In Iceland there seems to be endless amount of fresh water coming from the earth.
Rain is quite common in Iceland, mountains are full of water, and the water filters through natural underground water tunnels and lava. In the end, the water that you drink in Iceland is hundreds of years old, but fresh and cold and full of natural minerals. In most cases, it is perfectly safe to drink tab water. In cases where it is not safe or not recommended, people are warned about it. It rarely happens. If there is a faucet, it is usually safe to drink the water.
You can even have your bottle, stop by next clear water river, creek or spring and fill your bottle of water. Keep in mind though, that if there is a farm above you or if you are in a town, you should not do this. In the highlands this is always done by Icelandic hikers.
We often see travelers buying a whole lot of water in the supermarkets. It is completely unnecessary to do that. It is enough to by one bottle if you do not have one with you, and then you can fill it up in the next tab you can find. Even in public toilets.