I pressed alt + tab to change tabs, and when I went back into the game I could see every zombie on the map's health bars even in fog of war, which is obviously powerful (especially considering it was at the beginning of a new game).
When my unit spotted some zombies their health bars then disappeared *note the health bars of my units are not visible.