Just got my 3rd Fenix 5s replacement for this issue. I was told (this time - not in the past) that this is due to pool swimming and chlorine build up on the sensor and that I need to rinse it with fresh water after every swim and if I do that, it shouldn't happen.
The sensor is inside the right hole, so even you wash it is difficult to clean inside. I try also to put the watch into distilled water overnight weakly to try to dissolve impurities. But nothing fail happened anyway after 6 months. So maybe clorine oxide something