I used to do that and liked it, but dropped the habit somewhere along the way. It doesn't even feel that bad after the initial shock passes. It's even pleasant, and then when you get out, the water is cooler so won't evaporate as readily, do it's not just that you're more used to the cooler temperature, but it also doesn't get as cold because evaporation takes away more heat than the higher temperature water provides.

Also, use your hands as squeegees before you get out of the shower area and your towels won't go stinky as quickly (because they dry out quicker due to not needing to absorb as much water).