you'd be surprised but building houses is less of a trouble than you'd think.

the first big cities were built in late medieval age / early modern times. the great fire of london wiped out large parts of the city in 1666, up to which point most housings were built of wood. Yes, wood. After that, the city decided to rebuild the city in bricks and stones to guard against future fires. That was the first big cities on earth. (apart from some luxury cities for show-off in antiquity).

since then, almost all big cities have been built from scratch within the last 200 years. It was this rapid growth, together with an exponentially growing population, that caused all the demand for human work. Now, birth rates are declining in most of the northern hemisphere, and the population is gonna decline starting sometime around 2040. That means that you need less houses year after year, and if you completely stop building new houses, you'd probably still have enough after that, because old houses and city apartments tend to be freed up by old people dying.

So, no, building houses is not a big trouble. Maintaining houses (that we already have) is significantly less work than building new housing, and we already have a massive number of houses standing around today.