Mortgage rates in Canada were running over 15% in 1980 and peaked at near 22% in 1981. Minimum wage was about $3.25 depending on province, about $11.76/hr in today's money. They weren't buying houses. A lot of people were absolutely slammed if they had to renew their previous 11-13% mortgage at 22%.

There's a bunch of other factors involved too. Median house size has doubled since 1970 (1200sq ft to 2600 sq ft), restrictive zoning and forcing contractors to also develop single use subdivisions puts their costs up, and promoted higher margin developments. Also, the feds and provinces have drastically reduced the construction of public housing.