I think you are confusing personal growth with "number go up" as understood by GDP growth percentages and similar metrics.
Degrowth as a theory is mostly concerned with the latter and issues such as compound interest forcing continous and specifically continously accellerating economic growth.
Degrowth doesn't mean you as a person can't use your own labor to improve your living conditions and similar things. And such personal growth is only very indirectly related to economic growth.