If you have pre-trained model or a classical voice matching algorithm as the basis, few samples might suffice.

I don't think it seems like too few samples for it to work.

What they train for is rather specific. To identify anger and hostility characteristics, and adjust pitch and inflection.

Dunno if you meant it like that when you said "training people's voices", but they're not replicating voices or interpreting meaning.

learned to recognize and modify the vocal characteristics associated with anger and hostility. When a customer speaks to a call center operator, the model processes the incoming audio and adjusts the pitch and inflection of the customer's voice to make it sound calmer and less threatening.