The momentum is the same, the impulse (and therefore forces) are very different. The bullet is propelled down the barrel gradually - the force is spread through the entire time it takes the bullet to travel the length of the barrel, the reaction forces are applied to the stock gradually, and spread over the area of contact between the shooter and the gun.

A bullet stopped by a vest/plate has a much larger impulse. The bullet needs to be stopped essentially immediately, rather than gradually slowed down over a length equivalent to a rifle barrel, otherwise it kills you. The force is also more concentrated, occuring over the cross-sectional area of the bullet, rather than over the entire contact surface with the rifle.