There's an app called Fig that I use that makes it much easier. You tell it what you don't want to buy (there are presets for vegan, vegetarian, allergies, brand boycotts, etc.), and you take a picture of the ingredients list on a product. It'll tell you if it matches your preferences, or even if it's questionable.

Like most things, you learn as you go. I just learned my dishsoap isn't vegan, but I already have it, I'm not throwing it away now. I just wont buy it again. I'll be checking the next soap I buy. Each new thing needs to be researched. I'm just more comfortable with my choices than I was before.

It's definitely not easy and takes some research, but it's not impossible.

For example, carmine is used as a red dye and is made of small insects that are processed for the dye. Some sweets use these, baking ingredients can too. If you don't know that, you may end up supporting something non-vegan as a vegan.

It's these little things, "unknown" and obscure knowledge, that can get you if you're not well informed.