perhaps with something that when injected is inert but when ingested it mixes with the stomach acids and becomes poisonous?

Amygdalin comes to mind, but I'm not sure if it produces cyanide even when not in the stomach/intestine and just in the bloodstream, so obviously do not try it

Saline should be fine, it's just salt water. You wouldn't want to chug a ton of it, but even a glass certainly won't hurt you.

Edit: After some googling, swallowing insulin wouldn't hurt you either. It won't help your diabetes, but it's definitely not poisonous.

Syringe full of air?