Yeah, that would probably be ideal, or perhaps take the kid on vacation somewhere they'll get immersed. Watching media in your native language could also help, especially if you're in a country where dubbing is the norm and they don't get exposed to it organically.

In the meantime, you should probably take some pressure off. As you noted, your kids will learn English regardless, and if you keep speaking it they'll probably learn it faster than their peers and with less accents. If you push too hard they might start fighting it, which won't help their motivation at all.