There are "Emacs distributions" like Doom emacs, Spacemacs, and Emacs Prelude which do a better job of making sure all of Emacs uses the Vi key bindings.

As others have said, keybindings are fundamental for a text editor, and other packages are programmed with the assumption that you use the default Emacs key bindings. So it is takes a lot of work to make sure Emacs uses the same key binding "theme" for everything, and across all packages.

Fortunately, there are people who have done all of that work for you, and those are the people who have created Doom, Spacemacs, and Prelude. So I recommend you check out one of those distributions, follow the instructions to install and configure it, and use that instead of ordinary Emacs. Keep in mind that they change almost all of Emacs so you should read the documentation for the Emacs distribution instead of the ordinary Emacs documentation to learn how to do things.