Well the packages from the default repo are vetted by your distro maintainers. So if you just install a package from your distro's repo you're still relying on the security of your distro.
If you go outside of that, either to get a FOSS package that wasn't packaged for your distro, or to get a non-FOSS package, you have to do your own due diligence, just as when you're downloading a third party package for Windows or macOS. Either by reputation or by finding someone trustworthy who has actually checked the code.