Its just impossible to even start a VPN from these systems unless you have administrator privileges, so im not used to your way of doing it.

That's also the policy for the majority of the machines/users but there are a few that do have admin privileges like IT teams and whatnot and even if they manage to install a VPN solution (the app would most likely get blocked by endpoint security either way) they couldn't communicate to the outside because the firewalls, as I described, are all set to block VPN traffic. Except for those situations I specified above.

The bottom line is: distrust everything, everyone and anything. Even if you can ensure nobody can install a VPN application on their computers, assume someone might get around that and add proper firewall checks and blocks as well.