I have a simple VPS with OVH. I installed OpenVPN and use that for all my VPN needs. The external firewall is set to basically block everything except my home IP which I use to ssh into. However, I would like to now run some basic services like taskwarrior, etc that I can access remotely with my laptop and phone.

To keep security simple I thought I could just connect to the VPN, and only allow access to the web server, taskwarrior etc when I'm connected through OpenVPN.

I thought this would be obvious to set up, but so far, I've not succeeded. Also, is my method actually secure, or should I be doing something differently?