You have two options for setting up https certificates and then some more options for enabling it on the server:

1: you can generate a self signed certificate. This will make an angry scary warning in all browsers and may prevent chrome from connecting at all (I can't remember the status of this). Its security is totally fine if you are the one using the service since you can verify the key is correct

2: you can get a certificate to a domain that you own and then point it at the server. The best way to do this is probably through letsencrypt. This requires owning a domain, but those are like $12 a year, and highly recommended for any services exposed to the world. (You can continue to use a dynamic DNS setup, but you need one that supports custom domains)

Now that you have a certificate you need to know, Does the service your hosting support https directly. If it does, then you install the certificates in it and call it a day. If it doesn't, then this is where a reverse proxy is helpful. You can then setup the reverse proxy to use the certificate with https and then it will connect to the server over http. This is called SSL termination.

There's also the question of certificate renewal if you choose the letsencrypt option. Letsencrypt requires port 80 to do a certificate renewal. If you have a service already running on port 80 (on the router's external side), then you will have a conflict. This is the second case where a reverse proxy is helpful. It can allow two services (letsencrypt certificates renewal and your other service) to run on the same external port. If you don't need port 80, then you don't need it. I guess you could also setup a DNS based certificate challenge and avoid this issue. That would depend on your DNS provider.

So to summarize:

IF service doesn't support SSL/https OR (want a letsencrypt certificate AND already using port 80)

Then use a reverse proxy (or maybe do DNS challenge with letsencrypt instead)

ELSE:

You don't need one, but can still use one.