I am currently running Jellyfin on Btrfs and there is quite a performance impact due to CoW. If 2 clients decide to browse the libraries, both clients grind to a near standstill with regards to being able to see things.

CoW is not recommended for databases, all DB servers advise for turning it off for the actual database. You'll run into the same issue with a dedicated database if you leave CoW on I guess. You could also disable CoW for jellyfin's database right now and performance should increase.

I also follow the progress of a dedicated DB, but on the other hand I don't know how much sense it makes architecturally. The likeliness that you have multiple jellyfin server instances access the same database is low - after all, there is info very specific to the server in there like the file path. Just migration is already not easy, how likely is sharing the database live? And if each database is specific to an instance - why not use SQLite (like it's done right now) and allow for more specific parameter tuning, like used memory and the like?