The application needs to speak a protocol to be able to use it. If you use a X wm your apps need to be able to talk X's protocol to work, if you use a Wayland compositor your apps need to be able to talk Wayland's protocol (or run on Xwayland, which is basically an X server that runs inside Wayland).

The wm/compositor abstractions only work if your apps know how to use them via the correct protocol

X11 and Wayland are just protocols. These protocols are used to abstract the window drawing from the actual hardware and runtime environment as much as reasonably possible - because nobody wants to maintain 3215 versions of their app for different runtime environments. So in order to be shown on the screen an app needs to implement either the X11 or the Wayland protocol (or both!).

The piece of software that is on the other side depends on whether the app is using X11 or Wayland. For the sake of simplicity let's assume that the app does only support one of those. If the app supports Wayland then it will try to connect to a Wayland compositor. The compositor implements every part of the protocol and makes sure that the window is rendered on the screen and that user input is forwarded to the app. If the app supports X11 then it will try to connect to a X server and take the role of an X client. This is (on Linux, essentially) always X.org*. X.org also implements every part of the protocol and makes sure that the window is rendered on the screen and that user input is forwarded to the app.

* Unless you're running a Wayland compositor, then it will connect to XWayland which passes through the window to your compositor.

Wayland compositors have full control over the apps while the abilities of apps are purposefully restricted.
A window manager is just another regular, boring, old X client connecting to the X server. It doesn't actually abstract anything. It can move windows because the X11 protocol allows it to, but any other X client could just as well move all other windows around, read all user input to all other windows and even move the mouse around as it pleases.

So, to be specific, there is no mouse pointer bug in Virtualbox while using Wayland. There is a mouse pointer bug affecting specific Wayland compositors, likely because they enforce GPU hardware acceleration that is lacking in either your VM or the Linux kernel because of missing drivers. Try using a different compositor, (re)installing Virtualbox Guest Additions with the correct version on the guest system and/or check whether hardware acceleration is enabled for the VM and has enough video memory.

As an addition to other responses, think that most apps (specially smaller ones) are developed using some framework or set of libraries that might or might not support those protocols.

So let's pretend that I have an app buit using Electron and that framework does not support Wayland. There's nothing I can do on the app side until Electron supports Wayland in this fake example.

So it actually takes time for the libraries to support the new protocol and then app developers to update their apps to support it aswell.

That's why you see that the Wayland migration is incremental and not all at once.

Imagine the whole thing like a graphics card that is in a different PC. Your app wants to draw it's content on the remote screen. Only it's own content inside it's own window. This is not screen sharing. Your app cannot touch any other apps.

X11 is the connection between your app and the remote graphics card. It may be the local card as well, it is the same.

Technically, a wm is not needed. The app and X11 would work anyway.

Shouldn't window managers abstract all that for the software

The wm does not interrupt or change any communication between the app and the screen. It amends it with decoration and control buttons etc. for example it draws the window borders around the app's own window area.

In the case of Waydroid, it depends on features only available in Wayland; simple as that.

There are some applications (like autoclickers) that depend on features only available in X, as well (mainly because they directly ask X to do something)