I feel like this is true if the reader is meant to have the perspective of the person who feels that something is magic (the Hobbits, in the example from your video). However, not all magic in fiction is like this, and sometimes the reader is supposed to mostly have the perspective of Galadriel, or to gain her perspective over time.

An example is Lev Grossman's The Magicians. The reader has the perspective of the Hobbits at first, because that is the perspective of the main character. But the story has themes of "lifting the veil" of magic, and by the end both the main character and the reader have a more similar perspective to Galadriel.

I guess what I mean is, I agree with you and the video's author in large part... but like... to broadly say that magic "should" be used in literature in a certain way ignores how it can be used in different ways to great effect!