The people "trying to make it political" are the ones who want it to go away - ie, the bigots, who somehow always seem to be on the right wing. Everyone else has to push back because failing to do so means bending to the will of the bigots.

If you want it to stop being political, your energy should be directed against conservatives who keep trying to put people into arbitrary boxes and keep them there.