I never excelled at high school level math or above. During my career I haven't really needed any of what I was taught in those classes.

I think what matters more is having the temperament (and time) that allows you the patience to identify and work through a problem. The desire to truly understand the problem itself. I think any good programmer can just figure out the finer details when they encounter a problem that actually requires it as part of the solution. Whether that be higher level math, physics, graphics, audio, algorithmic optimization, or whatever.

Now, if you're the type of person that just gets frustrated at a problem and gives up easily, or would rather throw together/copy-paste something that "sorta works," but you don't know or care why, then it may not be the best fit. That is not to say those type of people can't be programmers, but they probably won't enjoy it so much.