I believe they're called heretics. Papal infallibility and divine representative positioning within Catholicism means disbelief in the word of the pope would basically be a renunciation of the faith.

They ain't called papists for nothin!