So this is kind of a breakdown of how it works. My 3 credit scores are all in the 800s, I only have one pay fee card and we're likely canceling it soon as we've got other cards that have better cash back deals. Here are a number of things that affect your score:

Using your cards, but not having a big balance on them monthly (we pay ours off, if not completely then the statement balance to avoid interest, but typically we pay them completely if we can)

Not missing any payments (haven't missed a payment in the 12ish years we've had cards)

Not having derogatory marks (credit dings from stuff like not paying debts or repossessions of cars or that sort of thing would hit hard)

Any temporary hits like applying for a new line of credit (cards, loans, mortgages)

Average age of credit (older lines help a lot, it hurts less to close a newer card than an older one)

Number of accounts, they want you to have a lot, over 12 I think is where it really positively impacts your score, and it can hurt you to only have a few

Anyway it's a complex system that's annoying and can be difficult to understand