You need enough lore to explain where the PCs fit into the world, both for their races/backgrounds and for their current adventure. Some of this can be plenty vague: you don't need to know everything about the elven city states yet, just that they exist and one of your PCs grew up there. You don't need an entire pantheon, just the gods your PCs worship. The details that are most important are for the local area: what might the PCs need to know, and what would they want to see? Who runs the town? Where are the rat catchers going? And what will hook them on their next adventure?

Generally I start with a vibe, usually generic fantasy land since that's what people are most familiar with.

Then I plan out the local town and the countryside around it: Portland Court used to be an ancient tiefling city-state but that was so long ago the only real reminders of that era are the large basalt bricks people still recycle for building and the occasional weird bones folks still find in the nearby fields. Now it's mostly just humans doing what humans do, scratching out a living using whatever they can. The name court refers to the central open market and the successful merchants who are the defacto leaders of the community.

Then I work with the PCs for a little background. There's a dwarf in my party and she hails from the nearby Iron Mountains. A cleric worships Sol and Luna, twin gods who embody the sun and moon. There's also a halfling who grew up south of the Court, in the Roving Prairies.

Finally I try to fit everything together: the local merchants need guards for a caravan going to the next town over because the demon-worshipping cultists who want to revive the dead empire keep knocking over wagons and stealing goods. This is good tension even if it's not very original. Plus it gives the party opportunities to continue adventuring and, if they choose, learn more about the ancient history most people would rather keep forgotten. The merchants will need more help, and the cultists will keep escalating until someone does something about it once and for all.

Hope that makes sense. I like to start small and build up as the party adventures and the stakes get higher, with just enough thought out that I can give the worldbuilding direction when I go back to it.