On if you have no clue what you're doing.

Off if you're good at making OP characters and familiar with 5e.

It always makes rolls favor the one rolling, with a bigger bonus the more they've failed. So if you have a low attack bonus and miss frequently you'll hit more often. But if you have a tank with 22 AC, they'll also get hit (and crit) far more often with it on.

I play with it off, because I prefer to not have my rolls messed with in either direction. I don't personally want to play with it giving me an artificial bonus and then likely dumping on me the next time I roll.

As someone else stated, if you don't know what you're doing keep it on for the sake of gameplay. Otherwise turn it off if you don't want it

Personally I will be playing without Karmic Dice. I believe Larian's concept behind it is like the DM that occasionally smudges the rolls during a game. The game is designed around you failing rolls just as much as succeeding so I will take what the dice wills.

That said people did testing during Early Access and found that on average you ended up hitting and getting hit more often in combat with Karmic Dice enabled as it sort of evens out your rolls. So if that interests you by all means go for it!

I hated the karmic dice. I found I failed skill checks noticeably more frequently with it on.