I haven't talked to these people on any regular basis, other than once attending one of their weird little seminars at someone's house once, so I've had little to no experience with how they apply these ideas. I just read the blog posts and mulled it over on my own for a while, and my main takeaways were something like "change your mind incrementally when you get new evidence".

That said everything you're saying does ring true, and I've been changing my mind about Yudkowsky and his ilk pretty gradually for a number of years. Hearing that he's in with dudes like SBF have made me ready to fully distance myself from their stuff now that I know what they get up to.

I never heard of applying the ad hoc Bayesian thing to moral stances. I'd only ever applied it to questions of fact. Creepy to think where that leads.

Thanks for chatting with me about this, it's been helpful.