See, if they just came out and admitted to that sort of strategy, I could at least respect it some, but instead they still seem to insist on acting like and planning for Trump being somewhat of a rational actor!

I’ve previously made an effort post summarizing this! https://awful.systems/post/4172436

tldr; Eliezer has criticized (more like mocked, or dare I say it, sneered at) expert systems, GOFAI, and neural networks, and he never really admitted he was wrong when it became apparent in the 2010s that neural networks really were a viable approach.

The exchange with niknoble in the comments was also my favorite. He notes the democrat party is actually the best bet for getting regulation on AI:

More generally, acceleration wins by default if everyone follows their local financial incentives, and the only force I’ve seen overpower capital at scale is the will of the Democratic Party of the United States.

He is maybe giving Democrat too much credit, but yeah, if the doomers were really trying to win, they should be looking at the party that at least has a chance of agreeing to more regulations.

It annoys me how much credit for predicting AI Eliezer gets away with claiming among his cult (and even outside his cult, like in interviews with reporters that could know better if they bothered to do their job) when all the actual details he predicted are nothing like LLM-based agents.

Likewise for claiming success on predicting instrumental convergence. LLM agents often can’t stay focused on the goal they are post trained and directly prompted to pursue, they are the furthest thing from trying to optimally maximize a bunch of secondary values in pursuit of master goal.

You know… from a big brain game theory perspective (rationalists love trying to take that perspective)… Eliezer is actually rewarding Trump’s tantrum by demanding EA bow to the right.

Rationalist have never run into a right-wing talking point or person they were unwilling to steelman nor a leftist they were unable to screw up steelmanning.

It would be hilarious if the AI doomers actually did manage to get Trump to sign some of their ideas into action with executive orders, only for him to reverse course or fail in implementation details, and then a hard tide of anti-Trump sentiment in the next administration to totally undo them.

Sadly it looks like we will only get the more immediate humor of Trump going hard against them.

Lol, I called it without even having to see it. Doubling down and blaming EAs for being too left even as the right curbstomps them. How does that boot taste?

Also, WTF does Eliezer even think “leftist” means?

This is hilarious. I didn’t think it was possible, but maybe the lesswrongers and EAs will finally realize they are in a leopards-eating-face moment?

Or maybe they will double down? I.e. they conclude that if they were less woke maybe Trump wouldn’t have turned on them? Haha.

Are we gearing up for a lepoard face moment from the cultist faction of the cultist / oligarch-technofedual-lord / grifter / authoritarian alliance?

Nah, lesswrongers don’t even have that level of introspection. Plus a lot of them are outright pro-oligarch-technofedual-lord, at least enough so that it outweighs their doomer-cultists allegiance.

“Democrat-polarized” is such a word choice. Like, ah yes, it is clearly the Democrat’s fault the right goes batshit at even a hint global warming is real or at any sort of gun regulation or any social safety net expansion.

The lesswrong reactions are hilarious: https://www.lesswrong.com/posts/s76hbGpaMkhiNwbTX/matrice-jacobine-s-shortform?commentId=B2WSExeg8wfgDqZiM

One commenter realizes the truth, but is heavily downvoted:

For anyone who was at all unclear on the subject: He’s not on your side. He’s against you. He wants you to fail. Stop trying to fix him. AI safety is in the same situation as measles prevention, free trade, military readiness, or the prosecution of the remaining Epstein offenders. Trump & co. are against these things, and they’re not going to be converted just because you tell them again how important these things are.

Sorry bud, if lesswrongers were capable of grasping this they wouldn’t have fallen for FTX, or kept falling for Anthropic’s half-hearted gestures of placation to doomer-cultists. And of course, many lesswrongers are genuinely very right wing and will thus rationalize justifications for the more centrist lesswrongers to suck up to Trump.

Because rationalists distrust mainstream normal science and skepticism, they are far more willing to dabble in mysticism and the supernatural than normal skeptics.

I am also questioning OpenAI’s strategy of strip-mining math for PR, since it seems like advances in math do not actually register that well in the public.

They want to sell the claim that they’ve “achieved AGI”. For the general public they can sell on the vibe of “the computer talks back to you”, but for more sophisticated audiences, they need something firmer. Everyone with even a fraction of the ability to think for themselves has stopped trusting benchmarks, so OpenAI and Anthropic has been looking for things associated with intelligence they can claim success on. Math proofs have the property of being somewhat automatically checkable, having lots of ideas scattered throughout the literature to let their LLMs sift through, and being associated with intelligence.

The /r/singularity and orange site responses have been gross, lots of commenters willing to totally overlook the multiple different ways OpenAI has been utter scum in this debacle in order to celebrate another LLM “success”.

I don’t think it’s a very novel insight

The slatestarcodex branch of lesswrongers have made “high-decoupling” and “steelmanning” and being “gray tribe” their entire identity so its extra funny (compared to normal cults in-group/out-group distinctions) seeing how hypocritical they are at it.

They’ve been talking about the recurrent layers… but my suspicion is that this “innovation” is to obfuscate “thinking” tokens/content instead of actually improving the model in any sort of useful way.

Benchmaxxing for one set of benchmarks can actually degrade performance on other benchmarks. They’ve plateaued for a while, all they can do is scale inference compute up and down (at logarithmically poor rates of exchange) and trade performance in one area for performance in others

A “no attacks on people’s characters” policy seems like a terrible idea

Rationalists: “Its not our fault that we got deceived and manipulated by SBF, lots of people were fooled by him! pay no attention to the fact we repeated the pattern of falling for it with OpenAI and Anthropic also”

Also Rationalists: “You can’t jsut call people in or adjacent to our in-group bad people!”

Also Eliezer: "There’s a standard Internet phenomenon (I generalize) of a Sneer Club of people who enjoy getting together and picking on designated targets. Sneer Clubs (I expect) attract people with high Dark Triad characteristics, which is (I suspect) where Asshole Internet Atheists come from - if you get a club together for the purpose of sneering at religious people, it doesn’t matter that God doesn’t actually exist, the club attracts psychologically f’d-up people. Bullies, in a word, people who are powerfully reinforced by getting in what feels like good hits on Designated Targets, in the company of others doing the same and congratulating each other on it. "

My first thought was maybe one of them isn’t focused enough on doomerism and is too concerned with practical shit, which in lesswrong and AI doomer spaces is a nono (practical concerns about GenAI and data centers are okay as a way to draw in normies, but they want to make sure you speak the shibboleths that indicate your real concern is Skynet). Thus whichever one is the less doomer one would get a smidgen of credit in my eyes.

I was disappointed to learn the real reason…

We endorse pointing out harmful actions. But we condemn attacks on people’s character.

…although it is probably a good thing that someone is willing to call Sam Altman a slimeball and Anthropic lying hypocrites? So I’ll give PauseAI-US a tiny bit of credit for that.

…or maybe not

When I read Holly’s Twitter feed, I get why someone would have a gut reaction like “this is an unhinged person who’s stooping to the level of personal attacks on people who seem nice and are just trying to do their best”.

Whatever credit this PauseAI-US person gets for willingness to call people out, they lose for doing so through twitter.