• 0 Posts
  • 239 Comments
Joined 3 years ago
cake
Cake day: May 20th, 2024

help-circle

  • I find it extremely extremely amusing that this is mixed with something that is almost the inverse:

    They are so desperate for what they are doing to be completely unprecedented. Something literally SINGULAR, with no comparison in the history of the universe itself. And yet, when you know how to properly plot what they are plotting, it becomes clear that it is anything but. It is a phenomenon that is ubiquitous across any optimizing system, be it industrial learning curves, traning ML systems, biological evolution, research and development return on investment… its just another utterly ordinary example.


  • Ahahahah oh man this is great

    The METR cultists are arguing the end of the world is next year because they see some kind of speedup in improvement of AI capabilities with respect to their favorite metrics.

    Others have rescaled their data according to total money spent on R&D and it’s… a decelerating returns power law curve, where linear increases require exponential effort. Just like the generic industrial learning curve, just like the scaling laws to training ML systems. They just started shoveling money in exponentially EVEN FASTER.

    https://www.siliconcontinent.com/p/are-we-approaching-an-intelligence

    (see figure 1 so I don’t keep filling this with even more images)

    This is just like Moore’s Law. We like to think of Moore’s Law as accelerating returns for computational hardware, but it was actually an utterly normal industrial learning curve with power law diminishing returns to R&D spend, coupled with an exponential increase in R&D spend per year that leveled off and went linear circa 2017 and since has manifested as decelerating return ordinary learning curve.

    Some are torturing the data to try to find if the exponent of the curve changed to SLOWER diminishing returns where they see the breakpoint, but its really tortured, you jiggle the datapoints just a little and you get wildly different conclusions.



  • ahem Back in my day…

    I remember finding and reading Lukeprog’s (Luke Muehlhauser) blog circa 2010 in my sweeps of morbid fascination, the rationalist convert who made MIRI less completely comically dysfunctional. It was a classic post-Bush-era new atheist deconversion annoyed at religion thing, who then discovers Big Yud’s writing and becomes more and more obsessed with it until he writes this big empassioned thing about how he has found the most important thing in the world and has to devote his life to it. Its classic epistle conversion story. I saw so so many of those on the LW website itself back in those days. I can’t find it right now but there was one that even included as a standalone line near the end “It was then that I accepted the reality of the singularity.” or something to that effect.

    It was so incredibly clear that the LW community swept up people who had fallen out of abusive religious contexts, filling the psychological hole that had been burned into them.

    And it’s so incredibly clear why so many of them continue to crash out into weird catholic conversions and ethnonationalist mythologies.



  • A LW type thinks we are past the ‘last revolution’ and irrevocably unfree and everything bad is now bad because of intent by those with power who are in control and cannot be dislodged

    https://www.lesswrong.com/posts/ZxBaCDjxyxKaLHqGf/the-last-great-revolution

    It feels like the common man lost a bit of his soul after we exterminated the vast majority of organized crime. Yes, crime sucks. But even if these organizations were horrible for the communities they preyed on, the existence of an underbelly at least suggested limits to state power… People watching old movies and TV shows can feel the guilty exhilaration at the idea of participating in or shaping a network outside the state, with its own rules.

    Nowadays even the petty crime you do witness feels like evidence of the state’s taunting indifference. There are cameras on every street corner; every child grows up with the tacit knowledge that everything they’ve ever said or done is going to be logged in some government database searchable by AI. The “government”, whatever that means, could stop the guy who’s shitting on the subway, if it wanted to. Crime has stopped being something that happens in spite of the government, or undermines it, and started being another boring reminder of your powerlessness in the face of the reigning regime, and of how nobody can ever do anything about it.

    I suppose it makes sense that people who believe in omnipotent computer gods also believe in omnipotent governments.






  • https://www.lesswrong.com/posts/7xJeaAKTKxbHL6F9a/a-new-kind-of-prediction-market-for-science

    Got a problem of randos shitting AI-generated ARXIV preprints into existence faster than the scientific community can deal with it? SHOEHORN IN PREDICTION MARKETS!

    (Incidentally, my boss has been complaining about the onslaught of low quality preprints that are shat out with the usual low-quality signifiers that let you filter things being harder to immediately see, and his conclusion is that he is sad to say science is going back towards and has to go back towards systems of reputation and patronage where you have to look at who lent their reputation to a work and who newcomers are being vouched for by to determine if it is worth looking at it at all.)

    (And I am again reminded of our electronic information sphere becoming like the Earth’s genetic information sphere in that it is full of selfish parasites looking to hijack you to use their nonproductive information, with productive cellular life jealously guarding its genomic nucleic acids and not letting it interact with the froth of selfish viral and mobile elements bouncing off it at all times.)


  • That’s the simplest way that this is wrong. But it’s also wrong in a much more interesting way that even a bunch of scientists I interact with don’t quite get sometimes.

    You can talk about the current complex system that has built up a bunch of internal dependencies over time, and wrongly talk about the odds of the specific sequence with all that embedded history appearing all at once.

    But you can also talk about canalization, and the metaphors from erosion are very apt and very intentional. If you have a hillside where it rains, there are a huge number of drainage networks that can transfer water down the hill, and any one of them is only a tiny subset of the huge of possibilities. But also, if you take a hillside and bulldoze it, you WILL get one of those valid networks back again, and it does not have to happen all at once. Instead, small changes to the inert starting point will be driven into existence by the pressures of the flowing water, causing the flow to slightly increase. Those highly contingent starting points will then drive an accelerating process of secondary changes that pushes the hillside towards an efficient drainage network, regardless of the starting point. The system is driven very predictably into a tiny corner of parameter space, from a huge number of initial starting points.

    There are very persuasive arguments that abiogenesis and evolution are very similar. That instead of talking about the random convergence of rare events, you should think of the process as a runaway, tiny initial changes making future ones more likely in a chain reaction that drives you towards function. Initially inert chemistry subject to disequilibrium being driven into becoming an efficient metabolic channel, and an efficient information channel, pushing you towards the living state in an initial environment that has enough disequlibrium and chemistry that can support structure. In this school of thought, the forces that drive life into existence are basically the same forces that sustain it once it exists.

    EDIT This is even relevant to machine learning! You literally initialize synthetic neural network with random numbers, and apply the training process, trying to push it towards having the output given an input match the ground truth. Regardless of the random inert starting point, you reliably drive the network into a state that produces something like the ground truth output from the training input, provided there is sufficient information in the input in the first place. You don’t have to initialize it with the perfect starting weights, almost every starting point is canalized towards some state that produces the relevant type of outputs.

    I recommend looking for the work of one particular guy, Dr. Eric Smith. There’s an old talk of his somewhere on youtube, where to roughly paraphrase, he says that you shouldn’t be wondering about the very specific chemical events at the origin of life, but instead, you should “ask what was so wrong with the dead Earth that it had to spontaneously break down into the living Earth” which is the stable state, the resting point that it is canalized towards.


  • https://www.lesswrong.com/posts/KrFu7kGk3x8aZcCzu/there-is-no-such-thing-as-cosmophagous-ai

    They are SO CLOSE to getting it. People get so crippled thinking of the possibilities of life and intelligence in the universe when they assume that the only possibilities are endless exponential expansion and destruction. and I abhor the discussion of abiogenesis in the comments SO MUCH.

    The probability of life spontaneously forming is related as inverse exponential to the complexity of the chemical reactions required to spontaneously generate that life. The simplest living organism we know of has a HEAVILY compressed DNA sequence of about 50,000 base pairs, all of which are most likely load-bearing. Back of the envelope, this means the spontaneous generation of this organism is less likely than 1 in 4^50,000, or about 1 in 1e30000, to say nothing of the machinery required for actually transcribing the DNA and synthesizing proteins

    This is PRECISELY analogous to being freaked out that a hillside where it rains can develop a river channel when you would need to move all the sediment in a branching network of ditches all at once for it to still transport water.



  • These scaling laws are even funnier than that, as are more recent ones.

    I continue to be extremely amused by the fact that you can take a LLM ‘reasoning’ trace, replace all the ‘reasoning’ with dashes or spaces, re-run the attention mechanism, and have what comes out the other end improve by a significant fraction as much as if you let it exude text that has the appearance of having to do with the topic at hand. It’s just more opportunity for the embedding vectors to dance against each other before you translate the abstract vectors back into natural language. Hence recent frontier models doing a thing where instead of doing a long reasoning trace, they build the attention matrix, allow the vectors to push each other around… then let those new vectors push each other around again, and again, and again, multiple rounds, before you start decoding.

    More recently, I have seen some vague math into the scaling of the speed with which running many parallel systems with access to a common pool of text produce a legible result. If I recall, it comes to roughly:

    N parallel systems

    Finish in ~1/N^0.5 as much time

    At ~N^0.5x the cost

    So 16 parallel systems crosstalking finish complex tasks in 1/4 the time at 4x the cost (and 16x the instantaneous rate of operation per second)