a decade of postponing the 2nd AI winter....

 its been morr than a decade since the demonstration that purely distributional models of languages could do a surprisingly good job, dispensing with many years clever work building semantics and symbolic approaches to the job.

so what has AI done for us recently? I mean really, compared with that first fanfare, how much more has been delivered? i'd say not very much.

when you compare it with any decade of internet research (which went from IP to WWW in roughly 1 decade, and went from WWW to smart phones in the next decade, and then went from that into orbit...)

nope, AI[*] is starting to look a bit like, um, fusion reactors, or, um, quantum computers - the next big thing is about 10 years off, whatever year you look at it. and what's really amazing is that the investors are still putting up with it...

* AI, by which I mean LLMs - there's lots of cool other AI - e.g. foundation models for weather prediction or for earth sensing (e.g. fastnet and tessera in my institutions) and some nice PhiML and so on...

AI re-writes capitalism

Money can be a means of exchange or a store of value. traditionally, we therefore could use money to shift the returns for goods or labour in space and time, instead of having to just immediately exchange (barter) goods for labour or labour for labour or goods for goods.

the system then enabled trade across distance - wealth of nations, and was to some extent regulated by the "invisible hand" of capitalism (thanks adam smith for these enduring, if somewhat flawed metaphors from 250 years ago).

so more recently (but still 80+ years ago) von Neumann took the theoretical ideas from Turing (& Church) and with Eckart and Machley, documented the idea of the stored program computer - the key clever trick is that memory holds code and data. indeed, at any given time, code may be data for other code (machine translation, compiling, self-modifying etc).

So the new thing about AI isn't about very big neural networks and data centers. It is that the eco-system took all the world's data, and all the world's code, including all the algorithms for how money and exchange work, and combined them - the AI industry is now a hidden brain instead of an invisible hand, and controls the rules for the means of exchange and the store of value as well as the goods and services for which those exchanges and stores operate. 

We can think of this new processing paradigm much as we think of computers - noting that of course it is very mutable, where previous systems involved soverign currencies backed by states and national banks, and to some extent, markets learned value based on production of physical goods, or the size of labour markets. Now that is all tears in the rain.

just to note: I don't believe capitalism works at all, but like democracy, it is a model that is perhaps just less bad than many others. to this end, the computational socio-economy described above is at least as bad as capitalism, and then some, as it bears even less relationship to reality (i.e. until humans sublime, we do depend on food, shelter, social contact etc, and virtualisation doesn't preserve the conservation laws - you can't eat bits, you can't shelter under a process).

the internet (and web, and napster) kind of said the cost of copying a bit is effectively zero. This undermined the huge investment that creatives made (books, music, film) and led to wholesale theft of their livelihoods. AI turns this on its head - the cost of a bit in an LLM is insanely high. time to go back to paying people to make nice things once more.

AI singularity (as flawed or as hyped as the human upload drivel)

 the new kid on Hype Block is recursive self-improvement for AI.

so of course software tools & die has always been about making tools that can make more refined tools, that can make more refined tools - from stone, to iron/bronze to steel, carbon, and on to  high level languages with optimising compilers and so on - nothing new there

but the steps require scaling up the tooling, and re-tooling - so famous numbers for new chips and new biotech are we're around £500 billion to build the new fab for the new thing. so if you wanna go meta on that, lets assume add a 0 for each generation (actually, each generation usually takes a decade to really bed in, and that's 3 orders of magnitude more infrastructure).

so what about the AI singularity - i.e. the point at which it just self-improves ("recursively" is a bit of extra hype-sauce)? couple of challenges

1. we're out of chips right now

2. we're out of new data right now.

Aint gonna happen

sure you can use synthetic data, and then some randomizing hacks to avoid model collapse (kind of).

but if you can't buy the memory for love or money, just forget it.

the opposite direction of travel (millions of really small, specialised AI boxen, running on smart phones, or worst case £1000 laptops) makes a lot more sense. and cents.

there's a book about people that invest in cryogenics for keeping their body on ice for when the medical tech can grant them immortality - see Mark O'Connell's fun read.

back to the football...

+/- 250 years

50+ years ago in france, we had no mains water and no electricy - we could fetch water from the well, and light a wood fire to cool. this was what most people had for the previous thousands of years, certain 250 years ago. most people weren't in cities, and had that experience - a lot of the day was just keeping body and soul together. now, some people still don't have electricity or safe water. but most people do.

now many people don't have safe and reliable internet.

so 250 years from now, i predict we'll be back to most people not having safe land, water or internet. we mighyt be better off if only we can maintain fish stocks, since we certainly wont have large herds of pigs and cows....unless they can become aquatic.

we should be planning for the 1000 year dark age, and establish a couple of foundations to rebuild after we get things back into a stable operating point on planet A. (apologies to Asimov, who of course based his Foundation series, at least the first 3 volumes on Gibbon's Decline & Fall).

royal society event on 250 years this way or that made me think a bit.

If 5 were 10.....generational cycles in academic life

One of my fave songs of jimi is If six was 9 which I had on vinyl on one of those were 1970s compilations of rock tracks with one band one side, and a different band on the other (usually very good).

However, I'm writing about 5s and 10s.

This year sees the alarming 50th anniversary of my matriculation at Trinity College Cambridge.

It also is the 25th anniversary of being elected as Marconi Professor in The Computer Lab (as it used to be called when I moved here).

LOoking at my academic diary, it looks like it is the 40th anniversary of me teaching undergrads, PhDs and Masters students (begun a few years into working at UCL, where I was between Cambridge and, errr, Cambridge, in what was, at first, the department of computer science and statistics).

It is the 5th anniversary of my playing live and seeing in most open mike and jam sessions at The Artery sessions at the MAP studio Cafe. I suppose it must be about 45 years since we played a session at the Cambridge Folk Festival, and a support gig at Kings College may ball, too. Oh, and Girton Garden Party.

Having an academic career is, of course, full of cycles. The academic year, the degree, promotion processes etc, and of course this leads to generations. I recently met a PhD student who was being advised by someome whose advisor I'd advised - so that's 4 generations. Looking at this (e.g. from the Maths Genealogy Site), it makes sense - stats suggest it is abot 10 years after you finish your own PhD that your own first advisee finishes (n the UK system). Of course, to reach some number of generations, the survival rate (i.e. does a student become an academic (hard), do they get students of their own (quite hard), do those students finish in reasonable time (fairly hard), and rinse, repeat, or recurse, as we prefer)....

I'm still working on figuring out a legitimate Erdos/Bacon/Sabbath number...I've got E (crowcroft :- luby :- noga :- erdos) and S (bit obscure but if you count jamming) but Bacon is a challenge - i was on the same stage as Brian Cox once for a Royal Society AI event at the Royal Festival Hall, so if I can count that, I can inherit his numbers, plus 1.

Gee what fun.

In numbers...:-

60 PhDs supervised

120 PhDs examined

Lost track of masters&undergradute numbers...

237 papers (on orcid)

FRS since 2013. FREng since (space) 1999.

Hot off the press: Cambridge has a retirement policy. This is my last year accordingly. my take.

Conservation Laws

 As we see the Giant Killer Asteroid of AI plummet towards the Home of the Human Race, it becomes clear that the laws of thermodynamics were incomplete.

As well as the conservation of energy, momentum, mass and spin (oh, ok, angular momentum), we must now add the conservation of Intelligence.

This is made obvious by the ease with which obviously stupid AIs are able to fool humans in the imitation game. The AI did not have to be as smart as the person, it only had to meet them half way.

This works both at the indivual level, and at the group level, and at the academic level (people can use an AI to write a terrible paper that human reviewers will accept) and the investor level (venture capitalists fall over each other to fund companies whose products will lead to the ends of the earth).

This is the existential threat to humanity. Not that the AI will wish to eliminate us as competition - it won't need to, because once it gets anywhere near that level, we will all dimmer than a Toc H Lamp

It is already too late to put an end to this.

the internet family's least famous kin

People talk about the "Father of the Internet" or the "Grandfather of the Internet", but no-one mentions the strange uncle of the Internet, or the fairy godmother of the Internet. Nor do they speak (except perhaps on the dark web) of the evil twin of the Internet, or the secret love child of the Internet.

Where are the kissing cousins of the Internet, or the mother-in-law of the Internet? What happened to the wastrel son of the Internet? Indeed, the whole bastard offspring of the Internet have been totally erased from public discourse.

The maiden aunts of the Internet should be given credit, as well as the confirmed bachelor great uncle.

And of course, the Giddy Aunt of the Internet.