uncanny cycle, hype valley - you choose

The cliche of the tech world is to trot out the infamous Gartner Hype Cycle and no-where is this more prevalent than in AI, post Chat-GPT (to be fair, post "Attention is all you need"). 

But the other slightly less worn hype is the phase that embodied (or perhaps even virtual) AI is said to go through, which is the Uncanny Valley

So where do these two curves collide, eh? Right about now...

zero trust

I'm pretty sure this sort of thing happens because of bad parenting - people that don't trust anything come up with the idea of distributed systems that have no need of any anchor for trust anywhere.

so i have lots of problems with this - starting from systems  - and the classic ken thompson reflections on trusting trust. These deecntealization extremists have to confront that they run software on hardware, and even if they build their own hardware and write their own software, they probably use an OS and a Compiler from somewhere else. However, it gets worse. Why should we trust the actual zero knowledge protocols they use? who has verified them, and how? why do we trust those verfication tools (peoples' brains too). And worse still. Why should we trust this new fangled idea zero. The Romans and Greeks and ancient mesopotamians got along fine without it.

No. I have zero trust in zero trust.

powerless trio

Three things we have that are between 50 and 100+ years old, still work, and do not require electricity - classic portable type writer from Remington and a Singer 66K sewing machine from 1917

 and an HMV wind up record player from the 1930s....

when civilisation collapses, we'll still be able to write letters, fix clothes and listen to some old 78s!!

 

Artificial Intelligences as Trusted Third Parties (AI as TTPs)

AI as TTPs is a recent posting by Bruce Schneier who has impeccable security credentials.

However,  I'm not convinced that the paper he is highlighting is as groundbreaking as he is.

The authors of the paper also have great track records and include AI, but I think they're missing something basic that means that a single or group of TCMEs ("Trusted Capable Model Environment") can't actually do anything different than any other computation, subject to basic privacy controls (e.g. access control authorisation, auditing, encryption of data at rest, in transit, during computation (e.g. using FHE and TEEs etc etc).

But also:

a) visible communication in/out of the computation - i.e. information flow control

b) control over specificity of that data (i.e. differential privacy - can you tell if an individual record is present or not, to put it. crudely)

c) secure multiparty computations and zero knowledge systems

which the paper compares and contrast with their new TCME notion. However, I think the dimensions they use for comparison are a bit of a stretch.

The main problem I think is that the TCME seems to be indistinguishable from any other trusted program.

Any shared secret between models (e.g. federated or decentralised learning) is just the same for AI/ML as for any other algorithm. Perhaps the intersection of probability distributions looks a bit different to juse being able to say "the richest person is A" without knowing how rich A (B, or C) actually is - but in the end, the distribution has some moments and can be described by some number of those more or less precisely - a distribution of distributions can be aggregated with more or less precision or uncertainty (e.g. respecting differential privacy, and some widest level, or preventing set membership inference at the finest grain) - the model itself can be protected from outside model inversion attacks by various schemes, but I don't see what TTP function is provided that isn't just a different mix of existing techniques for providing trust.

folding

 I asked my tame AI:

what if the plot unfolded like a bicycle?

That's a fun thought experiment! Here's how a plot might unfold like a bicycle:

• Two Wheels, Two Main Characters: The story revolves around two central figures, perhaps with contrasting personalities or backgrounds. They are interdependent, like the two wheels of a bicycle. One might be impulsive and adventurous, while the other is cautious and grounded.
  
• The Chain as Conflict: A central conflict or series of challenges acts as the "chain" driving the narrative forward. This could be an external threat, an internal struggle, or a shared goal.
  
• The Frame as Structure: The overall plot structure resembles the bicycle's frame – a sturdy foundation with distinct parts. There would be a clear beginning (the initial assembly), a middle section with rising action and obstacles (the journey), and a climax and resolution (reaching the destination).
  
• Pedals as Proactive Steps: The characters must actively "pedal" to move the plot forward. Their choices, actions, and interactions propel the story along.
  
• Steering as Decision-Making: Crucial decisions made by the characters determine the direction the story takes. Will they veer off course, encounter unexpected detours, or successfully navigate challenges?
  
• The Seat as Perspective: The "seat" represents the shifting perspectives of the characters. Their viewpoints and experiences evolve as the story progresses, influencing their actions and relationships.
  
• Wheels as Support: The characters provide mutual support and balance. They rely on each other to overcome obstacles, just as the two wheels work together to keep the bicycle upright.
  

Possible Story Examples:

• A buddy cop movie: Two detectives with vastly different personalities must work together to solve a complex case.
  
• A road trip adventure: Two friends embark on a cross-country journey, facing unexpected challenges and deepening their bond along the way.
  
• A sci-fi story: Two astronauts on a perilous mission to another planet must rely on each other for survival.
  

This bicycle analogy offers a unique framework for crafting a dynamic and engaging story with interconnected elements and a focus on character relationships and decision-making.

I'd love to hear more about what kind of story you have in mind! This "bicycle plot" could be a really interesting starting point for brainstorming.

so then I asked

what if the plot unfolded like a folding bicycle?

Ah, a folding bicycle! That adds an interesting layer of complexity and potential for unexpected twists. Here's how a plot unfolding like a folding bicycle could work:

• Compact Beginnings, Expanded Story:
  
  • The story might start with a seemingly small, contained situation or a limited cast of characters. Think of the folded bike, compact and unassuming.
    
  • As the plot progresses, it "unfolds," revealing hidden depths, unexpected connections, and a wider scope. This could involve:
    
    • Revealing hidden agendas or backstories.
      
    • Introducing new characters or locations that expand the narrative.
      
    • Escalating the conflict from a personal issue to a larger, societal one.
      
• Hidden Mechanisms and Twists:
  
  • The folding mechanism itself represents hidden elements and unexpected twists. Just when the audience thinks they understand the story, a "fold" occurs, changing the narrative's direction.
    
  • These twists could be:
    
    • Sudden betrayals or alliances.
      
    • Revealed secrets that alter the characters' motivations.
      
    • Shifting perspectives that reframe past events.
      
• Portability and Adaptability:
  
  • A folding bicycle is portable and adaptable, suggesting that the characters and plot can move and change easily.
    
  • This could translate to:
    
    • Characters being forced to relocate or change their plans abruptly.
      
    • The setting shifting unexpectedly, leading to new challenges.
      
    • The characters needing to adapt to rapidly changing circumstances.
      
• The "Folding" Climax:
  
  • The climax might involve a crucial "folding" moment, where all the hidden elements and twists come together.
    
  • This could be a moment of revelation, a sudden confrontation, or a dramatic shift in power.
    
  • The resolution would then be the bicycle being refolded, or the situation being brought back to a more stable state.
    
• Themes of Concealment and Revelation:
  
  • A folding bicycle plot could explore themes of concealment, deception, and the revelation of hidden truths.
    
  • It could also explore themes of adaptability, resilience, and the ability to find solutions in unexpected ways.
    

Examples:

• A spy thriller: A seemingly simple mission unfolds into a complex web of international intrigue, with hidden agents and double-crosses.
  
• A mystery: A small-town murder investigation reveals dark secrets and long-buried pasts.
  
• A psychological drama: A character's seemingly normal life unravels, revealing hidden traumas and psychological complexities.
  
• A science fiction story: a small team of researchers discover a device, that when used, unfolds a new reality.
  

The folding bicycle metaphor allows for a plot that is both intricate and dynamic, with the potential for surprising and satisfying twists.

ancient history

I was reminded of two things from the late 70s that a friend in cambridge did, and my cousin in london were doing at the start of their graduate research work.

The first was a study of medieval court records in East Anglia, trying to figure out what the demographcs were (in terms of single.married, how many times, how many kids, ages etc) - apparently for every day people in medieval tiems there were virtually no day-to-day records execpt that most people would at some point in their lives, show up in a court...so assuming most people didn't move about much in those day,s you could get a picture by looking at statistics and comparing villages/towns etc

Rhe second was a student of cave paintings from rather longer ago - the caves in France and Spain have depictions of animals from 17,000 to 22,000 years ago e.g. see

https://en.wikipedia.org/wiki/Lascaux#/media/File:Lascaux-IV_01.jpg

What my cousin was trying to do was figure out if the paintings were purely ritual, or perhaps actually a record of animals (especially ones hunted for food) - there's fossil records that give the spatial distribution of species, so you had ground truth - if the distribution of species in paintings was similar by area, then likely the primary record was of what people saw (even though of course it might also have ritual significance too) - 

Neither study was conclusive, but then AI tools were very hard to use 45 years ago, especially for historians and anthropologists...

So maybe we've made some progress since then...!

devaluation of ai

Change for the Machines (with apologies to Pat Cadigan)

AI was about models where money equals compute equals big data equals valuations.

So all the money going in was to finance compute, thinking where the value lies, 

and every valuation of every company was about how much compute they had so it was all fake. 

Companies valuations were just how many H100s they had (compute capacity), 

as if it correlates to better models (even though they’re usually just wrappers.

DeepSeek, and other Chinese models broke that which pissed everyone off, private and public investors 

included, because it casted doubt on the valuation methodologies, namely that energy, compute, data centres and number of chips were essentially fixed costs and the valuation of companies (and their output) could be measured on that alone.

Basically casted doubt on the last two years of public and private, markets not just of AI companies,

but the entire stack, energy, chips, data centres etc. Everyone felt like a dummy. Even though it’s 

been happening for a while. We knew this a couple of years earlier when Meta released Llama, and it was clear that much smaller models could be trained at much lower costs and yet achieve many of the same goals. In that case, it was better software engineering in the open source community. Perhaps being open sourced (despite origins in a hyperscale company) it attracted less attention, although perhaps the google memo "we have no moat"

should have been a clue.

https://www.theverge.com/2023/7/10/23790132/google-memo-moat-ai-leak-demis-hassabis

One of the ironies that the DeepSeek debacle also exacerbates is that one constraint on them that made them seek greater efficiency was export restrictions on higher end GPU - as with the open source research community, less is more. That constraint was already what drove people in the open source (often academic or hobbyist) community 

to develop affordable ATI technologies. In fact, outside of the LLM/GenAI world, many machine learning tools have been proving themselves perfectly useful and usable running on laptops on large datasets ("big data").

Denials at the time came thick and fast, perhaps because the huge investment in the new emperors

was not ready to be disrobed. Perhaps, also, as if OpenAI et al were deliberately trying to create artificially 

high barriers to entry to their tech market. For investors actually interested in innovation, this is

ironic given the entire direction of travel of much computing related tech has been to lower barriers 

so that innovation drives things with as low friction as possible (internet, cloud, processors, compilers, operating systems, SDKs/Appstores etc etc).

So up to and including future chip design, and certainly things like edge compute, 

federated machine learning, and of course, all things decentralised...

ad dare one also say warfare - cyber, and hybrid war has even increased the asymmetry 

in cost of effective weapons....  the military example is a very important one often overlooked and i think a large part of the world is scrambling to figure that out, same thing with cyber attacks too...

However its more than artificially high barriers to entry its also creating artificial or at least inflated markets because money goes out as investment and back in as infrastructure (think Microsoft investment or NVIDIA investment) when they aren't needed with a faulty way of valuing all the assets. 

What's best for innovation, and what happens usually in innovation? 

We would think they'd learn the lesson, barriers always lower, things get commoditised, and things get cheaper and easier. This is not always just second system or indeed, third version syndrome - some better understanding of the domain can lead to major efficiencies, and sometimes they arrive combined with other useful innovations - one example arose from work in explainable AI (XAI) where tooling to uncover what structures within a neural network ("deep learning") were responsible for detecting/recognising which input features (and hence classifying an input in some manner) - these tools for explainability also allow one to shrink the neutral network significantly by discarding nodes/edges that serve no useful classifier function - this has been used in face recognition in camera phones to make smaller, faster, and actually potentially more accurate AIs. The cost in training increases somewhat, but the payoff is that the cost in inference (done billions of times rather than just "once") is massively reduced. In some AI models that approach can actually be used during training to reduce training cost too. So an innovation in one space driven by a required feature (explainability) leads to efficiency gains too.

Another angle on this has been the use of physics models (in weather prediction and heavy engineering) combined with neural nets - there's a mutual benefit in reducing the computation costs of computing the physics model, and in optimising the neural network itself -recent advances (e.g. the Aardvaak weather predictor - see https://arxiv.org/abs/2404.00411) actually move the partial differential equations into approximations in the neural net (neural operations for the PDE) gaining huge efficiency, but retaining the fundamental explainability of the original physics. Applying the same technique to  continual updates to the models from real world inputs is another huge win.

Profligacy gets in the way of such giant steps.