by Helgi Páll Helgason

We now live in a world where generative AI can conjure photorealistic images of pretty much anything we can think of with results that are often indistinguishable from the real thing (this comes with its own set of problems but that’s a topic for another time). Then we have highly potent Large Language Models (LLMs) that can service very complex requests phrased in natural language, OpenAI’s GPT-4 reigning supreme at the moment. Consider that you can make absurd requests, such as…

The image was generated with Midjourney 5.1. The prompt used was simply “a man looking at the generative AI explosion”.

“Prove the Pythagorean theorem in a German poem and then list the elements in the periodic table in Chinese”

… and the model will usually generate a correct result from scratch in mere seconds. The same goes for more useful requests such as writing a piece of code and reviewing, rewriting or even generating written content on almost any topic. These examples just begin to scratch the surface of what is possible.

It is clear that LLMs at their present state of development can create significant business value already, but these models have limitations that are sometimes overlooked.

In the midst of the storm of progress and activity currently taking place with Generative AI, I’d like to stop for a moment to reflect, and offer a grounded and practical assessment.

LLMs are very large artificial neural networks. It is sometimes said that they simulate the inner workings of the human brain, but this is true to a far lesser extent than commonly perceived. Since neural networks were first introduced in the 80s, it has been well understood that they are function approximators. Even with the introduction of new features (e.g. attention) and new architectures (e.g. transformers) this fundamental nature remains unchanged. Although simplified, you can think of how they work as learning to map a set of training data points to their correct result values and then interpolating between these data points when given novel data. While often very effective, there is no guarantee that this will always produce correct results. An approximation of a function is not the same as the actual function. As statistician George Box famously said, “All models are wrong, but some are useful”.

Continue reading A Grounded Assessment of the Generative A.I. Explosion →

A recent article published in Computer Weekly describes discusses a paper by IIIM Managing Director Kristinn R. Thórisson, titled “The Future of AI Research: Ten Defeasible ‘Axioms of Intelligence’“. The paper, which he co-authored with Henry Minsky, son of AI pioneer Marvin Minsky and co-founder of Leela AI, details why contemporary methodologies in AI will not lead to general machine intelligence, and outlines the principles for charting a new course for the research field.

Published in Proceedings of Machine Learning Research vol. 192, the paper charts a visionary course for the field of AI, stating that:

“…to create autonomous self-contained systems that can rival human cognition—machines with ‘human-level general intelligence’ […] calls for a new kind of system that unifies in a single architecture the ability to represent causal relations, create and manage knowledge incrementally and autonomously, and generate its own meaning through empirical reasoning and control.”
(Thórisson & Minsky, 2022, p. 6)

The Computer Weekly article describing the paper and its main message, which the author says may lead to future machines with general intelligence. Contemporary AI approaches, according to the paper’s authors, are limited in that “… the current focus on deep neural networks is hampering progress in the field.”

“Being exclusively dependent on statistical representations – even when trained on data that includes causal information – deep neural networks cannot reliably separate spurious correlation from causally-dependent correlation,” says Thórisson. “As a result, they cannot tell you when they are making things up. Such systems cannot be fully trusted. … The only way to address the challenge is to replace top-down architectural development methodologies with self-organising architectures that rely on self-generated code. We call this ‘constructivist AI’.”
(Pat Brans, Computer Weekly)

Related article:
Computer Weekly: “IIIM Could Revolutionize AI”

A recent article by Jan Petter Myklebust explains how ChatGPT is having an effect on university teaching and evaluation. The article, published on March 4, 2023, titled “Universities adjust to ChatGPT, but the ‘real AI’ lies ahead,” quotes IIIM’s Director and Founder Dr. Kristinn R. Thórisson in the heading. Thórisson suggests that while technologies like ChatGPT are rather far from real intelligence, it provides the world with some good exercise in what may lay ahead in the coming years and decades of AI research.

The article covers academic developments related to AI in Norway, Finnland, Sweden, Denmark and Iceland. According to the article’s author, the Technical University of Denmark has officially banned the use of artificial intelligence in their exams, while Professor Philip John Binning, the university’s dean of graduate studies and international affairs, says that the institution does not have a good recipe to handle the issue. He considers technologies like ChatGPT as “contributing to the creation of a new reality,” comparing its impact to the advent of the Internet.

Dr. Thórisson, who has worked for 30 years on artificial general intelligence projects, applied AI projects, in both academia and industry, predicts that over the next 3 decades new methodologies will take over those based on artificial neural networks (which, despite the name, have very little to do with the ‘real’ neural networks found in nature), resulting in significant advances for automation, creating more trustworthy methods for achieving all sorts of control that will transform numerous industries and social arenas.