AI

Over the past week Open AI has opened up its private beta access to its latest natural language processing (NLP) model, Generative Pretrained Transformer 3, or GPT-3 for short. It immediately captivated the Twittersphere with a viral tweet by Sharif Shameem showing the impressive capabilities this optimized version has due to being trained on a way larger dataset of 175 billion parameters compared to his predecessor GPT-2, an increase of two magnitudes. A lot of interesting applications for GPT-3 followed and I was intrigued (as is evident from my tweets on GPT-3) to look into it in more detail to understand what it really could be capable off and how it could impact Bitcoin.

So let’s have a look at:

• what GPT-3 is
• what it is capable off and what not
• why it is important to understand it and its implications

What is GPT-3?

GPT-3 is a neural network for natural language processing and creation. This means that GPT-3 is capable of understanding human language (primarily in English but allegedly capable of understanding eleven languages at point of writing) and reacting to it with text generated by itself that is totally unique, not a regurgitation of pre-scripted sentences.

The API provided by Open AI allows to give GPT-3 inputs in form of words and sentences up to ten paragraphs and it will react with an output based on its 175 billion parameters that it was trained on. This huge set of parameters includes information available on the internet up to October 2019 and therefore does not only contain text but also mathematical formulas and code.

Since it’s a limited beta it’s currently not possible to train the model on a specific set of data as it was possible with its predecessor GPT-2 but that might change in the future. At this point GPT-3 is already far more advanced than any other model available and will be considered the de-facto standard for the time being on the road of development to a General Artificial Intelligence (GAI).

What is GPT-3 capable off and what not?

Since the release of the beta access we’ve seen myriads of use cases tested already and it seems the limits are more limited by human creativity than GPT-3’s limitations. This doesn’t mean that GPT-3 can be considered intelligent but it certainly reached the level of successfully tricking people in believing it to be so when the examples published were chosen carefully.

Some of the most extensive testing of GPT-3 regarding its creative use cases has been done by Gwern, as he did before with GPT-2. Additionally Max Woolf has done a lot of work on GPT-2 and now also put GPT-3 through the motions. Here’s a list of impressive results that were achieved by/with GPT-3:

• Kevin Lacker made GPT-3 take the Turing test to see whether it would pass as a human
• Arram Sabeti had GPT-3 create a poem about by Dr. Seuss about Elon Musk but it took some work
• Mac Woolf had GPT-3 create a unique text about unicorns
• Sushant Kumar made GPT-3 write tweets based on one-word inputs
• Sharif Shameem was able to get GPT-3 to create HTML code based on human language input
• Gwern made GPT-3 create literary content in the style of famous authors like Hemingway

There were many more interesting examples where GPT-3 was able to create SQL and React code, create Figma designs and translate difficult legalese or academic text into understandable explanations for non-experts, all highlighting the fascinating capabilities of the model but also surfacing its limitations.

The creative use cases have shown that GPT-3 became repetitive when creating longer texts, had difficulties rhyming properly and would get into trouble when being asked nonsensical questions. So while very impressive and clearly best in class, GPT-3 is not intelligent and will unlikely be in the near future. A GAI is not realistic yet but it is easily foreseeable that next iterations of GPT will blur the lines between human created works and machine created ones irrevocably.

Why it is important to understand GPT-3 and its implications?

To know where AI and machine learning (ML) is headed, we need to understand where we are. GPT-3 is impressive but even Sam Altman, co-founder of Open AI, is cautioning the hype. So we now have a machine that was able to read and understand a data set of 175 billion parameters – at least to a certain degree. Based on this understanding GPT-3 is able to learn new inputs and try to understand what is asked of it. One example would be that it was able to read 3 rows of Excel data with two populated columns and on the fourth row predict the missing data field based on the first one and how the two columns related to another in the previous rows.

With Moore’s law still intact we can expect the quantity of data a machine can learn and understand to increase exponentially. This exponential expansion makes it difficult to predict for the long- or even mid-term. As Bitcoiners we have a better understanding of exponential growth than many who cannot fathom such increases, but what an ever more knowledgeable machine will be able to do long-term is anyone’s best guess and better located in the world of science fiction.

So short-term, what might be next? I think we can expect GPT-4/5 to be able to communicate like a human so most of us will not be able to identify what is created by a human and what by a machine. This will have far-reaching implications in itself but likely even more in second and third order effects. Texts created by a machine will be translated into other mediums almost without problems. We will see books, audio plays, even movies based on machine generated content. We will have Twitter and Reddit bots filling and leading conversations online, emails will be written between machines; GPT-3 can already write emails based on minimal human input. So the lines will blur online as well as offline – but it will not end at text creation.

Pattern recognition is already strong in GPT-3 and the applications for this are immense. Pattern recognition is one of the most important, innate human capabilities and machines will be so much better at it. We will see constant iteration and continuous improvement on everything from product design, code creation, automation to levels unimaginable today, efficiencies in all disciplines will rise. Even art will not be “spared", the next GPT version might be so good at identifying valuable art and creating new pieces based on existing ones at unhindered speed, ever-creating.

Now we could say that these things will be limited to the digital realm but I wouldn’t be too sure about this. 3D-printing is already picking up speed and moving towards mainstream adoption with smaller and cheaper printers available each year (see also Jeff Booth’s excellent “Price of Tomorrow” on this). It might have a big part to play in this future of constant iteration and optimization by GPT-based machines.

And last but not least there are the financial markets which are already heavily automated and based on neural networks, consuming huge amounts of data to make nanosecond decision to get an edge above the competition. One can only speculate what an openly available (for a price) model like GPT could mean for active trading, it certainly will lead a lot of experimentation.

So where does all this leave us? Some have already wondered whether GPT-3 the "next big thing after Bitcoin” but I wouldn’t go so far just yet. Bitcoin stands on its own and will change the way humanity approaches money all by itself, thereby Bitcoin is still the current and next big thing. So GPT-3 is not the next big thing, GPT-4 might just be. I certainly noticed a leap being taken with the release of GPT-3 and a substantial change in terms of General AI might not be too far out, until then I plan to push Bitcoin adoption along and experiment with the possibilities that GPT-3 opened up.

These are exciting times and I’m keen to see where these technological advances lead to next.

Onward and upwards!