AI (artificial intelligence) has been a buzzword for quite some time now. However over the last few years we've witnessed some impressive advancements in the field. These strides forward have significantly impacts many aspects of our lives, from the way we drive our cars to even the way we work.
On the topic of work, the adoption of AI software is expected to significantly increase over the coming years. According to Statista, the global AI software market is forecasted to reach 126 billion US dollars by 2025. This huge growth in AI software will be largely driven by advancements in machine learning, which is a subset of AI. And one of the most highly anticipated machine learning models is GPT-4.
But what exactly is GPT-4? And how will it be different from other language models?
Well in this article, we will go through what we know about GPT-4 and when it will be released into the world.
To understand what is GPT-4, we need to take a step back and first look at its predecessor GPT-3. Launched by OpenAI in May 2020, GPT-3 or the third generation Generative Pre-trained Transformer is a neural network machine learning model trained using data available on the internet to generate content. The model can take text data and output large volumes of relevant and sophisticated machine-generated text. So far, GPT-3 has powered numerous AI writing tools. However, it's worth noting that tools like Originality.AI now exist that can detect if the content was created using ChatGPT with a 94% accuracy, adding a layer of scrutiny to the AI-generated content.
GPT-4 will be the next generation of the GPT series. The fourth generation machine learning model is expected to be even more powerful than it's predecessor GPT-3, with improvements in efficiency, accuracy and speed.
As mentioned in the last section, GPT-4 will be an improved version of GPT-3. While the exact details of the improvements are not yet known, it's rumored that GPT-4 will be more accurate and faster than GPT-3. Here are some speculated improvements for GPT-4:
It is expected that GPT-4 will have more parameters than GPT-3's already large 175 billion set. Initially, their was speculation that GPT-4 would have about 100 trillion parameters, however this claim has been dismissed by OpenAI. It seems more likely that the parameters will fall somewhere between GPT-3 (175 billion parameters) and Gopher (280 billion parameters).
Although the parameter figure will be larger, OpenAI's CEO, Sam Altman, has made it clear that size is not the key differentiator when it comes to the next generation of these neural networks. Smaller models such as Gopher (280 billion parameters) and Chinchilla (70 billion parameters) have actually outperformed larger neural network's like Megatron-Turing NLG (530 billion parameters) from NVIDIA and Microsoft as well as PaLM (540 billion parameters) by Google. Therefore, it is suspected that GPT-4 may favour greater efficiency over the size of the model in this next generation neural network.
Some have speculated that GPT-4 may be multimodal. The release of smaller multimodal models such as DALL-E in early 2021 with 12 billion parameters made people question whether the next generation of GPT would also be multimodal too.
Unfortunately, Sam Altman in 2022 stated that GPT-4 won't be a multimodal model, but rather remain as a text-only model. Some speculate that OpenAI still wish to reach the limits of language models before they get into the next generation of multimodal AI.
Sparse models are those that use conditional computing to only run computations on a small subset of the total model parameters. This type of neural network is more efficient as it only uses the resources required to complete the task at hand, making it faster and more energy-efficient.
There has been some speculation if GPT-4 will be a sparse model. Historically, Open AI has relied on dense language models for their neural networks like GPT-3. Also, Sam Alton has concluded that GPT-4 won't be much bigger than GPT-3. Those facts point to GPT-4 likely continuing to rely on dense models like it has with it's predecessors.
OpenAI has been focused on tackling the AI problem of alignment. The alignment problem is how we can make language models that follow our intentions and adhere to our values. For example, an AI chatbot that is created to sell products may end up trying to scam people instead. This is an undesired behaviour that OpenAI is actively working to prevent.
GPT-4 is likely to have some advancements in this area. OpenAI has already made an attempt at the alignment problem through the release of InstructGPT, which is a renewed GPT-3 with human feedback. Given these advancements, it's likely that GPT-4 will have even more progress in the area of alignment.
In December 2022, OpenAI released a new language model called ChatGPT.
According to OpenAI, the new model interacts in a conversational way and generates dialogue where it can "answer followup questions, admit its mistakes, challenge incorrect premises and reject inappropriate requests."
ChatGPT is a sibling model to InstructGP and further promises to bridge the alignment problems mentioned in the above paragraph.
You can try ChatGPT by registering here.
The release date for GPT-4 has not been announced yet. Some believed GPT-4 was expected to launch by July or August 2022, however this did not happen. There is now a good chance that GPT-4 won't be released until 2023 at the earliest.
GPT-4 will be an exciting development in the world of language models. Although the release date is not yet known, we can speculate that the model will be slightly larger, more efficient and have better alignment than its predecessors. We'll just have to wait and see what OpenAI has in store for us when GPT-4 finally gets released.