February 20, 2024
GPT-5 Predictions: What We Know So Far
Author:
CSO & Co-Founder
Reading time:
6 minutes
It’s been six years since OpenAI announced its groundbreaking large language model, GPT-1. Four iterations later, we’re anticipating the fifth generation of GPT. Unsurprisingly, the AI community is already buzzing with speculation- some true and some untrue, about what the future of GPT-5 holds.
Every iterative generation of GPT brought significant improvements and a couple of surprises, and GPT-5 will likely be no exception. GPT 4, for instance, introduced a multi-modal system that could accept images as input and produce text output. OpenAI filed a trademark application for the tag “GPT-5” with the United States Patent and Trademark Office (USPTO) in July 2023, meaning GPT-5 is right around the corner [1].
Read on to gain insight into what the fifth GPT iteration has to offer. We’ll highlight our top GPT-5 predictions and everything we know so far about the fifth GPT model. But first, let’s take an overview of previous GPT versions.
Insights from past GPT models
Open AI launched its first large language model with transformative architecture, GPT-1, in June 2018[2]. GPTs, or Generative Pre-Trained Transformers, utilize deep-learning technology and decoder-only architecture to “comprehend” input and give output resembling natural human language. It is a breakthrough technology, but OpenAI wasn’t done yet.
Roughly a year after releasing GPT-1, OpenAI introduced its second GPT iteration with massive improvements. While GPT-1 had 117 million parameters, GPT-2 boasted a whopping 1.5 billion parameters [3]. GPT-2 was also trained on a larger data set, unlike its predecessor, which was trained with unsupervised learning. Other improvements include a modified training algorithm and a more efficient text-generating sampling algorithm.
The third iteration, GPT-3, was introduced in 2020 and saw even more significant improvements, jumping from 1.5 billion parameters to 175 billion. It was also trained on a larger dataset and had improvements like the Gshard training methodology and few-shot learning capability.
OpenAI then introduced GPT 3.5 two years later, the first GPT model with 3 parameter variants. These variants had 1.3 billion, 6 billion, and 175 billion parameters each. Its biggest improvement, however, was reinforced learning with human feedback, which enhanced the relevancy, accuracy, and ethics of outputs.
The latest GPT version is GPT-4, released in 2023. As mentioned, this is OpenAI’s first multi-modal version and boasts a staggering 1 trillion parameters [4]. Its multi-modal system accepts images and text as input and produces the desired output.
Read more: What is an OpenAI API, and how to use it?
GPT-5 predictions: What we know so far
Information from reputable online sources and tweets by OpenAI’s president, Greg Brockman, has shed light on what GPT-5 offers. Here’s what we’ve learned so far.
Improved Accuracy
“Hallucinations” have been a major issue with previous GPT models. These hallucinations occur when the models present non-factual information as facts. This happens when AI models learn incorrect patterns from incomplete or biased data sets.
GPT-4 saw significantly fewer hallucinations than its predecessor, but we could see even better results with GPT-5. The fifth iteration is expected to have 10% fewer hallucinations than the fourth one, leading to improved output accuracy.
Data Training
Training data sets sit at the core of every successful GPT iteration, with successive iterations having larger data sets for training. GPT-5 is set to follow the same trend with an even larger data set from publicly available sources and proprietary information from private companies. It will take full advantage of OpenAI’s GPTbot to scour publicly available information to train the AI model.
Full-Circle Multimodality
Bard (rebranded for Gemini) and Bing Chat were forerunners on the multimodal front. OpenAI is looking to catch up and will likely introduce comprehensive multimodality to GPT-5. This means it will not only accept text and images but will also allow video and audio input and generate similar output. All this is speculation at the moment, but aligns with OpenAI’s mission to enhance usability.
GPT-5 predictions: Anticipating the innovations of GPT-5
GPT-5 could introduce new technologies, extending its capabilities and functionalities. Some innovations we expect in GPT-5 include:
Internal Autonomous Agents
Some tasks may be too complicated for simple LLMs, hence the need for internal autonomous agents. Internal autonomous agents refer to a network of specialized sub-agents that the AI model will delegate complex tasks to. These complex tasks include mathematics, programming, and bug testing.
Retrieval-Augmented Generation (RAG)
Retrieval-augmented generation is a method of optimizing LLMs to reference credible sources outside its dataset and produce quality and accurate output. Enhanced RAG will likely be a major selling point for GPT-5, coupled with the ability to recall previous interactions and contextually apply them to future prompts. The result is reduced hallucinations and almost human-like logic.
Use of Agency and Advanced Tools
GPT-5 is anticipated to learn by observation by utilizing agency and advanced tools. This would enable it to learn how to perform tasks by observation and then execute the tasks autonomously. The utilization of agency and tools is still a subject of debate- some are skeptical about the concept, while others show cautious optimism. However, given OpenAI’s ambitions to improve the AI model’s utility, they’re likely to pull it off.
Final Thoughts: From GPT-4 onwards
The future holds immense promise for the AI community. From the look of things, GPT-5 could be a significant upgrade from GPT-4. It is likely to transform various industries and enhance the way we interact with AI. However, its capabilities remain speculative until it’s trained and unveiled. Hopefully, the new GPT will be released soon. As for now, all we can do is wait and hope for the best.
References
[1]Searchenginejournal. com. OpenAI Files Trademark Application For GPT-5
https://www.searchenginejournal.com/openai-files-trademark-application-gpt-5/493040/. Accessed on February 14, 2024
[2]Forbes. com. A Short History Of ChatGPT: How We Got To Where We Are Today
https://www.forbes.com/sites/bernardmarr/2023/05/19/a-short-history-of-chatgpt-how-we-got-to-where-we-are-today/?sh=555bf9b2674f. Accessed on February 14, 2024
[3] Techtarget.com. ChatGPT vs. GPT: How are they Different?
https://www.techtarget.com/searchenterpriseai/feature/ChatGPT-vs-GPT-How-are-they-different. Accessed on February 14, 2024
[4] Semrush.com. What Is GPT-4? Key Facts and Features [August 2023]. https://www.semrush.com/blog/gpt-4/. Accessed on February 14, 2024
Category:
