Understanding images is just one way Chat GPT-4 goes beyond its predecessor

The creators behind the increasingly popular ChatGPT tool unveiled a new version of the generative artificial intelligence (AI) tool, known as GPT-4, Tuesday. 

The updated version of OpenAI’s powerful AI represents the next level of what the company’s technology is capable of — from passing a range of tests with flying colors to accepting prompts from images.

But the tool still has its limitations, according to the company’s announcement. 

Here are five things to know about GPT-4. 

GPT-4 accepts visual inputs

One of the most apparent changes with GPT-4, compared to the GPT-3.5 that was powering ChatGPT when it hit the public market in November, is that it accepts visual inputs alongside text. 

That update means users can ask questions based on queries that use images. 

In one example included in OpenAI’s announcement, a user asked “What is unusual about this image?” and included a photo of a man ironing while attached to a taxi cab.

GPT-4 responded, “the unusual thing about this image is that a man is ironing clothes on an ironing board attached to the roof of a moving taxi.”

GPT-4 is also getting up-to-date on meme culture and explaining funny pictures. In another example a user asked “can you explain this meme” with a photo of chicken nuggets laid out to show a map of the world with text that reads, “Sometimes I just look at pictures of the earth from space and I marvel at how beautiful it all is.” 

GPT-4 responded, “The humor in this meme comes from the unexpected juxtaposition of the text and the image. The text sets up an expectation of a majestic image of the earth, but the image is actually something mundane and silly.”

GPT-4 is acing exams

OpenAI’s technology was already making waves over how the tool performed on exams, such as the SAT and Advanced Placement-level tests. The new version is grading well above its predecessor. 

In a simulated bar exam, GPT-4 passed with a score around the top 10 percent of test takers, compared to the bottom 10 percent where GPT-3.5 passed. 

On the SAT reading and writing section, GPT-4 scored a 710 out of 800, 40 points higher than GPT-3.5. On the SAT math section, GPT-4 scored 700, marking a 110 point increase from GPT-3.5.

On AP Biology, GPT-4 went up to a 5 from the 4 that GPT-3.5 received. One of the biggest differences on tested AP exams was with AP Calculus BC, where GPT-4 received a 4, a significant step up from the 1 that GPT-3.5 received. 

But there are other tests where GPT-4 is still faltering. For example, on the AP English Language and the AP English Literature exams, both GPT-4 and GPT-3.5 only received scores of 2. 

GPT-4 can change its personality 

GPT-4 updates the AI’s “steerability” function, meaning it can alter its behavior on command. 

“Rather than the classic ChatGPT personality with a fixed verbosity, tone, and style, developers (and soon ChatGPT users) can now prescribe their AI’s style and task by describing those directions in the ‘system’ message,” OpenAI said. 

For example, when set as a “socratic tutor” and a user asked how to solve a linear equation, GPT-4 responded by saying, “Let’s start by analyzing the equations. Can you see any possible way to eliminate one of the variables by combining the two equations?” rather than spitting out the result. 

In another example, the system was set up as a “Shakespearean pirate.” When asked to help locate a user’s nonqualified plans on their w-2, GPT-4 responded, “Ahoy, dear mate, thou seeketh assistance with matters of taxation, In hopes to decipher thy monetary equation, Forsooth, gaze upon Box 11, with thou eyes keen, Where nonqualified plans oft findeth haven unseen.”

There are still limitations to GPT-4

Although the latest model marks an expansion of the earlier versions, OpenAI said that GPT-4 “has similar limitations as earlier GPT models.”

“Most importantly, it still is not fully reliable,” the company said. 

The AI still “hallucinates,” a term the industry has dubbed for when generative AI tools state false information as fact. 

The company said that GPT-4 generally lacks knowledge of events that occurred after the “vast majority of its data cuts off,” which is September 2021, and doesn’t learn from its experiences. 

OpenAI also acknowledged that the model can have “various biases in its output,” a concern critics have raised about the field of generative AI as tools enter the public sector. 

When and where is GPT-4 available?

Access to GPT-4 is available for ChatGPT Plus subscribers, a $20 monthly subscription. 

It is also the AI powering Microsoft’s new revamped Bing search engine, Bing’s corporate Vice President Jordi Ribas confirmed on Twitter. The search engine is available for a select group of users with advanced access to the new Bing. 

Users will also be interacting with GPT-4 technology through other products and services. OpenAI said in a blog post it has collaborated with a number of organizations to innovate products with GPT-4, including Duolingo, Morgan Stanley and Khan Academy.