xAI has announced the release of Grok-2, a major upgrade that boasts improved capabilities in chat, coding, and reasoning.
Alongside Grok-2, xAI has introduced Grok-2 mini, a smaller but capable version of the main model. Both are currently in beta on X and will be made available through xAI’s enterprise API later this month.
An early version of Grok-2 was tested on the LMSYS leaderboard under the pseudonym “sus-column-r”.
At the time of the announcement, xAI claims it is outperforming both Anthropic’s Claude 3.5 Sonnet and OpenAI’s GPT-4-Turbo. However, it’s worth noting that GPT-4o currently holds the top spot as the best AI assistant in terms of overall capabilities, followed by Google’s Gemini 1.5.
xAI’s internal evaluation process employs AI Tutors to assess the models across various real-world tasks. The company states that “Grok-2 has shown significant improvements in reasoning with retrieved content and in its tool use capabilities, such as correctly identifying missing information, reasoning through sequences of events, and discarding irrelevant posts”.
Benchmark results shared by xAI indicate that both Grok-2 and Grok-2 mini demonstrate substantial improvements over Grok-1.5. The models show competitive performance in areas such as graduate-level science knowledge, general knowledge, and maths competition problems. Notably, Grok-2 excels in vision-based tasks, delivering state-of-the-art performance in visual maths reasoning and document-based question answering.
The new Grok experience on X features a redesigned interface and new features. Premium and Premium+ subscribers will have access to both Grok-2 and Grok-2 mini. xAI describes Grok-2 as “more intuitive, steerable, and versatile across a wide range of tasks, whether you’re seeking answers, collaborating on writing, or solving coding tasks”.
xAI is also collaborating with Black Forest Labs to experiment with their FLUX.1 model to expand Grok’s capabilities on X.
For developers, xAI is launching an enterprise API platform later this month. The company promises enhanced security features, rich traffic statistics, and advanced billing analytics. A management API will also be available for integrating team, user, and billing management into existing tools and services.
Looking ahead, xAI plans to roll out multimodal understanding as a core part of the Grok experience on both X and the API. The company’s rapid progress since announcing Grok-1 in November 2023 is attributed to “a small team with the highest talent density”.
xAI’s focus remains on advancing core reasoning capabilities with its new compute cluster, as it aims to maintain its position at the forefront of AI development. However, the company recently agreed to halt the use of certain EU data for training its models.
While the release of Grok-2 marks a significant milestone for xAI, it’s clear that the AI landscape remains highly competitive. With ChatGPT-4o and Google’s Gemini 1.5 leading the pack, and other major players like Anthropic continuing to make advancements, the race for AI supremacy is far from over.
