xAI releases Grok-2

xAI released Grok-2 and a smaller Grok-2 mini in beta, its first major model update since Grok-1.5 and a significant capability jump for a company that had launched its first model only nine months earlier. Both variants were made available to X Premium subscribers rather than through a separate consumer product, continuing xAI’s strategy of distributing its models through Musk’s social platform rather than a standalone chat app.

xAI reported that Grok-2 was competitive with GPT-4-Turbo and Claude 3.5 Sonnet across academic benchmarks including graduate-level science questions (GPQA), general knowledge (MMLU and MMLU-Pro), and competition mathematics (MATH), and claimed particular strength on visual reasoning tasks such as MathVista and document question-answering. Independent of xAI’s own claims, an early version of the model — entered anonymously into the LMArena Chatbot Arena under the codename “sus-column-r” — placed second on the platform’s crowdsourced Elo leaderboard once its identity was revealed, ranking first on the arena’s mathematics category and second on coding and hard prompts, ahead of the contemporaneous GPT-4o release and tied with Google’s latest Gemini model.

The release’s more notable addition was image generation, built on FLUX.1, a model from Black Forest Labs — a company founded months earlier by former Stability AI researchers. Unlike most rival image generators, Grok-2’s integration carried few content restrictions, and it was quickly used to produce depictions of copyrighted characters, public figures and other content that competitors’ safety filters typically blocked, drawing criticism that xAI had prioritised capability and speed to market over the moderation standards other labs had adopted.