GPT-4o mini version tops the lmsys large model arena, tying for first place with the full version and surpassing Claude 3.5. This achievement excited CEO Altman, who stated that GPT-4o mini's performance is comparable to the full version, but at only 1/20 of the price.
The lmsys large model arena uses a user-driven question-asking and voting method for evaluation, which is closer to real user experience. GPT-4o mini's score on this leaderboard is only 7 points lower than the full version, and both are considered tied for first place. Claude 3.5 and the Gemini series follow closely behind.
OpenAI also announced that they will gradually open up GPT-4o mini's fine-tuning functionality, currently available to tier 4 and tier 5 users. From now until September 23, users can use 2 million training tokens for free daily.
The mini version focuses on cost-effectiveness, with prices of 15 cents and 60 cents per million input/output tokens respectively, less than half the price of GPT-3.5 Turbo. Compared to the text-davinci-003 version of GPT-3 from two years ago, the price has decreased by 99%.
Besides OpenAI, companies like Google, Anthropic, Hugging Face, Mistral, and Apple are also launching small models. Small models are gaining more attention as they meet usage needs while offering greater economic efficiency and potential advantages in edge computing.