LLM Model Selection Made Easy: The Most Useful Leaderboards for Real-World Applications

Recently, I am working to integrate the Rag (Badric Knowledge Base) into a chat boot. However, I was not sure how to choose the right LLM model.

When I started researching, I realized that there were countless leader boards available, which made me feel lost. To manage my understanding, I compiled this summary. I hope other engineers feel useful.

1. Leader boards for open source models

Open LLM Leader Board

This is the most famous leader board to compare open source models. It allows filtering based on filtering based on quality -based filtering as a model is provided by a government provider or improved for age devices, making it easier to find.

Open LLM Leader Board

Key standards include:

• iFeval: Evaluates the abilities to follow the instructions
• BBH (Big Bench tough): A challenging benchmark for future abilities
• Mathematics: Maths measure the skill of solving the problem
• GPQA: Google proof questions and answers, examining logical reasoning
• Engage: Tests using detective novels
• mmlu-pro: The language understanding the ability of the language

Large code Model Leader Board

This leader focuses on reviewing models for the board code generation. It provides a performance score for various programming languages ​​such as Azgar, Java, Javascript, and C ++, making it easier to select a model in your target language. The win rate The matriculation gives a overall diagnosis, which is useful if you are making a coding assistant.

Large code Model Leader Board

LLM Perf Leader Board

In real -world applications, accuracy is important, such as the use of speed and memory. This leader board prefers these practical concerns.

LLM Perf Leader Board

Find your best model The tab is especially useful. You can select your GPU sunglasses (such as, A10-24GB-150W in the screenshot) and quickly find the best model.

The chart is very intuitive-models in the left corner attack the best balance between speed and accuracy. Small bubbles indicate less memory use. If you plan to deploy on AWS g5.xlarge For example (24GBVRAM), you can quickly identify the best suitable model here.

2. Domain -related leader boards

The throat face also provides leader boards according to specific domains. If your project belongs to a special field, they are able to check.

Open Medical-LM Leader Board

Medical AI is reviewing the capabilities of a leader board, clinical knowledge and medical genetics dedicated to AI models. It is necessary for health applications.

Open Medical-LM Leader Board

Language Leader Boards

If your project needs to work with Japanese languages, these leader boards can be especially helpful.

Open the Japanese-LM Leader Board

3. To compare the open source and closed source model

Willm Leader Board

This leader board compares both closed source models (eg, GPT, cloud) and open source models (eg, Lama).

Willm Leader Board

It provides detailed insights, including model generation speed, delay, context window size, and per token pricing. In addition, the comparison feature allows you to directly review the new model against the previous generations. If you are deciding between the use of an API -based service or hosted a model yourself, the cost of this leader’s board is invaluable.

Seal Leader Board

This leader board reviews models based on specific practical skills rather than a normal benchmark.

Seal Leader Board

For example, Multichatinage Benchmarks evaluate capabilities such as consistency of conversation, specifications, and information recipes – key factors for chatboat applications.

LMS Chat Boat Arena

Unlike other leaders boards, the mechanical benchmarks rely on human decision -based models. Since it includes user preferences, it can help to predict real -world user experience more accurately.

LMS Chat Boat Arena

Additional views

Even after reviewing models through various leader boards, real -world barriers often order final choice.

In my case, I needed to make the Badric Knowledge Base in the Tokyo area, where only Claude 3.5 Smonts and Claude 3 Hikes were available (and surprisingly, there was not 3.5 hike). In addition, the Tokyo area does not support Imports of Customs ModelWhich restricted my powers.

Finally, I decided to move forward with Claude 3.5 Sant. It performed well, especially after reforms such as the declaring of the inquiry, the FMP, and the Chinking **, which significantly improved the reaction accuracy.

However, the delay in the rotation of the inquiry has increased (the reaction times exceed 10 seconds), so I am currently considering reducing strategies. Possible solutions include:

• Using badick in the Virginia area, accepting cross -region communication overhead but access to Claude 3.5 Hyco.
• Imports of open source model such as Lama as a custom model.

The right LLM is not just about the benchmark. Hopefully, this summary helps others to visit their model selection process!