Benchmarking Project · Jupyter Notebook
llm-calculation-benchmark
500 calculation problems with tool-verified ground truth for benchmarking LLM numerical reasoning — generation pipeline, evaluation notebook, report
What it is
The repository provides a 500-problem dataset for evaluating large language models' calculation abilities, including synthetic arithmetic, algebra, and real-world problems, plus competition problems translated via Gemini API. It also includes an evaluation notebook that runs any LLM (via LangChain) on the dataset and reports accuracy by category and difficulty.
Where it counts
- NLP & LLMs90%
- Machine Learning80%
- Data Science60%
How much of the project is about each field, as scoring counts it. It ranks in the field it is most about.
Around it on the NLP & LLMs board
Whole board- 30, shared rankaios-claude-codeML Application · TypeScript58.4AdvancedAngelescu Andrei-CiprianNLP & LLMs
- 30, shared rankatlas-builder-review-and-upgradesAI Developer Tool · Python58.4AdvancedRickNLP & LLMs
- 30, shared rankllm-calculation-benchmarkBenchmarking Project · Jupyter Notebook58.4Advancedjanwan2003NLP & LLMs
- 33atlas-inferenceOther AI Project · Python58.1ImplementationRickNLP & LLMs
- 34, shared ranksnap-cite-livive-netAI Developer Tool · Python57.4ImplementationJarad PeckenpaughNLP & LLMs
