Read, never run
Repositories are read as text through the GitHub API. Nothing is installed, built or executed, and nothing a repository says can instruct the evaluator.
Code reads every repository and checks what it can. A language model judges the rest against a fixed rubric, citing files. Fixed formulas turn that into six domain scores. Nothing in a repository is ever run.
Repositories are read as text through the GitHub API. Nothing is installed, built or executed, and nothing a repository says can instruct the evaluator.
Code establishes what it can before the model judges and again after: which libraries are used, whether a model is in the code, which tests CI runs. Where the two disagree, the check wins.
Each field is led by its strongest project, and more of the same adds less and less. A few substantial projects outweigh dozens of thin ones.
All 10 principles, as the scorer states them
Six steps. Code does five of them on its own; a language model does one, judging with the facts code found in front of it.
from GitHub account
Discover
300repositories per scan; 1,000 for an organisation
Lists the account's public repositories, most recently updated first, and pins each one to a commit. Every later step works from that one snapshot.
Output
Repository manifest
One entry per repository, each pinned to a commit
from Repository manifest
Filter
0model calls: fixed rules decide
Fixed rules decide which repositories hold AI work, from names, topics, languages, the README and the file tree. Before one is left out, the libraries it declares or imports are checked too. Infrastructure with no model in it does not qualify.
Output
Own AI work
The repositories worth reading; every other one gets a stated reason
from Own AI work
Read
24files per project, at most
Fixed rules pick the files that explain the project across its modules, files its README links to first, in any language, alongside the whole file tree. A long README is condensed so every section is read, not just its opening.
Output
Files and file tree
What the judge will see of each project
from Files and file tree
Check
Code turns the files into facts the model cannot argue with: which libraries are used, whether a model is in the code, where a model is called, which tests CI actually runs. Nothing is executed.
Output
Verified facts
Stated to the judge, and checked again when scoring
from Files and verified facts
Judge
7rubric parts, each citing files
A language model reads the files and the facts and rates the project on a fixed rubric, citing a file for every judgement. It returns rubric values, not the score.
Output
Rubric values
A value per rubric part, each with the files behind it
from Rubric values
Score
12projects count per domain, at most
Fixed, versioned formulas weigh the rubric into project merit, apply the checks (Machine Learning and MLOps need a model in the code), and build each domain from its strongest projects.
Output
Skills matrix
Six domain scores and the Atlas score
Why a repository can be left out · 8 reasons
Only repositories with AI work are scored. Every one that does not count is listed for its owner with one of these.
An Atlas is a shape, not one label. Four fields are core AI work; two count only beside it. Select one to see what counts there and how it is decided.
How the six fields relate
Core AI work
ML
Model architectures, training loops, optimisation, classical and deep learning method work — the mathematics and mechanics of learning systems.
Counts
Training code, model design, baselines and measured results, in any language, with or without a framework.
Not enough alone
Calling a hosted model, or an environment or solver with no learning in it.
How it is decided. The judge rates how much a project is about it, but it counts only if code finds a model in the files read: an ML library, learning code, or a model file.
Machine Learning covers learning in general; Vision applies it, or classical methods, to images and video. Training an image classifier can count for both.
Each project gets a merit from 0 to 100: seven rubric values from the judge, weighed by AI-Atlas with fixed, versioned weights.
How much was built, and how far below the surface it goes.
How hard the problem and the ideas behind it are.
What is new, in the idea or in how it is built.
A working path from input to result, not a fragment.
Whether it solves a real problem for someone.
How well the README and the code explain it.
Whether someone else could run it and check the result.
A project is judged at one commit, and its page links to that code and to what changed since. Later pushes count once the owner rescans: nothing is rescanned on its own, and an unchanged project keeps its judgement instead of drifting between scans.
Machine Learning counts only if the files read import or declare an ML library, contain learning code (a backward pass, a loss, an optimiser, a training loop, a layer) or the tree holds a model file. MLOps also accepts a model runtime or tracking tool (ONNX Runtime, BentoML, MLflow, Hugging Face Hub...). Otherwise the judge's relevance to them counts as 0. Calling a hosted model's API is not evidence of either.
Checked by code
Still the judge's call
A project's relevance to either is capped at 0.1 + 1.5 × its strongest Machine Learning, Computer Vision, NLP & LLMs or AI Agents relevance. With none, it stays under the 0.12 a field needs to count. MLOps whose model the code shows (a pretrained model downloaded, served and tested) is not capped: the check above answers it.
Documentation starts with the README. A repository without one has 20 points taken off Documentation (never below 0), noted on the project. Adding a README restores them on the next scan.
Stars, forks and activity never touch the rubric or a project's merit. A project with at least 100 stars and 10 forks adds a small adoption credit when it counts towards a domain, at most +5, shown on the project. Stars alone earn nothing.
Forks are left out, and so is a repository whose commits are almost all other people's (an imported project), whether one author wrote it or a whole community did, as in a re-uploaded copy of a large open-source project. A project that describes itself as a fork or mirror, or whose licence names another copyright holder, is also checked by a person: if it is someone else's work it is set aside like an imported project, and its Atlas is recomputed from the stored evaluations, without a new scan. Another copy of a stronger project in the same field (“-main”, “-v2”, a random tag) is damped to 35% rather than counted twice, and an archived repository counts at 90%.
Headroom, as roadmaps and predictions use it
What a rubric part can still add: the points it is missing, times its weight. A ceiling, never part of a score.
Each field is built from its strongest projects, and the overall from the strongest fields. Volume never beats depth.
Relative pull of each project on its domain, strongest first: 100%, 12%, 8%, 5%, 4%, 2%, 2%, 1%, 1%, 0%, 0%, 0%.
Projects in the domain, strongest first · bar height = pull on the score
Leaderboards
An Atlas joins the global, country, field and project boards once its owner claims it by signing in with that GitHub account.
Anyone can map a public account, but an unclaimed Atlas stays off the boards and out of search results until its owner verifies it.
GitHub organisations rank on a board of their own, by the same score, claimed or not. A member can claim one from its Atlas.
AI-Atlas can inspect hostile or broken code without giving it a way to run.
No script, package, build, binary or command from a repository is ever run. Files are read as text.
README text, comments and prompt-injection attempts are project content. They cannot change what the evaluator does.
Code that would attack a machine if run is still only inspected as evidence. Its payload never executes.
What it reads
What it cannot see
Known limitations · 8
What a score can and cannot tell you, stated plainly.
Every change to how projects are read or scored, and whether it moved any score.
Next: every ranked Atlas is re-scanned. Over the coming days we re-scan ranked Atlases with a stricter judge that scores the AI work in a project, not the whole app around it. Scores may move again. You don't need to do anything.
Try it on real work
Map a GitHub account and each number on this page becomes a project you can open, with the files behind its score. If one looks wrong, say so: disputed cases are how the rules get better.
scanner 0.4.12 · evaluator 0.6.0 · scoring 0.10.0