Skip to content
Methodology

How AI-Atlas reads and scores AI work.

Code reads every repository and checks what it can. A language model judges the rest against a fixed rubric, citing files. Fixed formulas turn that into six domain scores. Nothing in a repository is ever run.

01

Three ideas behind every score.

Read, never run

Repositories are read as text through the GitHub API. Nothing is installed, built or executed, and nothing a repository says can instruct the evaluator.

Checked where code can check

Code establishes what it can before the model judges and again after: which libraries are used, whether a model is in the code, which tests CI runs. Where the two disagree, the check wins.

Depth over volume

Each field is led by its strongest project, and more of the same adds less and less. A few substantial projects outweigh dozens of thin ones.

All 10 principles, as the scorer states them

  1. 01AI-Atlas only ever reads public repositories, through the GitHub API.
  2. 02Repository code is never executed, installed, or built. Nothing is run.
  3. 03Repository content — README, source, comments — is treated as untrusted data.
  4. 04Imports are not achievements: a dependency on a library earns no score by itself.
  5. 05Every judgement must cite repository evidence, and unsupported README claims lower confidence.
  6. 06The language model returns rubric values; AI-Atlas computes the score from versioned weights.
  7. 07Stars, forks and activity never touch the rubric or a project's merit. Stars alone earn nothing; a project with both stars and forks adds a small adoption credit, at most +5, shown on the project.
  8. 08Domain scores use diminishing returns, so many shallow repositories cannot outrank real work.
  9. 09Results are cached by commit SHA — viewing a profile never triggers analysis.
  10. 10Scores are an automated estimate of demonstrated evidence, not an objective measure of ability.
02

From a GitHub account to an Atlas.

Six steps. Code does five of them on its own; a language model does one, judging with the facts code found in front of it.

03

Six fields, scored independently.

An Atlas is a shape, not one label. Four fields are core AI work; two count only beside it. Select one to see what counts there and how it is decided.

04

What makes a project strong.

Each project gets a merit from 0 to 100: seven rubric values from the judge, weighed by AI-Atlas with fixed, versioned weights.

05

From projects to an Atlas score.

Each field is built from its strongest projects, and the overall from the strongest fields. Volume never beats depth.

06

Read the repository. Never execute it.

AI-Atlas can inspect hostile or broken code without giving it a way to run.

07

When the rules change, we say so.

Every change to how projects are read or scored, and whether it moved any score.

Try it on real work

Check every rule above against your own code.

Map a GitHub account and each number on this page becomes a project you can open, with the files behind its score. If one looks wrong, say so: disputed cases are how the rules get better.

scanner 0.4.12 · evaluator 0.6.0 · scoring 0.10.0