Automated evaluation
Every resource is scored by a script against the FAIR principles, ontology quality (upper-ontology alignment, reasoner consistency, ROBOT report, OOPS! pitfalls), maintenance and community adoption, minus a progressive inactivity penalty. The rubric is deliberately strict: 100 requires excellence on every axis. It queries OLS, TIB, BioPortal, Bioregistry, the OBO Foundry and its Dashboard, GitHub, git refs and Crossref, and reads the artefact itself to record exactly which version was evaluated. Scores describe how easy a resource is to find, trust and reuse today, not its scientific value.
Last run 25 Oct 2026, 19:10 UTC · Rubric v3.4 · 6 archived runs
The formula
Score = max(0, 100 · (F + A + I + R + Q + M + U) / possible − P)
The highlighted group is the FAIR subtotal (max 48). Q (ontology quality, max 20) applies to OWL/OBO/RDF ontologies; for schemas and other resources possible = 80 and the result is scaled to 100, otherwise possible = 100. P subtracts for inactivity (growing every year), logical inconsistency (−20, score capped at 54), unsatisfiable classes (−5), critical pitfalls (−3 each, up to −12) and ROBOT report errors (up to −10).
- F Findable max 12
4·[persistent identifier] + r(registries) + 3·[citable reference]- A Accessible max 12
3·[homepage reachable] + 6·[artefact retrievable over HTTP(S)] + 3·[metadata persists in a registry]- I Interoperable max 12
5·[formal format] + 3·[semantic-web IRIs] + i(imported vocabularies)- R Reusable max 12
license(5 open / 2 other) + version(4 in artefact / 2 elsewhere) + 3·(OBO principles pass rate)- Q Ontology consistency & quality max 20
c(logical consistency) + 2·[OWL 2 DL] + u(upper ontology) + q(quality report)- M Maintenance max 16
recency(months) + cadence(releases in last 12 months)- U Community adoption max 16
a(adopting projects) + reuse(ontologies importing it) + max(stars, citations)- P Penalties (inactivity & inconsistency) 0 to −50
min(50, inactivity + 10·[archived/legacy/OBO-inactive] + 20·[inconsistent] + 5·[unsatisfiable classes] + min(12, 3·critical pitfall types) + e(ROBOT errors)). Inconsistent ontologies are capped at 54 (grade D).
[x] is 1 if condition x holds, else 0.
r(registries): ≥ 4 → 5 · ≥ 3 → 4 · ≥ 2 → 3 · ≥ 1 → 1 · ≥ 0 → 0 · i(imports): ≥ 3 → 4 · ≥ 1 → 2 · ≥ 0 → 0
c (logical consistency): consistent with 0 unsatisfiable classes → 10 · 1–5 unsatisfiable → 5 · more → 2 · inconsistent or unchecked → 0 · u: imports BFO/COB/DOLCE/GFO → 3, RO → 1 · q: no ROBOT errors → 2, ≤ 0.05 warnings/class → 1, no critical pitfalls (OOPS!, else ROBOT) → 1
recency: ≤ 3 months → 10 · ≤ 6 months → 8 · ≤ 1 year → 5 · ≤ 2 years → 2 · older / unknown → 0 · cadence: ≥ 6 → 6 · ≥ 4 → 5 · ≥ 2 → 3 · ≥ 1 → 2 · ≥ 0 → 0
a(projects): ≥ 8 → 7 · ≥ 5 → 5 · ≥ 3 → 4 · ≥ 1 → 2 · ≥ 0 → 0 · reuse: ≥ 30 → 4 · ≥ 10 → 3 · ≥ 3 → 2 · ≥ 1 → 1 · ≥ 0 → 0
stars: ≥ 500 → 5 · ≥ 100 → 4 · ≥ 25 → 2 · ≥ 5 → 1 · ≥ 0 → 0 · citations: ≥ 1000 → 5 · ≥ 300 → 4 · ≥ 50 → 2 · ≥ 10 → 1 · ≥ 0 → 0
Grade: A ≥ 85 · B ≥ 70 · C ≥ 55 · D ≥ 40 · E ≥ 0
Worked example: PCL version 2025-07-07
Actual numbers from the latest run (25 Oct 2026, 19:10 UTC).
| Criterion | Step | Evidence | Points |
|---|---|---|---|
| F Findable FAIR = 9 / 12 | Persistent identifier (F1) | purl.obolibrary.org | +4 / 4 |
| Indexed in services (F4) | 5: OLS, BioPortal, OBO Foundry, Bioregistry, FAIRsharing | +5 / 5 | |
| Citable reference (F2) | no reference publication listed | 0 / 3 | |
| A Accessible FAIR = 12 / 12 | Homepage reachable | HTTP 200 | +3 / 3 |
| Artefact retrievable over HTTP(S) (A1) | download HTTP 200 | +6 / 6 | |
| Metadata persists in a registry (A2) | OBO Foundry, Bioregistry | +3 / 3 | |
| I Interoperable FAIR = 12 / 12 | Formal machine-readable format (I1) | OWL, OBO | +5 / 5 |
| Semantic-web IRIs (I2) | OWL, OBO | +3 / 3 | |
| References other vocabularies (I3) | imports 20: bfo, bspo, chebi, cl, cob, go, iao, mod… | +4 / 4 | |
| R Reusable FAIR = 11 / 12 | Clear license (R1.1) | CC BY 4.0 | +5 / 5 |
| Release provenance (R1.2) | 2025-07-07 (version IRI in the artefact) | +4 / 4 | |
| Community standards (R1.3) | OBO principles: 8/13 checks pass | +2 / 3 | |
| Q Ontology consistency & quality = 15 / 20 | Upper-ontology alignment | imports BFO, COB | +4 / 4 |
| Standard relations (RO) | imports RO | +2 / 2 | |
| Logical consistency | consistent, 0 unsatisfiable classes (OBO Dashboard) | +6 / 6 | |
| ROBOT report: no errors | 1138 errors | 0 / 2 | |
| ROBOT report: warning density | 14875 warnings / 27777 classes = 0.54 | 0 / 2 | |
| Scaled 12/16 → 20 | OOPS! pitfall scan unavailable for this run | +3 | |
| M Maintenance = 2 / 16 | Recency | 2025-07-08 → 15.6 months (≤ 2 years) | +2 / 10 |
| Release cadence (12 months) | 0 releases (GitHub releases) | 0 / 6 | |
| U Community adoption = 5 / 16 | Documented adopting projects | 1 project | +2 / 7 |
| Reused by other ontologies | 3: cl, hra, phipo | +2 / 4 | |
| Community signal | max(stars 12 → 1, citations 0 → 0) | +1 / 5 |
FAIR = 9 + 12 + 12 + 11 = 44 / 48
Penalty P = 1 (1.3 years → 4 × max(0, 1.3 − 1))
Score = max(0, 100 · (9 + 12 + 12 + 11 + 15 + 2 + 5) / 100 − 1) = 65 → 65 ≥ 55 → grade C
Criteria in plain words
F · Findable
12 ptsF1 — the artefact has a persistent identifier: PURL, w3id or DOI (4). F4 — it is indexed in searchable services; counted across OLS, TIB, BioPortal, OBO Foundry, Bioregistry and FAIRsharing: ≥ 4 → 5, 3 → 4, 2 → 3, 1 → 1. F2 — a citable reference publication with a DOI (3).
A · Accessible
12 ptsA1 — the machine-readable artefact (download PURL or repository) is retrievable over HTTP(S) (6) and the homepage resolves (3). A2 — metadata stays available even if the resource disappears, because it is registered in Bioregistry or the OBO Foundry (3). Hosts that refuse automated checks count as reachable.
I · Interoperable
12 ptsI1 — a formal, broadly applicable language (OWL, OBO, Turtle, RDF, JSON-LD, LinkML, JSON Schema, SHACL, YAML schema, XSD) (5). I2 — a semantic-web representation (OWL, OBO, Turtle, RDF, JSON-LD, SHACL) whose terms are resolvable IRIs (3). I3 — qualified references to other vocabularies, measured by imported ontologies (OLS, Bioregistry): ≥ 3 → 4, 1–2 → 2.
R · Reusable
12 ptsR1.1 — a clear license: a recognised open license scores 5, any other stated license 2. R1.2 — provenance of releases: a version IRI or version info inside the artefact scores 4; a release tag or registry version only, 2. R1.3 — meets community standards, measured by the share of OBO Foundry principle checks passed on the OBO Dashboard (× 3). Resources outside the OBO Foundry have no independent compliance report and score 0 here.
Q · Ontology consistency & quality
20 ptsApplies to ontologies (OWL/OBO/RDF artefacts). Logical consistency carries half the weight: a reasoner run (OBO Dashboard, or ROBOT + ELK in our CI) that finds the ontology consistent with no unsatisfiable classes scores 10; consistent with 1–5 unsatisfiable classes 5; more than 5 → 2; inconsistent or never checked 0. OWL 2 DL profile 2. Upper-ontology alignment: imports BFO or COB 3, the Relation Ontology 1. Quality report: no ROBOT errors 2, warnings per class ≤ 0.05 → 1, and a pitfall scan with no critical pitfalls 1 (OOPS! when reachable, otherwise the ROBOT report). Schemas, atlases and other non-ontology resources are not scored on Q; their total is scaled to 100 from the remaining 80 points.
M · Maintenance
16 ptsRecency of the latest commit, release or dated file: ≤ 3 months 10, ≤ 6 months 8, ≤ 1 year 5, ≤ 2 years 2, older 0. Release cadence — distinct releases in the last 12 months (GitHub releases, dated tags, version IRIs across runs): ≥ 6 → 6, ≥ 4 → 5, ≥ 2 → 3, 1 → 2.
U · Community adoption
16 ptsDocumented adopting projects (curated, OBO Foundry usages): ≥ 8 → 7, ≥ 5 → 5, ≥ 3 → 4, ≥ 1 → 2. Reuse — other ontologies that import it (OBO Dashboard, OLS, Bioregistry): ≥ 30 → 4, ≥ 10 → 3, ≥ 3 → 2, ≥ 1 → 1. Community signal, the better of GitHub stars (≥ 500 → 5, ≥ 100 → 4, ≥ 25 → 2, ≥ 5 → 1) and reference-paper citations (≥ 1000 → 5, ≥ 300 → 4, ≥ 50 → 2, ≥ 10 → 1).
P · Penalties (inactivity & inconsistency)
0 to −50Problems are marked down, not just left unrewarded. Inactivity: after a one-year grace period every further year without public activity deducts 4 points (up to 20); archived, legacy or OBO-inactive resources lose a further 10; with no activity evidence at all, 5. Consistency: a logically inconsistent ontology loses 20 and can score no higher than 54 (grade D); one with unsatisfiable classes loses 5. Critical pitfalls (OOPS!, e.g. cycles in the class hierarchy, multiple domains/ranges, ambiguous namespace) cost 3 per pitfall type, up to 12. ROBOT report errors cost 3 (1–9), 6 (10–99) or 10 (≥ 100) for OBO-style ontologies (the report checks OBO conventions). The penalty is capped at 50 and the total never drops below 0.
Which version is evaluated?
Each run records the exact version it looked at, in this order of preference: the owl:versionIRI or owl:versionInfo declared inside the artefact (or data-version / version for OBO, YAML and JSON files); the version reported by OLS; the latest GitHub release; a git tag on the evaluated commit; and otherwise the exact commit of the default branch. The artefact's ETag is stored too. Every entry page shows this under “Version evaluated”, and each run's snapshot keeps it permanently.
Sources & provenance
- Links: HTTP checks of each homepage, repository and download URL.
- Artefact: the first 256 KB of the download, read for version IRI / version info.
- Git refs: default-branch commit and release tags via git's HTTP endpoint.
- GitHub REST API: stars, forks, open issues, last push, latest release, detected license.
- OLS (EMBL-EBI) v2 API: classes, properties, individuals, imports and exports, ontology and version IRI, languages, load time.
- TIB Terminology Service: the same metrics from a second OLS instance, often holding a different release.
- BioPortal (with an API key): latest submission and metrics.
- Bioregistry: cross-portal identifiers (OLS, BioPortal, TIB, FAIRsharing, Wikidata…), dependencies and reuse.
- OBO Foundry registry: activity status, license and registered usages (shown as adopters).
- OBO Dashboard: OBO principle checks, ROBOT report, reasoner consistency and unsatisfiable classes, ontologies that reuse it.
- NeuroSchema CI: for ontologies outside the OBO Dashboard, ROBOT reasons (ELK) and reports on the artefact; every ontology is also sent to the OOPS! pitfall scanner.
- Crossref: citation counts of each resource's reference paper.
- Curated data: adopting projects, modules and tooling submitted through GitHub and reviewed before merge.
Each fact stores its source URL and retrieval time. Every run is archived as an immutable JSON snapshot, so any score can be traced back to the data behind it. If a source fails, the last good value is carried forward with its original timestamp and flagged. A low score often just means information is missing; adding a license, a PURL, a download link or documented adopters will raise it.
All scores
Click a column to sort| Resource | Version evaluated | F/12 | A/12 | I/12 | R/12 | Q/20 | M/16 | U/16 | P | FAIR/48 | Score | Last activity |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CL | 2026-06-08 | 12 | 12 | 12 | 12 | 20 | 16 | 15 | 0 | 48 | A99 | 23 Oct 2026 |
| UBERON | 2026-10-01 | 12 | 12 | 12 | 11 | 19 | 15 | 15 | 0 | 47 | A96 | 24 Oct 2026 |
| GO | 2026-07-26 | 12 | 12 | 12 | 12 | 20 | 12 | 14 | 0 | 48 | A94 | 25 Oct 2026 |
| MONDO | 2026-10-06 | 12 | 12 | 12 | 12 | 18 | 16 | 12 | 0 | 48 | A94 | 23 Oct 2026 |
| FBbt | 2026-10-01 | 12 | 12 | 12 | 12 | 20 | 15 | 6 | 0 | 48 | A89 | 20 Oct 2026 |
| ChEBI | 256 | 12 | 12 | 12 | 11 | 18 | 10 | 11 | 0 | 47 | A86 | 6 Oct 2026 |
| HPO | 2026-09-01 | 12 | 12 | 12 | 9 | 14 | 16 | 11 | 0 | 45 | A86 | 24 Oct 2026 |
| ZFA | 2026-07-16 | 9 | 12 | 12 | 12 | 14 | 15 | 5 | 0 | 45 | B79 | 23 Sept 2026 |
| NBO | 2023-07-04 | 9 | 12 | 12 | 11 | 19 | 5 | 6 | 0 | 44 | B74 | 14 Jan 2026 |
| WBbt | 2025-08-19 | 9 | 12 | 12 | 11 | 18 | 5 | 4 | 0 | 44 | B71 | 11 Mar 2026 |
| MF | 2025-07-08 | 9 | 12 | 12 | 11 | 14 | 5 | 5 | 0 | 44 | C68 | 6 Feb 2026 |
| MFOMD | 2020-04-26 | 9 | 12 | 12 | 10 | 13 | 5 | 4 | 0 | 43 | C65 | 6 Feb 2026 |
| PCL | 2025-07-07 | 9 | 12 | 12 | 11 | 15 | 2 | 5 | -1 | 44 | C65 | 8 Jul 2025 |
| MFOEM | 2025-07-31 | 9 | 12 | 12 | 10 | 14 | 2 | 4 | 0 | 43 | C63 | 17 Oct 2025 |
| AddictO | 2026-10-08 | 3 | 12 | 12 | 4 | 16 | 13 | 2 | 0 | 31 | C62 | 8 Oct 2026 |
| BIDS | v1.11.2 | 3 | 9 | 5 | 7 | n/a | 13 | 9 | 0 | 24 | C58 | 22 Oct 2026 |
| ADO | 2.0.1 | 12 | 12 | 12 | 9 | 16 | 0 | 3 | -8 | 45 | C56 | 21 Sept 2023 |
| NWB | 2.11.0 | 3 | 9 | 5 | 7 | n/a | 13 | 6 | 0 | 24 | D54 | 21 Aug 2026 |
| DANDI schema | 0.14.0 | 0 | 9 | 8 | 7 | n/a | 15 | 3 | 0 | 24 | D53 | 19 Oct 2026 |
| NeuroML | NMLv2.3.1 | 3 | 9 | 5 | 7 | n/a | 10 | 4 | 0 | 24 | D48 | 2 Sept 2026 |
| HED | mouse_v1.0.0 | 3 | 9 | 8 | 7 | 0 | 15 | 3 | 0 | 27 | D45 | 25 Oct 2026 |
| openMINDS | commit 19412fa | 0 | 9 | 8 | 5 | n/a | 10 | 3 | 0 | 22 | D44 | 21 Oct 2026 |
| FMA | 5.1.0 | 12 | 12 | 8 | 6 | 0 | 8 | 4 | -10 | 38 | D40 | 23 May 2026 |
| NIFSTD | v3.1 | 4 | 9 | 8 | 7 | 0 | 8 | 4 | 0 | 28 | D40 | 24 Jul 2026 |
| NPO | v3.1 | 3 | 9 | 8 | 7 | 0 | 8 | 4 | 0 | 27 | E39 | 24 Jul 2026 |
| HOMBA | v2026-04-28 | 4 | 9 | 8 | 2 | 0 | 13 | 0 | 0 | 23 | E36 | 28 Apr 2026 |
| SDS | 0.0.1.dev5 | 0 | 9 | 0 | 7 | n/a | 10 | 3 | 0 | 16 | E36 | 26 Aug 2026 |
| BICAN models | 0.2.0 | 0 | 9 | 5 | 2 | n/a | 10 | 2 | 0 | 16 | E35 | 22 Oct 2026 |
| ReproSchema | 1.0.0 | 0 | 9 | 8 | 2 | n/a | 5 | 3 | 0 | 19 | E34 | 6 Mar 2026 |
| MBA | v2025-07-04 | 4 | 9 | 8 | 2 | 0 | 5 | 2 | 0 | 23 | E30 | 29 Oct 2025 |
| BICANO | v2023-10-04 | 4 | 9 | 8 | 7 | 0 | 2 | 2 | -2 | 28 | E30 | 30 May 2025 |
| BICAN metadata | v2026.04.01 | 0 | 9 | 0 | 2 | n/a | 10 | 3 | 0 | 11 | E30 | 21 May 2026 |
| DHBA | v2025-02-10 | 4 | 9 | 8 | 2 | 0 | 5 | 2 | 0 | 23 | E30 | 13 Nov 2025 |
| Neuroshapes | v0.3.15 | 0 | 9 | 8 | 7 | n/a | 0 | 4 | -6 | 24 | E29 | 22 Mar 2024 |
| CAS | v1.1.0 | 0 | 9 | 5 | 2 | n/a | 2 | 3 | -1 | 16 | E25 | 16 Jun 2025 |
| DMBA | v2025-02-10 | 4 | 9 | 8 | 2 | 0 | 2 | 2 | -3 | 23 | E24 | 10 Feb 2025 |
| HBA | v2025-02-10 | 4 | 9 | 8 | 2 | 0 | 2 | 2 | -3 | 23 | E24 | 10 Feb 2025 |
| NIDM | NIDM-Results_1.3.0 + comm… | 3 | 9 | 8 | 2 | n/a | 0 | 4 | -10 | 22 | E23 | 22 Mar 2023 |
| SCAO | commit a5f68ee | 0 | 9 | 8 | 5 | 0 | 2 | 0 | -4 | 22 | E20 | 25 Oct 2024 |
| CogAt | — | 3 | 0 | 8 | 0 | n/a | 0 | 6 | -5 | 11 | E16 | — |
| PBAO | commit 3ce25f0 | 4 | 9 | 8 | 0 | 0 | 0 | 0 | -6 | 21 | E15 | 30 Apr 2024 |
| CogPO | — | 4 | 3 | 8 | 0 | 0 | 0 | 4 | -5 | 15 | E14 | — |
| CCFv3 | — | 3 | 3 | 0 | 0 | n/a | 0 | 7 | -5 | 6 | E11 | — |
| BITO | commit 0d1ec3c | 0 | 9 | 8 | 0 | 0 | 0 | 0 | -6 | 17 | E11 | 4 Jun 2024 |
| ND | commit 756a205 | 0 | 9 | 8 | 0 | 0 | 0 | 0 | -6 | 17 | E11 | 4 Jun 2024 |
| Julich-Brain | — | 3 | 3 | 0 | 0 | n/a | 0 | 6 | -5 | 6 | E10 | — |
| WHS | — | 3 | 3 | 0 | 0 | n/a | 0 | 6 | -5 | 6 | E10 | — |
| SynGO | — | 3 | 3 | 0 | 0 | n/a | 0 | 5 | -5 | 6 | E9 | — |
| CCN | — | 3 | 3 | 0 | 0 | n/a | 0 | 4 | -5 | 6 | E8 | — |
| InterLex | — | 0 | 3 | 8 | 0 | 0 | 0 | 2 | -5 | 11 | E8 | — |
| NeuroNames | — | 3 | 3 | 0 | 0 | n/a | 0 | 4 | -5 | 6 | E8 | — |
| EPSO | — | 1 | 3 | 8 | 0 | 0 | 0 | 0 | -5 | 12 | E7 | — |
| PDON | — | 1 | 3 | 8 | 0 | 0 | 0 | 0 | -5 | 12 | E7 | — |
| odML | commit ff4108b | 3 | 9 | 0 | 5 | n/a | 0 | 3 | -20 | 17 | E5 | 15 Nov 2019 |
| BAMS | — | 0 | 3 | 8 | 0 | n/a | 0 | 0 | -15 | 11 | E0 | — |
| CNO | — | 1 | 3 | 8 | 0 | 0 | 0 | 0 | -15 | 12 | E0 | — |
| NEMO | — | 1 | 3 | 8 | 0 | 0 | 0 | 0 | -15 | 12 | E0 | — |
Run history
runs/index.json| Finished | Scope | Entries | Avg. score | Trigger | Evaluator | Data |
|---|---|---|---|---|---|---|
| 25 Oct 2026, 19:23 UTC | full | 57 | 36 | local | v2.0.0 / rubric v3.3 | JSON |
| 25 Oct 2026, 19:10 UTC | full | 57 | 39 | push | v2.0.0 / rubric v3.0 | JSON · CI log |
| 25 Oct 2026, 18:49 UTC | full | 57 | 39 | local | v2.0.0 / rubric v3.0 | JSON |
| 25 Oct 2026, 18:21 UTC | full | 56 | 55 | push | v1.2.0 / rubric v2.2 | JSON · CI log |
| 25 Oct 2026, 18:08 UTC | full | 56 | 56 | local | v1.2.0 / rubric v2.0 | JSON |
| 25 Oct 2026, 17:55 UTC | full | 50 | 59 | local | v1.0.0 / rubric v1.0 | JSON |