Evaluating FAIR-Compliance Through an Objective, Automated, Community-Governed Framework

Mark D Wilkinson, Michel Dumontier, Susanna-Assunta Sansone, Luiz Olavo Bonino da Silva Santos, Mario Prieto, Peter McQuilton, Julian Gautier, Derek Murphy, Mercѐ Crosas, Erik Schultes

Research output: Working paper

52 Downloads (Pure)


With the increased adoption of the FAIR Principles, a wide range of stakeholders, from scientists to publishers, funding agencies and policy makers, are seeking ways to transparently evaluate resource FAIRness. We describe the FAIR Evaluator, a software infrastructure to register and execute tests of compliance with the recently published FAIR Metrics. The Evaluator enables digital resources to be assessed objectively and transparently. We illustrate its application to three widely used generalist repositories - Dataverse, Dryad, and Zenodo - and report their feedback. Evaluations allow communities to select relevant Metric subsets to deliver FAIRness measurements in diverse and specialized applications. Evaluations are executed in a semi-automated manner through Web Forms filled-in by a user, or through a JSON-based API. A comparison of manual vs automated evaluation reveals that automated evaluations are generally stricter, resulting in lower, though more accurate, FAIRness scores. Finally, we highlight the need for enhanced infrastructure such as standards registries, like FAIRsharing, as well as additional community involvement in domain-specific data infrastructure creation.
Original languageEnglish
Publication statusPublished - 16 Sept 2018
Externally publishedYes


Dive into the research topics of 'Evaluating FAIR-Compliance Through an Objective, Automated, Community-Governed Framework'. Together they form a unique fingerprint.

Cite this