Home
Quick reference links:
- Generation 1 Metrics
- Generation 2 Maturity Indicators
- FAIR Evaluator
- How do I create a new Maturity Indicator?
- How do I run a FAIR Evaluation?
- FAIR Maturity Mailing List
Evaluating FAIRness is a controversial issue! We (the Maturity Indicator Authoring Group) feel that these concerns can be eased through bringing more clarity regarding the "philosophy" behind FAIRness testing.
First, there is no such thing as "FAIR", and neither is there "unFAIR"! As described in this manuscript, we view FAIR as a continuum of 'behaviors' exhibited by a data resource that increasingly enable machine discoverability and (re)use. Moreover, the final FAIR sub-principles (specifically R1.2 and R1.3) speak directly to the fact that "FAIR" will have different requirements for different communities! Thus, a given Maturity Indicator (and its associated Maturity Indicator Test) may not be applicable to certain types of resources from a given community; or, alternatively, there may exist in that community some standard that is widely accepted within that community to enable machine-actionability of a data resource, but that would not be recognized by a "generic" computational agent. This is all completely acceptable, and is the reason that the Maturity Indicators and the Maturity Indicator Testing framework have been established as a community-driven initiative, rather than top-down.
For example: In the bioinformatics community, a widely used format for Sequence Features is GFF3 - a tab-delimited format with specific constraints on the content of each field. From the perspective of a "generic" computational agent, it would be ~impossible to interpret or reuse that data; however, there are constraints on that format that speak directly to the interoperability objectives of FAIR - for example, that the "type" field must consist of a term from the Sequence Ontology. Thus, it would be reasonable for the Bioinformatics community to create a FAIR Maturity Indicator, and its associated Maturity Indicator Test, that evaluated if a GFF3 file fulfilled those requirements. Thus, a community could create a Maturity Indicator Collection that eschewed the "generic" test for machine-readability of the data, and instead, utilized their own Maturity Indicator Test that validated the content of that GFF3 file.
In light of this, we insist that FAIR evaluations are not intended to be used as "judgement", but rather as a means to objectively (AND TRANSPARENTLY!) test if a resource has successfully fulfilled the FAIRness requirements that that community has established. In this light, FAIR Evaluation is a way for individual providers, or repository owners, to test their own compliance, and to take remedial action if their resources are not passing the tests. Confirmation and/or guidance, rather than judgement.
Certainly, we believe that some FAIR Maturity Indicator (for example, that the entity has a globally unique identifier, and that all data should have a license and citation information) are universal, and a prerequisite for FAIRness; however, many of the FAIR Principles must be interpreted by the individual communities, keeping as close to the "spirit" of FAIR as possible.