What is the Open Access Spectrum Evaluation Tool?

The Open Access Spectrum (OAS) Evaluation Tool quantitatively scores journals' degrees of openness. It uses the HowOpenIsIt?® guide as its basis.

What is the need for this type of program?

While the rapid growth of OA has seen an expansion in the availability of scholarly articles, it has also generated no small degree of confusion within the research community. Many journals claim to be “open” while actually placing moderate or severe restrictions on what an author or reader can do with an article, for example. It has become clear that not all "Open" is created equal. The OAS Evaluation Tool provides independent, expert analysis of journal OA policies beyond just "is this article free to read?"

What is the scoring grid on which OAS evaluations are based?

Why are the values of each of the six categories not equal?

The scale was developed over a period of 16 months, drawing from the input of a wide range of scholarly communication professionals. The team believes that reader and reuse rights have a wider impact on “opening” a journal’s communal value than the other categories. Automatic posting rights have the least impact because, even in their absence, the author can deposit his/her paper in PubMed Central or other trusted repositories provided his/her author posting rights are codified.

Within each category, why aren't the step classifications evenly spaced?

This reflects community input as to the degree of openness that each "step" represents. The “openness gap” between a twelve and six month reader rights embargo is inherently smaller than the gap between a twelve month embargo and the hybrid approach.

Has any journal received a perfect score of 100?

Not as yet. At present, full marks for Machine Readability are difficult to achieve, as a community standard API or protocol does not exist today. Machine readability and the automated discovery and use of content is perhaps the major challenge for scholarly publishers of the next decade. It must become more straightforward for machines to understand where to find articles and to identify the parts of an article. There is also work to be done on improving the way data is represented in research articles (e.g., tables) and more generally how data is described within the article and supplementary material. It is a reasonable observation that the highest rating within the Machine Readability category is currently aspirational.

Does the OAS evaluate publishers, journals, or articles?

OAS evaluations are focused at the journal level. It would be ineffective to evaluate at the publisher level, as many publishers with multiple journals have different policies within these titles. Evaluation at the article level is also inefficient. Individual articles within a given journal may be free to read without the entire journal being fully open. For example, some journals make all research articles free to read after 6 or 12 months behind a paywall. Other subscription journals (sometimes called "hybrid" journals) offer an option for authors to pay a fee and make their articles freely available for anyone to read. Journals with policies that allow for greater access and reuse of some but not all articles are accounted for in the OAS scoring system. In general, these journals are more open than subscription-only journals, and less open than journals that provide immediate open access and reuse rights to all of their published articles.

How do the HowOpenIsIt?® guide and the OAS Evaluation Tool fit together?

The HowOpenIsIt? guide lays out, in descriptive fashion, the array of polices a journal can have in the continuum between "Open" and "Closed". The OAS Evaluation Tool takes these policy distinctions and quantifies them.

Are there any other programs related to the HowOpenIsIt? project?

Yes. The reviews articles (identified by their DOIs or PubMed IDs) and identifies the licensing information associated with these publications. Where a license can be determined the results page gives the name of the license, a link and details of the associated re-use rights. The service is available as a website, an API, and as source code. The OACensus Tool is a reporting utility that builds a relevant list of scholarly outputs and then tests those outputs for accessibility and openness. It is a configurable system that can obtain data from multiple sources to build up a multidimensional view of how an institution, funder, or person is implementing open access. It then uses this information to provide customized reports for the user. Collectively, all the HOII programs are intended to be practical tools the community can use to better understand what it means to be "open".

Who is responsible for the OAS evaluations?

OAS evaluation are performed by a team of scholarly communication, library, and publishing experts, including representatives from DOAJ and librarians from the United States, Europe, and Africa. Collectively, they have decades of experience in the field.

What degree of control do the sponsoring organizations exert over the OAS evaluations?

OAS personnel do not answer to the sponsors. The sponsoring organizations have created the infrastructure for the program but do not exert any control over the evaluations.

Do publishers have a chance to vet their OAS evaluations?

Yes. Publishers are provided with a provisional evaluation and offered the opportunity to provide any clarifying documentation that might impact a journal’s score.

How was the initial batch of evaluated journals selected?

Journals from the freely available Scimago dataset were divided into open access and non-open access journals on the basis of a lookup of their ISSN in Directory of Open Access Journals (DOAJ). The top 600 journals from the non-OA set and the top 200 from the OA set based on ranking by Scimago Journal Rank were selected. Additionally, 200 journals were selected from across the Scielo, Redalyc, Bioline, AJOL, and DOAJ databases to ensure geographic and subject diversity.

What happens if a journal’s policy changes after it has been certified?

Each journal’s certified score will be time stamped to indicate precisely when the evaluation was performed. Should either the publisher or anyone else have reason to believe that the journal’s policies have changed since that time, they are encouraged to .

How is OAS data made available?

The OAS reports are searchable on this web site. Additionally, the raw data are made openly available for reuse via an open API and downloadable file. This allows third parties to apply the data for their own distinct purposes. For example, a research funder could use the raw OAS evaluation data to easily develop a whitelist of journals meeting its public access grant conditions.

What are the licensing rights associated with the OAS data?

The data on the site is made available under a Creative Commons Attribution (CC BY) license. We encourage third parties to use it, and ask only that attribution is given. On the issue of rights, also note that the image on the site banner is attributed to Christian Damasco et al., PLOS Computational Biology. 2008. 4(3) and licensed under CC BY.

If a journal scores above a certain number, is it considered "open"?

Not necessarily. Stakeholders can value specific dimensions of openness differently. For example, campuses that have passed open access declarations typically want to see a copy of the article deposited in their institutional repository. A journal might have a relatively high OAS score but nevertheless prohibit IR deposits. In an instance such as this, the overall score would tell only a partial story.

A journal’s website says it is “Open Access”, yet it received a low OAS score. How is that possible?

In some instances, journals use the term “open access” when they really mean “free to read”. Open access refers to the immediate, barrier-free online availability of scholarly and scientific articles, coupled with the rights to reuse these articles fully in the digital environment. Immediacy, the elimination of barriers, and full reuse are critical components of open access. This allows for accelerated discovery and follow-on research. It provides the widest possible impact for scholarly materials, and the best return on investment. The OAS Evaluation Tool therefore incorporates these elements into its scoring mechanism.

How are “hybrid” journals dealt with in the OAS Evaluation Tool?

Hybrid journals are publications that both collect subscription fees for their journal and offer an option for an author to pay a fee and make their article freely available for anyone to read. The Reader Rights and Reuse Rights dimensions of the OAS each have entries that specifically address hybrids. The OAS Evaluation Tool scoring gives journals limited credit for providing this option, as the default policy for these publications remains one which prohibits unfettered readership and reuse.

Why are some traditional subscription journals included here?

Many subscription journals have policies that address certain aspects of openness, such as allowing authors to post in institutional repositories, or automatically depositing funded papers into PubMed Central. The OAS Tool highlights how even "traditional" journals can promote openness in their own fashion.

I am an editor or publisher interested in getting my journal(s) certified. How do I do so?

Please .