Evaluating Journals for Quality and Legitimacy

The impact factor (IF) is the most widely-recognized method for attempting to gauge a journal’s rank or importance and is used extensively within the Sciences and Social Sciences.  It aims to demonstrate the frequency with which the “average article” in a journal is cited.  Calculating a journal’s impact factor is fairly simple, the number of citations to a given journal over the previous two years is divided by the number of research articles and review articles published within that journal over that same two-year period.  For example, a journal in 2024 with an impact factor of 2.45 means that the average article in the journal was cited 2.45 times in 2022 and 2023 combined.

Impact factors are often displayed on a journal’s website, but they can be found using the Journal Citation Reports (JCR).  The JCR provides rankings for journals included in the Web of Science database and includes journals in science and social science areas, and to a lesser degree in Arts & Humanities.  So, not only does it portray the impact factor and citation information, the JCR shows how the journal’s impact factor ranks when compared to the impact factor of other journals within the same subject category.

There are three additional ranking scores offered withing the Journal Citation Reports: the Eigenfactor Score (EF), Normalized Eigenfactor (NEF), and Article Influence Score (AI).

The Eigenfactor (EF) score is intended to measure a journal's total importance.  The scores are scaled so that the rankings of all journals listed within their category will equal 100. The Eigenfactor Score calculation is based on the number of times articles from the journal published in the past five years have been cited in the JCR year, but it also considers which journals have contributed these citations so that highly cited journals will influence the network more than lesser cited journals.  References from one article in a journal to another article from the same journal are removed, so that Eigenfactor Scores are not influenced by journal self-citation. If a journal has an EF of 1 then it has 1% of the total influence of all the indexed publications for a given year. That would be a very high score. The size of a journal will influence its score, with larger journals having higher scores.

The Normalized EF (NEF) score rescales the (EF) score so that it is easier to interpret. In this score the average journal has a score of 1, so that a journal with a NEF of 3 has three times the influence of a journal with a NEF of 1.

The Article Influence Score (AI) measures the average influence of the articles in a journal over 5 years.  It is calculated by dividing a journal’s Eigenfactor Score by the number of articles in the journal, normalized as a fraction of all articles in all publications.  The Article Influence Score is considered comparable to the JCR Impact Factor. This score has also been normalized so that the mean AI score is 1.0.  A score greater than 1.0 indicates that each article in the journal has above-average influence.  Scores below 1.0 signify that each article in the journal has below average-influence.

The Eigenfactor, Normalized Eigenfactor, and Article Influence Score may also be found at the Eigenfactor.org website, in addition to the JCR.

Limitations of Impact Factors as metrics: Impact factors must be interpreted carefully and have their limitations as they are only available for a small portion of the total number of peer-reviewed articles.  Use impact factors to only compare journals within the same discipline or subject.

The SCImago Journal Rank, or SJR, is a freely available journal ranking product which provides rankings for journals included within Elsevier’s Scopus Database, and includes science, social science, and arts and humanities titles.

The SJR measures the influence of a journal and is analogous to a journal  impact factor. The SJR algorithm includes both the number of citations and the prestige of the journals from which the citations come. Of two journals holding the same number of citations from other journals, the journal containing citations coming from more highly ranked journals will have a higher SJR. The intent is to look at both the number and the quality of citations. It is very similar to the Eigenfactor scoring model.

CiteScores are Elsevier’s alternative to the impact factor and the scores are calculated based on data from Scopus.  The CiteScore calculates the average number of citations received in a calendar year by all items published in that journal during the preceding three years.  CiteScore is not as established as impact factors but these metrics are available for a larger number of publications.