Methodology

Each stylized fact presented by our organization is published with a reading list of at least 15 references supporting its validity. There is, however, a wide range of information sources available when reviewing literature on a given topic, and so we believe it important to summarize our approach regarding the selection of articles for our reading lists:

 

Scientific articles, institutional research documents, and conferences

Not all research documents are published in scientific journals. Valuable empirical research (often descriptive) is also presented during conferences, or published by non-academic organizations.

Our current policy regarding references excludes research documents from nonprofit organizations and think-tanks, and strives to focus on journal articles and research from public institutions. Public institutions, in this context, will nearly always mean large international or national organizations such as the US’ Bureau of Labor Statistics or the Organization for Economic Co-operation and Development (OECD). In some rare cases, particularly when their findings were so original that they inspired research prior to their official publication, working papers and conferences may be selected for a reading list. This is, however, only in cases where research from these references was used in successful subsequent, peer-reviewed publications. 

 

Types of scientific articles selected

Not all scientific articles are the product of empirical work. Many are presentations of theoretical models which have yet to be validated empirically. Others are literature reviews, focusing on presenting an overview of the findings on a given topic (these are the closest to what our project currently produces). Some also deal in more normative ideas, presenting potential policy responses to prior research.

At the present, our project’s main focus is on empirical testing of the facts we present. Meta analyses and systematic reviews are also used in our analyses of stylized facts. Theoretical and policy articles, however, are rarely added to our reading lists. We understand and value their importance in scientific research, but because we specifically aim to present widely-validated empirical facts, there is little place for them in what we publish.

 

Journals of publication

Journal impact factors can be major sources of headaches for researchers. Submitting an original research to a publication often comes with the worry that the journal may not be prestigious enough in its disciplinary field or may not find the research presented valuable enough for its pages.

Our current policy is to disregard journal impact factors in the construction of our literature reviews. When selecting articles to add to our reading lists, we tend to favor those which have been the most significant on a given topic (number of citations), and which have the closest connection to each stylized fact’s original disciplinary field. Tangential research, building on the stylized facts we’re covering, may end up becoming stylized facts themselves, but to avoid publishing overly complicated posts, we prefer to avoid them.

 

Numbers of journal citations

Journal citations are an aspect of published articles that we do take in consideration when building our reading lists. Although the number of citations is not a guarantee that the article in question is valid or its findings are trustworthy, it tends to offer an idea of just how much discussion it generated in its given field (and others).

Currently, we prefer to avoid giving citation numbers for each source in our reading lists, instead differentiating between ‘significant’ and ‘additional’ reading when covering a stylized fact. The most significant articles tend to be those considered seminal in modelling and testing findings, as well as those which offer additional insight that is of particular importance in understanding a given dynamic. Because so much valuable research is produced on most topics, the difference between ‘significant’ and ‘additional’ sources may appear slightly arbitrary. We deal with limited amounts of space in order to keep published posts accessible, and it’s unfortunately a problem to which we see no easy solutions.

 

Publication dates

Scientific research can move surprisingly quickly, but certain findings may lie forgotten for some time before being re-evaluated and generating more discussion in their fields. While building our reading list, our team always focuses on finding the very earliest research having been done on the stylized fact covered, but also the more recent ones.

Certain findings are so well-known in their respective fields that they may receive very little empirical attention in contemporary research. In the rare event that we cannot find recent-enough scientific articles (as of now, more than five years) that validate the facts we present for contemporary society, we will usually put the topic on ice until further review.