The Economist’s Pursuit of (Un)biasedness
By: Christian Tangkere
Economics has undergone significant development through the extension of its inquiry on many aspects of the society. The application of formal approaches developed in economics to many disciplines has widened the economists’ role and enlarged their influence in the realm of policy making. It is then very important for economists to commit themselves for the integrity of the profession, since their technocratic works will have a huge impact on society. One key understanding to develop the integrity of economics profession, both in academia and applied sphere, is the understanding of bias in every scientific activity an economist may conduct. It is important to note that this writing does not deal with the discourse about positive versus normative approach in economics or the debate about ethical consideration in economic inferences. This writing deals with the issue of integrity of the economists in pursuing their work, including how they anticipate any potential biases.
Some evidences
One of the evidences of bias in economics research is found in the investigation done by Ioannidis et al (2017) upon the credibility of empirical works, with the focus on statistical power and bias. In this study, 159 meta-analysis literatures which contain 6700 empirical works are assessed. Figure 1 shows the study’s finding on the distribution of empirical studies’ results according to the level of research inflation which is defined as “the empirical lower bound of the magnitude of [the] exaggeration of reported results for a given area of economics research.” (Ioannidis et al, 2017). From figure 1, it is shown that around fifty percent of statistical effects found in observed empirical studies are upwardly biased by a factor of two or larger. This finding is one of alarming evidences about the credibility problems in the economics research field, and thus studying the sources and mechanism of bias can repair the trust of the public to the economics research.
Figure 1. Distribution of empirical studies according to the magnitude of bias.
In general, the field of work of an economist can be generally classified into applied realm and academic realm. Each field of work presents a unique context upon which potential biases can flourish. It is important, therefore, to discuss the source and mechanism of bias separately between these two realms.
Bias in applied realm of economics
One of the notable inquiries of the way biased studies are produced by applied economists can be found in a work done by DeMartino (2011) on the professional economic ethics. Economists who serve in the applied realm are embedded in the operation of the institutions they join, and thus are susceptible to the external pressures and limitations in their pursuit to do good works. These pressures and limitations, in some extent, catalyse bias in their works, especially for economists who value personal and career success as their utmost priority.
One privilege for academic economists that the applied economists do not have is time flexibility. Since applied economists are working for bureaucratic leaders in government, managers in private firms, or clients in consulting business, these stakeholders give assignments that constraint the time the economists have to conduct necessary studies. In this case, robust studies which are submitted lately are valued nothing. Meanwhile, the validity and reliability of researches (with the results) which are submitted on/in time cannot be sufficiently guaranteed due to the scarcity of data or immature methodological development.
Besides that, biased study can be traced from institutional pressures an applied economist receives. Some organizations utilise economic assessment only to make their goals or decisions look “scientifically-proved”. In this context, the decisions made by the stakeholders are not inductively developed from rigorous and reliable studies. Rather, the decisions are determined first, and the economists are asked (if not ordered) to conduct research which will serve as the justification grounds for predetermined decisions, with clear consequences on rebelling. Conducting studies with an aim to bring up particular outcomes requires the economists to manipulate data/methodology in a certain way that the outcome can support the interest of the stakeholders.
One of the studies that investigate bias in the applied realm of economics is done by Krol (2014) by using forecasted real GDP data published by two public agencies–Congressional Budget Office (CBO) and Office of Management and Budget (OMB)–and a private agency, Blue Chip Consensus. The study found that estimates done by OMB are upwardly biased, compared to forecasts by CBO and Blue Chip Consensus which are downwardly biased (though more aligned with other mainstream forecasts). One of the possible explanations comes from organisational design of OMB which is part of the executive branch, hence it is expected that the OMB’s estimates serve the interest of the president. On the other hand, CBO is part of Congress, and hence it has to serve both partisan affiliations in an independent manner to ensure its credibility as an internal think-tank.
Bias in academic realm of economics
In comparison with the applied realm of economics, biases by economists who work as academia are generally internal to the individual itself. The source of biases tends to come from an individual’s inherent characteristics; for instance, an economist’s moral belief, ideology, and political orientation. Randazzo & Haidt (2015) found that the choice of methodological approach is more influenced by an economist’s own moral conviction, particularly in research topics in which little consensus has been established. For instance, the tendency to utilise case studies rather than time series econometric approach is evident in the academic works of researchers who take Equality and Care as their preeminent values; a signal of consideration on left-leaning normative view. In the macroeconomic field, Saint-Paul (2018) analysed the tendency by macroeconomists to build economic models that favour the ideologically preferred policy, while keeping the consistency between the model and the data. He proved that model specification with steeper Phillips curve and lower magnitude of Keynesian multiplier is more favoured by conservative macroeconomists.
The mechanism of bias existence in economics research done by academics can be explained in the context of the relationship between an economist’s positive and normative beliefs. Caplan and Miller (2010) stated that bias may come from positive-to-normative mechanism or normative-to-positive mechanism. They found that there are the same underlying factors that determine the positive belief and normative belief of economists when they conduct research. In other words, it confirms the proposition that the positive belief and normative belief of an economist are not completely orthogonal. On the mechanism of connection between the two, they found that the positive-to-normative mechanism is explained by education background; higher education level means an economist is more equipped with information-sorting and critical thinking skills that prevent them from making biased beliefs. Meanwhile, normative-to-positive mechanism is explained by political affiliation; economists who actively affiliate themselves with a political belief/party are more likely to construct their concept about how the economy works (positive belief) by referring to their normative belief.
Javdani & Chang (2019) proposed some sources of bias which are unique to the economics realm. In their study, they argued that economics training received by the current economists and the incentive structure for economics research are two important catalysts of bias. Increasing tendency in many economics study programs to emphasise technical skills training with moderate-to-superficial depth of understanding in historical, theoretical, and philosophical aspects of economics makes the economics students more susceptible to ideological bias.
However, Javdani & Chang (2019) recognized that biases in economics research are not always ideological. The profession also suffers from the social structure of economics academia that emphasize the extensive publication of quantitatively-oriented works at reputable journals, as well as the elitism in economics consists of a small number of economists whose arguments have a significantly “authoritative” influence in the development of the body of knowledge. The exclusivity of social structure of economists–with its incentive system tied to publication quantity–results in authority bias, which gives perverse incentive for economists aspiring career success to make their findings become in line with mainstream economics understanding.
One kind of bias that is related to the incentive system described previously is publication bias. This kind of bias exists when the probability of a study being published depends on the statistical results the study presents. Christensen & Miguel (2018) found that studies with high statistical significance were more likely to be published at journals compared to less significant findings (as depicted in figure 2). When the journal editors tend to prioritize statistically significant studies over studies that fail to reject the null hypothesis, the economists have perverse incentive to conduct their research for the sake of finding statistically significant results rather than genuinely develop the body of knowledge according to reality.
Figure 2. Distribution of writings and publications of studies according to the statistical significance
Some remarks (and a caution)
From the exposition above, some remarks can be concluded that may arrive at some solutions. First, there is a need for serious academic discourses about how economists pursue ethical questions upon their works and their relationships with stakeholders. An honest discourse will be a good starting point in equipping the economists (especially those who work in the applied realm) to navigate the ethical problems they may encounter. The discourse can also provide foundation for economists to resist the institutional pressures and defend/explain their ethical decisions when the circumstances make it necessary for them to make public testimonies before the court/public inquiry. This will empower honest economists in (bravely) conducting credible research with unbiased/minimally biased data and inferences. Second, it is important to enforce economists to explicitly disclose the assumptions used in their research, as well as other aspects that may contribute to biases, in order to foster the profession’s accountability to the public. Christensen & Miguel (2018) propose the disclosure of some aspects, including research design, data presentation, funding sources, and potential conflict of interest, whereas Ioannidis et al (2017) emphasise the importance of requiring the researcher(s) to calculate statistical power of their own study before publishing it in the journal. Third, economics education should be arranged in such a way that enables the economics students to gain in depth understanding and discernment about historical, political, social, and psychological context of economic processes across the levels of society. The understanding of many noneconomic aspects can guard them from making biased judgement upon economic processes they study.
After all, it may be in the interest of the users of economic analysis that all economists recognise and take caution on the inherent bias in economics itself, particularly about how it conceptualises human as an object of scientific inquiry. In the study of economics as a social science, as Loury (2005) argued, economics places human “as an object of scientific inquiry”, and thus “the human subject must ultimately be reduced to a mechanism.” It implies that economists and policy makers cannot establish theoretical arguments that can powerfully provide consistent prediction, as well as incentive mechanisms which can deliver precise impacts. Since human does not “live by bread alone,” it is wise for economists working on research realm to acknowledge the biasedness of the relatively unbiased academic works, as well as to refine their understanding on the nature of human being by consistently facilitate discourses in that topic, although it is impossible to reach perfection on this matter. In the applied/policy realm, the understanding of the inherent bias in economics gives warning to the policy maker on taking a deterministic view about the policy they intend to develop and the behavioural outcomes they want to achieve. Consideration should be given on cross-discipline input to minimise the risk of reductionist approach to public intervention.
Due to limitation of discussion in this article, further inquiries are needed to understand some issues, including, but not limited to, the politics of economics publication, the indirect impact of economist’s bias on the economy, and the mechanisms in which cross-discipline contribution to economics education can help economist-in-training to minimise biased methodology choice and inference.
References
Caplan, B., and Miller, S. C. (2010). “Positive versus normative economics: what’s the connection? Evidence from the Survey of Americans and Economists on the Economy and the General Social Survey.” Springer Science+Business Media, LLC. Available at: https://doi.org/10.1007/s11127-010-9700-z (Accessed: 30 May 2020)
Christensen, G., and Miguel, E. (2018). “Transparency, Reproducibility, and the Credibility of Economics Research.” Journal of Economic Literature Vol. 56, no. 3: 920–980. Available at: https://doi.org/10.1257/jel.20171350 (Accessed: 30 May 2020)
DeMartino, G. F. (2011). The Economist’s Oath: On the Need for and Content of Professional Economic Ethics. New York: Oxford University Press.
Ioannidis, J. P. A., Stanley, T. D., and Doucouliagos, H. (2017). “The Power of Bias in Economic Research.” The Economic Journal 127 (October): F236-F265. https://doi.org/10.1111/ecoj.12461 (Accessed: 8 June 2020).
Javdani, M., and Chang, H. (2019). “Who Said or What Said? Estimating Ideological Bias in Views Among Economists.” Bonn: Institute of Labor Economics.
Krol, Robert. (2014). “Forecast Bias of Government Agencies.” Cato Journal 34, no. 1(Winter): 99–112.
Loury, G. C. (2005). “Hope in the Unseen: On Being a Christian and an Economist.” Address to Powell Center for Economic Literacy, Richmond, June. Available at: https://www.brown.edu/Departments/Economics/Faculty/Glenn_Loury/louryhomepage/ (Accessed: 10 June 2020)
Randazzo, A., and Haidt, J. (2015). “Are Economists Influenced By Their Moral Worldviews? Evidence From The Moral Foundations Of Economists Questionnaire.” Available at: http://ssrn.com/abstract=2700889 (Accessed: 8 June 2020).
Saint-Paul, Gilles. (2018). “The Possibility of Ideological Bias in Structural Macroeconomic Models.” American Economic Journal: Macroeconomics Vol. 10, no. 1: 216–241. Available at: https://doi.org/10.1257/mac.20140154 (Accessed: 1 June 2020).
Further reading:
DeMartino, G. F. (2011). The Economist’s Oath: On the Need for and Content of Professional Economic Ethics. New York: Oxford University Press.
