Environmental, Social and Governance (ESG) performance is becoming a bigger decision factor for companies and investors. As a result, ESG rating providers are now key actors of the business landscape. However, there is a divergence between the ESG ratings attributed by rating providers. Using the data from five rating companies, this paper finds three sources to the overall disagreement: scope divergence, measurement divergence and weight divergence.

The authors identify measurement divergence as the main driver of the disagreement, accounting for 53% of the aggregate difference in ESG ratings. Measurement divergence comes from the measure of the same attributes using different indicators and the spectrum of sources used by rating providers.

The paper introduces the ‘rater-effect’ as another significant cause of the measurement divergence. It refers to the bias demonstrated by the rater during the rating process. Once a rating is applied to an indicator, a similar rating (good or bad) is more likely to be attributed to other indicators. Two explanations for the ‘rater-effect’ are the perception of the firm by the rater and the willingness of the company to answer specific questions.

The research establishes that scope divergence weighs 44% of the total aggregate difference. The scope is defined by the range of attributes considered being a part of ESG performance. Scope divergence is a result of the different interpretations of ESG performance by rating providers, as it is still an evolving concept. The authors managed to replicate the ratings using a lighter taxonomy, signalling a potential redundancy in the indicators used by the agencies. Some raters do not consider the impact firms have with their product and only report on the companies’ operations.

Finally, the authors find that weight divergence accounts for 3% of the total disagreement. The source for weight divergence is the difference in opinions on the relative importance of attributes constituting ESG performance.

Measurement and scope divergence being the two main components of the total aggregate divergence, a standardisation of ESG indicators and measurement procedures are required to solve most of the discrepancy in ESG ratings.

The divergence in ESG ratings can lead to confusion for companies trying to improve their ESG performance. The lack of standardisation means their efforts might not be universally recognised by a rating provider and affect the firm’s ESG rankings.

Until the rating providers standardise their methods, academics might see the results of their studies change depending on the data used. The authors provide a taxonomy for researchers to develop their own ratings.

Investors should be aware of the reasons behind the difference in ESG ratings. When exposed to divergent ratings during their research, they can use the framework provided by the paper to improve their decision making.

This study is a working paper by the MIT Sloan School of Management in association with the University of Zurich.

1. Measurement divergence and scope divergence are the main drivers behind the aggregate disagreement in ESG ratings. Weight divergence is practically insignificant.
2. There is a ‘rater effect’ amplifying measurement divergence, where a firm that performs well in one indicator is likely to be attributed a good performance in other indicators by the rater.
3. A standardisation of the taxonomy and measurement procedures is required to eliminate a major portion of the measurement and scope divergence. Rating providers currently operate with their own methods and sources which leads to a discrepancy in ESG ratings.
4. The divergence in ESG ratings can be source of confusion and frustration for companies trying to improve their ESG performance. Their efforts might not be recognised by all rating providers and their actual performance might not be reflected in the rankings.
5. This report provides a framework and a taxonomy for stakeholders to replicate the ratings. It can be used by investors and academics to conduct their own research to avoid skewed results and improve decision making.
6. ESG performance is still an evolving concept and ESG rating providers can use the diagnosis in this paper to improve their processes. Specifically, eliminating the ‘rater effect’ is a necessary step for the legitimacy of the organisations.