Trust but verify: using ESG data to build environmentally friendly portfolios

With so much environmental, social, and governance data being created, sorting through the noise is no easy feat. We explore the data challenges and the solutions to create truly climate-friendly fixed-income portfolios.

During the Second World War, Allied military officers had a question for the Statistical Research Group (SRG), a team of Columbia University statisticians and mathematicians tasked with solving military problems. The officers brought data showing the distribution of bullet holes in Allied bomber planes that came back to base. Curiously, the bullet holes weren’t distributed evenly across the airplane; there were nearly twice as many hits around the fuselage and the wings than the engines. The officers wanted to know how much more armor they should place around these seemingly vulnerable spots to better protect them.1

The SRG’s answer? None.  

Scrutinizing the data, these bright minds realized that the reason the planes weren’t coming home with more bullet holes around the engines wasn’t because the planes weren’t taking bullets there—it was because the planes that took hits to the engines weren’t coming home at all. The bombers that took bullets to the fuselage and wings could make it back to base to be studied; the ones with damage to the engine, however, weren’t so lucky.  

Based on this analysis, the SRG suggested adding armor around the engines. Those recommendations were put into effect throughout the rest of the war.

Data, it seems, is only as good as the way it’s analyzed. Without proper interpretation, data becomes useless at best, and—as the above case study suggests—dangerous at worst. This philosophy is true across domains, including portfolio management. When it comes to building climate-friendly portfolios, proper data analysis is critical, since environmental, social, and governance (ESG) data is fairly new to the world of finance and standardization of that data remains essentially nonexistent. So how can managers build truly “green” fixed-income portfolios?


Too much data, not enough information

Data is simply a collection of facts and figures, raw and generally unorganized. On its own, it’s meaningless and can’t be used for decision-making. But once analyzed, interpreted, scrutinized, and given context, data becomes information, and information is what is used to make decisions—anything from a military analyst figuring out how to armor an airplane to a portfolio manager deciding what assets to buy.

The most basic challenge with ESG data isn't the lack of it, but rather the proliferation of it. The rising importance of sustainability in investors’ minds has led to both issuers and ratings agencies providing more ESG data of late. But this proliferation of data has led to a situation of investors having too much data but not enough information. Why?

From an issuer point of view, without any firm standards on disclosing environmental practices,2 corporations are free to choose which (if any) environmental metrics they disclose. And even if many companies do disclose environmental data such as scope 1 and 2 emissions (which is done by about 71% of the S&P 5003), companies interpret the standards differently, resulting in disclosures that aren't perfectly standardized. 
Moreover, measuring emissions becomes more complicated when it comes to scope 3 emissions as well as other environmental aspects such as water and biodiversity. This means there’s very little comparability between environmental data points in company filings, limiting the usefulness of the data and complicating the portfolio construction process.

And while ratings agencies and financial data providers such as Standard & Poor’s, MSCI, and Moody’s provide plenty of ESG data points (for example, greenhouse gas (GHG) emissions per dollar of revenue, amount of waste generated, etc.) and ratings for thousands of public companies, these metrics and ratings can vary wildly between providers. Differences between the metrics that the ratings agencies actually consider, how those metrics are measured, and the weight applied to each metric result in major differences between issuers’ ESG ratings. The end result is that correlations between various agencies’ ESG ratings are weak, averaging about 0.54 overall.4 To put that in context, corporations’ credit ratings from ratings agencies are correlated to the tune of about 0.99.4

Ratings agencies don't agree much on ESG matters

Correlations between ESG ratings from six ratings agencies

Table showing the correlation coefficients between 6 ratings agencies on overall ESG and individual environmental, social and governance criteria. It shows an avergae correlation of 0.54.

Source: "Aggregate Confusion: The Divergence of ESG Ratings," Berg, Florian and Kölbel, Julian and Rigobon, Roberto, August 15, 2019. Each letter represents a particular ESG ratings provider, anonymized.

In other words, it’s not rare to see a given company rank highly on an ESG basis from one ratings provider and poorly from another, so trusting a given agency’s ESG ratings without scrutinizing how the rating is derived effectively means aligning yourself with that provider’s definition of ESG—definitions that evolve over time, affecting different issuers in different ways. And while aligning to one (or more) data provider’s point of view may be enough for some managers, we don’t think it does justice to those investment professionals who truly want their portfolios to have a positive environmental impact. We don’t mean to say that corporate ESG ratings aren’t useful—they do provide a decent starting point when assessing a company’s sustainability—but we nevertheless believe they’re just a starting point and require heavy scrutiny on the part of investors and portfolio managers. 

Are ESG indexes the answer?

With so much confusion and lack of standardization in the industry, it’s easy to decide that building an active green portfolio isn’t worth the effort and instead turn to an index to outsource one’s ESG portfolio. Certainly, it’s not hard to find an ESG index or an investment vehicle that tracks it: MSCI alone boasts over 1,500 equity and fixed-income ESG indexes,5 and as of 2021, 534 sustainable open-ended and exchange-traded funds (of which 159 were passively managed) existed in the United States.6

But does investing in an ESG (or specifically a climate-focused) index really satisfy a true need to invest with a green conscience? We doubt it. ESG ratings are directly linked to materiality (i.e., how a particular ESG issue may affect the company's bottom line). In other words, companies with poor or declining emission profiles can actually be deemed strong environmental performers by the ratings agencies if climate change doesn’t present a material financial risk to that company.

It should therefore come as no surprise that “almost 90% of the stocks in the S&P 500 have wound up in ESG funds built with MSCI’s ratings.”7 Could that many companies truly be ESG leaders? We’re skeptical, which is why we think that building truly green portfolios requires not only the right data, but the right people to scrutinize, verify, and challenge it.

Trusting a given agency’s ESG ratings without scrutinizing how the rating is derived effectively means aligning yourself with that provider’s definition of ESG.

Building green portfolios

Clearly, ESG data in and of itself cannot be blindly relied on to build climate-friendly portfolios. What are the additional steps needed? We believe three major actions lie at the heart of the process of building financially performing climate-friendly strategies.

1 Define climate leaders, attaching quantifiable metrics and benchmarks

First, it’s critical to define what a climate-friendly company is. Without any universal definition of a climate leader, portfolio managers are forced to determine for themselves how they define it. This step is crucial as it aids in narrowing down the universe of potential investments, the first step in any portfolio construction process. Critically, these definitions are nonnegotiable: If a company doesn’t meet the definition, it’s excluded from the universe.

Defining a climate leader also requires measuring a company’s carbon footprint and its progress, so it’s also critical to have quantifiable and verifiable metrics and benchmarks for each of the portfolio holdings. These metrics can come directly from the company in question (e.g., corporate filings) or through a trusted framework such as the Science-Based Targets initiative (SBTi). The SBTi is one source that our team uses to help evaluate a company’s commitment and action plans to reduce its climate impact; covering over 2,200 companies globally, it’s been shown that companies that set SBTs are on a much better path to reduce GHG emissions than global emissions as a whole.

Companies with science-based targets are reducing emissions faster than the rest of the world

Annual percentage change in emissions, companies with SBTs vs. global emissions

Line graph of the annual change in emissions for emitters that set SBTs versus those that did not from 2015 to 2020. Emitters who set SBTs reduced their emissions by 12.1%, compared to 5.2% for those who did not.

Source: SBTi progress report, 2021.

How do these definitions and associated metrics come together? Ideally, each definition is kept honest by one or more metrics. For example, our team narrows down our universe using three criteria. 

Definition of climate leader Metric/benchmark
Companies with low relative GHG emissions 50% or less than the portfolio’s benchmark
Companies that are on a committed path to reducing GHG emissions Have set a science-based emission reduction target
Companies providing climate solutions to positively contribute to the world’s transition At least 20% of revenues from green solutions and clean tech

In addition to corporate bonds, climate-themed bonds, including green bonds, sustainable bonds, and sustainability-linked bonds, can also be a building block of any green portfolio, but even these seemingly no-brainer instruments require scrutiny. Some key questions to ask before investing in these bonds would be:

  • Are the proceeds of the bond issuance dedicated to climate-related activities?
  • Is the bond’s key performance indicator target linked to reducing GHG emissions or extracting GHG from the atmosphere?
  • Are the reduction targets rigorous enough?

Together, these definitions and metrics provide a framework that’s verifiable and quantifiable, helping us target the companies we’d be proud to own while also making sure those companies are delivering on commitments and on our expectations.

Fixed-income managers wouldn’t blindly trust ratings agencies’ analyses of a company’s creditworthiness, solvency, or liquidity. So why should it be any different for a company’s ESG or environmental rating? 

2 Analyze, interpret, and verify the data

As mentioned, we believe that ESG ratings provided by agencies are a useful starting point in the determination of a company’s environmental suitability, but using them without proper scrutiny isn’t prudent stewardship. After all, fixed-income managers wouldn’t blindly trust ratings agencies’ analyses of a company’s creditworthiness, solvency, or liquidity—analysts dig deeper to come up with their own conclusions that may or may not be aligned with the rating. So why should it be any different for a company’s ESG or environmental rating?

ESG analysis also plays a role in our determination of an investment’s risk/return profile. We believe that risks to a company’s ESG profile will inevitably become financial risks, particularly in an environment where capital allocators demand more transparency and better ESG practices from the companies they invest in. Similarly, companies that are improving environmental practices may not necessarily be rewarded with appropriate ESG ratings because they’re not ticking the “right” boxes provided by the ratings companies.

Yet we can find no correlation between credit ratings and ESG ratings. A changing environmental data point—for example, GHG intensity—might not change the agency’s credit rating of that company, but it could certainly change ours. By digging into a given company’s ESG metrics more deeply, comparing the data against industry competitors, and interpreting it within an appropriate framework, credit teams can take advantage of the divergence between credit and ESG ratings—uncovering opportunities created by today’s business practices that might only be rewarded or punished by the market in the future.

How far into the future? Investing with a sustainable mindset isn’t for day traders; environmental investment theses often take years to materialize. But one of the major drawbacks of individual data points or ratings is that they represent a static and fleeting point in time. Ignoring, or at least underemphasizing, how a company’s climate performance might change in the future, particularly years down the road. To build portfolios that are both climate-friendly and financially performing in the long term, an additional human element to the data analysis is necessary.

3 Engage with issuers

The reality is that data—ESG-related or otherwise—delivered through corporate reports is inherently backward-looking, and therefore, so too is any analysis based off that data. To get a true picture of where a company may be heading from an environmental perspective, it’s critical to assess its sustainable strategic direction and engage with management teams, giving a crucial human and forward-looking element to the data analysis process. By understanding—and sometimes challenging—companies’ environmental plans and commitment to those plans, portfolio managers can get a better understanding of the risks the company might face down the road. This will also help screen out potential greenwashing and thereby do justice to the mission of the portfolio objective. Specifically for climate change, we believe that engaging companies on their physical risks and transition risks helps us understand how their plans to manage climate risks and ultimately drive real-world impact.

By interacting directly with corporate decision-makers, managers gain the full gamut of information: ESG- and non-ESG related, backward- and forward-looking, quantitative and qualitative alike. With a mosaic-theory approach, properly analyzing as many types of data as possible helps turn that data into information, and having an information advantage can ultimately be a source of returns.  

When it comes to ESG data, trust—but verify

From medicine to military, financial to communications, data is omnipresent and pervades our everyday lives. When it comes to building a truly climate-friendly portfolio, data can be an incredibly powerful tool in the arsenal of portfolio managers, helping give them a critical information advantage. But it can also be dangerous: When data is misused or taken at face value, it can do more harm than good. The military’s Statistical Research Group clearly saw the importance of scrutinizing and analyzing data that was presented to it. Thankfully in the group’s case, its analysis had a positive outcome at one of the most critical times in the 20th century.

In the case of portfolio managers building climate-friendly strategies, data misuse can lead to portfolios that don’t deliver on sustainability promises for investors. Because of the danger of data misuse, we choose to take a page out of the SRG’s book: verify the data, share the learnings with your stakeholders, and drive a societal benefit. 



1 Armor slows down the plane and reduces maneuverability, making it critical to use it sparingly and put it in the right places. 2 On this matter, the European Union is more advanced than the United States, although changes may be coming in the latter thanks to the SEC’s proposal to standardize climate-related disclosures. 3 4 “Aggregate Confusion: The Divergence of ESG Ratings,” Berg, Florian and Kölbel, Julian and Rigobon, Roberto, August 15, 2019. Available at SSRN: 5 6 “Sustainable Funds U.S. Landscape Report 2021: Another year of broken records,” Morningstar, January 2022. 7 “The ESG Mirage,” Bloomberg Businessweek, December 10, 2021. 

A widespread health crisis such as a global pandemic could cause substantial market volatility, exchange-trading suspensions and closures, and affect portfolio performance. For example, the novel coronavirus disease (COVID-19) has resulted in significant disruptions to global business activity. The impact of a health crisis and other epidemics and pandemics that may arise in the future, could affect the global economy in ways that cannot necessarily be foreseen at the present time. A health crisis may exacerbate other pre-existing political, social and economic risks. Any such impact could adversely affect the portfolio’s performance, resulting in losses to your investment.

Investing involves risks, including the potential loss of principal. Financial markets are volatile and can fluctuate significantly in response to company, industry, political, regulatory, market, or economic developments.  These risks are magnified for investments made in emerging markets. Currency risk is the risk that fluctuations in exchange rates may adversely affect the value of a portfolio’s investments.

The information provided does not take into account the suitability, investment objectives, financial situation, or particular needs of any specific person. You should consider the suitability of any type of investment for your circumstances and, if necessary, seek professional advice.

This material is intended for the exclusive use of recipients in jurisdictions who are allowed to receive the material under their applicable law. The opinions expressed are those of the author(s) and are subject to change without notice. Our investment teams may hold different views and make different investment decisions. These opinions may not necessarily reflect the views of Manulife Investment Management or its affiliates. The information and/or analysis contained in this material has been compiled or arrived at from sources believed to be reliable, but Manulife Investment Management does not make any representation as to their accuracy, correctness, usefulness, or completeness and does not accept liability for any loss arising from the use of the information and/or analysis contained. The information in this material may contain projections or other forward-looking statements regarding future events, targets, management discipline, or other expectations, and is only current as of the date indicated. The information in this document, including statements concerning financial market trends, are based on current market conditions, which will fluctuate and may be superseded by subsequent market events or for other reasons. Manulife Investment Management disclaims any responsibility to update such information.

Neither Manulife Investment Management or its affiliates, nor any of their directors, officers or employees shall assume any liability or responsibility for any direct or indirect loss or damage or any other consequence of any person acting or not acting in reliance on the information contained here. All overviews and commentary are intended to be general in nature and for current interest. While helpful, these overviews are no substitute for professional tax, investment or legal advice. Clients should seek professional advice for their particular situation. Neither Manulife, Manulife Investment Management, nor any of their affiliates or representatives is providing tax, investment or legal advice.  This material was prepared solely for informational purposes, does not constitute a recommendation, professional advice, an offer or an invitation by or on behalf of Manulife Investment Management to any person to buy or sell any security or adopt any investment strategy, and is no indication of trading intent in any fund or account managed by Manulife Investment Management. No investment strategy or risk management technique can guarantee returns or eliminate risk in any market environment. Diversification or asset allocation does not guarantee a profit or protect against the risk of loss in any market. Unless otherwise specified, all data is sourced from Manulife Investment Management. Past performance does not guarantee future results.

Manulife Investment Management

Manulife Investment Management is the global wealth and asset management segment of Manulife Financial Corporation. We draw on more than a century of financial stewardship to partner with clients across our institutional, retail, and retirement businesses globally. Our specialist approach to money management includes the highly differentiated strategies of our fixed-income, specialized equity, multi-asset solutions, and private markets teams—along with access to specialized, unaffiliated asset managers from around the world through our multimanager model.

This material has not been reviewed by, is not registered with any securities or other regulatory authority, and may, where appropriate, be distributed by the following Manulife entities in their respective jurisdictions. Additional information about Manulife Investment Management may be found at

Australia: Manulife Investment Management Timberland and Agriculture (Australasia) Pty Ltd, Manulife Investment Management (Hong Kong) Limited. Brazil: Hancock Asset Management Brasil Ltda. Canada: Manulife Investment Management Limited, Manulife Investment Management Distributors Inc., Manulife Investment Management (North America) Limited, Manulife Investment Management Private Markets (Canada) Corp. Mainland China: Manulife Overseas Investment Fund Management (Shanghai) Limited Company. European Economic Area Manulife Investment Management (Ireland) Ltd. which is authorised and regulated by the Central Bank of Ireland Hong Kong: Manulife Investment Management (Hong Kong) Limited. Indonesia: PT Manulife Aset Manajemen Indonesia. Japan: Manulife Investment Management (Japan) Limited. Malaysia: Manulife Investment Management (M) Berhad 200801033087 (834424-U) Philippines: Manulife Investment Management and Trust Corporation. Singapore: Manulife Investment Management (Singapore) Pte. Ltd. (Company Registration No. 200709952G) South Korea: Manulife Investment Management (Hong Kong) Limited. Switzerland: Manulife IM (Switzerland) LLC. Taiwan: Manulife Investment Management (Taiwan) Co. Ltd. United Kingdom: Manulife Investment Management (Europe) Ltd. which is authorised and regulated by the Financial Conduct Authority United States: John Hancock Investment Management LLC, Manulife Investment Management (US) LLC, Manulife Investment Management Private Markets (US) LLC and Manulife Investment Management Timberland and Agriculture Inc. Vietnam: Manulife Investment Fund Management (Vietnam) Company Limited.

Manulife, Manulife Investment Management, Stylized M Design, and Manulife Investment Management & Stylized M Design are trademarks of The Manufacturers Life Insurance Company and are used by it, and by its affiliates under license.


Caryn E. Rothman, CFA

Caryn E. Rothman, CFA, 

Senior Portfolio Manager, Head of Global Credit, Manulife Investment Management

Manulife Investment Management

Read bio
Frederick W. Isleib III, CFA

Frederick W. Isleib III, CFA, 

Director of ESG Research and Integration, ESG Team

Manulife Investment Management

Read bio