By Ryan Adler
Tom Mahoney, CFA, CAIA, is a Private Wealth Advisor with UBS. He is a Chartered Financial Analyst® (CFA®) charterholder and is co-chair of the Private Wealth Management Committee and member of the Board of Governors of the CFA Society of Los Angeles. He is a Chartered Alternative Investment Analyst SM (CAIA®) and is the past president and co-founder of CAIA Los Angeles.
Tom, tell us a little bit about yourself. I’m the son of a college professor and art teacher from Platteville, Wisconsin. I moved to Los Angeles for the sun and haven’t been disappointed. For the last 31years, I have advised very successful entrepreneurs about their investments.
For how long have you been a charterholder? Since 2001.
Where and when did you first hear about Superforecasting and Good Judgment Inc? The smartest portfolio manager I know sent it to me as a Christmas present. One week later, the smartest investment consultant I know called me to recommend it. I picked it up and couldn’t put it down. I saw why everyone seemed to be talking about it: a group of novice forecasters, from retirees to realtors, outperformed the entirety of the U.S. intelligence community, including analysts with access to classified intercepts? That was absolutely stunning and is still hard for me to believe. It was an Economist Book of the Year for 2015, and was the second-most recommended book by Bloomberg’s top 50 contributors in finance and economics for that year.
The first sentence of the book is: “We are all forecasters. … When we think about changing jobs, getting married, buying a home, making an investment, launching a product, or retiring, we decide based on how we expect the future will unfold. These expectations are forecasts.”
The last sentence of the book says that “if you’d like to participate in the Good Judgment Open, go to www.gjopen.com. I registered and started competing in the contest. That’s when I noticed that a lot of the questions were about finance and economics. Questions about Chinese GDP, electric car orders, and Fed rate raises. It occurred to me that, although it is interesting to forecast when ISIS will be removed from Mosul, people are fascinated by economics. I thought that this would be an ideal fit for CFA charterholders.
Let’s talk about the CFA Society Los Angeles Finance & Economics tournament, which wrapped up last February. How did you enjoy it? The user interface is super-fast and intuitive. The scoring algorithm is brilliant – you get a Brier score, the lower, the better, as with golf. Users can disclose as much or as little about their professional lives as they want – and can compete anonymously. I typed in my explanation for my prediction, and others would comment on my thinking, either supporting it or suggesting a different way of looking at the question. I suddenly had a few followers, and I followed the better forecasters to learn their thinking. When a CFA charterholder from London sent me a very useful link, I felt connected to the global finance and economics community for the first time since I earned my charter. We had 1,899 participants in the challenge, making it the fourth most popular of 11 challenges last year on different themes – our challenge was more popular than those sponsored by The Economist, Wharton, and CNN–with almost no publicity. Ten percent were charterholders, but 20% of the top performers were charterholders, indicating a natural ability at this. CFA charterholders outperformed the group by 20% in forecast accuracy. We were delighted because CFA charterholders seemed naturally drawn to and good at the challenge.
Which was the most memorable forecasting question for you? “What percentage of U.S. equity funds will be outperformed by the S&P Composite 1500 index for the one-year period ending 30 June 2017?” What it’s really asking is whether active management outperforms passive indexing, a central question for everyone who wants to succeed at investing. It calls attention to the brilliant SPIVA report, which now has 15 years of data. At first, I forecast that 90% of funds would underperform, because that’s the long-term average. But a fellow charterholder from overseas sent me a link to an article in the FT that shows that, for any given year, only about 55% of funds underperform, but that this rate of failure compounds over time to produce the 90% figure that we’ve all heard. So I brought my forecast way down. The actual number turned out to be 47.51%. I learned a critical lesson from the exercise: news stories about the percent of funds that outperformed the market last year are not very useful, because this short-term noise masks an inexorable trend towards a high rate of failure over the long run. From a single question, I gained insight that has probably saved me 25 to 50 hours of time over the past year that would otherwise be wasted on researching the wrong questions in my practice.
How have the things you’ve learned about forecasting helped you in your work as a Chartered Financial Analyst? I used it in three ways. First, I always checked the crowd forecast, which was correct even as vocal pundits were wrong, on questions like the likelihood that the estate tax would be repealed in 2017, the direction of the euro/dollar exchange rate, the level of the VIX by year end, the probability of a bear market, and the full-year return on the Barclays Aggregate Index. These insights informed my thinking and my advice to clients by providing clarity and a superior source of information than expert forecasts, which, as I learned in the book Superforecasting, are not usually very accurate. Second, each question was like a mini-crash course on that topic that accelerated my learning curve. It’s amazing how much more you learn when you are focused by competition with a near-term scorecard. Third, I took stock of my errors and my wins, which gave me a clearer picture of how good my thinking is and where I could improve. It served as an excellent reminder of the danger of overconfidence as well as confirmation of areas in which I outperformed decidedly. In short, I improved.
But it’s not just me. Torsten Slok, Chief International Economist with Deutsche Bank Securities, whose Economics team has been top-ranked by Institutional Investor in fixed income and equities for the past five years, was recently interviewed by Barry Ritholz in his Bloomberg Masters in Business series. He was asked for his book recommendations and he chose only one: Superforecasting. Goldman Sachs announced two years ago that they were going to offer odds or probabilities on its near-term Fed calls, “consistent with the best practices as identified in the 2015 book Superforecasting by Philip Tetlock and Dan Gardener.” The book has earned widespread praise from AQR, Harvard Business Review, Ian Bremmer, Peter Orszag, Michael Mauboussin, and dozens of other firms, officers, and publications in our industry.
What is some advice that you can give to colleagues diving into forecasting like this for the first time? Start with the probability based on history. If recessions only happen in one of six years, then your starting odds of a recession in 2019 should be 1 in 6. Then adjust that forecast up or down based on what you know or can find that others may not have figured out. Read “The Ten Commandments for Aspiring Superforecasters” at the back of the book Superforecasting. Spend more time on questions on which you can gain an edge by researching, such as Chinese debt levels, rather than on questions forecasting a commodity price like the price of gold or oil, which depend more on random events.
If your time is extremely limited, then pick just one question, spend fifteen minutes on some Saturday researching it, and make a forecast. The very act of competing makes you better. Use the forecasts in your work–always go the Graphs & Stats link to see what the crowd’s forecast is and how it has changed over time–that forecast is probably more accurate than most experts’, giving you an informed edge on thirty questions throughout the year. We’re even going to introduce, for the first time that I know of, crowd-forecasted capital market return assumptions–which I think will be the holy grail of superforecasting intel for finance. Finally, realize that the main benefit isn’t your score; it’s improving your knowledge and thus confidence by interacting with lots of high-IQ people on questions of interest in a game format. The community is polite, collaborative, and smart.
I recommend that all CFA charterholders participate, even if it’s only on one question. Studies show that forecasting improves with practice and feedback. The interaction with other forecasters is strong, including a lot of charterholders around the world. Someone may send you a link to get you to think about a question in a different way that will immediately improve your effectiveness at work. Good Judgment Inc, which runs the challenge, can provide each participating society with user metrics and scoring, allowing them to present awards to their top 20 local performers. I believe that if we don’t have massive participation in the upcoming challenge, which is open to anyone, it’s because we haven’t communicated the benefits well.
My hope is that this experiment leads to an annual endeavor for all 160,000 charterholders globally, an online collaboration that, for the first time ever, links our global community together on burning questions of the day.
Any concluding thoughts? Give it a try. David Brailsford turned around British cycling by seeking little 1% improvements everywhere he could: the ergonomics of the seat, the diet, the weight of the tires, and even the best pillow for sleep and the right hand gel to fight infection. Within three years, his teams were consistently winning the Tour de France for the first time ever and they have dominated Olympic cycling medals from 2008 on. Brailsford calls his theory the “Aggregation of Marginal Gains.” There is power in small wins and slow gains. Everything in your life involves forecasting, including what time you will be home for dinner. Academic evidence shows that as little as one hour of study increases forecast accuracy by 14%. But even if you improve by just 1%, it’s worth it, especially if you seek such improvements everywhere in your life and career. I think Superforecasting is a superb career development tool, and I’m thankful to the CFA Institute and the Western U.S. societies for investing in it to bring us another year of improvement and fun.
Ryan Adler is a Superforecaster and Director at Good Judgment Inc. Click here to reach the Finance Forecasting Challenge hosted by the CFA® Societies of the Western US.