A Fee-Only Registered Investment Advisor
The Scientific Method
The scientific method can be characterized as a set of principles that include: questioning, hypothesizing, predicting, testing, and analyzing.
The scientific method is particularly useful when analyzing large data sets that blend useful information with useless information. For example, advances in DNA analysis and crime scene forensics in recent years have allowed us to find truth in places where it was previously hidden.
When it comes to investment management the scientific method has been applied to answer questions such as whether or not certain market segments or investment approaches provide a persistent and pervasive advantage over others.
It turns out some do.
For example, in the past it was suspected that stocks provide a statistically significant long-term performance advantage over bonds. In other words, it was suspected that stocks provide an advantage that is 'real' and not simply the product of random chance. To test this hypothesis predictions were made, tests were conducted, and the results were analyzed.
These tests confirmed that stocks have indeed exhibited a statistically significant performance advantage over bonds during most longer time periods. This discovery suggested that investing in stocks wasn't merely gambling and that investors were truly rewarded for the additional risks of stocks.
This discovery served as a platform for additional refinements in investment theory and it provides a key example of how science can be applied to gain a deeper understanding of important investment principles.
What's the t-Stat?
Perhaps the most important question in all of financial analysis is 'what's the t-stat?'
For example, a t-stat of 2.6 means there's a 99% chance that a data point (e.g. a risk premium - such as the historical difference in performance between stocks and bonds) is statistically meaningful. In other words, t-stats helps us determine if, for example, the average gap in performance between two asset classes is large enough and persistent enough to be considered 'real' as opposed to meaningless (i.e random) chance.
What we mean by 'real' is best illustrated with an example. Imagine a person flipping a coin ten times and getting seven heads (i.e. 70% heads). In this case, we can't say our coin-flipper is good at flipping heads because random chance easily explains this result. In other words, this is expected based on a normal probability curve. However, if our coin-flipper repeated this same feat 15 times in a row, now that's entirely different. This would not be expected based on normal probability.
In this case, our data would show (i.e. the t-stat would be close enough to 2.6) that random chance doesn't likely explain our coin-flipper's persistent out-performance (remember, random chance says on average heads should come up only 50% of the time not 70%). At some point (once the t-stat rises above, say, 2.0 or 3.0), our coin-flipper's ability to flip heads has to be recognized as not likely the result of random chance (i.e. perhaps he really is good at flipping heads after all).
In short, t-stats help us make determinations regarding whether random chance likely explains certain results and when it doesn't. This is important because it's vital to avoid unscientific, narrative or story-based investment discussions since they add nothing of value to the investment process. In fact, they represent the exact noise we're trying to filter out on our way to an actionable investment strategy.
Table 1 below shows the high level of statistical significance of the various dimensions of higher expected stock market returns.
The t-stats for the various equity risk premiums are robust and statistically meaningful.
In other words, they can reliably be considered real.
We believe in extracting as much of this kind of statistically significant information out of the past as possible when formulating investment strategy. To approach investing any other way by definition means we are formulating investment opinions based on information that is not robustly supported by the data.
We believe anecdotal or narrative-based evidence can never be as powerful as scientific evidence. That's why we don't believe in stories. We believe in statistically significant data.
In short, t-stats are one of our greatest weapons in the ongoing battle to separate meaningful information from useless noise in order to promote optimal financial health.
Confidence Intervals
Confidence intervals provide a range within which a 'real' gap in average performance likely resides. For example, from 1927 through 2010 stocks outperformed bonds by an arithmetic average of 8.05% annually with a standard deviation (a measure of variability around the average) of 21%.
This high variability of the average gap in performance in stocks versus bonds suggests that the average advantage for stocks (8.05%) might not actually be large enough to be statistically meaningful. In other words, with all that variability in returns, maybe 8.05% is just too small a gap to be considered different from zero from a statistical standpoint.
Said another way, despite what our sample data (i.e. the gap) seems to suggest, maybe stocks and bonds actually offer the same rate of return over time and the performance gap isn't 'real' (i.e. statistically significant). In simplest terms, to be statistically significant, a performance gap can either be very wide over relatively few periods or very small over many periods or moderately wide over a moderate period.
To find out if the 8.05% performance gap is big enough to be statistically different from 0% we run t-tests. Now, if the t-test is high enough to tell us, a 'true' performance gap likely does exist (a yes or no question), then the confidence interval test tells us, "it's 95% likely that the true performance gap is somewhere between 3.51% and 12.26% based on the historical data," (see Table 5 below).
In other words, a high t-stat (above 2.00) tells us the performance gap is likely real (i.e. different from zero); however, it does not tell us what the "true" gap is - the best we can do is to establish a confidence interval within which the actual gap likely resides based on the data.
Failure to understand this fundamental aspect of statistics can lead to investment decisions based on narratives or stories that may appear intellectually or emotionally satisfying but in the end can best be described as speculation.
Armed with this kind of scientific information we can make better informed investment decisions and put the odds in our favor (literally) that our investment actions are good for our clients' financial health.