Let me put my conclusion up front — now I feel these factor models are an economist's answer to the big mystery “why some securities have consistently higher excess return than other securities.” I assume this pattern is clear when we look long term like decades. I feel in this context the key assumption is iid, so we are talking about steady-state — All the betas are assumed time-invariant at least during a 5Y observation window.
There are many steady-state factor models including the Fama/French model.
Q: why do we say one model is better than another (which is often the CAPM, the base model)?
1) I think a simple benchmark is the month-to-month variation. A good factor model would “explain” most of the month-to-month variations. We first pick a relatively long period like 5 years. We basically “confine” ourselves into some 5Y historical window like 1990 to 1995. (Over another 5Y window, the betas are likely different.)
We then pick some security to *explain*. It could be a portfolio or some index of an asset class.
We use historical data to calibrate the 4 beta (assuming 4 factors). These beta numbers are assumed steady-state during the 5Y. The time-varying (volatile) factor values combined with time-invariant constant betas would give a model estimate of the month-to-month returns. Does the estimate match the actual returns? If good match, then we say the model “explains” most of the month-to-month variation. This model would be very useful for hedging and risk management.
2) A second benchmark is less intuitive. Here, we check how accurate the 2 models are at “explaining” _steady_state_ average return.
Mark Hendricks' Econs HW2 used GDP, recession and corp profits as 3 factors (without the market factor) to explain some portfolios' returns. We basically use the 5Y average data (not month-to-month) combined with the steady-state betas to come up with an 5Y average return on a portfolio (a single number), and compare this number to the portfolio actual return. If the average return matches well, then we say …”good pricing capability”!
I feel this is an economist's tool, not a fund manager's tool. Each target portfolio is probably a broad asset class. The beta_GDP is different for each asset class.
Suppose GDP+recession+corpProfit prove to be a good “pricing model”, then we could use various economic data to forecast GDP etc, knowing that a confident forecast of this GDP “factor” would give us a confident forecast of the return in that asset class. This would help macro funds like GMO making asset allocation decisions.
In practice, to benchmark this “pricing quality”, we need a sample size. Typically we compare the 2 models' pricing errors across various asset classes and over various periods.
When people say that in a given model (like UIP) a certain risk (like uncertainty in FX rate movement) is not priced, it means this factor model doesn't include this factor. I guess you can say beta for this factor is hardcoded to 0.