One of the talking points among those who deny that Freddie and Fannie played a large role in the housing bubble is that the agencies lost market share in 2003 and 2004, and that is when house prices took off. Some actual data on house prices might offer perspective.
The agency that regulates Freddie and Fannie maintains an index of house prices that is available for download. Using the seasonally-adjusted quarterly data, here are some values:
2002, Q4: 165.47
2004, Q4: 194.47
2006, Q4: 219.96
2007, Q2 (the peak): 224.02
2010, Q1 (the low point): 191.35
2010, Q2 (the latest data point): 196.38
From the end of 2002 to the end of 2004 (the period when Freddie and Fannie were losing market share), prices went up 17.5 percent. Over the next two years, as Freddie and Fannie got back into the market, prices only went up 13.1 percent.
So, was 2003 and 2004 a period when the bubble really took off? At what point did the level of house prices reach unsustainable levels?
If house prices hit bottom earlier this year (admittedly, that is a big if), then the house price index at the end of 2004 was at reasonable levels. The real bubble took off from 2005 through the middle of 2007.
To put this another way, if you stopped lending in 2004, there is a good chance that your borrowers have some equity in their homes--on average, they have retained their down payment plus whatever principal has been paid down on their mortgages. Defaults from loans made before 2005 ought to be much lower than default rates on loans made afterward (and in fact the analysis of loan defaults strongly supports this).
In hindsight, Freddie and Fannie chose the worst possible time to jump back into the market. In hindsight, the rise in house prices in 2003 and 2004 was not a bubble--buying a house in January of 2003 would have given you a capital gain, not a capital loss.
Even setting aside the contentious issue of Freddie and Fannie, I think it helps to have this perspective on house prices. People who write as if the entire run-up in house prices from the late 1990's through 2007 was a bubble are being misleading, unless they want to argue that the worst is yet to come in the housing market. If the worst is over, then house prices did not get out of hand until some time after 2004.