Actually, the Fed was quite worried about Freddie Mac and Fannie Mae.
In general, I think that Kaufman offers contradictory advice. On the one hand, he thinks that banks should have been closely regulated. On the other hand, he thinks that the growth of non-bank financial institutions should have been thwarted. You cannot have it both ways. Either you hamstring the banks, in which case non-banks will expand at their expense.* Or you allow banks to remain relevant by giving them room to compete against non-banks. In practice, the Fed compromised between those two approaches.
On the whole, I would rather see banks regulated more closely, in order to protect depositors. But one has to recognize that such a policy implies that the non-bank financial sector will be free to take advantage of regulatory restrictions on banks. If this approach is adopted, one has to be willing to let these uninsured competitors fail, rather than treat them as too big to fail. Since we have not demonstrated an ability to allow large financial institutions to fail, I cannot make any claims that my preferred approach is feasible.
*For example, when banks are not allowed to "abuse" credit card customers, the result will be fewer credit cards issued. This will allow alternative non-bank intermediaries, including loan sharks, to step into the vacuum, probably to the detriment of the alleged victims of credit card "abuse."