Bank and thrift MSR holdings peaked at $5.977 trillion back at the end of 2009.

The key factor is that some mortgage originators, the megabanks especially, are keeping conventional loans in portfolio that might otherwise be securitized by Fannie and Freddie.

According to Inside Mortgage Finance, among the top 15 jumbo originators eight are nonbanks.

All three agencies saw double-digit declines in new production, but Fannie Mae experienced the the biggest decline.

The top refi lender through the first nine months of 2014 was Wells Fargo with $38.56 billion, but Quicken wasn’t too far behind at $32.48 billion.

As one West Coast-based mortgage executive told IMFnews: “It can be a very tough space. You need lots of capital."

The sharp increase in industry holdings was fueled by a massive acquisition of credit card ABS by TD Bank...

At the same time, the market share of GSE business has been redistributed toward the smaller lenders.

The three biggest servicers in the group – Wells Fargo, JPMorgan Chase and Bank of America – all reported significant declines in servicing income.

Wells Fargo reported a 2.1 percent drop in its correspondent originations from the second quarter to the third, and its year-to-date volume was down 57.3 percent.