The titans of the industry – the four banks with over $1 trillion in total assets – shrank their mortgage portfolios by a combined 0.5 percent over that period.