Banks and thrifts that have a heavy concentration of mortgage assets on their books rely to a great deal on their retained portfolios as a source of profit, rather than fee income from mortgage banking activities, according to a new Inside Mortgage Profitability analysis. On average, the most profitable banks and thrifts concentrating on the mortgage business booked $305.2 million in net interest income during the fourth quarter of last year, but just $41.6