Banks and thrifts built up their retained portfolios of mortgage securities last year, but the smaller credit union industry pared back its MBS holdings over the course of 2004. According to data from the National Credit Union Administration, federal credit unions held $28.24 billion of mortgage securities at the end of 2004 – down 2.2 percent from the end of the previous year. The 50 most active credit union MBS investors, however, did increase their…