With refinance activity slowing to “normal” levels, a handful of mortgage servicers were able to build their portfolios aggressively during the first quarter of 2004 – while other top industry players were stagnant. Overall, the top 30 servicers grew their combined operations by 2.7 percent in the first quarter of this year, according to a new Inside Mortgage Finance ranking and analysis. With the overall