Fannie Mae and Freddie Mac have been struggling to grow their portfolios this past summer, even as market observers have been predicting that the companies are primed to grow again as they put their regulatory problems behind them. According to data tracked by Inside The GSEs, Fannie’s portfolio dropped 2.4 percent in July. But Fannie is down 11.6 percent through the first seven months of the year, having watched a whopping $116 billion in…