Fix-and-flip loans are offering attractive returns to some non-agency players even as others are reducing their exposure to the market, citing concerns about slowing home-price appreciation.