People who are concerned that forcing Fannie Mae and Freddie Mac to unload their portfolios would disrupt the mortgage market should find something else to worry about, according to a panel of experts who spoke at an American Enterprise Institute conference this week. The session, “Will Portfolio Limitations for Fannie Mae and Freddie Mac Affect Residential Finance?,” took place as chances for obtaining a GSE reform bill this year grew further remote due to a…