Regulatory challenges and rising origination costs are prompting community banks and credit unions to partner with nonbanks for mortgage lending.