You have a legal right to a good faith estimate. Get a copy of this document and go over it with a fine tooth comb, it will reveal where there is a real problem.
Make sure that your original mortgage does not have a pre-payment penalty or early payoff penalty of any kind. Sometimes people will get into their mortgage with the mortgage having a pre-payment penalty and they will not even know about it. Pre-payment penalties usually range from 6 months to 3 years with a penalty for an early payoff. The penalty is usually about the amount of 6 months worth of your mortgage loan interest, but this varies. You would have to be able to have some significant payment and interest savings on your refinance loan to justify refinancing a mortgage loan with a pre-payment penalty.
If your monthly payment on a fixed-rate loan includes escrow amounts for taxes and insurance, your payment each month could change over time due to changes in property taxes, insurance, or community association fees.
If your property value reduces and you refinance up to 80% of the reappraised value, your original mortgage amount may be higher than this amount. Thus, the new loan will not be sufficient enough to help you pay down the existing one.