When it comes to lowering your rates you will need to weight the benefits of having a lower rate vs. paying points up front. You may end up paying a lot more depending on your choice and how long you plan on keeping your mortgage.
Ask for more than just rates. Bad loan officers will tell you anything to keep you on the phone -- then change the details to suit them later. Instead, make them get real with you! Ask how long they've been in the industry. Probe them about their experience in the industry. Also, ask what their opinion is on the current market and where it's going.
Before you refinance a loan, make sure that you carefully analyze any fees that your lender is including on the loan. The Department of Housing and Urban Development can provide you with a list of standard fees. Use it to make sure that your lender isn't tacking on anything extravagant. And by all means, compare their fees with other lenders in the market.
You have a legal right to a good faith estimate. Get a copy of this document and go over it with a fine tooth comb, it will reveal where there is a real problem.