If you know that you will be moving in 3 to 5 years, you might want to consider refinancing to a 3 or 5 year ARM (adjustable rate mortgage). These loans typically have a much lower rate that a traditional fixed rate loan such as a 30 year fixed, but they do have a fixed rate for the first 3 or 5 years of the loan. This will enable you to benefit from the lower rate, but you won't ever have to worry about the risk of a rate adjustment because you will be selling the home before the fixed-rate period ends.
Refinancing may not be that useful if you have already used up 90% or more of your home value in taking out a mortgage or any home equity loan. You won't be able to get the best rates available in the market as when you refinance a 90% LTV loan, you will probably require a loan of that value or higher. This will be quite closer to being a 100% financing option and hence mortgage refinance rates will be comparatively higher. Moreover, 100% loans are hardly available in times of mortgage market crisis.
Resist "no cost" refinancing. No cost doesn't mean free. On the contrary: The closing costs are usually bundled into the new mortgage, which means you pay interest on them. The fees associated with a 30-year mortgage could cost you more than double what they would have had you simply written a check for them at closing. Or, if the costs aren't bundled in, you'll be charged a slightly higher interest rate. Either way, the lender wins.
If your new mortgage rate seems too good to be true then it probably is. Check for hidden fees in your mortgage that will make up that suspicious difference.