If you are making payments on a long term loan, say, 30 year mortgage for the past 10 to 20 years, then refinancing to another 30 year loan will not be a good option as it may increase your overall payment.
When lenders offer a "no-cost" loan, they may include a prepayment penalty to discourage you from refinancing within the first few years of the loan. Ask the lender offering a no-cost loan to explain all the fees and penalties before you agree to these terms.
Pull your credit report from the bureaus and review it for any negative items (late pays, collections etc) and inaccurate detail. Try to dispute negative items and remove them from the report. If required pay off any unpaid debt. Otherwise, you won't get a low rate and may not even qualify. Of course there are lenders in the subprime market who may offer you a bad credit refinance loan, but it's better to avoid them as they'll possible charge higher rates and fees.
It is feasible to go for a refinance when you have built up at least 10% equity in your home (For Fannie Mae owned mortgages, the value is 5%). It is also possible for you to choose the option if your equity is less than 5%, but you may have to pay a certain amount of cash in order to make up for the difference in equity.