Credit cards are dangerous. As many consumers have already discovered, the debts can pile up faster than a person can make enough money to pay it. Add in high interest rates and APRs, and you might find yourself beholden to your creditors. This has caused many a consumer to cut up their credit cards and close their accounts. However, this move is not as smart as you might think.

When it comes to credit card management (and with respect to Kenny Rogers), you need to know when to hold them, know when to fold them, know when to walk away, and know when to run.

When to Fold Them

The best medicine for overspending is to get rid of the credit card that enables the compulsion. However, it takes more than just cutting up the credit cards to break their spell. If you are in this situation, you need to cancel them.

Canceling the account takes away a compulsive shopping binge or the impulse buy in the middle of the night, simply because the account no longer exists for you to use it. Reducing the number of credit card accounts you leave open is also a good idea for people who don’t currently have difficulty with spending.

When to Hold Them

Don’t cancel every credit card account you have all at once. Prioritize your closures. Start with cards that charge you fees. They should be the first to go. But hold on to the cards you have had the longest. This gives you access to new opportunities for being a loyal customer and it also looks better on your credit history, especially with a long history of paying on time.

Hold on to cards with great rewards programs. This is especially true if you have a card with fantastic travel benefits and you work in a job that requires you to travel.

When to Walk Away

Be diligent, thorough and deliberate when canceling a credit card. Although you can always call up the credit card company, this is risky. Their customer service reps are experts at convincing you not to cancel by dangling new offers and rewards in an attempt to keep you around. But if you’ve already determined that the card and account must go, take the traditional route by writing to the company directly. Also complete the process online. Don’t overlook the follow-up process, since credit card companies are notorious for failing to follow your instructions. Always send a confirmation letter by snail mail to make sure that the account is canceled and the subsequent information on the credit report is accurate.

When to Run

If you cancel a card when it still has a balance, there are often bad consequences. Run away from these cards as fast as you can. But first pay that balance off or transfer it to another card. This way you avoid high interest rates and other fees.