This story is part of CNBC Make It's One-Minute Money Hacks series, which provides easy, straightforward tips and tricks to help you understand your finances and take control of your money.

There is no shortage of options when it comes to choosing a credit card that suits your financial needs. Most cards also offer a host of perks, from sign-up bonuses to cash back to discounts on travel and shopping.

The catch: Many of them also charge an annual fee. Whether it is worth it to pay an annual fee depends on your individual spending habits and how you use the card.

As part of sign-up packages, a lot of card issuers waive the annual fee for the first year, giving you a chance to try out higher cashback rates and more points-per-dollar.

But did you know you can often get the fee for the second year waived as well? It may take calling customer service more than once and always asking to speak to a representative, but many have had luck.

Before you call, have a short pitch justifying why you want to continue using this card, but feel you can't justify paying an annual fee for it. If you are a high spender and have a perfect or near-perfect payment history, leveraging your loyalty to the card could also help your case.

You can also ask for the retention department, which may help you negotiate a better deal. Some cardholders who weren't able to get their fee waived got offered statement credits or bonus points for spending a specified amount within the next few months.

If you are unable to get a retention offer and think that you will not be using enough of the card's perks to warrant paying a fee for it, take a look at what other cards from the same issuer charge lower fees or no annual fees.

Downgrading your card allows you to keep your line of credit open, which can help stabilize your credit score. Closing accounts can hurt your credit score, so it's best to avoid doing that unless you have to.

Check out: Meet the middle-aged millennial: Homeowner, debt-burdened and turning 40

More from this series:

• This simple formula will show you if you're on track to buy a home or retire early
• A typical family spends $4,600 a year on groceries—this free and easy hack could save you money
• Opening a Roth IRA in your 20s could set you up for success later in life