Yes! I did this exact thing last year. We maybe fly once every 5 years if that. I wanted to go out west to
Disneyland this past June. I opened up a
Chase Sapphire preferred. Got a nice sign up bonus and put everything on that card for the next 9 months. Transferred my rewards to southwest and it paid for 4 of us to fly for FREE. Just had to pay the $12 pp tax or whatever it was. It saved us almost $1400 on tickets. Because that card had a $95 fee which was waived the first year, I canceled right before the year was up.
This is the card we use now. I like it for its flexibility. I never thought I'd ever give up my SWA card but then I realized that we flew free so often that we rarely earned double points through any cc spending. (With the SWA card, only purchases made to SWA earn double points.)
With the Chase CSP card, all travel and dining earn double points. This isn't just flights, hotels, fancy restaurants, etc. It can be anything from pizza delivery to taxi/Uber or tickets for the train I take to work, even airport parking charges count as travel.
I like how flexible the points are. Sometimes you find cheap flights that you wouldn't mind just paying cash for so it's nice that I can then use points to book hotel or rental car in order to make the trip happen. It used to be possible to book Disney hotels with points but last time I tried, none were showing up, even during very off peak time. Also if I don't want to book travel through Chase (which is run by Expedia I think) I can transfer points to various airlines or hotel rewards programs.
Super flexible and we redeem way more than the $95/yr fee.
OP, I would do some research to see which airline flies non-stop (if possible) from your nearest airport and then compare the airline's bonus offers with non-airline travel reward cards such as CSP. See if any of the spending requirements are doable and which offers the best deal for the annual fee.
We are also in NJ but closer to PHL so our usual flight options are American, Southwest and Frontier. EWR is a hub for United and Frontier flies out of TTN also. Spirit flies out of AC and PHL but I've heard to many horror stories about their customer service.