If you can get your money back, then go for it. A thrift store may not refund your payment. There was somebody there who set the price. Either they believed it was real and didn't have a clue what it was or what it was worth or they knew what it was and priced it a bit high. Maybe they were the one who stamped the bogus info on the blade.
If the thrift store actually puts back into the community, like raising funds for charity or provide jobs for locals or the needy, and I mean really puts back in, not give 5% to an organization just to use their name and avoid taxes; then hold onto the sword. I wouldn't change it. You have an excellent example of a fake. The distinctive pommel, the info stamped on the blade 1 character at a time, even the scabbard with its distinctive throat that looks influenced by the British. These are all things you won't see on a genuine Civil War period cavalry saber.
You should get reference books, but it wouldn't hurt to look at some of the websites of companies that sell reproduction swords as reproductions. You won't see reproduction swords in reference books (mostly, I think there are at least one or two that do show them.) seeing reproductions will help keep you from thinking a fake is a rare variation because you haven't seen one before.
Years down the road you can pick up your fake sword and see how much you have learned and how far you have come. Then you'll wonder how you could ever have been fooled into believing it was real as you explain to someone else what makes it a fake.