Personally, I don't think you made a big mistake by putting olive oil on the blade. However, your blade might start to stink if and when the olive oil goes rancid on you, so don't trap it down in the scabbard! If you are seeing rust blooms coming out, I'll speculate you are seeing your own fingerprints underneath the olive oil. DON'T take any strong abrasives to the blade. DO apply some oil that won't go rancid on you: Mineral oil, gun oil, something of that nature--and apply it sparingly. Don't give the thing a bath. To repeat: No sandpaper. No coarse steel wool. If it were mine, I'd wash off the olive oil with mineral spirits and then apply a very light gun oil. Then, I would check it every couple of weeks and repeat the process as necessary.
After that, heed the aforementioned advice about keeping your skin oils off the blade.
You might also read up on Frontier Big 45 pads, which remove rust on old gun surfaces without removing blueing. They are probably gentle enough for your blade, but DO NOT plunge in. Experiment with them first. They are available online, and are big with vintage shotgun collectors. I have used them on various gun surfaces and would have no problem using them on a blade......but I have experimented. You do the same, please.