I've not ever had any long-term success stopping an oil leak from the outside of any device. That "rubber" that was pushed out is RTV silicone (there is no gasket), so there is now a tiny gap where that RTV had previously been applied. Removing and re-sealing the crankcase isn't a terrible job, once the engine is out of the chassis. Doing that will not change any gear lash or require any other adjustments. Just remove, clean the sealing surfaces aggressively and re-seal.
How is your compression? If the real issue is excessive blow-by due to worn rings, a catch can is just going to be lipstick on a pig. That might be all you need if you are taking the pig to market soon (trading your 14X in on something newer), but if you intend to keep it, it might be better to understand why you now have oil in the air box.