I believe with a keyed slot, the engineering is to handle the torque on the joint with the fit of the tapered shaft in the socket, correctly torqued via the castle nut, just like a prop. the key should only be there to guide it. But that theory didn't seem to work on my key or the prop I saw with a sheared key stock.
Your design would work even if things were not correctly torqued at assembly. I would bet the state of many rudders are unknown. They'll go forever in the right conditions, and fall apart rapidly in the wrong conditions.