I saw a Dr today for something similar. I've had two episodes of dizziness / nausea, once after swimming, once after Yoga. He suspects labyrinthitis but I am going to get an MRI, tilt test and see a specialist as I plan on a lot more offshore sailing in my life and I don't want this to be an issue. I think it also relates to crystals which metastasize in your ear canal and throw the sensors for a loop. I know one very well known offshore sailor has this, it's nice to see it hasn't slowed them down.

Labyrinthitis is an inflammation of the inner ear,[1] and a form of unilateral vestibular dysfunction. It derives its name from the labyrinths that house the vestibular system (which sense changes in head position). Labyrinthitis can cause balance disorders.
In addition to balance control problems, a labyrinthitis patient may encounter hearing loss and tinnitus. Labyrinthitis is usually caused by a virus, but it can also arise from bacterial infection, head injury, extreme stress, an allergy or as a reaction to a particular medication. Both bacterial and viral labyrinthitis can cause permanent hearing loss, although this is rare.
Labyrinthitis often follows an upper respiratory tract infection (URI).