With the kids. When you go overboard, it could be rough, could be calm, could be night, could be daylight. I guess a dinghy is just one more thing in the water that could help.
No matter what though, you're going to be swimming for something if you don't have a pfd on, and you'll likely be doing it more clothes than you want, etc. It could be the fastest object is that saves your life.