That both you and their mom are comfortable and feel safe onboard. I'm sure you both do, but that has to be so. That being said, there is absolutely no better place for a family with babies and young kids, than on their own boat. It's like being home, only so much better. Moms get total privacy with their family, which is something my wife craved(still does). It's complete and familiar security for everyone.
It's different with babies, you'll likely do all the sailing, she'll likely do more, taking care of two kids. You can switch anytime.
But it all comes nicely together, under sail, at anchor, or at a beach. I can't begin to imagine how many beaches we shared as a family up and down the east coast, completely alone. It was just an incredible experience.