I am still in Mission Bay, so all my sailing is in the ocean. (too tall to go under the bridges). But indeed you are right, sailing in the big bay with good winds and a flat sea IS a blast, not to mention the sights and destinations (restaurants et. al.)
I "cruise" down to the big bay from time to time and do Gloretta, and even Chula Vista... in mini vacations. Gives me someplace easy to get to. Chula Vista is very quiet and bring a bike... there are some great paths in the area.