I was having the same thought.
My compass is less than a foot away and it works OK. It's not accurate but OK.
So who needs a compass if your GPS is working? And if it's not working, unplug it and you're back to "plan b" (actually in my case it's plan "d" or "f" since I have alternate GPS devices).
BTW I like having a swivel mount, as it lets me view from anywhere in the cockpit or companionway. I tend to run the AP a lot and make small corrections as needed. Under sail I usually prefer to run the AP set to track the apparent wind to a set angle, unless constrained to a narrow channel or exact course.