If the clearance gets too large, it will start to get noisy and the valves won't open as far as intended, limiting flow. I think the noise would become an issue before reduced flow.
Of more concern is small clearances. If the clearance is too tight, the valves will heat up (particularly exhaust), get longer, not seat properly, and burn the valve and seat.