Aside from issues of balance of the rig, distribution of weight, trim differences (e.g. lead postions, angles and sheet tension), current, etc. --- another possible issue is the actual difference with apparent wind angle. If you windex guide lines or electronic instuments is off by as little as a few degrees, you will notice speed over water differences.

There are some differences in speed from differences in bottom growth on the hull. If the boat is docked (or moored) with one side to the South (for example), then you get more scum on the hull on one side. That can influence trim also.

One test, is to use the same sheet settings on both tacks for sheet tensions and lead positions and let the boat settle down without any rudder action. (i.e., let the boat balance itself by adjusting the sheets for a helm without using the rudder. Then switch tracks. See if the boat sets-up the same and balances. Measure the speed differences. You may also knotice the rudder position is actually somewhat different even though everything else appears to balance the same.