Could be the charger, the batteries, or the amp meter.
I'm not sure how the charger being off for a while wouldn't have been corrected in one cycle of charging?
You've checked the fluids in all batteries, made sure you don't have a bad cell, etc.
When you're drawing down the batteries without a charger, the batteries seem okay?
When you run the engine, something should happen to the accumulated (negative deficit) being reduced?