It's not enough to flush. You won't get good adhesion. I'd suggest you route it out with a Dremel or router to be sure you have a fresh surface, then fill with a wood shim sawed and trimmed to fit exactly, or epoxy with sawdust or other fiber thickening added. A polyurethane (e.g. Gorilla Glue outdoor formula) or epoxy might be best if you're going to glue a shim, but I'd defer to others. If the wood shim is used, you can cut flush with a sharp wood workers chisel and sand it perfectly level after it dries.