Or can you also access from the "thumb" side in the pic? If you can access from both sides, drill each hole half way thru, more or less. IF you have to drill from only the backing plate side..it kinda looks that way... drill one hole first thru as squarely as you can..focus on that, set the bolt thru both the backing plate and the unit. Before you tighten up the bolt, position the angle as you want the unit to be. Then drill the other two. You can do it with a bit of care.

Really, I'm sure you're perfectly capable of drilling a straight hole from the "thumb" side to be ok. But if you have to waller the hole out a bit from the "fingers" side to hit the hole in the unit itself, it really won't matter much. I like to chamfer holes (in this case on both sides) a bit with a countersink so it makes a little "doughnut" of caulk at each hole entry.

Don't skimp on caulk, tighten'er up. Rubbing alcohol cleans up most of the uncured caulks easily.