It looks like in your picture that if you could figure a way to keep the pole locking pins all the way in as you pulled the pole past the hole that they would slide under the collet and you'd be able to remove the pole. As it is, I would imagine that the pins stop on edge of the collet. I think you're right that the black pins hold the collet in. After the pole is out, then the collet should squeeze and pop out.