Thanks guys,
I was planning on just clamping a bit of wood so my dowel is sandwiched between the wood and my router tables fence, and feed the dowel into the bit, obviously being stopped eventually by the bearing. Slowly and carefully chomping away at it, pull it out, rotate it slightly and go...