It's not that hard to do, and +1 for CHJ's reference. You preferably need a dial gauge but for woodwork you can get away with a pointer on a stand; you'll have to estimate the mid points so it will take three or four iterations rather than one or two but is possible.
A four jaw chuck is fine for wood if the jaws are not too skinny. Most woodwind instrument makers turn their products on engineering lathes with three or six jaw self-cantering chucks or a 4-jaw independent chuck. It is normally worth allowing plenty of sacrificial material to be gripped in the chuck, which makes it usually OK for spindle work but very wasteful for bowl turning.
For economy, you could get either a screw chuck or a faceplate which fit in the 4-jaw, and off you go; only need to centre it once. A pic of your chuck and lathe would help to advise you how to do this cheaply.
Keith