You can simply pick a name and use as a sole trader, no need to register but due the necessary checks to ensure nobody else is using it.
to do the work legitimately, you will need to register as self employed and do a tax return each year. It is possible to be both PAYE and self employed. Note that while you are employed full time all of your tax allowance will be used up through your employment so any handyman work will be taxed on everything (less expenses).
The best starting point for a new business is SWAT analysis. I would spend lots of time researching who is currently working at your proposed handyman work in your catchment area, look at their advertising, sign written vans, websites etc. Make notes and see what USP your businees would have. Then research your potential market. Ask your friends and family what services they have used in recent years, how much they have paid etc.
Don't get a sheet printed that says: 'no job too small' or: 'I can do any sort of handyman work' or other vague statements.
We have a local free A5 booklet that is distributed in our town and surrounding areas. I came across an advertorial and advert for a local handyman. He had a clear businees: 'I only do small jobs that take no more than 2 days', 'I am a registered electirician and can do small electrical jobs like fit a single spur socket', 'I can do repairs to fence panels or fit a new panel, guttering' etc etc.
We used him to do our guttering as I didnt have a ladder, he confirmed a price and time to do the work. He arrived on time, was wearing a sign written polo shirt, had a sign written van. When he finished he showed me what he had done, and presented me with a hand written receipt on headed paper and I paid him then (he made his terms and pricing clear when I booked). My sister has used him a number of times to do small jobs. The problem she finds is that builders, plumbers, fencing contractor etc etc can't be bothered with small jobs or looks at the job but never comes back to quote. This guys USP is that he will clean, effiicient, professional, will do small jobs others arent interested in, and he can do a few jobs a day and charge from about £60 or so pounds for each one.
If you could have a website, with pricing for a range of jobs, eg guttering £60 for semi, front and back, max 15 metre run, easy access etc (subject to survey of course), that would really help with enquiries. People want to be able to look up on their ipad or see an advert that lists matches the service they require and has a price indication.
It will take you up to 2 or maybe 3 years to become established so include this in your business plan. When you make the jump and work at it full time you will need savings as a buffer to cover this period. Dont forget to allow for holidays, some sickness, lost days due to bad weather, and some time wasters and non payers in your costs. When you book in a job, always state how much it will be, and that you will need to be paid before you leave. If the job will cost more, or there are extras, get it agreed before you start and obtain a signature on a job sheet. If the job is too difficult -say and abandon it at the start, dont get bogged down in something that could lose you money.
I hope it goes well!