Hi! Aled, I work a lot with elm, (be it on an amateur level) and depending on the amount of worm damage and the area it covers I usually drill out the damaged section pop in a pellet of elm from the same board if possible, orient the grain direction and sand it flush when dry.
Give it a try on...