I do absolutely nothing - not sure if the climate is a factor (although I would have thought that Sussex gets more sunshine and warmth than Somerset). But 2022 was the first year we got actual olives, previously just the tiny potential fruit you are getting.