Fearne left the show as team captain after 10 years as she decided to take a new direction with her career.
Given that the hosts had a pact to all leave together when one decided to pack it in, Keith was worried that Holly was going to follow her pal out the door.
Speaking on Wednesday’s This Morning, Holly spoke about finding out Fearne was leaving.
She said: ‘When Fearne said she was leaving it was a bit of a shock for both of us, wasn’t it? I mean because you start it together from the beginning and it was always gonna be weird.’
Keith said: ‘I panicked and I thought you were gonna leave as well.’
‘I thought you were gonna say the same thing too,’ Holly replied.
Phil asked the hosts about their pact to leave the show together, to which Holly replied: ‘We did initially but we didn’t know that it was gonna be so long and so fun.
‘What happened was I think we just really enjoy making the show still and I don’t think we were ready for it to stop yet.’
Keith added: ‘You can’t just go, “Oh I’m not doing it” because you enjoy it so much.’
Paddy McGuinness has replaced Fearne as team captain for the new series, and said he had such a great time filming that ‘you do forget you’re on telly.’
Paddy, who filmed the episode on Tuesday night, explained: ‘I did feel nervous at first. I’ve done every series, I know everybody.
‘But when I sat there I thought “Oh I’ve gotta introduce people and be a little bit more authoritative”.
‘That lasted for about two minutes and a couple of pints! I were rolling on the floor with Mark Wright and I’m like we’re in. We’re off.’
This Morning airs weekdays from 10.30am on ITV
Got a showbiz story?
If you've got a story, video or pictures get in touch with the Metro.co.uk Entertainment team by emailing us celebtips@metro.co.uk, calling 020 3615 2145 or by visiting our Submit Stuff page - we'd love to hear from you.
MORE: Emilia Clarke compares Game Of Thrones ending to taking off a bra
MORE: Simon Cowell cuts out diary, sugar and gluten and calls it ‘the Eric Cowell diet’