They knocked out Spartan South Midlands League Premier Division Leverstock Green 4-1 at The Meadow.
The Ducks were 4 minutes from elimination in the original encounter before earning this replay, and were ahead after just 6 minutes through Sonny French from close range.
But the lead was wiped out midway through the first half when former Duck Chris Marsh bent a superb effort that came back off the post for Jonathan Lacey to convert.
Eight minutes later the lead was re-established with Alex Collard rising to head home inside a crowded area.
Soon after the break and there was clear daylight at 3-1 when Greg Williams headed in number three.
French added late gloss with a stoppage time fourth when, having been felled by the visiting goalkeeper in the box, he dusted himself down to score the penalty and send the Ducks on to a First Qualifying Round tie at Bostik League North outfit Tilbury.
Evo-Stik League South, West side Evesham United, however, bowed out, beaten 3-2 at home by Evo-Stik League South outfit Newcastle Town after extra-time.
A foul on James Campbell in the box allowed Matt Liddiard to give Evesham an 18th minute lead.
But Aaron Bott equalised a minute before the break.
And then on 65 minutes, Sam Wilson put Castle in front, only for Matt Bennett to level matters once again just 3 minutes later.
Newcastle keeper Rav Gill was having an excellent night and it was mainly thanks to him that the 90 minutes ended all-square.
And, with a penalty shoot-out looming, Bott popped up with his second and Castle`s winner in the 118th minute to earn them a shot at Evo-Stik League South Premier Stratford Town in the next round.
Keep up-to-date with our exclusive email newsletters.
Subscribe