Matt Wright gave the Peacocks a 28th minute lead that Ollie Chamberlain doubled a minute before the break.
Brad Hooper reduced the deficit from the penalty spot 4 minutes after half-time and then TJ Bohane grabbed `Marine`s equaliser on the hour-mark.
However, Dan Sullivan grabbed Taunton`s winner 5 minutes later.
Newcomers Bristol Manor Farm remain in second spot after winning 2-1 at another top six outfit, Winchester City.
But the Toolstation Western League champions left it very late after City led through Ollie Bailey`s 55th minute effort.
Dean Stamp popped up with the visitors` equaliser in the 82nd minute and then scenes of joy for away fans as Aron Robbins grabbed the winner 5 minutes later.
Salisbury FC are two points behind in third after they inflicted a 5-2 defeat on hosts Slimbridge.
A hat-trick from Lewis Benson led Whites to three points, getting the ball rolling by netting a free-kick to make it 1-0 after 32 minutes.b
But the hosts struck twice in 3 minutes through Sean Lawson and Max Harper-Emerson midway through the second half to turn the game on its head.
Danny Young levelled for the Whites on 75 minutes before Kane O'Keefe put Steve Claridge`s side in front in the 78th minute.
Benson then sealed the game with two more in stoppage time, the final one another free-kick coming 6 minutes into added time.
Evesham United were looking to join Bristol Manor Farm on twenty-two points when they hosted Didcot Town, but instead slipped to fourth after losing 2-0.
Stuart Cattell after 10 minutes and then new signing Felipe Barcelos 15 minutes from time were the Railwaymen on target.
AFC Totton were looking to move into the top six as they travelled to Cinderford Town.
But a goal from Dan Clare after 25 minutes proved to be the only one of the game to earn the Foresters all three points.
Bideford sealed a 3-0 home win over the other side relegated from the Premier last season, Cirencester Town.
Biddy had a great start with goals in the 4th and 7th minutes from Billy Tucker and Ryan Turner respectively - the second from the spot.
Craig Allen added a third 3 minutes before the break, and any hope the visitors had of getting back into the game disappeared on the hour-mark when `keeper Matt Wieczorek was sent-off for challenge on Turner.
Bottom-markers Bishops Cleeve were on the wrong end of a 5-1 home battering by Bideford`s Devon neighbours Barnstaple Town.
It was only 1-0 at the break, with Lloyd Gardner giving Barum a 33rd minute lead.
Elliot Kennedy equalised 4 minutes after the interval, but Jordan Charran on 54 minutes, Jed Harper-Penman 10 minutes later and then a Jordan Charran brace on 73 and 75 minutes completed the visitors nap hand.
Yate Town pulled themselves up to eighteenth place after a comfortable 3-0 home win over Shortwood United.
Loanee Lewis Leigh-Gilchrist gave Yate a 39th minute lead that Cameron Ebbutt and Ben Brooke eventually added to in the final 10 minutes.
The other two matches ended in 1-1 draws.
Toby Holmes gave Wimborne Town a 39th minute lead at North Leigh, whose Sam Mendes equalised just past the hour-mark.
And Lance Williams broke the deadlock to give Kidlington the lead at Larkhall Athletic in the 70th minute, only for Joe Tumelty to earn a point for the Larks from the spot 8 minutes from time.
By STEVE WHITNEY
Keep up-to-date with our exclusive email newsletters.
Subscribe