In front of a crowd of 909, the Poppies were looking to go at least joint-top alongside King`s Lynn Town with a win, while the Bulls are playing catch-up in the league after their various cup exploits.
In the end, it was the visitors who produced a very professional performance, running out 3-1 winners, much to the joy of their army of away followers.
Eliot Richards gave the Bulls the boost of an early 7th minute lead that top scorer John Mills doubled after 36 minutes.
Kettering battled hard to try and get back into the game and eventually did manage to reduce the deficit in the 81st minute through Aaron O`Connor.
But any hopes they had of mounting a late comeback virtually disappeared late on when Brett Solkhon was sent-off for a last-man pull-back on Mike Symons.
Symons extracted further revenge for that when he confirmed the points would be going back to Edgar Street when he made it 3-1 in the 89th minute and ensured the Bulls became the first side to gain a victory at Kettering in the league this season.
The win lifted Hereford to within three points of the Poppies, with four games in hand.
Hitchin Town moved up to sixth, three points outside the top five, after a 3-0 derby win at the ProKit UK Stadium against a disappointing Bishop`s Stortford side, who remain in nineteenth spot.
Dan Webb gave the Canaries a perfect start with a goal inside the opening 3 minutes.
Trey Charles then doubled the visitors` lead after 17 minutes, getting on the end of a Connor Vincent flick-on.
It wasn`t until a minute into stoppage time at the end of the game that the Canaries added a third, when Lucas Kirkpatrick converted a penalty for which the Blues` Joe Robinson was dismissed for a second yellow card.
Two sides sitting in sixteenth and seventeenth place at the start of play, Merthyr Town and Basingstoke Town, shared a 1-1 Penydarren Park draw in the evening`s other game.
In front of 302, it was goalless until the 67th minute when `Stoke took the lead when Callum Bunting headed a free-kick in at the far post.
But the battling Martyrs pulled back on level terms in the 81st minute, thanks to a terrific volley from Miles John.
By Steve Whitney
Keep up-to-date with our exclusive email newsletters.
Subscribe