The first half was a cagey affair with both sides having chances before the `home` side broke the deadlock right on half-time with a debut goal for Ramone Rose.
Just five minutes after the restart, the visitors had a great chance to get back on level terms when they were awarded a penalty for handball but David Pratt`s spot-kick was well saved by Danny Boness.
Hayes` Anthony Edgar then struck the post and Rose looked to extend their advantage before the league leaders equalised on 82 minutes when Will Richards headed home a James Guthrie corner.
Pratt made up for his penalty miss when he headed Chippenham winner in the first seconds of stoppage time at the end of the game.
Follow the EVO-STIK League Southern on Twitter at @SouthernLeague1
Keep up-to-date with our exclusive email newsletters.
Subscribe