Andy Butler spoiled a perfect week for ten-man Cheltenham with a late equaliser.
Defender Butler salvaged a point for the home side with a stoppage time header.
Rock bottom Cheltenham with back to back home wins over Hartlepool and Tranmere looked set for another famous victory.
But in the second minute of added time Butler pounced to deny them a nine point haul.
Cheltenham went ahead in the 5th minute when striker Leon Constantine, making his debut on loan from Northampton, cashed in on Tom Clarke's poor back header. Constantine collected the ball, took a couple of touches and blasted high into the net.
The Robins doubled their lead 13 minutes later when Andy Gallinagh's through ball found Elvis Hammond in the clear, he
turned Butler and fired hard and low past Alex Smithies.
Huddersfield boss Lee Clark was frustrated at his side's failings and cut an angry figure on the touchline.
He replaced out of sorts Lucas Jukkiewicz with Danny Cadamarteri and was quickly rewarded when Cadamarteri swept home from Gary Roberts' cross five minutes before the break.
Huddersfield piled on the pressure in the second half but Chris Lucketti's header straight at Scott Brown was the closest they came for the first half hour.
Cheltenham's hopes were dented in the 61st minute when winger Joshua Low saw red for an apparent stamp on Tom Clarke.
Low's straight red dismissal left Cheltenham with a battle on their hands but Huddersfield were wasteful with crosses.
As the frustration levels grew Anthony Pilkington and sub Michael Flynn let fly with wild efforts from well outside the box.
Huddersfield, who were denied what looked a clear penalty in the first half when Andy Lindegaard brought down Roberts, saw justice done six minutes from time.
Referee Kevin Wright pointed to the spot when Michael Flynn went down surrounded by Cheltenham defenders.
Robbie Williams stepped up to take the penalty but his weak strike to the goalkeeper's left was comfortably saved by Brown.
Just when it seemed Cheltenham had done enough to claim the points Butler struck deep into five minutes of added time.
Cadamarteri's cross was headed back by Pilkington and Butler twisted to nod home at the back post.