The 19-year-old strike was snapped up by new boss Lee Clark on a four-year deal from Ipswich in the summer and fresh from scoring as a substitute against Southend on Saturday, Rhodes netted either side of the break to make a dream start to his Terriers career.
Better was to come for the home side as fellow striker Theo Robinson matched Rhodes by scoring on his debut on 74 minutes moments after coming on as a sub.
Lee Clark opted to shuffle his pack after the weekend's 2-2 draw with Southend United.
Rhodes, who scored the equaliser 24 minutes into his debut against the Shrimpers, was promoted from the bench along with Lee Novak and Tom Clarke.
And it was the home side who were the first to go close when Anthony Pilkington saw his header flash just wide.
The Terriers continued to press as Anthony Kay drilled a long-range effort past the post and was again off target with a shot that cleared the bar when he was well placed inside the area on the half hour mark.
Just when it looked like the home side's dominance would go unrewarded and Rhodes pounced on 45 minutes.
Good build up play was finished off by the Rhodes inside the box as the former Ipswich youngster again demonstrated his razor sharp predatory instincts.
Those were again on hand when Rhodes lashed home from the edge of the area just two minutes after the break to all but seal the Terriers' passage into the next round.
To complete a man-of-the-display Rhodes provided the assist for Huddersfield's third with a great cross that was converted by sub Robinson.
Stockport salvaged some pride when David Poole pulled a goal back with two minute remaining.