Redknapp, who was sensationally appointed manager last week, was heading for a dream start when Lomano LuaLua scored a superb goal to give Pompey a half-time lead.
But Spurs fought back to equalise through skipper Ledley King and, with the aid of a controversial penalty, scored twice in the final five minutes to notch up their third successive victory.
Spurs had all the early play but struggled to convert their possession into clear chances. Robbie Keane, again preferred to Jermain Defoe in the Tottenham attack, was put through in the second minute but shot straight at keeper Jamie Ashdown.
Edgar Davids then put Mido through but again Ashdown saved the Egyptian's shot, and the keeper made an even better save when Keane broke through and shot at goal in the 16th minute.
It was one-way traffic, but then Portsmouth scored against the run of play in the 25th minute. Gary O'Neil hit a long crossfield pass towards LuaLua, and the Pompey forward hit the ball first time into the far corner of the net from 30 yards.
It was a strike that stunned Spurs, and they responded with Teemu Tainio shooting just over the bar from a tight angle, while Jermaine Jenas cured a free-kick over the bar.
Spurs started the second half where they had left off in the first, taking the game to Portsmouth, but as they became increasingly frustrated, so Redknapp's side got more into the game and created their own chances.
Vincent Pericard had a great chance to increase their lead, but Paul Robinson made a good save to turn away the striker's low shot.
Then, on 56 minutes, came the equaliser. King, who had been majestic in defence alongside Michael Dawson, ventured upfield and met Michael Carrick's corner with a powerful header that bounced down on the line and into the net.
Pompey were rocked but not sunk. Matthew Taylor broke through to shoot and brought a brilliant save from Robinson, who seemed to twist in mid-air.
Mido then had a header saved at the other end before heading over the bar, and the Egyptian had another shot deflected over the bar.
But he was not to be denied all night, and finally put Spurs ahead from a hotly-contested penalty. Defoe, on as a sub for Keane, was tripped just outside the penalty area in the 83rd minute.
Fellow sub Andy Reid, who had just come on for Wayne Routledge, stepped up to shoot at goal, and the ball hit O'Neil on the arm as he jumped. Referee Uriah Rennie pointed to the spot, Portsmouth's players protested to no avail, and Mido tucked the spot-kick under Ashdown's outstretched arm.
Spurs were now looking to seal all three points, and Defoe obliged with a brilliant goal in the final minute. He took the ball on the right of the penalty box, turned past his marker towards the byline and then hammered a low shot through the legs of Ashdown into the far corner of the goal from the tightest of angles.
It was all over for Redknapp and Portsmouth, as Spurs celebrated maintaining their push for a Champions League place by staying in fourth place.