Glen Johnson's stunning 30-yard strike looked to have earned Tony Adams his second win as Portsmouth manager before Windass bundled home one minute from time.
Portsmouth could have taken the lead inside the first two minutes when Papa Bouba Diop headed against the underside of the crossbar from Johnson's cross.
Pompey's players surrounded referee Stuart Atwell claiming the ball had crossed the line but linesman Martin Yerby kept his flag down and television replays later showed the officials were correct.
There was no let off for Hull when Peter Crouch opened the scoring 12 minutes later.
Johnson's left-foot shot was only cleared to Younes Kaboul, who delivered a pin-point cross to the head of Crouch and the lanky England striker nodded home from two yards out.
Diop then provided a contender for miss of the season when he sliced wide from four yards with only Boaz Myhill to beat after Crouch had headed John Utaka's cross into the midfielder's feet.
Hull came agonisingly close to making it 1-1 moments before half-time when former Pompey trialist Geovanni's stunning 25-yard curler crashed against the crossbar.
The Tigers did not have to wait long for their equaliser, which came nine minutes after the interval courtesy of Michael Turner's third goal of the season.
Dean Marney's in-swinging left-wing corner was flicked on by Kamil Zayatte and Turner rose unchallenged to head past home goalkeeper David James at the back post.
Portsmouth were back in front in spectacular fashion nine minutes later.
Sean Davis' corner was cleared as far as Johnson, who controlled the ball beautifully on his chest before firing past Myhill with a bullet shot from his weaker left foot.
Utaka then blazed over from three yards and the Nigerian was made to pay when Turner's long-range shot deflected off Noe Pamarot for Windass to nod the ball past a helpless James.