Craig Westcarr's late goal ensured that Pompey snatched a 1-0 victory from Oxford having not performed at their best.
Michael Appleton's men had the better of the first half with Paul Jones having to make a couple of excellent saves and Danny Hylton struck the crossbar late on.
James Dunne had to clear off the line to prevent Nigel Atangana scoring an own goal and there was also a strong penalty appeal when Hylton was taken down by Westcarr in the box. Ricky Holmes might have had a second yellow card in another game so could have considered himself fortunate to stay on.
Pompey's big chance in the first half fell to Atangana who sliced a shot wide whilst a couple of good crosses were wasted in what was a poor performance up to the break.
Andy Awford took Holmes off for Jed Wallace at the start of a second half which saw an improved Pompey performance driven by the substitute who was a willing runner at the Oxford defence.
The breakthrough came on the 80th minute when Westcarr bundled in Nicky Shorey's cross but Junior Brown had two great cances to equalise but his first effort was well saved by Jones and the second was a close range header which flew over the bar.
Pompey held on to record their fourth straight win and are now in second place in the early League Two table with Oxford rock bottom. Winning away from home when not playing particularly well is a sign of a team that can win promotion but it's best not to get carried away as there are still 42 games to play and by the sounds of things Pompey will need to play better in many of those games to stay in promotion contention.
Still, ten points from twelve is an excellent start to the season and many Pompey fans will fancy another win when Newport visit Fratton Park on Saturday.
Before that, however, is a midweek trip to Premier League Stoke in the Capital One Cup and that will be the biggest test of the season so far.