Pompey won their first away match of the season since 1988 courtesy of a couple of late penalties in each half of the game, although the large contingent of visiting fans faced a jittery end after the hosts scored late on.
Paul Cook picked the same starting eleven as the Dagenham game and the first half was an even affair with few clear cut chances despite some bright play from Pompey early on which saw Gareth Evans strike the ball straight at McCormick when he should have done better.
Both goalkeepers were forced to tip a shot over the bar but were rarely troubled as the match was lacking any quality final balls. Plymouth eventually played their way into the game which led to a few nervy moments at the back but some basic defending saw the hosts have little to shout about.
The travelling Pompey fans, however, did have something to shout about at the very end of the half after tenacity on the left from Gary Roberts led to Enda Stevens running into the box and being taken out by Hiram Boateng. Some would say it was a contentious decision but a shoulder charge like that would be given as a foul anywhere else on the pitch.
Matt Tubbs, who said after the match that he'd done his homework on goalkeeper McCormick, blasted the resulting spot kick straight down the centre to give Pompey the lead right on the stroke of half time.
That goal seemed to give Paul Cook's side a boost as the second half saw them create a few more chances whilst Plymouth barely had a sniff at Brian Murphy's goal.
Tubbs wasted a couple of good opportunities to double the lead as he struck a shot wide of the post and then completely skewed another effort well off target when played through on goal at an angle.
Kyle Bennett and Evans also had shots that narrowly missed the target but the second goal also came from a spot kick after Roberts beat Gary Sawyer in the box and was then brought down by the Argyle defender. If anything, that decision was softer than the previous one but referee Steve Martin had been giving soft decisions to both teams all match so the penalty award was hardly a surprise.
Roberts struck the ball into the far corner on 85 minutes to double Pompey's lead and secure what should have been a comfortable victory for the visitors.
However, four minutes later Gregg Wylde found the space to produce a sweet left-footed volley that gave Murphy no chance of saving it to make Pompey fans sweat during the five extra minutes of stoppage time.
Pompey sides of old may have crumbled under the pressure as Plymouth won two late corners and whipped them into the box only for them to be dealt with despite a shout for handball. Thankfully the Blues were able to see out the remaning minutes and record their third straight win, much to the delight of all those who travelled to Home Park.
Overall, it was a deserved victory considering Pompey created more chances and had more far more possession than their Devon counterparts.
Gary Roberts will rightly take the plaudits for having a hand in both goals but this game was won in midfield where Pompey pressed enough to win the ball back from Argyle whenever they lost it or at least force them into making mistakes.
It was a team effort once again but Tubbs still looks uncomfortable in the 4-2-3-1 system despite his goal. He was virtually anonymous in the first half and wasted two good chances in the second before he was taken off.
Reuben Reid has been touted as a possible replacement for him but he wouldn't get into the Pompey team based on yesterday's performance as he was dominated by the impressive Christian Burgess and Matt Clarke, ensuring he only fed off scraps throughout the entire match.
This result may well set the tone for the rest of the season as there won't be too many teams that take all three points from Plymouth. People may say "don't carried away" but there's no harm in enjoying supporting Pompey for a change after the last five or six years of dross.
Long may it continue with three points at Crawley on Tuesday.
Pompey Player Ratings
Brian Murphy: 6 - One decent save but defence dealt with most of Plymouth's threat.
Ben Davies: 7 - Solid and dependable.
Christian Burgess: 7 - A couple of nervy moments but dominant in the air and did the basics well.
Matt Clarke: 7 - Another performance to prove we're not missing Robinson.
Enda Stevens: 7 - Could be Pompey's "unsung hero" of the season.
Michael Doyle: 7 - Tirelessly closed down Argyle players.
Nigel Atangana: 7 - System suits him perfectly.
Gareth Evans: 6 - Wasted a couple of good opportunities.
Gary Roberts: 8 - The difference maker.
Kyle Bennett: 6 - In and out of the game but caused some problems.
Matt Tubbs: 6 - Got on the scoresheet but did little else.
Substitutes
Paul Jones
Adam Webster
Brandan Haunstraup
Danny Hollands (Atangana 71): N/A - Shored things up.
Adam McGurk (Roberts 90): N/A - Time eating change.
Conor Chaplin
Jayden Stockley (Tubbs 69): 5 - Got booked and never really got involved in play.