Ricardo Moniz made three changes from the team which was beaten 5-1 at Sheffield United in the Johnstone's Paint Trophy in midweek. Haydn Hollis dropped out of the squad to serve a suspension following his red card at Bramall Lane whilst Thierry Audel and loanee Jack Barmby also missed out.
Alan Smith returned to captain the team with Alan Sheehan recovering from a hamstring injury to make the starting line-up. Swerts was the final change, taking Hollis' place at the heart of the defence.
Kyle De Silva was named on the bench after two months out with an injury sustained in the 2-0 home defeat against Mansfield Town in the middle of August.
After a quiet opening period to the game, Notts threatened as Izale McLeod skipped beyond the challenge of Gary Sawyer and advanced towards goal before trying to play in Liam Noble, but the final ball was neither a cross nor a shot and trickled out of play for a goal-kick.
Notts were finding space on the edge of the opposition penalty area to get shots away with Burke and Noble having a go, only for the ball to end up in the gloves of Plymouth goalkeeper Luke McCormick on both occasions.
Plymouth had their first effort on Scott Loach's goal when winger Gregg Wylde cut in from the left and curled horribly off-target with his right foot.
Some neat interplay led to Notts' next chance as Hewitt scampered away on the right wing before slipping a pass into Burke, who couldn't get quite enough dip on his shot to trouble McCormick as it landed on the roof of the net.
Despite a bright opening, it was Plymouth who took the lead just after a quarter of an hour. Reid collected a long ball forward and span away from Swerts, giving himself enough room to drive a low shot across Loach and into the bottom corner.
It could have been 2-0 within a minute as former Magpies loanee Jake Jervis broke down the right and played the ball across the face of goal where Wylde blazed over from six yards.
County immediately came close to equalising. Adam Campbell's pass gave Hewitt an opportunity to cross for Burke, who saw his shot come back off the post from six yards out.
Noble whipped in a free-kick which was just beyond the reach of McLeod as Moniz's men pressed for the goal to get them back on level terms.
Slick passing between the midfield trio of Noble, Smith and Stanley Aborah ended with Smith's shot deflecting off Gary Sawyer to force McCormick into a terrific one-handed save to palm the effort over the bar. Sheehan slashed wildly over on his weaker right foot from the corner.
Craig Tanner curled straight at Loach whilst a Notts free-kick routine at the other end fizzled out to nothing. Noble miscued a header from Campbell's cross as Notts continued to push forward.
Jervis fired well wide from 20 yards and Swerts did well to force the ball out for a corner when a misplaced pass allowed goalscorer Reid a one-on-one run at the Belgian in the last minute of the half.
In the last action of the half, McLeod's knockdown found Burke, who brought a smart save from McCormick low down to his right. However, it was the visitors who took a slender lead into the break.
Sheehan had a header deflect wide from a corner in the opening minutes of the second period before the left back played a raking cross-field pass to Campbell, who drove a shot straight at McCormick at the goalkeeper's near post.
Burke's floated pass found Sheehan skedaddling down the left and his low cross wouldn't fall kindly enough for McLeod so the striker laid the ball off for Noble to hit a low shot which was held by the goalkeeper.
Noble's perfect through ball gave Campbell another sight of goal as he darted into the space between central defender and full back but, as with his previous effort, it went straight at the goalkeeper.
Tanner curled a free-kick wide of the upright from 30 yards as the clock ticked past the hour mark with Notts still trailing.
Substitute Jon Stead, who replaced Hewitt, turned his marker inside the area but couldn't repeat the finish of Reuben Reid and drilled an effort well wide of the target.
A misplaced header by Nelson gave Campbell a chance to run at a depleted defence. The attacker slipped in McLeod, who curled over the bar with his left foot.
Plymouth wasted a chance to double the lead from a counter-attack as goalscorer Reid broke down the left wing and crossed for Jervis who diverted the ball over the bar as he slid in to connect.
Nelson was allowed to advance half the length of the field and the Plymouth skipper almost got on the end of Tanner's cross but couldn't stretch far enough to make contact.
Moniz made a double substitution with 10 minutes left, bringing on De Silva and Julian Jenner for Burke and Campbell as the Magpies kept searching for the equalising goal.
Plymouth doubled their lead with just under five minutes remaining. A long ball over the top set Reid in on goal but the striker saw his powerful effort tipped away by Loach. However, Jervis collected the loose ball and played it across the face of goal where Reid was waiting to steer the ball into the bottom corner for his second of the afternoon.
The striker could have had a hat-trick but Loach came out to smother his close-range effort as the Pilgrims pilfered the points.
Notts: Loach; Hewitt (Stead, 66), Edwards, Swerts, Sheehan; Aborah, Smith; Campbell (Jenner, 80), Noble, Burke (De Silva, 80); McLeod.
Subs not used: Pilkington, Sprockel, Audel, Thompson.
Attendance: 5,266 (826 away)