Notts County were beaten 2-1 by Plymouth Argyle at Meadow Lane this evening.
Jon Stead levelled the scores for the second time in as many matches following Jake Jervis' opener.
But Nauris Bulvitis' header handed Argyle all three points before the break.
John Sheridan made one change to the side that faced Stevenage on Saturday, new loan signing Louis Laing replacing Haydn Hollis in central defence.
Sheridan opted for a flat back four with Duffy partnering Laing, and Thierry Audel and Carl Dickinson at full back.
Michael O'Connor shielded the defence, playing behind a midfield four of Alex Rodman, Matt Tootle, Stanley Aborah and Jonathan Forte. Stead took his place up front.
Laing endeared himself to Notts supporters during a previous loan spell with his committed approach to the game, and within minutes he was receiving extensive treatment for a head injury sustained in an aerial battle with a Plymouth man.
The home side had a glimpse of goal on six minutes when Aborah's teasing free-kick sailed just over Duffy's head.
Notts kept up their pressure and, shortly after Dickinson had been booked for a foul on Jervis, they carved the visitors open with some excellent play down the left. Rodman looped the ball into the channel for Forte, who raced into the area and cut the ball back for Stead who was hovering unmarked near the edge of the area. A defender got to the goal line to make an excellent block and the ball rebounded for Tootle to strike. Once again, however, an Argyle defender was on hand to make a heroic block.
Sheridan's frustrations were compounded on 14 minutes when the visitors took the lead. Gary Miller brought the ball forward from right back and fizzed it into Jervis' feet. He produced an excellent turn on the edge of the 'D' and struck the ball sweetly into the bottom corner.
It didn't take the Magpies long to respond, however. Stead controlled the ball down the right, cut inside, showed excellent feet to beat a defender and work an angle for the shot before stroking the ball low past the helpless Luke McCormick.
Jervis was proving himself to be a clear threat for Plymouth but he couldn't beat Adam Collin from 25 yards as he looked to restore his side's lead.
And Notts came within a lick of paint of taking the lead shortly afterwards. Laing's header was blocked well from Dickinson's corner but Sheridan's men kept the ball alive, fizzing it low back into the middle and seeing it flick up off a defender and on to the post. Still they kept going but Laing was unable to round off the frantic spell with a goal, heading the ball over the bar after a cross from the right.
The home side were on top - but had the stuffing knocked out of them on 32 minutes when Plymouth retook the lead from a corner. Bulvitis got the telling touch amid a crowd of bodies as players from both sides leapt to meet the ball.
Notts' keeper was called into action three minutes later to push Graham Carey's free-kick to safety, moments before Miller was booked for a foul on Magpies skipper O'Connor.
Buoyed by their second goal, Plymouth were pushing for another before the break. Bulvitis played an excellent long ball over the top for Oscar Threlkeld who controlled it well but miscued his shot, firing the ball across goal and well wide. There was a double booking shortly afterwards as Connor Smith and Aborah were punished following an off-the-ball altercation.
Notts had one last chance to level the scores before the break after they won a corner during three minutes of added time. Plymouth couldn't clear, leaving Forte with an opportunity to shoot, but the ball was blocked and the visitors ended the half in the lead.
Neither manager made changes at the break but it could have been 3-1 two minutes after the restart had it not been for an excellent block by Duffy to deny Jervis in the area.
Tootle conjured Notts' response with a nicely dinked ball into the area. Forte got away from his marker but could only nod the ball wide with his back to goal.
O'Connor was the next to try his luck, striking the ball cleanly from a 30-yard free-kick but failing to seriously trouble McCormick in the Argyle goal.
The hosts were dominating possession and had another opportunity on 64 minutes after they worked the ball to Rodman on the right. Tootle made a late run into the box to meet his cross but could only head it wide.
Notts were appealing for a penalty moments later. Tootle collected the ball on halfway and lofted it into the area towards Duffy, who had stayed up following a corner. The defender controlled it well on his chest and went down under a challenge but the referee saw no foul.
Plymouth were happy to concede most of the possession to the home side as they focused on protecting their slender lead.
And there was no sign of their resolute defending letting up as, on 75 minutes, a defender blocked Tootle's fierce strike following Forte's cross.
Sheridan made a double change seconds later as Vadaine Oliver and Adam Campbell replaced Tootle and Stead.
And Campbell had an excellent opportunity to make an almost immediate impact when he benefited from excellent work by Forte, receiving the ball on the edge of the area but blazing it over the bar.
It was now a case of all hands to the pump for Notts who, after seeing Collin make a good save to deny Carey, came close to an equaliser following an unlikely one-two between Duffy and O'Connor on the edge of Plymouth's area. The defender struck the ball low from the edge of the area but McCormick held on well.
The closing stages of the match were being played out at a frantic pace as the hosts desperately sought an equaliser. McCormick's attempts to slow the pace were penalised with a yellow card after he dallied over an 88th-minute goal kick.
Five minutes were added on at the end of the match, in the first of which Notts won a succession of corners. The second led to O'Connor striking the ball superbly from the edge of the area but McCormick got down low to his left to make an outstanding save, ensuring his side left the Lane with their first victory of the season.