Cole Stockton and Abdulai Bell-Baggie's first goals for Tranmere earned Rovers a place in the second round of the Capital One Cup.
Stockton struck early in extra time to give Ronnie Moore's injury hit side a deserved 1-0 lead but the home side equalised through Jack Lester. However, Bell-Baggie struck in injury time to the delight of the noisy travelling fans.
The game could have been over in normal time, with Rovers - missing Ian Goodison, Ash Taylor, Mark McChrystal, Paul Black, Adam McGurk and Jake Cassidy - hitting the woodwork in both halves and creating plenty of chances throughout.
The first chance of the game fell to Rovers in the second minute but Cole Stockton sent his header over the bar from Joe Thompson's corner.
Chesterfield attacked the Tranmere defence for the first time moments later but Zoumana Bakayogo soaked up the pressure and cleared the danger, while Kay headed clear soon after.
Owain Fon Williams, who was selected a day ahead of joining up with the Wales squad, saved comfortably from Danny Whitaker on five minutes.
Ben Gibson chased back to deny Craig Westcarr a shooting opportunity on 11 minutes with Neal Trotman heading wide from the resulting corner.
Cole Stockton controlled Andy Robinson's pass, turned and fired wide of the Chesterfield goal from outside the area on 14 minutes while Robinson hit the advertising boards behind the goal soon after.
Fon Williams pulled off an acrobatic save to keep out Sam Togwell's 20 yard half volley on 17 minutes and Gibson did well again to block Whitaker's swinging cross four minutes later.
Rovers were inches from the lead on 24 minutes when Jean-Louis Akpa Akpro rose to meet Robinson's cross. The striker's flick header hit the bar, bounced on the goal line and away from the net before being cleared.
After Bakayogo intercepted inside the Rovers area on 27 minutes, Ronnie Moore's side launched a counter attack that ended when James Wallace ran on to Thompson's lay off and saw his shot deflect wide of the target.
Tranmere had another chance to grab the opening goal on 42 minutes when Liam Palmer closed down a defender and won possession but his attempt to fire low past Tommy Lee when one on one went wide of the right hand post.
A foul on Wallace gave Robinson a shooting chance from a free kick 25 yards from goal but his effort hit the wall.
Danny Holmes calmly Trotman's header off the line from a corner in the first minute of the second half while Robinson's low curling shot was blocked by a home defender two minutes later.
Stockton's right foot attempt was deflected just past the post by Nathan Smith on 51 minutes and the young striker's powerful run ended with a shot that was tipped away from goal by Lee five minutes later, with Wallace heading wide from the resulting corner.
Robinson's 20 yard curler was caught in front of the top right corner by Lee on 59 minutes while Stockton broke into the right side of the area five minutes later and slammed a shot against the right hand post as Rovers continued to dominate.
Spireites substitute Jack Lester held the ball up for Whitaker on 69 minutes but Gibson got his body in the way of his shot and Lester fired wide of the target himself soon after.
The introduction of Lester had given the home side a lift and they went close again on 73 minutes when Jordan Bowery got past Kay but his low effort was saved by Fon Williams.
The keeper was called on again on 75 minutes, diving to keep out Bowery's goal bound header from Whitaker's free kick, while Wallace curled just past the top right corner from 25 yards at the other end.
Wallace and Robinson combined well on 81 minutes with the latter's shot directed just wide of the left hand upright from an acute angle.
Rovers passed the ball around the Chesterfield half on 89 minutes before Akpa Akpro picked out Robinson, however the former Leeds player's attempt was struck straight at Lee.
The home side almost stole a late winner moments later but Fon Williams pulled off a fine save to deny Lester and take the game into extra time.
Rovers took the lead on 93 minutes. Robinson and Akpa Akpro took charge of possession outside the area before the latter found Stockton who turned and fired across Lee into the top right corner from 10 yards for his first goal for the Club.
Bowery narrowly failed to make contact with a chipped pass over the Tranmere back four as Chesterfield looked for an immediate reply and Whitaker sent a free kick over the bar.
Abdulai Bell-Baggie replaced Akpa Akpro on 100 minutes with Robinson moving into a central role behind Stockton. It was Robinson who had the game's next chance, running at the Chesterfield back four before sending a shot just past the top left corner from the edge of the area.
The home side equalised three minutes into the second half of extra time. The ball reached Lester inside the area and the striker coolly finished over Fon Williams into the net.
Max Power and Danny Harrison came on for Robinson and Palmer with eight minutes remaining, adding much needed fresh legs to the side.
After a break in play when the net in the Tranmere goal came loose, Stockton's low shot was well saved by Lee and the keeper kept out Wallace's side footer soon after.
With the game looking destined for a penalty shoot out, Rovers sealed the win. Wallace ran out of room to progress inside the area and played the ball out to Bell-Baggie. He skipped between two challenges, cut back and fired high into the roof of the net from eight yards.
There was still time for a late scare. The ball fell to Lester eight yards out from a Whitaker free kick but his shot on the turn went inches wide of Fon Williams' goal.
Chesterfield: Lee, Forbes, Hird (Darikwa 76), Trotman, Smith, Talbot, Allott, Togwell, Whitaker, Richards (Lester 68), Westcarr (Bowery 55). Unused substitutes: O'Donnell, Boden, Randall, Clay.
Tranmere: Fon Williams, Holmes, Kay, Gibson, Bakayogo, Thompson, Palmer (Harrison 112), Wallace, Robinson (Power 112), Akpa Akpro (Bell-Baggie 100), Stockton. Unused substitutes: Mooney, Vaulks, Kirby.
Referee: David Webb
Yellow cards: Trotman (27), Hird (44), Togwell (66), Bakayogo (74), Whitaker (81), Wallace (113), Bell-Baggie (120)