Former Rovers loanee Jake Jervis came off the Portsmouth bench to deny Tranmere a victory at Prenton Park.
The striker curled in the equaliser moments after Tranmere had taken a 2-1 lead and earned the visitors a share of the points in the process.
Ronnie Moore's side had been ahead twice in the game, with Ash Taylor's header cancelled out by Gabor Gyepes early in the second half.
There were three changes to the team that started against Chesterfield in the FA Cup as Liam Palmer, Jake Cassidy and Ash Taylor came in for James Wallace, Cole Stockton and Ben Gibson.
An Ash Taylor header went high and wide from Andy Robinson's free kick on three minutes and it took another quarter of an hour for the next goalmouth action, with Portsmouth's John Ertl firing high over the bar from 25 yards.
Robinson twisted and turned inside the area on 19 minutes before his left foot shot was blocked by Mustapha Dumbuya, while a low, swerving effort from 20 yards was tipped around the near post by Simon Eastwood two minutes later.
The corner that followed led to a Tranmere free kick for encroachment and Rovers again went close to taking the lead. Palmer ran unmarked towards the back post and met Robinson's cross with a powerful volley but his shot was blocked by the body of a defender.
Robinson was denied what would have been a contender for goal of the season on 28 minutes. The former Leeds player ran at Dumbaya before jinking to his right and lifing a shot towards the top right corner which Eastwood superbly tipped over.
However, three minutes later Rovers finally got the breakthrough. A free kick from Robinson was cleared as far as McGurk on the right side of the area and his cross back into the six yard box was met by the head of Taylor who diverted the ball into the top right corner.
It was almost two a minute later. Bakayogo kept possession inside the Portsmouth area with a string of volleys with his final touch being a shot which was kept out at his near post by Eastwood.
A quickly taken throw in gave Ronnie Moore's side another opportunity in front of goal on 39 minutes. The ball was played to Palmer on the right wing and the on loan midfielder drilled in a low cross from which Jake Cassidy's shot was blocked for a corner.
Despite Rovers' first half dominance, the visitors were level within five minutes of the restart. A free kick from Brian Howard reached Gabor Gyepes at the far post and his header went into the top left corner.
McGurk tried to restore the lead with a spectacular overhead kick on 53 minutes but his attempt was high and wide, while Fon Williams easily saved from Nathaniel Mendez-Laing soon after.
Michael O'Halloran and Robinson couldn't fashion a shooting chance on 63 minutes and a McGurk cross went straight through the six yard box seven minutes later before the winger fired wide with another attempt moments later.
Rovers regained the lead on 74 minutes but it didn't last long. Palmer found Robinson on the right wing and his low cross was flicked into the path of O'Halloran who nodded in from close range.
However, less than 60 seconds later Pompey were back level. A ball up the left wing found substitute Jervis who superbly curled into the bottom right corner despite Owain Fon Williams getting his fingertips to the ball.
Bakayogo almost put Tranmere ahead for a third time on 77 minutes but his close range flick header from Robinson's free kick went over the bar.
Palmer twice went close to grabbing a winner on 90 minutes. First he ran on to Cassidy's flick on and cut inside a defender only for his one on one shot to be saved by the feet of Eastwood. Then a knock down from Cassidy fell into his path but he couldn't react quickly enough and the visitors cleared.
There was a late opportunity for Pompey to steal all three points deep in injury time after Robinson had lost possession but, under pressure from Goodison, Scott Allan shot wide of the target.
Tranmere: Fon Williams, Holmes, Taylor, Goodison, Bakayogo, McGurk (Stockton 72), Harrison, Palmer, Robinson, Cassidy, O'Halloran. Substitutes: Mooney, Kay, Daniels, Black, Power, Thompson.
Portsmouth: Eastwood, Howard, Harley, McLeod, Rocha, Russell (Williamson 83), Ertl, Benson, Dumbuya, Gyepes, Mendez-Laing (Jervis 67). Substitutes: Smith, Allan, Michalik, Compton, Webster.
Referee: David Webb
Yellow cards: McLeod (30), Russell (80), Williamson (90+1)