Chris Boyd's side came back from 14-0 and 24-17 down to secure their second win at Exeter Chiefs in 2021.
It was some way to back up the February success at the same venue as Saints bounced back from a horrible start and eventually held their nerve to claim a 26-24 victory.
First-half tries from Alex Mitchell and Matt Proctor pulled Saints back level at 14-14, and they took the lead with a James Grayson penalty.
Points were hard to come by in the second period, with Exeter going back in front at 24-17 up.
But Saints kept chipping away through the boots of Grayson and Furbank, and it was the latter who was the late hero with a fantastic penalty from to the right of the posts.
It means Saints have now won both Gallagher Premiership matches this season, while last season's runners-up, Exeter, have lost both of their games.
Saints had headed to Sandy Park with plenty of belief on the back of their win against Gloucester seven days earlier.
And the away side flew forward early on, breaking the Exeter line, but Furbank couldn't quite find the faster members of the side who had gone with him and the hosts survived.
After that early excitement subsided, Saints were quickly hit by not one, but two punches as Exeter delivered two tries in as many minutes.
First, a lineout in the Saints 22 laid the platform for a trademark home charge, with Don Armand eventually finding a way over under the posts.
Joe Simmonds converted, and he was soon doing so again after Exeter cut Saints apart, with Jack Maunder latching on to a kick ahead to score.
At 14-0 down after just seven minutes, Saints were already staring down the barrel, with the home fans dancing to the try tunes, which included the Macarena and Non, Je Ne Regrette Rien.
Saints were all at sea, struggling under a barrage of pressure, but after holding out at a lineout and winning the ball back, the away side pieced together a crucial move.
Grayson tiptoed through the traffic close to halfway and kicked ahead, leading to a Tom O'Flaherty fumble and consequently a try for the onrushing Mitchell.
Grayson converted to cut the gap to seven points and Saints suddenly had some belief, with only a good scrambling stop from Josh Hodge stopping Tom Collins from scoring on the left.
But Saints did soon level the scores as Tom Wood stole Exeter lineout ball before some patient play led to Proctor showing fine footwork and awarness to skate over the line.
Grayson converted and after a horrible start, the away side had parity.
And they had the lead eight minutes before the break as Grayson landed a long-range penalty.
Exeter were desperate for a response before the break and after Sam Matavesi was sin-binned, the hosts took immediate advantage, powering over through Rus Tuima.
Simmonds converted to make it 21-17 to the hosts on half-time.
Saints did well to ensure no further damage was done during the Matavesi sin-bin period, and they were still more than in the hunt with half and hour to go.
But Exeter held firm at a Saints lineout deep in the 22 and started to crank up the heat themselves, winning a penalty, which Simmonds kicked.
Simmonds missed with a long-range effort soon after as Saints continued to struggle to get any momentum in the half.
Grayson did notch their first second-half points with a penalty 17 minutes from time, setting up a big finish.
And when Furbank notched a penalty, Saints were just a point behind.
Exeter looked like they may be able to close it out, but Saints kept going and going, playing in the opposition half.
And when they won a penalty after a period of pressure close to the Exeter line, Furbank stepped up to put his side 26-24 up.
Exeter had one final chance as they booted a penalty into the Saints 22, but the hosts were penalised for a neck roll at a ruck and Furbank kicked the ball out to confirm the win.
Exeter Chiefs: Hodge; Cordero, Slade, Devoto (Whitten 64), O'Flaherty; J Simmonds, Maunder (Hidalgo-Clyne 56); Hepburn (Moon 50), Yeandle (c) (Innard 56), Williams (Iosefa-Scott 63); Witty, Lonsdale; Kirsten, Armand, Tuima (Tshiunza 56).
Saints: Furbank; Sleightholme, Proctor, Dingwall, Collins; Grayson (Freeman 64), Mitchell (Lomani 72); Waller (Auterac 53), Matavesi (Fish 72), Painter (Hill 40); Ribbans, Ratuniyarawa (Moon 64); Wood, Ludlam (c), Harrison (Augustus 52).
Referee: Christophe Ridley