Scott Clemmensen made 31 saves for the Devils, who beat the Rangers at MSG for the first time since a shootout win on February 22, 2007 -- and for the first time in regulation since December 17, 2006.
Nigel Dawes and Dmitri Kalinin each scored for New York, which has lost three in a row after winning three consecutive contests. Henrik Lundqvist stopped 31-of-35 shots in the loss.
The Devils entered Saturday's contest having not posted a goal since a December 21 shootout win over Philadelphia, but quickly snapped their scoring drought. Rolston's blast from the blueline on the power play, coming 2:54 into the game, gave New Jersey a 1-0 lead and ended its scoreless streak at 159:18.
Then, just over a minute later while the Rangers had a man advantage, Zajac harassed a pair of New York players by the blue line, intercepting a short pass. He fired the puck to Elias, who skated in on Lundqvist alone and buried the disc in the net for a 2-0 Devils lead, at 4:14.
But the Rangers managed to score on the same power play, exactly one minute after Elias' goal. Dawes grabbed the puck in the slot off a deflected shot, and sent a backhander between Clemmensen's pads to make it a 2-1 contest.
The score stayed that way headed into the second period, though Zajac tipped in Elias' shot with 8:48 left in the middle frame to give New Jersey a two- goal lead again.
Clemmensen was stellar in the second period, stopping all 17 shots he saw.
The Rangers got back within a goal at the 8:59 mark of the third period on Kalinin's first goal of the season, but Parise helped seal the game for the Devils.
Jamie Langenbrunner had the puck behind the net and peeked around the left side, shooting a short pass out front. Parise was there to put a shot by Lundqvist with 7:02 remaining for a 4-2 lead, and New Jersey held on from there.