Tampa Bay appeared to be headed for another loss to the New York Yankees when the first five batters reached against Rays starter Chris Archer.
Archer, though, settled down and Rays fought back to get their second victory in eight games with the Yankees this season.
Evan Longoria hit a tiebreaking sacrifice fly during a two-run eighth inning and the Rays rallied late to beat the Yankees 4-2 on Tuesday night.
"The way the game started I don't know if we saw an ending like that," Rays manager Kevin Cash said. "It's a credit to Arch, keeping his cool, keeping us right there in the ballgame. Extremely impressive outing."
Archer gave up two runs and seven hits in seven innings. He retired 15 of 16 over his final five innings.
The Yankees loaded the bases with no outs in the first, but scored just twice on consecutive RBI singles by Brian McCann and Carlos Beltran.
"Honestly not thinking about the situation," Archer said. "Just thinking about executing my pitches. I just stayed in the present moment and was able to work through that first."
Archer got key help from catcher Rene Rivera, who threw out both Jacoby Ellsbury and Brett Gardner attempting to steal second base during the second.
Kevin Kiermaier drew a one-out walk in the eighth from Nathan Eovaldi (3-1) and went to third on Steven Souza Jr.'s single. Dellin Betances entered and gave up Longoria's sac fly. Souza scored on Betances' wild pitch to make it 4-2.
"Dellin wasn't fully rested, and I was trying to stay away from using him but we got in a situation where we needed a strikeout and that's why I put him in," Yankees manager Joe Girardi said.
Tampa Bay tied it at 2 on David DeJesus' two-run single in the seventh.
Kevin Jepsen (1-2) pitched a scoreless eighth before Brad Boxberger got the final three outs for his ninth save.
Eovaldi gave up four runs and six hits over 7 1-3 innings.
Ellsbury went 2 for 3 against Archer, and has 12 hits in 18 at-bats off him.
Alex Rodriguez went 0 for 3 with a walk. A-Rod still has reached base safely in 12 of 23 plate appearances this year in five road games against the Rays, with three homers, five RBIs and eight runs scored.
The announced attendance was 10,417, which marks the second straight day that the turnout was the lowest for a Rays' home game against the Yankees. Monday's series opening crowd was 10,619.
Yankees: Rodriguez was in the lineup after having some left hamstring soreness in Monday night's 11-5 win over the Rays. "Worry about it a little bit, but we'll manage it," Girardi said. ... LHP Chris Capuano (quadriceps) allowed two runs and seven hits in six innings for Triple-A Scranton/Wilkes-Barre.
Rays: LHP Matt Moore (elbow surgery) will have his third batting practice session Wednesday. ... Closer Jake McGee (elbow surgery) will pitch for Triple-A Durham Wednesday and could return this weekend.
Girardi and several players, including Rodriguez and Ellsbury, talked on the phone from the manager's office with Yogi Berra on his 90th birthday Tuesday.
"He was happy to talk to all of us," Girardi aid. "He just told us to keep it going. It was good."
Yankees INF Stephen Drew took grounders at third base before game and will likely get time there backing up Chase Headley. Rodriguez has been the primary fill-in at third, starting two games at the position. Drew has started at second base and shortstop this year.
Yankees RHP Adam Warren (2-1) and Rays RHP Nathan Karns (2-1) are Wednesday night's scheduled starters.