The decision leaves the path open for NYSE Euronext to proceed with its previous $10 billion deal to combine with the German exchange operator Deutsche Boerse.
Nasdaq and IntercontinentalExchange are withdrawing their proposed $11 billion bid for the parent of the New York Stock Exchange after recognizing they would not receive regulatory approval for the transaction.
The decision clears the path for NYSE Euronext Inc. to proceed with its previous $10 billion deal to combine with the German exchange operator Deutsche Boerse.
Nasdaq OMX Group Inc. and IntercontinentalExchange Inc. said Monday that they held unsuccessful talks with the antitrust division of the Justice Department about their joint bid for NYSE Euronext.
Shares of NYSE Euronext dropped $4.45, or 10.9 percent, to $36.44 in pre-market trading, which Nasdaq OMX stock fell 80 cents, or 3 percent, to $26.11.
Nasdaq OMX CEO Bob Greifeld said in a statement that the companies had offered a variety of "substantial remedies" to try to secure regulatory approval, including the sale of the NYSE Self-Regulatory Organization and its related businesses.
"While we are surprised and disappointed in the antitrust division's conclusion, some of the uncertainty, at least as it relates to our joint proposal, has been resolved," he said.
The announcement comes one week after Nasdaq and ICE reached out directly to NYSE Euronext shareholders, issuing a letter saying that the NYSE Euronext board was rushing a vote without exploring better alternatives.
The board had twice rejected the Nasdaq and ICE bid in favor of the Deutsche Boerse offer despite the lower price.
NYSE shareholders are scheduled to vote in early July on the merger with the German company.