It's Long Island's little-known secret, a Russian compound sitting in the middle of Upper Brookville, and many longtime residents weren't even aware of its existence.
Now the 49-room mansion on a 14-acre compound, purchased by the Soviet Union in 1954, is being shut down by the U.S. government. The White House said Russia had been notified that Russia would be denied access to both that compound and another one in Maryland starting noon on Friday.
About a half-hour before a noon deadline, caravans of diplomatic vehicles, some carrying boxes, left both Russian compounds under the watch of U.S. State Department agents.
Once the last vehicle pulled away, a State Department official confirmed to NBC 4 New York the compound was vacant and under its control.
The longtime getaway for Russian diplomats in Upper Brookville was also used for Russian intelligence purposes, according to U.S. government officials, and President Obama Thursday ordered both the New York and the Maryland compounds shut down, part of several actions the U.S. is taking to punish Russia for election hacking.
"I didn't even really know," a resident named Cornelia said of the compound's existence. "It's kinda really scary, in your own backyard."
Others say they heard the luxurious mansion was Russian-owned, but what went on inside was always a mystery.
"Who knows what kind of activity they're engaged in?" said Jared Greenman. "Are they diplomats, spies, hackers -- who really knows?"
"Ever since the late '70s, that's when I knew about it," said another resident named Johnny. "I was in high school."
"Never seen any activity there. Gates open, gates close, and that's it," he said.
Upper Brookville village mayor Elliot Conway said although he's never actually been invited inside the compound, "they've been quiet neighbors as long as they've been here."
Meanwhile, Penny Hallman, 68, whose home abuts the estate, told The Associated Press that the diplomats were "wonderful neighbors."
"They brought a bottle of vodka and chocolates to wish us a Merry Christmas," she said. "It's mostly a social club, a vacation spot."
The historic website pastvu.com identifies it as a true Gold Coast property, finished around 1920. It's a part of Long Island made famous in F. Scott Fitzgerald's 1925 novel "The Great Gatsby."
Its main house originally had 27 rooms and 11 baths and was constructed for an executive at a Brooklyn company that made heavy machinery and torpedoes. Later it became the home of a former New York governor, Nathan Miller.
Satellite photography shows that the grounds today include a tennis court, gardens, a soccer field and another large, modern building.
Although the White House announced at the same time it was kicking out Russian officials and closing facilities, it said those were responses to other troubling Russian behavior: harassment of U.S. diplomats by Russian personnel and police.
The 35 Russian diplomats being kicked out are intelligence operatives, Obama said. They were declared "persona non grata," and they were given 72 hours to leave the country. The State Department declined to identify them.
Congressional leaders on both sides of the aisle called the sanctions against Russia long overdue.
"I just wish he had done this several years ago," said Republican congressman Peter King, who represents Long Island. "The Russians have been hacking for several years."
New York's U.S. Sen. Charles Schumer, a Democrat, said in a statement: "I hope the incoming Trump administration, which has been far too close to Russia throughout the campaign and transition, won't think for one second about weakening these new sanctions or our existing regime."
"Both parties ought to be united in standing up to Russian interference in our elections, to their cyberattacks, their illegal annexation of Crimea and other extra-legal interventions," he said.
For his part, Trump issued a statement Thursday saying it was "time for our country to move on to bigger and better things." Yet in the face of newly public evidence, he pledged to meet with members of the intelligence community next week.
Russia responded angrily in anticipation of President Obama's announcement and suggested it might retaliate against American diplomats. A Russian Foreign Ministry spokeswoman called it a last blow by Obama to U.S.-Russia relations and added, "We are tired of lies about Russian hackers that continue to be spread in the United States from the very top."
Russian U.N. Ambassador Vitaly Churkin told reporters at U.N. headquarters that the Obama administration was destroying holiday fun for the children of Russian diplomats.
"I think it's quite scandalous that they chose to throw out our kids," he said. "They know full well that those two facilities they mentioned, they are vacation facilities for our kids and this is Christmas, and this is vacation time for our schools. This is the time when the kids go to those facilities. So to close our access to them just while those holidays were starting, to me was rather silly."
A short drive away from the Upper Brookville compound, Russian diplomats stay at another grand Gold Coast estate, the Killenworth mansion, not far from the city of Glen Cove. It, too, was bought during the Cold War. Glen Cove Mayor Reggie Spinello said Friday that Killenworth was not being closed down by the government.
Both Long Island properties were the subject of long-running property-tax battles between the Russian government and local officials. Those disputes have been resolved, and for years Oyster Bay has waived parking and beach fees for Russia's U.N. diplomats as a goodwill gesture.