The drill, part of the "Securing the Cities" program, was a coordinated effort to see how 150 officers from seven agencies could work together to track and stop a vessel carrying a dirty bomb.
"We're obviously concerned about the possibility of a nuclear device or radiological material for a dirty bomb coming into New York Harbor," said New York City Police Commissioner Ray Kelly.
Some 18 police and Coast Guard vessels participated in the drill, which set up checkpoints where officers watched monitors and tried to find the four pleasure crafts outfitted with simulated radiological material that set off detectors.
NYPD counter-terrorism chief James Waters explained that harmful substances could be hidden in small spaces aboard the boats.
"Something that could easily be in a cooler on a boat could be used to manufacture a dirty bomb," Waters said.
New technology allows the police to tell the difference between harmless radiation emitting from medical equipment and the kind used in a weapon. During the drill, computers linked with command centers enabled the readings to be quickly analyzed by experts.
Kelly stressed that this drill was not launched as a result of a specific threat against the city but added. "We do know that Osama Bin Laden several years ago obtained a fatwa to use nuclear weapons and our goal is to make certain that that fatwa does not come to fruition."
Police say these drills amount to a last line of defense and while it's important to prepare the best defense is stopping a device from ever reaching New York's shores.