A statue of Mary and another of Jesus on a cross were both doused with paint at St. Anselm, while the doors of Holy Cross Greek Church and Bay Ridge Jewish Center, both a few blocks away in opposite directions, were painted with "no."
One parishioner at St. Anselm made the sign of the cross upon seeing the paint-covered statues and said the vandalism gave her chills.
Another said she was disgusted by what she saw as acts of violence.
U.S. Rep. Michael Grimm said the graffiti was repulsive.
Kelly said the suspect lives in the area, and walked up to detectives Tuesday to say he was the vandal.
Police sources say he approached a captain and said "you got me. I did it all."
The motive is still unclear.