Spokesman George Arzt says the 84-year-old Koch had a coronary angioplasty procedure on Monday to increase the blood flow to his heart but the procedure was not successful.
He says Koch has decided to remain at New York Presbyterian Hospital and will undergo surgery to replace his aortic heart valve sometime later this week or early next week.
Koch had suffered a heart attack in 1999. He served as mayor of New York City from 1977 to 1989.
His spokesman says Koch has joked to friends that "this is one hospital where the food is good."