Speaking at a candidate's forum in Harlem Monday night, Rangel said the president hasn't "been around long enough to determine what my dignity is.''
The embattled 80-year-old congressman said it was more likely that he would protect Obama's dignity over the next two years.
Obama's remark was made three weeks ago.
A House ethics panel has accused the 20-term Democrat from Harlem of ethics violation charges. Rangel has vowed to fight the charges and is refusing to resign.
He says he is focusing on his re-election to his district in Manhattan's Harlem neighborhood.