The complaint filed Thursday in an administrative court in Virginia accused the college of imposing unnecessary and discriminatory hurdles to employment for more than 100 non-U.S. citizens since 2007.
Under federal law, employers can't pose different or greater eligibility standards on non-citizens.
"Every individual who is authorized to work in this country has the right to know they will be free from discrimination as they look for a job, and that they will be on the same playing field as every other applicant or worker," said Thomas Perez, assistant attorney general for the Justice Department's civil rights division.
John Jay College is a public institution part of the City University of New York. The college did not immediately comment.
In one instance described in the complaint, the college is accused of firing a woman who provided a Social Security card and driver's license to re-verify her employment eligibility.
Those documents were legally sufficient, but she was also asked to produce her green card, the complaint said.
The complaint asks for relief for the losses suffered by the employees, plus civil penalties of $1,100 for each person who was a victim of discrimination.
The college said in a statement that it was working to address the issues that the Justice Department had raised.
"We will be instituting a comprehensive training program to prevent any recurrence," the statement said.