A majority of MSPs backed the Scottish Conservative motion to scrap the law, which has been consistently criticised by fans and the legal profession.
The issue was brought to Holyrood as part of Scottish Conservative business, spearheaded by shadow justice secretary Douglas Ross.
And the vote in favour of the motion of 64 votes to 63 dealt the SNP a rare defeat in the chamber.
Scottish Conservative shadow justice secretary Douglas Ross said:
“The SNP can now make no mistake – the Scottish Parliament has voted to repeal this hated law.
“There’s nowhere for the Scottish Government to go, it has to scrap what was unpopular and ill-thought out legislation.
“This is a historic moment for Holyrood – the days of the SNP railroading through this kind of thing are over.
“Football fans and the legal community will welcome this vote, which formally puts on record the Scottish Parliament’s opposition to the legislation.
“It wrongly targeted football fans, the overwhelming majority of whom are law-abiding, and didn’t even work on those who did cause problems.
“I look forward to ministers now detailing the timetable for repealing this law.”