I’m trying to remove duplicates from a MySQL table using ALTER IGNORE TABLE + an UNIQUE KEY. The MySQL documentation says:
IGNORE is a MySQL extension to standard SQL. It controls how ALTER TABLE works if there are duplicates on unique keys in the new table or if warnings occur when strict mode is enabled. If IGNORE is not specified, the copy is aborted and rolled back if duplicate-key errors occur. If IGNORE is specified, only the first row is used of rows with duplicates on a unique key. The other conflicting rows are deleted. Incorrect values are truncated to the closest matching acceptable value.
When I run the query …
ALTER IGNORE TABLE table ADD UNIQUE INDEX dupidx (field)
… I still get the error #1062 – Duplicate entry ‘blabla’ for key ‘dupidx’.
IGNORE keyword extension to MySQL seems to have a bug in the InnoDB version on some version of MySQL.
You could always, convert to MyISAM, IGNORE-ADD the index and then convert back to InnoDB
ALTER TABLE table ENGINE MyISAM; ALTER IGNORE TABLE table ADD UNIQUE INDEX dupidx (field); ALTER TABLE table ENGINE InnoDB;
Note, if you have Foreign Key constraints this will not work, you will have to remove those first, and add them back later.
Or try set session old_alter_table=1 (Don’t forget to set it back!)
The problem is that you have duplicate data in the field you’re trying to index. You’ll need to remove the offending duplicates before you can add a unique index.
One way is to do the following:
CREATE TABLE tmp_table LIKE table; ALTER IGNORE TABLE tmp_table ADD UNIQUE INDEX dupidx (field); INSERT IGNORE INTO tmp_table SELECT * FROM table; DROP TABLE table; RENAME TABLE tmp_table TO table;
this allows you to insert only the unique data into the table