Currently I'm using LimeSurvey 1.85 for a large scale survey, i.e. more than 20,000 tokens. I've had to import tokens in batches using CSV files. The more tokens in the database, the slower the import.

I've fixed the problem temporarily by creating an index in the token table (a table lime_tokens_nnnnn that was created dynamically) with fields 'email', 'firstname' and 'lastname'. Now it's much faster to import and find duplicates. However I'd have to create these indexes manually each time a token table is created.

The instruction is as follows:

"CREATE UNIQUE INDEX limetokens<nnnnn>_names_idx ON limetokens<nnnnn>( email(120), firstname, lastname );"

It was successfully tested and checked against a MySQL database.