View Issue Details

This bug affects 1 person(s).
 6
IDProjectCategoryView StatusLast Update
17319Bug reportsSurvey participants (Tokens)public2021-07-09 15:31
Reporteralorenc Assigned Toc_schmitz  
PriorityhighSeveritypartial_block 
Status closedResolutionnot fixable 
Product Version3.25.19 
Summary17319: Generating tokens for more than 10,000 participants
Description

In our case, we imported over 13,000 participants at the client's place, and choosing the option to generate tokens for each of them - after generating 5,000, an error 500 appears. We noticed that it generates too many SQL queries, and that's why.

You would need to optimize the script:

  • for INSERT multiple rows
  • and the use of transactions (e.g. packages of 100 SQL statements)
TagsNo tags attached.
Bug heat6
Complete LimeSurvey version number (& build)v3.25.19+210323
I will donate to the project if issue is resolvedNo
Browser
Database type & versionSQL Server 2019
Server OS (if known)
Webserver software & version (if known)
PHP Versionv7.4.15 NTS x64

Users monitoring this issue

There are no users monitoring this issue.

Activities

ollehar

ollehar

2021-05-21 10:32

administrator   ~64505

Can you enable debug = 2 in application/config/config.php and try again, please? Paste the complete error message here.

alorenc

alorenc

2021-05-21 11:51

reporter   ~64506

In the test environment, we have debugging = 2 enabled. But it only returns error 500 as in the screenshot. Unfortunately, I do not have access to SQL logs, and those on the PHP side do not return anything, just like IIS.

I had a similar error while saving several thousand participants to a database in PHP. Because in general, with fewer queries, the above script works. So I think their amount is killing him.

In my case of the attendee import script, using the transaction was enough to fix the problem.

2021-05-21_11h49_13.png (6,395 bytes)   
2021-05-21_11h49_13.png (6,395 bytes)   
ollehar

ollehar

2021-05-21 12:08

administrator   ~64507

Can you share some of the code you tested? You can do a pull-request too, if you want.

c_schmitz

c_schmitz

2021-05-21 13:09

administrator   ~64508

I don't understand. Why would there be too many queries?
Do you maybe get a timeout on the server?

c_schmitz

c_schmitz

2021-05-21 14:23

administrator   ~64509

There is no such thing of too many queries. The server does not care about the number of queries, as they are all sequential.

There are only two scenarios that could cause this:
Either running out of memory (less like) or out of time. The reason for the error should be visible in your webserver logs.

In general don't think that it is a LimeSurvey problem but a server configuration problem.

alorenc

alorenc

2021-05-28 14:21

reporter   ~64633

I will come back to this topic when I have more free time, I will describe the problem in more detail.

alorenc

alorenc

2021-06-24 16:21

reporter   ~65055

In fact, token generation can be a matter of server configuration.

But moving participants from the Central Base to the survey or from the survey to the Central Base. If there are, for example, 20,000, you have to be patient, because when sending more than 2,000 records, an error 500 will appear every time.

c_schmitz

c_schmitz

2021-07-09 15:31

administrator   ~65266

It sounds to me a like a timeout problem.
You should try to increase max_execution_time in your PHP configuration and in your webserver settings (the exact setting depends on the webserver software you are using)
That will most likely fix the problem.
I will close this issue now as it is not an original problem of LimeSurvey but of resources.
Thank you for your understanding.

Issue History

Date Modified Username Field Change
2021-05-21 10:19 alorenc New Issue
2021-05-21 10:30 ollehar Priority none => high
2021-05-21 10:32 ollehar Note Added: 64505
2021-05-21 11:51 alorenc Note Added: 64506
2021-05-21 11:51 alorenc File Added: 2021-05-21_11h49_13.png
2021-05-21 12:08 ollehar Note Added: 64507
2021-05-21 13:09 c_schmitz Note Added: 64508
2021-05-21 13:09 c_schmitz Assigned To => c_schmitz
2021-05-21 13:09 c_schmitz Status new => feedback
2021-05-21 14:23 c_schmitz Note Added: 64509
2021-05-28 14:21 alorenc Note Added: 64633
2021-05-28 14:21 alorenc Status feedback => assigned
2021-05-28 14:40 gabrieljenik Status assigned => feedback
2021-06-24 16:21 alorenc Note Added: 65055
2021-06-24 16:21 alorenc Status feedback => assigned
2021-07-09 15:31 c_schmitz Note Added: 65266
2021-07-09 15:31 c_schmitz Status assigned => resolved
2021-07-09 15:31 c_schmitz Resolution open => not fixable
2021-07-09 15:31 c_schmitz Status resolved => closed