View Issue Details

This bug affects 1 person(s).
 10
IDProjectCategoryView StatusLast Update
17481Bug reportsSurvey editingpublic2021-09-14 08:50
Reportersamarta Assigned Togabrieljenik  
PriorityhighSeverityblock 
Status resolvedResolutionfixed 
Product Version5.x 
Summary17481: Error adding a large number of answer options (max_input_vars needs to be higher in LS5)
DescriptionError adding large number of answer options to single choice questions

I checked with the types: List (Drop down) and List (radio)

When adding a large number of answer options to single-choice questions, some (1 or 2) of the answer options do not appear and others are out of order
Steps To ReproduceI experimented with answer options with very simple texts: 1, 2, 3.....
Tests I made:

Test 1 - With 200 answer options - instead of 70 the 145 appeared (the answer option "70" disappeared)
Test 2 - with 150 answer options - it worked
Test 3 - with 175 answer options - it worked
Test 4 - With 190 answer options - instead of 65 appeared 169 (answer option "65" disappeared)

This situation was reported by a user of ours who is implementing a survey and one of the questions has a dropdown list with the 308 Portuguese councils
TagsNo tags attached.
Bug heat10
Complete LimeSurvey version number (& build)5.0.11+210727
I will donate to the project if issue is resolvedNo
BrowserSafari, Chrome
Database & DB-VersionMysql 8.0.17
Server OS (if known)
Webserver software & version (if known)
PHP Version 7.4.21

Users monitoring this issue

User List There are no users monitoring this issue.

Activities

ollehar

ollehar

2021-07-28 13:41

administrator   ~65674

Is the problem happening when you're saving or when you're adding them?
samarta

samarta

2021-07-28 15:07

reporter   ~65686

I add the answer options, saved them, and then I checked the errors
samarta

samarta

2021-07-28 15:09

reporter   ~65688

There were no error messages during the process.
galads

galads

2021-07-28 16:06

administrator   ~65693

I am able to import the 308 Portuguese councils into a dropdown list. I attached a sample question with the 308 entries.
galads

galads

2021-07-28 16:11

administrator   ~65695

The only thing i noticed is that some answer code is not numbered correctly.
samarta

samarta

2021-07-28 18:04

reporter   ~65717

The issue is to tell if (or not) the answer options that you have on your question on LS are the same (and that is no one missing) regarding the ones that you added on Quick Add.

The 308 Portuguese councils you have in your example : have you checked that ALL the councils you added are on the answer options (that's the issue)? That's way I've made the tests with very simple texts that I can easly detect some missing. BTW, the language you used in your survey is french and it must match the base language on my survey so I can import the question. I've made a new survey wtith french as the base language, imported your question with sucess but that does not answer the issue . I don't know the lines that you have "pasted" on your answer options with "quick add"
galads

galads

2021-07-29 08:31

administrator   ~65719

Maybe you should send us a list "simple text", so we can try to re-produce the issue
samarta

samarta

2021-07-29 09:13

reporter   ~65721

My "simple text" was the simpliest possible: 1, 2,3,.......,308
I send you a txt file
1_a_308.txt (1,124 bytes)   
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
1_a_308.txt (1,124 bytes)   
DenisChenu

DenisChenu

2021-07-29 09:19

developer   ~65723

https://manual.limesurvey.org/Troubleshooting#After_clicking_.27Next.27_on_a_page_which_contains_a_big_number_of_answers.2Fsubquestions.2C_not_all_responses_are_being_saved

But with with max_input_vars at 1000 (default)
in 3.X : ~200 answer is OK
5.X : ~200 answer is not OK

OR :
In 3.X : ~200 answer + ~200 subquestion : OK
5.X : 200 answer + 200 subquestion : not OK

Because in 5.X : we send : answers + subquestions + settings + question model
galads

galads

2021-07-29 09:25

administrator   ~65724

Some values are missing and also, some question codes are changed. I have created a task for this. Thank you for reporting
ollehar

ollehar

2021-07-29 11:39

administrator   ~65728

This is not related to PHP post size limit?
ollehar

ollehar

2021-07-29 11:40

administrator   ~65729

Oh, Denis already tested that. Thanks.
ollehar

ollehar

2021-07-29 11:50

administrator   ~65730

The fix should check for input vars and throw an exception if it's too small, right? @DenisChenu
DenisChenu

DenisChenu

2021-07-29 11:54

developer   ~65731

Define « too small» …

Maybe : adding a POST at end of each post request and test if it' there ?

But here : 3X is better than 5.X. Something working in 3.X didn't work with 5.X
ollehar

ollehar

2021-07-29 11:55

administrator   ~65733

You can hash the post on JS, then hash it again in PHP and compare. Not equal == messed up.
DenisChenu

DenisChenu

2021-07-29 11:55

developer   ~65734

PS : it surely a max_input_vars issue, @samarta can you update it to 10000 and test again ?
DenisChenu

DenisChenu

2021-07-29 11:56

developer   ~65735

> You can hash the post on JS, then hash it again in PHP and compare. Not equal == messed up.

How do you send the hash ?

And what happen without JS (public survey need nojs)
ollehar

ollehar

2021-07-29 11:57

administrator   ~65736

You send the hash in the POST or GET. Maybe GET...
DenisChenu

DenisChenu

2021-07-29 12:03

developer   ~65737

Seems clearer to add a simple post at end of the forum in my opinion.

Here : you need to disable submit and do your submit in JS : a lot of error can happen.
ollehar

ollehar

2021-07-29 12:05

administrator   ~65738

> Seems clearer to add a simple post at end of the forum in my opinion.

What do you mean?

> And what happen without JS (public survey need nojs)

I'm only talking about admin right now.
DenisChenu

DenisChenu

2021-07-29 12:16

developer   ~65740

> What do you mean?

HTML :
```
<input type="hidden" name="checkAllPosted" value="1">
</form>
```
ONLY at end of the form

PHP :
```
if(empty(App()->getRequest()->getPost('checkAllPosted')) {
App()->addFllashMessage("All values are not posted : please check max_input_vars");
}
```

More specific :
Seems there are something started in 5.X :
- https://github.com/LimeSurvey/LimeSurvey/blob/13830b848f34192a7d87331b2d246b6bea8e9b01/application/views/questionAdministration/subquestions.twig#L140
- https://github.com/LimeSurvey/LimeSurvey/blob/13830b848f34192a7d87331b2d246b6bea8e9b01/application/views/questionAdministration/answerOptions.twig#L148

But here : in my opinion : need to separate answers and subquestion_X subquestion_Y.
With array : bFullPOST is set in subquestion before set in answers : then tested only for subquestions, not for answers.
ollehar

ollehar

2021-07-29 13:56

administrator   ~65741

> <input type="hidden" name="checkAllPosted" value="1">

OK, that's enough? The POST is always posted in proper order, and it's always the end that gets truncated?
samarta

samarta

2021-07-29 16:39

reporter   ~65746

@DenisChenu
PS : it surely a max_input_vars issue, @samarta can you update it to 10000 and test again ?PS : it surely a max_input_vars issue, @samarta can you update it to 10000 and test again ?

I've updated max_input_vars to 10 000, restarted httpd and still have errors:

I have inserted 308 answer options and 3 have disapeared
DenisChenu

DenisChenu

2021-07-29 16:56

developer   ~65747

> OK, that's enough? The POST is always posted in proper order, and it's always the end that gets truncated?

Unsure, need testing time.

> I've updated max_input_vars to 10 000, restarted httpd and still have errors:

Argl… no idea here.
samarta

samarta

2021-07-29 16:58

reporter   ~65748

I've updated max_input_vars to 100 000, restarted httpd and still have errors:

I have inserted 308 answer options and 2 have disapeared
DenisChenu

DenisChenu

2021-07-29 17:12

developer   ~65749

10000 must work : you check on phpinfo ?
ollehar

ollehar

2021-07-29 17:13

administrator   ~65750

Which two? Which are so special about those? :) Is it the same data as you pasted above?
samarta

samarta

2021-08-19 12:43

reporter   ~66071

Hi,

I apologize for the delay in my reply, just returning from summer holidays.

>Which two? Which are so special about those? :)

Nothing special about those two. The answer options are just a sequence : 1, 2, ...,308

>Is it the same data as you pasted above?
Yes

Done a new test (now on version 5.1.3+210817) just with 308 answers options . The answers options are 1,2,3, 4, ...,308

I've pasted those 308 values and the result is:
answers options 3, 8,149, 187, 189 are missing

answers 40, 35, 265, 203, 288 appears at the place of the missing answer options


You can check this out at:
https://ls.uc.pt/index.php/796285


-- João
ollehar

ollehar

2021-08-19 12:58

administrator   ~66072

Is it possible to add the missing rows manually after? One by one.
samarta

samarta

2021-08-20 10:01

reporter   ~66084

>Is it possible to add the missing rows manually after? One by one.
No, I can't.

Trying to add a missing option (clicking in the "+" - "Insert a new answer otion after this one"), inserted a new answer option and when I clicked in the button "Save" an error message appeared:

code is not set in data: {"oldcode":"AO250"}
samarta

samarta

2021-08-20 10:03

reporter   ~66086

I started a new fresh survey with a single question and tried to insert the 308 answer options and an error ocurred:

An error happened: Could not save answer option Array ( [code] => Array ( [0] => Code is too long (maximum is 5 characters). ) )
ollehar

ollehar

2021-08-20 10:11

administrator   ~66088

OK, maybe we need to prioritize this issue. We've been busy with multiple security issues this week.
DenisChenu

DenisChenu

2021-08-30 12:31

developer   ~66174

samarta : did you get the php error log ?
samarta

samarta

2021-08-30 14:21

reporter   ~66187

DenisChenu: "did you get the php error log ?"

No entries in the /var/log/php-fpm/www-error.log
neither in the
/var/log/httpd/php_errors.log
The value of error_log in Phpinfo () is:
                         Local value Master value
error_log /var/log/php-fpm/www-error.log /var/log/php-fpm/www-error.log
gabrieljenik

gabrieljenik

2021-08-31 15:59

manager   ~66248

PR: https://github.com/LimeSurvey/LimeSurvey/pull/2026

Moved and started to re-use the bFullPOST to show a warning message if some fields are not processed because of max_input options being too small.

The answerOptions.twig and subquestions.twig views had the hidden "bFullPOST" field, but was not being checked.
Database.php used to check it, but seems that mechanism "got lost in translation".

Moved the bFullPost field to create.php following the new screen reorganization (subquestions and answers editable on the main screen)
galads

galads

2021-09-06 09:53

administrator   ~66334

@samarta can you please check if this is solved?
samarta

samarta

2021-09-06 10:23

reporter   ~66336

In version 5.1.5+210830 , no, it's not solved
gabrieljenik

gabrieljenik

2021-09-06 10:56

manager   ~66342

Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=32644
samarta

samarta

2021-09-06 15:03

reporter   ~66344

Hi,
Done some further tests.

Scenario 1:
If I "ADD" (not Replace) the 308 answer options when adding the 308 Answer Options are ALL there . But when I click on "Save" the error persists:
"An error happened: Could not save answer option Array ( [code] => Array ( [0] => Code is too long (maximum is 5 characters). ) )"

Scenario 2:

If I choose "Replace" (instead of "Add") the 308 answer options when adding the 308 Answer Options are ALL there and when I click on "Save" it works fine and the 308 Answer Options are ALL there. Everything is correct.



I am sending a video with scenario 1


-- João
galads

galads

2021-09-06 15:31

administrator   ~66348

I tested with the PR from @Gabriel and it seems to be fixed. We have pushed the fix to the master branch.
samarta

samarta

2021-09-06 15:40

reporter   ~66349

Hi,
As you can see in the video I've sent, the version I've used is today's version 5.1.6 - 210906 and error in Scenario 1 ("ADD") persists.


-- João
gabrieljenik

gabrieljenik

2021-09-07 23:58

manager   ~66386

Hi All,

I think this new issue is not related to the fix, but it is just a separate existing issue.
Still, will fix it of course.

Thanks!
galads

galads

2021-09-14 08:50

administrator   ~66475

I created a new bug report for the new issue and assigned it to @Gabriel : https://bugs.limesurvey.org/view.php?id=17593

I will close this now.

Related Changesets

LimeSurvey: master ed6dd58f

2021-09-06 10:56:13

gabrieljenik


Committer: GitHub Details Diff
Fixed issue 17481: Error adding a large number of answer options (#2026) Affected Issues
17481
mod - application/controllers/QuestionAdministrationController.php Diff File
mod - application/views/questionAdministration/answerOptions.twig Diff File
mod - application/views/questionAdministration/create.php Diff File
mod - application/views/questionAdministration/subquestions.twig Diff File

Issue History

Date Modified Username Field Change
2021-07-28 10:02 samarta New Issue
2021-07-28 13:41 ollehar Note Added: 65674
2021-07-28 13:45 ollehar Sync to Zoho Project => |Yes|
2021-07-28 14:01 galads Assigned To => galads
2021-07-28 14:01 galads Status new => testing
2021-07-28 15:07 samarta Note Added: 65686
2021-07-28 15:09 samarta Note Added: 65688
2021-07-28 16:06 galads Note Added: 65693
2021-07-28 16:06 galads File Added: limesurvey_question_21.lsq
2021-07-28 16:11 galads Note Added: 65695
2021-07-28 18:04 samarta Note Added: 65717
2021-07-29 08:31 galads Note Added: 65719
2021-07-29 08:31 galads Status testing => feedback
2021-07-29 09:13 samarta Note Added: 65721
2021-07-29 09:13 samarta File Added: 1_a_308.txt
2021-07-29 09:13 samarta Status feedback => assigned
2021-07-29 09:19 DenisChenu Note Added: 65723
2021-07-29 09:21 galads Status assigned => acknowledged
2021-07-29 09:25 galads Note Added: 65724
2021-07-29 11:33 ollehar Priority none => high
2021-07-29 11:33 ollehar Sync to Zoho Project Yes => |Yes|
2021-07-29 11:39 ollehar Note Added: 65728
2021-07-29 11:40 ollehar Note Added: 65729
2021-07-29 11:50 ollehar Note Added: 65730
2021-07-29 11:54 DenisChenu Note Added: 65731
2021-07-29 11:55 ollehar Note Added: 65733
2021-07-29 11:55 DenisChenu Note Added: 65734
2021-07-29 11:56 DenisChenu Note Added: 65735
2021-07-29 11:57 ollehar Note Added: 65736
2021-07-29 12:03 DenisChenu Note Added: 65737
2021-07-29 12:05 ollehar Note Added: 65738
2021-07-29 12:16 DenisChenu Note Added: 65740
2021-07-29 13:56 ollehar Note Added: 65741
2021-07-29 14:13 ollehar Summary Error adding a large number of answer options => Error adding a large number of answer options (max_input_vars needs to be higher in LS5)
2021-07-29 14:13 ollehar Sync to Zoho Project Yes => |Yes|
2021-07-29 16:39 samarta Note Added: 65746
2021-07-29 16:56 DenisChenu Note Added: 65747
2021-07-29 16:58 samarta Note Added: 65748
2021-07-29 17:12 DenisChenu Note Added: 65749
2021-07-29 17:13 ollehar Note Added: 65750
2021-08-13 15:12 galads Status acknowledged => confirmed
2021-08-19 12:43 samarta Note Added: 66071
2021-08-19 12:58 ollehar Note Added: 66072
2021-08-20 10:01 samarta Note Added: 66084
2021-08-20 10:03 samarta Note Added: 66086
2021-08-20 10:11 ollehar Note Added: 66088
2021-08-30 11:48 galads Assigned To galads => gabrieljenik
2021-08-30 11:48 galads Sync to Zoho Project Yes => |Yes|
2021-08-30 12:31 DenisChenu Note Added: 66174
2021-08-30 14:21 samarta Note Added: 66187
2021-08-31 15:59 gabrieljenik Status confirmed => testing
2021-08-31 15:59 gabrieljenik Note Added: 66248
2021-08-31 15:59 gabrieljenik Bug heat 8 => 10
2021-09-06 09:53 galads Note Added: 66334
2021-09-06 10:23 samarta Note Added: 66336
2021-09-06 10:56 gabrieljenik Changeset attached => LimeSurvey master ed6dd58f
2021-09-06 10:56 gabrieljenik Note Added: 66342
2021-09-06 10:56 gabrieljenik Resolution open => fixed
2021-09-06 10:57 c_schmitz Status testing => resolved
2021-09-06 10:59 c_schmitz Status resolved => closed
2021-09-06 15:03 samarta Note Added: 66344
2021-09-06 15:03 samarta File Added: 308answeroptions_scenario_01.mp4
2021-09-06 15:31 galads Note Added: 66348
2021-09-06 15:40 samarta Note Added: 66349
2021-09-06 15:57 galads Status closed => confirmed
2021-09-07 23:58 gabrieljenik Note Added: 66386
2021-09-14 08:50 galads Note Added: 66475
2021-09-14 08:50 galads Status confirmed => resolved