View Issue Details

IDProjectCategoryView StatusLast Update
14965Bug reportsSurvey participants (Tokens)public2020-08-07 06:00
Reporterblocka Assigned Toollehar  
PrioritynormalSeverityminor 
Status acknowledgedResolutionopen 
Product Version3.17.x 
Summary14965: Group randomization issue
Description

I have a survey that has a set of question groups in the same randomization group.

Token persistence is enabled, and when a participant clicks on their email invitation and starts the survey, no problems of course.

But, if the participant doesn't complete the survey, and then returns at a later time, the survey loads with a blank question group.

As per the forum post:
https://www.limesurvey.org/forum/design-issues/116150-group-randomization-issue?start=0

Group randomization (at the group level) has always worked with no tokens for me. Its just when you mix in response persistence, a new session and a partially answered survey where they stopped in the middle of the randomized groups. It re-randomized the groups putting an unanswered mandatory question before the last question answered, then tried to send you back to the last question answered which resulted in the a group loading with error classes and hidden questions, unable to proceed forwards.

TagsNo tags attached.
Complete LimeSurvey version number (& build)3.17.4 Build 190529
I will donate to the project if issue is resolvedNo
BrowserAll
Database & DB-Version359
Server OS (if known)
Webserver software & version (if known)
PHP Version7.2

Activities

DenisChenu

DenisChenu

2019-06-06 09:07

developer   ~52313

@ollehar : related to SEED ?

ollehar

ollehar

2019-06-06 10:14

administrator   ~52314

Will have a look today.

ollehar

ollehar

2019-06-06 10:50

administrator  

ollehar

ollehar

2019-06-06 11:04

administrator   ~52316

I need clearer instructions on how to reproduce. This is what I did:

  1. Import the attached survey
  2. Activate it
  3. Create token table with participant
  4. Run survey with token (from the admin interface)
  5. Write answers, close tab in middle of randomized group
  6. Run survey again (tried both with and without newtest=Y in URL)

Can't see any different in group randomization. Also tried with clicking "Resume later".

ollehar

ollehar

2019-06-06 11:04

administrator   ~52317

Using 3.17.5+190604.

Bigred01

Bigred01

2019-07-11 00:39

reporter   ~52821

I was just able to replicate on the first attempt on the https://demo.limesurvey.org site which looks to be running Version 3.17.3. This issue has existed for as long as I can remember and no one seems to know about it so I think we can safely assume it still exists in 3.17.5.

https://demo.limesurvey.org/index.php?r=admin/tokens/sa/browse/surveyid/959263

  1. Token based response persistence must be on.
  2. Enter test survey without newtest=Y in url in a private window. Populate survey until halfway through randomized groups.
  3. Re-enter the survey with the same token in a new private window.

See attached screenshot for what you will see when experiencing the bug.

If you don't see it on the first time, just immediately close the window and re-enter in a new private window. Sooner or later you will see it.

Now chances are, no one will be entering the survey like above. But its no different than starting a survey on your phone then completing on your desktop later. Its just the easiest way to replicate.

groupRandBug.JPG (71,295 bytes)   
groupRandBug.JPG (71,295 bytes)   
groupRandBugQuestionClasses.JPG (76,662 bytes)   
groupRandBugQuestionClasses.JPG (76,662 bytes)   
DenisChenu

DenisChenu

2019-07-11 07:46

developer   ~52823

@Bigred01 : you can use demo for checking, but it reset all night.
We have a 404: Not Found not foun n,ow for this survey.

Can you send the related lss please ?

Bigred01

Bigred01

2019-07-11 07:54

reporter   ~52824

I used the one posted above by ollehar. I believe it’s the original one I posted to the forums 10 months ago.

DenisChenu

DenisChenu

2019-07-11 08:04

developer   ~52825

OK :) Great :)

gabrieljenik

gabrieljenik

2020-08-06 17:51

developer   ~59349

Shuodldn't be using newtest=Y all time?
I think it is mandatory for cleaning old sessions which could have been loaded before.
I know it doesn't match a lot with the scenario because of being private windows, but still, making some initial cleanup never hurts.
Actually, I always try to ad it to the survey urls and I think they are.

Bigred01

Bigred01

2020-08-06 17:59

reporter   ~59352

The issue is more than that though. When you come in with a clean session after already answering some questions previously, the groups would be re-randomized and it might put an unanswered mandatory group/question before the group it starts you at. It then renders a page with all questions hidden due to relevance error classes and you cannot proceed. My quick hack was to store the group order in a db table using the sid & token as the primary key and use that on entry instead of setting a random order again. I cant remember where in the code i did that now, that was a while ago. That paired with newtest=Y appeared to work but its not a real solution, just a quick hack to get a survey out.

DenisChenu

DenisChenu

2020-08-06 18:51

developer   ~59353

@ollehar : seed must be used to have same radomization for same token ? No ?

gabrieljenik

gabrieljenik

2020-08-06 22:00

developer   ~59356

I wouldn't rely on the seed to get reproducible numbers.

Besides what it says here (https://www.php.net/manual/en/function.srand.php#16513) you can;t controll that every time that you rturn, the same number of random numbers will be retrieved. You "may" guarantee the sequence is the same. But can you guarantee that there will be no extra random number request in between the sequence?

One of the problems should have got solved by using the newTest parameter on the url.

Having a constant randomness is another issue and is because random sequences are not saved.
Although it is not the expected behaviour on this case, I am not sure it is a bug (sorry, don't want to be rude) but more a design issue that should be tackled.

I will leave @cdorin review what is the suggested next steps.

Thanks!

Bigred01

Bigred01

2020-08-06 23:13

reporter   ~59362

It does prevent people from completing the survey though. How i found it in the first place was when we started to receive a series of complaint emails. Group randomization is essentially useless when you have token based response persistence on and someone doesn't complete the survey in a single sitting. There is a very good chance the next time they enter they will encounter this.

gabrieljenik

gabrieljenik

2020-08-07 06:00

developer   ~59364

I understand you mean.

Issue History

Date Modified Username Field Change
2019-06-05 20:22 blocka New Issue
2019-06-06 09:07 DenisChenu Note Added: 52313
2019-06-06 10:14 ollehar Note Added: 52314
2019-06-06 10:14 ollehar Assigned To => ollehar
2019-06-06 10:14 ollehar Status new => assigned
2019-06-06 10:50 ollehar File Added: limesurvey_survey_959263.lss
2019-06-06 11:04 ollehar Status assigned => feedback
2019-06-06 11:04 ollehar Note Added: 52316
2019-06-06 11:04 ollehar Note Added: 52317
2019-07-11 00:39 Bigred01 File Added: groupRandBug.JPG
2019-07-11 00:39 Bigred01 File Added: groupRandBugQuestionClasses.JPG
2019-07-11 00:39 Bigred01 Note Added: 52821
2019-07-11 07:46 DenisChenu Note Added: 52823
2019-07-11 07:54 Bigred01 Note Added: 52824
2019-07-11 08:04 DenisChenu Note Added: 52825
2019-07-11 10:29 ollehar Status feedback => assigned
2020-08-05 17:24 cdorin Priority none => normal
2020-08-05 17:24 cdorin Status assigned => acknowledged
2020-08-06 17:51 gabrieljenik Note Added: 59349
2020-08-06 17:59 Bigred01 Note Added: 59352
2020-08-06 18:51 DenisChenu Note Added: 59353
2020-08-06 22:00 gabrieljenik Note Added: 59356
2020-08-06 23:13 Bigred01 Note Added: 59362
2020-08-07 06:00 gabrieljenik Note Added: 59364