View Issue Details

IDProjectCategoryView StatusLast Update
05068Bug reports[All Projects] Conditionspublic2012-01-20 13:08
Reporterc_schmitzAssigned ToTMSWhite 
Status closedResolutionfixed 
Product Version1.91RC5 
Target VersionFixed in Version1.92RC2 
Summary05068: Performance issue with long surveys and conditions
DescriptionWhen using conditions on a long survey and in the later of the course conditions are used then ALL previous values are embedded in the question page (in group by group mode) where any conditions are used. Naturally with a huge result set the page becomes pretty big AND slow.

Could there be a 'smart' embedding so that only values are embedded in the page that are really used by the condition on that page?

TagsNo tags attached.
Complete LimeSurvey version number (& build)1
I will donate to the project if issue is resolvedNo
Database & DB-Version.
Operating System (Server).
Webserver software & version.
PHP Version.


related to 04787 closedmdekker Bug in getArrayFilterExcludesCascadesForGroup() function causing querying for too many questions 
has duplicate 05346 closedc_schmitz Firefox & IE Freezes 




2011-04-04 12:49

developer   ~14647

retrieveConditionInfo in qanda.php does a db query for every field, which could make things slow. Better would be to do 1 query, save the resulting array and filter out the qid you need by looping. This way a lot of db queries van be saved improving the speed.

To identify all the queries executed, edit the mysql to activate query log. This would be the best start I think as just inserting all fields should not be much of a problem.


2011-04-04 12:51

administrator   ~14649

It definately is a problem if there are 1000+ hidden fields in every page loaded.
Besides that it is a unnecessary waste of ressources.


2011-04-05 09:08

developer   ~14670

I would expect that to be a problem for javascript execution, looping over all input fields and page rendering in the browser, not really on the server side. Aren't all fields in the session anyway? If we could get our hands on the problem survey we could do some profiling to see where the problems are. Although I do agree resources should never be wasted.

I think long surveys and conditions have always been an issue and we did various things to improve speed in the past. Maybe we can use a survey from an old topic to check. I remember one that had many pages of arrays with filters that got slower and slower while you got more towards the end of the survey.

One recent thing added is using tokens in condition. If the problem survey doesn't use that, but does use tokens, it could be a nice start to rule out the newly added token feature by disabling tokens and seeing what performance does then.


2011-04-05 13:35

developer   ~14673

I did a fix to the array filter which caused a lot of unneeded repetition. Commited the fix in 9964 for bug 04787. Please check if this is still an issue.


2011-09-11 10:03

administrator   ~16258

Thomas, is this still an issue with the expresison engine?


2011-09-11 14:58

reporter   ~16266


This is still somewhat of an issue. I was planning to do smart embedding, but have not gotten there yet. Right now, all substitutions are written to the page both from PHP (so that the starting values are correct), and as JavaScript-based substitutions. This is instantaneous for Firefox and Chrome, but causes a second delay per navigation event in Internet Explorer. So, the plan is to only create JavaScript substitutions for items that can actually be dynamically changed on the page.

As to the database query issue, my hope is to see CreateFieldMap() re-designed to do a single DB query to popuplate everything $fieldmap('full') needs and store it in $_SESSION, including the answers.

EM will prevent the need for the long queries for getting Conditions, once we remove Conditions (or as long as people don't use them).

Lastly, for performance, we might want to add Relevance as a field in the Questions table so that I don't have to retrieve it from QuestionAttributes, but that isn't currently a problem, since we almost always have to get QuestionAttributes anyway, so I just create some relevance-related $_SESSION[] variables.


2012-01-10 22:02

reporter   ~16783

This is no longer an issue in 1.92.
Conditions are no longer read at run-time, so that slow-down is eliminated.
Although qanda does query the DB for every question (something that will be fixed in a plus release or 2.x), Conditions were the biggest source of slow-down.


2012-01-20 13:08

administrator   ~16841

1.92RC2 released.

Issue History

Date Modified Username Field Change
2011-03-31 17:08 c_schmitz New Issue
2011-03-31 17:08 c_schmitz Status new => assigned
2011-03-31 17:08 c_schmitz Assigned To => lemeur
2011-04-04 12:49 mdekker Note Added: 14647
2011-04-04 12:51 c_schmitz Note Added: 14649
2011-04-05 09:08 mdekker Note Added: 14670
2011-04-05 09:11 mdekker Relationship added related to 04787
2011-04-05 13:35 mdekker Note Added: 14673
2011-07-21 17:59 c_schmitz Relationship added has duplicate 05346
2011-09-11 10:03 c_schmitz Assigned To lemeur => TMSWhite
2011-09-11 10:03 c_schmitz Note Added: 16258
2011-09-11 10:09 c_schmitz Status assigned => feedback
2011-09-11 14:58 TMSWhite Note Added: 16266
2012-01-10 22:02 TMSWhite Note Added: 16783
2012-01-10 22:02 TMSWhite Status feedback => resolved
2012-01-10 22:02 TMSWhite Fixed in Version => 1.92RC1
2012-01-10 22:02 TMSWhite Resolution open => fixed
2012-01-20 13:07 c_schmitz Fixed in Version 1.92RC1 => 1.92RC2
2012-01-20 13:08 c_schmitz Note Added: 16841
2012-01-20 13:08 c_schmitz Status resolved => closed