LimeSurvey issue tracker
Registration

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
05068Bug reports[All Projects] Conditionspublic2011-03-31 17:082012-01-20 13:08
Reporterc_schmitz 
Assigned ToTMSWhite 
PrioritynormalSeveritymajor 
StatusclosedResolutionfixed 
Product Version1.91RC5 
Target VersionFixed in Version1.92RC2 
Summary05068: Performance issue with long surveys and conditions
DescriptionWhen using conditions on a long survey and in the later of the course conditions are used then ALL previous values are embedded in the question page (in group by group mode) where any conditions are used. Naturally with a huge result set the page becomes pretty big AND slow.

Could there be a 'smart' embedding so that only values are embedded in the page that are really used by the condition on that page?

I will donate to the project if issue is resolved within 48 hrsNo
LimeSurvey build number OR git commit ID1
Browser.
Database & DB-Version.
Operating System (Server).
Webserver software & version.
PHP Version.
Attached Files

- Relationships
related to 04787closedmdekker Bug in getArrayFilterExcludesCascadesForGroup() function causing querying for too many questions 
has duplicate 05346closedc_schmitz Firefox & IE Freezes 

-  Notes
User avatar (14647)
mdekker (developer)
2011-04-04 12:49

retrieveConditionInfo in qanda.php does a db query for every field, which could make things slow. Better would be to do 1 query, save the resulting array and filter out the qid you need by looping. This way a lot of db queries van be saved improving the speed.

To identify all the queries executed, edit the mysql to activate query log. This would be the best start I think as just inserting all fields should not be much of a problem.
User avatar (14649)
c_schmitz (administrator)
2011-04-04 12:51

It definately is a problem if there are 1000+ hidden fields in every page loaded.
Besides that it is a unnecessary waste of ressources.
User avatar (14670)
mdekker (developer)
2011-04-05 09:08

I would expect that to be a problem for javascript execution, looping over all input fields and page rendering in the browser, not really on the server side. Aren't all fields in the session anyway? If we could get our hands on the problem survey we could do some profiling to see where the problems are. Although I do agree resources should never be wasted.

I think long surveys and conditions have always been an issue and we did various things to improve speed in the past. Maybe we can use a survey from an old topic to check. I remember one that had many pages of arrays with filters that got slower and slower while you got more towards the end of the survey.

One recent thing added is using tokens in condition. If the problem survey doesn't use that, but does use tokens, it could be a nice start to rule out the newly added token feature by disabling tokens and seeing what performance does then.
User avatar (14673)
mdekker (developer)
2011-04-05 13:35

I did a fix to the array filter which caused a lot of unneeded repetition. Commited the fix in 9964 for bug 04787. Please check if this is still an issue.
User avatar (16258)
c_schmitz (administrator)
2011-09-11 10:03

Thomas, is this still an issue with the expresison engine?
User avatar (16266)
TMSWhite (reporter)
2011-09-11 14:58

Carsten-

This is still somewhat of an issue. I was planning to do smart embedding, but have not gotten there yet. Right now, all substitutions are written to the page both from PHP (so that the starting values are correct), and as JavaScript-based substitutions. This is instantaneous for Firefox and Chrome, but causes a second delay per navigation event in Internet Explorer. So, the plan is to only create JavaScript substitutions for items that can actually be dynamically changed on the page.

As to the database query issue, my hope is to see CreateFieldMap() re-designed to do a single DB query to popuplate everything $fieldmap('full') needs and store it in $_SESSION, including the answers.

EM will prevent the need for the long queries for getting Conditions, once we remove Conditions (or as long as people don't use them).

Lastly, for performance, we might want to add Relevance as a field in the Questions table so that I don't have to retrieve it from QuestionAttributes, but that isn't currently a problem, since we almost always have to get QuestionAttributes anyway, so I just create some relevance-related $_SESSION[] variables.
User avatar (16783)
TMSWhite (reporter)
2012-01-10 22:02

This is no longer an issue in 1.92.
Conditions are no longer read at run-time, so that slow-down is eliminated.
Although qanda does query the DB for every question (something that will be fixed in a plus release or 2.x), Conditions were the biggest source of slow-down.
User avatar (16841)
c_schmitz (administrator)
2012-01-20 13:08

1.92RC2 released.

- Issue History
Date Modified Username Field Change
2011-03-31 17:08 c_schmitz New Issue
2011-03-31 17:08 c_schmitz Status new => assigned
2011-03-31 17:08 c_schmitz Assigned To => lemeur
2011-04-04 12:49 mdekker Note Added: 14647
2011-04-04 12:51 c_schmitz Note Added: 14649
2011-04-05 09:08 mdekker Note Added: 14670
2011-04-05 09:11 mdekker Relationship added related to 04787
2011-04-05 13:35 mdekker Note Added: 14673
2011-07-21 17:59 c_schmitz Relationship added has duplicate 05346
2011-09-11 10:03 c_schmitz Assigned To lemeur => TMSWhite
2011-09-11 10:03 c_schmitz Note Added: 16258
2011-09-11 10:09 c_schmitz Status assigned => feedback
2011-09-11 14:58 TMSWhite Note Added: 16266
2012-01-10 22:02 TMSWhite Note Added: 16783
2012-01-10 22:02 TMSWhite Status feedback => resolved
2012-01-10 22:02 TMSWhite Fixed in Version => 1.92RC1
2012-01-10 22:02 TMSWhite Resolution open => fixed
2012-01-20 13:07 c_schmitz Fixed in Version 1.92RC1 => 1.92RC2
2012-01-20 13:08 c_schmitz Note Added: 16841
2012-01-20 13:08 c_schmitz Status resolved => closed


Copyright © 2000 - 2014 MantisBT Team
Powered by Mantis Bugtracker