06299: LimeSurvey 2 is much slower than 1.19 - LimeSurvey bugs, patches and feature requests

(19597)
Mazi (developer)
2012-07-03 15:56

This is the third or fourth such report I noticed so we should have a look at this.
I have attached a report of my local version which loads 750KB of data in about 6 seconds for the root page /Limesurvey/index.php/admin/survey/index

(19601)
TMSWhite (developer)
2012-07-03 18:57

coiby-

If you want to insert some timing code yourself, you can use the code in LimeSurveyExpressionManager as a model. See the FinishProcessingPage() function, which processes LEM->runtimeTimings[] that have been collected.

I found this more helpful than using a PHP profiler, since this let me focus on coarse-grained tasks rather than the fine-grained details that are shown by a profiler.

/Tom

(19610)
user20690
2012-07-04 09:43

Hi Tom,

I'm glad to show runtimeTImings to help you. But how can I enable the output?

(19612)
DenisChenu (developer)
2012-07-04 10:20

It's the admin page no ?

Not the public page, then not EM specific.

Here there are a lot of CSS for display participant, and clearly "js table" are longer the first time.

Denis

(19614)
TMSWhite (developer)
2012-07-04 14:54

coiby-

I meant that someone could follow the example used by LEM-runtimeTimings to calculate timings of other parts of LS to identify the source of the slowdown. EM is effectively identical between 1.92 and 2.0, so I agree that it is unlikely to be the cause.

(19634)
c_schmitz (administrator)
2012-07-06 10:00

The screenshot are pretty much meaningless. They are taken with two different tools on two different browsers.

Menno: Would you be interested to have a look at this in general? In general I doubt that 2.0 is 4 times slower as indicated.

(19643)
user20690
2012-07-06 11:32

Tom: Sorry, I'm not familiar with this and don't have time to follow the example.

c_schmitz: what do you mean by "two different browsers". They are the same! I don't know how much slower 2.0 is than 1.92 exactly. But it's much slower indeed.

(19646)
DenisChenu (developer)
2012-07-06 12:00

Hello,

With the same page, second call (for cache)
http://limesurvey-yii/index.php/admin/user/index [^] : 2.37s (onload: 2.68s),
http://limesurvey/admin/admin.php?action=editusers [^] : 2.5s (onload: 2.95s)

The PHP file waiting:
Yii: 510ms waiting
192 : 210 ms waiting

With Yii : i use blobbluish admin template (Same as 192)

(19682)
mdekker (developer)
2012-07-09 11:38
edited on: 2012-07-09 11:39

Not able to reproduce slow loading a fresh Xampp for windows installation and on my server. If it really is slow, it must be some kind of server configuration.

I do get 2 404 errors on the downloaded RC4:
GET http://localhost/yii/limesurvey/scripts/jquery/css/jquery.multiselect.css [^] 404 (CHttpException) index:14
GET http://localhost/yii/limesurvey/scripts/jquery/css/jquery.multiselect.filter.css [^] 404 (CHttpException) index:15

Maybe that causes a slow redirect and the delay. Maybe bug owner can verify that these two files are the problem by moving them from /styles-public to /scripts/jquery/css

(19683)
user20690
2012-07-09 11:56

Hi,

I've moved them from /styles-public to /scripts/jquery/css. But the problem is the same.

(19684)
mdekker (developer)
2012-07-09 13:39

Does your server have a public url and if so, could you provide links and login to the admin part? Mark your response as private so only developers can see your response. As none of the devs seems to be able to reproduce, this is the only way we can get some clue on what is wrong.

(20029)
c_schmitz (administrator)
2012-07-27 14:03

The URL does not seem to work?

(20424)
mdekker (developer)
2012-08-14 13:59

http://www.ipv6proxy.net/ [^] helps a little, but too difficult to see use. To rule out ipv6, could you try on a different location using ipv4?

(20606)
TMSWhite (developer)
2012-08-31 13:23

The survey attached to bug #06534 is a good example. When I test the survey, it loads in 3-4 seconds for LimeSurvey 1.92+, and 22-25 seconds for 2.0RC9.

I highly recommend tracking down the performance issues before saying that 2.0 is stable, since more than about 5 seconds for load time is, in my mind, unacceptable.

(20619)
mdekker (developer)
2012-09-02 16:44
edited on: 2012-09-02 16:45

The survey from http://bugs.limesurvey.org/view.php?id=6534 [^] uses randomization and executes an insane amount of db queries, when randomization is reset it works at normal speed again. (still showing a lot of duplicate queries though)

for mysql this statement i used to reset the random attributes:
update `lsyii__question_attributes` set value = '' WHERE `attribute` LIKE '%rand%' ORDER BY `lsyii__question_attributes`.`value` ASC

Don't really have the time to dive into it but who wants to take this bug should look into randomization.

(20635)
Mazi (developer)
2012-09-06 15:26

20-25 second is really quite long...

We should either extend the loading time so it takes long enough to get a coffee (and display an according message) or try to fix it.

Carsten will probably be very bored when coming back from holiday so I'll assign this one to him :-)

(22004)
seanrife (developer)
2012-11-08 05:33

I'm experiencing the same problem. I just uploaded a couple of surveys that have been extremely slow (sometimes timing out while as few as 10 people are taking it). Also comes up when viewing responses (page will sometimes take up to a minute to load). This is now on a VPS with 4GB memory.

Two other notes: (1) running MySQL queries is very snappy. (2) I'm almost certain this has to do with memory (CPU load has never exceeded .14), as I've seen the server use up to 300MB just pulling up results (a single user). I'm no expert, but this certainly seems excessive. Or am I missing something?

(22110)
mdekker (developer)
2012-11-13 12:25

Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=10263 [^]

(22265)
mdekker (developer)
2012-11-16 12:32

@seanrife: Some improvements have been done to make surveys with randomization faster. Also a bottleneck with checking for updates has been fixed. Could you please test latest release (121116) and report if it helped to get faster response times so we can close this issue?

(22291)
seanrife (developer)
2012-11-16 19:47

Thanks mdekker... I just upgraded, and things seem to be working well. I definitely notice a significant difference. I also (about a week ago) switched to php 5.2.17 and enabled XCache, which seems to have helped too.

All in all, I think we're good to go here. Thanks for all your hard work!

Related Changesets
LimeSurvey: master a5efb1ea Timestamp: 2012-11-13 10:50:54 Author: mdekker Committer: Menno Dekker [ Details ] [ Diff ]	Fixed 06299: Slow on randomization groups dev: first bottleneck eliminated dev: also added a little cache to survey like on 2.1 to help save some db calls
	mod - application/helpers/frontend_helper.php	[ Diff ] [ File ]
	mod - application/models/Survey.php	[ Diff ] [ File ]

Issue History
Date Modified	Username	Field	Change
2012-07-03 14:41	user20690	New Issue
2012-07-03 14:41	user20690	File Added: screenshot1.19VS2.zip
2012-07-03 15:47	Mazi	Assigned To	=> c_schmitz
2012-07-03 15:47	Mazi	Status	new => assigned
2012-07-03 15:56	Mazi	Note Added: 19597
2012-07-03 15:56	Mazi	File Added: ls2_loading_times.PNG
2012-07-03 18:57	TMSWhite	Note Added: 19601
2012-07-04 09:43	user20690	Note Added: 19610
2012-07-04 10:20	DenisChenu	Note Added: 19612
2012-07-04 14:54	TMSWhite	Note Added: 19614
2012-07-06 09:59	c_schmitz	Assigned To	c_schmitz => mdekker
2012-07-06 10:00	c_schmitz	Note Added: 19634
2012-07-06 11:32	user20690	Note Added: 19643
2012-07-06 12:00	DenisChenu	Note Added: 19646
2012-07-09 11:38	mdekker	Note Added: 19682
2012-07-09 11:39	mdekker	Note Edited: 19682	View Revisions
2012-07-09 11:56	user20690	Note Added: 19683
2012-07-09 13:39	mdekker	Note Added: 19684
2012-07-27 14:03	c_schmitz	Note Added: 20029
2012-07-27 14:03	c_schmitz	Status	assigned => feedback
2012-08-14 13:59	mdekker	Note Added: 20424
2012-08-31 13:19	TMSWhite	Relationship added	related to 06534
2012-08-31 13:23	TMSWhite	Note Added: 20606
2012-09-02 16:44	mdekker	Note Added: 20619
2012-09-02 16:45	mdekker	Note Edited: 20619	View Revisions
2012-09-02 16:53	mdekker	Assigned To	mdekker =>
2012-09-06 15:26	Mazi	Note Added: 20635
2012-09-06 15:26	Mazi	Assigned To	=> c_schmitz
2012-09-06 15:26	Mazi	Status	feedback => assigned
2012-10-22 14:32	c_schmitz	Assigned To	c_schmitz =>
2012-10-22 14:32	c_schmitz	Status	assigned => new
2012-11-04 21:04	c_schmitz	Assigned To	=> c_schmitz
2012-11-04 21:04	c_schmitz	Status	new => assigned
2012-11-08 04:56	seanrife	File Added: limesurvey_survey_917871.lss
2012-11-08 04:58	seanrife	File Added: limesurvey_survey_114635.lss
2012-11-08 05:03	seanrife	File Added: phpinfo.pdf
2012-11-08 05:33	seanrife	Note Added: 22004
2012-11-13 10:12	mdekker	Assigned To	c_schmitz => mdekker
2012-11-13 12:25	mdekker	Changeset attached	=> LimeSurvey master a5efb1ea
2012-11-13 12:25	mdekker	Note Added: 22110
2012-11-13 12:25	mdekker	Resolution	open => fixed
2012-11-16 12:32	mdekker	Note Added: 22265
2012-11-16 19:47	seanrife	Note Added: 22291
2012-11-16 22:02	c_schmitz	Status	assigned => closed
2012-11-16 22:02	c_schmitz	Fixed in Version	=> 2.00+

Notes
(19597) Mazi (developer) 2012-07-03 15:56	This is the third or fourth such report I noticed so we should have a look at this. I have attached a report of my local version which loads 750KB of data in about 6 seconds for the root page /Limesurvey/index.php/admin/survey/index

(19601) TMSWhite (developer) 2012-07-03 18:57	coiby- If you want to insert some timing code yourself, you can use the code in LimeSurveyExpressionManager as a model. See the FinishProcessingPage() function, which processes LEM->runtimeTimings[] that have been collected. I found this more helpful than using a PHP profiler, since this let me focus on coarse-grained tasks rather than the fine-grained details that are shown by a profiler. /Tom

(19610) user20690 2012-07-04 09:43	Hi Tom, I'm glad to show runtimeTImings to help you. But how can I enable the output?

(19612) DenisChenu (developer) 2012-07-04 10:20	It's the admin page no ? Not the public page, then not EM specific. Here there are a lot of CSS for display participant, and clearly "js table" are longer the first time. Denis

(19614) TMSWhite (developer) 2012-07-04 14:54	coiby- I meant that someone could follow the example used by LEM-runtimeTimings to calculate timings of other parts of LS to identify the source of the slowdown. EM is effectively identical between 1.92 and 2.0, so I agree that it is unlikely to be the cause.

(19634) c_schmitz (administrator) 2012-07-06 10:00	The screenshot are pretty much meaningless. They are taken with two different tools on two different browsers. Menno: Would you be interested to have a look at this in general? In general I doubt that 2.0 is 4 times slower as indicated.

(19643) user20690 2012-07-06 11:32	Tom: Sorry, I'm not familiar with this and don't have time to follow the example. c_schmitz: what do you mean by "two different browsers". They are the same! I don't know how much slower 2.0 is than 1.92 exactly. But it's much slower indeed.

(19646) DenisChenu (developer) 2012-07-06 12:00	Hello, With the same page, second call (for cache) http://limesurvey-yii/index.php/admin/user/index [^] : 2.37s (onload: 2.68s), http://limesurvey/admin/admin.php?action=editusers [^] : 2.5s (onload: 2.95s) The PHP file waiting: Yii: 510ms waiting 192 : 210 ms waiting With Yii : i use blobbluish admin template (Same as 192)

(19682) mdekker (developer) 2012-07-09 11:38 edited on: 2012-07-09 11:39	Not able to reproduce slow loading a fresh Xampp for windows installation and on my server. If it really is slow, it must be some kind of server configuration. I do get 2 404 errors on the downloaded RC4: GET http://localhost/yii/limesurvey/scripts/jquery/css/jquery.multiselect.css [^] 404 (CHttpException) index:14 GET http://localhost/yii/limesurvey/scripts/jquery/css/jquery.multiselect.filter.css [^] 404 (CHttpException) index:15 Maybe that causes a slow redirect and the delay. Maybe bug owner can verify that these two files are the problem by moving them from /styles-public to /scripts/jquery/css

(19683) user20690 2012-07-09 11:56	Hi, I've moved them from /styles-public to /scripts/jquery/css. But the problem is the same.

(19684) mdekker (developer) 2012-07-09 13:39	Does your server have a public url and if so, could you provide links and login to the admin part? Mark your response as private so only developers can see your response. As none of the devs seems to be able to reproduce, this is the only way we can get some clue on what is wrong.

(20029) c_schmitz (administrator) 2012-07-27 14:03	The URL does not seem to work?

(20424) mdekker (developer) 2012-08-14 13:59	http://www.ipv6proxy.net/ [^] helps a little, but too difficult to see use. To rule out ipv6, could you try on a different location using ipv4?

(20606) TMSWhite (developer) 2012-08-31 13:23	The survey attached to bug #06534 is a good example. When I test the survey, it loads in 3-4 seconds for LimeSurvey 1.92+, and 22-25 seconds for 2.0RC9. I highly recommend tracking down the performance issues before saying that 2.0 is stable, since more than about 5 seconds for load time is, in my mind, unacceptable.

(20619) mdekker (developer) 2012-09-02 16:44 edited on: 2012-09-02 16:45	The survey from http://bugs.limesurvey.org/view.php?id=6534 [^] uses randomization and executes an insane amount of db queries, when randomization is reset it works at normal speed again. (still showing a lot of duplicate queries though) for mysql this statement i used to reset the random attributes: update `lsyii__question_attributes` set value = '' WHERE `attribute` LIKE '%rand%' ORDER BY `lsyii__question_attributes`.`value` ASC Don't really have the time to dive into it but who wants to take this bug should look into randomization.

(20635) Mazi (developer) 2012-09-06 15:26	20-25 second is really quite long... We should either extend the loading time so it takes long enough to get a coffee (and display an according message) or try to fix it. Carsten will probably be very bored when coming back from holiday so I'll assign this one to him :-)

(22004) seanrife (developer) 2012-11-08 05:33	I'm experiencing the same problem. I just uploaded a couple of surveys that have been extremely slow (sometimes timing out while as few as 10 people are taking it). Also comes up when viewing responses (page will sometimes take up to a minute to load). This is now on a VPS with 4GB memory. Two other notes: (1) running MySQL queries is very snappy. (2) I'm almost certain this has to do with memory (CPU load has never exceeded .14), as I've seen the server use up to 300MB just pulling up results (a single user). I'm no expert, but this certainly seems excessive. Or am I missing something?

(22110) mdekker (developer) 2012-11-13 12:25	Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=10263 [^]

(22265) mdekker (developer) 2012-11-16 12:32	@seanrife: Some improvements have been done to make surveys with randomization faster. Also a bottleneck with checking for updates has been fixed. Could you please test latest release (121116) and report if it helped to get faster response times so we can close this issue?

(22291) seanrife (developer) 2012-11-16 19:47	Thanks mdekker... I just upgraded, and things seem to be working well. I definitely notice a significant difference. I also (about a week ago) switched to php 5.2.17 and enabled XCache, which seems to have helped too. All in all, I think we're good to go here. Thanks for all your hard work!

Relationships