View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
06567 | Bug reports | Central participant database | public | 2012-09-19 11:58 | 2012-11-15 14:47 |
Reporter | lolo | Assigned To | mdekker | ||
Priority | normal | Severity | minor | ||
Status | closed | Resolution | fixed | ||
Product Version | 2.00RC9 | ||||
Fixed in Version | 2.00+ | ||||
Summary | 06567: Performance issue | ||||
Description | My participant database currently has:
On the panel screen, to display the full list of participants it takes:
There has to be a bootleneck somewhere, as it's not that much data to handle even for modest hardware. | ||||
Steps To Reproduce |
| ||||
Tags | No tags attached. | ||||
Bug heat | 14 | ||||
Complete LimeSurvey version number (& build) | 120817 | ||||
I will donate to the project if issue is resolved | No | ||||
Browser | Mozilla Firefox 15.0.1 | ||||
Database type & version | Mysql 5.5.24 | ||||
Server OS (if known) | Linux 2.6.27.52 & Win XP SP3 | ||||
Webserver software & version (if known) | Apache 2 | ||||
PHP Version | 5.3.14 | ||||
Forgot last step in "Steps to reproduce":
|
|
Fix committed to 2.1 branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=10037 |
|
Hey Lolo, I did some testing with 3749 participants and 26 attributes, and yes there was a bottleneck that has been fixed and committed. I want to share the test results so as to bring something to your notice. Initially these participants without any attribute visible in the display participant screen were taking 19.4 seconds to load. After removing the bottleneck it came down to 2.25 seconds Then I enabled all the 26 attributes to be visible in the display participant screen , the load time was 30 seconds and with 6 attributes it was 9.12 seconds and with 1 attribute visible it was 4 seconds. I would suggest that you don't keep the unnecessary attributes visible in the participant screen. Even if the attribute value is empty, that will still have an impact on performance. Keep the attributes visible to only the ones you require :) |
|
Aniessh, for a real panel provider, having 20 or more attributes is absolutely normal. So if +1 attribute adds +1sec loading time, there still seems to be some kind of bottleneck I would guess. Maybe you can have another look before more bug report about perfomance issues with >10 attributes show up? |
|
Hi Aniessh, If panel contains N participants and M attributes, I suspect that the application is performing N x M requests (HTTP+database). This is never going to scale with large amounts of data, as the performance penalty is exponential. The solution, in my opinion, is to aim for an algorithm of linear complexity (or even better, a constant complexity). The best solution would be to fetch all participants and their respective attributes in one single SQL query (and therefore one single HTTP/json request). In issue 06454 I gave an example of such a SQL query. On my MySQL server I have created a view that contains such a query (it returns all participants and all their attribute values in one single SQL query, so I know it works). I know it's easier said than done, and that it would require quite a bit of changes, but it may be the only way to definitely solve the issue. Hope it helped. |
|
Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=10056 |
|
Fix committed to 2.1 branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=10066 |
|
Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=10068 |
|
Hey Mazi,Lolo @Mazi : sure :) @lolo : thank you very much for the example of the query you gave, I implemented that query and it's working like a charm. Please note I have done this for users with super admin rights only, will try to do for normal users tomorrow ( feeling sleepy ) ,it's quite different from super admin. Although I have one question for you, if you can answer that then we can probably reduce the computation time further, do you have any idea of how to handle the survey count in that single query, I mean it's a join on the number of counts on survey_links table, if that's possible and if you can give an example of the same, I would be more than happy to implement that and the entire data will then come from a single query which will reduce the turn around time even further. As far as the number of http/json requests is concerned, it's has always been a single request to the server, but because of the execution of large number of queries it was running slow. As for statistics , It was taking around 31.33 with 26 attributes and 3749 participants which is now reduced to 4.08 seconds with the same number of attributes, thanks to your example of the query :) |
|
The fix contains hard-coded table prefixes, see https://github.com/LimeSurvey/LimeSurvey/commit/3c6ea940e6c2d04fccbc33056ab61dd4e5c6bb70#L1R215 Aniessh, can you please fix this? |
|
Added some fixes to participant panel, including the prefix issue |
|
Hi, Recently updated to build 121115, and the rendering performance is now miles away than what it was before. So, congrats for the speed-ups guys, it's much more usable now ! |
|
LimeSurvey: 2.1 c2e438b0 2012-10-27 17:23 Details Diff |
Fixed issue 06567: Performance issue Dev Removed unnecessary processing giving display participant better load times |
Affected Issues 06567 |
|
mod - application/controllers/admin/participantsaction.php | Diff File | ||
LimeSurvey: master 7a08f153 2012-10-30 09:20 Details Diff |
Fixed issue 06567: Performance issue Dev Removed unnecessary processing giving display participant better load times |
Affected Issues 06567 |
|
mod - application/controllers/admin/participantsaction.php | Diff File | ||
LimeSurvey: 2.1 b8a1073c 2012-10-30 12:44 Details Diff |
Fixed issue 06567: Performance issue Dev This commit changes the way attributes are extracted for superadmins only |
Affected Issues 06567 |
|
mod - application/controllers/admin/participantsaction.php | Diff File | ||
mod - application/models/Participants.php | Diff File | ||
LimeSurvey: master 3c6ea940 2012-10-30 12:58 Details Diff |
Fixed issue 06567: Performance issue Dev This commit changes the way attributes are extracted for superadmins only |
Affected Issues 06567 |
|
mod - application/controllers/admin/participantsaction.php | Diff File | ||
mod - application/models/Participants.php | Diff File | ||
LimeSurvey: master a88ebb5c 2012-11-07 16:45 Details Diff |
dev: minor fixes to participant panel |
Affected Issues 06567 |
|
mod - application/controllers/admin/participantsaction.php | Diff File | ||
mod - application/models/Participants.php | Diff File | ||
LimeSurvey: master 65e4d015 2012-11-07 16:48 Details Diff |
dev: removed debug & reset |
Affected Issues 06567 |
|
mod - application/controllers/admin/participantsaction.php | Diff File | ||
LimeSurvey: master 670eaaa8 2012-11-07 16:52 Details Diff |
dev: another small fix |
Affected Issues 06567 |
|
mod - application/models/ParticipantAttributeNames.php | Diff File |
Date Modified | Username | Field | Change |
---|---|---|---|
2012-09-19 11:58 | lolo | New Issue | |
2012-09-19 11:58 | lolo | Status | new => assigned |
2012-09-19 11:58 | lolo | Assigned To | => jcleeland |
2012-09-19 12:12 | lolo | Note Added: 20729 | |
2012-09-19 12:13 | lolo | Issue Monitored: lolo | |
2012-10-14 13:42 | Mazi | Issue Monitored: aniesshsethh | |
2012-10-28 00:32 | aniesshsethh | Changeset attached | => LimeSurvey 2.1 c2e438b0 |
2012-10-28 00:32 | aniesshsethh | Note Added: 21736 | |
2012-10-28 00:32 | aniesshsethh | Assigned To | jcleeland => aniesshsethh |
2012-10-28 00:32 | aniesshsethh | Resolution | open => fixed |
2012-10-28 01:02 | aniesshsethh | Note Added: 21737 | |
2012-10-28 01:02 | aniesshsethh | Status | assigned => feedback |
2012-10-28 01:05 | aniesshsethh | Note Edited: 21737 | |
2012-10-28 01:13 | aniesshsethh | Note Edited: 21737 | |
2012-10-28 14:12 | Mazi | Note Added: 21738 | |
2012-10-28 16:39 | lolo | Note Added: 21741 | |
2012-10-28 16:39 | lolo | Status | feedback => assigned |
2012-10-28 21:44 | rizaemet | Issue Monitored: rizaemet | |
2012-10-28 21:45 | rizaemet | Issue End Monitor: rizaemet | |
2012-10-30 16:21 | aniesshsethh | Changeset attached | => LimeSurvey master 7a08f153 |
2012-10-30 16:21 | aniesshsethh | Note Added: 21780 | |
2012-10-30 19:45 | aniesshsethh | Changeset attached | => LimeSurvey 2.1 b8a1073c |
2012-10-30 19:45 | aniesshsethh | Note Added: 21796 | |
2012-10-30 19:58 | aniesshsethh | Changeset attached | => LimeSurvey master 3c6ea940 |
2012-10-30 19:58 | aniesshsethh | Note Added: 21797 | |
2012-10-30 20:07 | aniesshsethh | Note Added: 21798 | |
2012-10-30 20:07 | aniesshsethh | Status | assigned => feedback |
2012-10-30 20:12 | aniesshsethh | Note Edited: 21798 | |
2012-10-30 20:15 | aniesshsethh | Note Edited: 21798 | |
2012-10-30 20:16 | aniesshsethh | Note Edited: 21798 | |
2012-11-06 10:24 | c_schmitz | Status | feedback => closed |
2012-11-06 10:25 | c_schmitz | Fixed in Version | => 2.00+ |
2012-11-07 16:28 | c_schmitz | Note Added: 22001 | |
2012-11-07 16:28 | c_schmitz | Status | closed => feedback |
2012-11-07 16:28 | c_schmitz | Resolution | fixed => reopened |
2012-11-07 16:58 | mdekker | Changeset attached | => LimeSurvey master a88ebb5c |
2012-11-07 16:59 | mdekker | Changeset attached | => LimeSurvey master 65e4d015 |
2012-11-07 16:59 | mdekker | Changeset attached | => LimeSurvey master 670eaaa8 |
2012-11-07 16:59 | mdekker | Note Added: 22002 | |
2012-11-15 14:24 | lolo | Note Added: 22181 | |
2012-11-15 14:24 | lolo | Status | feedback => assigned |
2012-11-15 14:25 | lolo | Note Edited: 22181 | |
2012-11-15 14:47 | c_schmitz | Status | assigned => closed |
2012-11-15 14:47 | c_schmitz | Assigned To | aniesshsethh => mdekker |
2012-11-15 14:47 | c_schmitz | Resolution | reopened => fixed |
2021-08-02 19:35 | guest | Bug heat | 10 => 14 |