View Issue Details

This bug affects 1 person(s).
 6
IDProjectCategoryView StatusLast Update
19589Bug reportsSpeed optimizationpublic2024-08-06 16:55
ReporterDenisChenu Assigned ToDenisChenu  
PrioritynoneSeverityminor 
Status closedResolutionfixed 
Product Version6.5.x 
Summary19589: Multiple call of Survey->find a lot of time
Description

Wde have a bunck of system.db.CDbCommand.query(SELECT survey.sid AS t1_c0, survey.owner_id AS t1_c1, survey.gsid AS t1_c2, survey.admin AS t1_c3, survey.active AS t1_c4, survey.expires AS t1_c5, survey.startdate AS t1_c6, survey.adminemail AS t1_c7, survey.anonymized AS t1_c8, survey.format AS t1_c9, survey.savetimings AS t1_c10, survey.template AS t1_c11, survey.language AS t1_c12, survey.additional_languages AS t1_c13, survey.datestamp AS t1_c14, survey.usecookie AS t1_c15, survey.allowregister AS t1_c16, survey.allowsave AS t1_c17, survey.autonumber_start AS t1_c18, survey.autoredirect AS t1_c19, survey.allowprev AS t1_c20, survey.printanswers AS t1_c21, survey.ipaddr AS t1_c22, survey.ipanonymize AS t1_c23, survey.refurl AS t1_c24, survey.datecreated AS t1_c25, survey.showsurveypolicynotice AS t1_c26, survey.publicstatistics AS t1_c27, survey.publicgraphs AS t1_c28, survey.listpublic AS t1_c29, survey.htmlemail AS t1_c30, survey.sendconfirmation AS t1_c31, survey.tokenanswerspersistence AS t1_c32, survey.assessments AS t1_c33, survey.usecaptcha AS t1_c34, survey.usetokens AS t1_c35, survey.bounce_email AS t1_c36, survey.attributedescriptions AS t1_c37, survey.emailresponseto AS t1_c38, survey.emailnotificationto AS t1_c39, survey.tokenlength AS t1_c40, survey.showxquestions AS t1_c41, survey.showgroupinfo AS t1_c42, survey.shownoanswer AS t1_c43, survey.showqnumcode AS t1_c44, survey.bouncetime AS t1_c45, survey.bounceprocessing AS t1_c46, survey.bounceaccounttype AS t1_c47, survey.bounceaccounthost AS t1_c48, survey.bounceaccountpass AS t1_c49, survey.bounceaccountencryption AS t1_c50, survey.bounceaccountuser AS t1_c51, survey.showwelcome AS t1_c52, survey.showprogress AS t1_c53, survey.questionindex AS t1_c54, survey.navigationdelay AS t1_c55, survey.nokeyboard AS t1_c56, survey.alloweditaftercompletion AS t1_c57, survey.googleanalyticsstyle AS t1_c58, survey.googleanalyticsapikey AS t1_c59, survey.tokenencryptionoptions AS t1_c60 FROM lime_surveys survey WHERE (survey.sid=:ypl0). Bound with :ypl0=282267)

On each page of survey
It's OK with good SQL server using cache a lot, but some other can put it at top
161 time for Samppe ls5 survey / Single choice question for example

Steps To Reproduce

Steps to reproduce

Install a new master version
Import demo5 survey
Activate debug=2 and debugsql=1
Activate survey
Lauch
Move next
Move next
Search for SELECT survey.sid in SQL debug

Expected result

Find it, maybe 4/5 times

Actual result

Find it a bunch of time : 131 for Single choice

TagsNo tags attached.
Attached Files
Bug heat6
Complete LimeSurvey version number (& build)6.5.11
I will donate to the project if issue is resolvedNo
Browsernot relevant
Database type & versionnot relevant
Server OS (if known)not relevant
Webserver software & version (if known)not relevant
PHP Versionnot relevant

Users monitoring this issue

There are no users monitoring this issue.

Activities

DenisChenu

DenisChenu

2024-06-06 12:00

developer   ~80229

tested :

    public function getRelated($name, $refresh=false, $params=array()) {
        if ($name == 'survey' && !$refresh && empty($params)) {
            return Survey::model()->findByPk($this->sid);
        }
        return parent::getRelated($name, $refresh,$params);
    }

Seems to wok

TonisOrmisson

TonisOrmisson

2024-06-06 14:43

developer   ~80232

If you have a chance, it would be interesting to know what would be the query result with my suggested PR to reduce reasonably the calling of survey finds

https://github.com/LimeSurvey/LimeSurvey/pull/3819/files

I tend to be a bit skeptical about overriding the framework logic. If you have a general knowledge about how the framework works and there is one model where you have overridden it, it might cause problems later in other places.

For me it seems more logical to start from fixing bad code and assess the situation after fixing obvious code inefficiency issues. A good example of what I mean is this
https://github.com/LimeSurvey/LimeSurvey/pull/3819/files#diff-af8e690b7162e715acc4d95c2dd74fb97479066cf40a03d33de4c393df0250e3L1563 (image)

having a survey find in such a loops can possibly explode these queries. Maybe fixing inefficient code will cut most of the queries so that it will not be that big of an issue any more to fix?

image.png (81,097 bytes)   
image.png (81,097 bytes)   
DenisChenu

DenisChenu

2024-06-06 14:49

developer   ~80233

I tend to be a bit skeptical about overriding the framework logic. If you have a general knowledge about how the framework works and there is one model where you have overridden it, it might cause problems later in other places.

I check the framework logic before.

Else “ remove duplicate survey-find-by-pk ” is unrelated since Survey::model()->findByPk($surveyId); use static variable , then same number of DB call.

DenisChenu

DenisChenu

2024-06-06 15:08

developer   ~80234

@TonisOrmisson

I confirm Survey::model()->findByPk($this->sid) don't throw again the event : since it use a static variable

Without public function getRelated($name, $refresh=false, $params=array()) { : 63 event
With 2 event

(on public page : demo survey)

DenisChenu

DenisChenu

2024-06-06 15:23

developer   ~80235

This lower the number of event afterFindSurey too : 2 times on public survey, 63 por more time before (using debug and previous plugin)

DenisChenu

DenisChenu

2024-06-06 15:23

developer   ~80236

Let's discuss

DenisChenu

DenisChenu

2024-06-06 15:28

developer   ~80237

afterFindSurey event

questionAdministration/view&surveyid=282267&gid=4&qid=22

without the fix : 6 times
with the fix : 3 times

Logixc file

without the fix : 4 times
with the fix : 3 times

Really better on public part (happen for each question multiple times)

DenisChenu

DenisChenu

2024-06-06 15:30

developer   ~80238

3 times for admin part seems a constant

TonisOrmisson

TonisOrmisson

2024-06-06 17:16

developer   ~80241

I traced this event being mostly triggered from here:
https://github.com/LimeSurvey/LimeSurvey/blob/master/application/models/QuestionAttribute.php#L258

Maybe inject languages as a dependency here, the db queries will be gone then, most of them

DenisChenu

DenisChenu

2024-06-06 18:44

developer   ~80242

Last edited: 2024-06-06 18:45

Both can be interesting : https://github.com/LimeSurvey/LimeSurvey/pull/3871

(I test it with one of my client)

DenisChenu

DenisChenu

2024-06-12 17:07

developer   ~80280

I assign to Carsten maybe ?

2 solution currently :

  1. Review to use findByPkCache fpr question (like 3.X) : 1 (or 2/3) calls by page
  2. Improve other part to minimize request (maybe 10 by page, but need to improve Events too in my opinion)
DenisChenu

DenisChenu

2024-08-06 14:50

developer   ~80749

Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=36510

LimeBot

LimeBot

2024-08-06 16:55

administrator   ~80754

Fixed in Release 6.6.1+240806

Related Changesets

LimeSurvey: master 3163c250

2024-08-06 14:50:08

DenisChenu


Committer: GitHub Details Diff
Fixed issue 19589: Multiple call of Survey->find for public survey (#3871) Affected Issues
19589
mod - application/models/Question.php Diff File
mod - application/models/Survey.php Diff File

Issue History

Date Modified Username Field Change
2024-06-06 11:45 DenisChenu New Issue
2024-06-06 11:45 DenisChenu File Added: Capture d’écran du 2024-06-06 11-37-34.png
2024-06-06 11:45 DenisChenu File Added: Capture d’écran du 2024-06-06 11-37-43.png
2024-06-06 11:45 DenisChenu File Added: Capture d’écran du 2024-06-06 11-38-08.png
2024-06-06 11:59 DenisChenu Category Other => Speed optimization
2024-06-06 12:00 DenisChenu Note Added: 80229
2024-06-06 12:00 DenisChenu Bug heat 0 => 2
2024-06-06 14:43 TonisOrmisson Note Added: 80232
2024-06-06 14:43 TonisOrmisson File Added: image.png
2024-06-06 14:43 TonisOrmisson Bug heat 2 => 4
2024-06-06 14:49 DenisChenu Note Added: 80233
2024-06-06 15:08 DenisChenu Note Added: 80234
2024-06-06 15:08 DenisChenu File Added: AfterFindSurveyEventCount.zip
2024-06-06 15:17 DenisChenu Summary Multuiple call of same request a lot of time => Multiple call of Survey->find a lot of time
2024-06-06 15:23 DenisChenu Note Added: 80235
2024-06-06 15:23 DenisChenu Assigned To => TonisOrmisson
2024-06-06 15:23 DenisChenu Status new => ready for code review
2024-06-06 15:23 DenisChenu Note Added: 80236
2024-06-06 15:28 DenisChenu Note Added: 80237
2024-06-06 15:30 DenisChenu Note Added: 80238
2024-06-06 17:16 TonisOrmisson Note Added: 80241
2024-06-06 18:44 DenisChenu Note Added: 80242
2024-06-06 18:45 DenisChenu Note Edited: 80242
2024-06-12 17:05 DenisChenu Assigned To TonisOrmisson => c_schmitz
2024-06-12 17:07 DenisChenu Note Added: 80280
2024-08-06 14:50 DenisChenu Changeset attached => LimeSurvey master 3163c250
2024-08-06 14:50 DenisChenu Note Added: 80749
2024-08-06 14:50 DenisChenu Assigned To c_schmitz => DenisChenu
2024-08-06 14:50 DenisChenu Resolution open => fixed
2024-08-06 15:17 tibor.pacalat Status ready for code review => resolved
2024-08-06 16:55 LimeBot Note Added: 80754
2024-08-06 16:55 LimeBot Status resolved => closed
2024-08-06 16:55 LimeBot Bug heat 4 => 6