View Issue Details

This issue affects 1 person(s).
 4
IDProjectCategoryView StatusLast Update
20653Bug reportsSpeed optimizationpublic2026-08-14 14:19
Reportersigun Assigned To 
PrioritynoneSeverityminor 
Status newResolutionopen 
Product Version7.0.x 
Summary20653: Listing submitted responses for surveys take LOTS of time
Description

Browsing survey responses (Responses → Browse responses) is extremely slow for surveys with a non-trivial number of questions and responses, because getExtendedAnswer() in application/helpers/common_helper.php runs a full, unfiltered SELECT * FROM questions WHERE sid = :sid query on every single rendered cell of the responses grid, in addition to a Survey::model()->findByPk() query per cell. For a page of 10 responses × ~80 columns this adds up to 500+ avoidable queries per page load, and is the dominant cost of the page.

Steps To Reproduce

Steps to reproduce

  1. Create (or use) a survey with a moderate number of questions (our reproduction case: 81
    questions) and several hundred responses (our case: 516).
  2. Go to Responses → Browse responses for that survey.
  3. Observe page load time for a single page of 10 responses.
  4. Enable the MySQL general query log while loading one page and inspect the query counts.

Actual behavior

Loading a single page of 10 responses took ~3 seconds, and the general query log showed the query

SELECT * FROM `questions` WHERE sid = <sid>

executed 526 times for that one page load (once per non-empty answer cell), each time loading every question row for the survey, only to run:

$found = false;
foreach ($rawQuestions as $rawQuestion) {
    $found = $found || (strpos($sFieldCode, "Q{$rawQuestion->qid}") === 0);
}

i.e. a linear scan just to decide whether $sFieldCode might be a question field, before optionally building the field map. Survey::model()->findByPk($iSurveyID) is likewise re-fetched on every call.

Expected behavior

The responses grid should not re-run survey-wide queries once per cell. Rendering N rows × M columns should cost O(1) (or at most O(rows)) survey/question metadata queries, not O(rows × cols).

Root cause

application/helpers/common_helper.php, function getExtendedAnswer() (called from SurveyDynamic::getExtendedData(), which is used as the CGridView column value expression for the responses grid — i.e. once per cell):

function getExtendedAnswer($iSurveyID, $sFieldCode, $sValue, $sLanguage, $question = null)
{
    if ($sValue == null || $sValue == '') {
        return '';
    }
    $survey = Survey::model()->findByPk($iSurveyID);
    $rawQuestions = Question::model()->findAll("sid = :sid", [":sid" => $iSurveyID]);
    $found = false;
    foreach ($rawQuestions as $rawQuestion) {
        $found = $found || (strpos($sFieldCode, "Q{$rawQuestion->qid}") === 0);
    }
    if ($found) {
        $fieldmap = createFieldMap($survey, 'short', false, false, $sLanguage);
        if (isset($fieldmap[$sFieldCode])) {
            $fields = $fieldmap[$sFieldCode];
        } else {
            return '';
        }
        ...

Patch example code provided, the performance regression is fixed on our local installation.

TagsNo tags attached.
Attached Files
fix-getExtendedAnswer-n-plus-one.patch (1,787 bytes)   
diff --git a/application/helpers/common_helper.php b/application/helpers/common_helper.php
--- a/application/helpers/common_helper.php
+++ b/application/helpers/common_helper.php
@@ -1000,16 +1000,24 @@ function getExtendedAnswer($iSurveyID, $sFieldCode, $sValue, $sLanguage, $ques
     if ($sValue == null || $sValue == '') {
         return '';
     }
-    $survey = Survey::model()->findByPk($iSurveyID);
-    $rawQuestions = Question::model()->findAll("sid = :sid", [":sid" => $iSurveyID]);
-    $found = false;
-    foreach ($rawQuestions as $rawQuestion) {
-        $found = $found || (strpos($sFieldCode, "Q{$rawQuestion->qid}") === 0);
+    // createFieldMap() is cached per survey/language in the user session, but
+    // Survey::model()->findByPk() is not - cache it per request (keyed off the
+    // current Yii app instance, which is recreated fresh for every request) so
+    // that rendering many cells of the same survey (e.g. the responses grid)
+    // doesn't re-fetch the survey row for every single cell.
+    static $surveyCache = [];
+    $appId = spl_object_id(Yii::app());
+    if (!isset($surveyCache[$appId])) {
+        $surveyCache = [$appId => []];
     }
+    if (!array_key_exists($iSurveyID, $surveyCache[$appId])) {
+        $surveyCache[$appId][$iSurveyID] = Survey::model()->findByPk($iSurveyID);
+    }
+    $survey = $surveyCache[$appId][$iSurveyID];
+
     //Fieldcode used to determine question, $sValue used to match against answer code
     //Returns NULL if question type does not suit
-    if ($found) {
-        //Only check if it looks like a real fieldcode
+    {
         $fieldmap = createFieldMap($survey, 'short', false, false, $sLanguage);
         if (isset($fieldmap[$sFieldCode])) {
             $fields = $fieldmap[$sFieldCode];
Bug heat4
Complete LimeSurvey version number (& build)7.0.10+260813
I will donate to the project if issue is resolvedNo
Story point estimate0
BrowserAny
Database type & versionMySQL 8.0.46
Server OS (if known)Ubuntu 24.04
Webserver software & version (if known) nginx + php-fpm
PHP Version8.3

Users monitoring this issue

DenisChenu

Activities

DenisChenu

DenisChenu

2026-08-14 14:19

developer   ~85474

Survey::model()->findByPk() is static : this part is not needed
https://github.com/LimeSurvey/LimeSurvey/blob/0e01ea590c60f8b879c858281d41d609744ef390/application/models/Survey.php#L1044

The issue Question::model()->findAll("sid = :sid", [":sid" => $iSurveyID]); with is real

Before only check if it's start by surveyId.
https://github.com/LimeSurvey/LimeSurvey/blame/e78a12233fa631a1d3190c3efe5a08a9d3497a1c/application/helpers/common_helper.php#L1007

Unsure if it's OK to remove it, but surely more quick :)

Issue History

Date Modified Username Field Change
2026-08-14 11:18 sigun New Issue
2026-08-14 11:18 sigun File Added: fix-getExtendedAnswer-n-plus-one.patch
2026-08-14 14:08 DenisChenu Issue Monitored: DenisChenu
2026-08-14 14:08 DenisChenu Bug heat 0 => 2
2026-08-14 14:19 DenisChenu Note Added: 85474
2026-08-14 14:19 DenisChenu Bug heat 2 => 4