View Issue Details

This issue affects 1 person(s).
 8
IDProjectCategoryView StatusLast Update
17908Bug reportsImport/Exportpublic2022-04-12 10:25
ReporterMazi Assigned Toc_schmitz  
PrioritynoneSeverityblock 
Status closedResolutionfixed 
Product Version3.25.20 
Fixed in Version5.x 
Summary17908: Export/Import a survey at LS 3.x fails with "Call to a member function getColumnNames() on null"
Description

A customer is running a large survey using Version 3.27.17+210911. That survey comes with 671 questions and shows 2571 columns at the export screen, though several hundred seem to be timing details. Thing is that when exporting results to XSL, only ~980 columns get exported. Then the file gets cut off for unknown reasons,

When exporting the survey as LSA file (file attached) and importing it at the latest 3.28 version I get this error (screenshot attached):
"Call to a member function getColumnNames() on null"
ls/application/helpers/admin/import_helper.php(1825)

Importing the survey as LSS file is possible but when activating it I get the famous "Too many columns" error:
"The survey response table could not be created. Usually this is caused by having too many (sub-)questions in your survey. Please try removing questions from your survey."

Several questions arise:

  1. Why was the customer able to activate the survey at their system (slightly different configuration?)?
  2. Is the LSA import error related to the activation error or is something else (survey structure?) causing the probem?
  3. When exporting data to Excel, is there a column limit? Or can something like a memory limit lead to the export including only a part of the columns?
Steps To Reproduce

Steps to reproduce

Try importing the attached LSA file. Try exporting data afterwards

Expected result

Import shoiuld work fine. Export should include all columns

Actual result

Import fails with error, thus export is not possibe,

TagsNo tags attached.
Bug heat8
Complete LimeSurvey version number (& build)3.28
I will donate to the project if issue is resolvedNo
BrowserChrome
Database type & versionmysqlnd 7.4.28
Server OS (if known)Ubuntu 18
Webserver software & version (if known)Apache 2.0
PHP Version7.4

Users monitoring this issue

There are no users monitoring this issue.

Activities

DenisChenu

DenisChenu

2022-02-24 09:01

developer   ~68388

Your client DB is ?

Mazi

Mazi

2022-02-24 09:04

updater   ~68389

@DenisChenu, client is using PHP 7.4.3 and mysqlnd 7.4.3

DenisChenu

DenisChenu

2022-02-24 09:07

developer   ~68390

Strange , mysql with such number of columns …

ollehar

ollehar

2022-02-24 09:09

administrator   ~68391

Do you have a db dump too?

DenisChenu

DenisChenu

2022-02-24 09:13

developer   ~68392

lss too maybe

c_schmitz

c_schmitz

2022-02-24 09:31

administrator   ~68394

Yeah, we know that problem. The response table is so close to the table field limit, that the length of the field names become relevant.

So if you copy that survey to a very low survey ID , like 2, you will be able to activate it because the field names are much shorter, and thus that internal field limit is not reached.

Mazi

Mazi

2022-02-24 09:34

updater   ~68395

@c_schmitz, thanks a lot for the very helpful hint!

Do you think this is also causing the export problem? How can we approach this?
Currently they are merging 2-3 file exports but that is painful since they need to analyze data weekly or even more often and new responses pop in every day.

Mazi

Mazi

2022-02-24 09:35

updater   ~68396

@ollehar I can not provide a full DB dump. There are dozens of surveys and there is a lot of medical data in it so it is too confidential. Sorry!

c_schmitz

c_schmitz

2022-02-24 09:37

administrator   ~68397

Response export limit is most likely caused by PHP max_input_vars setting, which defaults to 1000. Try to raise that to 2000.

Mazi

Mazi

2022-02-24 09:44

updater   ~68398

@c_schmitz, I was able to get the data into our test system by:

  1. Adjusting the survey ID at the LSS file to "111".
  2. Importing the surveys with the adjusted LSS file which then COULD be activated
  3. Exporting data using VV export
  4. Importing using VV import

I was also able to export ALL data because we always raise "max_input_vars" to 10.000 at our system.
So that was VERY helpful, thanks a lot.

Maybe you should code a warning if someone tries to expoirt a survey with the number of columns being > max_input_vars!?

I am attaching the new survey 111 as LSA file for further testing.

DenisChenu

DenisChenu

2022-02-24 10:06

developer   ~68400

Maybe you should code a warning if someone tries to expoirt a survey with the number of columns being > max_input_vars!?

I think we already have seom issue reported about max_input_vars , and already add some warnings in Question edition maybe ?

c_schmitz

c_schmitz

2022-02-24 11:09

administrator   ~68402

Last edited: 2022-02-24 13:26

Added a warning to response export if setting is too low in LS5.

Issue History

Date Modified Username Field Change
2022-02-24 08:44 Mazi New Issue
2022-02-24 08:44 Mazi File Added: image.png
2022-02-24 08:44 Mazi File Added: survey_archive_599599.lsa
2022-02-24 09:01 DenisChenu Note Added: 68388
2022-02-24 09:01 DenisChenu Bug heat 0 => 2
2022-02-24 09:04 Mazi Note Added: 68389
2022-02-24 09:04 Mazi Bug heat 2 => 4
2022-02-24 09:07 DenisChenu Note Added: 68390
2022-02-24 09:09 ollehar Note Added: 68391
2022-02-24 09:09 ollehar Bug heat 4 => 6
2022-02-24 09:13 DenisChenu Note Added: 68392
2022-02-24 09:19 c_schmitz View Status public => private
2022-02-24 09:19 c_schmitz Bug heat 6 => 12
2022-02-24 09:22 c_schmitz Bug heat 12 => 14
2022-02-24 09:30 c_schmitz Bug heat 14 => 12
2022-02-24 09:31 c_schmitz Note Added: 68394
2022-02-24 09:31 c_schmitz Bug heat 12 => 14
2022-02-24 09:32 c_schmitz File Deleted: survey_archive_599599.lsa
2022-02-24 09:32 c_schmitz File Deleted: image.png
2022-02-24 09:34 Mazi Note Added: 68395
2022-02-24 09:35 Mazi Note Added: 68396
2022-02-24 09:37 c_schmitz Note Added: 68397
2022-02-24 09:44 Mazi Note Added: 68398
2022-02-24 09:44 Mazi File Added: survey_archive_111.lsa
2022-02-24 10:06 DenisChenu Note Added: 68400
2022-02-24 11:09 c_schmitz Assigned To => c_schmitz
2022-02-24 11:09 c_schmitz Status new => resolved
2022-02-24 11:09 c_schmitz Resolution open => fixed
2022-02-24 11:09 c_schmitz Note Added: 68402
2022-02-24 11:09 c_schmitz File Deleted: survey_archive_111.lsa
2022-02-24 11:09 c_schmitz View Status private => public
2022-02-24 11:09 c_schmitz Bug heat 14 => 8
2022-02-24 13:26 c_schmitz Note Edited: 68402
2022-04-12 10:25 galads Status resolved => closed
2022-04-12 10:25 galads Fixed in Version => 5.x