View Issue Details

Jump to NotesJump to History
This bug affects 1 person(s).
 10
IDProjectCategoryView StatusDate SubmittedLast Update
06952Bug reportsOtherpublic2012-11-26 02:292012-12-07 09:58
Reporterseanrife Assigned Toseanrife  
PriorityurgentSeverityblock 
Status closedResolutionfixed 
Product Version2.00+ 
Fixed in Version2.00+ 
Summary06952: Fatal error when exporting large amounts of data
Description

Getting the following message when trying to export data:

Fatal error: Allowed memory size of 201326592 bytes exhausted (tried to allocate 24 bytes) in /home/sr_lime2/surveys.lyceum.ws/ls/framework/db/CDbDataReader.php on line 122

This occurs when any attempt is made to export results from a survey with a large number of responses. For example, when doing a full SQL dump, running ComfortUpdate (at the stage where a backup of the database is created), or exporting results.

My assumption is that this has to do with the way Yii is reading results (i.e., return $this->_statement->fetchAll();, which puts all the responses into a single array). This is obviously contingent on memory_limit, which I have set to 192M (I really can't do more, as my host's hard limit is 300MB, and I have a couple of other sites).

Steps To Reproduce
  1. Create a new survey using the enclosed .lss file.
  2. Accumulate 1,400+ responses.
  3. Attempt to export the data as described above.
Additional Information

Unfortunately, my institution's privacy regulations prohibit me from providing external entities with participant responses. However, you should be able to reproduce the problem with any kind of simulated data.

TagsNo tags attached.
Attached Files
limesurvey_survey_917871.lss (226,326 bytes)
Bug heat10
Complete LimeSurvey version number (& build)121116
I will donate to the project if issue is resolvedNo
BrowserN/A
Database type & versionMySQL 5.1.63
Server OS (if known)Debian 6.0
Webserver software & version (if known)Apache 2.2
PHP Version5.2.17

Users monitoring this issue

seanrife

Activities

c_schmitz

c_schmitz

2012-11-30 14:19

administrator   ~22735

I cannot reproduce this at all. I can export surveys with 50,000 responses without problems. Maybe your records are very big.

Try setting

'maxdumpdbrecords'=>1000,

in config.php (below the debug setting) and report if that solves the problem.
seanrife

seanrife

2012-11-30 20:56

reporter   ~22770

Changing maxdumpdbrecords to 1000 wasn't enough (still getting the fatal error), but taking it down to 100 at least allowed me to update. Is there any significant disadvantage to using such a low setting? There seems to be a monotonic increase in the size of the SQL dump file along with the decrease of the maxdumpdbrecords value, but I'm not sure what data are being included/excluded.

However, even with maxdumpdbrecords set to 100, I'm still getting the same error when I try to run an export to an SPSS or R data file. It DOES work when I try to export to a CSV file using the "Export results to an application" option.

I'm not sure what constitutes a "very big" record, but the survey in question (the attached lss file) has over 800 variables (a lot of matrix questions with checkboxes).
c_schmitz

c_schmitz

2012-11-30 21:25

administrator   ~22771

There is no real disadvantage - it just takes a little longer and the file is a bit bigger. No data is excluded.
The exact limit depends on your mySQL configuration. Glad the setting worked out for you.
seanrife

seanrife

2012-11-30 21:38

reporter   ~22772

Sorry - perhaps I wasn't clear... I'm still getting the error when I try to export data to an SPSS or R file. Should I file this as a new bug report?
inspire

inspire

2012-12-03 18:27

reporter   ~22830

This show-stopper is still an issue here, w/ 2.00-121127, even w/ maxdumpdbrecords set low.
mdekker

mdekker

2012-12-03 20:48

reporter   ~22833

Please file a new report for spps/r export. They work different and are not capable of limiting the amount of exported data and probably max execution time will be the problem you hit. It is on the wishlist to fix, but probably has to wait for 2.1 to be released.
inspire

inspire

2012-12-03 21:08

reporter   ~22834

LS seems to crash for any "application" export for us, w/ ~ 1800 records.

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 2 bytes) in /var/www/limesurvey/framework/db/CDbDataReader.php on line 122
seanrife

seanrife

2012-12-03 21:45

reporter   ~22835

New bug report filed. See http://bugs.limesurvey.org/view.php?id=7007
inspire

inspire

2012-12-03 22:20

reporter   ~22837

OK, sorry for the noise here. Was able to get export to complete by boosting memory_limit in php.ini
seanrife

seanrife

2012-12-04 21:50

reporter   ~22885

Fixed by lowering maxdumpdbrecords value and resolution to 7007.
c_schmitz

c_schmitz

2012-12-07 09:58

administrator   ~23015

2.00+ build 121207 released.

Issue History

Date Modified Username Field Change
2012-11-26 02:29 seanrife New Issue
2012-11-26 02:29 seanrife File Added: limesurvey_survey_917871.lss
2012-11-26 02:31 seanrife Issue Monitored: seanrife
2012-11-26 10:27 c_schmitz Assigned To => c_schmitz
2012-11-26 10:27 c_schmitz Status new => assigned
2012-11-30 14:19 c_schmitz Note Added: 22735
2012-11-30 14:19 c_schmitz Status assigned => feedback
2012-11-30 20:56 seanrife Note Added: 22770
2012-11-30 20:56 seanrife Status feedback => assigned
2012-11-30 21:25 c_schmitz Note Added: 22771
2012-11-30 21:25 c_schmitz Status assigned => closed
2012-11-30 21:25 c_schmitz Resolution open => no change required
2012-11-30 21:38 seanrife Note Added: 22772
2012-11-30 21:38 seanrife Status closed => feedback
2012-11-30 21:38 seanrife Resolution no change required => reopened
2012-12-03 18:27 inspire Note Added: 22830
2012-12-03 20:48 mdekker Note Added: 22833
2012-12-03 21:08 inspire Note Added: 22834
2012-12-03 21:45 seanrife Note Added: 22835
2012-12-03 21:45 seanrife Status feedback => assigned
2012-12-03 22:20 inspire Note Added: 22837
2012-12-03 23:00 c_schmitz Assigned To c_schmitz => seanrife
2012-12-04 21:50 seanrife Note Added: 22885
2012-12-04 21:50 seanrife Status assigned => resolved
2012-12-04 21:50 seanrife Fixed in Version => 2.00+
2012-12-04 21:50 seanrife Resolution reopened => fixed
2012-12-07 09:58 c_schmitz Note Added: 23015
2012-12-07 09:58 c_schmitz Status resolved => closed
2021-08-02 19:44 guest Bug heat 8 => 10