View Issue Details

This bug affects 1 person(s).
 6
IDProjectCategoryView StatusLast Update
05992Bug reportsSurvey takingpublic2012-04-16 18:30
ReporterTmac Assigned ToTMSWhite  
PriorityhighSeveritypartial_block 
Status closedResolutionfixed 
Product Version1.92+ 
Fixed in Version1.92+ 
Summary05992: After upgrade to Version 1.92+ Build 120405 the uploaded file information gets corrupted.
Description

After upgrade the JSON format record for a 'file upload' question in an existing survey database seems to get corrupted and no longer displays the file information and images.

Steps To Reproduce

Using earlier build 1.92+, create a survey with a file upload type of question.
Execute the survey and upload some image files.
Verify that the image files display correctly by editing/executing the same survey again.
Upgrade to build 12045 and execute the same survey (with the same token)
Now the images and information is corrupted and if you check the database entry
after submission, the JSON record is changed to [<span style=.
Before the value was [{"title":"","comment":"","size":"11.859","name":"fsq.jpg","filename":"fu_3ivwau56u7mez4x","ext":"jpg"}]

Additional Information

I am suspicious of an encoding/collation change from old to new database
See enclosed image.

TagsNo tags attached.
Attached Files
Clipboard01.jpg (64,822 bytes)   
Clipboard01.jpg (64,822 bytes)   
Bug heat6
Complete LimeSurvey version number (& build)120405
I will donate to the project if issue is resolvedNo
BrowserChrome
Database type & versionMySQL
Server OS (if known)Linux
Webserver software & version (if known)Apache
PHP VersionPHP Version 5.3.3-7+squeeze8

Users monitoring this issue

There are no users monitoring this issue.

Activities

Tmac

Tmac

2012-04-07 16:54

reporter   ~18248

I think I have found the reason...
In build 120405 the JSON string has an extra space (x20) just after the {
eg:

[{ "title"......
5b7b20227469746c6522

and the earlier build did not add the extra space eg:
[{"title"......
5b7b227469746c6522

If the space is missing the latest build cannot interpret the JSON code and corrupts the database when saving.

TMSWhite

TMSWhite

2012-04-07 17:08

reporter   ~18249

Those look like EM errors. There must be JSON encoding that does NOT have an extra space next to the opening and closing curly braces. Thus, EM thinks it is an expression and shows syntax errors around the colons.

JSON must be passed through this for EM to avoid trying to parse it:

str_replace(array('{','}'), array('{ ', ' }'), $json)

I'd be very surprised if JSON can't cope with the internal space - but perhaps someone can check that on some other JSON code.

Tmac

Tmac

2012-04-07 17:23

reporter   ~18252

I may be wrong but after a quick look at the code, Limesurvey uses it's own JSON decoding which may work differently to standard php.

The quick fix is to update the database with a replace function:
UPDATE TABLE_NAME SET FIELD_NAME = REPLACE( FIELD_NAME , '[{"', '[{ "' ) WHERE 1
using MYSQL

TMSWhite

TMSWhite

2012-04-07 18:43

reporter   ~18259

Tac, do you recall which older version of 1.92 you were using for your original tests?

Also, when using that older version, did you try going through the same steps (executing the same survey with the same token) - and if so, did it fail there too?

Tmac

Tmac

2012-04-07 20:29

reporter   ~18264

I think the build was 120311, and I had no problem with modifying/viewing previously uploaded files. (the JSON code had no extra space then)

I'm using LS in a slightly unusual way, members are allowed to update their info but filling out a survey (additional invites are sent of request). The info then populates a tourism website and displays member information.

Now after upgrading, the extra space(s) is needed to execute the survey more than once. (If a brand new survey is executed there is no problem)
To test, just remove the extra space directly in the database table.

TMSWhite

TMSWhite

2012-04-07 21:33

reporter   ~18265

Extra spaces must be allowable in JSON. See examples here (http://en.wikipedia.org/wiki/JSON) and here (http://json.org/example.html).

So, the core problem is probably a little different.

TMSWhite

TMSWhite

2012-04-08 04:41

reporter   ~18269

Tac-

I'm having trouble replicating a variant of this. If I use the current build, I can upload some pictures, resume later, and see the pictures again. The values in the database are correct, and editing of images is fine.

There was a bug in browsing of individual responses, but I just fixed that.

Is manipulating images working OK for you on the current version? Does this issue only occur when upgrading from a prior version?

TMSWhite

TMSWhite

2012-04-08 06:55

reporter   ~18270

Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&amp;id=8112

Tmac

Tmac

2012-04-08 07:24

reporter   ~18271

Correct! You can upload pics and edit them later with the current build. But if you have records in the database of uploaded files from the previous build, there is a problem as evidenced above.
You can replicate quite easily - just use the new build to upload some files via a survey question. (Set the global settings to allow updating of survey questions)
Then use Mysql to view the record and in the field where the JSON record is stored, observe the space just after the {. If you remove the space and then try to view/update the same survey record, you get those red syntax errors and garbled info, no images and the database record gets corrupted.

TMSWhite

TMSWhite

2012-04-08 07:25

reporter   ~18272

Tac-

I think that patch should fix it, but since I couldn't replicate the the problem, can you test it?

TMSWhite

TMSWhite

2012-04-08 07:33

reporter   ~18273

Our notes crossed paths. Just tried the Mysql approach you mentioned and view/update the same record works fine with the latest patch (and the database record gets fixed)

Tmac

Tmac

2012-04-08 07:56

reporter   ~18274

Thanks TMS, I will test it and let you know.

Tmac

Tmac

2012-04-08 09:01

reporter   ~18275

Confirmed TMS, this is now resolved. Even JSONs without the extra space are being decoded correctly. Thanks for your help.
I have posted another issue which is related to the latest version. An easy fix I think.

c_schmitz

c_schmitz

2012-04-12 00:42

administrator   ~18328

New 1.91+ version released.

TMSWhite

TMSWhite

2012-04-16 18:30

reporter   ~18381

Fix committed to Yii branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&amp;id=8173

Related Changesets

LimeSurvey: master 1bb18a23

2012-04-07 21:54:56

TMSWhite

Details Diff
Fixed issue 05992: After upgrade to Version 1.92+ Build 120405 the uploaded file information gets corrupted.
Dev ensures that all json_encode output has spaces added after opening curly braces so that EM doesn't try to parse it.
Affected Issues
05992
mod - admin/conditionshandling.php Diff File
mod - admin/dataentry.php Diff File
mod - admin/questionhandling.php Diff File
mod - admin/surveylist.php Diff File
mod - admin/translate_google_api.php Diff File
mod - classes/expressions/LimeExpressionManager.php Diff File
mod - common_functions.php Diff File
mod - delete.php Diff File
mod - qanda.php Diff File
mod - upload.php Diff File

LimeSurvey: Yii 8755a06c

2012-04-16 08:50:52

TMSWhite

Details Diff
Fixed issue 05992: After upgrade to Version 1.92+ Build 120405 the uploaded file information gets corrupted.
Dev ensures that all json_encode output has spaces added after opening curly braces so that EM doesn't try to parse it.
Affected Issues
05992
mod - application/controllers/UploaderController.php Diff File
mod - application/controllers/admin/conditionsaction.php Diff File
mod - application/controllers/admin/dataentry.php Diff File
mod - application/controllers/admin/labels.php Diff File
mod - application/controllers/admin/participantsaction.php Diff File
mod - application/controllers/admin/question.php Diff File
mod - application/controllers/admin/surveyadmin.php Diff File
mod - application/controllers/admin/tokens.php Diff File
mod - application/controllers/admin/translate.php Diff File
mod - application/helpers/admin/statistics_helper.php Diff File
mod - application/helpers/common_helper.php Diff File
mod - application/helpers/expressions/em_manager_helper.php Diff File
mod - application/helpers/qanda_helper.php Diff File
mod - application/views/admin/export/statistics_graph_view.php Diff File

Issue History

Date Modified Username Field Change
2012-04-07 16:11 Tmac New Issue
2012-04-07 16:11 Tmac File Added: Clipboard01.jpg
2012-04-07 16:54 Tmac Note Added: 18248
2012-04-07 17:08 TMSWhite Note Added: 18249
2012-04-07 17:23 Tmac Note Added: 18252
2012-04-07 18:20 c_schmitz Assigned To => TMSWhite
2012-04-07 18:20 c_schmitz Status new => assigned
2012-04-07 18:43 TMSWhite Note Added: 18259
2012-04-07 20:29 Tmac Note Added: 18264
2012-04-07 21:33 TMSWhite Note Added: 18265
2012-04-08 04:41 TMSWhite Note Added: 18269
2012-04-08 06:55 TMSWhite Changeset attached => LimeSurvey master 1bb18a23
2012-04-08 06:55 TMSWhite Note Added: 18270
2012-04-08 06:55 TMSWhite Resolution open => fixed
2012-04-08 07:24 Tmac Note Added: 18271
2012-04-08 07:25 TMSWhite Note Added: 18272
2012-04-08 07:25 TMSWhite Status assigned => feedback
2012-04-08 07:33 TMSWhite Note Added: 18273
2012-04-08 07:56 Tmac Note Added: 18274
2012-04-08 07:56 Tmac Status feedback => assigned
2012-04-08 09:01 Tmac Note Added: 18275
2012-04-08 11:20 TMSWhite Status assigned => resolved
2012-04-08 11:20 TMSWhite Fixed in Version => 1.92+
2012-04-12 00:42 c_schmitz Note Added: 18328
2012-04-12 00:42 c_schmitz Status resolved => closed
2012-04-16 18:30 TMSWhite Changeset attached => LimeSurvey Yii 8755a06c
2012-04-16 18:30 TMSWhite Note Added: 18381