View Issue Details

This bug affects 1 person(s).
 10
IDProjectCategoryView StatusLast Update
19661Bug reportsImport/Exportpublic2024-12-18 15:23
Reportermguzzinati Assigned Totibor.pacalat  
PrioritynoneSeveritypartial_block 
Status closedResolutionfixed 
Product Version6.5.x 
Summary19661: Data inconsistency in responses after importing LSA files
Description

After creating a questionnaire and a response which includes special characters including " ' ", " ] ", etc., I export the questionnaire with the LSA format, the %_responses.lsr file is correct, but when I import it of the .lsa file, the text of the questions in the new imported questionnaire is correct but the answers present some anomalies, all the " ' " disappear.

Steps To Reproduce

Steps to reproduce

-Create a questionnaire.
-Answer a question by entering characters like " ' ".
-Export the questionnaire in .lsa format
-Import the previously exported questionnaire (in the same environment by first deleting the previously exported one or in a new environment - tested with v6.5 and v5.6)
-read the answers you just posted

Expected result

see the answer correctly

(Write here what you expected to happen)

Actual result

all " ' " will disappear.

TagsNo tags attached.
Attached Files
Bug heat10
Complete LimeSurvey version number (& build)LimeSurvey Community Edition Version 6.5.4+240422
I will donate to the project if issue is resolvedNo
Browser
Database type & versionpostgres
Server OS (if known)
Webserver software & version (if known)
PHP Version8.2.7

Users monitoring this issue

There are no users monitoring this issue.

Activities

DenisChenu

DenisChenu

2024-07-22 11:53

developer   ~80680

Do you have the same thing with VV export/import ?

mguzzinati

mguzzinati

2024-07-22 12:08

reporter   ~80681

I hadn't tried it yet, I don't have this error with VV export/import. I still get it just by importing the entire questionnaire with the answers with the .lsa file

mguzzinati

mguzzinati

2024-11-13 10:37

reporter   ~81374

news?

tibor.pacalat

tibor.pacalat

2024-11-13 17:56

administrator   ~81378

@gabrieljenik I exported and imported the survey and then looked at the answers. They are indeed missing single quote. Is this maybe because we are sanitizing those strings somehow?

gabrieljenik

gabrieljenik

2024-11-20 16:06

manager   ~81470

I will review and be back.

gabrieljenik

gabrieljenik

2024-11-25 14:49

manager   ~81502

https://github.com/LimeSurvey/LimeSurvey/pull/4056

SurveyDynamic's insertRecords method was being used, which effectively has a sanitization, but it's not clear to me why...

Didn't dare to take it out.
Opted for something more similar to importing responses in CSV.

tibor.pacalat

tibor.pacalat

2024-11-25 17:53

administrator   ~81504

@gabrieljenik some tests are failing on your branch, can you please take a look at that?

gabrieljenik

gabrieljenik

2024-12-11 14:29

manager   ~81606

The automatic tests for this ticket were failing not because we introduced an error but because the fix impacted the import process (and fixed it).

The failing test was trying to import a survey for reviewing stats.
The LSA used in the test has a response with a lot of empty nodes, which technically means that the response is an empty string, but it was being imported as null (not relevant) missmatch!. Now they are imported as empty, not null. That's why the statistics don't match now.

Seems there was some manual editing on the file because in the other responses it has missing nodes (as if they hadn't been exported because they were null) and in the last one it has all the nodes empty, as if they had data and had been cleaned to simulate that they weren't answered.

We created a new branch based on master with an automatic test to show the issue:
https://github.com/LimeSurvey/LimeSurvey/actions/runs/12263965074/job/34216749065#step:11:146

That test is now green on this PR.

PS: There is a red on the PR automatic test because of an issue on master regarding the versioning.

DenisChenu

DenisChenu

2024-12-13 12:20

developer   ~81623

There are a lot of test to fix
https://github.com/SondagesPro/LimeSurvey-SondagesPro/commit/cf021d3671db3e9078d983e3e4f7ea4e71c536d3
https://github.com/SondagesPro/LimeSurvey-SondagesPro/commit/b325feb0de30509ed4ebb35dc702afc561ebc5ab
https://github.com/SondagesPro/LimeSurvey-SondagesPro/commit/9dd97d01c593fdfa645afdd2bf4f968d150db23f
https://github.com/SondagesPro/LimeSurvey-SondagesPro/commit/0c6534f1d73d17e59de9f450faedfb3baaf87ee5

gabrieljenik

gabrieljenik

2024-12-13 14:35

manager   ~81624

PR for fixing tests:
https://github.com/LimeSurvey/LimeSurvey/pull/4108

LimeBot

LimeBot

2024-12-18 15:23

administrator   ~81656

Fixed in Release 6.9.0+241218

Issue History

Date Modified Username Field Change
2024-07-22 11:47 mguzzinati New Issue
2024-07-22 11:47 mguzzinati File Added: survey_archive_182564.lsa
2024-07-22 11:53 DenisChenu Note Added: 80680
2024-07-22 11:53 DenisChenu Bug heat 0 => 2
2024-07-22 12:08 mguzzinati Note Added: 80681
2024-07-22 12:08 mguzzinati Bug heat 2 => 4
2024-11-13 10:37 mguzzinati Note Added: 81374
2024-11-13 17:54 tibor.pacalat Assigned To => gabrieljenik
2024-11-13 17:54 tibor.pacalat Status new => assigned
2024-11-13 17:56 tibor.pacalat Note Added: 81378
2024-11-13 17:56 tibor.pacalat File Added: Screenshot 2024-11-13 at 17.54.26.png
2024-11-13 17:56 tibor.pacalat Bug heat 4 => 6
2024-11-20 16:06 gabrieljenik Note Added: 81470
2024-11-20 16:06 gabrieljenik Bug heat 6 => 8
2024-11-25 14:49 gabrieljenik Assigned To gabrieljenik => DenisChenu
2024-11-25 14:49 gabrieljenik Status assigned => ready for code review
2024-11-25 14:49 gabrieljenik Note Added: 81502
2024-11-25 15:48 DenisChenu Assigned To DenisChenu => tibor.pacalat
2024-11-25 15:48 DenisChenu Status ready for code review => ready for testing
2024-11-25 17:53 tibor.pacalat Note Added: 81504
2024-12-11 14:29 gabrieljenik Note Added: 81606
2024-12-12 15:15 tibor.pacalat Status ready for testing => resolved
2024-12-12 15:15 tibor.pacalat Resolution open => fixed
2024-12-13 12:20 DenisChenu Status resolved => in testing
2024-12-13 12:20 DenisChenu Note Added: 81623
2024-12-13 14:35 gabrieljenik Note Added: 81624
2024-12-13 15:02 DenisChenu Status in testing => ready for merge
2024-12-13 17:21 tibor.pacalat Status ready for merge => resolved
2024-12-18 15:23 LimeBot Note Added: 81656
2024-12-18 15:23 LimeBot Status resolved => closed
2024-12-18 15:23 LimeBot Bug heat 8 => 10