View Issue Details

This bug affects 1 person(s).
 10
IDProjectCategoryView StatusLast Update
02713Bug reportsOtherpublic2009-11-13 16:40
Reporterlime_junkie Assigned Tomdekker  
PrioritynormalSeveritypartial_block 
Status closedResolutionfixed 
Product Version1.72 
Fixed in Version1.87RC1 
Summary02713: SPSS export of data is in a strange order
Description

Export of spss data file seems to work ok, but when the file is opened in spss the order of questions is not the order in which they are in the survey.

Question numbers in the survey are in order, but not in the data file. Maybe this is an issue AFTER the question order is changed in the survey construction, which i think happened during the construction of this survey.

IE when a new question is created, it is given an QID by the system. However, if later, once all the questions are added, the order is changed of a couple of questions, maybe the export is then incorrect. The response data is correct it seems, just the question order in the spss file. But this maybe worth checking also.

TagsNo tags attached.
Attached Files
patch.diff (3,205 bytes)   
### Eclipse Workspace Patch 1.0
#P Limesurvey_SVN
Index: admin/export_data_functions.php
===================================================================
--- admin/export_data_functions.php	(revision 7796)
+++ admin/export_data_functions.php	(working copy)
@@ -54,12 +54,18 @@
 
 	$result=db_execute_num($query) or safe_die("Couldn't get results<br />$query<br />".$connect->ErrorMsg()); //Checked
 	$num_fields = $result->FieldCount();
-
-	while ($row = $result->FetchRow()) {
-		$fieldno = 0;
-		while ($fieldno < $num_fields)
+	
+	//This shouldn't occur, but just to be safe:
+	if (count($fields)<>$num_fields) safe_die("Database inconsistency error");
+	
+	while (!$result->EOF) {
+		$row = $result->GetRowAssoc(true);	//Get assoc array, use uppercase
+		reset($fields);	//Jump to the first element in the field array
+		$i = 1;
+		foreach ($fields as $field)
 		{
-			if ($fields[$fieldno]['SPSStype']=='DATETIME23.2'){
+			$fieldno = strtoupper($field['sql_name']);
+			if ($field['SPSStype']=='DATETIME23.2'){
 				#convert mysql  datestamp (yyyy-mm-dd hh:mm:ss) to SPSS datetime (dd-mmm-yyyy hh:mm:ss) format
 				if (isset($row[$fieldno]))
 				{
@@ -75,7 +81,7 @@
 				{
 					echo ($na);
 				}
-			} else if ($fields[$fieldno]['LStype'] == 'Y')
+			} else if ($field['LStype'] == 'Y')
 			{
 				if ($row[$fieldno] == 'Y')    // Yes/No Question Type
 				{
@@ -85,7 +91,7 @@
 				} else {
 					echo($na);
 				}
-			} else if ($fields[$fieldno]['LStype'] == 'G')    //Gender
+			} else if ($field['LStype'] == 'G')    //Gender
 			{
 				if ($row[$fieldno] == 'F')
 				{
@@ -95,7 +101,7 @@
 				} else {
 					echo($na);
 				}
-			} else if ($fields[$fieldno]['LStype'] == 'C')    //Yes/No/Uncertain
+			} else if ($field['LStype'] == 'C')    //Yes/No/Uncertain
 			{
 				if ($row[$fieldno] == 'Y')
 				{
@@ -107,7 +113,7 @@
 				} else {
 					echo($na);
 				}
-			} else if ($fields[$fieldno]['LStype'] == 'E')     //Increase / Same / Decrease
+			} else if ($field['LStype'] == 'E')     //Increase / Same / Decrease
 			{
 				if ($row[$fieldno] == 'I')
 				{
@@ -119,7 +125,7 @@
 				} else {
 					echo($na);
 				}
-			} elseif (($fields[$fieldno]['LStype'] == 'P' || $fields[$fieldno]['LStype'] == 'M') && (substr($fields[$fieldno]['code'],-7) != 'comment' && substr($fields[$fieldno]['code'],-5) != 'other'))
+			} elseif (($field['LStype'] == 'P' || $field['LStype'] == 'M') && (substr($field['code'],-7) != 'comment' && substr($field['code'],-5) != 'other'))
 			{
 				if ($row[$fieldno] == 'Y')
 				{
@@ -128,7 +134,7 @@
 				{
 					echo("'0'");
 				}
-			} elseif (!$fields[$fieldno]['hide']) {
+			} elseif (!$field['hide']) {
 				$strTmp=mb_substr(strip_tags_full($row[$fieldno]), 0, $length_data);
 				if (trim($strTmp) != ''){
 					$strTemp=str_replace(array("'","\n","\r"),array("''",' ',' '),trim($strTmp));
@@ -145,10 +151,11 @@
 					echo $na;
 				}
 			}
-			$fieldno++;
-			if ($fieldno<$num_fields && !$fields[$fieldno]['hide']) echo ',';
+			if ($i<$num_fields && !$field['hide']) echo ',';
+			$i++;
 		}
 		echo "\n";
+		$result->MoveNext();
 	}
 }
 
patch.diff (3,205 bytes)   
Bug heat10
Complete LimeSurvey version number (& build)Version 1.72 (5737)
I will donate to the project if issue is resolved
BrowserNA
Database type & versionMySQL 5
Server OS (if known)linux
Webserver software & version (if known)10341100
PHP Version5.2.3

Users monitoring this issue

There are no users monitoring this issue.

Activities

user372

2008-12-04 14:58

  ~06496

Last edited: 2008-12-04 14:59

Hi, can you reproduce the problem?
If yes, please tell us step-by-step how we can reproduce the problem.
Would be nice, if you could also provide screenshot and a sample survey (survey structure as .csv-file).

Can you reproduce the problem on v1.80RC2 - some things on SPSS export changed between v1.72 and v1.80!!!

lime_junkie

lime_junkie

2008-12-04 19:45

reporter   ~06497

will give that a try ASAP as i have 1.80rc2 on another box. Will let you know.
will include sample as suggested.

ta

user659

2008-12-05 09:28

  ~06499

@ lime_junkie: Can you confirm that the exported questions are ordered by question groups if you're using group-by-group questions and then by question code (as displayed when selecting questions for editing)? This order probably matches the order of your Variable Labels in the SPSS syntax file.

Regards.

lime_junkie

lime_junkie

2008-12-12 15:34

reporter   ~06516

Ive uploaded an SPSS error that came about when trying to import the spss file from the survey.

I was trying to test if the 'out of order problem' happened on 1.80RC1 (5885), but for some reason, i could not import to spss.

Ive enclose the spss files, the survey file, a pdf containing the error that came up. Im using spss 15

c_schmitz

c_schmitz

2008-12-14 22:35

administrator   ~06522

Limesurvey's SPSS export is optimized for working with SPSS v16. If you are running older versions and want to open exported data you have to delete "SET UNICODE=ON" from the syntax and then exchange variable formats that don't work with the older version with ones that do.

c_schmitz

c_schmitz

2008-12-14 22:37

administrator   ~06523

LimeSurvey exports data to SPSS in the same order as the survey was in when it got activated. Changing that would be too much of an effort for being purely cosmetic.

c_schmitz

c_schmitz

2009-11-01 13:07

administrator   ~09936

Menno, do you think you can change this as our SPSS expert?

mdekker

mdekker

2009-11-02 09:25

reporter   ~09947

Is there any chance that you can change order after you activated? I thought that was prohibited.

If that is possible, the variables and data can get out of sync. I can change the export to read the export data as an associative array and then match it to the sorted fieldlist. This should prevent any bugs when sort order is changed after activation. I'll work on a quick fix and submit the patch here so it can be tested before I commit it.

mdekker

mdekker

2009-11-02 10:13

reporter   ~09949

I attached a patch. I think the createFieldMap function always uses the correct group and question order. When exporting the data I read an associative array and export it in the order as defined in the fieldmap. The only fix needed then would be to sort the fieldmap if that is not correct.

I am going on a holiday in two days so I won't be able to test this patch myself but in my local install it seemed to work fine. Please review and commit if it is ok.

c_schmitz

c_schmitz

2009-11-02 17:13

administrator   ~09957

Patch looks fine to me - applied it.

Thank you very much. Great help!!

Issue History

Date Modified Username Field Change
2008-12-04 14:45 lime_junkie New Issue
2008-12-04 14:45 lime_junkie Status new => assigned
2008-12-04 14:45 lime_junkie Assigned To => user372
2008-12-04 14:45 lime_junkie LimeSurvey Build Number (only Plus versions) => Version 1.72 (5737)
2008-12-04 14:45 lime_junkie Browser => NA
2008-12-04 14:45 lime_junkie Database & DB-Version => MySQL 5
2008-12-04 14:45 lime_junkie Operating System (Server) => linux
2008-12-04 14:45 lime_junkie PHP Version => 5.2.3
2008-12-04 14:45 lime_junkie Webserver => 10341100
2008-12-04 14:58 user372 Note Added: 06496
2008-12-04 14:58 user372 Status assigned => feedback
2008-12-04 14:59 user372 Note Edited: 06496
2008-12-04 19:45 lime_junkie Note Added: 06497
2008-12-05 09:28 user659 Note Added: 06499
2008-12-12 15:29 lime_junkie File Added: SPSS Text Output in Output1 [Document1].pdf
2008-12-12 15:29 lime_junkie File Added: limesurvey_survey_33673.csv
2008-12-12 15:30 lime_junkie File Added: survey_33673_SPSS_data_file.dat
2008-12-12 15:30 lime_junkie File Added: survey_33673_SPSS_syntax_file.sps
2008-12-12 15:34 lime_junkie Note Added: 06516
2008-12-12 21:05 user372 Status feedback => assigned
2008-12-12 21:05 user372 Assigned To user372 => c_schmitz
2008-12-14 22:35 c_schmitz Note Added: 06522
2008-12-14 22:37 c_schmitz Note Added: 06523
2008-12-14 22:37 c_schmitz Status assigned => closed
2008-12-14 22:37 c_schmitz Resolution open => won't fix
2009-11-01 13:04 c_schmitz Status closed => assigned
2009-11-01 13:04 c_schmitz Assigned To c_schmitz => mdekker
2009-11-01 13:07 c_schmitz Note Added: 09936
2009-11-02 09:25 mdekker Note Added: 09947
2009-11-02 10:09 mdekker File Added: patch.diff
2009-11-02 10:13 mdekker Note Added: 09949
2009-11-02 10:13 mdekker Assigned To mdekker => c_schmitz
2009-11-02 10:13 mdekker Status assigned => feedback
2009-11-02 17:12 c_schmitz Status feedback => assigned
2009-11-02 17:12 c_schmitz Assigned To c_schmitz => mdekker
2009-11-02 17:13 c_schmitz Status assigned => resolved
2009-11-02 17:13 c_schmitz Fixed in Version => 1.87RC1
2009-11-02 17:13 c_schmitz Note Added: 09957
2009-11-02 17:13 c_schmitz Resolution won't fix => fixed
2009-11-13 16:40 c_schmitz Status resolved => closed
2010-05-06 10:27 c_schmitz Category Import / Export => (No Category)