View Issue Details

This bug affects 1 person(s).
 4
IDProjectCategoryView StatusLast Update
03967Bug reportsOtherpublic2009-12-11 12:16
Reporteruser4084Assigned Toc_schmitz  
PrioritynormalSeveritypartial_block 
Status closedResolutionfixed 
Product Version1.85+ 
Fixed in Version1.87RC5 
Summary03967: Native Languages support in csv file export and import
Description

i discovered a bug in CSV File export and import regards Arabic and possibly Japanese Language.

Most of us use Microsoft Excel to manipulate CSV files
The problems are:

1- comma as a separator are different with localizations in windows operating system, some use semicolon(default in arabic) and some use other characters
so you either patch the user's windows system , or patch the limesurvey it self to export semicolon separated values

2- Limesurvey exports UTF-8 Without BOM encoded CSV Files, Excell only can read UTF WITH BOM (BYTE ORDER MARK) Correctly. This doesn't affect english only foregin characters e.g. arabic language

3- Excel Exports CSV Files as Windows/Dos File format. in Windows Encoding (cp1256 in arabic language) a special Converter is required to convert this to utf-8.

Additional Information

SOLUTIONS:

1- Use This Reg File To patch client system

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Control Panel\International]
"sList"=","

2- patch limesurvey
File: admin/exportresults.php
line: 864
code:

// Export UTF8 WITH BOM
$firstline= chr(hexdec('EF')).chr(hexdec('BB')).chr(hexdec('BF')).$firstline;

File: admin/tokens.php
line : 155
code:

// Export UTF8 WITH BOM
$tokenoutput=chr(hexdec('EF')).chr(hexdec('BB')).chr(hexdec('BF')).$tokenoutput;

3- there are an open source arabic php projext at
(http://www.ar-php.org/en_index_php_arabic.html)
that allows correct code page conversions

add the arabic package to your site and insert this code to convert the input to utf

For import vv functions
file: vvimport.php
Line : 146
//Correct import arabic issue
$uploadcharset='cp1256';
include_once('Arabic.php');
$Arabic = new Arabic('ArCharsetC');

  $Arabic->setInputCharset('windows-1256'); 

  $Arabic->setOutputCharset('utf-8'); 
  $charset = $Arabic->getOutputCharset(); 
  $bigarray[]= $Arabic->convert($buffer)

You can do the same thing in token file for csv token import

TagsNo tags attached.
Bug heat4
Complete LimeSurvey version number (& build)1.85
I will donate to the project if issue is resolved
BrowserALL
Database type & versionXAMPP Windows 1.70
Server OS (if known)Windows 7 7600 Ultimate
Webserver software & version (if known)APACHE
PHP VersionPHP Version 5.2.8

Relationships

related to 03968 closedc_schmitz Arabic Support in Grpahs Legend 

Users monitoring this issue

There are no users monitoring this issue.

Activities

user372

2009-12-07 22:11

  ~10507

@ c_schmitz: please have a look at the suggested modifications!

c_schmitz

c_schmitz

2009-12-08 02:35

administrator   ~10514

I implemented suggestion 2.) - Thank you!

VVIMport is only designed to import/export UTF-8 therefore we won't convert anything there.

Tokens import already has a cp1256 input filter

Thank you very much!

Issue History

Date Modified Username Field Change
2009-12-07 22:09 user4084 New Issue
2009-12-07 22:09 user4084 Status new => assigned
2009-12-07 22:09 user4084 Assigned To => user372
2009-12-07 22:09 user4084 LimeSurvey build number => 1.85
2009-12-07 22:09 user4084 Browser => ALL
2009-12-07 22:09 user4084 Database & DB-Version => XAMPP Windows 1.70
2009-12-07 22:09 user4084 Operating System (Server) => Windows 7 7600 Ultimate
2009-12-07 22:09 user4084 Webserver => APACHE
2009-12-07 22:09 user4084 PHP Version => PHP Version 5.2.8
2009-12-07 22:11 user372 Note Added: 10507
2009-12-07 22:11 user372 Assigned To user372 => c_schmitz
2009-12-07 23:10 user372 Relationship added related to 03968
2009-12-08 02:35 c_schmitz Note Added: 10514
2009-12-08 02:35 c_schmitz Status assigned => resolved
2009-12-08 02:35 c_schmitz Fixed in Version => 1.87RC5
2009-12-08 02:35 c_schmitz Resolution open => fixed
2009-12-08 13:37 gwenguillaume Issue Monitored: gwenguillaume
2009-12-08 13:37 gwenguillaume Issue End Monitor: gwenguillaume
2009-12-11 12:16 c_schmitz Status resolved => closed
2010-05-06 10:27 c_schmitz Category Import / Export => (No Category)