View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
06478 | Bug reports | Import/Export | public | 2012-08-13 13:12 | 2012-09-20 14:12 |
Reporter | DenisChenu | Assigned To | c_schmitz | ||
Priority | normal | Severity | minor | ||
Status | closed | Resolution | fixed | ||
Product Version | 2.00+ | ||||
Fixed in Version | 2.00+ | ||||
Summary | 06478: Unable to import excel survey | ||||
Description | When i try to import excel survey , i have: PHP notice | ||||
Steps To Reproduce | Export a survey in xls format, import the survey. Or try to import joined survey | ||||
Additional Information | I remove all accent (à/é) from survey: same think | ||||
Tags | No tags attached. | ||||
Attached Files | limesurvey_survey_probaccents-OO.txt (12,880 bytes)
class type/scale name relevance text help language validation mandatory other default same_default allowed_filetypes alphasort answer_width array_filter array_filter_exclude assessment_value category_separator display_columns display_rows dropdown_dates_year_min dropdown_dates dropdown_dates_year_max dropdown_prefix dropdown_prepostfix dropdown_separators dropdown_size dualscale_headerA dualscale_headerB em_validation_q_tip em_validation_q em_validation_sq em_validation_sq_tip equals_num_value exclude_all_others exclude_all_others_auto hidden hide_tip input_boxes location_city location_country location_defaultcoordinates location_mapheight location_mapservice location_mapwidth location_mapzoom location_nodefaultfromip location_postal location_state max_answers max_filesize max_num_of_files max_num_value max_num_value_n maximum_chars min_answers min_num_of_files min_num_value min_num_value_n multiflexible_checkbox multiflexible_max multiflexible_min multiflexible_step num_value_int_only numbers_only other_comment_mandatory other_numbers_only other_replace_text page_break prefix public_statistics random_group random_order reverse scale_export show_comment show_grand_total show_title show_totals slider_accuracy slider_default slider_layout slider_max slider_middlestart slider_min slider_rating slider_separator slider_showminmax suffix text_input_width time_limit_action time_limit_countdown_message time_limit_disable_next time_limit_disable_prev time_limit_message time_limit_message_delay time_limit_message_style time_limit_timer_style time_limit_warning_2_display_time time_limit_warning_2_message time_limit_warning_2_style time_limit_warning_2 time_limit_warning time_limit time_limit_warning_display_time time_limit_warning_message time_limit_warning_style use_dropdown S sid 452982 S owner_id 1 S admin Problème accent : èàù S active N S anonymized N S format G S savetimings N S template citronade S language fr S datestamp N S usecookie N S allowregister N S allowsave Y S autonumber_start 0 S autoredirect N S allowprev N S printanswers N S ipaddr N S refurl N S datecreated 2012-08-19 S publicstatistics N S publicgraphs N S listpublic N S htmlemail Y S sendconfirmation Y S tokenanswerspersistence N S assessments N S usecaptcha D S usetokens N S attributedescriptions a:0:{} S tokenlength 15 S showxquestions Y S showgroupinfo B S shownoanswer Y S showqnumcode X S bounceprocessing N S showwelcome Y S showprogress Y S allowjumps N S navigationdelay 0 S nokeyboard N S alloweditaftercompletion N SL surveyls_title Problème accent : è à ù fr SL surveyls_description Problème accent : è à ù fr SL surveyls_welcometext Problème accent : è à ù fr SL surveyls_endtext Problème accent : è à ù fr SL surveyls_url http:// fr SL surveyls_urldescription Problème accent : è à ù fr SL surveyls_email_invite_subj Invitation à participer à un questionnaire fr SL surveyls_email_invite "Dear {FIRSTNAME}, you have been invited to participate in a survey. The survey is titled: ""{SURVEYNAME}"" ""{SURVEYDESCRIPTION}"" To participate, please click on the link below. Sincerely, {ADMINNAME} ({ADMINEMAIL}) -----------------------------------" fr SL surveyls_email_remind_subj Rappel pour participer à un questionnaire fr SL surveyls_email_remind "Dear {FIRSTNAME}, Recently we invited you to participate in a survey. We note that you have not yet completed the survey, and wish to remind you that the survey is still available should you wish to take part. The survey is titled: ""{SURVEYNAME}"" ""{SU" fr SL surveyls_email_register_subj Confirmation d'enregistrement au questionnaire fr SL surveyls_email_register Dear {FIRSTNAME}, You, or someone using your email address, have registered to participate in an online survey titled {SURVEYNAME}. To complete this survey, click on the following URL: {SURVEYURL} If you have any questions about this survey, or if you fr SL surveyls_email_confirm_subj Confirmation de votre participation à notre questionnaire fr SL surveyls_email_confirm Dear {FIRSTNAME}, this email is to confirm that you have completed the survey titled {SURVEYNAME} and your response has been saved. Thank you for participating. If you have any further questions about this email, please contact {ADMINNAME} on {ADMINEMAI fr SL surveyls_dateformat 1 fr SL surveyls_attributecaptions a:0:{} fr SL email_admin_notification_subj Soumission de réponse pour le questionnaire {SURVEYNAME} fr SL email_admin_notification Hello, A new response was submitted for your survey '{SURVEYNAME}'. Click the following link to reload the survey: {RELOADURL} Click the following link to see the individual response: {VIEWRESPONSEURL} Click the following link to edit the individual r fr SL email_admin_responses_subj Soumission de réponse pour le questionnaire {SURVEYNAME} avec résultats fr SL email_admin_responses Hello, A new response was submitted for your survey '{SURVEYNAME}'. Click the following link to reload the survey: {RELOADURL} Click the following link to see the individual response: {VIEWRESPONSEURL} Click the following link to edit the individual r fr SL surveyls_numberformat 0 fr G Probleme accent : 汉语à/漢語 1 Problème accent : è à ù fr Q S Q1 1 Problème accent : èàù Problème accent : è à ù fr N 1 Q L QA 1 Problème accent : èàù Problème accent : è à ù fr N 1 A 0 A1 A1 Problème accent : è à ù fr Q M QSQ 1 Problème accent : è à ù Problème accent : è à ù fr N 1 SQ 0 SQ001 Problème accent : è à ù fr | ||||
Bug heat | 12 | ||||
Complete LimeSurvey version number (& build) | 130812 | ||||
I will donate to the project if issue is resolved | No | ||||
Browser | not relevant | ||||
Database type & version | Mysql 5 | ||||
Server OS (if known) | debian/linux | ||||
Webserver software & version (if known) | apache | ||||
PHP Version | PHP Version 5.3 | ||||
RC7 or 8 ? My Xampp install of RC8 120803 will execute the import of that file, the success page shows up, listing the survey contents question count and etc., but the actual survey is not accessible using the URL at the bottom. When clicked on, the return for that URL is blank below the top icon. I don't get a PHP error echo'd The last four lines of the served HTML page are the help button ping and a <p...> line and a blank line: [code] [/code]["..." added so the code shows up here...] Additionally, the newly imported survey does -not- show on the list of surveys. From which I assume that the survey id is removed due to an error I'm not seeing, at the end of the excelimportsurvey function. This same fault occurs with files which I had previously successfully imported into RC7. However, sometimes I get this error: "Invalid Survey ID" for those files. I don't recall if I 'upgraded' from 7-->8, or if I am using a full out new install of RC8 The reporter's file and my other files continue to import correctly on 192+, using Tom's enhanced version that included excel import/export |
|
abita1, Your right: upload another file, eror on the first. And another with: My server is a linux error, with UTF8. It's an real excel fole (not an OOo calc file). |
|
For iconv, with my server and in 1.92 patched: case 'iconv' : $result = @iconv('UTF-16LE', $this->_defaultEncoding, html_entity_decode($string)); And: For ExcelImportSurvey import function. I think it's a server configuration issue: But a lot of installation are on linux box. |
|
in RC8 120803, Execl Exporting an already existing Survey, and then importing that excel file, drives this error: Internal Server Error An internal error occurred while the Web server was processing your request. Please contact the webmaster to report this problem. Thank you.
|
|
I am able to replicate this error. Windows + XAMPP. PHP error: The error occurs here: C:\xampp\htdocs\LS2\application\helpers\common_helper.php(1447) There is nothing wrong with this line of code. I suspect that the error occurs much earlier, and that the survey record doesn't get stored properly in the upstream code. I am not going to be able to trace this, so am assigning back to Carsten. |
|
I can replicate the issue with the attached survey as well as any newly exported survey. Have tried to trace, but am unable to find the issue. |
|
Hello, Thinks it's server configuration, patch for one server don't work with another one. Maybe try another Class (this one is in PHP licence) |
|
I Installed RC9, and I got the same error, as my first note 20394 -- i.e., a blank page. HOWEVER....I then looked at the database, by "Check Data Integrity", and in a few cycles, the database content was completely deleted, reporting problems with groups, questions and answers. [This had test surveys, only....] For the install, I saved my 'config.php' from the RC8, deleted the contents of my LS directory, and exploded the entire RC9 7Zip there, finally dropping in the old "config.php". Did I do something wrong....? Did I need to "install", again when moving from RC8 to RC9?Update: That is weird: I went back [restarted LS], and now my test surveys show, and a data check shows no problems [still]. |
|
This used to work about 2-4 months ago, so someone must have changed something since then that broke this feature. All of these samples (http://docs.limesurvey.org/Excel+Survey+Structure&structure=English+Instructions+for+LimeSurvey) load just fine in the 1.92 patch (https://github.com/LimeSurvey/LimeSurvey/pull/7), which is the original version of the Excel import/export routines. I see that none of these load properly in 2.0 RC9 - they say no errors, but when you click "go to survey", you get a blank page and numerous database consistency errors. |
|
Tom, I try with 1.92 patch on 2 different linux server (each php5/Mysql5). 1st : iconv problem on both : just put a @iconv seems OK. On one server: On the second : don't work. Then there are different problem between server. Surely utf8 connexion between php and mysql. I think there can be different problem with pg or mssql or ... I take a try with http://phpexcel.codeplex.com/ . Maybe different problem. Dis you try functionnality on Yii with your server ? @andrie : did you try with 1.92 patch on wamp server ? Denis |
|
I fixed both issues. @Andrie: The key was that the survey ID was not valid because no survey entry was created in the first place. |
|
@Carsten : i Know, ;). Do youn replace excel_reader or put a patch on excel_reader ? |
|
Fix committed to Yii branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=9374 |
|
Both. |
|
OK for survey, OK for è, but not for é à ù and surely some other in: I upload a survey (limesurvey_survey_probaccents.xls) |
|
No problems here - limesurvey_survey_probaccents.xls imports just fine. |
|
Send 3 picture:
Same thing for answers and sub question. |
|
I can reproduce - after further tests I think the problem is within the Excel library because it gets much worse after opening and saving the .xls file with OpenOffice (where I am pretty sure that OpenOffice creates a valid XLS format). Current I am thinking about dropping the whole feature - the other option is to include phpExcel. The problem with PHPExcel is that it is almost as big as LimeSurvey itself. This situation is exactly the reason I did not want this feature to be part of the core. |
|
Attached the file after being saved in OO for reference. |
|
After further research I found out that the used Excel import library is completely bug-ridden (hundreds of open bugs since 2009) The only viable option currently is to switch to phpExcel. However I don't have the time to do this - so either someone else does it or we will have to drop this feature for now. Anyway, the feature in its current form of no use if it does not work for non-English languages or for Excel files that are not created with a particular Excel version. |
|
The other option is to just use a tab separated value format (.tsv). That way people can still use Excel to author the surveys, but don't have to be locked into a particular .xls format. |
|
Fix committed to Yii branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=9486 |
|
The core problems with excel_reader have been fixed. However, this still has trouble with the accents example. Since this works perfectly in the 1.92 extension (pull request #7), those accent problem are likely in another part of the code. |
|
Frankly speaking I have no clue why the accent à is a problem (and it seems to be as such). When sanitizing the UTF-8 by doing a $v = mb_convert_encoding($v, 'utf-8', 'utf-8'); the 'à' gets dropped from the string and it inserts correctly. Chinese and other UTF-8 chars work fine. To me it seems to be some bug in Yii or PHP . Both survey_languagesettings model and groups model use the exact same way to insert the data - but it works only in the survey_languagesettings . Maybe someone else can take a look at this because I am currently out of ideas. |
|
Fix committed to Yii branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=9487 |
|
Ok, found the issue. The problem was the line: $row = str_replace(chr(0xA0),' ',$row); which is not UTF-8 safe and causes invalid UTF-8 with the à char. It works in the 1.92 version because the line does not exist there (doh!). |
|
Great work Carsten \o/ Thanks ! |
|
Thanks for catching that oversight, Carsten. I had removed that row from the 1.92 version, but missed it when porting to Yii. Of course, the only reason it is there is trying to remove the non-breaking space character (0xC20xA0) that Excel likes to add, which breaks inserts into MySql. Regardless, glad it is finally working. |
|
LimeSurvey: Yii 3e96fa10 2012-08-17 06:20 Details Diff |
Fixed issue 06478: Unable to import excel survey Dev Fixed incorrect URL description reading in Excel reader |
Affected Issues 06478 |
|
mod - application/helpers/admin/import_helper.php | Diff File | ||
mod - application/libraries/admin/excel/excel_reader2.php | Diff File | ||
mod - tmp/.gitignore | Diff File | ||
LimeSurvey: Yii cd1590c8 2012-09-12 21:51 Details Diff |
Fixed issue 06478: Unable to import excel survey Dev Removed dependency upon excel_reader; instead read/write to tab separated value (.txt) file Dev This works fine in the 1.92 extension. Dev However, this still can has trouble (like with accents) in 2.0; but since the code changes are identical in 1.92, the source of that problem must be elsewhere (perhaps in the model->insertRecords() functions?) |
Affected Issues 06478 |
|
mod - application/controllers/admin/export.php | Diff File | ||
mod - application/controllers/admin/surveyadmin.php | Diff File | ||
mod - application/helpers/admin/import_helper.php | Diff File | ||
rm - application/libraries/admin/excel/excel_reader2.php | Diff | ||
mod - application/views/admin/survey/subview/tabImport_view.php | Diff File | ||
mod - application/views/admin/survey/surveybar_view.php | Diff File | ||
LimeSurvey: Yii c026442f 2012-09-13 04:32 Details Diff |
Fixed issue 06478: Unable to import excel survey |
Affected Issues 06478 |
|
mod - application/helpers/admin/import_helper.php | Diff File |
Date Modified | Username | Field | Change |
---|---|---|---|
2012-08-13 13:12 | DenisChenu | New Issue | |
2012-08-13 13:12 | DenisChenu | File Added: limesurvey_survey_298419-remove-accent.xls | |
2012-08-13 18:58 | abita1 | Note Added: 20394 | |
2012-08-13 19:02 | abita1 | Note Edited: 20394 | |
2012-08-13 19:03 | abita1 | Note Edited: 20394 | |
2012-08-13 19:05 | abita1 | Note Edited: 20394 | |
2012-08-13 19:54 | DenisChenu | File Deleted: limesurvey_survey_298419-remove-accent.xls | |
2012-08-13 19:55 | DenisChenu | File Added: limesurvey_survey_TEXTEXPORT-noacccent.xls | |
2012-08-13 19:57 | DenisChenu | Note Added: 20396 | |
2012-08-13 19:57 | DenisChenu | File Added: limesurvey_survey_TESTIMPORT (Excel 1997-2003).xls | |
2012-08-13 20:07 | DenisChenu | Note Added: 20397 | |
2012-08-13 21:20 | abita1 | Note Added: 20398 | |
2012-08-14 12:25 | c_schmitz | Assigned To | => user9189 |
2012-08-14 12:25 | c_schmitz | Status | new => assigned |
2012-08-16 22:53 | andrie | Note Added: 20489 | |
2012-08-16 22:58 |
|
Assigned To | user9189 => c_schmitz |
2012-08-16 22:59 |
|
Note Added: 20490 | |
2012-08-16 22:59 |
|
Status | assigned => confirmed |
2012-08-17 01:58 | DenisChenu | Note Added: 20494 | |
2012-08-17 02:30 | abita1 | Note Added: 20495 | |
2012-08-17 02:32 | abita1 | Note Edited: 20495 | |
2012-08-17 02:37 | abita1 | Note Edited: 20495 | |
2012-08-17 05:09 | TMSWhite | Note Added: 20496 | |
2012-08-17 12:03 | DenisChenu | Note Added: 20497 | |
2012-08-17 13:24 | c_schmitz | Note Added: 20498 | |
2012-08-17 13:24 | c_schmitz | Status | confirmed => resolved |
2012-08-17 13:24 | c_schmitz | Fixed in Version | => 2.00+ |
2012-08-17 13:24 | c_schmitz | Resolution | open => fixed |
2012-08-17 13:27 | DenisChenu | Note Added: 20499 | |
2012-08-17 13:47 | c_schmitz | Changeset attached | => LimeSurvey Yii 3e96fa10 |
2012-08-17 13:47 | c_schmitz | Note Added: 20501 | |
2012-08-17 13:48 | c_schmitz | Note Added: 20502 | |
2012-08-17 13:55 | c_schmitz | Note Edited: 20498 | |
2012-08-19 20:12 | DenisChenu | Note Added: 20535 | |
2012-08-19 20:12 | DenisChenu | Status | resolved => feedback |
2012-08-19 20:12 | DenisChenu | Resolution | fixed => reopened |
2012-08-19 20:12 | DenisChenu | File Added: limesurvey_survey_probaccents.xls | |
2012-08-19 20:13 | DenisChenu | Note Edited: 20535 | |
2012-08-22 23:44 | c_schmitz | Assigned To | c_schmitz => DenisChenu |
2012-08-22 23:44 | c_schmitz | Status | feedback => assigned |
2012-08-22 23:47 | c_schmitz | Note Added: 20570 | |
2012-08-22 23:47 | c_schmitz | Status | assigned => feedback |
2012-08-23 12:35 | DenisChenu | File Added: Capture du 2012-08-23 12:16:01.png | |
2012-08-23 12:35 | DenisChenu | File Added: Capture du 2012-08-23 12:16:27.png | |
2012-08-23 12:35 | DenisChenu | File Added: Capture du 2012-08-23 12:17:15.png | |
2012-08-23 12:37 | DenisChenu | Note Added: 20578 | |
2012-08-23 12:37 | DenisChenu | Status | feedback => assigned |
2012-08-23 12:37 | DenisChenu | Assigned To | DenisChenu => c_schmitz |
2012-09-12 20:24 | c_schmitz | Note Added: 20659 | |
2012-09-12 20:25 | c_schmitz | Note Edited: 20659 | |
2012-09-12 20:28 | c_schmitz | File Deleted: limesurvey_survey_TEXTEXPORT-noacccent.xls | |
2012-09-12 20:28 | c_schmitz | File Deleted: limesurvey_survey_TESTIMPORT (Excel 1997-2003).xls | |
2012-09-12 20:28 | c_schmitz | File Added: limesurvey_survey_probaccents-OO.xls | |
2012-09-12 20:29 | c_schmitz | Note Added: 20660 | |
2012-09-12 21:20 | c_schmitz | Assigned To | c_schmitz => TMSWhite |
2012-09-12 21:24 | c_schmitz | Note Added: 20661 | |
2012-09-12 22:28 | TMSWhite | Note Added: 20663 | |
2012-09-13 04:52 | TMSWhite | Changeset attached | => LimeSurvey Yii cd1590c8 |
2012-09-13 04:52 | TMSWhite | Note Added: 20666 | |
2012-09-13 05:12 | TMSWhite | Note Added: 20667 | |
2012-09-13 05:12 | TMSWhite | Status | assigned => resolved |
2012-09-13 05:12 | TMSWhite | Resolution | reopened => fixed |
2012-09-13 11:06 | c_schmitz | Assigned To | TMSWhite => c_schmitz |
2012-09-13 11:06 | c_schmitz | Status | resolved => assigned |
2012-09-13 11:10 | c_schmitz | Note Added: 20668 | |
2012-09-13 11:11 | c_schmitz | File Added: limesurvey_survey_probaccents-OO.txt | |
2012-09-13 11:32 | c_schmitz | Changeset attached | => LimeSurvey Yii c026442f |
2012-09-13 11:32 | c_schmitz | Note Added: 20669 | |
2012-09-13 11:33 | c_schmitz | Note Added: 20670 | |
2012-09-13 11:34 | c_schmitz | Status | assigned => resolved |
2012-09-13 11:52 | DenisChenu | Note Added: 20671 | |
2012-09-13 13:03 | TMSWhite | Note Added: 20672 | |
2012-09-20 14:12 | c_schmitz | Status | resolved => closed |