View Issue Details

This bug affects 1 person(s).
 6
IDProjectCategoryView StatusLast Update
15792Bug reportsInstallationpublic2023-09-27 18:21
Reportertbart Assigned Toc_schmitz  
PrioritynormalSeveritypartial_block 
Status closedResolutionreopened 
Product Version4.0.1 
Summary15792: Database upgrade errors DB version 359>425 (Incorrect string value, data too long for column)
Description

When executing the DB update, I get several errors.
The first one:
CDbCommand konnte das SQL-Statement nicht ausführen: SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xE2\x80\x8BWel...'
can be fixed by
update questions set question=replace(question,unhex('E2808B'),'');

These characters seem to stem from copying texts from word.

The next error,
Data too long for column 'code' at row
is not so easy to fix.

It seems we have longer "code"s than expected?
mysql> select code, length(code) as codelen from answers order by codelen desc limit 10;
+------+---------+
| code | codelen |
+------+---------+
| A1.2 | 4 |
| A1.1 | 4 |
| A1.3 | 4 |
| A1.4 | 4 |
| A1.5 | 4 |
| A1.6 | 4 |
| A19 | 3 |
| A14 | 3 |
| A18 | 3 |
| A17 | 3 |
+------+---------+
10 rows in set (0.01 sec)

The updater should fix both errors.
Or alternatively, how can I fix the DB in the meantime?

Steps To Reproduce

Update from 3.16.1 to 4.0.1 having mentioned code specifications and unicode characters in strings.

TagsNo tags attached.
Bug heat6
Complete LimeSurvey version number (& build)4.0.1
I will donate to the project if issue is resolvedNo
Browser
Database type & versionmysql Ver 14.14 Distrib 5.7.29
Server OS (if known)Ubuntu 18.04.3 LTS
Webserver software & version (if known)
PHP VersionPHP 7.2.24

Users monitoring this issue

There are no users monitoring this issue.

Activities

cdorin

cdorin

2021-02-04 10:13

reporter   ~61931

Hey there, is it possible to test again the upgrade process and see which issues are popping up? Fixes were pushed in LS 4.4.x

tbart

tbart

2021-02-09 16:05

reporter   ~62083

Last edited: 2021-02-16 12:23

Sure. Tested again today, from 3.25.x to 4.4.4.

I can confirm the special char encoding bug is gone!

However, I still get

CDbCommand konnte das SQL-Statement nicht ausführen: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'code' at row 291. The SQL statement executed was: INSERT INTO labels
(lid, code, sortorder, assessment_value)
SELECT lid, code, min(sortorder), min(assessment_value)
FROM labels_update400
GROUP BY lid, code

The respective table content looks like this:

MariaDB [eval]> SELECT lid, code, min(sortorder), min(assessment_value) FROM labels_update400 GROUP BY lid, code limit 25;
+-----+-----------------+----------------+-----------------------+
| lid | code | min(sortorder) | min(assessment_value) |
+-----+-----------------+----------------+-----------------------+
| 1 | engagiert | 2 | 0 |
| 1 | erklaertgut | 1 | 0 |
| 1 | gehtaufBeduerfn | 3 | 0 |
| 1 | vorbereitet | 0 | 0 |
| 2 | A1 | 0 | 0 |
| 2 | A10 | 9 | 0 |
| 2 | A2 | 1 | 0 |
| 2 | A3 | 2 | 0 |
| 2 | A4 | 3 | 0 |
| 2 | A5 | 4 | 0 |
| 2 | A6 | 5 | 0 |
| 2 | A7 | 6 | 0 |
| 2 | A8 | 7 | 0 |
| 2 | A9 | 8 | 0 |
| 3 | A1 | 0 | 0 |
| 3 | A2 | 1 | 0 |
| 3 | A3 | 2 | 0 |
| 3 | A4 | 3 | 0 |
| 3 | A5 | 4 | 0 |
| 4 | SQ001 | 0 | 0 |
| 4 | SQ002 | 1 | 0 |
| 4 | SQ003 | 2 | 0 |
| 4 | SQ004 | 3 | 0 |
| 5 | SQ001 | 0 | 0 |
| 5 | SQ002 | 4 | 0 |
+-----+-----------------+----------------+-----------------------+

cdorin

cdorin

2021-02-09 16:39

reporter   ~62085

Last edited: 2021-02-16 12:23

Thx - will get it in our backlog :)

c_schmitz

c_schmitz

2021-02-15 13:48

administrator   ~62243

Last edited: 2021-02-16 12:23

I cannot reproduce the issue. THe label code should never be longer than 5 chars and this limitation has been in the database forever.
Most likely you have modified the table to allow for longer codes - might that be?

tbart

tbart

2021-02-16 12:23

reporter   ~62283

Last edited: 2021-02-16 12:23

This is a vanilla install dating back to 2016 approx., no plugins whatsoever.
Setup on 2.x if I remember correctly.

No, the DB has never been touched, this must have been possible via the standard web GUI.

I can adapt this for this installation, I guess (hope it does not need changes in other locations?).

Still - this might not be the only installation where this happened and the fix should be in the upgrade process, i.e. shorten to 5 chars, see whether the label is still unique.

Column "code" is defined as varchar(15), so the application should support that. If this is not intended, it should be reduced to varchar(5) so we do not end up with unexpectedly long data in the DB. Better throw an error right when the application tries to write beyond bounds not expected than write unsupported data to the DB!

c_schmitz

c_schmitz

2021-02-16 17:32

administrator   ~62294

I checked back our source code to the beginning of LimeSurvey. The code never had a length of 15, it has been always 5 since LimeSurvey was created. This must have been manually modified.
I will close this issue now.

tbart

tbart

2023-09-27 18:21

reporter   ~77387

As a note to myself and anyone running into the same issue:
There do not seem to be any more references to these codes in the DB, so they can be shortened.

https://github.com/LimeSurvey/LimeSurvey/blob/25f63d5758134ad5eca607453deb8a9e94b7f2bd/installer/create-database.php#L177 creates the column with string(20) now, but this happened only in DB version 483.
If longer codes are needed, they can be restored on LS5.x or LS6.x.
(Though I still don't know why mine where longer.. Don't need this, actually)

Issue History

Date Modified Username Field Change
2020-01-28 17:11 tbart New Issue
2020-02-05 08:54 cdorin Assigned To => cdorin
2020-02-05 08:54 cdorin Status new => assigned
2021-02-04 10:13 cdorin Assigned To cdorin =>
2021-02-04 10:13 cdorin Status assigned => feedback
2021-02-04 10:13 cdorin Note Added: 61931
2021-02-09 16:05 tbart Note Added: 62083
2021-02-09 16:05 tbart Status feedback => new
2021-02-09 16:39 cdorin Priority none => normal
2021-02-09 16:39 cdorin Status new => confirmed
2021-02-09 16:39 cdorin Sync to Zoho Project => |Yes|
2021-02-09 16:39 cdorin Note Added: 62085
2021-02-15 13:48 c_schmitz Assigned To => c_schmitz
2021-02-15 13:48 c_schmitz Status confirmed => feedback
2021-02-15 13:48 c_schmitz Note Added: 62243
2021-02-15 13:53 c_schmitz Status feedback => closed
2021-02-15 13:53 c_schmitz Resolution open => unable to reproduce
2021-02-16 12:23 tbart Status closed => feedback
2021-02-16 12:23 tbart Resolution unable to reproduce => reopened
2021-02-16 12:23 tbart Note Added: 62283
2021-02-16 17:32 c_schmitz Status feedback => closed
2021-02-16 17:32 c_schmitz Note Added: 62294
2021-02-18 14:17 c_schmitz Sync to Zoho Project Yes =>
2023-09-27 18:21 tbart Note Added: 77387