View Issue Details

IDProjectCategoryView StatusLast Update
15679Feature requests[All Projects] Installationpublic2020-01-14 16:14
Reporterdschirge Assigned To 
PrioritynoneSeverityfeature 
Status newResolutionopen 
Product Version 
Target VersionFixed in Version 
Summary15679: Creation of database migration scripts
Description

hi guys,

we currently are migrating some old LS 1.9 installations to LS 3. Because of missing tables it's a pretty long and rough way.
Yii 1 provides migration classes, described in https://www.yiiframework.com/doc/guide/1.1/en/database.migration#applying-migrationse
For LS 4 and LS 5 it appears to be necessary for creating such migrations as you guys planned for doing some biger database changes.

TagsNo tags attached.

Activities

Mazi

Mazi

2019-12-19 14:37

partner   ~55081

So just to give an insight of the currently faced issues: We tried updating a large Limesurvey system (DB size: 4.5GB) from 1.92 to the latest 3.21.2 version and totally failed. We wasted some hours trying to debug errors before going from version to version in small steps which finally worked fine.

This is what happened:

  1. Test: Updating directly to 3.21.2
    -> Error when updating DB: "Alias "application.helpers.expressions.warnings.EMWarningInvalidComparison" is invalid.", see bug report at https://bugs.limesurvey.org/view.php?id=15677. Not sure if this is a different issue.

  2. Test: Use the previous 3.20.0 version (also test 3.19)
    -> Error: Table {{permissions}} for ActiveRecord class "Permission" not found at DB. That's true since the "permissions" table didn't exist at 1.92. But the updater didn't notice that, though special updating code is there at https://github.com/LimeSurvey/LimeSurvey/blob/c0c0c6e9abaaf2f59f0b22aea2409d618b690b9d/application/helpers/update/updatedb_helper.php#L1018.

  3. Test: Try an older 2.73 version
    -> Same issues as #2 above

  4. Test: Try an older 2.05 version
    -> Same issues as #2 above

  5. Test: Test with 2.00
    -> Finally, the DB updater was able to update to DB version 164

  6. Go from 2.00 to 2.05
    -> The updater worked fine and created the new permission table when updating to DB version 178

  7. Go from 2.05 to 2.73
    -> Updated to DB version 263

  8. Go from 2.73 to 3.19
    -> Updated to DB version 359

  9. Lastly, we updated from 3.19 to 3.21.2 using ComfortUpdate

That took several hours including yelling at the PC and searching documentation for possible solutions.

Mazi

Mazi

2019-12-19 14:39

partner   ~55082

Assuming that there will be huge DB adjustments when going from LS 4 to LS 5 (more object oriented question models, new DB schema for strong responses in a non-flat-table system, ...) we should find better ways to run such updates.

Feel free to ping us/David if you want to discuss using Yii migration classes.

Mazi

Mazi

2019-12-19 14:40

partner   ~55083

@DenisChenu, have you ever faced similar problems when updating stone-old systems?

What do you suggest to provide better updating features?

DenisChenu

DenisChenu

2019-12-19 14:48

developer   ~55084

Error: Table {{permissions}} for ActiveRecord class "Permission" not found at DB

I think it was managed in previous system ?

What do you suggest to provide better updating features?

I already say we need update $to : then we can upgrade really step by step.

Using Yii system seem great too, but REALLY need more work (we need to create whole update system from 1.80 …)

ollehar

ollehar

2020-01-14 15:15

administrator   ~55233

Test: Updating directly to 3.21.2

Did you clear cache after update?

Mazi

Mazi

2020-01-14 15:27

partner   ~55236

@ollehar: I do not really remember if I cleared the cache but at other updates I noticed that not clearing cache can indeed cause weird issues like being unable to log in.

ollehar

ollehar

2020-01-14 16:14

administrator   ~55242

@Mazi, to be precise, I'm talking about the Yii runtime cache. Hm, wait, I remember someone else talking about that problem. Strange.

Issue History

Date Modified Username Field Change
2019-12-19 13:30 dschirge New Issue
2019-12-19 14:37 Mazi Note Added: 55081
2019-12-19 14:39 Mazi Note Added: 55082
2019-12-19 14:40 Mazi Note Added: 55083
2019-12-19 14:48 DenisChenu Note Added: 55084
2020-01-14 15:15 ollehar Note Added: 55233
2020-01-14 15:27 Mazi Note Added: 55236
2020-01-14 16:14 ollehar Note Added: 55242