View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 20516 | Bug reports | Installation | public | 2026-05-19 12:57 | 2026-06-17 19:17 |
| Reporter | riqcles | Assigned To | |||
| Priority | none | Severity | crash | ||
| Status | new | Resolution | open | ||
| Product Version | 7.0beta x | ||||
| Summary | 20516: Updating an existing database failed - Postgres | ||||
| Description | To be done in a development environment. Choose a LimeSurvey 6.x version with several thousand questionnaires. Install a LimeSurvey 7 beta version. Modify the config.ini file of version 7 to point to the database of version 6. The following errors may appear:
In my case, the second error occurred after I deleted 1,500 questionnaires. It would be good to be able to create a log file to record these errors for analysis (and then disable this file) during an update. | ||||
| Steps To Reproduce | Steps to reproduce(Replace this text with detailed step-by-step instructions on how to reproduce the issue) Expected result(Write here what you expected to happen) Actual result(Write here what happened instead) | ||||
| Tags | No tags attached. | ||||
| Bug heat | 10 | ||||
| Complete LimeSurvey version number (& build) | Limesurvey 7 Beta | ||||
| I will donate to the project if issue is resolved | No | ||||
| Browser | |||||
| Database type & version | Postgresql 17 | ||||
| Server OS (if known) | |||||
| Webserver software & version (if known) | Rocky Linux | ||||
| PHP Version | PHP Version 8.2.4 | ||||
|
Thanks for reporting the issue. Will raise the idea to have logs in the team. In the meantime, can you provide further information? I would need:
|
|
|
LimeSurvey Community Edition Version 7.0.0-beta1+260121
[root@serveurIntranet httpd]# php /var/www/html/limesurvey7/application/commands/console.php updatedb |
|
|
Thanks for the detailed answer. Before I get into the threads we opened, let me provide some general information. This is an old-ish version from January and we changed quite a lot since then. This is a newer version with lots of fixes:https://github.com/LimeSurvey/LimeSurvey/releases/tag/7.0.0-RC1%2B260430 . We are not logging issues as you correctly pointed out and I raised this idea in the team. As about your responses:
What happens under the hood:
So some fields representing some answers you removed are still in the responses table and the script attempts to convert them and finds no answer in the {{answers}} table to convert. (in current version it would be subquestion in the {{questions}} table). There could be other inconsistencies from some old tables that I cannot predict at this time. So a way to solve it would be to count the answers of ranking questions versus fields in the responses table, like select q.sid, count(*) vs. select count(*) You can have these as subqueries and join by qid and a HAVING clause for the group by where the count(*) differs at the two sides. Those are potential issues. I presumed here your table prefix was lime_ . If it's something different, then you can adjust the queries. In this version, line 1471 of common_helper.php is $aid = $cd ? $index : $questions[0]->answers[(substr($fieldName, strlen("{$sid}X{$gid}X{$qid}")) - 1)]->aid; and the two arrays potentially causing the problem are:
and it is very likely that the {{answers}} table has not enough answers for the responses table to map to.
$aid = $cd ? $index : $questions[0]->answers[((int)substr($fieldName, strlen("{$sid}X{$gid}X{$qid}")) - 1)]->aid; may help as long as the substring ends up being numeric, but that's hacky, a newer version could be a better point of departure.
$suffixText .= "_S" . $scales[$index]; and apparently there are no scales for $index, so your data is inconsistent there, but it's a different table.I asked the team about logging, there is no decision yet. If you wish to debug and log in the meantime, then here: https://github.com/LimeSurvey/LimeSurvey/blob/68e30d06a8dabd9611c6c4cab16c1abe86e12de7/application/helpers/update/updates/Update_700.php#L1119 You find
you can do
to see what table it attempted to create just before crashing. And then you can send that definition as well as the actual table structure in your database, note that we rename the tables, so your {{survey_123456}} is renamed to {{responses_123456}} and your {{survey_123456_timings}} is renamed to {{timings_123456}} |
|
|
Thank you for the information. I'm not a developer, so it will be difficult for me to carry out all these steps. As you indicated, it's probably old questionnaires and response tables that are causing the problem. Since I can't run the integrity check (due to the data volume), there has been an accumulation of incorrect data (orphaned tables). The scenario you described is entirely possible: I have several hundred managers who create dozens (or more) of questionnaires per week. It's impossible to keep track of them all :) I think this bug can be fixed. It provides a starting point (and goodness knows you have plenty of work to do: thank you to the team) that we can address in a near-final version in September/October? Otherwise, I modified my configuration to rerun the tests:
I'm increasing the file server memory from 2 GB to 16 GB. (The database server remains at 2 GB and Postgres 17.7.) I'm removing php-fpm to be as close as possible to my environments. I modify the php.ini file by adding: max_input_time = 300 I enable 'debug'=>2 in the config file. And I rerun the update via the command line. It's longer, but at the end this: CException: Please fix this error in your database and try again in /var/www/html/limesurvey7/application/commands/UpdateDbCommand.php:51 But I have the application.log file in the runtime which is more verbose: 20 MB 2026/05/19 19:15:41 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[42601]: Syntax error: 7 ERROR: zero-length delimited identifier at or near """" ^. The SQL statement executed was: CREATE TABLE lime_old_responses_114533_2021020511460 I have this 6714 times in my file: I have attached the log file compressed with WinRAR. |
|
|
Thank you for your response. The good news is that fixing this ranking behavior is approved and I'll do it today. We still did not get confirmation on whether we wish to do logs. The zero-length identifier is the empty fieldname in the create table statement. It's definitely the case in the ranking issue we discussed and that's gonna be fixed. It's also possible that some other issues with the symptoms may still exist, so with the fix I expect the number of this error at your end to be reduced to hopefully 0, but even a significant reduction would mean we removed this bug. We just don't know whether this is the last bug or not. |
|
|
I have been pointed to https://bugs.limesurvey.org/view.php?id=19237 |
|
|
@lajosarpad : here, in my opinon, there are 2 issue we can try to fix
Your opinion ? |
|
|
@DenisChenu thanks for the ideas, I'll notify the team about them. Personally I think 1. is a good idea as well as 2. As per the change issue with rankings, we have a fix already merged into develop-major at https://github.com/LimeSurvey/LimeSurvey/pull/4985 |
|
|
Yes, i know :) Did I report both issues: one for check integrity by survey + one for SQL error ? |
|
|
I'm not sure whether they were reported separately, but I notified the team about both ideas. |
|
|
Hello,
^. The SQL statement executed was: CREATE TABLE lime_old_responses_114533_20210205114609
2026/05/26 16:33:11 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[42704]: Undefined object: 7 ERROR: index "lime_idx2_users" does not exist. The SQL statement executed was: DROP INDEX "lime_idx2_users".
The redundancy check verifies the presence of tables left after deactivating questionnaires. You can delete them if you no longer need them. I couldn't check all the boxes at once: I need to check 5,495 boxes to be able to click the <Clear checked items!> button.
2026_05_26_2_V7_application.log (1,212 bytes)
2026/05/26 16:33:11 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[42704]: Undefined object: 7 ERROR: index "lime_idx2_users" does not exist. The SQL statement executed was: DROP INDEX "lime_idx2_users".
2026/05/26 16:33:11 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[25P02]: In failed sql transaction: 7 ERROR: current transaction is aborted, commands ignored until end of transaction block. The SQL statement executed was: CREATE UNIQUE INDEX "lime_idx2_users" ON "lime_users" ("email").
2026/05/26 16:33:59 [error] [exception.CException] CException: Please fix this error in your database and try again in /var/www/html/limesurvey7/application/commands/UpdateDbCommand.php:51
Stack trace:
#0 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleCommandRunner.php(71): UpdateDBCommand->run()
#1 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
#2 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/base/CApplication.php(185): CConsoleApplication->processRequest()
#3 /var/www/html/limesurvey7/application/commands/console.php(77): CApplication->run()
#4 {main}
---
|
|
|
sorry the name is lime_oldreponses not OLD_answer |
|
|
Thanks for the detailed report. Can you test what the DB version is after running the change script? You can find that out by running: select stg_value from lime_settings_global where stg_name = 'DBVersion'; |
|
|
The result is 705. In CLI : I'm going to revert to the original database before the migration, then clean up all the OLD files, and then rerun the script from the command line (I'll also try a variation by running the update via the web interface). Lots more testing to do :). |
|
|
Thanks for the information! The great news is that 700 finally succeeded as well as 701, 702, 703, 704 and 705 and it crashed at 706 for you. 705 is about creating the react editor plugin and updating the priorities: https://github.com/LimeSurvey/LimeSurvey/blob/master/application/helpers/update/updates/Update_705.php 706 is about deduplicating emails as we can see at https://github.com/LimeSurvey/LimeSurvey/blob/master/application/helpers/update/updates/Update_706.php The part which fails for you is:
however, the code catches any exception being thrown here in case you did not have the index. Can you confirm you have the same code for this function as above? |
|
|
That's right, I started looking for the cause and I've come to the same conclusion: and by using the web interface for the update (with debugging set to 1), you can see the text below which indicates where the error is. So, it's indeed the migration to 706 that causes the creation of this unique index for emails. Why a DROP, since this data didn't exist before? I reinstalled, creating a blank database from V7, and by looking at the users table, I see that the indexes are indeed created. Here is the error text: CDbException CDbCommand could not execute the SQL command: SQLSTATE[25P02]: In failed sql transaction: 7 ERROR: current transaction is aborted, commands ignored until end of transaction block. The SQL statement executed was: CREATE UNIQUE INDEX "lime_idx2_users" ON "lime_users" ("email") /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/db/CDbCommand.php(358) 346 { Stack Trace
140) #2 19 public function up() #3 42 */ #4
|
|
|
The reason for needing the drop is that it was created for some installations. The reasoning is that if it fails, we need to silently ignore it, because it was not created in the first place. The solution I would do if I was to administer your PostgreSQL RDBMS would be to manually add the key by that name to the table, so when it attempts to drop it, it will succeed instead of throwing an error and possibly reestablishing connection. Unfortunately I can only guess about the exact details that you have, but I do think this approach would help, it's at least worth a try. If this solves the issue for you, then we'll need to fix this script to check for the existence of the index before attempting to drop it. Thanks a lot for your patience. |
|
|
@tibor.pacalat, can we have a developer look into this? |
|
|
@Mazi I will check what is the status of this. |
|
In one of my plugins, I don't find a way to check if an index exists: https://gitlab.com/SondagesPro/coreAndTools/reloadAnyResponse/-/blob/master/reloadAnyResponse.php?ref_type=heads#L1590 |
|
|
You can differentiate between the drivers that are used for the supported RDBMS (MySQL/MariaDB, PostgreSQL or SQL Server) and have the appropriate SQL being executed. |
|
|
Okay, testing is complete.
SELECT FROM information_schema.tables, generate_series(2011, 2024) AS annee -- Generates the years from 2011 to 2024 WHERE table_catalog = 'limesurveyv7' AND table_schema = 'public' AND table_type = 'BASE TABLE' AND table_name ~ ('^lime_oldresponses[0-9]+_' || annee || '[0-9]{10}$') or table_name ~ ('^lime_oldtokens[0-9]+_' || annee || '[0-9]{10}$') or table_name ~ ('^lime_oldsurvey[0-9]+_' || year || '[0-9]{10}$') ORDER BY table_name;
real 11m15.607s So it works. But keep in mind that some people might be experiencing the same issue as me. Question: I didn't have indexes with idx2 in previous versions (as you can see in the attachments). After the migration, the other indexes haven't changed (no idx, just the old names from the screenshot). P.S.: I have another bug in the new editor: ------------ The new editor is currently only compatible with the 'Fruity TwentyThree' theme and is displayed each time a questionnaire is selected. ------------ And if I select one of the menus on the left (e.g., General), I get a "checking permission" message that keeps running for a long time. Perhaps it's related to my database size... I'll check if others have reported this before doing so. |
|
|
I restarted the process:
The migration from version 651 to 708 stops with the following message: CException: Please fix this error in your database and try again in /var/www/html/limesurvey7/application/commands/UpdateDbCommand.php:51 I have attached the generated log file. Note: I have the following permissions with the limesurveyV7 user of the database to be migrated: Create Role, Database, Can log in. I'm retesting with the Postgres user (not good :( ) who is superuser. I haven't added the index creation (as seen previously). I think this will work, but what permission is the limesurveyV7 user missing, given that my previous migrations didn't require any privilege escalation? Well, it didn't work because I'm still getting the lime_idx2_users message, so I'm going to create it again and run the script again. Okay, we're making progress: database migration error from 705 to 708. Okay, this time it's working. But what about the database user? |
|
|
the file application.log (8,038 bytes)
2026/06/17 16:23:35 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[42501]: Insufficient privilege: 7 ERROR: permission denied for schema public. The SQL statement executed was:
CREATE OR REPLACE FUNCTION show_create_table(table_name text, join_char text = E'
' )
RETURNS text AS
$BODY$
SELECT 'CREATE TABLE ' || $1 || ' (' || $2 || '' ||
string_agg(column_list.column_expr, ', ' || $2 || '') ||
'' || $2 || ');'
FROM (
SELECT ' "' || column_name || '" ' || data_type ||
coalesce('(' || character_maximum_length || ')', '') ||
case when is_nullable = 'YES' then '' else ' NOT NULL' end as column_expr
FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = $1
ORDER BY ordinal_position) column_list;
$BODY$
LANGUAGE SQL STABLE
;
.
2026/06/17 16:23:56 [error] [exception.CException] CException: Please fix this error in your database and try again in /var/www/html/limesurvey7/application/commands/UpdateDbCommand.php:51
Stack trace:
#0 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleCommandRunner.php(71): UpdateDBCommand->run()
#1 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
#2 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/base/CApplication.php(185): CConsoleApplication->processRequest()
#3 /var/www/html/limesurvey7/application/commands/console.php(77): CApplication->run()
#4 {main}
---
2026/06/17 16:31:34 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[42501]: Insufficient privilege: 7 ERROR: permission denied for schema public. The SQL statement executed was:
CREATE OR REPLACE FUNCTION show_create_table(table_name text, join_char text = E'
' )
RETURNS text AS
$BODY$
SELECT 'CREATE TABLE ' || $1 || ' (' || $2 || '' ||
string_agg(column_list.column_expr, ', ' || $2 || '') ||
'' || $2 || ');'
FROM (
SELECT ' "' || column_name || '" ' || data_type ||
coalesce('(' || character_maximum_length || ')', '') ||
case when is_nullable = 'YES' then '' else ' NOT NULL' end as column_expr
FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = $1
ORDER BY ordinal_position) column_list;
$BODY$
LANGUAGE SQL STABLE
;
.
2026/06/17 16:31:56 [error] [exception.CException] CException: Please fix this error in your database and try again in /var/www/html/limesurvey7/application/commands/UpdateDbCommand.php:51
Stack trace:
#0 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleCommandRunner.php(71): UpdateDBCommand->run()
#1 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
#2 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/base/CApplication.php(185): CConsoleApplication->processRequest()
#3 /var/www/html/limesurvey7/application/commands/console.php(77): CApplication->run()
#4 {main}
---
2026/06/17 16:44:13 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[53200]: Out of memory: 7 ERROR: out of shared memory
HINT: You might need to increase "max_locks_per_transaction".. The SQL statement executed was: CREATE TABLE lime_responses_265375 (
"id" serial PRIMARY KEY,
"submitdate" timestamp without time zone,
"lastpage" integer,
"startlanguage" character varying(20) NOT NULL,
"seed" character varying(31),
"startdate" timestamp without time zone NOT NULL,
"datestamp" timestamp without time zone NOT NULL,
"Q387650" timestamp without time zone,
"Q387651" text,
"Q387637" character varying(5),
"Q387638" character varying(5),
"Q387639" character varying(5),
"Q387640" text,
"Q387652" text,
"Q387644" character varying(5),
"Q387642" character varying(5),
"Q387643" character varying(5),
"Q387646" character varying(5),
"Q387657" character varying(5),
"Q387645" character varying(5),
"Q387653" character varying(5),
"Q387678" character varying(5),
"Q387654" character varying(5),
"Q387661" text,
"Q387662" text,
"Q387660" text,
"Q387655" character varying(5),
"Q387658" character varying(5),
"Q387656" character varying(5),
"Q387659" character varying(5),
"Q387648" character varying(5),
"Q387649" character varying(5)
);.
2026/06/17 16:44:37 [error] [exception.CException] CException: Please fix this error in your database and try again in /var/www/html/limesurvey7/application/commands/UpdateDbCommand.php:51
Stack trace:
#0 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleCommandRunner.php(71): UpdateDBCommand->run()
#1 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
#2 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/base/CApplication.php(185): CConsoleApplication->processRequest()
#3 /var/www/html/limesurvey7/application/commands/console.php(77): CApplication->run()
#4 {main}
---
2026/06/17 17:02:46 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[42704]: Undefined object: 7 ERROR: index "lime_idx2_users" does not exist. The SQL statement executed was: DROP INDEX "lime_idx2_users".
2026/06/17 17:02:46 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[25P02]: In failed sql transaction: 7 ERROR: current transaction is aborted, commands ignored until end of transaction block. The SQL statement executed was: CREATE UNIQUE INDEX "lime_idx2_users" ON "lime_users" ("email").
2026/06/17 17:03:14 [error] [exception.CException] CException: Please fix this error in your database and try again in /var/www/html/limesurvey7/application/commands/UpdateDbCommand.php:51
Stack trace:
#0 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleCommandRunner.php(71): UpdateDBCommand->run()
#1 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
#2 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/base/CApplication.php(185): CConsoleApplication->processRequest()
#3 /var/www/html/limesurvey7/application/commands/console.php(77): CApplication->run()
#4 {main}
---
2026/06/17 17:06:18 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[42704]: Undefined object: 7 ERROR: index "lime_idx2_users" does not exist. The SQL statement executed was: DROP INDEX "lime_idx2_users".
2026/06/17 17:06:18 [error] [system.db.CDbCommand] CDbCommand::execute() failed: SQLSTATE[25P02]: In failed sql transaction: 7 ERROR: current transaction is aborted, commands ignored until end of transaction block. The SQL statement executed was: CREATE UNIQUE INDEX "lime_idx2_users" ON "lime_users" ("email").
2026/06/17 17:06:39 [error] [exception.CException] CException: Please fix this error in your database and try again in /var/www/html/limesurvey7/application/commands/UpdateDbCommand.php:51
Stack trace:
#0 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleCommandRunner.php(71): UpdateDBCommand->run()
#1 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
#2 /var/www/html/limesurvey7/vendor/yiisoft/yii/framework/base/CApplication.php(185): CConsoleApplication->processRequest()
#3 /var/www/html/limesurvey7/application/commands/console.php(77): CApplication->run()
#4 {main}
---
|
|
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2026-05-19 12:57 | riqcles | New Issue | |
| 2026-05-19 14:54 | lajosarpad | Note Added: 84834 | |
| 2026-05-19 14:54 | lajosarpad | Bug heat | 0 => 2 |
| 2026-05-19 16:39 | riqcles | Note Added: 84838 | |
| 2026-05-19 16:39 | riqcles | Bug heat | 2 => 4 |
| 2026-05-19 18:51 | lajosarpad | Note Added: 84853 | |
| 2026-05-19 23:16 | riqcles | Note Added: 84859 | |
| 2026-05-19 23:16 | riqcles | File Added: application.rar | |
| 2026-05-20 12:42 | lajosarpad | Note Added: 84863 | |
| 2026-05-20 13:58 | lajosarpad | Note Added: 84865 | |
| 2026-05-20 15:59 | DenisChenu | Note Added: 84869 | |
| 2026-05-20 15:59 | DenisChenu | Bug heat | 4 => 6 |
| 2026-05-21 10:36 | lajosarpad | Note Added: 84874 | |
| 2026-05-21 10:36 | lajosarpad | Note Edited: 84874 | |
| 2026-05-21 10:37 | DenisChenu | Note Added: 84875 | |
| 2026-05-21 11:44 | lajosarpad | Note Added: 84876 | |
| 2026-05-27 09:48 | riqcles | Note Added: 84919 | |
| 2026-05-27 09:48 | riqcles | File Added: 2026_05_26_V7_application.log | |
| 2026-05-27 09:48 | riqcles | File Added: 2026_05_26_2_V7_application.log | |
| 2026-05-27 09:59 | riqcles | Note Added: 84920 | |
| 2026-05-27 10:48 | lajosarpad | Note Added: 84921 | |
| 2026-05-27 11:50 | riqcles | Note Added: 84922 | |
| 2026-05-27 12:00 | lajosarpad | Note Added: 84923 | |
| 2026-05-27 15:11 | riqcles | Note Added: 84926 | |
| 2026-05-27 15:44 | lajosarpad | Note Added: 84928 | |
| 2026-05-29 11:00 | Mazi | Note Added: 84952 | |
| 2026-05-29 11:00 | Mazi | Bug heat | 6 => 8 |
| 2026-05-29 11:44 | tibor.pacalat | Note Added: 84954 | |
| 2026-05-29 11:44 | tibor.pacalat | Bug heat | 8 => 10 |
| 2026-05-29 12:10 | DenisChenu | Note Added: 84956 | |
| 2026-05-29 14:18 | lajosarpad | Note Added: 84961 | |
| 2026-05-29 16:24 | riqcles | Note Added: 84964 | |
| 2026-05-29 16:24 | riqcles | File Added: 2026-05-29_index_1.png | |
| 2026-05-29 16:24 | riqcles | File Added: 2026-05-29_index_2.png | |
| 2026-05-29 16:24 | riqcles | File Added: 2026-05-29_index_3.png | |
| 2026-05-29 16:24 | riqcles | File Added: 2026-05-29_index_4.png | |
| 2026-06-17 19:16 | riqcles | Note Added: 85066 | |
| 2026-06-17 19:17 | riqcles | Note Added: 85067 | |
| 2026-06-17 19:17 | riqcles | File Added: application.log |