View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|16330||Bug reports||Encryption||public||2020-05-25 14:55||2021-07-12 11:53|
|Fixed in Version||5.x|
|Summary||16330: Encrypted fields not decrypted correctly when viewing responses using web UI or when exported to .lsa|
|Description||Survey includes one question where encryption has been specified (a multiple short text question requesting Firstname, Lastname). |
There are no errors when respondents submit, and the database field is showing ciphertext for responses to this question.
However when responses are examined using the Web UI, or exported in a .lsa file, they are NOT shown correctly decrypted but in what also appears to be ciphertext (but is NOT the same character sequence as in the database, so some attempt at decryption seems to be happening).
The security.php file has not been changed during the currency of this survey.
|Steps To Reproduce||1) Import and activate attached survey, which has 16 questions in 4 groups.|
2) Complete one response - apart from the first question, null responses can be given to most questions but enter text for [G01Q07] "Please give your name"
3) View response for this entry - field is not decrypted as expected.
|Additional Information||1) A simple test survey with one encrypted question of the same type (no other questions in survey) does NOT reproduce the problem - the fields in the question are decrypted correctly. |
2) I don't think this is relevant, but "for info" the survey exhibiting the bug, and also the simple test survey noted here, have been moved between LimeSurvey instances - behaviour is the same for each survey regardless of server (other server is on 4.2.3)
|Tags||No tags attached.|
|Complete LimeSurvey version number (& build)||4.4.11 git|
|I will donate to the project if issue is resolved||No|
|Database & DB-Version||MariaDB 10.3.23|
|Server OS (if known)||linux|
|Webserver software & version (if known)||LiteSpeed|
|related to||15669||feedback||p_teichmann||Crypted data are not exported uncrypted|
|related to||15668||closed||p_teichmann||Crypted data are not shown uncrypted|
|related to||15670||closed||p_teichmann||Reloading response via token : crypted data|
|related to||16971||closed||p_teichmann||Unable to use of TOKEN attribute encrypted during survey (expression manager)|
limesurvey_survey_597794.lss (159,608 bytes)
About LSA export : i think it's great if we have a "LSA decrypted" export, then user can choose if done for backup for same server then can be crypted. Or not.
The worst one are [Reloading response via token : crypted data](https://bugs.limesurvey.org/view.php?id=15670)
> 1) A simple test survey with one encrypted question of the same type (no other questions in survey) does NOT reproduce the problem - the fields in the question are decrypted correctly.
? Are you sure ?
Try to import included lsa : i think it broke with a 500 error (Wrong decryption key! Decryption key has changed since this data were last saved, so data can't be decrypted. Please consult our manual at https://manual.limesurvey.org/Data_encryption#Errors.)
survey_archive_crypted.lsa (5,493 bytes)
Hi - I think I may have diverted attention away from my main issue here by talking about exporting / importing survey archives with encrypted fields.
I did spot issues 15668 and 15669 before posting, and have verified in a one-question survey with encryption that in this case, my export is decrypted - obviously that will be necessary to migrate it to another server with different encryption keys. That may not be so wise for a server backup but that's another question for debate / feature request to select as part of the export.
I did import your test survey and that had an error as expected when I tried to display a response:
"500: Internal Server Error
Call to a member function decrypt() on null"
This is my issue:
With the survey I uploaded here, I get NO errors when I activate the survey, enter one response including 'First Name, Surname', and then display the response.
In the DB I can see that the relevant fields are encrypted, but when displaying the response I see:
[G01Q07_SQ01] (First name) Ys1jwmKfS1+rYB274j2gO06GPrYdL...
[G01Q07_SQ02] (Surname) 5PORCDiCzCidyf4hN0LbX0wdG74d...
(I've shortened the strings above). No error is shown but the decryption is incorrect (the above strings are different from the DB fields for those questions)
Are you able to replicate the problem with my survey?
There seems to be something specific in my uploaded survey that's stopping the correct decryption, though the simple one-question survey with encryption DOES work.
I can confirm i have encrypted field in browse response.
The strange part : « though the simple one-question survey with encryption DOES work.» : you mean a survey with only ONE question (encrypoted) ?
Because a survey with 2 question : one crypted + on not crypted show (and export) only crypted data.
OK : database are not the same, but export as LSA, import as LSA : work and show same data.
Capture d’écran du 2020-05-26 11-51-21.png (16,884 bytes)
Capture d’écran du 2020-05-26 11-51-21.png (16,884 bytes)
Yes that seems to replicate what I'm seeing - both web UI and exported LSA are showing the same data (WRONGLY decrypted version of the DB contents).
Re your previous note - I have made two test surveys - one with only one question that was encrypted, the second survey with two questions, one of which was encrypted. BOTH behaved as expected with the DB fields being encrypted, but shown correctly in the Web UI and being exported unencrypted (as in the attached LSA for the two-question survey).
It is strange that the problem seems to be present in some surveys but not others. If if you are seeing the problem in your survey with only two questions (one encrypted) that may make debugging easier hopefully.
survey_archive_971865-2.lsa (4,641 bytes)
It's not really wrongly encryted since we can export as lsa and import (in same instance) without issue.
If there are a bad system here : we can not do it :)
|Yes I understand what you are saying, the decryption on the 'bad' surveys / fields isn't showing an error so must the actual decryption must be happening correctly, but if so, then the correct plaintext string resulting from that decryption isn't being shown in the UI or the LSA. I have checked to see if a small sub-string of what is displayed in the UI / LSA field is present in the encrypted DB field, and it isn't. So I'm not sure where the output shown in UI / LSA is coming from.|
When a person drops out of the survey without a submit the encrypted data appears fine in the show responses for that record.
However, when the person completes the survey with a submit at the end the encrypted data is not properly displayed in the UI (show responses) for that record.
To follow with more examples - I have a short survey done on Version 4.3.3+200707. An export is attached.
The two questions "appear" to be the same and both are encrypted. The email that is sent to the admin shows with answers without clear text for either (attached)
The Admin "show responses" shows the first question garbled, but the second question is properly decrypted in the UI (response1)
You can create multiple questions of different types and some will be correct and others are not.
IMPORTANT - changing the ORDER of the questions has an impact. (response2) as the first is wrong and the second correct.
Adding another question appears to work properly. (response3)
limesurvey_survey_274463.lss (34,760 bytes)
email.txt (1,201 bytes)
Email that gets sent to admin - encryted fields are "hidden" Hello, A new response was submitted for your survey 'testing'. Click the following link to see the individual response: https://guernseyresearch.com/survey/index.php/admin/responses/sa/view/surveyid/274463/id/11 Click the following link to edit the individual response: https://guernseyresearch.com/survey/index.php/admin/dataentry/sa/editdata/subaction/edit/surveyid/274463/id/11 View statistics by clicking here: https://guernseyresearch.com/survey/index.php/admin/statistics/sa/index/surveyid/274463 The following answers were given by the participant: Response ID : 11 Date submitted : 01-01-1980 00:00:00 Last page : 3 Start language : en Seed : 1238572615 My first question group <p>What color</p> : Red [AO01] <p>email</p> : aKcreBSO7Gov58aIL6MkyrOn8ECq3BWpSsxBYIyR4i6UBq2DjkXV50+Z7PLLHwihHxXLwCATrHvuE7LzycJiBWQvblMxbUJzZEtVaDE3WktaV2M3TGZ6V1dLRThxM3ZDRWhMZnJQUVBpRTJ0bEp5WmkzNllCeEh5NGxsWkNUaThnTmNNaGVqWmsremFVR0RYZzNGRkRXUmhkbVZBWldVPQ== <p>name</p> : 1su1M7yg9GwdTt2P4mSCWTCYnJIwECbYcWUkX3CLd0y4mNPSI5IF8DhdxODp1KdrXiDilOlWAbPSO7OzBfmtBURhdmU=
email.txt (1,201 bytes)
Response1.jpg (142,187 bytes)
|Checked both : seems fixed now (export lsa from one system to another instance)|
@GuernseyResearch : i confirm the 2 issue on last master.
Can you report separatly maybe ? @cdorin : need 2 issue or can be fixed here ?
|This appears to be fixed in Version 4.4.0-RC3+210112|
> However when responses are examined using the Web UI
No need token : just
5. Browse, view and edit
limesurvey_survey_CryptedToken.lss (30,544 bytes)
Capture d’écran du 2021-03-05 08-06-07.png (33,050 bytes)
Capture d’écran du 2021-03-05 08-06-07.png (33,050 bytes)
|Having the same problem with a few forms that were conducted before 4.4.0 was out (some time last month, I believe). The encryption was only enabled on some fields, probably by accident, and only these fields are being shown non-decrypted. I tried copying the database and setting up a new LimeSurvey instance pulled directly from master with the old security.php - no luck. Issue persists across Web UI and all exports, which makes it impossible for us to extract any of the answers. As some of those surveys were pretty critical and the results are going to be needed soon, is there any way that they could possibly be decrypted "manually"? Do I understand correctly that AES-256-GCM is used for encryption? Is there any other information I can provide or something I should try?|
|Fix committed to master branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=31818|
@GuernseyResearch, can you confirm that this problem is fixed for you at the latest version?
Did you notice any performance issues when exporting encrypted data of a larger survey (our survey has ~300 questions and 1000 responses)?
|Our key survey that uses encryption has 40 questions (some are skipped) and TWO fields near the end that are encrypted because we ask for an optional name and email address if they want to participate in another survey. We have close to 1000 responses and don't appear to have a problem. When we do an export of the results we don't need to see the encrypted data (that is done as we process each response and communicate individually with the people responding). As a result, we have not seen any performance issues in the actual survey data.|
|Thanks for sharing your feedback!|
We have many encrypted fields (~300 questions) and 1600 responses and the screen just freezes on export.
When exporting a smaller amount of responses, we get empty or "n/a" at the responses, though there IS some data. But that may be related to testing at a different system and not using the correct key for decryption... not sure. Maybe anyone has seen those emtpy responses at the export as well?
Do you need ALL those fields to be encrypted?
We are trying to follow privacy rules and protect personal information (name, email). As long as those are blocked the other data is not associated with a person. The database itself is protected so we feel that we are adequately following the requirements.
What causes you to encrypt so much of the survey?
> not using the correct key for decryption...
Sure it broke somewhere here …
|2020-05-25 14:55||pns1953||New Issue|
|2020-05-25 14:55||pns1953||File Added: limesurvey_survey_597794.lss|
|2020-05-26 10:04||DenisChenu||Relationship added||related to 15669|
|2020-05-26 10:04||DenisChenu||Relationship added||related to 15668|
|2020-05-26 10:04||DenisChenu||Relationship added||related to 15670|
|2020-05-26 10:06||DenisChenu||Note Added: 58020|
|2020-05-26 10:09||DenisChenu||Note Added: 58021|
|2020-05-26 10:09||DenisChenu||File Added: survey_archive_crypted.lsa|
|2020-05-26 11:42||pns1953||Note Added: 58022|
|2020-05-26 11:49||DenisChenu||Note Added: 58023|
|2020-05-26 11:49||DenisChenu||Note Added: 58024|
|2020-05-26 11:49||DenisChenu||File Added: Capture d’écran du 2020-05-26 11-47-31.png|
|2020-05-26 11:52||DenisChenu||Note Added: 58025|
|2020-05-26 11:52||DenisChenu||File Added: Capture d’écran du 2020-05-26 11-51-21.png|
|2020-05-26 11:52||DenisChenu||File Added: Capture d’écran du 2020-05-26 11-50-34.png|
|2020-05-26 12:02||pns1953||Note Added: 58026|
|2020-05-26 12:02||pns1953||File Added: survey_archive_971865-2.lsa|
|2020-05-26 12:31||DenisChenu||Note Added: 58028|
|2020-05-26 12:59||pns1953||Note Added: 58032|
|2020-05-27 15:36||ollehar||Category||Response browsing => Encryption|
|2020-06-30 18:35||GuernseyResearch||Note Added: 58599|
|2020-07-09 00:58||GuernseyResearch||Note Added: 58817|
|2020-07-09 00:58||GuernseyResearch||File Added: limesurvey_survey_274463.lss|
|2020-07-09 00:58||GuernseyResearch||File Added: email.txt|
|2020-07-09 00:58||GuernseyResearch||File Added: Response1.jpg|
|2020-07-09 00:58||GuernseyResearch||File Added: Response2.jpg|
|2020-07-09 00:58||GuernseyResearch||File Added: Response3.jpg|
|2020-08-07 19:54||kjnerhus||Issue Monitored: kjnerhus|
|2020-09-16 19:29||DenisChenu||Note Added: 59855|
|2020-10-26 14:51||DenisChenu||Note Added: 60412|
|2021-01-12 17:30||GuernseyResearch||Note Added: 61502|
|2021-01-12 17:35||ollehar||Assigned To||=> ollehar|
|2021-01-12 17:35||ollehar||Status||new => resolved|
|2021-01-12 17:35||ollehar||Resolution||open => fixed|
|2021-01-12 17:35||ollehar||Fixed in Version||=> 4.4.0-RC2|
|2021-01-12 17:35||ollehar||Note Added: 61503|
|2021-01-13 08:49||DenisChenu||Relationship added||related to 16971|
|2021-01-29 08:23||cdorin||Status||resolved => closed|
|2021-03-05 08:05||DenisChenu||Status||closed => feedback|
|2021-03-05 08:05||DenisChenu||Resolution||fixed => reopened|
|2021-03-05 08:05||DenisChenu||Complete LimeSurvey version number (& build)||4.2.4+200520 => 4.4.11 git|
|2021-03-05 08:05||DenisChenu||Note Added: 62744|
|2021-03-05 08:07||DenisChenu||Note Added: 62745|
|2021-03-05 08:07||DenisChenu||File Added: limesurvey_survey_CryptedToken.lss|
|2021-03-05 08:07||DenisChenu||File Added: Capture d’écran du 2021-03-05 08-06-07.png|
|2021-03-05 08:07||DenisChenu||File Added: Capture d’écran du 2021-03-05 08-06-14.png|
|2021-03-28 16:31||xieve_||Note Added: 63663|
|2021-03-29 11:09||ollehar||Assigned To||ollehar => p_teichmann|
|2021-05-17 11:03||p_teichmann||Status||feedback => assigned|
|2021-05-21 10:29||p_teichmann||Changeset attached||=> LimeSurvey master bd1ae3af|
|2021-05-21 10:29||p_teichmann||Note Added: 64504|
|2021-05-21 10:29||p_teichmann||Resolution||reopened => fixed|
|2021-06-25 10:31||Mazi||Note Added: 65058|
|2021-06-25 16:31||GuernseyResearch||Note Added: 65064|
|2021-06-25 22:02||Mazi||Note Added: 65065|
|2021-06-25 22:05||Mazi||Note Added: 65066|
|2021-06-25 22:18||GuernseyResearch||Note Added: 65067|
|2021-06-26 16:49||DenisChenu||Note Added: 65104|
|2021-07-06 09:20||p_teichmann||Status||assigned => closed|
|2021-07-06 09:20||p_teichmann||Fixed in Version||4.4.0-RC2 => 5.x|
|2021-07-06 09:26||p_teichmann||Status||closed => resolved|
|2021-07-12 11:53||c_schmitz||Note Added: 65280|
|2021-07-12 11:53||c_schmitz||Status||resolved => closed|