View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|15664||Bug reports||Security||public||2019-12-17 12:50||2023-01-25 13:25|
|Summary||15664: OAuth support for SMTP & IMAP|
Implement OAuth for LS email settings
Email providers are shifting to OAuth to allow app access so requesting that LS support OAuth in email settings.
Example: LS currently accesses Gsuite from Google by taking the login details of a Gsuite account in LS global email settings. Gsuite classifies applications that do this as 'LSAs' - Less Secure Applications and to allow LS to access Gsuite this way currently, you have to provide an extra permission.
Google plans to turn off access for LSAs and only permit access via OAuth: "
|Tags||No tags attached.|
|Complete LimeSurvey version number (& build)|
|I will donate to the project if issue is resolved|
|Database type & version|
|Server OS (if known)|
|Webserver software & version (if known)|
It's one of the reason to rewrite mail system in 4 and extend clearly PHPMailer.
According to this http://www.pmail.com/newsflash.htm Google will require Oauth2 by the end of May 2022 - this month. Would it be worth having this issue bumped up the priority list?
leave google …
And again : it can be done in plugin
"...can be done in plugin": Most certainly but many of the existing features could be plugins, including all the SMTP settings. I think a more nuanced rule is used in practice ;)
Did you really read the link
"Every app that requests access to restricted scope Google user´s data … may see costs at a lower threshold of $4,500."
I confirm : leave google.
Irrelevant to this issue (relates to Android app publishing)?
LimeSurvey doesn't seem to be leaving Google - see image for three Google specific settings in Global Configuration.
NB Oauth2 isn't even just for Google, it's a standard (rammed through by Google) for what that's worth
So according the schedule Google has deactivated LSA a long time ago.
I think they planned to do it 2020 but delayed it due to the pandemic.
Seems that it the switchover date for mandatory Oauth2 is now 30 May 2022:
Found the actual announcement on LSAs:
Please find below some comments about the analysis of the ticket
About the protocol
What's difficult about XOAUTH2 is obtaining that access token. To get hold of it requires that you run an HTTP service (on top of whatever other services you are running) and provide it with the correct ID and secret, which you submit and receive an access token in response, which you then use in your XOAUTH2 request. PHPMailer provides a script that helps you to do this for Google, Microsoft and Yahoo services, and others via the numerous adapters that are available for the PHPLeague's OAuth2 client package.
IMHO, OAuth2's monumentally confusing structure, ambiguity, and complexity creates excessive overhead and many opportunities for error, and it's all ultimately secured by a username and password, all of which undermines the security of the whole exercise. What doesn't help is that there is no standardisation, and the procedures you must follow to obtain tokens can be completely different across providers (hence the need for different adapters), and is subject to frequent change, making it hard to document. However, since Google imposes it on everyone, you have no choice but to go along with it.
Please find below some comments about the analysis done:
For each OAuth2 provider a specific library is needed
In general, I understand that to send the mail it will be needed a client_id, a client_secret, and the Refresh Token.
Endpoint for getting the Refresh Token
The Refresh Token is assigned by the provider.
UI for Refresh Token
Global Settings > Email Settings should be updated
We could also fetch the settings with a plugin event.
Ideally there would be new Plugin Events added so that plugins can register new providers.
When fetching the refresh token
We could implement the UI for refresh token as plugin settings.
When sending the mail.
To get the AbstractProvider instance that PHPMailer needs, additional parameters may be needed (such as the tenantId in the case of Azure), so somehow the plugin would have to indicate what additional fields should be add in Global Settings.
Also, we could set the provider on a plugin event (ex: beforeEmail)
Don't try to develop OAuth system,
I really think it must be a Plugin , core plugin yes, but a plugin.
Never intended to
Seems need specific library for GMail
Then really better in a plugin
I would suggest that as a first step you add OAUTH support for Google as described in the PHPMailer docs.
You mean a PoC?
If you intended the latter, I guess you are OK with the analysis and approach, right?
A lot of configuration screen + a complete library
A good reason to set it as Plugin ?
Picking this up. As a plugin :)
Let me show you how about the progress on this story.
We have developed a core plug-in that:
1- Pickups the credentials for connecting to Google services.
2 - Calls Google services to get a refresh-token, that will be used when sending emails.
3 - Intervene during the email sending process to avoid sending the emails through the regular means and send them directly through Google services.
Please find below a few screenshot.
Working on this proof-of-concept we have come with the following issues or observations.
Google requires an https URL callback when setting up the service. So HTTP will be required for using this plugin. We will be adding I checked on that as well as a warning if not having the proper schema.
Google seems to overwrite the from address and use the one related to the account.
New Email Events
Right now we only have beforeEmail event, which is used generally for decorating the email (Ex. adding a header), or for sending it through a different non-standard mean.
As the plug-ins don't have a synchronization mechanism, if there are two plug-ins using the same beforeEmail event, is hard to know for sure which plug-in will run first and which will run later.
Of that we believe we need a specific event named beforeSendingEmail. This way we will be able to separate two different processes:
A - Decorating the email (as adding a header or setting specific form addresses depending on the survey)
B - Sending it (as sending it through SMTP, OAuth, or even other gateways).
Global setting screen
We will need to inform the administrators that the OAuth plugin is active. If not administrator could get confused and think that the emails will be sent through regular means.
We don't have a clear view on this but we picture having an email plugin coordinator, which can query the different email plug in and, for example, identify if the emails will be sent to a regular mean or through a plug-in email.
The global settings screen could use this new blocking email coordinator for knowing if the SMTP credential fields should be disabled or not, and also for showing a text message that says which plug-in will be handling emails.
Porting code to new plugins
Looking to the Future the current meaning for sending emails could be relocated in one or many core plug ins.
image-2.png (233,769 bytes)
image-3.png (312,429 bytes)
beforeEmail receive whole LimeMailer (extend PHPMailer) as ->get('mailer'), then it can be used for anything that is related to mail in my opinon.
You can use priority in config.xml to be sure to use this way to send email after all olther are done. It's done for this.
But : we always have incompatibility issue with other plugin.
Or : developer can not really be sure other plugin broke somthing : https://gitlab.com/SondagesPro/mailing/noMailToExample must happen before google here for example.
Maybe PR as draft ? Hard to compare here
|2019-12-17 12:50||Indispirit||New Issue|
|2019-12-18 16:00||DenisChenu||Issue Monitored: DenisChenu|
|2019-12-18 16:01||DenisChenu||Note Added: 55069|
|2022-05-18 14:51||Leithy||Note Added: 69782|
|2022-05-18 14:51||Leithy||Bug heat||254 => 256|
|2022-05-18 14:52||Leithy||Issue Monitored: Leithy|
|2022-05-18 14:52||Leithy||Bug heat||256 => 264|
|2022-05-18 14:56||DenisChenu||Note Added: 69784|
|2022-05-18 14:56||DenisChenu||Note Added: 69785|
|2022-05-19 13:09||Leithy||Note Added: 69846|
|2022-05-19 13:09||Leithy||File Added: image.png|
|2022-05-19 14:15||DenisChenu||Note Added: 69852|
|2022-05-19 14:20||Leithy||Note Added: 69853|
|2022-05-20 09:15||c_schmitz||Note Added: 69897|
|2022-05-20 09:15||c_schmitz||Bug heat||264 => 266|
|2022-05-20 11:25||Leithy||Note Added: 69906|
|2022-05-20 11:42||Leithy||Note Added: 69907|
|2022-09-25 19:25||netikja||Issue Monitored: netikja|
|2022-09-25 19:25||netikja||Bug heat||266 => 274|
|2022-10-18 17:00||c_schmitz||Priority||none => immediate|
|2022-10-18 17:00||c_schmitz||Assigned To||=> gabrieljenik|
|2022-10-18 17:00||c_schmitz||Status||new => assigned|
|2022-10-18 17:01||c_schmitz||Summary||[feature request] OAuth implementation => OAuth support for SMTP & IMAP|
|2022-10-18 17:01||c_schmitz||Project||Feature requests => Bug reports|
|2022-10-20 21:49||gabrieljenik||Note Added: 72371|
|2022-10-20 21:49||gabrieljenik||Bug heat||274 => 276|
|2022-10-20 21:49||gabrieljenik||Note Edited: 72371|
|2022-10-21 08:45||DenisChenu||Note Added: 72372|
|2022-10-21 13:59||gabrieljenik||Note Added: 72373|
|2022-10-21 14:09||DenisChenu||Note Added: 72374|
|2022-10-26 10:22||c_schmitz||Note Added: 72428|
|2022-10-26 15:40||gabrieljenik||Note Added: 72436|
|2022-10-26 17:58||DenisChenu||Note Added: 72440|
|2023-01-03 20:50||gabrieljenik||Note Added: 73340|
|2023-01-09 17:54||gabrieljenik||Note Added: 73405|
|2023-01-09 17:54||gabrieljenik||File Added: image-2.png|
|2023-01-09 17:54||gabrieljenik||File Added: image-3.png|
|2023-01-09 18:02||DenisChenu||Note Added: 73406|
|2023-01-09 18:07||DenisChenu||Note Edited: 73406|
|2023-01-24 21:58||gabrieljenik||Note Added: 73551|
|2023-01-25 07:53||DenisChenu||Note Added: 73554|
|2023-01-25 13:25||gabrieljenik||Note Added: 73556|