It's one of the reason to rewrite mail system in 4 and extend clearly PHPMailer.
https://github.com/LimeSurvey/LimeSurvey/blob/3ca1be07878d3e3953a7aa38bf7fb5f85345ccb8/application/core/LimeMailer.php#L495
Sample plugin : https://github.com/LimeSurvey/LimeSurvey/tree/develop/application/core/plugins/mailSenderToFrom (more easy this one)

According to this http://www.pmail.com/newsflash.htm Google will require Oauth2 by the end of May 2022 - this month. Would it be worth having this issue bumped up the priority list?

The cost of the assessment typically varies between $10,000 -$75,000 (or more) depending on the size and complexity of the application; smaller applications may see costs at a lower threshold of $4,500.

leave google …

And again : it can be done in plugin

"...leave Google"

Lol...

"...can be done in plugin": Most certainly but many of the existing features could be plugins, including all the SMTP settings. I think a more nuanced rule is used in practice ;)

image.png (9,169 bytes)   

image.png (9,169 bytes)   

Did you really read the link

"Every app that requests access to restricted scope Google user´s data … may see costs at a lower threshold of $4,500."

I confirm : leave google.

Irrelevant to this issue (relates to Android app publishing)?

LimeSurvey doesn't seem to be leaving Google - see image for three Google specific settings in Global Configuration.

NB Oauth2 isn't even just for Google, it's a standard (rammed through by Google) for what that's worth

So according the schedule Google has deactivated LSA a long time ago.
I did not hear from our user base so far that sending invitations using Gmail would not work.
So, it is not possible to use Google Mail with LimeSurvey now?

I think they planned to do it 2020 but delayed it due to the pandemic.

Seems that it the switchover date for mandatory Oauth2 is now 30 May 2022:

https://support.google.com/mail/thread/153515128/which-email-client-will-work-with-gmail-after-30th-may-2022?hl=en
https://github.com/k9mail/k-9/issues/5956
https://www.ghacks.net/2022/04/06/thunderbird-91-8-0-makes-important-changes-to-google-mail-accounts/

Found the actual announcement on LSAs:

https://support.google.com/accounts/answer/6010255

Please find below some comments about the analysis of the ticket

About the protocol

https://stackoverflow.com/questions/65228456/how-does-phpmailer-use-oauth/65230616#65230616

What's difficult about XOAUTH2 is obtaining that access token. To get hold of it requires that you run an HTTP service (on top of whatever other services you are running) and provide it with the correct ID and secret, which you submit and receive an access token in response, which you then use in your XOAUTH2 request. PHPMailer provides a script that helps you to do this for Google, Microsoft and Yahoo services, and others via the numerous adapters that are available for the PHPLeague's OAuth2 client package.

IMHO, OAuth2's monumentally confusing structure, ambiguity, and complexity creates excessive overhead and many opportunities for error, and it's all ultimately secured by a username and password, all of which undermines the security of the whole exercise. What doesn't help is that there is no standardisation, and the procedures you must follow to obtain tokens can be completely different across providers (hence the need for different adapters), and is subject to frequent change, making it hard to document. However, since Google imposes it on everyone, you have no choice but to go along with it.

Analysis

Please find below some comments about the analysis done:

Library

For each OAuth2 provider a specific library is needed

Data

In general, I understand that to send the mail it will be needed a client_id, a client_secret, and the Refresh Token.
In the case of Azure you also need a tenantId.
I don't know how it will be with other providers.

Endpoint for getting the Refresh Token

The Refresh Token is assigned by the provider.
An endpoint is required to receive it from the provider and save it.

• It has to be HTTPS
• The URL is configured in the Google console (or the provider setting).

UI for Refresh Token

Global Settings > Email Settings should be updated

• It should have some warning that OAUTH only works if HTTPS is enabled.
• It should ask you:
  • Provider (Google, Microsoft, Yahoo, etc.)
  • Client ID
  • Client Secret
    When saving (or maybe with a new get refresh token button), it should send you to the provider page to confirm the permissions. The new endpoint would receive the message and save the refresh token in a global setting (no UI).

We could also fetch the settings with a plugin event.

Plugin event

Ideally there would be new Plugin Events added so that plugins can register new providers.
The provider is used in two moments:

When fetching the refresh token

We could implement the UI for refresh token as plugin settings.
This common behaviour could be on a BaseSMTPOauth class.

When sending the mail.

To get the AbstractProvider instance that PHPMailer needs, additional parameters may be needed (such as the tenantId in the case of Azure), so somehow the plugin would have to indicate what additional fields should be add in Global Settings.

Also, we could set the provider on a plugin event (ex: beforeEmail)

Don't try to develop OAuth system,
PHPMailer already have solution : https://github.com/LimeSurvey/LimeSurvey/blob/master/application/third_party/phpmailer/src/OAuth.php

The doc is here …
https://github.com/PHPMailer/PHPMailer/wiki/Gmail-XOAUTH2-Using-Google-API-Client

I really think it must be a Plugin , core plugin yes, but a plugin.

Don't try to develop OAuth system,

Never intended to

Never intended to

Seems need specific library for GMail

composer require google/apiclient ^2.11

Then really better in a plugin

I would suggest that as a first step you add OAUTH support for Google as described in the PHPMailer docs.
After that is implemented we will see how we go from there.

I would suggest that as a first step you add OAUTH support for Google as described in the PHPMailer docs.

You mean a PoC?
Or adding that to Lime with all the configuration screens?

If you intended the latter, I guess you are OK with the analysis and approach, right?

Or adding that to Lime with all the configuration screens?

A lot of configuration screen + a complete library

A good reason to set it as Plugin ?
No ?
@c_schmitz ?

Picking this up. As a plugin :)

Hi All,

Let me show you how about the progress on this story.

We have developed a core plug-in that:

1- Pickups the credentials for connecting to Google services.

2 - Calls Google services to get a refresh-token, that will be used when sending emails.

3 - Intervene during the email sending process to avoid sending the emails through the regular means and send them directly through Google services.

Please find below a few screenshot.

Working on this proof-of-concept we have come with the following issues or observations.

Https required

Google requires an https URL callback when setting up the service. So HTTP will be required for using this plugin. We will be adding I checked on that as well as a warning if not having the proper schema.

From Address

Google seems to overwrite the from address and use the one related to the account.

New Email Events

Right now we only have beforeEmail event, which is used generally for decorating the email (Ex. adding a header), or for sending it through a different non-standard mean.

As the plug-ins don't have a synchronization mechanism, if there are two plug-ins using the same beforeEmail event, is hard to know for sure which plug-in will run first and which will run later.

Of that we believe we need a specific event named beforeSendingEmail. This way we will be able to separate two different processes:

A - Decorating the email (as adding a header or setting specific form addresses depending on the survey)

B - Sending it (as sending it through SMTP, OAuth, or even other gateways).

Global setting screen

We will need to inform the administrators that the OAuth plugin is active. If not administrator could get confused and think that the emails will be sent through regular means.

Right now for this proof-of-concept we are going to be doing it through JavaScript on an event. We believe we may need something a little bit more robust as there could be many other email plug-in.

We don't have a clear view on this but we picture having an email plugin coordinator, which can query the different email plug in and, for example, identify if the emails will be sent to a regular mean or through a plug-in email.

The global settings screen could use this new blocking email coordinator for knowing if the SMTP credential fields should be disabled or not, and also for showing a text message that says which plug-in will be handling emails.

Porting code to new plugins

Looking to the Future the current meaning for sending emails could be relocated in one or many core plug ins.

As the plug-ins don't have a synchronization mechanism, if there are two plug-ins using the same beforeEmail event, is hard to know for sure which plug-in will run first and which will run later.
Of that we believe we need a specific event named beforeSendingEmail. This way we will be able to separate two different processes:

beforeEmail receive whole LimeMailer (extend PHPMailer) as ->get('mailer'), then it can be used for anything that is related to mail in my opinon.

You can use priority in config.xml to be sure to use this way to send email after all olther are done. It's done for this.

But : we always have incompatibility issue with other plugin.
For example : you can not use https://github.com/Xites/LS_mailDKIM with google.

Or : developer can not really be sure other plugin broke somthing : https://gitlab.com/SondagesPro/mailing/noMailToExample must happen before google here for example.

POC: https://github.com/LimeSurvey/LimeSurvey/tree/feature/15664-Google-OAuth-plugin

Great !

Maybe PR as draft ? Hard to compare here

https://github.com/LimeSurvey/LimeSurvey/pull/2868