The emailing security has been raised in the latest version.
This error happens because your mail server uses an invalid certificate.

yes, I just noticed I had to change the SMTP server name. Please close the issue, thanks.

On the same server I have installed L.S. 2.59.1+ build 170116. This version is using phpmailer 5.6.22. This works fine, using the same smtp server.
I have tried copying phpmailer 5.6.22 files into the 2.6.3 installation (which uses version 5.6.21) but that didn't work.
So - is there a difference between versions 2.59 and 2.6.3 with respect to the way phpmailer is called?

It seems I can fix it by replacing lines 3992-3994 in common_helper.php:
require_once(APPPATH.'/third_party/phpmailer/class.phpmailer.php');
require_once(APPPATH.'/third_party/phpmailer/class.smtp.php');
$mail = new PHPMailer;
by:
require_once(APPPATH.'/third_party/phpmailer/PHPMailerAutoload.php');
$mail = new PHPMailer;
$mail->SMTPAutoTLS=false;

as taken from the 2.59.1 source files.

@fvanderstarre: Thank you for the insight. Will apply that as fix.

Fix committed to 2.06lts branch: http://bugs.limesurvey.org/plugin.php?page=Source/view&id=22221