I have done some further investigation and was at least able to narrow down the problem and found a related but different problem in the process.
First of all, i found where the 5000 number came from, which turns out to be a general setting (mail batch size), so I think you can ignore my earlier comments about that not being mentioned anywhere (my bad on that part). Although it might still be nice to have a "Note that the configured batch size is applied" to the documentation.

Playing around with this setting, I was able to exclude some of my earlier hunches on a managable testscenario. I used a batch size of 5 for the steps below

• Adding 7 (valid) participants and sending invitations (via invite_participants) does not result in an error. It correctly sends 5 emails first and then sends the remainder in the next call to invite_participants.
• Adding 12 participants and sending invitations does not result in an error. It correctly sends 5 emails.
• Adding 7 participants with a startdate in the future does not result in an error. No emails are sent (correct)
• Adding 1 participant with a startdate in the past AFTER adding 7 participants with a startdate in the future does not trigger any emails. There is no error, just nothing happens at all. This is wrong. It expect it to invite this 1 participant that can be invited. The 7 participants with a startdate in the future should have no influence on the amount of emails to send.
• Adding 6 or 11 participants with a startdate in the past AFTER adding 7 participants with a startdate in the future does not trigger any emails. This seems to have the exact same effect as the previous bullet. This is also wrong for the same reason. (I would have expected 5 emails in this case and then more in the next batch(es)). The 7 participants with a startdate in the future should have no influence on the amount of emails to send.
• Adding 5 participants with a startdate in the past and enddate of today and then adding 1 participant with a startdate in the past (no enddate) does not trigger any emails. This seems to have the exact same effect as the previous bullet. This is also wrong for the same reason. I expect 1 email in this case. The 5 'expired' participants should have no influence on the amount of emails to send.

So far I was not able to reproduce the HTTP 500 error on a (managable) testset, but the above findings should still be looked at because it just doesn't do what I would expect. I did not investigate the remind_participants functionality but I am expecting the same behavior there.

By the way, the results described in my previous comment were done on version 6.12.3

My guess is the 500 error is the result of a timeout. Either the php worker itself is timing out (max_execution_time in php.ini or set_time_limit in code somewhere) Or the web server is timing out waiting for it (for example fastcgi_read_timeout/proxy_read_timeout for nginx.)

Could also be both with one masking the other. I.e. if you fix the shorter one, the other one could still come into play.

You might also play with the email batch size and delay between batches to get it to finish before the timeout.

Interesting suggestion. I don't fully understand how the batch delay is relevant, because i'm only calling the endpoint once? But it's definitely worth looking into. Thanks for the pointer.

Anyway, even disregarding the HTTP 500 error (potential timeout), the behavior isn't what I would functionally expect either. I hope that can be addressed

i think it works something like this
number of emails to send = n
batch size = b
wait time = w

sent = 0
while(sent < n) {
send_emails(b)
sent+=b
wait(w)
}

so when you call if they're 50,000 emails to send and batch size is 500 and delay is 1 minute, it would easily take over an hour just in delays, not to mention how long it takes to send the emails themselves

never mind just read through the code, the delay only seems relevant for sending through the admin interface

the remote control endpoint just sends one round of the batch size

so perhaps the time it takes to send 5000 emails is greater than the relevant timeout in your case

Ok, that would mean that I should set a lower batch size. However, the effect would be even fewer mails being sent (likely none at all) because of the unwanted behavior that it considers the future participants as well, so that won't really work for me.