[Catalyst] Sending 3000 emails.

[Len Jaffe]

On Wed, Dec 2, 2015 at 10:10 AM, Andrew <catalystgroup at unitedgames.co.uk>
wrote:

> Hi Len and Darren,
> Atm I'm using Email::Stuffer to send email from within a perl script, in
> the
> Controller folder of my Catalyst app.
> I'm assuming this results in it being queued to be sent by the local SMTP
> server.
> My VPS server has Cpanel, and uses Exim, which I'm guessing is what the
> scripts are using to send the email.
>

According to the docs
http://search.cpan.org/~rjbs/Email-Stuffer-0.012/lib/Email/Stuffer.pm
unless you explicitly set the transport, Email::Stuffer defers to
Email::Sender for the default transport,
which appears to be sendmail, which means you'll be handing it off locally
for delivery.


As noted, it all works until about 1020 emails in, at which point the
> Catalyst app crashes (although this has only happened once - I do not wish
> to try again and risk another crash). I am using Email::Stuffer in a for
> loop, iterating over an array of 3000 email addresses. For my next step I
> will alter my code to only attempt to send to 500 email addresses at any
> one
> time, since below 1000 everything seems to work fine.
>

If you can consistently to 500, but 1000 consistently crashes you, and you
don't want to
spend the time on forensics, then just go with 500. But be aware that if
you're not freeing
memory, then it's possible to hit the resource usage limits if e.g. you
start sending a bigger message.

You seem to be mentioning using a background process, Darren. Is Exim
> already the background process that sends email?
>

It is a web service trope now that one does not send any email directly as
the result of a browser interaction,
but rather one notes that an email needs to be sent (queues a job request),
and a demon or cron job (a queue
processor) creates ans sends the mail. Thus the web infrastructure can
finish serving the web request sooner,
which is the greatest good at web scale.


If the long term solution is to use mailgun or sendgrid - before I delve
> into all their documentation - do you lot have any tips I should consider,
> on how to integrate them with a Catalyst app, or is it all pretty straight
> forward once I read the documentation?

I would not explore them unless:
1) you start sending a lot more mail.
2) you start ending up in spam control black lists.

As for tips, my usage of sendmail involves having my local sendmail (I use
postfix) hand off to
sendgrid for delivery.  I'd like to make use of their
thousand-addresses-at-a-time capabilities, but
my client doesn't feel the need to optimize their delivery processes yet.



Len.



> ----- Original Message -----
> From: "Darren Duncan" <darren at darrenduncan.net>
> To: "The elegant MVC web framework" <catalyst at lists.scsys.co.uk>
> Sent: Monday, November 30, 2015 11:41 PM
> Subject: Re: [Catalyst] Sending 3000 emails.
>
>
> On 2015-11-30 9:29 AM, Andrew wrote:
> > ...given that my attempt to send 3000 emails with nothing but my own VPS
> server,
> > and Cpan's Email::Stuffer, only sent around 1000, before the
> > plack/fastcgi/catalyst app crashed.... could I have been running into a
> similar
> > limitation? I'm now thinking of altering my code to split the attempts to
> send
> > email into six batches of 500 emails, as a short term fix, until I've had
> time
> > to research which transactional email partner to use, and how their APIs
> work,
> > or integrate with Catalyst apps.
>
> If you are trying to send the emails directly from the process serving your
> web
> API, then this is absolutely the wrong approach, and you should give up
> immediately on trying to do that.  Your API process should just queue the
> messages to be sent by a separate background process, and then return
> immediately such as with a message "3000 messages queued to be sent soon,
> click
> here to check queue progress" or such.  Sending in-process may be
> reasonable
> for
> a single email, eg email a copy of the form the user just submitted, but
> that's
> it. -- Darren Duncan
>
>
> _______________________________________________
> List: Catalyst at lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>
>
> _______________________________________________
> List: Catalyst at lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>



-- 
Len Jaffe - Information Technology Smoke Jumper - lenjaffe at jaffesystems.com
614-404-4214    @LenJaffe <https://www.twitter.com/lenJaffe>
www.lenjaffe.com
Host of Code Jam Columbus <http://www.meetup.com/techlifecolumbus/>  -
@CodeJamCMH <https://www.twitter.com/CodeJamCMH>
Curator of Advent Planet <http://www.lenjaffe.com/AdventPlanet/> - An
Aggregation of Online Advent Calendars.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.scsys.co.uk/pipermail/catalyst/attachments/20151202/8a69d8f9/attachment.htm>