[Catalyst] Sending 3000 emails.

Mon Nov 30 17:29:40 GMT 2015

I also read on the sendgrid website that:

"With SMTP, 100 messages can be sent with each connection"
and
"Customers should utilize SMTPAPI if this is an option. As with SMTP, 100 messages can be sent with each connection, but there can be 1000 recipients for each message."

...given that my attempt to send 3000 emails with nothing but my own VPS server, and Cpan's Email::Stuffer, only sent around 1000, before the plack/fastcgi/catalyst app crashed.... could I have been running into a similar limitation? I'm now thinking of altering my code to split the attempts to send email into six batches of 500 emails, as a short term fix, until I've had time to research which transactional email partner to use, and how their APIs work, or integrate with Catalyst apps.

  ----- Original Message ----- 
  From: Len Jaffe 
  To: The elegant MVC web framework 
  Sent: Sunday, November 29, 2015 1:31 AM
  Subject: Re: [Catalyst] Sending 3000 emails.

  I'll add that you should look into a transactional email partner like sendgrid, mailgun, etc.  They'll help you stay out of the  black lists by showing you what not to do, ans how to set up DKIM, etc. for authentication of your email.

  I use sendgrid professionally, with one of my cusotmers that sends a lot of email.

  If your volume is under 10k emails per month, sendgrid is free, and they have a templating feature which allows you to upload a template (parameterizable) and let them to the mail merge on their servers, for up to one thousand addresses per tx.

  So instead of generating 3000 emails and pushing them through your smtp server, you'd upload the template file, and push three 1000-name lists to to sendgrid and let them take care of delivery.  Something to think about.

  Len.

  On Thu, Nov 26, 2015 at 12:38 PM, Lasse Makholm <lasse at unity3d.com> wrote:

    Maybe not the answer you're looking for but I'd go with an email service such as sendgrid which takes most if not all of the pain out of sending emails. They have a simple REST API you can POST your email to and provides callbacks for delivery notifications etc as well...

    I'm not advocating sendgrid over any other service (I'm sure there are plenty others) - it's just the one I have experience with... They apparently have a free tier as well....

    /L

    On Thu, Nov 26, 2015 at 5:53 PM, Octavian Rasnita <orasnita at gmail.com> wrote:

      A queue in Perl is better when there is a need of sending thousand messages.
      Without a queue, if the Catalyst-based code just sends the messages directly and a browser is waiting for a page to load after the web app sent them, it may time-out. But otherwise it should work and not crash the web app.
      3000 messages should be sent pretty fast if they are sent to the local SMTP server, but not fast enough for a pleasant experience.

      Regarding the success of those messages... if you don't want them to reach in the spam folder, it helps to sign them by using SPF/DKIM. If you'll find this complicated, a better solution might be to create an account on a site like mailgun.com or mandrill.com and send the messages using their servers. They offer APIs that you can use in your app very easy. mailgun.com allows sending 10,000 messages/month for free and after that limit the prices are pretty cheap. You will have reports with the list of email messages that bounced that you can get programaticly, plus a few other helpful features.

      --Octavian

        ----- Original Message ----- 
        From: Andrew 
        To: The elegant MVC web framework 
        Sent: Thursday, November 26, 2015 5:32 PM
        Subject: [Catalyst] Sending 3000 emails.

        Hello,

        You lot are all experts at Catalyst and Perl,
        and because I'm using Perl to code a Catalyst program, I wanted to ask you about a problem I've encountered.

        The Catalyst app I'm coding is to replace a PHP website, which presently allows an admin to manage sending out an email to all members. The site has 3000 odd members. From what I could tell, the PHP code was simply using the PHP mail function to send email (although it did in my experience typically take two days for everyone to receive an email sent with it).
        I've also been asking other Perl programmers when I've had the chance - and the only concerns raised, were about IP blacklisting or emails bouncing back - not about actual code working.

        So could sending 3000 emails be as simple as Matt's old CGI form mail?

        I had a look on CPAN, and found the simple looking Email::Stuffer,
        and wrote a script, sending an email to myself, and it worked.
        I then modified the script to pick five email addresses from a MySQL database, and email them, and again - it worked.
        Okay - one final change - throwing 3000 email addresses at it.
        How does that do....?
        Starts to work - then throws up an Internal Server Error in the browser window,
        and I find out my Fast CGI Plack Up server thingie isn't up, nor running my Catalyst app anymore.
        I have to type plackup and my details, etc, again from the command prompt.
        A log in to Web Host Manager (this is an apache2 VPS with CPanel) and a check of the "View Sent Summary" shows 1020 emails sent - 611 successful, 642 deferrals and 329 failures...curious, as that adds up to more than 1020, =S. If I click through for more details, it shows 1582 records - some with green ticks, and others with amber or red exclamation marks, ^_^.
        I can worry about IPs and failure rates later....

        ....I just want to know why the whole FastCGI Plack loaded Catalyst app came down half way through,
        and how I can have a Perl script process sending 3000 emails without that happening every time, =S.

        Do I need to use a queue within Perl? Or does the postfix/sendmail on the server, automatically queue everything for me? If so - why did my code crash the Catalyst app, and not send all the emails? >_<.

        I'm guessing this is a common problem - how does a website email all its signed up members - and so there must surely be a common solution among Perl / Catalyst users....?

        Any help, appreciated!

        Yours,
        Andrew.

------------------------------------------------------------------------

        _______________________________________________
        List: Catalyst at lists.scsys.co.uk
        Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
        Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
        Dev site: http://dev.catalyst.perl.org/

      _______________________________________________
      List: Catalyst at lists.scsys.co.uk
      Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
      Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
      Dev site: http://dev.catalyst.perl.org/

    _______________________________________________
    List: Catalyst at lists.scsys.co.uk
    Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
    Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
    Dev site: http://dev.catalyst.perl.org/

  -- 

  Len Jaffe - Information Technology Smoke Jumper - lenjaffe at jaffesystems.com 
  614-404-4214    @LenJaffe  www.lenjaffe.com

  Host of Code Jam Columbus  - @CodeJamCMH
  Curator of Advent Planet - An Aggregation of Online Advent Calendars.

------------------------------------------------------------------------------

  _______________________________________________
  List: Catalyst at lists.scsys.co.uk
  Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
  Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
  Dev site: http://dev.catalyst.perl.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.scsys.co.uk/pipermail/catalyst/attachments/20151130/22929eaf/attachment.htm>