[Catalyst] Production session issue - commercial support inquiry?

[J. Shirley]

On Mon, Jan 26, 2009 at 7:32 PM, Matt Pitts <mpitts at a3its.com> wrote:
>> -----Original Message-----
>> From: J. Shirley [mailto:jshirley at gmail.com]
>> Sent: Monday, January 26, 2009 4:14 PM
>> To: The elegant MVC web framework
>> Subject: Re: [Catalyst] Production session issue - commercial support
>> inquiry?
>>
>> On Mon, Jan 26, 2009 at 12:27 PM, Matt Pitts <mpitts at a3its.com> wrote:
>> >> -----Original Message-----
>> >> From: Matt Pitts [mailto:mpitts at a3its.com]
>> >> Sent: Monday, January 26, 2009 10:53 AM
>> >> To: The elegant MVC web framework
>> >> Subject: RE: [Catalyst] Production session issue - commercial
>> support
>> >> inquiry?
>> >>
>> >> > -----Original Message-----
>> >> > From: Matt Pitts [mailto:mpitts at a3its.com]
>> >> > Sent: Friday, January 23, 2009 4:22 PM
>> >> > To: The elegant MVC web framework
>> >> > Subject: RE: [Catalyst] Production session issue - commercial
>> > support
>> >> > inquiry?
>> >> >
>> >> > We had another major incident of session cross-over this morning
>> >> which
>> >> > happened right after I pushed out a small change and
>> > reload/restarted
>> >> > the instances.
>> >> >
>> >> > I personally experienced the session cross-over twice. In each
>> >> instance
>> >> > I captured my session id and went to compare it to a dump of all
>> >> > sessions and found that no other session had the same data as
>> mine.
>> >> > However, the 2 session id's *were* the same and I had cleared my
>> >> > cookies
>> >> > between these two instances.
>> >> >
>> >> > So, I could be getting dups out of generate_session_id, but I'm
>> also
>> >> > wondering if it's not some weird process issue where the same
>> > session
>> >> > ID
>> >> > is not being generated twice, but is just getting shoved into the
>> >> > response headers of 2 different requests. Anyone know if this is
>> >> > plausible?
>> >> >
>> >> > If P::Session is generating duplicate IDs I doubt I'd be the only
>> > one
>> >> > having the issue. BTW, the cat app only averages about 12 req/s
>> and
>> >> > this
>> >> > issue doesn't seem to be very dependent on traffic.
>> >> >
>> >> > Along the lines of the process question...
>> >> >
>> >> > What, if any, are the ramifications of running two instances of
>> the
>> >> app
>> >> > (one HTTP::Prefork and one FCGI) out of the same "root". In my
>> >> > particular case the FCGI instance is for talking to a local
> Apache
>> >> > instance that is providing SSL access and the HTTP::Prefork
>> instance
>> >> is
>> >> > to provide direct HTTP access to areas where SSL isn't needed. I
>> >> wasn't
>> >> > originally worried about this because they're separate process,
>> but
>> >> > right now I'm looking at anything that might be causing my
>> problems.
>> >> > Could a situation like this cause strange behavior in regards to
>> >> > session
>> >> > ids.
>> >> >
>> >> > Right now I have a set of changes primed to push out that
>> completely
>> >> > removed P::Session and uses P::CookiedSession instead. I guess if
>> I
>> >> > don't get a resolution to the issues with P::Session by Monday
>> I'll
>> >> be
>> >> > pushing this out and then I'll know it's not P::Session problem
> if
>> > it
>> >> > continues.
>> >> >
>> >> > TIA
>> >> >
>> >> > -matt
>> >>
>> >> My saga continues...
>> >>
>> >> I have pushed out my CookiedSession based changes and can confirm
>> that
>> >> the cross-over issues are still present. Myself and my client have
>> > been
>> >> able to replicate the issue twice by clearing cookies and accessing
>> >> Cart
>> >> related pages. After a few refreshes my Cart page in IE shows the
>> same
>> >> Cart that I built out in FF and after examining the cookie headers
>> the
>> >> encrypted data appears to be identical.
>> >>
>> >> P::Session and its related modules have been completely removed
> from
>> >> use
>> >> Catalyst qw/.../, so I now know that it's not being caused by these
>> >> modules.
>> >>
>> >> Now that everything is stored in the browser's cookie, how is it
>> >> possible that two browsers get sent the same cookie header?
>> >
>> > Ok, now that I've looked at this some more I think this may actually
>> be
>> > caused by the proxy server. Currently the app is load balanced
> behind
>> an
>> > Apache/mod_proxy_balancer instance and I'm unable to replicate the
>> > problem when not going through the proxy. I had spent quite a bit of
>> > time looking at the proxy back when these issues started, but now
> I'm
>> > doing it again because it's about the only thing left.
>> >
>> > If it is Apache, I imagine it's related to mod_cache, although I
>> > *thought* I had it properly configured.
>> >
>> > As an exercise; after a simple restart of Apache on the proxy I was
>> > *unable* to duplicate the cookie issue after ten minutes of trying
>> > whereas this morning I had been able to do it after just a few
>> > refreshes.
>> >
>> > Has anyone experienced good/bad things with Apache as a frontend
>> > proxy/balancer/cache?
>> >
>> > I've seen mention of Varnish on this list as a good frontend proxy.
>> Any
>> > personal recommendations of it or others?
>> >
>> > TIA
>> > -matt pitts
>> >
>>
>> Varnish is very good, but it simply doesn't cache something if it has
>> a session cookie for exactly this reason.  There are ways to configure
>> it to do so, but if your performance requirements are such that you
>> must proxy you may want to couple Varnish with ESI.
>>
>> Jay Kuri wrote a nice article on this:
>> http://www.catalystframework.org/calendar/2008/17
>>
>> -J
>
> Thanks for the pointer!
>
> My first looks at Varnish have peaked my interest, but a big stumbling
> block is be the lack of SSL support. MyApp has some eCom I need
> end-to-end SSL, which Apache does nicely for me. My only option w/
> Varnish would be to put Apache, or an SSL accelerator ($$) in front of
> it and do SSL tunnels between the non-SSL stuff. Double-proxy with 2
> tiers of SSL tunnels and now my head is spinning.
>
> I guess if I continue to have issues w/ Apache I won't have a choice but
> to either figure it out or move on to something else.
>
> v/r
> -matt pitts
>

I run the SSL outside of Varnish completely, using nginx to proxy to
fastcgi.  Varnish is only for static assets as a caching proxy.

To handle serving pages between SSL and not, I simply flip my static
path whilst instead of SSL using this macro:

sub static_uri {
    my ( $c, $asset, $query ) = @_;
    my $static_path = $c->stash->{page}->{static_root} ||
$c->view('TT')->config->{static_root} || '/static';
    my $uri;
    if ( $static_path =~ /^https?/ ) {
        $static_path =~ s#/$##;
        $uri = URI->new( "$static_path/$asset" );
        if ( ref $query eq 'HASH' ) {
            $uri->query_form( $query );
        }
    } else {
        # TODO: Find out why query needs to be defined, otherwise
        # local URIs have a trailing slash
        if ( $query and $query eq 'HASH' ) {
            $uri = $c->uri_for( $static_path, $asset, $query );
        } else {
            $uri = $c->uri_for( $static_path, $asset );
        }
    }
    return $uri;
}


This is well off the original subject, and I should probably wiki this up :)

-J