[Catalyst] Distributed session storage problems/questions

[Jay K]

Hello Matt,

This looks like a version mismatch of the  
Catalyst::Authentication::Store::DBIx::Class module on the two machines.

Recently I added functionality to the module which allow users to  
avoid the DB hit when restoring a user from the session, part of this  
change entailed changing the data stored in the session for a user.

The old module simply stored the user's ID in the session.  The new  
way stores all the user information in the session, and then attempts  
to restore it either using the id field or, if caching is turned on,  
all the fields stored.

It sounds like one of your machines is using the older version of the  
module and one is using the newer version of the module.

Bring the modules into sync and the problem should go away.  Though I  
expect that at least half of your sessions will need to be reset  
because you probably have 2 different types of data in there now from  
authentication on the two machines.

Jay


On Feb 20, 2008, at 11:08 AM, Matt Pitts wrote:

> We have a load balanced application where the proxy does *not* force  
> any type of sticky session with each backend. The app was running in  
> production as a single instance (pre load-balanced) and we moved it  
> behind the proxy to be able to add backends and make it load  
> balanced. I had the app in test for several weeks as a load-balanced  
> setup using memcached to store sessions across the 2 backends and I  
> am pretty confident that the auth areas that relied on session  
> storage were working fine.
>
> As of last night the app throws errors (in UAT) about every other  
> request when inside an auth area with the 2 backends enabled. I  
> can’t get the actual message as this time, but it basically was a  
> “Can’t use 12345 as a HASHREF…” type of error where ‘12345’ is the  
> ID of the user I was authenticated as. If I disable one of the  
> backends, clear my cookies and re-login the entire session works  
> fine. As soon as I re-enable the other backend I start getting the  
> errors again and it doesn’t matter which backend I use first, they  
> both work independent of one another. Obviously, the first backend  
> that stores the session works and the other doesn’t, but what I  
> don’t know is why.
>
> As I stated above, I’m pretty confident that all the auth areas of  
> the site were functioning in the load-balanced setup before we went  
> to PROD. So, I looked at what changed and the only thing that  
> changed related to session storage was that I added a 2nd memcached  
> backend instance to the config file. I wondered if this might have  
> been part of the problem, but I have since dropped back to only one  
> memcached instance, restarted it and the errors still persist.
>
> Here is the relevant config section:
>
> session:
>   flash_to_stash: 1
>   expires: 3600
>   memcached_new_args:
>     data:
>       - 10.xxx.xxx.xxx:11211
>     namespace: tangeroutlet_tangerweb_session_uat
>
> In an effort to troubleshoot this I have also tried to implement  
> memcached storage using C::P::Session::Store::Cache and  
> C::P::Cache::Memcached, but I haven’t gotten this to work yet.
>
> I’m hoping that answers to the following questions will guide me to  
> a solution:
>
> 1. What is the acceptable way to use memcached for session storage?  
> Is it using C::P::Session::Store::Memcached or is it  
> C::P::Session::Store::Cache in conjunction with  
> C::P::Cache::Memcached?
>
> 2. Is memcached even a “good” way to store sessions? DBIC seems to  
> be a popular option, but I figured memcached’s considerable speed  
> would make it the preferred choice.
>
> 3. When using memcached to store sessions, is it OK to have more  
> than one backend? At first I didn’t think anything about adding  
> another backend, but then I thought that the 2 memcached instances  
> wouldn’t have the same set of data which would cause problems with  
> sessions, unless of course the session writes get distributed to all  
> the backends. I’ve looked into the source of  
> C::P::Session::Store::Memcached, Cache::Memcached::Managed and  
> Cache::Memcached and cannot tell if the writes are spread across all  
> backends. Does anyone know the answer to this?
>
> 4. Is it just crazy to run a load balanced setup without some type  
> of sticky session setup on the proxy? If so, any implementations of  
> this using Apache 2.x mod_proxy(_balancer) as the frontend would be  
> greatly appreciated.
>
> At this point I think I’d be willing to take a performance hit and  
> store sessions in DBIC if it’s going to work better in a distributed  
> environment.
>
> TIA for any input.
>
> v/r
>
> -matt pitts
> _______________________________________________
> List: Catalyst at lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/

---
"May we not return to those scoundrels of old, the illustrious  
founders of superstition and fanaticism, who first took the knife from  
the altar to make victims of those who refused to be their disciples."  
- Voltaire