[Catalyst] Caching access to objects.

[Bill Moseley]

I have a request that looks like /object/$id/big_thing which returns a
related big_thing after first making sure the user has access to the object
of the given $id.

It turns out that checking the access and fetching big_thing are both slow
operations so only want to do them once and cache.  Also, big_thing is so
big it needs to be returned to the client in parts -- and returned in any
order.  So, the client may request:

   /object/$id/big_thing/2  -- part 2
   /object/$id/big_thing/211  -- part 211

First, what's a good approach to caching access?  What I've done in a
similar case is create a digest of the "$id.$user_id.$secret" so
in subsequent requests an access token can be used.  If a valid token is
provided then I don't need to do the expensive access check:

   /object/$id/big_thing/4?auth=3D1717a8f87b87a8c787f219

I'll sometimes include an "expires" time (and include in digest) to expire
the request.  Anyone have a different suggestion?


I also have to figure out caching for big thing which can be up towards
10MB, although most are a few 100K.  It comes from a third-part web service
and can be quite large and made up of many small parts. The parts are then
fetched by the client via AJAX requests so they should be returned very
fast.  Only a few parts may ever be requested.  So, I have to look at
breaking it up into all the parts when first fetched and caching all the
parts vs. caching the entire "big_thing" and for each part request pull out
the part.   I think part of the answer here is the Catalyst app should not
be serving this at all.

The Catalyst app probably needs to grant access and then have the client
fetch from a separate set of servers that can broker multiple request via a
queue into a single fetch of big_thing and then serve statically (perhaps
using byte ranges).

Anyone doing something similar?




-- =

Bill Moseley
moseley at hank.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20100929/43f03=
fbd/attachment.htm