[Catalyst] Hypothetical Site and Scalability Planning

[J. Shirley]

On 10/28/07, Robert Mah <rmah at pobox.com> wrote:
>
> Memcached is not distributed.  Thus, you can't support distributed
> session state with it.  Having only one server is obviously bad from
> a scalability and reliability point of view.
>
> Graphics in the database means that you are not taking advantage of
> the web server's built-in sendfile(2) handling of static files.
> This is bad for performance and thus, scalability.


I don't think that storing images in a database is a good idea, but you can
still use sendfile by pulling out of the database and caching them on disk
using any number of caching structures.

They key to scalability can be boiled down to distribution.  That is,
> can you distribute the load (CPU, memory, filesystem, database, etc.)
> to multiple servers in a near-linear way.  If you do it right, you
> also gain much increased reliability because you have no single point
> of failure.
>
> Distributed filesystems such as Lustre, Isilon, NetApp, OpenFiler
> or even something hacked together with DRDB or rsync replication
> will help.  Databases replication is available for most major RDBMs,
> including mysql, postgres, oracle, etc.  Consider using hardware
> loadbalancers to front-end your web applications.  Think about
> splitting your mod_perl processes from static file serving httpd
> processes.  I could go on, but I won't.
>
> Scalability planning isn't something you can do on the cheap (both
> mentally and financially).  Think hard, try to consider all sides
> and plan carefully.



Six Apart has built many things that help do it "on the cheap" and it's
worked pretty well for them and their users.  memcached and mogileFS do a
great job at what they do.  MySQL replication and clustering is also
available for free.  Tie that in with perlbal and you have 100% OSS
near-linear scalability.

Does it require more technical prowess?  Sure... but if you don't have that,
this is likely never going to be an issue.

-J
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20071028/95871=
43e/attachment.htm