[Catalyst] Beginner Info (Part 1, perhaps with more to follow)

[Geoffrey Ferrari]

I made a request recently for better documentation for beginners. As  
Kieren Demont pointed out, Catalyst is a community project, so if I  
think there's a lack of beginner documentation, perhaps I ought to  
help write some.

A few other helpful bods have posted (links to) introductory  
documentation in response to my request. But I saw nothing that  
answered specifically to my own needs. In particular, what I think is  
lacking is a general conceptual overview of Catalyst which answers  
some very basic questions like those listed in my previous email.  
Since this is likely to be long enough as it is, I'd prefer not to  
include a potted history of CGI programming (as did raptor on 13th  
January), nor jump straight in an example application (as does Pete  
Sergeant's tutorial in his 13th Jan email). Don't get me wrong, both  
of these fulfil useful functions. But to Pete I say that I followed  
several example application tutorials, and still had difficulty  
inferring general principles from the particular examples. And to  
Raptor I say that I suspect his potted history of CGI was intended  
mainly as just one way of giving a conceptual overview of catalyst by  
illustrating the design choices and the problems they are supposed to  
solve. If that's the case, it's more direct to aim straight for the  
conceptual overview.

So, with this in mind, I've written an extremely general overview of  
Catalyst. It's adopts the principle that things should be explained  
so as to make Catalyst comprehensible to an audience with a specified  
set of background knowledge. If anyone wants to extend it, I beg them  
to abide by that principle!

Two things count against my writing this kind of beginner's  
documentation. First, I'm a beginner myself, which may result in the  
blind attempting to lead the blind. Second, my free time is limited.  
Nevertheless, I think there is a great need for beginner's  
documentation, so I'll try to help write some as best I can.

Feedback to this list please.

Beginner Info Part 1
===============

=head1 What is Catalyst?

If the answer "Catalyst is an elegant MVC web application framework"  
does not help you, you've come to the right place. This document is  
for beginners: beginners to web application frameworks in general,  
and beginners to Catalyst in particular. The only knowledge it  
presupposes is a reasonable level of familiarity with:

- the perl programming language
- how to split your perl code into packages and modules
- object-oriented programming in perl
- perl programming for the web (to give its proper name, CGI  
programming)

If you are not reasonably familiar with all of these, there are many  
resources on the net and some excellent books that cover these topics.

The purpose of this document is to give someone with only the  
knowledge presupposed above a conceptual overview of Catalyst. A  
conceptual overview is not a manual or a tutorial, though a beginner  
should find it a helpful complement to a manual or tutorial. A  
conceptual overview aims to do two things:

(1) to describe at the most general level what Catalyst is
(2) to help the user to understand why Catalyst was designed to be as  
it is.


=head1 So again, what is Catalyst?

Catalyst, at the most basic level, is a perl application that aims to  
help you write perl applications. I shan't try to define what an  
application is - I assume that you already have a reasonable grasp of  
that - but let's assume for simplicity that an application is made up  
of some files stored on disk, and that these files, when activated,  
result in a certain pattern of behaviour. Catalyst itself, and the  
applications that it will help you to write, are like that. We can  
also assume for present purposes that all the applications that  
Catalyst will help you to write are CGI programmes. CGI stands for  
Common Gateway Interface, and it refers to a set of protocols which  
govern requests and responses over the internet. The vast majority of  
web sites conform to CGI specifications. To forestall potential  
confusion, you should know that using the famous perl module CGI.pm  
is only one way of doing web programming in Perl. Both Catalyst and  
CGI.pm have the aim of helping you to write web (that is, CGI)  
applications.

So, Catalyst is a perl application which tries to help you write CGI  
applications. The key question, then, is how does Catalyst help?  
Catalyst provides help in several forms, but the most central kind -  
and it is most central because the majority of the help that Catalyst  
provides depends on it - is a set of programming conventions.

A set of programming conventions is a set of default rules - some of  
which can be overriden, some of which can't - about how your perl web  
application will be structured. We'll get on to looking at some of  
those rules later, but why, you might ask, is it worth the trouble of  
following the Catalyst rules? There are basically only two reasons  
why you should follow the rules of Catalyst, or any other good web  
application framework. (A web application framework, by the way, is  
really just a set of conventions for writing web applications,  
perhaps along with an application that will help you write  
applications that conform to those conventions. See! You're making  
progress already. Anyway, back to the reasons for following the rules):

Following the rules of a web application framework is good because:
(i) it solves a coordination problem
(ii) it gives elegant solutions to some common problems.

=head2 The Coordination Problem

Let's talk about the co-ordination problem first. Co-ordination  
problems arise when you have at least two agents working towards a  
common goal. In the context of a web application framework like  
Catalyst, one set of agents includes the end users, the programmers  
like you and me. The other set of agents strictly speaking is the  
Catalyst development team, who write the Catalyst programme and make  
it available to the end users. But for simplicity, we'll think of the  
second agent as being the Catalyst programme itself. So now we have  
our two agents, Catalyst and you. The goal, of course ,is to write  
web applications.

On your own you can probably write some pretty cool web applications.  
But, if you write your web applications in accordance with the  
Catalyst conventions, then Catalyst will be able to do at least some  
of the hard work for you. This has a powerful and potentially  
frightening result: your perl code now works differently. Don't worry  
- it's still perl! But now Catalyst may take your perl code and do  
things with it that you wouldn't normally expect and at first won't  
be able to understand. This is precisely the sort of thing that is  
frightening for a beginner! But let me reassure you. The things that  
Catalyst will do with your code are things that most people, most of  
the time, find extremely helpful. And whilst you may be frightened by  
what Catalyst does, there is an obvious solution: what all beginners  
must do is learn the Catalyst conventions. That way you will  
understand what Catalyst will do with your code, and thus come to  
write code which takes advantage of all the help that Catalyst offers.

To help you to understand how conventions can be helpful, let's  
consider an analogy. Usually, if you twitch your finger it  
achieves... nothing much. But that's not always the case. In the  
context, for example, of an auction house, twitching one's finger  
hooks into a set of conventions which give it additional  
significance. In an auction, a twitch of the finger can register a  
bid, indicate a legally-binding commitment to buy, and indicate that  
one will stop by the office after the auction to settle one's  
account, thus ensuring that if the auctioneers want your money,  
there'll be someone there to take it. The conventions of the auction  
house, just like the conventions of Catalyst, are apt to frighten  
beginners. But for those in the know, having conventions ensures that  
things can be bought and sold with minimal effort. And just as with  
Catalyst, the conventions of the auction house solve a co-ordination  
problem - in this case, the problem is how you and the auctioneers  
can achieve the common aim of a legal, efficient, exchange of goods.

=head2 Elegant Solutions to Common Problems

The second kind of help that Catalyst provides is that some of its  
rules were chosen to help you avoid common pitfalls, or, where the  
problems are unavoidable, to help you to solve them in an efficient -  
perhaps even beautiful or elegant - way. Of course, you could just  
try to follow the Catalyst rules without understanding them, but it  
will probably make your life easier if you understand why some of  
those rules were chosen.

One of the most common problems that Catalyst tries to help you solve  
is that of designing one's application so that it is easy to  
understand and easy to maintain. Over time, it became clear to some  
programmers that most applications have three areas of concern.  
What's an area of concern? Basically, an area of concern is one well- 
defined problem. All applications try to solve problems. But it's  
also possible to sub-divide the big problem that an application tries  
to solve into smaller, sub-problems, which you can then deal with  
individually. So what I really mean is that over time it became clear  
that most applications try to solve three general kinds of sub-problem.

First of all, there is the problem of accessing a store of data,  
which your application will certainly want to read, and may also want  
to update. The part of your application that tries to solve this  
problem is called the Model. For our purposes, we don't care about  
the specifics of the data. It can be anything you like - your BT  
phone records, weather data, or the results of a poll. It's all data,  
and the problem of accessing is solved by the Model part of an  
application.

Second, of course, there is the problem of reporting things back to  
the user. The part of your application that tries to solve this  
problem is called the View. Again, we don't care what is reported  
back to the user, but most of the time your user will want some  
feedback on the results of his actions.

Finally, there is the problem of doing the right thing in response to  
your user's input. The part of your application which tries to solve  
this problem is called the Controller. This way of conceiving of the  
the various sub-problems of your application is called the Model-View- 
Controller design pattern. It's a design pattern because it's a set  
of rules for separating your code into three different parts each of  
which tries to solve just one of the three sub-problems just described.


=head2 An example

Let's recap for a second and discuss an example. I said above that  
the most central kind of help that Catalyst provides is a set of  
conventions. If you abide by those conventions when you write your  
web application, then your perl code will take on added significance  
and behave differently than it normally would. Let me give you an  
example. Whether you think this is an example of Catalyst solving the  
co-ordination problem or Catalyst providing an elegant solution to a  
common problem doesn't really matter. In fact, it's an example of  
Catalyst doing both.

The CGI protocol ensures that all web applications consist of a  
request from the user and a response back to the user from the  
application. The way the user makes a specific request is by making a  
request for a specific resource. The names for these resources comine  
in the form of a URI e.g. 'http://www.greatapp.com/users/list'. Now,  
one of the recurrent problems designers of web applications must face  
is how to ensure that the correct piece of code is executed in  
response to a request for a specific URI. One way or another, the  
application (broadly conceived) must have a way of mapping individual  
requests to invididual sections of code.

This is such a common problem that the Catalyst development team came  
up with an elegant solution to the problem. The way to solve this  
problem, according to Catalyst, is to let your Controller modules  
mirror your URLs. Of course, there is some flexibility in the  
mirroring. But in the simplest case, the Catalyst convention is that  
a URI that ends in, say, '.../users/list', will call the 'list'  
method of the 'users' controller module. As we shall see in a moment,  
Catalyst gets you to organise your packages and modules under three  
directories, one for each of the Model, View and Controller problems  
that your code tries to solve. So, in this case, we would have a  
controller module called Users.pm under the Controller directory, and  
in that module we would have a subroutine called 'list'. There are  
other ways to achieve the required URI to code mapping, but this is  
the simplest and one of the most often used.

But that's not all that Catalyst does for you. So far, all we've done  
is come up with a way of organising you code so that there is an  
intuitive mapping from URIs to subroutines. We haven't yet done  
anything to implement that the user requests are despatched according  
to that mapping. And that's the brilliant thing! If you use Catalyst,  
you don't have to! In simple terms, Catalyst automatically handles  
the URI to subroutine despatching for you. If you want to do  
something in response to a request for '../users/list', all you have  
to do is make a module, Users.pm, under your Controller directory,  
and then put a subroutine, 'list', in that module, and that is enough  
to ensure that that piece of code will get called in response to that  
request. So you see, Catalyst not only provides an elegant solution  
to the common problem of URI to code mappings, but since it 'knows'  
that you will follow those rules, Catalyst can do something else for  
you, too. Knowing that you'll follow the elegant solution to the  
problem of mapping URIs to code also solve sa co-ordination problem  
that allows Catalyst to perform automatic URI to subroutine  
despatching. And that's a really helpful thing for Catalyst to do!

In fact, as we'll see later, Catalyst does one more thing which is  
really helpful in this example. You know from past experience that if  
you have code in a module, say in .../Controller/Users.pm, then in  
order to deploy that module you have to have a  
'use ..::Controller::Users;' somewhere in your application. Well,  
there is a convention in Catalyst that all your code is located under  
the directories ../Model, ../Controller, and ../View. If you follow  
that convention, then, because Catalyst knows where to look for your  
modules, it can load them automatically for you. That way you don't  
need to worry about maintaining a lot of 'use' statements. If you put  
your modules where Catalyst knows to find them, they'll be loaded  
automatically.