[Catalyst-dev] Important Question About Recommended Environment For The Tutorial

[Tomas Doran]

On 8 Dec 2008, at 17:12, hkclark at gmail.com wrote:
> I have talked to MST over the years about trying to have a way to  
> automatically regression test the tutorial on an ongoing basis.  If  
> such a thing existed, it seems like a great solution -- we could  
> have a nightly cron job automatically tell us if some new module  
> broke either the install *or* the tutorial itself.

Assuming that the tutorial and all of the code it uses lives in  
subversion, then I've got a smoke-testing solution already setup  
which could be extended a little to do (some of) this.

> However, I'm not aware of any frameworks or tools to automatically  
> test all that.  Does anyone else know of one?  I spent some time  
> thinking about how to use some =for tags in the POD to automate the  
> testing, but I didn't get very far with it -- unless I'm missing  
> something, it seems like a big job to do it right.

Yeah, pulling the tutorial fragments out is somewhat harder..

> So far I have done it manually: I start with a "minimal" install of  
> Linux in VMWare, I run cat-install following the exact directions  
> in Catalyst::Manual::Installation::CentOS4 (almost always updating  
> them because of some dependency or module change requiring a hack),  
> manually walking through every command and and cutting and pasting  
> every chunk of code into the right place.  It's obviously very slow  
> (even though I have done it so many times I can almost do it in my  
> sleep), :-)  but it's the only way I know to make sure it works (at  
> least I know it worked the day I finished my testing... all bets  
> were off as soon as the underlying modules get updated the next day).

It would be nice to be able to use some of the CPAN testers smoke  
test tools to do this for you, but privately - i.e. build all the  
dependencies but report privately.

I'd certainly be interesting in helping to work on some sort of  
general project-wide smoking solution, as having an aggressive  
Catalyst (and dependencies) specific  testing would be good for the  
general quality / perception of quality of the project..

We're good at (and people have put a lot of effort into) making the  
dependency stack work well, but it'd be nice to be able to be  
proactive rather than reactive about these issues..

> Thoughts on this?  Baring an automated testing methodology that  
> really exercises all parts of the tutorial process (the install,  
> running the helper commands, copying code from the pod files,  
> etc.), it seems like something along the lines of Ubuntu is a  
> pretty good compromise.  Especially if we don't have a volunteer to  
> automate the testing process. :-)

I think we should try to do all of the above ;_)

> PS -- Note that with the Ubuntu approach, we do have the option of  
> having them use CPAN for one or more modules if we want to get  
> around a serious bug and/or pull in a module that newer than can be  
> found in Ubuntu universe.  And, because apt-get would do the heavy  
> lifting of getting the 172 modules/packages installed first, the  
> job would still be a lot faster, simplier, and more tightly  
> controlled than a raw build against CPAN.

I'd say that you show pulling everything using apt-get, but then  
building a local::lib, and install Task::Catalyst::Tutorial into it,  
so that you get any modules you need updating.. Does that sound like  
a sane compromise?

> PPS - Another idea I have thought about and seen discussed is  
> having a VMWare virtual appliance image available.  It sounds like  
> a great way to go, but we would need to find a way to maintain it.

Ah, well that's easy.

You can script vmware from perl, so you just make a new disk, attach  
it to a pre-existing machine, boot, login, debootstrap the new disk,  
install everything and do some setup, shut down the vmware machine..

I'm not volunteering for this, as basically I don't have the hardware  
to mess about with it outside of work time, but having done something  
similar, I can say it's possible and not too hard..

Cheers
t0m