[Catalyst] Re: Development environments and performance

[kevin montuori]

>>>>> "DR" == Dave Rolsky <autarch at urth.org> writes:


 DR> I'm thinking that a sane budget is $2,500 per developer machine
 DR> (with monitor). Is that unreasonable? Do you really need to save a
 DR> few hundred bucks on ram?

it's not so much that it's unreasonable (though the extra $700 apple
charges for 4 instead of 2 GB certainly is) as unnecessary.  actually,
the "at least" part made me chuckle.  i've honestly never felt
constrained by 2 GB, but whatever.

 >> i'd have to disagree.  if you have a bunch of junior developers
 >> writing code, a shared (to some extent) development environment can
 >> aid in enforcing good development habits.

 DR> This is what automation was invented for. At several past positions,
 DR> I've set things up so that development consisted of checking out the
 DR> source and running a "set up my dev env" script that created/updated
 DR> the database, inserted test data, set up any servers needed, etc.

so resources should not be shared in a development environment at all?
everyone gets their own oracle server *instance* (not database)?  it's
one thing to crank up a new database and slap in a schema, something
else entirely to maintain a separate instance of oracle.  i've seen
grown men cry trying to install it (and get the instantclient libs,
tnsnames.ora, &c working).  

 DR> That said, I think good developers should have some minimal sysadmin
 DR> skills, and should be comfortable setting up a DBMS or LDAP server on
 DR> their own machine, and difficult installations can be scripted.

that's naive.  not that i don't agree (i do, very much so).  the sad
reality is that we're having a difficult enough time finding developers
(for a $75k/year, 2-4 years experience position) who can even discuss
what LDAP is, never mind try to configure it.  maybe everyone in your
shop is a reasonable sysadmin, but that's not what i've seen.

 >> my suspicion is that in shops with poor shared development
 >> environments, the systems administration is more to blame for the
 >> suitability issues than the fact that the environment is shared.

 DR> Well, not if there's a _resource_ issue. If, as J Rockway described,
 DR> you have 40 people sharing one machine, you're probably screwed no
 DR> matter how good your sysadmins are.

of course you're right, but a good sysadmin wouldn't try to put 40
people on one machine.  and i wouldn't say that machine equals
environment.  whenever possible it's nice to develop (yes, actually
develop, not just test) on an environment similiar to what production
looks like.  if that's distributed fast CGI across more than one
machine, having developer access to a similiar environment will save on
having unplesant surprises later.

 >> catalyst allows for a particularly nice sandbox though, using the
 >> devlopment httpd.  we're having a lot of luck providing a (robust, but
 >> not 4GB per devloper!) shared dev/sandbox environment with each of 8
 >> or so developers running a dev httpd.  we then releasing code to
 >> integration for regression testing.  i'm certainly not seeing the
 >> performance problems that have been reported on this list.

 DR> Presumably that depends on how many devs you have. However, I'd be
 DR> going nuts if I had to deal with other devs changing the database
 DR> schema, or even just changing the state of the data while I'm trying
 DR> to develop against it.

presently, development (database and LDAP) schema changes are managed by
the release manager (which makes sense if the changes are going to end
up in integration, test, and production).  developers have access to
their own copies of the schema and data that can be refreshed at their
leisure, but DBA services are provided centrally, so backups (and
restores) "just happen" on sandbox databases.

 DR> I stand by my position that any place not providing individual
 DR> environments is backwards.

and i'd maintain that i've seen very successful setups that provided
centralized services for development.  i wouldn't advocate that everyone
try to share a working directory, or a database, or an apache instance,
but having decent administrators provide a reliable and consistent
dev environment has benefited us in a number of ways:

  -- code that's been tested by the developer in development will almost
     always run in integration because the versions of the libraries are
     consistent.  furthermore, there's a good understanding of what's
     required to deploy code that's been written against the development
     environment.  less so when people tell me that "it runs on my
     machine, i don't konw why it doesn't run in integration."  

  -- experienced developers new to the environment waste almost no time
     trying to futz around getting all the working parts configured
     correctly.  we run a setup script and volia, their databases, ldap
     instances, apache sandbox, SSO configuration, firewall
     configuration, &c. are complete and correct.  when upgrades are
     done to any of the software, these are mostly transparent to the
     developers.  and backups (and recovery!), security, access from
     off-site, redundant power, &c. are all included free of charge.

  -- junior developers end up in the middle of a working environment
     rather than frustrated trying to piece together their own bit by
     bit.  i realize that our shop is a little abnormal in that we're
     higher ed and end up with our share of just-graduated and
     work-study types, so this point is actually very important to me.

note that just because an environment is provided doesn't preclude
cranking out some code elsewhere.  but it does provide a proving ground
*before* commits are made to VC.  this almost 100% eliminates the "it
works on my desktop" complaint.

please don't misunderstand me, it's extremely possible to have lousy
centralized services (jonathan's was a great description of that
too commonplace scenario).  it's also possible to provide developers
with services, tools, and resources they need to get the job done
while removing the headaches of doing it all yourself.  if the choice
were between the former and "do it yourself", i'd be in favor of the
former too; my argument is that the latter can (and does, sometimes)
work.

obCatalyst (finally): part of the reason i like catalyst so much is the
ease with which it's possible to crank up individual development servers
(even, for instance, two versions of the same code) and i'd repeat that
i've found them to be very lightweight and work well in a shared
resource setting, your mileage probably varied.  

cheers.
k.

-- 
kevin montuori