[Catalyst-commits] r7250 - in
trunk/examples/CatalystAdvent/root/2007: . pen
zarquon at dev.catalyst.perl.org
zarquon at dev.catalyst.perl.org
Sun Dec 9 00:00:47 GMT 2007
Author: zarquon
Date: 2007-12-09 00:00:46 +0000 (Sun, 09 Dec 2007)
New Revision: 7250
Added:
trunk/examples/CatalystAdvent/root/2007/9.pod
Removed:
trunk/examples/CatalystAdvent/root/2007/pen/15.pod
Modified:
trunk/examples/CatalystAdvent/root/2007/8.pod
Log:
fixed xml parse error and bring day 9 live
Modified: trunk/examples/CatalystAdvent/root/2007/8.pod
===================================================================
--- trunk/examples/CatalystAdvent/root/2007/8.pod 2007-12-08 01:40:12 UTC (rev 7249)
+++ trunk/examples/CatalystAdvent/root/2007/8.pod 2007-12-09 00:00:46 UTC (rev 7250)
@@ -52,13 +52,9 @@
% make test
% make install
-=over 1
-
Note: If any of the above steps hang for any reason, you may need to go
into the build directory (~/.cpan/build/CPAN-1.92) and run make install.
-=back
-
You should now be setup with a ~/perl5 directory, which will be used for
further module installations via CPAN. To complete installation, setup your
environment by adding it to your .bashrc for subsequent logins and reload the
Copied: trunk/examples/CatalystAdvent/root/2007/9.pod (from rev 7189, trunk/examples/CatalystAdvent/root/2007/pen/15.pod)
===================================================================
--- trunk/examples/CatalystAdvent/root/2007/9.pod (rev 0)
+++ trunk/examples/CatalystAdvent/root/2007/9.pod 2007-12-09 00:00:46 UTC (rev 7250)
@@ -0,0 +1,252 @@
+=head1 Advanced Search in web DBIx::Class based applications (with tags, full text search and searching by location)
+
+There is a bit of irony that I write that article, for people to learn from it,
+while in fact it is my failing to properly wrap my head around the problem and encapsulate
+my solution into a CPAN library that forces me to write an article in the first
+place. But maybe someone smarter then me will read it and write that CPAN
+module?
+
+=head2 The Problem
+
+It is a common case that on a web site you need an 'advanced search' feature
+that let's the user combine simple predicates into more elaborated queries.
+Usually all the predicates are joined with an 'AND' - and the technique I
+describe here is based on this assumption. At first this task looks pretty
+simple. You have a list of parameters from the web form, corresponding the the
+columns of some database table, you have the values of those parameters and you
+need to find all the records in that table that have those values in those
+columns. You just do:
+
+ my @records = $schema->ResultSet( 'MyTable' )->search(
+ $reqest->params,
+ { page => 1, rows => 5 }
+ );
+
+Simple.
+
+Then of course you add parameter validation and filtering - but this is outside
+of the scope of this article.
+
+Then you need to add checks on columns not only in the searched table, but also
+on columns from related records and things become more complicated. What I
+propose here is a solution that works for the simple case, solves the related
+tables case, and also is easily extendable to cover more complicated predicates
+like searching by a conjunction of tags, full text searches or searches by
+location. I also add implementation of those 'advanced' predicates (using the
+PostgreSQL extensions for full text search and location based search).
+
+=head2 The Solution
+
+The solution I propose is this simple module:
+
+ package Ymogen2::DB::RSSearchBase;
+
+ use strict;
+ use warnings;
+
+ use base qw( DBIx::Class::ResultSet );
+
+ sub advanced_search {
+ my ( $self, $params, $attrs ) = @_;
+ for my $column ( keys %$params ){
+ if( my $search = $self->can( 'search_for_' . $column ) ){
+ $self = $self->$search( $params );
+ next;
+ }
+ my ( $full_name, $relation ) = simple_predicate( $column );
+ my $join;
+ $join = { join => [ $relation ] } if $relation;
+ $self = $self->search(
+ { $full_name => $params->{$column} },
+ $join,
+ );
+ }
+ $self = $self->search( {}, $attrs );
+ return (wantarray ? $self->all : $self)
+ }
+
+You use it like that:
+
+ my @records = $schema->ResultSet( 'MyTable' )->advanced_search(
+ \%search_params,
+ { page => 1, rows => 5 }
+ );
+
+But first you need to make your ResultSet class inherit from it. This can be
+done in several ways, what we do is adding:
+
+ __PACKAGE__->resultset_class(__PACKAGE__ . '::ResultSet');
+
+ package Ymogen2::DB::Schema::Users::ResultSet;
+
+ use base qw( Ymogen2::DB::RSSearchBase );
+
+
+
+to MyTable.pm.
+
+For the simple case it works just like the familiar 'search' method of the
+L<DBIx::Class::ResultSet class>. But it also works for searching in related
+records. For that we have the simple_predicate function. It looks like that:
+
+ sub simple_predicate {
+ my $field = shift;
+ if( $field =~ /(.*?)\.(.*)/ ){
+ my $first = $1;
+ my $rest = $2;
+ my( $column, $join ) = simple_predicate( $rest );
+ if ( $join ) {
+ return $column, { $first => $join };
+ }else{
+ return $first . '.' . $column, $first;
+ }
+ }elsif( $field ){
+ return $field;
+ }else{
+ return;
+ }
+
+What it does is parsing column names of the format:
+'relationship1.relationship2.relationship3.column' into 'relationship3.column'
+- the fully qualified column name and a
+'{ relationship1 => { relationship2 => relationship3 } }' hash used for joining
+the appriopriate tables.
+
+(I had also a non-recursive version - but it was not simpler)
+
+So now you can do this:
+
+ my @records = $schema->ResultSet( 'MyTable' )->advanced_search(
+ {
+ column1 => 'value1',
+ column2 => 'value2',
+ some_relation.column => 'value3',
+ some_other_relation.some_third_relation.column => 'value4',
+ },
+ { page => 1, rows => 5 }
+ );
+
+Useful?
+We use it.
+
+=head2 The Extensions
+
+But the real advantage of this approach is how easily it can be extended.
+
+=head3 Tags
+
+For example let say we need to search by conjunction of tags like that:
+
+ my @records = $schema->ResultSet( 'MyTable' )->advanced_search( {
+ column1 => 'value1',
+ some_other_relation.some_third_relation.column => 'value4',
+ tags => [ qw/ tag1 tag2 tag3/ ],
+ });
+
+What we need is a method called 'search_for_tags' that will do the search. The
+nice thing is that we don't need to warry how this will be combined with the
+rest of the predicates - DBIC will do the right thing (for and 'AND' relation).
+
+Here is the method:
+
+ sub search_for_tags {
+ my ( $self, $params ) = @_;
+ my @tags = @{$params->{tags}};
+ my %search_params;
+ my $suffix = '';
+ my $i = 1;
+ for my $tag ( @tags ){
+ $search_params{'tags' . $suffix . '.name'} = $tag;
+ $suffix = '_' . ++$i;
+ }
+ my @joins = ( 'tags' ) x scalar( @tags );
+ $self = $self->search( \%search_params, {
+ join => \@joins,
+ }
+ );
+ return $self;
+ }
+
+It builds a query like that:
+
+ SELECT * FROM MyTable me, Tags tags, Tags tags_2, Tags tags_3
+ WHERE tags.mytable_id = me.id AND tags.tag = 'tag1' AND
+ tags_2.mytable_id = me.id AND tags_2.tag = 'tag2' AND
+ tags_3.mytable_id = me.id AND tags_3.tag = 'tag3'
+
+This query will use indices and should be fast (a more detailed cover of this
+technique you can find at my blog at:
+http://perlalchemy.blogspot.com/2006/10/tags-and-search-and-dbixclass.html).
+
+*Attention:* You need the 0.08008 version of DBIx::Class for this to work properly.
+
+=head3 Full Text Search
+
+For full text search I use the PostgreSQL tsearch2 engine here.
+
+=head3 Search by Proximity
+
+For searching by proximity I use the PostgreSQL geometric functions
+(http://www.postgresql.org/docs/8.2/interactive/functions-geometry.html).
+There is
+one problem with it - the distance operator assumes planar coordinates,
+while for the interesting thing is to search geografic data with the standard
+latitude/longitude coordinates. In our solution we just don't care about
+being exact and just multiply the 'distance' in degrees by 50 to get approximate
+distance in miles. The actual proportion is about 43 for latitude and 69 for
+longitude at about the London's longitude, it would be possible to get quite
+good results by dividing the latitude and longitude by those numbers in the
+database - but I would rather have good data in the database then more exact
+results. Maybe at some point we shell switch to use some real geografic
+distance functions (I've seen a PosgreSQL extension to do that - but I was
+scared a bit by it's experimental status).
+
+So here is the function used to filter the results by proximity to a place:
+
+sub search_for_distance {
+ my ( $self, $rs, $params ) = @_;
+ my $lat_long = $params->{lat_long};
+ my $distance = $params->{distance} / 50;
+ # around London the actual proportions are around 43 for latitude
+ # and 69 for longitude
+ return $rs->search(
+ { "(lat_long <-> '$lat_long'::POINT) < " => \$distance },
+ { join => 'location' }
+ );
+}
+
+This function assumes there are two parameters on the $params hash: distance
+and lat_long (lattitude/logintude coordinates). The location data in our
+database are in a separate table called 'location'.
+
+We also use another search extension:
+
+sub search_for_lat_long {
+ my ( $self, $rs, $params ) = @_;
+ my $lat_long = $params->{lat_long};
+ $rs = $rs->search( undef,
+ {
+ join => 'location',
+ '+select' => [ \"(lat_long <-> '$lat_long'::POINT) AS distance" ],
+ '+as' => 'distance',
+ order_by => 'distance ASC',
+ }
+ );
+ return $rs;
+}
+
+This function sorts the results by proximity to the point determined by the
+lat_long coordinates. This way the user does not need to specify the
+maximum distance - the closest results are displayed on the first pages
+anyway - and that is enough for most of the searches.
+
+=head2 And Beyond
+
+In the search by proximity extension I've used ordering of the results. There
+is one problem with this. We use many 'search' calls on the resultset
+to cumulate the predicates - but we cannot do this with the order. Only the
+last 'order_by' parameter used in the 'search' calls is effective. I believe
+it would be useful to have a similar 'cumulative' behaviour for 'order_by'
+and we can add this to 'advanced_search' (or perhaps it can be added to
+the core DBIC search method).
+
Deleted: trunk/examples/CatalystAdvent/root/2007/pen/15.pod
===================================================================
--- trunk/examples/CatalystAdvent/root/2007/pen/15.pod 2007-12-08 01:40:12 UTC (rev 7249)
+++ trunk/examples/CatalystAdvent/root/2007/pen/15.pod 2007-12-09 00:00:46 UTC (rev 7250)
@@ -1,252 +0,0 @@
-=head1 Advanced Search in web DBIx::Class based applications (with tags, full text search and searching by location)
-
-There is a bit of irony that I write that article, for people to learn from it,
-while in fact it is my failing to properly wrap my head around the problem and encapsulate
-my solution into a CPAN library that forces me to write an article in the first
-place. But maybe someone smarter then me will read it and write that CPAN
-module?
-
-=head2 The Problem
-
-It is a common case that on a web site you need an 'advanced search' feature
-that let's the user combine simple predicates into more elaborated queries.
-Usually all the predicates are joined with an 'AND' - and the technique I
-describe here is based on this assumption. At first this task looks pretty
-simple. You have a list of parameters from the web form, corresponding the the
-columns of some database table, you have the values of those parameters and you
-need to find all the records in that table that have those values in those
-columns. You just do:
-
- my @records = $schema->ResultSet( 'MyTable' )->search(
- $reqest->params,
- { page => 1, rows => 5 }
- );
-
-Simple.
-
-Then of course you add parameter validation and filtering - but this is outside
-of the scope of this article.
-
-Then you need to add checks on columns not only in the searched table, but also
-on columns from related records and things become more complicated. What I
-propose here is a solution that works for the simple case, solves the related
-tables case, and also is easily extendable to cover more complicated predicates
-like searching by a conjunction of tags, full text searches or searches by
-location. I also add implementation of those 'advanced' predicates (using the
-PostgreSQL extensions for full text search and location based search).
-
-=head2 The Solution
-
-The solution I propose is this simple module:
-
- package Ymogen2::DB::RSSearchBase;
-
- use strict;
- use warnings;
-
- use base qw( DBIx::Class::ResultSet );
-
- sub advanced_search {
- my ( $self, $params, $attrs ) = @_;
- for my $column ( keys %$params ){
- if( my $search = $self->can( 'search_for_' . $column ) ){
- $self = $self->$search( $params );
- next;
- }
- my ( $full_name, $relation ) = simple_predicate( $column );
- my $join;
- $join = { join => [ $relation ] } if $relation;
- $self = $self->search(
- { $full_name => $params->{$column} },
- $join,
- );
- }
- $self = $self->search( {}, $attrs );
- return (wantarray ? $self->all : $self)
- }
-
-You use it like that:
-
- my @records = $schema->ResultSet( 'MyTable' )->advanced_search(
- \%search_params,
- { page => 1, rows => 5 }
- );
-
-But first you need to make your ResultSet class inherit from it. This can be
-done in several ways, what we do is adding:
-
- __PACKAGE__->resultset_class(__PACKAGE__ . '::ResultSet');
-
- package Ymogen2::DB::Schema::Users::ResultSet;
-
- use base qw( Ymogen2::DB::RSSearchBase );
-
-
-
-to MyTable.pm.
-
-For the simple case it works just like the familiar 'search' method of the
-L<DBIx::Class::ResultSet class>. But it also works for searching in related
-records. For that we have the simple_predicate function. It looks like that:
-
- sub simple_predicate {
- my $field = shift;
- if( $field =~ /(.*?)\.(.*)/ ){
- my $first = $1;
- my $rest = $2;
- my( $column, $join ) = simple_predicate( $rest );
- if ( $join ) {
- return $column, { $first => $join };
- }else{
- return $first . '.' . $column, $first;
- }
- }elsif( $field ){
- return $field;
- }else{
- return;
- }
-
-What it does is parsing column names of the format:
-'relationship1.relationship2.relationship3.column' into 'relationship3.column'
-- the fully qualified column name and a
-'{ relationship1 => { relationship2 => relationship3 } }' hash used for joining
-the appriopriate tables.
-
-(I had also a non-recursive version - but it was not simpler)
-
-So now you can do this:
-
- my @records = $schema->ResultSet( 'MyTable' )->advanced_search(
- {
- column1 => 'value1',
- column2 => 'value2',
- some_relation.column => 'value3',
- some_other_relation.some_third_relation.column => 'value4',
- },
- { page => 1, rows => 5 }
- );
-
-Useful?
-We use it.
-
-=head2 The Extensions
-
-But the real advantage of this approach is how easily it can be extended.
-
-=head3 Tags
-
-For example let say we need to search by conjunction of tags like that:
-
- my @records = $schema->ResultSet( 'MyTable' )->advanced_search( {
- column1 => 'value1',
- some_other_relation.some_third_relation.column => 'value4',
- tags => [ qw/ tag1 tag2 tag3/ ],
- });
-
-What we need is a method called 'search_for_tags' that will do the search. The
-nice thing is that we don't need to warry how this will be combined with the
-rest of the predicates - DBIC will do the right thing (for and 'AND' relation).
-
-Here is the method:
-
- sub search_for_tags {
- my ( $self, $params ) = @_;
- my @tags = @{$params->{tags}};
- my %search_params;
- my $suffix = '';
- my $i = 1;
- for my $tag ( @tags ){
- $search_params{'tags' . $suffix . '.name'} = $tag;
- $suffix = '_' . ++$i;
- }
- my @joins = ( 'tags' ) x scalar( @tags );
- $self = $self->search( \%search_params, {
- join => \@joins,
- }
- );
- return $self;
- }
-
-It builds a query like that:
-
- SELECT * FROM MyTable me, Tags tags, Tags tags_2, Tags tags_3
- WHERE tags.mytable_id = me.id AND tags.tag = 'tag1' AND
- tags_2.mytable_id = me.id AND tags_2.tag = 'tag2' AND
- tags_3.mytable_id = me.id AND tags_3.tag = 'tag3'
-
-This query will use indices and should be fast (a more detailed cover of this
-technique you can find at my blog at:
-http://perlalchemy.blogspot.com/2006/10/tags-and-search-and-dbixclass.html).
-
-*Attention:* You need the 0.08008 version of DBIx::Class for this to work properly.
-
-=head3 Full Text Search
-
-For full text search I use the PostgreSQL tsearch2 engine here.
-
-=head3 Search by Proximity
-
-For searching by proximity I use the PostgreSQL geometric functions
-(http://www.postgresql.org/docs/8.2/interactive/functions-geometry.html).
-There is
-one problem with it - the distance operator assumes planar coordinates,
-while for the interesting thing is to search geografic data with the standard
-latitude/longitude coordinates. In our solution we just don't care about
-being exact and just multiply the 'distance' in degrees by 50 to get approximate
-distance in miles. The actual proportion is about 43 for latitude and 69 for
-longitude at about the London's longitude, it would be possible to get quite
-good results by dividing the latitude and longitude by those numbers in the
-database - but I would rather have good data in the database then more exact
-results. Maybe at some point we shell switch to use some real geografic
-distance functions (I've seen a PosgreSQL extension to do that - but I was
-scared a bit by it's experimental status).
-
-So here is the function used to filter the results by proximity to a place:
-
-sub search_for_distance {
- my ( $self, $rs, $params ) = @_;
- my $lat_long = $params->{lat_long};
- my $distance = $params->{distance} / 50;
- # around London the actual proportions are around 43 for latitude
- # and 69 for longitude
- return $rs->search(
- { "(lat_long <-> '$lat_long'::POINT) < " => \$distance },
- { join => 'location' }
- );
-}
-
-This function assumes there are two parameters on the $params hash: distance
-and lat_long (lattitude/logintude coordinates). The location data in our
-database are in a separate table called 'location'.
-
-We also use another search extension:
-
-sub search_for_lat_long {
- my ( $self, $rs, $params ) = @_;
- my $lat_long = $params->{lat_long};
- $rs = $rs->search( undef,
- {
- join => 'location',
- '+select' => [ \"(lat_long <-> '$lat_long'::POINT) AS distance" ],
- '+as' => 'distance',
- order_by => 'distance ASC',
- }
- );
- return $rs;
-}
-
-This function sorts the results by proximity to the point determined by the
-lat_long coordinates. This way the user does not need to specify the
-maximum distance - the closest results are displayed on the first pages
-anyway - and that is enough for most of the searches.
-
-=head2 And Beyond
-
-In the search by proximity extension I've used ordering of the results. There
-is one problem with this. We use many 'search' calls on the resultset
-to cumulate the predicates - but we cannot do this with the order. Only the
-last 'order_by' parameter used in the 'search' calls is effective. I believe
-it would be useful to have a similar 'cumulative' behaviour for 'order_by'
-and we can add this to 'advanced_search' (or perhaps it can be added to
-the core DBIC search method).
-
More information about the Catalyst-commits
mailing list