[Catalyst] Out of Memory - File delivery issue

neil.lunn neil at mylunn.id.au
Fri May 3 07:55:10 GMT 2013


On 3/05/2013 2:19 AM, Craig Chant wrote:
> I can't work this out?
As life allows :) This works for me as an example.
Clearly, just replace the loop with whatever is generating your data, 
convert to CSV as shown and output via the abstracted handle.

In Controller:

sub index :Path :Args(0) {
   my ( $self, $c ) = @_;

   $c->response->headers->header(
     Content_Type =>  'application/vnd.ms-excel'
   );
   $c->response->headers->header(
     Content_Disposition => 'attachment;filename=NBCS_Export.csv'
   );

   $c->model('Export')->get_data();
}

sub end : Private {}

In Model:

package CSVExporter::Model::Export;
use base 'Catalyst::Model::Factory::PerRequest';

__PACKAGE__->config(
   class => 'CSVExporter::Domain::CSVExport'
);

sub prepare_arguments {
   my ( $self, $c ) = @_;
   return { res => $c->res };
}

1;

And finally in a seperate Class, for abstraction.

use Modern::Perl;
use MooseX::Declare;

class CSVExporter::Domain::CSVExport {

   use Text::CSV;
   use Moose::Util::TypeConstraints;

   has csv => (
     is          => 'ro',
     isa         => 'Text::CSV',
     lazy_build  => 1,
     handles     => [qw/ combine string /]
   );

   has res => (
     is          => 'ro',
     isa         => duck_type( [qw/ print write /] ),
     required    => 1,
     handles     => [qw/ print write /]
   );

   has data => (
     traits      => ['Array'],
     is          => 'ro',
     isa         => 'ArrayRef',
     lazy_build  => 1,
     handles     => {
       allData   => 'elements'
     }
   );

   method _build_csv() {
     my $csv = Text::CSV->new();
     $csv->eol( "\n" );
     return $csv;
   }

   method _build_data() {
     return [
       [ "one",    1, "First"  ],
       [ "two",    2, "Second" ],
       [ "three",  3, "Third"  ]
     ];
   }

   method get_data() {

     foreach my $line ( $self->allData ) {
       $self->combine( @{$line} );
       $self->print( $self->string() );
     }

   }

}


>
> I have in my model...
> ------------------------------------------------------------------
>        my $xls = "col1,col2,col3\n";
>
>          # open io handle
>          $io_handle = IO::Handle->new();
>          open my ($str_fh), '>', \$xls;
>
>          if ($io_handle->fdopen($str_fh,"w"))
>          {
>          $io_handle->print('"row1","row2","row3"' . "\n");
>         }
>
>     return $io_handle;
> --------------------------------------------
>
> in my controller...
> --------------------------------------------------------------
> # output header
>          $c->response->header(
>              Content_Type =>  'application/vnd.ms-excel',
>              Content_Disposition => 'attachment;filename=NBCS_Export.csv'
>              );
>
>          # output XLS data
>          $c->response->body($io_handle);
> ----------------------------------------------------------------------
>
> All I get is a blank XLS file?
>
> I can't work out how I create the IO::Handle object with the XLS data inside it?
>
> Thanks,
>
> Craig.
>
> -----Original Message-----
> From: Craig Chant [mailto:craig at homeloanpartnership.com]
> Sent: 02 May 2013 15:17
> To: The elegant MVC web framework
> Subject: RE: [Catalyst] Out of Memory - File delivery issue
>
> Awesome Luke, really appreciate the advice and guidance.
>
> It didn't feel right the way I have it and I get enough boots up my behind from the IRC , without adding to the problem with more bad code!
>
> I like this option (IO::Handle-style object instead of a string.) , just going to have to read up on how I create one!
>
> If Catlayst is going to baulk trying to parse strings, but likes IO::Handle objects, I can put my header output back in the controller and not couple the model to the response object.
>
> I can then pass back the IO::Handle object from my model and let catalyst output that!
>
> I'll give it a whirl and let you know how I get on.
>
> Regards,
>
> Craig.
>
> -----Original Message-----
> From: Lukas Thiemeier [mailto:spamcatcher at thiemeier.net]
> Sent: 02 May 2013 15:04
> To: The elegant MVC web framework
> Subject: Re: [Catalyst] Out of Memory - File delivery issue
>
> Hi again,
>
> FIRST:
>
> Your Controller Code looks good. I would use build-in  functionality whenever possible. In this case, using $c->res->content_type instead of setting the content type by hand. But your code should work just fine.
>
> Your are right. You  have to set the headers before using $c->res->print (or write). But this does not mean that you have to do this in the model.  You can set the headers in your Controller before running the model code.
>
> SECOND:
>
> IMO you are right about your concerns regarding the MVC architecture.
> The Model should provide the data and not deal with HTTP responses. On the other hand, the data has to be provided in some format. XML is a well known standard format for textual data presentation. Providing the data as XML is as good or as bad as providing the data as DBIC objects.
> (Well, not really. But close enough for this explanation).
>
> The cleanest (most MVC conform) way to do this would be to fetch the raw data from your model and create your XML in a special view. There are several ways to do this. You can create XML using a TT view. ( I have done this before, it works fine). Or you can use existing XML views like Catalyst::View::XML::Generator or Catalyst::View::XML::Hash::LX. (I have not used any of them, but the names are promising). I guess there are even more ways to do it...
>
> In your Case, you already have a Model which provides the XML Data (which is fine, as I said before). IMO, one of the great things about Catalyst is that it allows you to get the job done quickly. It makes reusing existing code easy. There is no reason to abandon your existing XML-creation code just because it doesn't fit the MVC layout. Doing so would be contra-productive.
>
> So, what can be done to re-use your XML Model and still fit into the MVC architecture? I see two ways:
>
> The first one would be to update your model that it writes its data to any IO::Handle compatible object. You can pass $c->res to your Model, which is IO::Handle compatible. Your model uses a Catalyst independent API to write out the data. Catalyst streams the data to the client. Your Model Code is still Catalyst independent and does not know that it is writing to a Catalyst::Response object. No tight coupling. You can reuse your model in non-catalyst applications and easily test its functionality using Test::More or any other test suite. ( I think this is more or less the way proposed by Neil)
>
> The second way (which I would prefer, since it is even more MVC conform) is the following: Update your Model to return an IO::Handle-style object instead of a string. You can fetch this object from the model in your controller, and pass it to $c->res->body. Catalyst will take care of streaming the data to the client in small chunks. You don't have to pass any Catalyst related objects to the model anymore. Your model returns the data as a well known standard object which happens to be suitable for streaming large amounts data with catalyst. No tight coupling at all. Problem solved. Have a Tea and celebrate.
>
> This is my opinion on this topic. I hope it helps you to find a way which fits your needs, and reduces your confusion about Models, Views and controllers in this specific case.
>
> Lukas
>
> _______________________________________________
> List: Catalyst at lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
> This Email and any attachments contain confidential information and is intended solely for the individual to whom it is addressed. If this Email has been misdirected, please notify the author as soon as possible. If you are not the intended recipient you must not disclose, distribute, copy, print or rely on any of the information contained, and all copies must be deleted immediately. Whilst we take reasonable steps to try to identify any software viruses, any attachments to this e-mail may nevertheless contain viruses, which our anti-virus software has failed to identify. You should therefore carry out your own anti-virus checks before opening any documents. HomeLoan Partnership will not accept any liability for damage caused by computer viruses emanating from any attachment or other document supplied with this e-mail. HomeLoan Partnership reserves the right to monitor and archive all e-mail communications through its network. No representative or employee of HomeLoan Partnership has the authority to enter into any contract on behalf of HomeLoan Partnership by email. HomeLoan Partnership is a trading name of H L Partnership Limited, registered in England and Wales with Registration Number 5011722. Registered office: 26-34 Old Street, London, EC1V 9QQ. H L Partnership Limited is authorised and regulated by the Financial Conduct Authority.
>
> _______________________________________________
> List: Catalyst at lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
> This Email and any attachments contain confidential information and is intended solely for the individual to whom it is addressed. If this Email has been misdirected, please notify the author as soon as possible. If you are not the intended recipient you must not disclose, distribute, copy, print or rely on any of the information contained, and all copies must be deleted immediately. Whilst we take reasonable steps to try to identify any software viruses, any attachments to this e-mail may nevertheless contain viruses, which our anti-virus software has failed to identify. You should therefore carry out your own anti-virus checks before opening any documents. HomeLoan Partnership will not accept any liability for damage caused by computer viruses emanating from any attachment or other document supplied with this e-mail. HomeLoan Partnership reserves the right to monitor and archive all e-mail communications through its network. No representative or employee of HomeLoan Partnership has the authority to enter into any contract on behalf of HomeLoan Partnership by email. HomeLoan Partnership is a trading name of H L Partnership Limited, registered in England and Wales with Registration Number 5011722. Registered office: 26-34 Old Street, London, EC1V 9QQ. H L Partnership Limited is authorised and regulated by the Financial Conduct Authority.
>
> _______________________________________________
> List: Catalyst at lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/





More information about the Catalyst mailing list