[Catalyst] $c->write() buffering?

[John Napiorkowski]

Daniel,

Have you taken a look at what Plack's FCGI handlers does? =A0I'm not so fam=
iliar with the intricacies of FCGI but here's the source:

https://metacpan.org/source/MIYAGAWA/Plack-1.0029/lib/Plack/Handler/FCGI.pm


psgi.input is set to STDIN it appears. =A0Not sure if that is what you are =
looking for.

the PSGI based Catalyst introduced a way to allow you to have more fined co=
ntrol over the write process. =A0I wonder if you could experiment with that=
 and let us know how it goes? =A0

John



On Thursday, November 7, 2013 3:07 PM, Daniel J. Luke <dluke at geeklair.net> =
wrote:
 =

I've done some more experimentation and this doesn't end up working very we=
ll for us. For the list archive:

You can accomplish what I was going for using mod_fastcgi and a standalone =
FCGI-based script. mod_fastcgi needs to be configured with its -flush optio=
n /and/ you need to be able to call FCGI::Flush($request) when you want out=
put sent out to the client.

Catalyst::Engine:FastCGI doesn't expose the FCGI request (lexical variable =
in the run() method), though, so there's not an easy way to do this (and it=
's icky anyway). Catalyst::Engine::FastCGI should possibly FCGI::Flush in i=
ts write() method [note that Plack::Handler::FCGI doesn't expose the FCGI r=
equest either and also doesn't call FCGI::Flush anywhere].

So, I guess no one is doing $c->res->write() with mod_fastcgi and http://wi=
ki.catalystframework.org/wiki/longcomputations should probably be edited to=
 explain that this doesn't work (unless you pad each write() you want to go=
 to the client to 8k).

We'll likely implement some AJAX ui (possibly tying up 2 catalyst processes=
 instead of just one and/or fork/exec a worker) for now, until we can get s=
ome time to move things over to a job queue that isn't as horrible as the o=
ne we currently have :-)


On Nov 1, 2013, at 11:42 AM, John Napiorkowski <jjn1056 at yahoo.com> wrote:
> I was reviewing code last night for release of Catalyst Hamburg dev5 and =
saw that $c->res->write is just calling the writer under the hood.=A0 So as=
 long as your headers are ready to finalize, that should be fine as well.=
=A0 Just remember this approach is going to block, so be careful with anyth=
ing=A0 that is taking a lot of time.=A0 --jnap
> =

> =

> On Thursday, October 31, 2013 4:34 PM, John Napiorkowski <jjn1056 at yahoo.c=
om> wrote:
> Recent releases of Catalyst makes it possible to stream write, although y=
ou need to be careful when using a blocking web server (long stream will of=
 course block the available listener).
> =

> Older versions of Catalyst had similar ability with the write method, I n=
ever used it, and would be happen to get patches or test cases to make it w=
ork as we'd like.
> =

> Here's a link to an example, this one assume AnyEvent as a loop, but you =
could use a similar technique with any web server, as long as you don't min=
d blocking:
> =

> https://metacpan.org/pod/release/JJNAPIORK/Catalyst-Runtime-5.90049_005/l=
ib/Catalyst/Response.pm#res-write_fh
> =

> If you follow that, you will see stuff 'stream' as you wish unless there =
is some buffering going on at some other level of the stack.
> =

> Johnn
> =

> =

> On Thursday, October 31, 2013 4:22 PM, Daniel J. Luke <dluke at geeklair.net=
> wrote:
> I replicated this today outside of Catalyst (just a small FCGI/FCGI::Proc=
Manager test script). If anyone else has seen this/fixed it, I'd appreciate=
 a pointer. I'll report back to the list if/when I get it resolved so that =
there's an answer in the list archives for future people to google otherwis=
e.
> =

> On Oct 31, 2013, at 10:12 AM, Daniel J. Luke <dluke at geeklair.net> wrote:
> > We're actually running Catalyst::Runtime 5.80031 (currently), so I beli=
eve it's using Catalyst::Engine::FastCGI which just does *STDOUT->syswrite()
> > =

> > I guess I try to do some testing with newer Catalyst (and maybe alterna=
te deployment methods), to see if that changes anything. Looking through th=
e Changelog, I didn't see anything specifically related to this, though (al=
though I imagine the Plack stuff makes it somewhat different in the more re=
cent releases). =

> > =

> > On Oct 30, 2013, at 11:03 PM, Hailin Hu <i at h2l.name> wrote:
> >> It is an engine relevant stuff.
> >> Find which engine you are using ( for example, Plack::Handler::FCGI )
> >> and look around codes around write(r), you may find something.
> >> Good luck :)
> >> =

> >> On Wed, Oct 30, 2013 at 9:51 AM, Daniel J. Luke <dluke at geeklair.net> w=
rote:
> >>> I've got some legacy CGI code that does a bunch of processing and use=
s the old hack of $| =3D 1; print "foo\n"; do_work(); print "foo done\n"; e=
tc. (solution #1 from http://wiki.catalystframework.org/wiki/longcomputatio=
ns)
> >>> =

> >>> While I'll eventually convert it to a job queue, I'd like to create a=
n output-identical implementation first with Catalyst, however it seems lik=
e I'm getting output buffering when I don't want it.
> >>> =

> >>> As a very simple test, I've set up apache 2.2, mod_fastcgi (FastCgiEx=
ternalServer with -flush and without -flush) and a method like this:
> >>> =

> >>> sub test: Local {
> >>>=A0 my ($self, $c) =3D @_;
> >>> =

> >>>=A0 $c->res->body('');
> >>>=A0 $c->response->content_type( 'text/plain; charset=3Dutf-8' );
> >>>=A0 $c->finalize_headers;
> >>> =

> >>>=A0 my $i;
> >>>=A0 for($i=3D1;$i<8;$i++) {
> >>>=A0 =A0 =A0 $c->write("$i: foo bar baz\n");
> >>>=A0 =A0 =A0 sleep(1);
> >>>=A0 }
> >>> }
> >>> =

> >>> I see all the data at once in my browser instead of a line every seco=
nd, and with tcpdump, can see that all of the data is coming back in one pa=
cket and not in 8+ smaller packets like I expect. If I make the string that=
 gets passed to write longer, I get several packets, but all at once (and n=
ot with each iteration through the for loop).
> >>> =

> >>> Am I missing something obvious? Is there some way to get the behavior=
 I'm expecting?

--
Daniel J. Luke=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =

+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D+=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =

| *---------------- dluke at geeklair.net ----------------* |=A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =

| *-------------- http://www.geeklair.net -------------* |=A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =

+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D+=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =

|=A0  Opinions expressed are mine and do not necessarily=A0  |=A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =

|=A0 =A0 =A0 =A0 =A0 reflect the opinions of my employer.=A0 =A0 =A0 =A0 =
=A0 |=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =

+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D+
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20131108/2c7de=
e8a/attachment.htm