From ton.voon at opsera.com Wed Sep 1 10:24:10 2010 From: ton.voon at opsera.com (Ton Voon) Date: Wed Sep 1 10:24:17 2010 Subject: [Catalyst-dev] Security issue with Catalyst::Action::REST Message-ID: <1882D900-BA40-4B4D-BAB7-88A800D87B27@opsera.com> Hi! I raised this on IRC a few days back. Basically, using Data::Serializer through Catalyst::Action::Rest takes the input data and evals it. However, there is no checking on the input data, which means arbitrary code could be executed. I tested with putting a "die()" as input and this gets run. mst suggested using Data::Pond instead, but this didn't support undef and also was erroring with some of my data structures. mst also suggested not patching Data::Serializer as that would break other applications. Instead, I've patched Catalyst::Action::Deserialize::Data::Serializer so that if the serializer is Data::Dumper, pass it through a Safe compartment instead. This limits the input to JSON-like input in perl style which I guess is the most you would use REST input as. I'm guessing that all Catalyst::Action::REST instances have this issue as the default map allows Data::Serializer to get invoked. The begin action could get called before authentication routines, which means that any cat app using this module is exposed. Can you please confirm if this patch is a suitable one. Are there any other holes left open? Ton -------------- next part -------------- A non-text attachment was scrubbed... Name: catalyst_action_rest_safe_eval_data_dumper.patch Type: application/octet-stream Size: 2596 bytes Desc: not available Url : http://lists.scsys.co.uk/pipermail/catalyst-dev/attachments/20100901/83ec4432/catalyst_action_rest_safe_eval_data_dumper.obj -------------- next part -------------- From peter at peknet.com Wed Sep 1 18:46:29 2010 From: peter at peknet.com (Peter Karman) Date: Wed Sep 1 18:46:36 2010 Subject: [Catalyst-dev] Security issue with Catalyst::Action::REST In-Reply-To: <1882D900-BA40-4B4D-BAB7-88A800D87B27@opsera.com> References: <1882D900-BA40-4B4D-BAB7-88A800D87B27@opsera.com> Message-ID: <4C7E9F85.2040509@peknet.com> Ton Voon wrote on 09/01/2010 05:24 AM: > Instead, I've patched Catalyst::Action::Deserialize::Data::Serializer so > that if the serializer is Data::Dumper, pass it through a Safe > compartment instead. This limits the input to JSON-like input in perl > style which I guess is the most you would use REST input as. What if the serializer is Data::Dump, et al.? I.e., is the special check for Data::Dumper echoing some other, similar test in the module or in Catalyst core? -- Peter Karman . http://peknet.com/ . peter@peknet.com From ton.voon at opsera.com Wed Sep 1 20:51:01 2010 From: ton.voon at opsera.com (Ton Voon) Date: Wed Sep 1 20:51:08 2010 Subject: [Catalyst-dev] Security issue with Catalyst::Action::REST In-Reply-To: <4C7E9F85.2040509@peknet.com> References: <1882D900-BA40-4B4D-BAB7-88A800D87B27@opsera.com> <4C7E9F85.2040509@peknet.com> Message-ID: <835F2BA2-FAC4-4949-8AD5-76503CB60E58@opsera.com> On 1 Sep 2010, at 19:46, Peter Karman wrote: > Ton Voon wrote on 09/01/2010 05:24 AM: > >> Instead, I've patched >> Catalyst::Action::Deserialize::Data::Serializer so >> that if the serializer is Data::Dumper, pass it through a Safe >> compartment instead. This limits the input to JSON-like input in perl >> style which I guess is the most you would use REST input as. > > What if the serializer is Data::Dump, et al.? I.e., is the special > check > for Data::Dumper echoing some other, similar test in the module or in > Catalyst core? > I'm not sure what you mean. Data::Serializer is a front end to other serializers: http://search.cpan.org/~neely/Data-Serializer-0.49/lib/Data/Serializer.pm Data::Dump is not one of the serializers available, though I guess that doesn't preclude it from being included in future. Data::Denter looks like some YAML type format, Data::Taxi is based on an XML like structure, FreezeThaw is some variable only structure, PHP::Serialization is PHP strings, Storable doesn't include code, XML::Dumper/XML::Simple are XML based and YAML is YAML So I think Data::Dumper is the only serialization that could execute other code based on blindly eval'ing input. Ton -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.scsys.co.uk/pipermail/catalyst-dev/attachments/20100901/20791d0c/attachment.htm From peter at peknet.com Wed Sep 1 21:17:29 2010 From: peter at peknet.com (Peter Karman) Date: Wed Sep 1 21:17:35 2010 Subject: [Catalyst-dev] Security issue with Catalyst::Action::REST In-Reply-To: <835F2BA2-FAC4-4949-8AD5-76503CB60E58@opsera.com> References: <1882D900-BA40-4B4D-BAB7-88A800D87B27@opsera.com> <4C7E9F85.2040509@peknet.com> <835F2BA2-FAC4-4949-8AD5-76503CB60E58@opsera.com> Message-ID: <4C7EC2E9.6040504@peknet.com> Ton Voon wrote on 09/01/2010 03:51 PM: > So I think Data::Dumper is the only serialization that could execute > other code based on blindly eval'ing input. ok, I see what you mean. Thanks for clarifying. -- Peter Karman . http://peknet.com/ . peter@peknet.com From bobtfish at bobtfish.net Wed Sep 1 21:54:53 2010 From: bobtfish at bobtfish.net (Tomas Doran) Date: Wed Sep 1 21:54:49 2010 Subject: [Catalyst-dev] Security issue with Catalyst::Action::REST In-Reply-To: <1882D900-BA40-4B4D-BAB7-88A800D87B27@opsera.com> References: <1882D900-BA40-4B4D-BAB7-88A800D87B27@opsera.com> Message-ID: <53580E35-5987-4289-942C-BC0976384610@bobtfish.net> On 1 Sep 2010, at 11:24, Ton Voon wrote: > Thanks, applied as ef81163, will be in a release shortly. Cheers t0m From bobtfish at bobtfish.net Wed Sep 1 22:18:23 2010 From: bobtfish at bobtfish.net (Tomas Doran) Date: Wed Sep 1 22:18:09 2010 Subject: [Catalyst-dev] Patch for Catalyst-Action-REST to allow json options In-Reply-To: <93694845-058C-41A4-8B46-F34681C03027@gmail.com> References: <93694845-058C-41A4-8B46-F34681C03027@gmail.com> Message-ID: <36EA1426-8387-4482-954E-CF9B6447C342@bobtfish.net> On 27 Aug 2010, at 16:58, Ton Voon wrote: > Applied as 838f49d Cheers t0m From bobtfish at bobtfish.net Wed Sep 1 22:20:39 2010 From: bobtfish at bobtfish.net (Tomas Doran) Date: Wed Sep 1 22:20:24 2010 Subject: [Catalyst-dev] uri_for() and URI fragments In-Reply-To: References: Message-ID: <56947BD7-1ADA-491E-B5C5-3B80A87B71E1@bobtfish.net> On 27 Aug 2010, at 14:45, Ronald J Kimball wrote: > 5.7007 used the URI module to create the URI. Starting in 5.7008 [*] > a big block of custom code is used instead. While this provides > better performance, it is much less reliable. > > > Is this a bug that should be fixed, or is there a different, correct > way of specifying a URI fragment with uri_for? No, it's almost certainly a bug. Any chance of a failing test case? Cheers t0m From rkimball at pangeamedia.com Fri Sep 3 13:07:36 2010 From: rkimball at pangeamedia.com (Ronald J Kimball) Date: Fri Sep 3 13:07:42 2010 Subject: [Catalyst-dev] uri_for() and URI fragments In-Reply-To: <56947BD7-1ADA-491E-B5C5-3B80A87B71E1@bobtfish.net> References: <56947BD7-1ADA-491E-B5C5-3B80A87B71E1@bobtfish.net> Message-ID: On Wed, Sep 1, 2010 at 6:20 PM, Tomas Doran wrote: > > On 27 Aug 2010, at 14:45, Ronald J Kimball wrote: >> >> Is this a bug that should be fixed, or is there a different, correct >> way of specifying a URI fragment with uri_for? > > No, it's almost certainly a bug. > > Any chance of a failing test case? This test should be added to t/aggregate/unit_core_uri_for.t: is( Catalyst::uri_for( $context, '/bar#anchor', { param1 => 'value1' } )->as_string, 'http://127.0.0.1/foo/bar?param1=value1#fragment', 'URI for path with fragment and query params' ); # Failed test 'URI for path with fragment and query params' # at t/aggregate/unit_core_uri_for.t line 61. # got: 'http://127.0.0.1/foo/bar#anchor?param1=value1' # expected: 'http://127.0.0.1/foo/bar?param1=value1#fragment' Ronald From uwe at uwevoelker.de Fri Sep 3 14:12:04 2010 From: uwe at uwevoelker.de (=?ISO-8859-1?Q?Uwe_V=F6lker?=) Date: Fri Sep 3 14:12:09 2010 Subject: [Catalyst-dev] uri_for() and URI fragments In-Reply-To: References: <56947BD7-1ADA-491E-B5C5-3B80A87B71E1@bobtfish.net> Message-ID: Did you mix up #anchor and #fragment? Uwe > This test should be added to t/aggregate/unit_core_uri_for.t: > > is( > ? ?Catalyst::uri_for( $context, '/bar#anchor', { param1 => 'value1' } > )->as_string, > ? ?'http://127.0.0.1/foo/bar?param1=value1#fragment', > ? ?'URI for path with fragment and query params' > ); > > > # ? Failed test 'URI for path with fragment and query params' > # ? at t/aggregate/unit_core_uri_for.t line 61. > # ? ? ? ? ?got: 'http://127.0.0.1/foo/bar#anchor?param1=value1' > # ? ? expected: 'http://127.0.0.1/foo/bar?param1=value1#fragment' From rkimball at pangeamedia.com Fri Sep 3 14:19:46 2010 From: rkimball at pangeamedia.com (Ronald J Kimball) Date: Fri Sep 3 14:19:49 2010 Subject: [Catalyst-dev] uri_for() and URI fragments In-Reply-To: References: <56947BD7-1ADA-491E-B5C5-3B80A87B71E1@bobtfish.net> Message-ID: 2010/9/3 Uwe V?lker : > Did you mix up #anchor and #fragment? I did, thank you. Here's a corrected test: is( Catalyst::uri_for( $context, '/bar#fragment', { param1 => 'value1' } )->as_string, 'http://127.0.0.1/foo/bar?param1=value1#fragment', 'URI for path with fragment and query params' ); # Failed test 'URI for path with fragment and query params' # at t/aggregate/unit_core_uri_for.t line 61. # got: 'http://127.0.0.1/foo/bar#fragment?param1=value1' # expected: 'http://127.0.0.1/foo/bar?param1=value1#fragment' Ronald From tamas at eisenberger.hu Mon Sep 13 22:01:43 2010 From: tamas at eisenberger.hu (=?UTF-8?B?RWlzZW5iZXJnZXIgVGFtw6Fz?=) Date: Mon Sep 13 22:01:50 2010 Subject: [Catalyst-dev] C::P::Unicode::Encoding vs. C::P::Compress:Gzip conflict Message-ID: <4C8E9F47.5000507@eisenberger.hu> Hy! I've found a bug that makes impossible to use the two /subj/ plugins to be used together. Here is a test that shows the problem: http://www.iwstudio.hu/~u-foka/compress-gzip-test.tar.gz Thanks! From tamas at eisenberger.hu Tue Sep 14 17:23:06 2010 From: tamas at eisenberger.hu (=?UTF-8?B?RWlzZW5iZXJnZXIgVGFtw6Fz?=) Date: Tue Sep 14 17:23:11 2010 Subject: [Catalyst-dev] C::P::Unicode::Encoding vs. C::P::Compress:Gzip conflict In-Reply-To: <4C8E9F47.5000507@eisenberger.hu> References: <4C8E9F47.5000507@eisenberger.hu> Message-ID: <4C8FAF7A.9010302@eisenberger.hu> Has anyone seen inside this? In the meantime, I digged into this more deeply, and I found a solution, that moving the code from finalize to finalize_headers in Compress::Gzip (like it's moved in Unicode::Encoding for some reason, and that's caused the problem). I don't know if there are any consequences, so please someone, confirm that is the proper solution? Thanks! Eisenberger Tam?s (u-foka) On 09/14/2010 12:01 AM, Eisenberger Tam?s wrote: > Hy! > > I've found a bug that makes impossible to use the two /subj/ plugins to > be used together. > Here is a test that shows the problem: > http://www.iwstudio.hu/~u-foka/compress-gzip-test.tar.gz > > Thanks! > > _______________________________________________ > Catalyst-dev mailing list > Catalyst-dev@lists.scsys.co.uk > http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst-dev From joseph.he.2008 at gmail.com Tue Sep 28 14:07:57 2010 From: joseph.he.2008 at gmail.com (Joseph He) Date: Tue Sep 28 14:08:02 2010 Subject: [Catalyst-dev] random .dll loading error when running fastCGI Message-ID: Hi, all, I know this might be more like fastCGI or Apache problem instead of Catalyst issue, but there are more experts in this list than anywhere else so I will not be shy to ask you guys. I appologize if this is offending anyone here. The story is here: [Tue Sep 28 08:35:46 2010] [error] [client 127.0.0.1] FastCGI: server "C:..../test001.fcgi" stderr: install_driver(ODBC) failed: Can't load 'C:/Perl/site/lib/auto/DBD/ODBC/ODBC.dll' for module DBD::ODBC: load_file:A dynamic link library (DLL) initialization routine failed at C:/Perl/lib/DynaLoader.pm 230 The environment is Windows XP Pro, Apache 2.2.10, SQL Server 2005, FastCGI and Perl 5.5.8. When this error appears, refreshing the page or whatever cannot solve this problem while other pages are still fine. When refreshing, I can see the server generates the same log again but seems the server can never load this particular script successfully. After restarting server, it might happen to other scripts once in a while, which makes debugging is impossible to me. Very appreciate if anyone can share some experience or point me to some direction. Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.scsys.co.uk/pipermail/catalyst-dev/attachments/20100928/6= 11a2e96/attachment.htm From bobtfish at bobtfish.net Wed Sep 29 01:12:01 2010 From: bobtfish at bobtfish.net (Tomas Doran) Date: Wed Sep 29 01:10:33 2010 Subject: [Catalyst-dev] random .dll loading error when running fastCGI In-Reply-To: References: Message-ID: <77CFF2F0-79EF-4563-9B78-B68C839D45CF@bobtfish.net> On 28 Sep 2010, at 15:07, Joseph He wrote: > [Tue Sep 28 08:35:46 2010] [error] [client 127.0.0.1] FastCGI: > server "C:..../test001.fcgi" stderr: install_driver(ODBC) failed: > Can't load 'C:/Perl/site/lib/auto/DBD/ODBC/ODBC.dll' for module > DBD::ODBC: load_file:A dynamic link library (DLL) initialization > routine failed at C:/Perl/lib/DynaLoader.pm 230 > > The environment is Windows XP Pro, Apache 2.2.10, SQL Server 2005, > FastCGI and Perl 5.5.8. You mean 5.8.8, right? > > When this error appears, refreshing the page or whatever cannot > solve this problem while other pages are still fine. When > refreshing, I can see the server generates the same log again but > seems the server can never load this particular script successfully. > After restarting server, it might happen to other scripts once in a > while, which makes debugging is impossible to me. > > Very appreciate if anyone can share some experience or point me to > some direction. Try upgrading stuff like the DBD driver, and/or perl? (Sorry, not very helpful) Or run your .fcgis as external servers rather than having apache dynamically start them? Cheers t0m From bobtfish at bobtfish.net Wed Sep 29 01:36:30 2010 From: bobtfish at bobtfish.net (Tomas Doran) Date: Wed Sep 29 01:35:01 2010 Subject: [Catalyst-dev] Patch for current_view & default_view not existing In-Reply-To: References: Message-ID: <5FB6FEC7-972C-4FFA-B35E-1591565FA3E1@bobtfish.net> On 27 Aug 2010, at 18:22, Brad Lhotsky wrote: > So without the patch you get this: > [ Thanks for the patch! Applied with some changes as r13626. Cheers t0m From joseph.he.2008 at gmail.com Wed Sep 29 21:15:53 2010 From: joseph.he.2008 at gmail.com (Joseph He) Date: Wed Sep 29 21:15:56 2010 Subject: [Catalyst-dev] Re: Catalyst-dev Digest, Vol 63, Issue 6 In-Reply-To: References: Message-ID: Thank you, T0m. Actually it is 5.8.7. I have done more testing while having windows Tast Manager open. I find out that these issues only happen when I already have about 35-38 Perl processes running. Mainly two types of errors: 1. [Wed Sep 29 15:06:11 2010] [error] [client 127.0.0.1] (OS 109)The pipe has been ended. : FastCGI: comm with server "C:.....fcgi" aborted: GetOverlappedResult() failed, referer ...... ## I searched and know that "(OS 109)The pipe has been ended" is SQL server error message. 2. [Wed Sep 29 15:10:52 2010] [error] [client 127.0.0.1] FastCGI: server "C:......fcgi" stderr: install_driver(ODBC) failed: Can't load 'C:/Perl/site/lib/auto/DBD/ODBC/ODBC.dll' for module DBD::ODBC: load_file:A dynamic link library (DLL) initialization routine failed at C:/Perl/lib/DynaLoader.pm line 230., referer: .... So basically, when more than 35 Perl processes are launched, failure can happen to any of my new scripts. I tried to start some scripts as static server, say 20 scripts, then I will meet the same problem when the processes number reach to 35ish or more. If I try to initiate more than 40 as static server, in the initiation log, it will show some warn message like "[Wed Sep 29 16:02:44 2010] [warn] FastCGI: server "C:.....fcgi" (pid 6860) terminated with exit with status '-1073741502' [Wed Sep 29 16:02:44 2010] [warn] FastCGI: server "C:.....fcgi" terminated with exit with status '-1073741502'. It seems to me the total number of processes that can be run on windows depends on CPU, Memory, etc. FastCGI can eat up system resources quickly. I don't know whether this is a normal issue for fastCGI/Apache/Windows or maybe this can be optimized by some configuration. FYI: my PC is with 4G ram and Intel Core 2 T5600 @1.83G. Really really appreciate some help! Joe -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://lists.scsys.co.uk/pipermail/catalyst-dev/attachments/20100928/6=3D > 11a2e96/attachment.html > > ------------------------------ > > Message: 2 > Date: Wed, 29 Sep 2010 02:12:01 +0100 > From: Tomas Doran > Subject: Re: [Catalyst-dev] random .dll loading error when running > fastCGI > To: Development of the elegant MVC web framework > > Message-ID: <77CFF2F0-79EF-4563-9B78-B68C839D45CF@bobtfish.net> > Content-Type: text/plain; charset=3DUS-ASCII; format=3Dflowed; delsp=3Dyes > > > On 28 Sep 2010, at 15:07, Joseph He wrote: > > [Tue Sep 28 08:35:46 2010] [error] [client 127.0.0.1] FastCGI: > > server "C:..../test001.fcgi" stderr: install_driver(ODBC) failed: > > Can't load 'C:/Perl/site/lib/auto/DBD/ODBC/ODBC.dll' for module > > DBD::ODBC: load_file:A dynamic link library (DLL) initialization > > routine failed at C:/Perl/lib/DynaLoader.pm 230 > > > > The environment is Windows XP Pro, Apache 2.2.10, SQL Server 2005, > > FastCGI and Perl 5.5.8. > > You mean 5.8.8, right? > > > > > When this error appears, refreshing the page or whatever cannot > > solve this problem while other pages are still fine. When > > refreshing, I can see the server generates the same log again but > > seems the server can never load this particular script successfully. > > After restarting server, it might happen to other scripts once in a > > while, which makes debugging is impossible to me. > > > > Very appreciate if anyone can share some experience or point me to > > some direction. > > Try upgrading stuff like the DBD driver, and/or perl? (Sorry, not very > helpful) > > Or run your .fcgis as external servers rather than having apache > dynamically start them? > > Cheers > t0m > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.scsys.co.uk/pipermail/catalyst-dev/attachments/20100929/9= 24f99b5/attachment.htm From peter at dragonstaff.co.uk Wed Sep 29 21:32:28 2010 From: peter at dragonstaff.co.uk (Peter Edwards) Date: Wed Sep 29 21:32:31 2010 Subject: [Catalyst-dev] Re: Catalyst-dev Digest, Vol 63, Issue 6 In-Reply-To: References: Message-ID: On 29 September 2010 22:15, Joseph He wrote: > Thank you, T0m. Actually it is 5.8.7. > > I have done more testing while having windows Tast Manager open. I find o= ut > that these issues only happen when I already have about 35-38 Perl proces= ses > running. Mainly two types of errors: > > 1. [Wed Sep 29 15:06:11 2010] [error] [client 127.0.0.1] (OS 109)The pipe > has been ended. : FastCGI: comm with server "C:.....fcgi" aborted: > GetOverlappedResult() failed, referer ...... > > ## I searched and know that "(OS 109)The pipe has been ended" is SQL serv= er > error message. > > 2. [Wed Sep 29 15:10:52 2010] [error] [client 127.0.0.1] FastCGI: server > "C:......fcgi" stderr: install_driver(ODBC) failed: Can't load > 'C:/Perl/site/lib/auto/DBD/ODBC/ODBC.dll' for module DBD::ODBC: load_file= :A > dynamic link library (DLL) initialization routine failed at > C:/Perl/lib/DynaLoader.pm line 230., referer: .... > # Sounds like an out of memory condition triggering a problem. What does the Windows monitor show with respect to memory usage? I have seen random errors from FastCGI on Linux, too, which aren't all that easy to track down. How about trying it through an Apache server on Windows and see if that works or gives more information. Or running your app on Linux if that's an option. Regards, Peter http://perl.dragonstaff.co.uk -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.scsys.co.uk/pipermail/catalyst-dev/attachments/20100929/c= a8de828/attachment.htm