[Redis] Perl Redis Client news

[Steffen Mueller]

On 11/18/2013 07:20 PM, Pedro Melo wrote:
> According to my profiling, parsing the Redis protocol is the bottlenet
> *most* of the time. Adding a way to improve that part seems to me a good
> way to improve the overall performance.

Of course, this entire exchange is valid only without considering 
network latencies which at a hefty half millisecond trump most things. 
But on a multi-use client machine, CPU capacity is a separate thing to 
optimize for, so making things faster never hurts!

> Having a XS backend for that, with the PP fallback, seems to be the
> standard approach for a lot of CPAN modules.

Hmm. And here I was, thinking that maybe there's low hanging fruit on 
the encoding side. Here's a quick snippet[2]:

void
redis_encode(...)
   PREINIT:
     SV *retval;
     int i;
     char *str;
   PPCODE:
     retval = sv_2mortal(newSVpvf("*%lu\r\n", (unsigned long)items));

     for (i = 0; i < items; ++i) {
       /* FIXME UTF8? */
       sv_catpvf(retval, "$%lu\r\n", (unsigned long)SvCUR(ST(i)));
       sv_catsv(retval, ST(i));
     }

     PUSHs(retval);


$ time perl -Mblib -MXSFun -le '@d=(qw(SET foo), "x" x 10);for 
(1..1e6){$x = ""; $x .= "*" . scalar(@d) . "\r\n"; $x .= 
"\$".length($_)."\r\n$_\r\n" for @d; }'
real    0m1.552s
user    0m1.536s
sys     0m0.008s

$ time perl -Mblib -MXSFun -le '@d=(qw(SET foo), "x" x 10);for 
(1..1e6){$x=redis_encode(@d)}'
real    0m1.112s
user    0m1.100s
sys     0m0.008s

With a bigger value:

$ time perl -Mblib -MXSFun -le '@d=(qw(SET foo), "x" x 10000);for 
(1..1e6){$x = ""; $x .= "*" . scalar(@d) . "\r\n"; $x .= 
"\$".length($_)."\r\n$_\r\n" for @d; }'
real    0m2.870s
user    0m2.856s
sys     0m0.004s

$ time perl -Mblib -MXSFun -le '@d=(qw(SET foo), "x" x 10000);for 
(1..1e6){$x=redis_encode(@d)}'
real    0m1.629s
user    0m1.616s
sys     0m0.004s

Meh. Just a modest win. :)

> Ok, then against RedisDB XS component…

That seems a better bet indeed.

> RedisDB (I *think* its RedisDB…) has a XS interface. We could compare it
> with our read()-based performance…

Just checked. Yes, it's RedisDB.

>     What's the main benefit you'd hope for? Maybe there's an easy win?
>
> Moare speed. I have *one* use case where redis protocol parser showed up
> on my app profiling… it's a rather eavy use of Redis.

We use Redis a fair amount at work, but IIRC the miscellaneous 
marshalling code of all cases is more expensive than the Redis.pm logic.

>     [1] Example:
>              reply = redisCommand(c,"LPUSH mylist element-%s", buf);
>
>
> YUCK!!!

I know, right?

--Steffen

[2] I tried a more slightly elaborate solution which would calculate an 
upper bound on the output string length first and then allocated that 
all in one go. Of course, that would have to peek at each stacked 
parameter at least twice and that turned out SLOWER than pure-Perl. Rare 
occasion that shaving of mallocs isn't a clear win.