[Redis] Perl Redis Client news

[Damien Krotkine]

Hi,

My goals were (and still are) to :

- make sure current code works properly again
- properly benchmark the different read methods
- check out XS parsing (I was actually thinking to benchmark
Protocol::Redis::XS)
- but above all, I'd like the code to be robust, easily maintainable,
and if possible relatively fast. And if not all PP, at least have a PP
fallback.

The benchmarking I mentioned is for me to have a way to measure how much
we'd loose if we go back to sysread. Because I very much would go back
to sysread :) The first result we have are not conclusive. The speed is
very dependant of the packet size and possibly other criterias, so meh.
Maybe it's not worth it, but I'll start gathering common commands
performed on real-life production Redis, and benchmark the most used
ones.


On Mon, Nov 18, 2013, at 12:18 PM, Steffen Mueller wrote:
> On 11/18/2013 07:20 PM, Pedro Melo wrote:
> > According to my profiling, parsing the Redis protocol is the bottlenet
> > *most* of the time. Adding a way to improve that part seems to me a good
> > way to improve the overall performance.
> 
> Of course, this entire exchange is valid only without considering 
> network latencies which at a hefty half millisecond trump most things. 
> But on a multi-use client machine, CPU capacity is a separate thing to 
> optimize for, so making things faster never hurts!
> 
> > Having a XS backend for that, with the PP fallback, seems to be the
> > standard approach for a lot of CPAN modules.
> 
> Hmm. And here I was, thinking that maybe there's low hanging fruit on 
> the encoding side. Here's a quick snippet[2]:
> 
> void
> redis_encode(...)
>    PREINIT:
>      SV *retval;
>      int i;
>      char *str;
>    PPCODE:
>      retval = sv_2mortal(newSVpvf("*%lu\r\n", (unsigned long)items));
> 
>      for (i = 0; i < items; ++i) {
>        /* FIXME UTF8? */
>        sv_catpvf(retval, "$%lu\r\n", (unsigned long)SvCUR(ST(i)));
>        sv_catsv(retval, ST(i));
>      }
> 
>      PUSHs(retval);
> 
> 
> $ time perl -Mblib -MXSFun -le '@d=(qw(SET foo), "x" x 10);for 
> (1..1e6){$x = ""; $x .= "*" . scalar(@d) . "\r\n"; $x .= 
> "\$".length($_)."\r\n$_\r\n" for @d; }'
> real    0m1.552s
> user    0m1.536s
> sys     0m0.008s
> 
> $ time perl -Mblib -MXSFun -le '@d=(qw(SET foo), "x" x 10);for 
> (1..1e6){$x=redis_encode(@d)}'
> real    0m1.112s
> user    0m1.100s
> sys     0m0.008s
> 
> With a bigger value:
> 
> $ time perl -Mblib -MXSFun -le '@d=(qw(SET foo), "x" x 10000);for 
> (1..1e6){$x = ""; $x .= "*" . scalar(@d) . "\r\n"; $x .= 
> "\$".length($_)."\r\n$_\r\n" for @d; }'
> real    0m2.870s
> user    0m2.856s
> sys     0m0.004s
> 
> $ time perl -Mblib -MXSFun -le '@d=(qw(SET foo), "x" x 10000);for 
> (1..1e6){$x=redis_encode(@d)}'
> real    0m1.629s
> user    0m1.616s
> sys     0m0.004s
> 
> Meh. Just a modest win. :)
> 
> > Ok, then against RedisDB XS component…
> 
> That seems a better bet indeed.
> 
> > RedisDB (I *think* its RedisDB…) has a XS interface. We could compare it
> > with our read()-based performance…
> 
> Just checked. Yes, it's RedisDB.
> 
> >     What's the main benefit you'd hope for? Maybe there's an easy win?
> >
> > Moare speed. I have *one* use case where redis protocol parser showed up
> > on my app profiling… it's a rather eavy use of Redis.
> 
> We use Redis a fair amount at work, but IIRC the miscellaneous 
> marshalling code of all cases is more expensive than the Redis.pm logic.
> 
> >     [1] Example:
> >              reply = redisCommand(c,"LPUSH mylist element-%s", buf);
> >
> >
> > YUCK!!!
> 
> I know, right?
> 
> --Steffen
> 
> [2] I tried a more slightly elaborate solution which would calculate an 
> upper bound on the output string length first and then allocated that 
> all in one go. Of course, that would have to peek at each stacked 
> parameter at least twice and that turned out SLOWER than pure-Perl. Rare 
> occasion that shaving of mallocs isn't a clear win.