[Dbix-class] Regarding Oracle's 1000 element limit in IN clause

[Rob Kinyon]

On Mon, Oct 24, 2011 at 07:46, Matija Grabnar <matija at serverflow.com> wrote:
> On 10/24/2011 11:55 AM, Jorge Gonzalez wrote:
>>
>> B. Create a temp table in SMALL database with the contents of the HUGE
>> database I want to filter. Thi would mean transfering a copy of the table
>> with _millions_ of rows to local, just to discard it afterwards. Seems not
>> very reasonable.
>
> The reasonable solutions are the ones you aren't allowed to do. What is left
> is the unreasonable ones.
>>
>>  or move the HUGE one where the SMALL one is (not reasonable).
>
> Or cache PART of the HUGE one on your side. Which may or may not be
> reasonable, depending
> on exact details of your problem.

I think this suggestion makes a key point. If there is a relatively
small subset (relative to the total size of the warehouse, maybe even
up to 100GB), then maybe the right solution is to create a separate
service that does nothing but mirror this subset to your database.
Then, you can access those tables as if they were in your database,
without your application caring how they are populated.

The important design concept is to separate the service that maintains
the tables from the service(s) that *consumes* the tables.

Rob