Ferret acts_as_ferret and performance

Ruby Apps Ferret
1

hello,

I am actually indexing thousands of 1ko text documents using ferret and
acts_as_ferret, and i face performance problems.
I takes me hours to index 20 000 1ko text documents.

Methology used :

I create and object, fulfill it with the text, and save it. So it is
automaticly indexed.

Is there a way to make it faster ? ( remove the auto optimize option
somewhere ?)

Thank you if you have any ideas…
Regards

2

On Fri, Aug 31, 2007 at 06:24:35PM +0200, Hakita Hakita wrote:

Is there a way to make it faster ? ( remove the auto optimize option
somewhere ?)

You should disable Ferret indexing before you start creating your
records, then create them, enable Ferret again and index them as a
whole:

Model.disable_ferret

create records here, collect ids in id_array

Model.enable_ferret
Model.bulk_index(id_array)

bulk_index temporarily turns off auto_flush, and optimizes the index
after finishing.

I just committed these functions to trunk, so let us know how it works
:wink:

Jens

–
Jens Krämer
http://www.jkraemer.net/ - Blog
http://www.omdb.org/ - The new free film database

3

Good idea

Once again,
Thank you Jens

4

On Sun, Sep 02, 2007 at 02:10:32AM +0200, Hakita Hakita wrote:

This is going really faster now !
As i am indexing thousands of rss feeds, it now takes one hour to
retrieve and index it all ( several hours before )

glad to hear that :slight_smile:

Jens

–
Jens Krämer
http://www.jkraemer.net/ - Blog
http://www.omdb.org/ - The new free film database

5

This is going really faster now !
As i am indexing thousands of rss feeds, it now takes one hour to
retrieve and index it all ( several hours before )