Problem with stemming and AAF

I’m sure I’m missing something completely obvious here, so I hope
someone can point me in the right direction!

I’ve implemented a basic search with AAF, which works as expected; I’m
running a ferret drb server, and using will_paginate to page results.
The code in my search_controller.rb:

search_text = params[:query] || " "
@products = Product.find_with_ferret(search_text, :page =>
params[:page], :per_page => #$ItemsPerPage, :limit => $ItemsPerPage,
:offset => $offset)
@results_pages = Product.paginate_search(search_text, :page =>
params[:page], :per_page => $ItemsPerPage)

The next step was to implement stemming, which seemed straightforward
enough. I created the stemmed_analyzer.rb file in the lib directory,
as follows:

require ‘rubygems’
require ‘ferret’

class StemmedAnalyzer < Ferret::Analysis::Analyzer
include Ferret::Analysis
def initialize(stop_words = ENGLISH_STOP_WORDS)
@stop_words = stop_words
end
def token_stream(field, str)
StemFilter.new(StopFilter.new(LowerCaseFilter.new(StandardTokenizer.new(str)),
@stop_words))
end
end

And added the call to the analyzer in my model file:

acts_as_ferret( :fields => { :name => { :boost => 1,
:store => :yes },
:product_number => { :boost => 2 },
:description => { :boost => 0,
:store => :yes },
:care => { :boost => -2 },
:manufacturer_name => { :boost => 1,
:store => :yes },
:collection_name => { :boost => 1,
:store => :yes },
:category_name => { :boost => 0 }
},
:remote => true,
:analyzer => StemmedAnalyzer.new )

Straight forward, no errors. But also no results. Searching for chairs
returns only results for that word, not chair or chairs. I know the
actual analyzer works, as when I explicity call it as follows, it
returns the correct root words to the log files:

search_terms = StemmedAnalyzer.new.token_stream(nil, params[:query])
while token = search_terms.next
puts token
end
Like so: Search for “chairs tables” returns

token[“chair”:0:6:1]
token[“tabl”:7:13:1]

but the front end throws up on me with a:
TypeError (wrong argument type DRb::DRbObject (expected Data))

I’m fully confused. I’m sure it’s something obvious that I’m just not
seeing, and after beating my head against this for two days, I’m
hoping someone can point it out to me! Or at least get me moving in
the right direction. Thanks for any help!

claudia

Hi!

the analyzer option belongs to the set of options which aaf directly
passes on to Ferret, and therefore the call has to read:
acts_as_ferret(:fields => { },
:remote => true,
:ferret => {
:analyzer => StemmedAnalyzer
})

Cheers,
Jens

On Fri, Nov 09, 2007 at 02:41:17PM -0500, phedre wrote:

:offset => $offset)
require ‘ferret’
end
:manufacturer_name => { :boost => 1,
:store => :yes },
:collection_name => { :boost => 1,
:store => :yes },
:category_name => { :boost => 0 }
},
:remote => true,
:analyzer => StemmedAnalyzer.new )


Jens Krämer
http://www.jkraemer.net/ - Blog
http://www.omdb.org/ - The new free film database

Such a simple solution. That’s what I get for spending days staring at
the silly thing. Thanks for the help!

claudia