Bug or "feature"?

Ruby Apps Ferret
1

hi, i’m trying ferret, i’ve a model which has some records and two of
them have a title with the word ‘again’ (one or more time), so i’ve
tried to do a search for ‘again’, but i didn’t found anything…i’ve
edited the title with ‘test again’, searched for ‘test’, and i’ve found
them…another time with ‘again’, but nothing, so i’ve tried with ‘my
test’…searched for ‘my’, and…nothing…is it a problem or like a
“feature”?

another little problem i’ve found is that i’ve written this for the
search:

acts_as_ferret :fields => {:title => {},
:category_id => {},
:bought_at_int => {:index =>
:untokenized_omit_norms, :term_vector => :no},
:gift => {:index =>
:untokenized_omit_norms, :term_vector => :no}}

def self.full_text_search(query, category_id)
return nil if query.nil? or (query == ‘’)
query += " +category_id:{category_id}
+bought_at_int:>#{Time.now.to_i} +gift:false"
sort = Ferret::Search::SortField.new(:bought_at_int, :type =>
:byte, :reverse => false)
self.find_by_contents(query, {:limit => :all, :sort => sort}, {
:include => [:user] })
end

this works…initially i’ve tried with
query = “+title:#{query} +bought_at_int:>#{Time.now.to_i}
+gift:false”

but this doesn’t wok, can someone tell me what? i think is the
same…(the second is better because the query is only for the title
and not for other fields)

2

mix schrieb:

hi, i’m trying ferret, i’ve a model which has some records and two of
them have a title with the word ‘again’ (one or more time), so i’ve
tried to do a search for ‘again’, but i didn’t found anything…i’ve
edited the title with ‘test again’, searched for ‘test’, and i’ve found
them…another time with ‘again’, but nothing, so i’ve tried with ‘my
test’…searched for ‘my’, and…nothing…is it a problem or like a
“feature”?

try this:

bash$ script/console

Ferret::Analysis::FULL_ENGLISH_STOP_WORDS

and see: http://ferret.davebalmain.com/api/classes/Ferret/Analysis.html

Ben :slight_smile:

3

Benjamin K. wrote:

mix schrieb:

hi, i’m trying ferret, i’ve a model which has some records and two of
them have a title with the word ‘again’ (one or more time), so i’ve
tried to do a search for ‘again’, but i didn’t found anything…i’ve
edited the title with ‘test again’, searched for ‘test’, and i’ve found
them…another time with ‘again’, but nothing, so i’ve tried with ‘my
test’…searched for ‘my’, and…nothing…is it a problem or like a
“feature”?

try this:

bash$ script/console

Ferret::Analysis::FULL_ENGLISH_STOP_WORDS

and see: http://ferret.davebalmain.com/api/classes/Ferret/Analysis.html

Ben :slight_smile:

ok, now is more clear :slight_smile:
but i’ve just another problem… if i search something like +, “”, <abc
and so on it found me all records O_o do you know why?
is it correct the query ? about the title:#{query} any words? :frowning:
thanks :slight_smile:

4

mix wrote:

mix wrote:

but i’ve just another problem… if i search something like +, “”, <abc
and so on it found me all records O_o do you know why?
is it correct the query ? about the title:#{query} any words? :frowning:
thanks :slight_smile:

Maybe just me, but I can’t understand your question. Can you try
rephrasing or perhaps including some code and results?

5

mix wrote:

but i’ve just another problem… if i search something like +, “”, <abc
and so on it found me all records O_o do you know why?
is it correct the query ? about the title:#{query} any words? :frowning:
thanks :slight_smile:

anyone? :frowning:

6

Sam G. wrote:

mix wrote:

mix wrote:

but i’ve just another problem… if i search something like +, “”, <abc
and so on it found me all records O_o do you know why?
is it correct the query ? about the title:#{query} any words? :frowning:
thanks :slight_smile:

Maybe just me, but I can’t understand your question. Can you try
rephrasing or perhaps including some code and results?

sure…so, i’ve a model which has this code:

acts_as_ferret :fields => {:title => {},
:all_categories => {},
:user_id => {},
:bought_at_int => {:index =>
:untokenized_omit_norms, :term_vector => :no},
:gift => {:index =>
:untokenized_omit_norms, :term_vector => :no}}

def self.full_text_search(query, category_id, extra)
return nil if query.nil? or (query == ‘’)
query += " +all_categories:#{category_id}
+bought_at_int:<#{15.days.from_now.to_i} +gift:false #{extra}"
sort = Ferret::Search::SortField.new(:expires_at_int, :type =>
:byte, :reverse => false)
self.find_by_contents(query, {:limit => :all, :sort => sort}, {
:include => [:user, :categories] })
end

the search controller is:
def show
@category = Category.find_by_id(params[:category_id] || 1)
@obj = Model.full_text_search(params[:query], @category.id, ‘’)
end

(i use the extra variable in another part to search for a user, like: ’
+user:1’)
anyway, if i search for a title (which i have in the db) ‘test’, ferret
found me without problem, or another title like ‘fishing guide’, and the
result is pretty good (ok ok, is perfect :)), but when i search for
something like: ‘+’, ‘""’, ‘<abc’ ferret found me all the records in the
db… why? :frowning:
and so on it found me all records

7

On Thu, Mar 08, 2007 at 12:15:55AM +0100, mix wrote:
[…]

(i use the extra variable in another part to search for a user, like: ’
+user:1’)
anyway, if i search for a title (which i have in the db) ‘test’, ferret
found me without problem, or another title like ‘fishing guide’, and the
result is pretty good (ok ok, is perfect :)), but when i search for
something like: ‘+’, ‘“”’, ‘<abc’ ferret found me all the records in the
db… why? :frowning:
and so on it found me all records

could you please have a look into your application’s log file, and look
for the query aaf actually runs against the ferret index? the relevant
line should start with 'query: ’

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

8

On Thu, Mar 08, 2007 at 02:22:45PM +0100, mix wrote:

this works, now with ‘?’ it doesn’t found anything, but… if i search a
stop word (my, all, your, etc) it found all books :frowning:

the log:

Query: +title:all +all_categories:1 +bought_at_int:<1174655577
+gift:false

I guess this is perfectly correct behaviour. The stop word can’t be
searched for, since it gets stripped from your query (and it has been
stripped from all your indexed documents, too, so searching for it would
be pointless). What remains is the query
‘+all_categories:1 +bought_at_int:<1174655577 +gift:false’ which will
then be run against the index.

You can tell ferret to not use any stop words by specifying an empty
list for the stop words:

acts_as_ferret { :fields => { … } }, { :analyzer =>
StandardAnalyzer.new([]) }

Jens

–
Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

9

Jens K. wrote:

could you please have a look into your application’s log file, and look
for the query aaf actually runs against the ferret index? the relevant
line should start with 'query: ’

Jens

“Query: +all_categories:1 +bought_at_int:<1174655577 +gift:false”

with this and a search ‘+’ it found all books (it changes ‘+’ with ’ ')
same for ‘?’

now i’ve changed the query with:

query = “+title:#{query} +all_categories:#{category_id}
+bought_at_int:<#{15.days.from_now.to_i} +gift:false #{extra}”
this works, now with ‘?’ it doesn’t found anything, but… if i search a
stop word (my, all, your, etc) it found all books :frowning:

the log:

Query: +title:all +all_categories:1 +bought_at_int:<1174655577
+gift:false

10

Jens K. wrote:

I guess this is perfectly correct behaviour. The stop word can’t be
searched for, since it gets stripped from your query (and it has been
stripped from all your indexed documents, too, so searching for it would
be pointless). What remains is the query
‘+all_categories:1 +bought_at_int:<1174655577 +gift:false’ which will
then be run against the index.

You can tell ferret to not use any stop words by specifying an empty
list for the stop words:

acts_as_ferret { :fields => { … } }, { :analyzer =>
StandardAnalyzer.new([]) }

ok, thanks, i’ll try :slight_smile: