Another issue with highlighting

Hi,

I’m encountering another highlighting issue.
(about the first one “range search and highlighting”, i received no
response. I don’t even know if somebody tried to reproduce and/or if
it’s normal behavior)

about the new issue, an example will be easier for you to reproduce:
I’m filling an index with random data, i try to match for “1” and
then highlight the matched tokens. If it’s matched and not highlighted
i put it in z

It works as expected when there’re 100 entries (replace 500.times by
100.times), in that case z contains empty arrays.
When having 500 entries it doesn’t highlight every matches !

This example has been tested with 0.11.4
(r770 has been tested with the application i first discovered this
issue with)

I would appreciate if you could test and tell me if I’m the only one
having this problem.
TIA

require 'ferret' include Ferret

filling

index=Index::Index.new(:path => ‘/tmp/test’)
chars1 = chars2 = chars3 = chars4 = (“a”…“z”).to_a + (“0”…“9”).to_a
chars2.concat(["-", “_”, " “])
chars3 << " "
chars4 << “-”
chars5 = chars6 = (“0”…“9”).to_a
chars6 << “.”
500.times do
z={}
t=”"
1.upto(15+rand(10)) { |i| t << chars4[rand(chars4.size-1)] }
z[:un] = t
t=""
1.upto(40+rand(40)) { |i| t << chars2[rand(chars2.size-1)] }
z[:deux] = t
t=""
1.upto(30+rand(10)) { |i| t << chars4[rand(chars4.size-1)] }
z[:trois] = t
t=""
1.upto(30+rand(10)) { |i| t << chars1[rand(chars1.size-1)] }
z[:quatre] = t
t=""
1.upto(30+rand(10)) { |i| t << chars2[rand(chars2.size-1)] }
z[:cinq] = t
t=""
1.upto(12) { |i| t << chars5[rand(chars5.size-1)] }
z[:six] = t
t=""
1.upto(12) { |i| t << chars6[rand(chars6.size-1)] }
z[:sept] = t
t=""
1.upto(12) { |i| t << chars6[rand(chars6.size-1)] }
z[:huit] = t
t=""
1.upto(24+rand(24)) { |i| t << chars3[rand(chars3.size-1)] }
z[:neuf] = t
t=""
1.upto(100+rand(100)) { |i| t << chars2[rand(chars2.size-1)] }
z[:dix] = t
index << z
end

#testing
q=“1
z={}
index.search_each(q,:limit => :all) do |id,score|
for b in [:un, :deux, :trois, :quatre, :cinq, :six, :sept, :huit, :neuf,
:dix]
z[b]=[] if not z[b]
z[b] << id.to_s + " : " + index.highlight(q,id,:field => b, :pre_tag =>
”, :post_tag => “”, :num_excerpts => :all, :excerpt_length =>
:all).join(" | “) if index[id][b].match(/1/) and
index.highlight(q,id,:field => b, :pre_tag => “”, :post_tag =>
”, :num_excerpts => :all, :excerpt_length => :all) and not
index.highlight(q,id,:field => b, :pre_tag => “”, :post_tag =>
”, :num_excerpts => :all, :excerpt_length => :all).join(” |
“).match(//)
end
end
z
index.search(”*",:limit => :all).total_hits

[email protected] writes:

i put it in z
having this problem.
chars2.concat(["-", “_”, " “])
1.upto(40+rand(40)) { |i| t << chars2[rand(chars2.size-1)] }
t=”"
z[:neuf] = t
for b in [:un, :deux, :trois, :quatre, :cinq, :six, :sept, :huit, :neuf, :dix]
z[b]=[] if not z[b]
z[b] << id.to_s + " : " + index.highlight(q,id,:field => b, :pre_tag => “”, :post_tag => “”, :num_excerpts => :all, :excerpt_length => :all).join(" | “) if index[id][b].match(/1/) and index.highlight(q,id,:field => b, :pre_tag => “”, :post_tag => “”, :num_excerpts => :all, :excerpt_length => :all) and not index.highlight(q,id,:field => b, :pre_tag => “”, :post_tag => “”, :num_excerpts => :all, :excerpt_length => :all).join(” | “).match(//)
end
end
z
index.search(”*",:limit => :all).total_hits

Hi,

Could somebody mind trying to reproduce this please ?

Highlighting is a very important feature for me, I need to know if I’m
doing something wrong or if it’s a dirty bug.

Thanks in advance

On Wed, Jun 20, 2007 at 12:30:03AM +0200, [email protected] wrote:

then highlight the matched tokens. If it’s matched and not highlighted
I would appreciate if you could test and tell me if I’m the only one
having this problem.

Here z is not empty in the 500 case, too. Strange behaviour indeed,
imho highlight should either return strings with some highlighted
content, or nothing at all when there’s nothing to highlight…

Jens


Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

Jens K. [email protected] writes:

Here z is not empty in the 500 case, too. Strange behaviour indeed,
imho highlight should either return strings with some highlighted
content, or nothing at all when there’s nothing to highlight…

Hi,

Thanks Jens for your answer.
hmm since content has to be stored the default behavior doesn’t
surprise me.
But the issue does ! :slight_smile:
I’m looking forward to read Dave about this :slight_smile: