Re: Splitting a sentence with delimiter preserved

Ruby
1

From: Ajithkumar Warrier [mailto:[email protected]]

puts b
p string.scan( /\w[^.!?]+\S+/ )

#=> [“This is an example string.”, “The purpose is to save the delimiter
during split.”, “Does this work.”, “Great!!!.”]

2

On 10/17/06, Gavin K. [email protected] wrote:
That was very quick.

Thank you.

3

Hi –

On Wed, 18 Oct 2006, Gavin K. wrote:

b = string.split(’#’)
puts b

p string.scan( /\w[^.!?]+\S+/ )

#=> [“This is an example string.”, “The purpose is to save the delimiter
during split.”, “Does this work.”, “Great!!!.”]

Also, in 1.9, with oniguruma, you can do:

string.split(/(?<=.)\s+/)

(negative lookbehind).

David

4

[email protected] wrote:

Also, in 1.9, with oniguruma

Is the current (1.8.5) regex engine some (other) well-known engine? For
example it is very like PCRE, but I take it that it is not PCRE. Just
curious. m.

5

[email protected] wrote:

Also, in 1.9, with oniguruma, you can do:

string.split(/(?<=.)\s+/)

(negative lookbehind).

Er, positive lookbehind, I believe you mean.

For completeness, if you wanted to use this form and also wanted to
allow exclamation points and question marks as sentence delimiters in
addition to periods, you could use:

string.split( /(?<=[.!?])\s+/ )

6

On Oct 17, 2006, at 9:55 PM, matt neuburg wrote:

[email protected] wrote:

Also, in 1.9, with oniguruma

Is the current (1.8.5) regex engine some (other) well-known engine?
For
example it is very like PCRE, but I take it that it is not PCRE.

Ruby’s current regex engine is pretty limited compared to PCRE or
Oniguruma. I’m not aware of the name for the current engine.

James Edward G. II

7

Robert K. wrote:

during split. Does this work. Great!!!. It costs 0.1 dollars."

string.scan( /\w[^.!?]+\S+/ )
=> [“This is an example string.”, “The purpose is to save the\ndelimiter
during split.”, “Does this work.”, “Great!!!.”, “It costs 0.1”, “do
llars.”]

Down this path leads the madness that is trying to use simple regexp to
parse something as complex as English grammar. That said, here’s
another regexp that still works and fixes that particular case:

string = “This is an example string. The purpose is to save the
delimiter during split. Does this work. Great!!!.”
string = string + " It costs 0.1 dollars."
p string.scan( /\w.+?[.!?]+(?=\s|\Z)/ )
#=> [“This is an example string.”, “The purpose is to save the
delimiter during split.”, “Does this work.”, “Great!!!.”, “It costs 0.1
dollars.”]

It’ll still fail on sentences with embedded quotes that have
sub-sentences within them.

8

On 17.10.2006 22:37, Gavin K. wrote:

b = string.split(’#’)
puts b

p string.scan( /\w[^.!?]+\S+/ )

#=> [“This is an example string.”, “The purpose is to save the delimiter
during split.”, “Does this work.”, “Great!!!.”]

string = “This is an example string. The purpose is to save the
delimiter during split. Does this work. Great!!!.”

string = string + " It costs 0.1 dollars."
=> “This is an example string. The purpose is to save the\ndelimiter
during split. Does this work. Great!!!. It costs 0.1 dollars.”

string.scan( /\w[^.!?]+\S+/ )
=> [“This is an example string.”, “The purpose is to save the\ndelimiter
during split.”, “Does this work.”, “Great!!!.”, “It costs 0.1”, “do
llars.”]

Hm…

robert