Need some advice : scientific scripting language project

[emmanuel]

Hello,

Ok, I am writing a (pretty big) project trying to use ruby as the main
scripting language for performing math computation, scientific display
and display interaction.
Here is roughly what I want

• a terminal window with a ruby interpreter in it (I want to be able to
  control all the completions/editions capabilities)
• a bench of ruby classes to manipulate signals/images,etc…
• I want everything to be really really fast (I’ll be manipulating huge
  signals/images)
• simple commands to display instances of these classes
• graphic (programable) interaction on these displays (e.g. moving the
  mouse on a such a window should display image values, clicking should
  zoom, etc…

Now my questions are the following

• Should I use irb ? Can I wrap it easily in a wxWidget window ? Can I
  reprogram some new capabilities (e.g., history completion, multi-line
  history commands…)

• A unix terminal window displays output text really really fast… what
  object should I use in wxRuby to display text as fast (wxTextCtrl is
  pretty slow, even if I use a buffer)

• I want to be able to interrupt long computations (clicking on a stop
  button or typing ctrl-c on the terminal window) ==> I should be able to
  send SIGINT signal to the interpreter. Should I use 2 processes : one
  for the terminal and one for the rest ?
  The problem if I do that is that both processes will have a
  wxMainLoop… now what will be the best way for them to communicate ?

Thank you very much

Emmanuel

[emmanuel]

Hi Emmanuel

On 19/05/2010 07:47, Emmanuel E. wrote:

Ok, I am writing a (pretty big) project trying to use ruby as the main
scripting language for performing math computation, scientific display
and display interaction.

Sounds like a big and interesting project. I originally got interested
in wxRuby building research tools (for sociology) so I’d certainly be
interested to hear your progress.

Here is roughly what I want

• a terminal window with a ruby interpreter in it (I want to be able to
  control all the completions/editions capabilities)

readline?

• a bench of ruby classes to manipulate signals/images,etc…
• I want everything to be really really fast (I’ll be manipulating huge
  signals/images)

ruby/gsl? r-ruby?

• simple commands to display instances of these classes
• graphic (programable) interaction on these displays (e.g. moving the
  mouse on a such a window should display image values, clicking should
  zoom, etc…

wxRuby is well suited to this sort of application.

Now my questions are the following

• Should I use irb ? Can I wrap it easily in a wxWidget window ? Can I
  reprogram some new capabilities (e.g., history completion, multi-line
  history commands…)

I don’t know of any examples of wrapping IRB in wxRuby, although it does
provide a library interface. You might look into Wx::StyledTextCtrl
which offers syntax highlighting and other more sophisticated support
for programming code text.

• A unix terminal window displays output text really really fast… what
  object should I use in wxRuby to display text as fast (wxTextCtrl is
  pretty slow, even if I use a buffer)

The text display classes are Wx::TextCtrl, Wx::LogTextCtrl,
Wx::StyledTextCtrl, Wx::RichTextCtrl and Wx::HtmlControl. I’d expect
simple TextCtrl or StyledTextCtrl to be fastest, but I’ve never tested.

Judicious use of #freeze and #thaw might help to manage updates on
screen. Generally Ruby is fairly slow, and wxRuby’s bridge to C++ adds
additional overhead to each Wx method call, so you might improve speed
by batching calls to the Wx API to update the TextCtrl’s context.

• I want to be able to interrupt long computations (clicking on a stop
  button or typing ctrl-c on the terminal window) ==> I should be able to
  send SIGINT signal to the interpreter. Should I use 2 processes : one
  for the terminal and one for the rest ?
  The problem if I do that is that both processes will have a
  wxMainLoop… now what will be the best way for them to communicate ?

You have a couple of options:

• if the computational task can be broken down into small discrete
  elements, you can execute these in an evt_idle handler, which runs when
  the GUI isn’t doing anything else. This is the easiest route because you
  can allow interruptions using normal wxRuby event handling techniques.

• you could use Ruby threads and run the computation in a slave thread.
  This is somewhat more complex because the slave thread should not
  directly update the GUI, but should signal its state by posting Wx
  events to the main GUI thread. A class that does this sort of thing is
  here (afraid it’s embedded in a fairly complex Wx app):

http://weft-qda.rubyforge.org/svn/trunk/weft-qda/lib/weft/wxgui/gui_subscriber.rb

If you use wxRuby and threads in Ruby 1.8 you need to explicitly
allocate time to the slave thread - there are numerous discussions of
this in the list archives.

• you could fork / pipe and run the computation as a separate process.
  There is no need for this to be a GUI process, I presume, so you don’t
  need to worry about a separate GUI main_loop. One good thing about this
  route is that it will take advantage of multiple cores if they’re
  available, and will separate the GUI and analytic code. But it’ll
  involve working out how to pass data to the subsidiary process (eg
  popen3?) and manage and signal to it (SIGINT etc?). Given Ruby’s unix
  background, managing a child process is easier if you only need the
  program to work on *nix platforms, not Windows.

hth
alex

[emmanuel]

First of all, Thank you Alex for all your interesting comments !

Alex F. wrote:
Hi Emmanuel

Sounds like a big and interesting project. I originally got interested
in wxRuby building research tools (for sociology) so I’d certainly be
interested to hear your progress.

I will let you know. Actually I already wrote a script language (called
LastWave, LastWave Home)
that has been quite used by a lot of persons… I started it a long time
ago, so I did not use the modern script language (python or ruby)…
it is version 3.0 now and using wxWidget… but it clearly needs to be
rewritten using a good scripting language… it needs to be entirely
redesigned…
However I learned a lot, and I am pretty clear about some of the issues
(memory, speed, some graphics issue).

I know there exist some language such as Matlab ($$$) or R (free), but I
do think these languages suffer from some drawbacks (I could develop
that point but it is not the subject!).
Moreover, if I write such a soft using Ruby for instance, I could
benefit from both Matlab an R (there are libraries to encapsulate their
engine)…

Here is roughly what I want

• a terminal window with a ruby interpreter in it (I want to be able to
  control all the completions/editions capabilities)

readline?

I am not sure to understand… I want a window in which I can redirect
any key being hit so that I can implement multi-line editing commands,
history completion and so on …
readline is a library ?

• a bench of ruby classes to manipulate signals/images,etc…
• I want everything to be really really fast (I’ll be manipulating huge
  signals/images)

ruby/gsl? r-ruby?

nope and this is clear from my experience… I know pretty precisely
what I want … and it is not on the market… I’ll have to do it (I
alredy started and it is not the hardes part at all).
But yes, I want my soft to be able to communicate with the libraries
that deal with the Matlab engine and the R engine

• simple commands to display instances of these classes
• graphic (programable) interaction on these displays (e.g. moving the
  mouse on a such a window should display image values, clicking should
  zoom, etc…

wxRuby is well suited to this sort of application.

Yes I know it is, since LastWave has been done fully on wxWidget !

Now my questions are the following

• Should I use irb ? Can I wrap it easily in a wxWidget window ? Can I
  reprogram some new capabilities (e.g., history completion, multi-line
  history commands…)

I don’t know of any examples of wrapping IRB in wxRuby, although it does
provide a library interface. You might look into Wx::StyledTextCtrl
which offers syntax highlighting and other more sophisticated support
for programming code text.

Is there a irb documentation (of the class I mean) anywhere on the web ?
I could not find it!

• A unix terminal window displays output text really really fast… what
  object should I use in wxRuby to display text as fast (wxTextCtrl is
  pretty slow, even if I use a buffer)

The text display classes are Wx::TextCtrl, Wx::LogTextCtrl,
Wx::StyledTextCtrl, Wx::RichTextCtrl and Wx::HtmlControl. I’d expect
simple TextCtrl or StyledTextCtrl to be fastest, but I’ve never tested.

No wxTextCtrl is what I used on LastWave and I have been doing some test
on wxRuby recently. It is fast, but really nothing compared to a unix
terminal…

Judicious use of #freeze and #thaw might help to manage updates on
screen.
There are methods of which class ?

Generally Ruby is fairly slow, and wxRuby’s bridge to C++ adds
additional overhead to each Wx method call, so you might improve speed
by batching calls to the Wx API to update the TextCtrl’s context.
Yes I agree and plan to do that (though I heard ruby 1.9 is much faster!
Do you think I should use python instead?)

• I want to be able to interrupt long computations (clicking on a stop
  button or typing ctrl-c on the terminal window) ==> I should be able to
  send SIGINT signal to the interpreter. Should I use 2 processes : one
  for the terminal and one for the rest ?
  The problem if I do that is that both processes will have a
  wxMainLoop… now what will be the best way for them to communicate ?

You have a couple of options:

• if the computational task can be broken down into small discrete
  elements, you can execute these in an evt_idle handler, which runs when
  the GUI isn’t doing anything else. This is the easiest route because you
  can allow interruptions using normal wxRuby event handling techniques.

No, what you call the “comoutational part” is not only computational…
It has graphics drawing (that must be fast) … I want to write
something “a la matlab” so there is not a clear cut in between
terminal/graphics and computations

• you could use Ruby threads and run the computation in a slave thread.
  This is somewhat more complex because the slave thread should not
  directly update the GUI, but should signal its state by posting Wx
  events to the main GUI thread. A class that does this sort of thing is
  here (afraid it’s embedded in a fairly complex Wx app):

http://weft-qda.rubyforge.org/svn/trunk/weft-qda/lib/weft/wxgui/gui_subscriber.rb

If you use wxRuby and threads in Ruby 1.8 you need to explicitly
allocate time to the slave thread - there are numerous discussions of
this in the list archives.
Yes I tried (maybe too naively) doing threads… but it slowed down the
computations a lot … again I maight have done it the wrong way !

• you could fork / pipe and run the computation as a separate process.
  There is no need for this to be a GUI process, I presume, so you don’t
  need to worry about a separate GUI main_loop. One good thing about this
  route is that it will take advantage of multiple cores if they’re
  available, and will separate the GUI and analytic code. But it’ll
  involve working out how to pass data to the subsidiary process (eg
  popen3?) and manage and signal to it (SIGINT etc?). Given Ruby’s unix
  background, managing a child process is easier if you only need the
  program to work on *nix platforms, not Windows.

I want to work on ALL platforms. You think I sould not use 2 processes
then ?
Now how would the 2 processes communicate if they are both running
wxMainLoop (one for the terminal and the other one for all other
graphics…) ?

There is one thing VERY important for me : I need to be able to stop a
computation if the user decides it’s taking too long for instance… and
I don’t want the soft to crash !

Thank you again

Emmanuel

hth
alex

[emmanuel]

On 19/05/2010 14:10, Emmanuel E. wrote:

I will let you know. Actually I already wrote a script language (called
LastWave, LastWave Home)
that has been quite used by a lot of persons… I started it a long time
ago, so I did not use the modern script language (python or ruby)…
it is version 3.0 now and using wxWidget… but it clearly needs to be
rewritten using a good scripting language… it needs to be entirely
redesigned…

Looks good, although I don’t understand the topic of analysis

history completion and so on …
readline is a library ?

I was wondering if the library could be used to provide
command-completion and history features - but on looking now maybe it’s
quite tied to the terminal which hosts it.

ruby/gsl? r-ruby?

nope and this is clear from my experience… I know pretty precisely
what I want … and it is not on the market… I’ll have to do it (I
alredy started and it is not the hardes part at all).
But yes, I want my soft to be able to communicate with the libraries
that deal with the Matlab engine and the R engine

I’d be really interested to hear how you find the Ruby-R bindings, as
I’ve been doing increasing amounts of stat analysis with R over the past
year.

I don’t know of any examples of wrapping IRB in wxRuby, although it does
provide a library interface. You might look into Wx::StyledTextCtrl
which offers syntax highlighting and other more sophisticated support
for programming code text.

Is there a irb documentation (of the class I mean) anywhere on the web ?
I could not find it!

I can’t find anything either - it doesn’t appear on ruby-doc.org even
though it’s part of the standard library. I think when I did a bit of
work extending it I just relied on the source files.

The 1.9 module structure and method listing is here:
http://ruby-doc.org/ruby-1.9/classes/IRB.html

on wxRuby recently. It is fast, but really nothing compared to a unix
terminal…

No, I imagine not. Like to know whether there is a perceptible speed
difference between wxTextCtrl (C++) and Wx::TextCtrl (Ruby) performing a
comparable task.

Judicious use of #freeze and #thaw might help to manage updates on
screen.

There are methods of which class ?

Wx::Window, so applies to all things that are drawn on screen.

Generally Ruby is fairly slow, and wxRuby’s bridge to C++ adds
additional overhead to each Wx method call, so you might improve speed
by batching calls to the Wx API to update the TextCtrl’s context.

Yes I agree and plan to do that (though I heard ruby 1.9 is much faster!
Do you think I should use python instead?)

Ruby 1.9 brought particular speed improvements to some numeric
operations, which might be relevant to your project.

Python?! I don’t know. It probably is faster. But it’s not so much
faster that it should outweigh your preference of language, and tools.
wxPython is of course very well established, with a longer history than
wxRuby.

No, what you call the “comoutational part” is not only computational…
It has graphics drawing (that must be fast) … I want to write
something “a la matlab” so there is not a clear cut in between
terminal/graphics and computations

It sounds like you have a decomposable task - ie there are interim or
partial results from part of the whole computation. So any option is
open to you. It’s a question of what’s fast enough, and how it’s most
comfortable to pass data (results) back to the GUI layer to be drawn on
screen.

If you use wxRuby and threads in Ruby 1.8 you need to explicitly
allocate time to the slave thread - there are numerous discussions of
this in the list archives.

Yes I tried (maybe too naively) doing threads… but it slowed down the
computations a lot … again I maight have done it the wrong way !

Yes, it’ll be pretty inefficient because the Ruby thread scheduler is
being forced to jump (rather clumsily) between the GUI event loop and
the computational task in the slave thread. You might test evt_idle to
see if it processes the same task faster.

I want to work on ALL platforms. You think I sould not use 2 processes
then ?

Ordinary ruby will not take advantage of more than one core - but if you
spawn a separate process it may run on a different core. This may be a
big advantage for your use case.

Now how would the 2 processes communicate if they are both running
wxMainLoop (one for the terminal and the other one for all other
graphics…) ?

You could perhaps open a subprocess with IO#popen
(class IO - RDoc Documentation), write the data to be
processed to the process’s STDIN, and then read the results back as
they’re available on the process’s STDOUT (and do any graphical display
that’s wanted).

There is one thing VERY important for me : I need to be able to stop a
computation if the user decides it’s taking too long for instance… and
I don’t want the soft to crash !

Several ways - perhaps just closing the pipes to the subprocess will
kill it; you could send it a system signal; or kill it with Process.
Afraid I don’t know what is needed to make this portable to Windows.

a

[emmanuel]

Emmanuel E. wrote:

Thanks again for teh time you spend answering me !

You’re welcome.

No, I imagine not. Like to know whether there is a perceptible speed
difference between wxTextCtrl (C++) and Wx::TextCtrl (Ruby) performing a
comparable task.

No there is none

That’s useful to know, thanks.

yes, so for that, the receiving process should have idle events reading
the stdin right to wait for inputs right?

Something like that, yes. Use evt_idle if possible since you can
directly update the GUI state in that handler. Dealing with threads,
posting events, synchronisation etc is much more hassle.

Do you know if this works well on Windows platform ?

IIRC Ruby’s own fork, popen and popen3 DON’T work on Windows. However
there are definitely libraries which provide comparable functionality
and similar API through the Win32 API.

This is the signal thing you are afraid it might be not portable or even
more than that ?

I’m not sure but I think SIGINT / SIGKILL are *nix specific. Just having
the pipe might be enough to control the slave process and terminate
when needed. Otherwise there are various other options for communicating
between processes - have a search for ‘ruby IPC’.

best
alex

[emmanuel]

Not to jump in and ruin the discussion, quite good stuff. I just have
one
thing to comment on:
popen3/popen/etc/etc.

When I was working on my Ruby IDE, I found the limitation of
popen/popen3,
and the very many different implementations of how to properly utilize
the
API to do what you wanted. So I created my own Library, called
xprocess.
It utilizes the same methods that are used across all platforms, and
provides a singular, unified API, that is consistent, and the same
across
all platform.

I don’t have any Gem build of it, or published on RubyForge, as I only
had
the need for the library at the time. It will need to be compiled on
any
platform you wish to implement it on, but you can get it from
http://repos.ruby-im.net/xprocess There is currently no documentation
for
it, but you can look in the test directory, and see a couple of samples
of
how to use it.

hth,

Mario

[emmanuel]

Thanks again for teh time you spend answering me !

I’d be really interested to hear how you find the Ruby-R bindings, as
I’ve been doing increasing amounts of stat analysis with R over the past
year.

Ok, let me go into it further and I’ll develop that in an other post.

No, I imagine not. Like to know whether there is a perceptible speed
difference between wxTextCtrl (C++) and Wx::TextCtrl (Ruby) performing a
comparable task.
No there is none

Judicious use of #freeze and #thaw might help to manage updates on
screen.

There are methods of which class ?

Wx::Window, so applies to all things that are drawn on screen.
thanks !

Ruby 1.9 brought particular speed improvements to some numeric
operations, which might be relevant to your project.

Python?! I don’t know. It probably is faster. But it’s not so much
faster that it should outweigh your preference of language, and tools.
wxPython is of course very well established, with a longer history than
wxRuby.

ok

Yes I tried (maybe too naively) doing threads… but it slowed down the
computations a lot … again I maight have done it the wrong way !

Yes, it’ll be pretty inefficient because the Ruby thread scheduler is
being forced to jump (rather clumsily) between the GUI event loop and
the computational task in the slave thread. You might test evt_idle to
see if it processes the same task faster.

I’ll try!

I want to work on ALL platforms. You think I sould not use 2 processes
then ?

Ordinary ruby will not take advantage of more than one core - but if you
spawn a separate process it may run on a different core. This may be a
big advantage for your use case.

Now how would the 2 processes communicate if they are both running
wxMainLoop (one for the terminal and the other one for all other
graphics…) ?

You could perhaps open a subprocess with IO#popen
(class IO - RDoc Documentation), write the data to be
processed to the process’s STDIN, and then read the results back as
they’re available on the process’s STDOUT (and do any graphical display
that’s wanted).

yes, so for that, the receiving process should have idle events reading
the stdin right to wait for inputs right?

Do you know if this works well on Windows platform ?

There is one thing VERY important for me : I need to be able to stop a
computation if the user decides it’s taking too long for instance… and
I don’t want the soft to crash !

Several ways - perhaps just closing the pipes to the subprocess will
kill it; you could send it a system signal; or kill it with Process.
Afraid I don’t know what is needed to make this portable to Windows.

This is the signal thing you are afraid it might be not portable or even
more than that ?

Emmanuel

[emmanuel]

Ok,

let me describe what I have done and working very well on a macos X…
I am going to sum up the code … (just have in mind that the goal is to
get an interaction between a terminal window and computations/graphics
in the “same” way as R for instance).

I still have a few questions (to both of you !) and any comments will be
great !

=====
The father process opens a child process (using popen mode “r+”) and
opens a window with a TextCtrl in it and just waits with wxMainLoop.
This process has basically two possible status : i) we are typing in a
command (so this is a basic line-editor program) and as soon as we hit a
return key, this command is sent to the child process using pipe.puts
(pipe is the object returned by the popen command for now) and then it
switches to the ii) waiting mode.
In this last mode, the process (every 10ms) reads the stdout of the
child process (pipe.read_nonblock) and prints in the terminal window
whatever is in it as long as a specific character sequence is found
indicating that the command has been executed by the child process ==>
return to the command mode i)

=====
The child process is very simple : a wxMainLoop (again!) and every 10ms
an Idle procedure is ran.
It reads the $stdin using $stdin.read_nonblock. If there is something
then it corresponds to a command that will be executed, e.g.,
$ans = eval(command,TOPLEVEL_BINDING)
(I take care of eventual errors of course)
And then, it just $stdout.puts the specific character sequence that
indicates command execution is over and $stdout.flush

Notice that this process can open windows, draw graphics, handle
interactions with these graphics …

++++++++++++++

That’s it.
In this way, the display on the terminal is really fast (never got
something as fast). Be aware that the (ruby) command that is executed
with eval can use puts … and it will print on the terminal.
The output buffering system is the one of the system…!

So for instance

100000.times {|i| puts i}

takes less that 2 seconds !
Which is really fast.

Moreover if a ctrl-C is typed in the terminal window I can send a SIGINT
to the child process (I do it using C raise) which interrupts the
execution of the ruby command !?!

I was really happy… till I realized that, it will not work on Windows
XP!

==> First I looked on the web to see if there is anything that could
replace the popen. I found popen4… BUT
a- You have to pass a block (at the end of the block the process
dies)… which is not good for me since I want the child process to live
through the wxMainLoop
b- (which is related to a- I think) I want to do a read_nonblock (so I
don’t reach the eof … )

==> MARIO : does your library will help me ? Does it manage the
stdin/stdout of the child process in a standard way (buffering and
all)? Is it non blocking ? Can I read and write (non blocking)
alternatively without closing the process ?

==> Do you have any other idea (Alex, I guess you understand exactly
what I want to do since you know R)

Thank you for everything !!

Emmanuel

[emmanuel]

Hello Emmanuel,

The XProcess library I developed, is asynchronous, and has methods in
which
to detect if there is data available on a pipe object or not. I setup
the
library specifically for the need to be able to be async to keep the GUI
responsive while the sub-process is being executed.

As I said before, there are many different methods across Windows,
Linux,
and OS X, in which to create a pipe to a sub-process, and Windows lacks
the
major feature in which to allow sub-proceses naturally in ruby, which is
Fork. What I have done, is created a single API interface in which to
be
able to create processes, and send and receive data from the
sub-process,
without blocking the wxWidgets API.

I did think about wrapping wxProcess, but after looking through the
documentation for it, I felt that it would just be simpler to create the
API
myself in another class, then to go through the trouble of trying to
wrap
wxProcess.

The simplest way I can explain the API for XProcess, is that you
initialize
the class with the executable you wish to run. EG:

xproc = XProcess.new(“ruby”)

Then you use the method execute to pass along command line arguments to
the
sub-process, such as -e, -v, or whatever command line arguments your
sub-process needs.

Then the xproc object has 4 attributes, pid, which is the process id,
in,
out, and err, which are the representations of STDIN, STDOUT, and STDERR
for
the sub-process.

You can get output from the process by doing:

if xproc.out.is_data_avail?
data = xproc.out.gets
puts “Received: #{data}”
end

The method is_data_avail? on STDOUT and STDERR will not block the
process,
it will either return true, or false.

If you want to see if the process has closed either STDOUT or STDERR,
you
simply check by using the eof? method on the STDOUT or STDERR.

if xproc.out.eof?
puts “The process has closed STDOUT”
end

XProcess doesn’t close the IO handles for input, output or error, so you
will need to close them yourself, if you don’t want to use them. And
if
you want to kill the process if it’s taking to long, simply call the
kill
method.

I will need to write up some documentation for the XProcess library, but
it’s fairly straight forward, and simple to use. I try to keep things
simplified, and unified across all platforms.

I have had confirmation that this library works on all 3 platforms, but
if
you find any bugs, please feel free to let me know, off list, and I will
see
what I can do to fix the problems.

hth,

Mario

[emmanuel]

ouups :

==> what about using wxProcess (i.e., creating a process using wxWigets
API) ??

E.

[emmanuel]

Thanks Mario for this long answer !

Few questions

1- Is there also a way to read a maximum number of bytes (non blocking,
knowing that there are some data available) ?
(the equivalent of .read_nonblock()

2- Is it working with ruby 1.9 ?

3- How do I install it, is there a small doc ?

Thanks

Emmanuel

[emmanuel]

ouups sorry for this message (again!)

I was wrong in my last post… it is not blocking. Something really
weired happens.
As I already explained both processes run a wxMainLoop and both have
wxTimers that wake up the process regularly.

So when the child process is processing the stdin the parent process
does not receive any alarm from its timer any more ??!

[emmanuel]

Okay Mario,

I did install it, it seems to work … except that, I am doing
something wrong maybe, but in order to send to the child process some
strings into the stdin, I do

cmd.in.puts(string)

Now, when the child process finds a string in the stdin, it processes it
… and really weired… but this cmd.in.puts is blocking as long as
the child process has finished its processing ???

I really don’t understand what’s hapenning here

Could you help me ?

Emmanuel

[emmanuel]

Hello Mario,

I identified the problem (first of all no I am NOT using ruby 1.9… I
was just asking …) :

When the child process is writing quickly to the stdout then the call
(from the main process) to

cmd.out.data_avail?

basically waits for the child process to have finished writing…

---

Example :
parent process runs Idle every 10ms. Idle looks at the stdout of the
child process and prints on a textCtrl whatever is in it

child process runs 100000.times {|i| puts i}

==> The whole thing takes 3 seconds to run (macos X) but the flush is
done just once !
(Idle is called once and blocks at cmd.out.data_avail?, wait for child
process to finish writing and then displays all the numbers at once)

Let’s note that if I put a sleep after the puts then the display is
updated very often during the loop…

Emmanuel

[emmanuel]

hi

Glad you’re making progress; not surprised to hear that Windows is the
sticking point. Mario knows a lot more than me about using pipes etc
with wxRuby so just a few observations that might or might not be
useful:

Emmanuel E. wrote:

The child process is very simple : a wxMainLoop (again!) and every 10ms
an Idle procedure is ran.

Are you sure child process needs to be a wx program at all? I would
expect it to be a simple Ruby non-GUI program if it just uses STDIN and
STDOUT to communicate in an efficient format. With that you can test
different options, eg

• a long-running server to handle many jobs (which starts up and then
  enters an endless “loop do … end”) vs a per-job new process. If it’s
  like R, the overhead of starting a new process per-job is insignificant
  against the overall job computation time. And you can close the write
  pipe on each job and then read data incrementally, rather than the more
  complex popen3 model.

• I wonder if Drb (distributed ruby) might work? I have used it with
  some success in attaching a wxRuby GUI to an application running
  separately. One other bonus of this is it would then be easy to move to
  a client/server model on different pcs if wanted in the future.

Notice that this process can open windows, draw graphics, handle
interactions with these graphics …

Is this in the child process? That seems an unusual arrangement but of
course there might be very good reasons for this…

++++++++++++++

That’s it.
In this way, the display on the terminal is really fast (never got
something as fast).

Great. I’m away on hols for a few weeks now so won’t be picking up, but
best of luck with the project

alex

[emmanuel]

Hey Emmanuel,

First off, I haven’t really tested Ruby 1.9, so if it’s working for you,
that’s great.

Second off, once any output to stdin is executed, it should
automatically
return, without any delays. I will have to investigate this more,
unless
it’s this third thing.

Third, are you using Ruby 1.9, Wx::Timer, and Threads in the setup?
Cause,
if your using Ruby 1.9, there is no need to utilize Wx::Timer, to wake
up
your application, as Ruby 1.9 has native Thread support, which is
compatible
with wxWidgets thread utilization setup. The only problem you would
have,
is attempting to update the GUI from any other thread, outside the Main
Thread. Which is the base level thread, that your app runs in. If you
attempt to update the GUI from any other thread, wxWidgets will panic,
and
seg-fault on you, or lock your program up.

I will be in IRC, if you wish to discuss more about my library. irc://
irc.freenode.net:6667/#wxruby

hth,

Mario