Global variable mysteriously becoming nil half way through execution of the script

Hi all.

I have something of a black box mystery to solve. A user has reported
an error coming from this line:

$utilities.exporter.export(item, output_file)

The error is:

export.rb:32:in `export_files': undefined method `exporter' for

nil:NilClass (NoMethodError)

What’s odd about this is that the code is being called in a loop, and
works for several iterations before failing later. So $utilities must
have existed initially, and then vanished later on.

Needless to say, this is the only usage of $utilities in the entire
script. The value for it is put into the ScriptContext before the
script is executed and not touched at any other time.

Has anyone seen anything like this before? I can’t help thinking that
perhaps some other error occurred and then the wrong message was
printed.

Unfortunately the things I don’t know are a bigger problem. For
instance I don’t know if they might have tried running another script
while the first one was still running.

TX


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Hi,

On Sun, Sep 5, 2010 at 9:12 PM, Trejkaz [email protected] wrote:

nil:NilClass (NoMethodError)

What’s odd about this is that the code is being called in a loop, and
works for several iterations before failing later. So $utilities must
have existed initially, and then vanished later on.

Needless to say, this is the only usage of $utilities in the entire
script. The value for it is put into the ScriptContext before the
script is executed and not touched at any other time.

So, did this happen when the script was evaluated using JSR223 impl?
Was the default configuration used?
This sounds bug to me, but let me clarify how this happened. JSR223
requires no state should be remained after the evaluation. To fulfill
this requirement, JSR223 impl sets null to global vars on runtime that
are tied to vars in ScriptContext when the evaluation ends. The global
vars in ScriptContext should be set again for successive evaluation,
but this part seems to have a problem.

-Yoko


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

On Wed, Sep 8, 2010 at 1:27 AM, Yoko H. [email protected] wrote:

What’s odd about this is that the code is being called in a loop, and
works for several iterations before failing later. Â So $utilities must
have existed initially, and then vanished later on.

Needless to say, this is the only usage of $utilities in the entire
script. Â The value for it is put into the ScriptContext before the
script is executed and not touched at any other time.

So, did this happen when the script was evaluated using JSR223 impl?
Was the default configuration used?

Yes on both. We’re using JSR223 in a very generic way, with no
knowledge of JRuby specifically (the same code supports JavaScript and
I know at least one user who installed Jython themselves and is using
that.)

This sounds bug to me, but let me clarify how this happened. JSR223
requires no state should be remained after the evaluation. To fulfill
this requirement, JSR223 impl sets null to global vars on runtime that
are tied to vars in ScriptContext when the evaluation ends. The global
vars in ScriptContext should be set again for successive evaluation,
but this part seems to have a problem.

We do use a fresh ScriptContext for each run so I would expect them to
be isolated. But instead of asking whether this was actually the
case, I did an experiment.

Longer-running script:
20.times do
puts “$utilities = #{$utilities}”
sleep 5
end

Shorter-running script:
puts “shorter script run”

I start the longer-running script and then run the shorter one part
way through, and the output was quite mysterious:
$utilities = script.impl.DefaultUtilities@8864af
$utilities = script.impl.DefaultUtilities@8864af
$utilities = script.impl.DefaultUtilities@8864af
shorter script run
Script completed successfully with return value: null

Then a minute later:
Script completed successfully with return value: null

So it looks like the original context loses its output stream at the
time the second script either starts or completes.

I tried another one, this time calling a method, and bingo:

exporter = script.impl.exporter.DefaultExporter@1d2052b
exporter = script.impl.exporter.DefaultExporter@1d2052b
shorter script run
Script completed successfully with return value: null
Script failed due to an error:
C:\Scripts\Longer Running.rb:2: undefined method `exporter' for

nil:NilClass (NoMethodError)
from C:\Scripts\Longer Running.rb:1:in `times’
from C:\Scripts\Longer Running.rb:1
…internal jruby stack elided…
from (unknown).(unknown)(C:\Scripts\Longer Running.rb:1)
from Integer.times(C:\Scripts\Longer Running.rb:1)
from (unknown).(unknown)(:1)

For the sake of comparison, I tried writing the same script using
JavaScript to see what Rhino would do in this situation:

Longer running:
for (var i = 0; i < 20; i++) {
println("utilities = " + utilities.exporter);
java.lang.Thread.sleep(1000);
}

Shorter running:
println(“shorter script run”);

Rhino behaves as I would expect - the script contexts are isolated
from each other and the message continues to print after the short
script terminates.

TX


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

On Tue, Sep 7, 2010 at 7:34 PM, Trejkaz [email protected] wrote:

Was the default configuration used?
vars in ScriptContext should be set again for successive evaluation,
end
Script completed successfully with return value: null
exporter = script.impl.exporter.DefaultExporter@1d2052b
from (unknown).(unknown)(:1)

OK. Thanks for explaining in detail. It was helpful to figure the
problem out.

This is the expected behavior because the code used a global variable.
As you know, Ruby has scopes of variables. The global variables are
globally referenced vars on Ruby runtime, which are similar to Java’s
static variables. To make the code work in the same way as JavaScript,
you need to use a local variable instead of global variable. So, you
should choose transient variable behavior and take $ out from
variable. gist:569506 · GitHub is the sample that won’t lose
the variable value. This code outputs:

utilities = OK?
utilities = OK?
shorter script run
utilities = OK?

Please be aware of. Even though transient type is chosen, global vars
will be lost when another script is evaluated. If you change the code
of the gist as in below:

    context1.setAttribute("$utilities", "OK?", 

ScriptContext.ENGINE_SCOPE);
String script1 =
“20.times do\n” +
" puts "$utilities = #{$utilities}"\n" +
" sleep 5\n" +
“end”;

you’ll get:

$utilities = OK?
$utilities = OK?
shorter script run
$utilities =

JSR223 requires that values used in the script should be available to
use in Java code, in this case, the value of $utilities should be
reuse in Java. Because of this, when the evaluation of “short run”
script ends, $utilities is saved in Java side and set nil in its
value. Java program can’t discern $utilities is originally in other
ScriptContext because it is a global variable. Don’t use the global
variables for this sort of purposes while parallel processing is going
on.

println(“shorter script run”);

Rhino behaves as I would expect - the script contexts are isolated
from each other and the message continues to print after the short
script terminates.

Thanks for this JavaScript example.

I’m hoping this solves the trouble of variables.

-Yoko

TX


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

On Wed, Sep 8, 2010 at 12:57 PM, Yoko H. [email protected] wrote:

So it looks like the original context loses its output stream at the
time the second script either starts or completes.

I tried another one, this time calling a method, and bingo:

the variable value. This code outputs:
It would be nice to use local variables, actually. From my
perspective, it’s JRuby’s JSR223 implementation which is forcing me to
make them global even though my code isn’t asking for it at all. :frowning:

Here’s what we are using in the code:

    context.setAttribute("utilities",

container.getComponent(Utilities.class), ScriptContext.ENGINE_SCOPE);

So we have:

  • no dollar sign
  • ENGINE_SCOPE, not GLOBAL_SCOPE
  • a brand new ScriptEngine object being used for every run

Under these conditions, it’s quite surprising that even a “global”
variable from the script’s perspective is being shared between two
contexts.

Incidentally, if I try referencing this “utilities” variable as a
local variable, I get:

<script>:1: undefined local variable or method `utilities' for

main:Object (NameError)

It would be nice if “utilities” without the dollar sign did introduce
a local variable. Though it would require updating all our
documentation and example scripts, I think it would be better in the
long run.

TX


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

On Tue, Sep 7, 2010 at 10:57 PM, Yoko H. [email protected] wrote:

So, did this happen when the script was evaluated using JSR223 impl?
are tied to vars in ScriptContext when the evaluation ends. The global
sleep 5
shorter script run
exporter = script.impl.exporter.DefaultExporter@1d2052b
from Integer.times(C:\Scripts\Longer Running.rb:1)
you need to use a local variable instead of global variable. So, you
Please be aware of. Even though transient type is chosen, global vars
you’ll get:
script ends, $utilities is saved in Java side and set nil in its
value. Java program can’t discern $utilities is originally in other
ScriptContext because it is a global variable. Don’t use the global
variables for this sort of purposes while parallel processing is going
on.

There’s one more workaround. Ruby originated global vars won’t be
lost. When the global var value is assigned to another global vars in
Ruby code:

    String script1 =
        "$ruby_utilities = $utilities\n" +
        "20.times do\n" +
        "  puts \"$utilities = #{$utilities}\"\n" +
        "  puts \"$ruby_utilities = #{$ruby_utilities}\"\n" +
        "  sleep 5\n" +
        "end";

$ruby_utilities doesn’t lost its value. You’ll get the output:

$utilities = OK?
$ruby_utilities = OK?
$utilities = OK?
$ruby_utilities = OK?
shorter script run
$utilities =
$ruby_utilities = OK?

-Yoko

I’m hoping this solves the trouble of variables.


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

On Thu, Sep 9, 2010 at 2:52 PM, Yoko H. [email protected] wrote:

This is the default behavior for JSR223 to work as the reference
implementation did. So, you can change this default to transient and
persistent even though you are using JSR223 impl,
[…]
Yes, you can do this using JRuby’s JSR223 impl. Would you just try it?

OK, setting “org.jruby.embed.localvariable.behavior” does make it work.

It doesn’t necessarily please me, though. As a plugin API, a user is
going to expect a JRuby implementation to drop into an application
which may not have been coded with JRuby in mind. This user won’t
necessarily be able to set that system property, depending on how the
application is being launched. I guess in a very optimistic
situation, if it’s an app which is launched from a shell script or
batch file, they could edit that… if it’s launched from an .exe they
might be out of luck.

Usability-wise, I think things should work the “right way” out of the
box. But it’s unfortunate, because the only “reference
implementation” we have is Rhino, and JavaScript doesn’t really have
global variables so there is no true basis for comparison.

I think that this still makes JRuby weird though, because it’s taking
a variable which is local in some other implementation, and making it
global, even though the code made it ENGINE_SCOPE in the first place,
which is a clear sign to me that the code didn’t want it to be global.
“Principle of least surprise”, and all that.

TX


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Hi,

On Wed, Sep 8, 2010 at 10:15 PM, Trejkaz [email protected] wrote:

This is the expected behavior because the code used a global variable.
make them global even though my code isn’t asking for it at all. :frowning:
That’s just a default. You can choose transient variable behavior on
JSR223 implementation. Did you see my sample code,
gist:569506 · GitHub ? This uses JSR223 and transient
variable behavior. Also, please see
http://kenai.com/projects/jruby/pages/RedBridge#Transient_Local_Variable_Behavior

Under these conditions, it’s quite surprising that even a “global”
variable from the script’s perspective is being shared between two
contexts.

This is the default behavior for JSR223 to work as the reference
implementation did. So, you can change this default to transient and
persistent even though you are using JSR223 impl,

Incidentally, if I try referencing this “utilities” variable as a
local variable, I get:

:1: undefined local variable or method `utilities' for main:Object (NameError)

Again, please see gist:569506 · GitHub. You’ll learn how to
use local variables on JSR223.

It would be nice if “utilities” without the dollar sign did introduce
a local variable. Though it would require updating all our
documentation and example scripts, I think it would be better in the
long run.

Yes, you can do this using JRuby’s JSR223 impl. Would you just try it?

-Yoko

TX


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

I think we could elect to change this for JRuby 2.0 (or next major
release). I expect since we ship our own JSR223 impl now no one will
use the one from Sun’s repo. We could even ask sundar to change it
(if he even still works for Oracle anymore :frowning: ). I agree that we
should not change this behavior until we find a nice breaking point
(like a major release).

-Tom

On Thu, Sep 9, 2010 at 9:46 AM, Yoko H. [email protected] wrote:

That’s great!
The option is Java’s system property, so you can set that by a command
as the “JRuby’s JSR223 reference implementation.” Current
JSR223 engine for JRuby should be weird, too. I wanted to change the
JSR223 users follow this discussion. Perhaps, we’d better start new


blog: http://blog.enebo.com twitter: tom_enebo
mail: [email protected]


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

On Thu, Sep 9, 2010 at 1:14 AM, Trejkaz [email protected] wrote:

On Thu, Sep 9, 2010 at 2:52 PM, Yoko H. [email protected] wrote:

This is the default behavior for JSR223 to work as the reference
implementation did. So, you can change this default to transient and
persistent even though you are using JSR223 impl,
[…]
Yes, you can do this using JRuby’s JSR223 impl. Would you just try it?

OK, setting “org.jruby.embed.localvariable.behavior” does make it work.

That’s great!

It doesn’t necessarily please me, though. As a plugin API, a user is
going to expect a JRuby implementation to drop into an application
which may not have been coded with JRuby in mind. This user won’t
necessarily be able to set that system property, depending on how the
application is being launched. I guess in a very optimistic
situation, if it’s an app which is launched from a shell script or
batch file, they could edit that… if it’s launched from an .exe they
might be out of luck.

The option is Java’s system property, so you can set that by a command
line option. The command line option works both in a shell script and
batch file. Does this make sense to you?

Usability-wise, I think things should work the “right way” out of the
box. But it’s unfortunate, because the only “reference
implementation” we have is Rhino, and JavaScript doesn’t really have
global variables so there is no true basis for comparison.

I think you forgot Sun has released JSR223 engine for JRuby at
https://scripting.dev.java.net/. I mentioned the implementation there
as the “JRuby’s JSR223 reference implementation.” Current
implementation included in JRuby has default settings to work in the
same way as the Sun’s JSR223 engine for JRuby.

I think that this still makes JRuby weird though, because it’s taking
a variable which is local in some other implementation, and making it
global, even though the code made it ENGINE_SCOPE in the first place,
which is a clear sign to me that the code didn’t want it to be global.
“Principle of least surprise”, and all that.

If you think current JSR223 implementation behaves weirdly, Sun’s
JSR223 engine for JRuby should be weird, too. I wanted to change the
default variable behavior to transient model since it is natural to
Ruby. However, that would force people to rewrite their code, so I
kept the same variable behavior. I guess Sun’s JSR223 engine for JRuby
used global variables because the implementation might have been easy.
The only one person wrote about twenty implementations of JSR223. He
was not good at Ruby and didn’t have much time to think about a design
just for Ruby.

Changing default variable behavior would make you happy, but, at the
same time, that might make other people unhappy. I’m not sure how many
JSR223 users follow this discussion. Perhaps, we’d better start new
discussion about this?

-Yoko

TX


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

On Thu, Sep 9, 2010 at 1:08 PM, Thomas E Enebo [email protected]
wrote:

I think we could elect to change this for JRuby 2.0 (or next major
release). I expect since we ship our own JSR223 impl now no one will
use the one from Sun’s repo. We could even ask sundar to change it
(if he even still works for Oracle anymore :frowning: ). I agree that we
should not change this behavior until we find a nice breaking point
(like a major release).

I agree that the change should be made in the next major release. I’m
hoping all JSR223 users will be happy to have transient variable
behavior as a default. The behavior is more Ruby friendly than the
current default setting.
Besides, the next major release will have a makeover of JSR223 impl.
Currently, I’m working to fix bugs of JSR223 impl.

-Yoko

Yes, you can do this using JRuby’s JSR223 impl. Would you just try it?
situation, if it’s an app which is launched from a shell script or
global variables so there is no true basis for comparison.
which is a clear sign to me that the code didn’t want it to be global.
just for Ruby.
TX
To unsubscribe from this list, please visit:
mail: [email protected]


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

+1 for making transient the default, given the concurrency issues. Also,
you should consider making
“org.jruby.embed.localcontext.scope”=“threadsafe”
the default as well. This is the only way I can get JRuby to work with
high concurrency under JSR223