JRuby org.yecht.Emitter.scanScalar(Emitter.java:446) error and how to prevent jruby from completely

aris · December 1, 2012, 8:34pm

Hi,

I’m looking for workaround for an interesting problem I have. Or some
explanation why this can happen so I can think about workaround.

For certain data (I’m still unable to figure out what is offending
part),
calling YAML.dump in jruby is crashing jruby process with
the following stack which start with “*
org.yecht.Emitter.scanScalar(Emitter.java:446)*” (full stack is
attached).
The offending data is just a simple json object - relatively big.

The problem is that I this error crashes the entire process - even
putting begin/rescue does not help.

So I have two questions:

Is there some way to protect my jruby process
from completely crashing?
What could be causing the error like this in YAML.dump?
(Unfortunately, the data which causes this problem is client’s data
so I
cannot share with it - I’m still trying to figure out what is
offending
part).

Thanks,
KB

P.S. Here is the stack trace:

ERROR: 209

org.yecht.Emitter.scanScalar(Emitter.java:446)
org.yecht.Emitter.emitScalar(Emitter.java:503)
org.yecht.ruby.RubyEmitterHandler.handle(RubyEmitterHandler.java:42)
org.yecht.Emitter.emit(Emitter.java:280)
org.yecht.Emitter.emitItem(Emitter.java:949)
org.yecht.ruby.RubyEmitterHandler.handle(RubyEmitterHandler.java:28)
org.yecht.Emitter.emit(Emitter.java:280)
org.yecht.Emitter.emitItem(Emitter.java:949)
org.yecht.ruby.RubyEmitterHandler.handle(RubyEmitterHandler.java:36)
org.yecht.Emitter.emit(Emitter.java:280)
org.yecht.Emitter.emitItem(Emitter.java:949)
org.yecht.ruby.RubyEmitterHandler.handle(RubyEmitterHandler.java:28)
org.yecht.Emitter.emit(Emitter.java:280)
org.yecht.Emitter.emitItem(Emitter.java:949)
org.yecht.ruby.RubyEmitterHandler.handle(RubyEmitterHandler.java:28)
org.yecht.Emitter.emit(Emitter.java:280)
org.yecht.ruby.YEmitter.emit(YEmitter.java:115)
org.yecht.ruby.YEmitter$s$0$1$emit_DBG.call(YEmitter$s$0$1$emit_DBG.gen:65535)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:211)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:176)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:182)
org.jruby.ast.CallOneArgBlockPassNode.interpret(CallOneArgBlockPassNode.java:60)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:255)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:210)
org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:221)
org.jruby.ast.CallTwoArgBlockNode.interpret(CallTwoArgBlockNode.java:62)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:167)
org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:167)
org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
org.jruby.runtime.Block.yieldSpecific(Block.java:99)
org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:142)
org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:153)
org.jruby.ast.FCallNoArgBlockNode.interpret(FCallNoArgBlockNode.java:32)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:201)
org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.ast.IfNode.interpret(IfNode.java:117)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.ast.IfNode.interpret(IfNode.java:117)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:167)
org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:120)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:134)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:69)
org.jruby.ast.FCallManyArgsNode.interpret(FCallManyArgsNode.java:60)
org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.ast.IfNode.interpret(IfNode.java:117)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:201)
org.jruby.ast.FCallTwoArgNode.interpret(FCallTwoArgNode.java:38)
org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.IfNode.interpret(IfNode.java:117)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:203)
org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
org.jruby.runtime.Block.yieldSpecific(Block.java:99)
org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:212)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:176)
org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.ast.IfNode.interpret(IfNode.java:119)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:201)
org.jruby.ast.FCallTwoArgNode.interpret(FCallTwoArgNode.java:38)
org.jruby.ast.DAsgnNode.interpret(DAsgnNode.java:110)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
org.jruby.runtime.Block.yieldSpecific(Block.java:99)
org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
org.jruby.ast.DAsgnNode.interpret(DAsgnNode.java:110)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
org.jruby.ast.EnsureNode.interpret(EnsureNode.java:96)
org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:328)
org.jruby.runtime.BlockBody.call(BlockBody.java:73)
org.jruby.runtime.Block.call(Block.java:89)
org.jruby.RubyProc.call(RubyProc.java:270)
org.jruby.RubyProc.call(RubyProc.java:224)
org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:95)
java.lang.Thread.run(Thread.java:722)

Kristy_Branko · December 1, 2012, 9:12pm

Thanks!

How can I force to use Psych YAML parser for this case? I’m using jruby
1.6.8 and it is somehow calling Yecht when I do “bla”.to_yaml
There might some gem which needs Yecht but for particular case it should
be
ok to use Psych parser.

Any suggestion?

t_7$ jruby -v
jruby 1.6.8 (ruby-1.8.7-p357) (2012-09-18 1772b40) (Java HotSpot™
64-Bit
Server VM 1.7.0_09) [linux-amd64-java]

Thanks,
KB

Kristy_Branko · December 2, 2012, 12:41am

Hmm. I might have to take that back. In order to use Psych, your
application needs to run in the 1.9 mode:

$ jruby -v --1.9 -ryaml -e ‘p YAML.parser’
jruby 1.6.8 (ruby-1.9.2-p312) (2012-09-18 1772b40) (Java HotSpot™
64-Bit Server VM 1.7.0_09) [darwin-x86_64-java]
#<Psych::Parser:0x54f21f54 @handler=#<Psych::TreeBuilder:0x4286ae77
@root=nil, @last=nil, @stack=[]>>

Kristy_Branko · December 2, 2012, 6:59am

That looks like a long stack trace. Hiro, do you think it might be
something as simple as running out of stack space, and increasing the
stack size might help?

Keith

Keith R. Bennett

Kristy_Branko · December 1, 2012, 8:49pm

Hmm. Yecht. That might be a problem #1. Is it possible to move to Psych
as the YAML parser instead? I do not believe Yecht[1] is maintained any
longer.

While it is almost impossible to diagnose this problem remotely, I
suspect that there is an array element violation on the line indicated.
If you are not using JRuby 1.7.0[2], I believe you can rescue Java
exception. You might want to try that.

[1] GitHub - olabini/yecht: Yecht is a YAML processor based on Syck - it is 1.0 semi compatible.
[2] This will be fixed in 1.7.1
http://jira.codehaus.org/browse/JRUBY-6978

Kristy_Branko · December 2, 2012, 7:27am

I tend to think not. If we run out of stack, then we should get an
Exception indicating that instead.

My guess is that AIOOBE is thrown here:

github.com

olabini/yecht/blob/master/src/main/org/yecht/Emitter.java#L446


      
          
          
for(int i=0; i<len; i++) {
              int ci = (int)(cursorb[cursor+i]&0xFF);
              if(! ( ci == 0x9 ||
                     ci == 0xA ||
                     ci == 0xD ||
                     (ci >= 0x20 && ci <= 0x7E))) {
                  flags |= SCAN_NONPRINT;
              } else if(ci == '\n') {
                  flags |= SCAN_NEWLINE;
                  if(len - i >= 3 && cursorb[cursor+i+1] == '-' && cursorb[cursor+i+2] == '-' && cursorb[cursor+i+3] == '-' ) {
                      flags |= SCAN_DOCSEP;
                  }
                  if(i+1 < len && (cursorb[cursor+i+1] == ' ' || cursorb[cursor+i+1] == '\t')) {
                      flags |= SCAN_INDENTED;
                  }
                  if(req_width > 0 && (i - start) > req_width) {
                      flags |= SCAN_WIDE;
                  }
                  start = i;
              } else if(ci == '\'') {

This is dependent on input data, and even if we did have data to verify
it, there is little reason to fix Yecht.