Hi ,
I am trying Storm ( GitHub - nathanmarz/storm: Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC, and more ) using RedStorm
(
GitHub - colinsurprenant/redstorm: JRuby on Storm ). The data passed between
tasks need to be serializable but I got exception when passing JRuby
Hashes
around. The error message is like the following:
1461 [Thread-49] ERROR backtype.storm.util - Async loop died!
java.io.NotSerializableException: org.jruby.RubyHash$RubyHashEntry
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
at
java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
at
backtype.storm.serialization.JavaSerialization.serialize(JavaSerialization.java:20)
at
backtype.storm.serialization.JavaSerialization.serialize(JavaSerialization.java:14)
at
backtype.storm.serialization.FieldSerialization.serialize(FieldSerialization.java:26)
at
backtype.storm.serialization.ValuesSerializer.serializeInto(ValuesSerializer.java:50)
at
backtype.storm.serialization.TupleSerializer.serialize(TupleSerializer.java:31)
at
backtype.storm.daemon.worker$fn__3076$exec_fn__860__auto____3077$fn__3137$fn__3138.invoke(worker.clj:189)
at
backtype.storm.daemon.worker$fn__3076$exec_fn__860__auto____3077$fn__3137.invoke(worker.clj:184)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:540)
at backtype.storm.util$async_loop$fn__445.invoke(util.clj:215)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:680)
I also posted this in RedStorm issue tracker (
java.io.NotSerializableException when serialize Hash between tasks · Issue #8 · colinsurprenant/redstorm · GitHub ). Can any one have
experience work with JRuby & Java serialization give some more
suggestions
?
Thanks
Best Regards,
Stone