I’m encountering what looks like a fairly textbook deadlock in the core
IronRuby code when creating Classes:
Thread 0:
-> calls RubyContext.GetOrCreateClass(Type) - acquires
RubyContext.ModuleCacheLock
-> function body invokes RubyContext.GetOrCreateClassNoLock(Type)
-> The class does not exist, and there are Clr Mixins present, so
calls
if(clrMixins != null) } using(ClassHierarchyLocker())… - deadlocks
trying to acquire RubyContext._classHierarchyLock
My Ruby code on this thread is calling .to_module on a CLR type
Thread 6:
-> Calls RubyContext.GetQualifiedConstant(RubyScope, ConstantSiteCache,
string[], bool)
-> function body invokes using (context.ClassHierarchyLocker()) -
acquires RubyContext._classHierarchyLock
-> … other intermediate calls …
-> calls RubyContext.GetOrCreateModule(NamespaceTracker) - deadlocks
trying to acquire RubyContext.ModuleCacheLock
This thread is a background thread (pretty sure it’s just a regular CLR
threadpool thread) which is responding to an event fired by an external
application.
Basically I have a .NET library which fires events, and I’m subscribing
to
those events from ruby. The events are normal and boring.
It doesn’t look like any ruby code on this thread has run yet as it is
not
mentioned in the callstack. It appears to be trying to get a ruby module
or class, perhaps so it can begin running some ruby code.
Basically, GetOrCreateClassNoLock is lying to us, and is acquiring a
lock
in spite of being called “NoLock”.
The code is complicated enough however that I’m not sure of the best way
to resolve it, but the standard deadlock-resolving steps suggest one of
these
-
Move the “expandMixins” code outside the “NoLock” method
This is the most sensible option, but I’ve no idea where it could be
moved to. -
Collapse to a single lock (eg: Remove the separate moduleCacheLock
and
just use _classHierarchyLock for everything)
note however there is also a seperate NamespaceCacheLock,
ReferenceTypeInstanceDataLock, ValueTypeInstanceDataLock, and perhaps
others which would also likely need collapsing. -
Enforce a lock order such that you must always acquire
_classHierarchyLock before acquiring the other locks
(if we do this we may as well just collapse the locks as it would be
simpler)
Is there any performance benefit to be gained from having 5 locks, or
would it make more sense to just have a single “TypeSystem” lock ?
Hopefully someone with more knowledge of the IronRuby internals can give
a
better perspective on this.
Thanks a lot, Orion
Appendix:
Here’s the output from SOS which illustrates it: I took a memory dump of
IR running with -D command line parameter.
I built the IronRuby executable off the github master code about 2
months
ago, there are no modifications to the code
0:000> !syncblk
Index SyncBlock MonitorHeld Recursion Owning Thread Info SyncBlock
Owner
112 08ab0780 3 1 08a55e38 148c 6 02608cc8
IronRuby.Runtime.CheckedMonitor
153 08ab0fd4 3 1 004d9d38 1970 0 0260a240
System.Collections.Generic.Dictionary`2[[System.Type,
mscorlib],[IronRuby.Builtins.RubyModule, IronRuby]]
Callstack for thread 0:
0:000> !clrstack
OS Thread Id: 0x1970 (0)
Child SP IP Call Site
001fbd4c 77e200ed [GCFrame: 001fbd4c]
001fbed8 77e200ed [GCFrame: 001fbed8]
001fbef4 77e200ed [HelperMethodFrame: 001fbef4]
System.Threading.Monitor.ReliableEnter(System.Object, Boolean ByRef)
001fbf44 6fcde0c4 System.Threading.Monitor.Enter(System.Object, Boolean
ByRef)
001fbf54 5ea0b8ff IronRuby.Runtime.CheckedMonitor.Enter(Boolean ByRef)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\CheckedMonitor.cs
@
32]
001fbf84 5e9f242b
IronRuby.Runtime.RubyContext.GetOrCreateClassNoLock(System.Type)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyContext.cs @
876]
001fbfd4 5e9f2c40
IronRuby.Runtime.RubyContext.GetOrCreateClass(System.Type)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyContext.cs @
815]
001fc008 5e9f0dea IronRuby.Runtime.RubyContext.GetModule(System.Type)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyContext.cs @
1355]
001fc018 0a3ad40d
DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, IronRuby.Runtime.RubyScope,
System.Object)
001fc038 099266c7
DynamicClass.ℑℜ:wrap_method:133:1688:D:/Dev/TestFramework/lib/com_method_gen.rb(System.Runtime.CompilerServices.Closure,
IronRuby.Runtime.BlockParam, System.Object, System.Object)
001fc078 5d5dd25f
Microsoft.Scripting.Interpreter.LightLambda.Run3[[System.__Canon,
mscorlib],[System.__Canon, mscorlib],[System.__Canon,
mscorlib],[System.__Canon, mscorlib]](System.__Canon, System.__Canon,
System.__Canon)
001fc0b4 5eaa1886
IronRuby.Runtime.Calls.BlockDispatcher1.Invoke(IronRuby.Runtime.BlockParam,
System.Object, IronRuby.Builtins.Proc, System.Object)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\Calls\BlockDispatchers.cs
@ 133]
001fc0c8 5ea2067d IronRuby.Runtime.RubyOps.Yield1(System.Object,
IronRuby.Builtins.Proc, System.Object, IronRuby.Runtime.BlockParam)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 336]
001fc100 5ea601e6 IronRuby.Runtime.BlockParam.Yield(System.Object,
System.Object ByRef)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\BlockParam.cs @
205]
001fc110 5ecfe2a1
IronRuby.Builtins.Enumerable+<>c__DisplayClassa.b__6(IronRuby.Runtime.BlockParam,
System.Object, System.Object)
[c:\Dev\ironlanguages-main\Languages\Ruby\Libraries\Builtins\Enumerable.cs
@ 117]
001fc12c 5eaa1886
IronRuby.Runtime.Calls.BlockDispatcher1.Invoke(IronRuby.Runtime.BlockParam,
System.Object, IronRuby.Builtins.Proc, System.Object)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\Calls\BlockDispatchers.cs
@ 133]
001fc140 5ea2067d IronRuby.Runtime.RubyOps.Yield1(System.Object,
IronRuby.Builtins.Proc, System.Object, IronRuby.Runtime.BlockParam)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 336]
001fc178 5ecea6eb
IronRuby.Builtins.IListOps.Each(IronRuby.Runtime.BlockParam,
System.Collections.IList)
[c:\Dev\ironlanguages-main\Languages\Ruby\Libraries\Extensions\IListOps.cs
@ 885]
001fc190 05afe35f
DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure,
System.Runtime.CompilerServices.CallSite, System.Object,
IronRuby.Builtins.Proc)
001fc1c8 5ecf2750
IronRuby.Builtins.Enumerable.Each(IronRuby.Runtime.CallSiteStorage1<System.Func4<System.Runtime.CompilerServices.CallSite,System.Object,IronRuby.Builtins.Proc,System.Object>>,
System.Object, IronRuby.Builtins.Proc)
[c:\Dev\ironlanguages-main\Languages\Ruby\Libraries\Builtins\Enumerable.cs
@ 37]
… elided …
001fef74 5d61bb5c
Microsoft.Scripting.Hosting.Shell.CommandLine.RunFile(Microsoft.Scripting.Hosting.ScriptSource)
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\CommandLine.cs
@ 182]
001fefa4 5eaa641d
IronRuby.Hosting.RubyCommandLine.RunFile(System.String)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Hosting\RubyCommandLine.cs
@ 69]
001fefb8 5d61bc9c Microsoft.Scripting.Hosting.Shell.CommandLine.Run()
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\CommandLine.cs
@ 144]
001fefc4 5d61bd4a
Microsoft.Scripting.Hosting.Shell.CommandLine.Run(Microsoft.Scripting.Hosting.ScriptEngine,
Microsoft.Scripting.Hosting.Shell.IConsole,
Microsoft.Scripting.Hosting.Shell.ConsoleOptions)
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\CommandLine.cs
@ 112]
001ff004 5d61d1da
Microsoft.Scripting.Hosting.Shell.ConsoleHost.RunCommandLine()
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs
@ 402]
001ff040 5d61d577
Microsoft.Scripting.Hosting.Shell.ConsoleHost.ExecuteInternal()
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs
@ 337]
001ff04c 5d61d987
Microsoft.Scripting.Hosting.Shell.ConsoleHost.Execute()
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs
@ 316]
001ff05c 5d61c9da
Microsoft.Scripting.Hosting.Shell.ConsoleHost.Run(System.String[])
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Hosting\Shell\ConsoleHost.cs
@ 213]
001ff09c 6eb253e5 Host.Main(System.String[])*** WARNING: Unable to
verify
checksum for ir.ni.exe
[c:\Dev\ironlanguages-main\Languages\Ruby\Console\Program.cs @ 80]
Callstack for thread 6:
0:006> !clrstack
OS Thread Id: 0x148c (6)
Child SP IP Call Site
0955e490 77e200ed [GCFrame: 0955e490]
0955e5a8 77e200ed [GCFrame: 0955e5a8]
0955e5c4 77e200ed [HelperMethodFrame_1OBJ: 0955e5c4]
System.Threading.Monitor.ReliableEnter(System.Object, Boolean ByRef)
0955e61c 6fcde0c4 System.Threading.Monitor.Enter(System.Object, Boolean
ByRef)*** WARNING: Unable to verify checksum for mscorlib.ni.dll
0955e62c 5e9f2f47
IronRuby.Runtime.RubyContext.GetOrCreateModule(Microsoft.Scripting.Actions.NamespaceTracker)***
WARNING: Unable to verify checksum for IronRuby.ni.dll
0955e660 5e9f011d
IronRuby.Runtime.RubyContext.TrackerToModule(System.Object)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyContext.cs @
1661]
0955e670 5ea15b5e
IronRuby.Builtins.RubyModule.TryGetConstantNoAutoloadCheck(System.String,
IronRuby.Runtime.ConstantStorage ByRef)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @
1132]
0955e698 5ea563a0
IronRuby.Builtins.RubyModule+<>c__DisplayClass6.b__5(IronRuby.Builtins.RubyModule)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @
1102]
0955e6a4 5ea15096
IronRuby.Builtins.RubyModule.ForEachDeclaredAncestor(System.Func2<IronRuby.Builtins.RubyModule,Boolean>) [c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @ 876] 0955e6b8 5ea18026 IronRuby.Builtins.RubyClass.ForEachAncestor(System.Func2<IronRuby.Builtins.RubyModule,Boolean>)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyClass.cs @
622]
0955e6c8 5ea15a6d
IronRuby.Builtins.RubyModule.TryResolveConstantNoAutoloadCheck(Boolean,
System.String, IronRuby.Runtime.ConstantStorage ByRef)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @
1102]
0955e6e8 5ea15967
IronRuby.Builtins.RubyModule.TryLookupConstantNoLock(Boolean, Boolean,
IronRuby.Runtime.RubyGlobalScope, System.String,
IronRuby.Runtime.ConstantStorage ByRef)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Builtins\RubyModule.cs @
1063]
0955e71c 5ea49915
IronRuby.Runtime.RubyScope.TryResolveConstantNoLock(IronRuby.Runtime.RubyGlobalScope,
System.String, IronRuby.Runtime.ConstantStorage ByRef)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyScope.cs @
530]
0955e740 5ea1edd1
IronRuby.Runtime.RubyOps.ResolveQualifiedConstant(IronRuby.Runtime.RubyScope,
System.String[], IronRuby.Builtins.RubyModule, Boolean,
IronRuby.Runtime.ConstantStorage ByRef, Boolean ByRef)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 997]
0955e7b0 5ea1f711
IronRuby.Runtime.RubyOps.GetQualifiedConstant(IronRuby.Runtime.RubyScope,
IronRuby.Runtime.ConstantSiteCache, System.String[], Boolean)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 811]
0955e808 5eaccd74
Microsoft.Scripting.Interpreter.FuncCallInstruction`5[[System.__Canon,
mscorlib],[System.__Canon, mscorlib],[System.__Canon,
mscorlib],[System.Boolean, mscorlib],[System.__Canon,
mscorlib]].Run(Microsoft.Scripting.Interpreter.InterpretedFrame)***
WARNING: Unable to verify checksum for Microsoft.Dynamic.ni.dll
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Instructions\CallInstruction.Generated.cs
@ 764]
0955e840 5d5ae299
Microsoft.Scripting.Interpreter.Interpreter.Run(Microsoft.Scripting.Interpreter.InterpretedFrame)
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Interpreter.cs
@ 126]
0955e870 5d5dd2c6
Microsoft.Scripting.Interpreter.LightLambda.Run3[[System.__Canon,
mscorlib],[System.__Canon, mscorlib],[System.__Canon,
mscorlib],[System.__Canon, mscorlib]](System.__Canon, System.__Canon,
System.__Canon)
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\LightLambda.Generated.cs
@ 130]
0955e8ac 5eaa1886
IronRuby.Runtime.Calls.BlockDispatcher1.Invoke(IronRuby.Runtime.BlockParam,
System.Object, IronRuby.Builtins.Proc, System.Object)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\Calls\BlockDispatchers.cs
@ 133]
0955e8c0 5ea2067d IronRuby.Runtime.RubyOps.Yield1(System.Object,
IronRuby.Builtins.Proc, System.Object, IronRuby.Runtime.BlockParam)
[c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 336]
0955e8f8 5ecea6eb
IronRuby.Builtins.IListOps.Each(IronRuby.Runtime.BlockParam,
System.Collections.IList)*** WARNING: Unable to verify checksum for
IronRuby.Libraries.ni.dll
[c:\Dev\ironlanguages-main\Languages\Ruby\Libraries\Extensions\IListOps.cs
@ 885]
0955e910 5d624a86
Microsoft.Scripting.Interpreter.FuncCallInstruction`3[[System.__Canon,
mscorlib],[System.__Canon, mscorlib],[System.__Canon,
mscorlib]].Run(Microsoft.Scripting.Interpreter.InterpretedFrame)
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Instructions\CallInstruction.Generated.cs
@ 716]
0955e93c 5d5ae299
Microsoft.Scripting.Interpreter.Interpreter.Run(Microsoft.Scripting.Interpreter.InterpretedFrame)
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Interpreter.cs
@ 126]
0955e96c 5d5dd649
Microsoft.Scripting.Interpreter.LightLambda.Run4[[System.__Canon,
mscorlib],[System.__Canon, mscorlib],[System.__Canon,
mscorlib],[System.__Canon, mscorlib],[System.__Canon,
mscorlib]](System.__Canon, System.__Canon, System.__Canon,
System.__Canon)
[c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\LightLambda.Generated.cs
@ 165]
0955e9ac 6f6ebfc2
System.Dynamic.UpdateDelegates.UpdateAndExecute3[[System.__Canon,
mscorlib],[System.__Canon, mscorlib],[System.__Canon,
mscorlib],[System.__Canon,
mscorlib]](System.Runtime.CompilerServices.CallSite, System.__Canon,
System.__Canon, System.__Canon)*** WARNING: Unable to verify checksum
for
System.Core.ni.dll
0955ea08 5d62d2dd
Microsoft.Scripting.Interpreter.DynamicInstruction4[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Run(Microsoft.Scripting.Interpreter.InterpretedFrame) [c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Instructions\DynamicInstructions.Generated.cs @ 193] 0955ea38 5d5ae299 Microsoft.Scripting.Interpreter.Interpreter.Run(Microsoft.Scripting.Interpreter.InterpretedFrame) [c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\Interpreter.cs @ 126] 0955ea68 5d5dd649 Microsoft.Scripting.Interpreter.LightLambda.Run4[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]](System.__Canon, System.__Canon, System.__Canon, System.__Canon) [c:\Dev\ironlanguages-main\Runtime\Microsoft.Dynamic\Interpreter\LightLambda.Generated.cs @ 165] 0955eaa8 5eaa1aea IronRuby.Runtime.Calls.BlockDispatcher2.Invoke(IronRuby.Runtime.BlockParam, System.Object, IronRuby.Builtins.Proc, System.Object, System.Object) [c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\Calls\BlockDispatchers.cs @ 229] 0955eac0 5ea20590 IronRuby.Runtime.RubyOps.Yield2(System.Object, System.Object, IronRuby.Builtins.Proc, System.Object, IronRuby.Runtime.BlockParam) [c:\Dev\ironlanguages-main\Languages\Ruby\Ruby\Runtime\RubyOps.cs @ 362] 0955eafc 0a092f4a DynamicClass.CallSite.Target(System.Runtime.CompilerServices.Closure, System.Runtime.CompilerServices.CallSite, System.Object, System.Object, MyApplication.SomeEventArgs) 0955eb20 6f6ebfc2 System.Dynamic.UpdateDelegates.UpdateAndExecute3[[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib],[System.__Canon, mscorlib]](System.Runtime.CompilerServices.CallSite, System.__Canon, System.__Canon, System.__Canon) 0955eb7c 05afe257 DynamicClass._Scripting_(System.Object[], System.Object, MyApplication.SomeEventArgs) 0955eb90 098df5fd MyApplication.SomeClass.RaiseEvents(Boolean, System.Collections.Generic.IEnumerable1<MyApplication.DataStruct>)
0955ebd8 098df45f MyApplication.SomeClass.OnEvent(UInt32, Boolean,
Boolean, UInt32, MyApplication.DataStruct[])
0955ec28 098df3c9 DomainBoundILStubClass.IL_STUB_COMtoCLR(Int32, Int32,
Int32, Int32, IntPtr)