Green threads in 1.9.*?

Ruby
41

“Green? I think perhaps this word does not mean what you think it
means.”

Perhaps this discussion would proceed more smoothly if we defined what’s
meant by a “green” thread, or avoided the term. Ruby programmers usually
use the term “green threads” to mean implementations of Ruby’s Thread
class where the scheduling doesn’t use OS support. These threads are
otherwise middle-weight, akin to a Linux pthread. The expected use-case
for Ruby Thread objects would usually have only a smallish number of
objects, often approximating the number of cores in a server. For this
sort of model, moving to an OS supported thread means that multicore
parallelism is easier to achieve.

Rex, it sounds like the “green” threads you are thinking of may be a
flyweight sort of object, where a program might have hundreds or
thousands of entities. Threads of this type aren’t usually supported by
OS, so each language or application may support them, or not. Ruby
Thread instances don’t map well to this usage, independent of whether
they use a “green” internal scheduler or external OS scheduler.

Sent from my iPhone

42

Bill P. wrote in post #1067690:

Ruby programmers usually
use the term “green threads” to mean implementations of Ruby’s Thread
class where the scheduling doesn’t use OS support. These threads are
otherwise middle-weight, akin to a Linux pthread. The expected use-case
for Ruby Thread objects would usually have only a smallish number of
objects, often approximating the number of cores in a server. For this
sort of model, moving to an OS supported thread means that multicore
parallelism is easier to achieve.

Rex, it sounds like the “green” threads you are thinking of may be a
flyweight sort of object, where a program might have hundreds or
thousands of entities.

Very helpful. Indeed, the green threads I was thinking of were a
virtual flyweight sort of object similar to what I’ve used in other
systems. I wasn’t aware that there were different usages of the term.

Thanks for the Rubyist usage of the term, and the clarification. I’m
willing to call the discussion closed and move on.

Regards.

43

On Fri, Jul 6, 2012 at 5:40 AM, rex goxman [email protected] wrote:

Use cases for green threads aren’t controversial, and there is no debate

on it anywhere that I’m aware of… except perhaps here I guess.

You’re right, there is no debate! Pretty much all major virtual machines
have moved from green threads to native threads. There are still systems
that offer M:N userspace “microthreads” to N native thread schedulers,
such
as Erlang and Go, but these are different from green threaded systems
which
eschew use of native threads entirely and attempt to emulate them
entirely
in userspace.

On Thu, Jul 5, 2012 at 11:52 PM, Ryan D. [email protected]
wrote:

Do you HAVE to have the last word? Is that what this is?

There are plenty of use cases for green threads, and YOU know that. The
fact that Rex is not telling you his means nothing. Stop being a jackass.

Actually, I really do believe that green threads are an antiquated
concept
whose time has passed. Problems with thread creation time can be
mitigated
using thread pools, and systems like Disruptor show you can have
extremely
low-latency cross-thread communication with native threads, even when
there’s contention around data structures, by doing the majority of your
synchronization in userspace.

44

I want a return to the happy, friendly ruby-talk of yesterweek.

One small step would be to stop (gently) mocking people for using
well-accepted variants of colloquial idioms. :slight_smile:

I could care less about 1.8 or any other implementation.

Couldn’t. Unless your level of caring is greater than your minimum. Or you can
have negative carativitiy.

http://articles.boston.com/2010-10-24/lifestyle/29303907_1_care-peeves-decades
http://itre.cis.upenn.edu/~myl/languagelog/archives/001182.html

TL;DR: “I could care less” is at least 50 years old, and can be
explained as sarcasm, hyperbole, and/or a long-ago confusion now lost
in history (like “una narancia” became “a norange” became “an orange”)
– not as evidence of the speaker’s idiocy.

That said, rex was being pretty defensive and peevish. Group hug!

45

On Jul 5, 2012, at 14:22, rex goxman wrote:

Eric H. wrote in post #1067586:

Ruby schedules use of the GVL for threads created by the ruby VM.

A Ruby thread may release the GVL to run non-ruby code.

For these answers I am ignoring Windows. You can probably treat “POSIX
thread” as the equivalent userland threads on Windows, but don’t take my
word for it.

What is a ruby thread?

A thread spawned from ruby. Non-ruby threads can be spawned by other
libraries. For example, if you’ve embedded ruby in a game engine the
game engine can have many of its own threads which ask ruby threads to
do work.

A green thread?

Any program that manages thread creation and scheduling for itself
without use of the OS uses green threads.

A kernel thread?

The most literal definition is a thread that runs in, and only in, the
kernel. You can ignore them for most purposes since you don’t have any
direct control over when or how many are created by your process (this
is true for all programs, not just ruby programs). I think that most
frequently, kernel threads are mapped 1:1 to userland threads, but this
may vary between different OSs.

At one point FreeBSD had multiple pthread implementations you could link
in your program. One had 1:1 mapping between kernel threads and
userland threads, one had 1:N mapping (one kernel thread per process)
and one M:N mapping (multiple userland threads per kernel thread).

Userland threads are what userland programs run. The POSIX thread API
(pthread) handles userland threads and takes care of spawning the
threads, allocating stack space for them, etc. Depending upon the
mapping to kernel threads they may be scheduled from userland or from
the kernel. (I think, most frequently, from the kernel.)

A green thread mapped to a kernel thread?

Since a green thread is scheduled from within a program the kernel has
no knowledge of it so they won’t be mapped.

The depiction at the link somewhere above seems to suggest it is a green
thread which is mapped one-to-one to a kernel thread.

Since pthreads typically make some communication with the kernel it
isn’t correct to call them green threads. It’s possible to have a
pthread library that makes no communication with the kernel (a library
similar to FreeBSD’s 1:N thread library could do this), but in practice
I doubt this happens. It’s difficult for a thread library to schedule a
process’ threads on multiple CPUs without some communication with the
kernel.

Some here suggest there is no such thing, just kernel threads. I will assume it
is a kernel thread unless you say otherwise.

I think what they describe as “kernel threads” I call “userland
threads”. The OS often has its own threads to perform OS tasks such as
userland scheduling, handling interrupts, polling for IO from hardware,
etc. I don’t wish to confuse the two.

Digging in further to thread.c, the GVL is not directly part of userland
thread scheduling or ruby thread scheduling, it’s just a lock around the
ruby VM that allows a thread to use the ruby VM. (If a thread calls
methods in the Ruby VM without the GVL it will likely cause corruption
and crashes.) Ruby uses pthread API to switch threads so it’s up to the
OS which threads get run.

Ruby will interrupt threads that have run for “too long” and allow the
OS to schedule another thread. This is all performed through pthread
mutexes and condition variables. (Perhaps I’m missing it, but I’m no
longer seeing a list of threads waiting to run as I recall 1.8 having
which is a requirement for a green thread implementation.)

46

On Sat, Jul 7, 2012 at 12:57 AM, rex goxman [email protected]
wrote:

Tony A. wrote in post #1067715:

You’re right, there is no debate! Pretty much all major virtual machines
have moved from green threads to native threads.

Perhaps it depends on the definition of what is “major.” I won’t get
into that, everybody has their own definition I guess. But all
Smalltalk VMs I am aware of still use green threads. Smalltalk VM is
state of the art and has been developed for decades, so I don’t see how
it couldn’t be considered “major.”

It’s more like Smalltalk VMs used to be state-of-the-art decades ago.
Sun
hired the Anamorphic team (the creators of Strongtalk) to incorporate
their
technology into the JVM. They also hired Cliff Click.

Cliff Click threw away the Strongtalk JIT compiler and rewrote it from
scratch. With the advent of pthreads, which weren’t available at the
time
these Smalltalk VMs you’re referencing were written, green threads were
ditched for native threads. The result was HotSpot, the JVM we know
today,
and it’s slightly more state-of-the-art than any Smalltalk VM. Azul ran
the
JVM on systems with 768 cores and heaps of 512GB or more.

Erlang processes are entirely “green,” in the same way that a green

thread is “green.” Yes, nowadays they can be mapped to ‘real’ threads
running on multiple cores, and that’s a good thing.

Sorry to get pedantic, but the term “green threads” was termed
specifically
to distinguish the threading models of systems that don’t use native
threads from ones which do (i.e. the opposite of “green threaded” is
“native threaded”). As soon as you have parallel OS threads (i.e. on a
multicore computer) managing the scheduling of multiple userspace
threads/“microthreads”, you no longer have a “green threaded” system,
you
have one which is using native threads. These sorts of systems have very
different properties from green threaded systems as there is true
parallelism. Different threads in the system will have different views
of
memory at the same time.

Erlang used to be green threaded, however with the introduction of the
SMP
scheduler in R12B Erlang gained native thread/multicore support and
Erlang
processes began to run in parallel.

You are colluding terms and that is what is making the discussion
difficult.

47

Tony A. wrote in post #1067845:

Erlang used to be green threaded, however with the introduction of the
SMP
scheduler in R12B Erlang gained native thread/multicore support and
Erlang
processes began to run in parallel.

You are colluding terms and that is what is making the discussion
difficult.

I am using the terms I see in the literature, and in the links provided
to you above. I am using the terms in the same way the links provided
use the terms.

I have not seen any literature say that Erlang is no longer green
threaded (er, green processed). In fact I was just reading more stuff
today that referred to Erlang as having green processes. Erlang’s own
docs say that it has green processes.

I am reading Erlang documentation right now which which states that an
Erlang process doesn’t amount to much more than a pointer within the
Erlang VM pointing off to some chunk of code/memory inside the Erlang
VM, which explains why the processes are so lightweight.

Again, it doesn’t say it’s a pointer pointing off to something OUTSIDE
THE VM. Rather, it points to something WITHIN IT.

Again I say I can spawn off several hundred thousand in a second with my
crappy notebook (I can’t do that with pthreads, kernel threads, OS
threads, whatever). Again I say I can have literally millions of the
things running (latest version of Erlang). And while doing this, I can
bring up the OS process viewer, and also view the threads running within
the processes. Guess what? NOTHING CHANGES. There aren’t thousands or
millions of new processes or threads being added to the OS. There’s the
same processes/threads that were there before. In other words, the OS
doesn’t know about any of these new processes I fired off within Erlang,
and it isn’t creating new threads to handle them. But I can bring up a
process viewer within Erlang, and the millions of processes I created
are all right there.

The fact that the Erlang VM can now take advantage of multiple cores and
move these green processes (which are sparked within the VM and live
within the VM) between the cores by moving them between VMs running one
per core (one OS thread per core is the recommended approach) doesn’t
mean the processes aren’t green. Just because they can now actually run
truly in parallel (because they are running on different VMs which are
running on different cores) doesn’t mean they aren’t green.

Can we agree that my Erlang code (compiled to Erlang VM bytecode) has to
be run within an Erlang VM (I’m not compiling it to native code)? If
so, then it means that if some kind of OS or kernel thread was used to
model Erlang processes, an Erlang VM would have to be fired up for each
kernel thread to execute my code. You are talking hundreds of thousands
of VMs being created per second, and millions running on the system. Of
course that doesn’t happen.

Erlang didn’t change the processes. It didn’t change the threading. It
added a scheduler and the infrastructure to allow the green processes
living within the VM to be scheduled across VMs running on other cores.
Erlang’s own docs recommend running only one VM in a single thread per
core, because it says you aren’t going to get any speedup if you add
more threads and more VMs per core.

Sorry to get pedantic, but the term “green threads” was termed
specifically
to distinguish the threading models of systems that don’t use native
threads from ones which do (i.e. the opposite of “green threaded” is
“native threaded”).

All systems have always used native threads at some point - otherwise
the systems couldn’t run at all. At the end of the day, there has
always been and always will be a native thread running somewhere. So
the term “green threaded” doesn’t distinguish between systems which
don’t use native threads from ones which do, because at the end of the
day they all do. The term “green threaded” (or in Erlang’s case, “green
processed”) refers to whether or not and how virtual threads or
processes being created and living in the VM map to ‘real threads’ in
the OS.

Therefore, if you have a system that spawns one ‘real thread’ (call it a
kernel thread, a pthread, whatever) on the OS for each thread that is
spawned in the VM, then it is not green threaded - the threads are OS
threads. If you have a system that spawns no ‘real threads’ on the OS
for any threads created in the VM, you have a green threaded system.

As soon as you have parallel OS threads (i.e. on a
multicore computer) managing the scheduling of multiple userspace
threads/“microthreads”, you no longer have a “green threaded” system,
you
have one which is using native threads.

That’s entirely incorrect, because you could have one thread running on
each core, with a single GREEN-THREADED VM running on each of those
threads (x cores, x threads total, x VMs running total). This is what
Erlang does now. The only thing different now than in the past is that
the VMs will communicate back and forth and schedule their GREEN THREADS
across each other.

48

On Sun, Jul 8, 2012 at 2:19 AM, rex goxman [email protected] wrote:

I am reading Erlang documentation right now which which states that an
Erlang process doesn’t amount to much more than a pointer within the
Erlang VM pointing off to some chunk of code/memory inside the Erlang
VM, which explains why the processes are so lightweight.

Erlang processes have separate heaps and are independently garbage
collected, however within the SMP scheduler they share the same address
space across cores. While they have an affinity for particular
schedulers,
since they operate in a shared address space any scheduler can
potentially
execute them.

Erlang didn’t change the processes. It didn’t change the threading. It
added a scheduler and the infrastructure to allow the green processes
living within the VM to be scheduled across VMs running on other cores.
Erlang’s own docs recommend running only one VM in a single thread per
core, because it says you aren’t going to get any speedup if you add
more threads and more VMs per core.

You’re colluding the word “virtual machine” with schedulers running
within
the virtual machine. If I launch BEAM for example:

$ erl
Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:8:8] [rq:8]
[async-threads:0] [hipe] [kernel-poll:false]

This is a single instance of the Erlang virtual machine. However you
will
note smp:8 and rq:8. This is because this single instance of the Erlang
virtual machine is running 8 scheduler threads, one for each core of my
computer:

Eshell V5.8.5 (abort with ^G)
1> erlang:system_info(schedulers).
8

That’s entirely incorrect, because you could have one thread running on
each core, with a single GREEN-THREADED VM running on each of those
threads (x cores, x threads total, x VMs running total). This is what
Erlang does now. The only thing different now than in the past is that
the VMs will communicate back and forth and schedule their GREEN THREADS
across each other.

Again, there’s only one instance of BEAM, i.e. there is only one virtual
machine, however there are 8 schedulers running inside it. These
schedulers
each have their own thread, however they share an address space and
contend
on things like memory allocation (again, because there’s only one
virtual
machine with a single pool of resources that much be shared among
schedulers).

In the past, when Erlang really did use green threads, it was necessary
to
run a single Erlang VM per CPU core and use distributed Erlang (even in
a
single system) to get multicore parallelism. However, now that Erlang
has
an SMP scheduler, one Erlang VM can run a scheduler thread per CPU core
and
affect multicore parallelism that way.

49

Tony A. wrote in post #1067919:

Again, there’s only one instance of BEAM, i.e. there is only one virtual
machine, however there are 8 schedulers running inside it. These
schedulers
each have their own thread, however they share an address space and
contend
on things like memory allocation (again, because there’s only one
virtual
machine with a single pool of resources that much be shared among
schedulers).

In the past, when Erlang really did use green threads, it was necessary
to
run a single Erlang VM per CPU core and use distributed Erlang (even in
a
single system) to get multicore parallelism. However, now that Erlang
has
an SMP scheduler, one Erlang VM can run a scheduler thread per CPU core
and
affect multicore parallelism that way.

Ah! I’ll take your word for this, and assume my Erlang distribution
isn’t as updated as I thought it was (could have sworn I had the
latest)! Apparently I’ve been doing the older method of “running a
single VM per core” to achieve parallelism, and this is no longer
necessary. Got it.

Now that that’s cleared up, I think we seem to be using different terms
to describe essentially the same thing. For you, it seems no threads
are green anywhere if a VM or scheduler runs a native thread per core to
achieve parallelism. It seems you call the whole shebang “native
threaded.”

For me, I draw a line between those millions of threads living in the
VM, and the one native scheduler thread per core. I call the things in
the VM “green threads,” and the things the scheduler is running “real
threads” or “kernel threads” or “OS threads” or “native threads” or
whatever. Furthermore, if there are what I call “green threads”
anywhere, I say the whole shebang “has green threads” or “is green
threaded,” regardless of whether or not parallelism is achieved by a
scheduler running a kernel thread per core.

If this is true, and we actually agree as to what’s going on (albeit
using different terms), I think we can shake hands and call it a day at
this point. And again, thanks for the updated info.

50

Tony A. wrote in post #1067715:

You’re right, there is no debate! Pretty much all major virtual machines
have moved from green threads to native threads.

Perhaps it depends on the definition of what is “major.” I won’t get
into that, everybody has their own definition I guess. But all
Smalltalk VMs I am aware of still use green threads. Smalltalk VM is
state of the art and has been developed for decades, so I don’t see how
it couldn’t be considered “major.”

From Cincom website:

http://www.cincomsmalltalk.com/userblogs/cincom/blogView?content=what_can_smalltalk_do

“A single Smalltalk process (image) runs in the context of one operating
system process, using a single OS level thread. Within that process, you
can manage an arbitrary number of lightweight (green) threads…”

I have used green threads on Haskell (GHC). Unless something has
changed, it still has them. It also has other threading models, but I
don’t believe they removed green threads in order to have the other
threading models.

Erlang processes are still green.

There are still systems
that offer M:N userspace “microthreads” to N native thread schedulers,
such
as Erlang and Go, but these are different from green threaded systems
which
eschew use of native threads entirely and attempt to emulate them
entirely
in userspace.

Erlang processes are entirely “green,” in the same way that a green
thread is “green.” Yes, nowadays they can be mapped to ‘real’ threads
running on multiple cores, and that’s a good thing. But they are still
green. My crappy notebook can spawn in the hundreds of thousands a
second. Each one takes up a ridiculously small amount of memory. I can
have millions of the things running in just a few lines of code (latest
version of Erlang, btw).

Here’s the wikipedia entry for green threads. It lists the systems I
listed above, and more: Green thread - Wikipedia

I’m not trying to “have the last word” here, just saying that it seem to
me some ‘major’ VMs still use green threads or processes. You are free
to disagree, and if you want the last word, go for it.

51

On Sun, Jul 8, 2012 at 3:19 AM, rex goxman [email protected] wrote:

I am reading Erlang documentation right now which which states that an
Erlang process doesn’t amount to much more than a pointer within the
Erlang VM pointing off to some chunk of code/memory inside the Erlang
VM, which explains why the processes are so lightweight.

And here is, I think, the crux of the cross communication.

It’s pretty trivial to use Ruby Proc objects in exactly this way.
Write yourself a simple ProcScheduler, and you can, in Ruby, have very
lightweight green “processes”. You can spawn a hundred thousand in a
second, and depending on what they are doing, all hundred thousand can
run in the same second.

Create a subclass of the Proc that you call a ScheduleableProc, add a
few methods to control runnable states, and to indicate whether a
given green process is alive or dead, and you can have a scheduler
that lets GC clean up those dead green processes, while the processes
themselves can manipulate whether or not they are runnable, and can
die when no longer needed.

Voila! Very lightweight “processes” that are similar in concept to the
kind of green threads or processes that you continue to refer to.

It’s not really the same thing that Tony is talking about, though.

Kirk H.

52

On Fri, Jul 6, 2012 at 2:40 PM, rex goxman [email protected] wrote:

Robert K. wrote in post #1067663:

So far rex provides a solution he wants
to use but I cannot really see the reasoning behind it.

Sigh. I did not provide a “solution I want to use.” I provided a
CRAPPY EXAMPLE off the top of my head that was advertised as a CRAPPY
EXAMPLE before giving the CRAPPY EXAMPLE.

I am sorry, but you are wrong: I am not referring to that example.
You said you wanted to use green threads right from the start. That’s
a solution to a problem but not a problem.

I do not intend to divulge any real use cases for green threads because
they are intuitively obvious, apparent, easy to find or dream up for
one’s self, and the literature is abundant and easily available. It’s
sort of like being asked to give a use case for a hammer. I’m not here
to educate. If people want education, it’s up to them to get it, not up
to me to provide it.

Actually it’s not that simple. A use case for green threads should
have specific properties that prevent usage of kernel threads or make
it less optimal for the case at hand. I provided one such case (i.e.
platform does not support kernel threads) and I doubt there are so
many others. That’s what I asked for other use cases. Of course
there are tons of use cases for concurrent processing but that’s not
really the point.

But by now I believe the discussion has uncovered that what you really
want is pieces of code with not too much overhead. Kirk gave an
example, Fibers might be another one. Eventually you can view any
object as a “green thread” in the way of that Erlang definition (which
I don’t know) you gave: “pointer within the Erlang VM pointing off to
some chunk of code/memory inside the Erlang VM”.

Regards

robert

53

On 07/09/2012 05:58 AM, Tony A. wrote:

Erlang (even in a single system) to get multicore parallelism.
However, now that Erlang has an SMP scheduler, one Erlang VM can run a
scheduler thread per CPU core and affect multicore parallelism that way.

–
Tony A.

All interesting stuff. This lead me to another interesting read
elaborating on this somewhat:
http://erlang.2086793.n4.nabble.com/Some-facts-about-Erlang-and-SMP-td2108770.html

So that OTP R12B release was running smp mode by default - I wonder how
far back the smp switch was available. I seem to remember running up
rabbits on earlier releases and I guess we must’ve been using that
option (perhaps lshift scripts did that for us?). Sorry, all, for the
continued digression from ‘Ruby T.’.

Sam

54

On Mon, Jul 9, 2012 at 10:32 AM, rex goxman [email protected]
wrote:

accepted that answer, but the point is, I was probably told no by
someone using a different definition than me.

Just as clarification: my definition of a “green thread” includes a
program counter (into code which records current position of execution
for the time the thread lies dormant) and a stack (which records the
call chain). If you do not need to interrupt execution of a task you
do not need these and a simple reference to an object and method or
lambda is sufficient.

Some questions someone can answer which will be helpful:

  1. When a thread is created and used with the Thread class, does a new
    kernel thread or native thread have to be spawned to deal with it?

= 1.9: yes
<1.9: no

  1. Related to the first question, would it be common or the norm to have
    as many kernel threads spawned as threads created with the Thread class?
    What I’m asking is, say you make a loop which creates AND STORES a few
    hundred thousand threads from the Thread class. Have you just created a
    few hundred thousand kernel or native threads, meaning I can go out to
    the OS and see these things in a process or thread viewer? If it helps,
    assume this is either a 1 core processor, or a 2-4 core processor (in
    other words, the number of Threads you create are huge compared to the
    number of cores).

= 1.9: yes, more correctly: at least as many. The runtime or some library might
create native threads of their own.

If the answers are “yes” and “yes” I will say Ruby doesn’t have green
threads any longer. If the answers are “no” and “no” I will say “ah,
Ruby kept the green threads after all.”

Well, but there are other things like Fiber and Continuation which
might be more appropriate in this case. If you do not need to ensure
that execution of a task can be interrupted by another tasks execution
then you can simply use procs / lambdas (i.e. pieces of code to
execute) and throw them in a queue served by a thread pool. That’s
why we keep asking for the real use case you want to solve.

Cheers

robert

55

I believe I have definitive answers now - thanks.

56

Robert K. wrote in post #1067956:

Eventually you can view any
object as a “green thread” in the way of that Erlang definition (which
I don’t know) you gave: “pointer within the Erlang VM pointing off to
some chunk of code/memory inside the Erlang VM”.

What I’m wondering now (and someone who knows is welcome to answer) is
that since we now know I have a totally different definition of “what
constitutes green threads” than some here, does Ruby in fact have green
threads according to my definition? I was told “no” before, and
accepted that answer, but the point is, I was probably told no by
someone using a different definition than me.

Some questions someone can answer which will be helpful:

  1. When a thread is created and used with the Thread class, does a new
    kernel thread or native thread have to be spawned to deal with it?

  2. Related to the first question, would it be common or the norm to have
    as many kernel threads spawned as threads created with the Thread class?
    What I’m asking is, say you make a loop which creates AND STORES a few
    hundred thousand threads from the Thread class. Have you just created a
    few hundred thousand kernel or native threads, meaning I can go out to
    the OS and see these things in a process or thread viewer? If it helps,
    assume this is either a 1 core processor, or a 2-4 core processor (in
    other words, the number of Threads you create are huge compared to the
    number of cores).

If the answers are “yes” and “yes” I will say Ruby doesn’t have green
threads any longer. If the answers are “no” and “no” I will say “ah,
Ruby kept the green threads after all.”