When counting lines in Ruby randomly failed our deployments

1

Recently, we observed the occasional failing deployments only on two of our servers. The failed servers even were closing our regular SSH connection.

tl;dr. str.count($\) was 1.5x faster compared to str.lines.count and didn’t allocate additional memory.

P.S. ruby-forum didn’t allow me to include the blog post with the details. If you’re interested, the it can be found at SerpApi blog.

2

Hi Illia Zub,

The performance between str.count($\) and str.lines.count can indeed vary. However, the issue you encountered with failed deployments could be due to other issues within your Ruby code, such as memory allocations, server resources, or network issues. It may be useful to thoroughly check the server logs during the failure instance.

Also, if it’s becoming a consistent issue you might want to use exception tracking with tools like Sentry or Bugsnag to gain more insight into the failure reasons.

Best,
Bobby the Bot

3

This is the kind of opportunities where laziness and inmutability could provide some advantage.

With laziness, str.lines could return a collection that will not be materialized until really needed.
And with inmutability, we can be safe that that str will not change for the moment we materialize the collection.

Also, we can be safe that that str will not change by the moment we call str.lines.count. Because of this, count could be an specialized method implemented as str.count($\), or the actual counting if the processing of each element happened before, or it’s cached value.

With all this in place, the programmer could have the freedom to say str.lines.count and have the response in an efficient manner: having your cake and eating it too.

4

Thanks @jgomo3,

I may have made some things unclear in the blog. There are multiple threads making requests and processing the HTML. str.lines.count was one of the steps in the HTML processing in the different threads with the possibly different HTML contents.

How would you use laziness and immutability in this particular str.lines.count example?

5

The main idea is that str.lines.count should be a wrapper of str.count($\).

So if str.count($\) is good, the hypothetical lazy version of str.lines.count would be as good too.

The only thing one wins as a programmer is the ability to express str.lines.count without realizing the lines.

But let me clarify:

I’m using your case just as an example of something that explains one of the advantages (a small one in this case) of laziness and immutability, and also, independently of Ruby.

I’m not suggesting you do that, it is just a reflection.

Having said that, let me continue with this reflection.

Another way to think about this: Imagine str.lines returns an object that is Enumerable, response to count with exactly the equivalent of str.count($\) and only when you call each on it is that the memory is allocated. An object of the class Lines explained here:

class String
  alias materialized_lines lines

  class Lines
    include Enumerable

    def initialize(str)
      @str = str.freeze
    end

    def each
      @str.materialized_lines
    end

    def count
      @str.count($\)
    end
  end

  def lines
    Lines.new(self)
  end
end

The expression str.lines.count would be simply a wrapper of str.count($\). (This is hypotetical, this code doesn’t work).