From: sam.saffron@...
Date: 2017-07-27T16:17:35+00:00
Subject: [ruby-core:82199] [Ruby trunk Bug#13772] Memory leak recycling stacks for threads in 2.4.1

Issue #13772 has been updated by sam.saffron (Sam Saffron).


OK I have a standalone repro here: https://gist.github.com/SamSaffron/f43996d5989cefbfbf69e9557ef13b23 

Also impacts Ruby head

```
i = 0

def parse_param(s)
  unless block_given?
    return enum_for(__method__, s)
  end

  while s[0] == ';'
    s = s[1..-1]
    ends = s.index(';')
    while ends && ends > 0 \
          && (s[0...ends].count('"') -
              s[0...ends].scan('\"').count) % 2 != 0
      ends = s.index(';', ends + 1)
    end
    if ends.nil?
      ends = s.length
    end
    f = s[0...ends]
    yield f.strip
    s = s[ends..-1]
  end
  nil
end

def parse_line(line)
  parts = parse_param(';' + line)
  key = parts.next
  pdict = {}

  begin
    while (p = parts.next)
      i = p.index('=')
      if i
        name = p[0...i].strip.downcase
        value = p[i+1..-1].strip
        if value.length >= 2 && value[0] == '"' && value[-1] == '"'
          value = value[1...-1]
          value = value.gsub('\\\\', '\\').gsub('\\"', '"')
        end
        pdict[name] = value
      end
    end
  rescue StopIteration
  end

  [key, pdict]
end


while i < 10_000
  Thread.new { parse_line("Content-Type:application/json; charset=ISO-8859-1") }.join
  slots = GC.stat[:heap_live_slots]
  _,rss = `ps ax -o pid,rss | grep -E "^[[:space:]]*#{$$}"`.strip.split.map(&:to_i)
  puts "slots #{slots} #{i+=1} threads #{Thread.list.count} rss #{rss}"
end

puts "done #{RUBY_VERSION}"
gets


##
#  Ruby 2.3.3
#  slots 20388, rss 40708

#  Ruby 2.4.1
#  slots 14448, rss 133688
#
#  Ruby latest
#  slots 16071 rss 132968
```

----------------------------------------
Bug #13772: Memory leak recycling stacks for threads in 2.4.1
https://bugs.ruby-lang.org/issues/13772#change-65965

* Author: sam.saffron (Sam Saffron)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 2.4.1
* Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN
----------------------------------------
Per: 

https://github.com/rest-client/rest-client/issues/611

gem install rest-client

```
100000.times.each_slice(32) do |slice|
    slice.map { Thread.new { RestClient.get('echo.jsontest.com/key/value/one/two') } }.each(&:join)
    GC.start(full_mark: true, immediate_sweep: true)
    slots = GC.stat[:heap_live_slots]
    puts "slots #{slots} #{i+=1} threads #{Thread.list.count}"

end
```

Causes unbound memory growth on 2.4.1, this was not the case in earlier versions of Ruby or Ruby master. 

When running through heaptrack and massif-visualizer it looks like all unreclaimed allocations come from thread_recycle_stack

Ruby heaps do not grow during tests, all the growth is around unmanaged memory. 

I feel getting this sorted is urgent, we have seen similar leaks at Discourse. 



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>