Skip to content

Commit

Permalink
open-uri: clear string after buffering
Browse files Browse the repository at this point in the history
Since r58846 (in Ruby 2.5), it is safe to clear the string
yielded to Net::HTTPResponse#read_body methods.  This
reduces malloc garbage (anonymous RSS) using the Linux-only
script below:

before:  user     system      total        real
      0.030000   0.250000   0.280000 (  0.280511)
    RssAnon:	   60240 kB

 after:  user     system      total        real
      0.050000   0.223333   0.273333 (  0.273118)
    RssAnon:	    6676 kB

------
  # warning this script requires 1G free space for buffering
require 'open-uri'
require 'socket'
require 'benchmark'

s = TCPServer.new('127.0.0.1', 0)
len = 1024 * 1024 * 1024
buf = ((0..255).map(&:chr).join * 128)
nr = len / buf.size
pid = fork do
  c = s.accept
  c.readpartial(16384).clear
  c.write("HTTP/1.1 200 OK\r\n" \
	  "Content-Length: #{len}\r\n" \
          "Content-Type: application/octet-stream\r\n" \
          "\r\n")
  buf.freeze # speeds up IO#write slightly
  nr.times { c.write(buf) }
  c.close
end

addr = s.addr
open("http://#{addr[3]}:#{addr[1]}/", "rb") do |fp|
  bm = Benchmark.measure do
    while fp.read(16384, buf)
    end
  end
  puts bm
end
puts File.readlines("/proc/#$$/status").grep(/RssAnon/)[0]
Process.waitpid2(pid)
------

* lib/open-uri.rb: clear string yielded by Net::HTTPResponse#read_body
  [ruby-core:84662] [Feature #14320]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61664 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
  • Loading branch information
normal committed Jan 8, 2018
1 parent b02fc0f commit e1dd1fc
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions lib/open-uri.rb
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,7 @@ def OpenURI.open_http(buf, target, proxy, options) # :nodoc:
if options[:progress_proc] && Net::HTTPSuccess === resp
options[:progress_proc].call(buf.size)
end
str.clear
}
}
}
Expand Down

0 comments on commit e1dd1fc

Please sign in to comment.