From: Martin Bosslet <Martin.Bosslet@...>
Date: 2012-02-18T02:54:40+09:00
Subject: [ruby-core:42717] [ruby-trunk - Feature #5995] calling io_advise_internal() in read_all()


Issue #5995 has been updated by Martin Bosslet.


Motohiro KOSAKI wrote:

> Because of, when we read whole file contents, we only need _one_ read syscall if the file is regular. In other words, current read_all() suck.  It read a few kilo bytes and append them to a string. That's said it create tons realloc. That doesn't make sense. we need fix the root cause.  btw, read_all() also abuse BUFSIZ.

I noticed that, too, lately, when I had to implement something similar. read_all[1] resizes the buffer in a linear fashion. This means a linear number of reallocs - wouldn't it make sense to grow the buffer exponentially by multiplying with a constant factor (1.5 or 2 maybe) instead? That way, we would only have a logarithmic number of reallocs, which would probably already give better performance. It's a bit more wasteful on memory usage, but I assume that's tolerable because in the end we would resize to the exact total size anyway, no?

[1] https://github.com/ruby/ruby/blob/trunk/io.c#L2152 


----------------------------------------
Feature #5995: calling io_advise_internal() in read_all()
https://bugs.ruby-lang.org/issues/5995

Author: Masaki Matsushita
Status: Rejected
Priority: Normal
Assignee: Motohiro KOSAKI
Category: core
Target version: 


=begin
I propose to call io_advise_internal() in read_all().
It will increase performance.

I created a dummy file:
 dd if=/dev/zero of=dummy bs=1M count=100

Then, I ran the following:

 require 'benchmark'
 
 Benchmark.bm do |x|
   x.report do
     f = File.open("dummy") # dummy file(about 100MB )
     f.read
   end
 end

I freed page cache before each test:
 sudo sysctl -w vm.drop_caches=1

results on Ubuntu 11.10(3.0.0-15-server):

r34462:

 user     system      total        real
   0.050000   0.220000   0.270000 (  0.356033)
 
       user     system      total        real
   0.050000   0.190000   0.240000 (  0.332243)

       user     system      total        real
   0.060000   0.210000   0.270000 (  0.347758)

patched ruby:

 user     system      total        real
   0.030000   0.130000   0.160000 (  0.225866)

       user     system      total        real
   0.040000   0.170000   0.210000 (  0.250172)

       user     system      total        real
   0.040000   0.150000   0.190000 (  0.254654)

It shows the patch increases performance.
=end


-- 
http://bugs.ruby-lang.org/