From: Masaki Matsushita <glass.saga@...>
Date: 2012-02-15T12:03:10+09:00
Subject: [ruby-core:42649] [ruby-trunk - Feature #5995] calling io_advise_internal() in read_all()


Issue #5995 has been updated by Masaki Matsushita.


=begin
>In other words, current read_all() suck.  It read a few kilo bytes and append them to a string.

I modified io.c to show how many bytes read_all() reads on each syscall.

io.c:1834

 static long
 io_bufread(char *ptr, long len, rb_io_t *fptr)
 {
    long offset = 0;
    long n = len;
    long c;

    if (READ_DATA_PENDING(fptr) == 0) {
        while (n > 0) {
          again:
            c = rb_read_internal(fptr->fd, ptr+offset, n);
            if (c == 0) break;
            printf("%ld/%ld\n", c, len); /* how many bytes? */
            if (c < 0) {
                if (rb_io_wait_readable(fptr->fd))
                    goto again;
                return -1;
            }
            offset += c;
            if ((n -= c) <= 0) break;
            rb_thread_wait_fd(fptr->fd);
        }
        return len - n;
    }

io.c:2137
 for (;;) {
    READ_CHECK(fptr);
    n = io_fread(str, bytes, siz - bytes, fptr);
    if (n == 0 && bytes == 0) {
        rb_str_set_len(str, 0);
        break;
    }
    bytes += n;
    rb_str_set_len(str, bytes);
    if (cr != ENC_CODERANGE_BROKEN)
        pos += rb_str_coderange_scan_restartable(RSTRING_PTR(str) + pos, RSG_PTR(str) + bytes, enc, &cr);
    if (bytes < siz) break;
    printf("%ld/%ld\n", n, siz); /* how many bytes? */
    siz += BUFSIZ;
    rb_str_modify_expand(str, BUFSIZ);
 }

Then, I ran the test same as [ruby-core:42471] and I got:
 user     system      total        real
 102400000/102400000
 102400000/102400000
 0.020000   0.170000   0.190000 (  0.254729)

It shows current read_all() reads file at a time.
=end
----------------------------------------
Feature #5995: calling io_advise_internal() in read_all()
https://bugs.ruby-lang.org/issues/5995

Author: Masaki Matsushita
Status: Rejected
Priority: Normal
Assignee: Motohiro KOSAKI
Category: core
Target version: 


=begin
I propose to call io_advise_internal() in read_all().
It will increase performance.

I created a dummy file:
 dd if=/dev/zero of=dummy bs=1M count=100

Then, I ran the following:

 require 'benchmark'
 
 Benchmark.bm do |x|
   x.report do
     f = File.open("dummy") # dummy file(about 100MB )
     f.read
   end
 end

I freed page cache before each test:
 sudo sysctl -w vm.drop_caches=1

results on Ubuntu 11.10(3.0.0-15-server):

r34462:

 user     system      total        real
   0.050000   0.220000   0.270000 (  0.356033)
 
       user     system      total        real
   0.050000   0.190000   0.240000 (  0.332243)

       user     system      total        real
   0.060000   0.210000   0.270000 (  0.347758)

patched ruby:

 user     system      total        real
   0.030000   0.130000   0.160000 (  0.225866)

       user     system      total        real
   0.040000   0.170000   0.210000 (  0.250172)

       user     system      total        real
   0.040000   0.150000   0.190000 (  0.254654)

It shows the patch increases performance.
=end


-- 
http://bugs.ruby-lang.org/