From: "felix.wolfsteller@... (Felix Wolfsteller) via ruby-core" Date: 2023-04-28T09:04:15+00:00 Subject: [ruby-core:113372] [Ruby master Bug#19621] Resolv::Hosts uses ineffective File.read, making using big hosts file 'impossible' Issue #19621 has been reported by felix.wolfsteller@betterplace.org (Felix Wolfsteller). ---------------------------------------- Bug #19621: Resolv::Hosts uses ineffective File.read, making using big hosts file 'impossible' https://bugs.ruby-lang.org/issues/19621 * Author: felix.wolfsteller@betterplace.org (Felix Wolfsteller) * Status: Open * Priority: Normal * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- By default, `Resolv` will read `/etc/hosts` once. Privacy- and security aware people might use the file to prevent unwanted traffic, developers use it to quickly manipulate address resolution. `Resolv::Hosts` uses [`IO.read`](https://github.com/betterplace/ruby/blob/9b07d30df8c6bf65c2558c023fd6452405915610/lib/resolv.rb#LL195C4-L195C4), which seems to be inefficient when dealing with large amounts of data that should be consumed by line. E.g. if you install the `/etc/hosts` additions by [hblock](https://hblock.molinero.dev/hosts) (https://github.com/hectorm/hblock), the first call to resolve an address will likely take **minutes**. We believe the solution is easy: Use streaming `IO.foreach` (see patch and PR attached). Benchmarking with examplary /etc/host from xyz done like this ``` require 'resolv' require 'benchmark' Benchmark.measure do Resolv::Hosts.new.lazy_initialize end ``` With `read`: ... With `foreach`: ... -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/