Reorganizing Website Architecture for HTTP/2 and Beyond

Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved.
Reorganizing Website Architecture
for HTTP/2 and Beyond
DeNA Co., Ltd.
Kazuho Oku

Who am I
n  lead developer of H2O HTTP/2 server
⁃  one of the most sophisticated HTTP/2 impl.
⁃  initial public release: 2014/10 (license: MITL)
⁃  used by Fastly, etc.
n  author of HTTP performance-related I-Ds
⁃  Cache Digests for HTTP/2
⁃  103 Early Hints
n  lives in Japan, works for DeNA Co., Ltd.
Reorganizing Website Architecture for HTTP/2 and Beyond

Current State of HTTP

Why use HTTP/2?
n  latency has become the bottleneck of the Web
n  HTTP/2 to conceal latency by raising concurrency
⁃  6 concurrent requests in HTTP/1
⁃  ~100 in HTTP/2

Current state of HTTP
n  HTTP/2 (RFC 7540) released on May 2015
1: https://github.com/HTTPWorkshop/workshop2016/blob/master/talks/http2-review-data.pdf
45
28
37
41
18
31
0% 20% 40% 60% 80% 100%
2015/7
2016/7
# of transac+ons by Firefox1
HTTP HTTPS (H1) HTTPS (H2)

Achievements
n  header compression (HPACK)
⁃  compression ratio: 90% (at median) according to
Mozilla
n  multiplexing
⁃  the tiled benchmark

So is everything optimal now? Not quite.

Issues
n  errors in prioritization
n  TCP head-of-line blocking
n  ineﬃciency of HTTP/2 push
n  compression ratio of small responses

Prioritization

イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピュー
ターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開
くことができないか、イメージが破損している可能性がありま
す。コンピューターを再起動して再度ファイルを開いてくださ
い。それでも赤い x が表示される場合は、イメージを削除して
挿入してください。
イメージを表示できません。メモリ不足のためにイメージを
開くことができないか、イメージが破損している可能性があ
ります。コンピューターを再起動して再度ファイルを開いて
ください。それでも赤い x が表示される場合は、イメージを
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピュー
ターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示さ
れる場合は、イメージを削除して挿入してください。
The prioritization tree
Root
Leader G
Follower G
weight: 1
HTML
weight: 32
Image
weight: 22
Image
weight: 22
Image
weight: 22
CSS
weight: 32
CSS
weight: 32
n  hybrid approach using weights and chaining
⁃  servers are expected to obey to the priority
speciﬁed by the clients
n  Firefoxʼs prioritization tree is shown below
JS
weight: 32
JS
weight: 32
イメージを表示できません。メモリ不足のためにイメージを
開くことができないか、イメージが破損している可能性があ
ります。コンピューターを再起動して再度ファイルを開いて
ください。それでも赤い x が表示される場合は、イメージを
削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピュータ
ーを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。

The prioritization tree
n  some web browsers fail to specify priority
⁃  Safari, Blink
⁃  older versions of Chrome also had issues
⁃  server-side countermeasures required
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再
度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して
再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イ
メージが破損している可能性があります。コンピューターを再起動して再度ファイルを
開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してく
ださい。
イメージを表示で
きません。メモリ
不足のためにイメ
ージを開くことが
イメージを表示できませ
ん。メモリ不足のために
イメージを開くことがで
きないか、イメージが破
イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージ
が破損している可能性があります。コンピューターを再起動して再度ファイルを開いてくださ
い。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
イメージを表示できません。メモリ不足のためにイメージを開くことができないか
ァイルを開いてください。それでも赤い x が表示される場合は、イメージを削除し
Root
HTML
weight: 16
CSS
weight: 16
JS
weight: 16
Image
weight: 16
Image
weight: 16
Image
weight: 16

HTTP/2 prioritization: the solution
n  bandwidth distribution on server-side:
⁃  use Weighted Fair Queuing (WFQ) or Deﬁcit
Round Robin (DRR)
⁃  some servers do it right:
•  nghttp2 (and Apache) implements WFQ in O(log N)
•  H2O approximates WFQ in O(1)
n  detect dumb clients and fallback to server-driven
prioritization
⁃  H2O reprioritizes CSS, JS for clients that do not
use priority chains

HTTP/2 prioritization: benchmark
n  diﬀerences between the times spent until ﬁrst-paint
(red bar)

TCP head-of-line blocking

Typical sequence of HTTP/2
HTTP/2 200 OK
<!DOCTYPE HTML>
...
<LINK HREF="style.css"...
client server
GET /
GET /style.css
need to switch from sending HTML
to sending CSS at this very moment,
or from sending images to sending
CSS in case the CSS in discovered
late (i.e. hidden assets)
1 RTT
*

n  head-of-line blocking:
⁃  high-priority data blocked by preceding data in
ﬂight
n  TCP head-of-line blocking:
⁃  data in TCP send buﬀer blocks following data of
higher priority

n  typical H2 server writes much more than that can be
sent immediately
⁃  unsent data in TCP send buffer (and TLS buffer)
head-of-line blocks following data
TCP send buffer
CWND
unacked poll threshold
TLS buf.
TLS Records
sent immediately not immediately sent
HTTP/2 frames

TCP head-of-line blocking: the solution
n  write only what can be sent immediately
⁃  obtain CWND and unacked size using TCP_INFO
n  adjust poll threshold to delay write notiﬁcation until
TCP becomes ready to send some data immediately
CWND
unacked poll threshold
TLS Records
sent immediately not immediately sent
HTTP/2 frames
TCP send buﬀer

TCP head-of-line blocking: benchmark 1
n  conditions:
⁃  server in Ireland, client in Tokyo (RTT 250ms)
⁃  load tiny js at the top of a large HTML
n  result: delay decreased from 511ms to 250ms
⁃  i.e. JS fetch latency was 2 RTT, became 1 RTT
•  similar results in other environments

TCP head-of-line blocking: benchmark 2
n  using same data as previous
n  server and client both in Japan (25ms RTT)
0
50
100
150
200
250
300
HTML JS
milliseconds
downloading HTML (and JS within)
RTT ~25ms
master latopt

TCP buffers exist everywhere
n  HTTP/2 serverʼs TCP send buffer
⁃  the negative impact can be avoided by a clever
HTTP/2 server (as shown by H2O)
n  TLS terminator
⁃  better to avoid using, if performance matters
n  mobile networks
⁃  cannot be controlled by a web-site admin
⁃  QUIC would be the final solution

Push

Push
n  positive reports:
⁃  “20-30% speedup on page render time”2
n  negative comments:
⁃  many unnecessary pushes (47% are reset2)
⁃  increased render time in anti-patterns3
⁃  “consider preload instead of push”3
n  what is the cause of these negative views?
2: https://github.com/HTTPWorkshop/workshop2016/blob/master/talks/server-push.pdf
3: https://docs.google.com/document/d/1K0NykTXBbbbTlv60t5MyJvXjqKGsCVNYHyLEXIxYMv0/edit

Itʼs the cached resources
n  server shouldnʼt push resources already cached
⁃  but HTTP/2 doesnʼt provide a way to determine
⁃  pushing same resources multiple times is waste
of bandwidth
n  solution: cache-digests
⁃  now an IETF HTTP WG draft
⁃  https://tools.ietf.org/html/draft-ietf-httpbis-
cache-digest-01

Cache-digests
n  client sends a digest of cached responses along with
an HTTP request
n  server examines the digest, pushes only the
resources that arenʼt cached
n  digest: list of SHA(URL)s stored in Golomb-coded sets (a space-
eﬃcient variant of bloom ﬁlter)
n  size of digest: ~ N*log2(1/P) bits
⁃  N: # of resources
⁃  P: false-positive-rate

Example
n  set false positive rate to: P=1/27
n  calculate SHA256 of two URLs (N=2)
⁃  https://example.com/style.css (SHA256: baf9…)
⁃  https://example.com/script.js (SHA256: 1174…)
n  truncate to log2(N*1/P) bits and sort
⁃  0x11, 0xba
n  emit log2(N), log2(1/P) in 5 bits, then the deltas of
sorted list using Golomb coding (subtracted by 1)
⁃  00001 00111 1 0010001 01 010 1000
N 1/P 0x11 0xa8 (0xba-0x11-1)

Overhead of sending a Cache Digest
n  # of fresh responses in browser cache per domain is
expected to be <~1K
⁃  looking at about:cache
•  *.facebook.com: 790
•  *.google.com: 373
n  HTTP request + cache-digest can be packed into a
single TCP packet
⁃  when setting P to 1/128 or 1/256
•  or, P can be set to a bigger value to conserve space,
or cache-digest could be split into multiple packets
following the HTTP request

Implementation Status
n  three ways to send a cache-digest
⁃  I-D proposes the use of HTTP/2 frame
⁃  header-based scheme is TBD
•  so that websites can implement cache-digest using
ServiceWorker (e.g. cache-digest.js)
1: the listed softwares may have interoperability issues due to implementing diﬀerent versions of the draft
2: approximating the cache state using cookies has accuracy issues
Conveyer Server Implementa+on Client Implementa+on
HTTP/2 frame nghYp2 N/A
HTTP header1 Apache, H2O cache-digest.js
HTTP cookie2 H2O not needed

but even after solving the never-push-cached-
resources issue, push doesnʼt necessarily improve
performance

Benchmark scores
n  ﬁrst-paint does not change
⁃  since the browser needs both HTML and CSS
⁃  push only changes the order between the two
n  less time spent showing a blank page
⁃  since unload happens when HTML arrives

so is push an useless technology? no

Three use-cases of H2 push
n  prioritization
n  push while processing request
n  push from edge

Pushing for prioritization
client server
GET /
GET
/style.css
HTTP/2 200 OK
<html>
<link
rel=style.css
...
HTTP/2 200 OK
body: ...
#title: ...
1.  send CSS, JS ﬁrst
2.  then send HTML
(can be rendered
progressively)
without push
client server
GET /
GET /style.css HTTP/2
HTTP/2 200 OK
body: ...
#title: ...
with push
HTTP/2 200 OK
<html> 
<link
rel=style.css ...

Push while processing request
n  web applications involving DB access, etc.
req.
process request
push-asset
HTML
push-asset
push-asset
push-asset
req.
process request
asset
HTML
asset
asset
asset
req.
450ms (5 RTT + processing =me)
without push with push

Push from edge
n  CDNsʼ use-case
⁃  utilize the conn. while waiting for app. response
req.
push-asset
HTML
push-asset
push-asset
push-asset
client edge server (CDN) app. server
req.
HTML

Impact of push
n  prioritization doesnʼt have much beneﬁt
⁃  as discussed
n  push-while-processing, and push-from-edge are the
real use-cases
⁃  but how to use?

Push while processing
req.
process request
push-asset
HTML
push-asset
push-asset
push-asset
req.
process request
asset
HTML
asset
asset
asset
req.
without push with push
n  One way is to conﬁgure the H2 server
mruby.handler: |
Proc.new do |env|
push_paths = []
if /(/|.html)$/.match(env["PATH_INFO"])
push_paths << "/style.css”
...
end
[399, push_paths.empty? ? {} : {
"link" => push_paths.map{|p|
"<#{p}>; rel=preload”
}.join("n")
}, []]
end
fastcgi.connect: ...

Push while processing
n  or use 103 Early Hints
⁃  application sends 103 then processes the request
GET / HTTP/1.1
Host: example.com
HTTP/1.1 103 Early Hints
Link: </style.css>; rel=preload
HTTP/1.1 200 OK
Content-Type: text/html
Link: </style.css>; rel=preload
<!DOCTYPE HTML>
...
HTTP/2 server app. server
GET /
103 Early Hints
Link: …
200 OK
process request
Web Browser
GET /
200 OK
push

103 Early Hints
n  a new I-D (published last week)
⁃  https://tools.ietf.org/html/draft-kazuho-early-
hints-status-code-00
n  the status code indicates that the server is likely to
send a ﬁnal request with the headers included in the
informational response
⁃  can be used to hint preloading 3rd party assets
(e.g. ads) as well
n  already supported by: nghttp2, H2O
⁃  other server / web-application framework
developers have shown interest

Push from edge
n  at least some CDNs are likely to provide
conﬁguration directives at the edge-server level
req.
push-asset
HTML
push-asset
push-asset
push-asset
client edge server (CDN) app. server
req.
HTML

compression ratio of small responses

File size vs. compression ratio
n  claim: no need to concatenate, since H2 improves
request concurrency
n  reality: small files suffer from low compression ratio
⁃  problem for API as well
note: first N bytes of jquery-3.1.1.js compressed with best quality settings
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
256 1,024 4,096 16,384 65,536 86,709
file size
Compression ra/o of a JavaScript file
gzip
brotli

Two proposals
n  Shared Dictionary Compression over HTTP (SDCH)
⁃  uses a pre-shared compression dictionary
⁃  https://tools.ietf.org/html/draft-lee-sdch-spec-00
n  Compression Dictionaries for HTTP/2
⁃  re-uses the compression context of a previously
transmitted response
⁃  https://tools.ietf.org/html/draft-vkrasnov-h2-
compression-dictionaries-01

Differences between the approaches
n  pre-shared dictionary:
⁃  achieves high compression ratio when the
dictionary is cached by the client (i.e. when
revisiting)
⁃  cannot be used at first visit
n  re-using compression context of another response:
⁃  works effectively at first visit
⁃  easier to use, since no configuration needed
⁃  may have security issues
•  i.e. CRIME attack

Compression ratio of small responses
n  beneﬁcial to concatenate small ﬁles (to ~32KB or
larger) for the time being

Summary

HTTP/2 is not perfect (yet)
n  choice of servers / CDNs causes difference
⁃  be careful when looking at benchmarks
n  avoid using:
⁃  TLS terminators
⁃  hidden assets (or un-hide them w. preload links)
n  use push for either of:
⁃  push-while-processing (103 Early Hints)
⁃  push-from-edge (configure your CDN)
n  continue to concatenate files, at least the tiny ones

Reorganizing Website Architecture for HTTP/2 and Beyond

More Related Content

What's hot (20)

Similar to Reorganizing Website Architecture for HTTP/2 and Beyond (20)

More from Kazuho Oku (20)

Recently uploaded (20)

Reorganizing Website Architecture for HTTP/2 and Beyond