<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.3.4">Jekyll</generator><link href="https://zyhe.github.io/feed.xml" rel="self" type="application/atom+xml" /><link href="https://zyhe.github.io/" rel="alternate" type="text/html" /><updated>2026-11-17T14:06:33+00:00</updated><id>https://zyhe.github.io/feed.xml</id><title type="html">Zhiyu He</title><subtitle>Homepage of Zhiyu He</subtitle><author><name>Zhiyu He</name></author><entry><title type="html">Random Matrix Theory</title><link href="https://zyhe.github.io/2026/09/15/random-matrix-theory.html" rel="alternate" type="text/html" title="Random Matrix Theory" /><published>2026-09-15T08:00:00+00:00</published><updated>2026-09-15T08:00:00+00:00</updated><id>https://zyhe.github.io/2026/09/15/random-matrix-theory</id><content type="html" xml:base="https://zyhe.github.io/2026/09/15/random-matrix-theory.html"><![CDATA[<h1 id="random-matrix-theory">Random Matrix Theory</h1>

<p>Zhiyu He, September 15, 2026</p>

<p>This note is adapted from the following sources:</p>
<ul>
  <li>Tropp, J. A. (2015). <em>An Introduction to Matrix Concentration Inequalities.</em> Foundations and Trends in Machine Learning, 8(1–2), 1–230. <a href="https://doi.org/10.1561/2200000048">doi</a> &amp; <a href="https://arxiv.org/pdf/1501.01571">pdf</a></li>
  <li>Chen, Y. (2020). <em>Matrix Concentration Inequalities.</em> Lecture notes, Princeton University (ELE 520: Mathematics of High-Dimensional Data). <a href="https://yuxinchen2020.github.io/ele520_math_data/lectures/matrix_concentration.pdf">pdf</a></li>
</ul>

<hr />

<h2 id="1-what-is-a-random-matrix-and-why-care">1. What Is a Random Matrix, and Why Care?</h2>

<p>A random matrix is a matrix whose entries are drawn at random — say a symmetric \(\mathbf{X} \in \mathbb{R}^{d \times d}\) with \(X_{ij} = X_{ji} \sim \mathcal{N}(0,1)\) independent and identically distributed. Classical random matrix theory asks what the <em>spectrum</em> of such a matrix looks like, and answers in the <strong>asymptotic</strong> regime when \(d \to \infty\), where the empirical eigenvalue distribution converges to a limiting spectral density.</p>

<p><img src="/img/posts/2026-9-15-random-matrix-theory/fig_random_matrix.png" alt="Random matrix and its limiting spectral density" style="display:block; max-width:90%; height:auto; margin:2rem auto;" /></p>

<p>This tutorial takes a different route. Instead of examining one large matrix, we study a <strong>sum of independent random matrices</strong>,</p>

\[\mathbf{S} = \sum_{k=1}^{n} \mathbf{X}_k \in \mathbb{R}^{d_1 \times d_2}, \qquad (\mathbf{X}_1, \ldots, \mathbf{X}_n) \ \text{independent},\]

<p>and we ask for <strong>finite-sample</strong> (non-asymptotic) guarantees — bounds that hold for <em>every</em> \(n\) and \(d\), not only in the limit.</p>

<p><img src="/img/posts/2026-9-15-random-matrix-theory/fig_independent_sum.png" alt="Sum of independent random matrices" style="display:block; max-width:90%; height:auto; margin:2rem auto;" /></p>

<p>The object of interest is the deviation of \(\mathbf{S}\) from its expectation \(\mathbb{E}[\mathbf{S}]\), measured in the <strong>spectral norm</strong>. The key question is</p>

\[\mathbb{P}\left\{ \|\mathbf{S} - \mathbb{E}[\mathbf{S}]\| \geq t \right\} \leq \ \underline{\quad \textbf{?} \quad}\]

<p>There are three things to take away:</p>

<ul>
  <li><strong>Applications</strong> in statistics and machine learning;</li>
  <li>The <strong>setup and guarantee</strong> of the matrix Bernstein inequality;</li>
  <li>The <strong>proof</strong> for the basic case with a symmetric matrix.</li>
</ul>

<!-- **Warm up.** For a symmetric $$\mathbf{A} \in \mathbb{R}^{2 \times 2}$$ with eigendecomposition $$\mathbf{A} = \mathbf{U} \boldsymbol{\Lambda} \mathbf{U}^*$$,

$$\mathbf{A} = \begin{bmatrix} -1 & 2 \\ 2 & -1 \end{bmatrix}, \qquad
\mathbf{U} = \frac{1}{\sqrt{2}} \begin{bmatrix} 1 & 1 \\ -1 & 1 \end{bmatrix}, \qquad
\boldsymbol{\Lambda} = \begin{bmatrix} -3 & 0 \\ 0 & 1 \end{bmatrix},$$

the spectral norm is $$\|\mathbf{A}\| = \max_i \lvert\lambda_i\rvert = 3$$ — the largest eigenvalue *in magnitude*, not the largest signed one. -->

<hr />

<h2 id="2-the-relevance-of-the-independent-sum-model">2. The Relevance of the Independent Sum Model</h2>

<h3 id="sample-covariance-estimation">Sample covariance estimation</h3>

<p>Let \(\mathbf{x}_1, \ldots, \mathbf{x}_n \in \mathbb{R}^{d}\) be i.i.d., zero mean, with unknown covariance \(\mathbf{C} = \mathbb{E}[\mathbf{x} \mathbf{x}^*]\). The empirical covariance</p>

\[\hat{\mathbf{C}}_n = \frac{1}{n} \sum_{k=1}^{n} \mathbf{x}_k \mathbf{x}_k^*, \qquad \mathbb{E}[\hat{\mathbf{C}}_n] = \mathbf{C},\]

<p>has error \(\hat{\mathbf{C}}_n - \mathbf{C} = \sum_{k=1}^{n} \mathbf{X}_k\) with \(\mathbf{X}_k = \tfrac{1}{n}(\mathbf{x}_k \mathbf{x}_k^* - \mathbf{C})\). This is an independent sum of centered matrices.</p>

<p>The question is <em>how many samples guarantee \(\|\hat{\mathbf{C}}_n - \mathbf{C}\| \leq \varepsilon \|\mathbf{C}\|\)?</em></p>

<h3 id="least-squares-with-random-data">Least squares with random data</h3>

<p>Given i.i.d. data \((\mathbf{a}_k, y_k)_{k=1}^{n}\), the empirical risk and its Hessian are</p>

\[\hat{f}_n(\mathbf{x}) = \frac{1}{2n} \sum_{k=1}^{n} (\mathbf{a}_k^* \mathbf{x} - y_k)^2,
\qquad
\hat{\mathbf{H}}_n \triangleq \nabla^2 \hat{f}_n(\mathbf{x}) = \frac{1}{n} \sum_{k=1}^{n} \mathbf{a}_k \mathbf{a}_k^*.\]

<p>With the population Hessian \(\mathbf{H} = \mathbb{E}[\mathbf{a} \mathbf{a}^*]\) we again get \(\hat{\mathbf{H}}_n - \mathbf{H} = \sum_{k=1}^{n} \mathbf{X}_k\), \(\mathbf{X}_k = \tfrac{1}{n}(\mathbf{a}_k \mathbf{a}_k^* - \mathbf{H})\). Controlling this deviation means the empirical problem inherits the curvature of the population one.</p>

<p><em>How many samples ensure \(\|\hat{\mathbf{H}}_n - \mathbf{H}\| \leq \varepsilon \|\mathbf{H}\|\)?</em></p>

<h3 id="randomized-matrix-multiplication">Randomized matrix multiplication</h3>

<p>Computing \(\mathbf{A} \mathbf{B}\) with \(\mathbf{A} \in \mathbb{R}^{d_1 \times N}\), \(\mathbf{B} \in \mathbb{R}^{N \times d_2}\) costs \(\mathcal{O}(N d_1 d_2)\). This operation can become prohibitive when the inner dimension \(N\) is huge. The column–row view</p>

\[\mathbf{A} \mathbf{B} = \sum_{i=1}^{N} \mathbf{a}_i \mathbf{b}_i^*\]

<p>writes the product as a sum of \(N\) rank-one terms, so we may keep only \(n \ll N\) <strong>sampled</strong> terms.</p>

<p><img src="/img/posts/2026-9-15-random-matrix-theory/fig_colrow.png" alt="Column-row view of the matrix product" style="display:block; max-width:90%; height:auto; margin:2rem auto;" /></p>

<p>Sample indices \(i_1, \ldots, i_n\) i.i.d. with \(\mathbb{P}(i_k = j) = p_j\), then reweight to stay unbiased:</p>

\[\mathbf{M}_{i_k} = \frac{1}{p_{i_k}} \mathbf{a}_{i_k} \mathbf{b}_{i_k}^*, \qquad
\hat{\mathbf{M}}_n = \frac{1}{n} \sum_{k=1}^{n} \mathbf{M}_{i_k}, \qquad
\mathbb{E}[\hat{\mathbf{M}}_n] = \mathbf{A} \mathbf{B}.\]

<p>This is once more an independent sum, now at cost \(\mathcal{O}(n d_1 d_2)\).</p>

<p>The question is again <em>how many samples \(n \ll N\) suffice for a given accuracy?</em></p>

<h3 id="the-common-structure">The common structure</h3>

<p>All three problems share the model \(\mathbf{S} = \sum_{k=1}^{n} \mathbf{X}_k\) with independent summands, and all three ask for \(\|\mathbf{S} - \mathbb{E}[\mathbf{S}]\|\) in spectral norm.</p>

<p>In the scalar case \(d_1 = d_2 = 1\) the answer is the classical Bernstein inequality, whose tail is driven by the variance \(\sum_k \operatorname{Var}(X_k)\); the matrix case needs a matrix analogue of that variance. The similar machinery extends beyond independence, e.g. to matrix martingales with \(\mathbb{E}[\mathbf{X}_{k+1} \mid \mathbf{X}_1, \ldots, \mathbf{X}_k] = \mathbf{X}_k\).</p>

<hr />

<h2 id="3-the-matrix-bernstein-inequality">3. The Matrix Bernstein Inequality</h2>

<p>Let \(\mathbf{X}_1, \ldots, \mathbf{X}_n \in \mathbb{R}^{d \times d}\) be independent and symmetric — they may follow <em>different</em> distributions — with</p>

\[\mathbb{E}[\mathbf{X}_k] = \mathbf{0}, \qquad \|\mathbf{X}_k\| \leq B, \qquad
v \triangleq \Big\|\mathbb{E}\Big[\sum_{k} \mathbf{X}_k^2\Big]\Big\|, \qquad
\mathbf{S} = \sum_{k=1}^{n} \mathbf{X}_k.\]

<blockquote style="background:#f4f7fa; border-left:4px solid #0085A1; padding:1.25rem 1.5rem; border-radius:4px; margin:1rem 0; color:#212529; font-style:normal;">
  <p><strong>Matrix Bernstein (symmetric case).</strong> For all \(t \geq 0\),</p>

\[\begin{align*}
\mathbb{P}\{\lambda_{\max}(\mathbf{S}) &amp;\geq t\} \leq d \exp\left(\frac{-t^2/2}{v + Bt/3}\right), \\
\mathbb{E}\, \lambda_{\max}(\mathbf{S}) &amp;\leq \sqrt{2 v \log d} + \tfrac{1}{3} B \log d.
\end{align*}\]
</blockquote>

<p>Two quantities shape the bound: the <strong>matrix variance statistic</strong> \(v\) and the <strong>problem dimension</strong> \(d\), the latter entering only logarithmically in expectation.</p>

<p>The bound interpolates between two tail behaviors, with the crossover at \(t \approx v/B\):</p>

\[\mathbb{P}\{\lambda_{\max}(\mathbf{S}) \geq t\} \lesssim
\begin{cases}
d \exp\big(-t^2/(2v)\big), &amp; t \ll v/B \quad \text{(Gaussian tail)} \\[.4em]
d \exp\big(-t/B\big),      &amp; t \gg v/B \quad \text{(exponential tail)}
\end{cases}\]

<p><img src="/img/posts/2026-9-15-random-matrix-theory/fig_tails.png" alt="Gaussian core and exponential tails" style="display:block; max-width:90%; height:auto; margin:2rem auto;" /></p>

<p>Small deviations are governed by the variance statistic \(v\); large deviations by the uniform bound \(B\).</p>

<hr />

<h2 id="4-application-randomized-matrix-multiplication">4. Application: Randomized Matrix Multiplication</h2>

<p>Return to \(\hat{\mathbf{M}}_n\) and choose the sampling probabilities proportional to squared norms,</p>

\[p_j = \frac{\|\mathbf{a}_j\|^2 + \|\mathbf{b}_j\|^2}{\|\mathbf{A}\|_{\mathrm{F}}^2 + \|\mathbf{B}\|_{\mathrm{F}}^2}, \quad j = 1, \ldots, N,\]

<p>which costs only \(\mathcal{O}(N(d_1 + d_2))\) to form. Normalize \(\|\mathbf{A}\| = \|\mathbf{B}\| = 1\) and define the <strong>average stable rank</strong></p>

\[\operatorname{asr} \triangleq \tfrac{1}{2}\big(\operatorname{srank}(\mathbf{A}) + \operatorname{srank}(\mathbf{B})\big), \qquad
\operatorname{srank}(\mathbf{X}) = \frac{\|\mathbf{X}\|_{\mathrm{F}}^2}{\|\mathbf{X}\|^2} \leq \operatorname{rank}(\mathbf{X}).\]

<p>The Bernstein parameters then satisfy \(\|\mathbf{M}_j\| \leq \operatorname{asr}\) and \(v(\mathbf{M}_j) \leq 2\operatorname{asr}\), so with \(d \triangleq d_1 + d_2\),</p>

\[\mathbb{E}\,\|\hat{\mathbf{M}}_n - \mathbf{A} \mathbf{B}\| \leq \sqrt{\frac{4 \operatorname{asr} \log d}{n}} + \frac{2 \operatorname{asr} \log d}{3 n}.\]

<p>Consequently \(n \geq \varepsilon^{-2} \cdot \operatorname{asr} \cdot \log d\) samples give \(\mathbb{E}\|\hat{\mathbf{M}}_n - \mathbf{A}\mathbf{B}\| \leq 2\varepsilon + \tfrac{2}{3}\varepsilon^2\). The <strong>stable rank replaces the inner dimension \(N\)</strong>, for a total cost \(\mathcal{O}(\varepsilon^{-2} \operatorname{asr} d_1 d_2 \log d)\).</p>

<h3 id="numerical-experiments">Numerical experiments</h3>

<p>Take \(\mathbf{A} \in \mathbb{R}^{80 \times 4000}\) and \(\mathbf{B} \in \mathbb{R}^{4000 \times 60}\) with \(\|\mathbf{A}\| = \|\mathbf{B}\| = 1\) and \(d = 140\). Here \(\operatorname{srank}(\mathbf{A}) = \operatorname{srank}(\mathbf{B}) = 9.6\), so \(\operatorname{asr} = 9.6 \ll N = 4000\) — the regime where sampling pays off. Each trial draws \(n\) index pairs with \(p_j \propto \|\mathbf{a}_j\|^2 + \|\mathbf{b}_j\|^2\), reweights by \(1/(n p_j)\), and averages.</p>

<p><em>Try it yourself:</em> fill in <code class="language-plaintext highlighter-rouge">rand_matmul(A, B, n, p, rng)</code> and run the code in <a href="https://colab.research.google.com/drive/1ET9YarT---UeYfj2YZ_lUhHkl7rO33Do?usp=sharing">Google Colab</a>.</p>

<p><img src="/img/posts/2026-9-15-random-matrix-theory/error_vs_n.png" alt="Error versus sample size" style="display:block; max-width:90%; height:auto; margin:2rem auto;" /></p>

<p>The estimation error follows the predicted \(n^{-1/2}\) rate; the bound is loose only in the constant.</p>

<p><img src="/img/posts/2026-9-15-random-matrix-theory/error_distribution.png" alt="Distribution of the error over 800 trials" style="display:block; max-width:90%; height:auto; margin:2rem auto;" /></p>

<p>Across 800 trials at \(n = 250\), the error is sharply concentrated — exactly what a matrix concentration inequality promises.</p>

<p><img src="/img/posts/2026-9-15-random-matrix-theory/error_heatmap.png" alt="Heatmap of the true product and its estimate" style="display:block; max-width:90%; height:auto; margin:2rem auto;" /></p>

<p>At \(n = 1000\) the estimate already reproduces \(\mathbf{A}\mathbf{B}\) closely, entry by entry.</p>

<p><img src="/img/posts/2026-9-15-random-matrix-theory/wall_clock_times.png" alt="Accuracy versus wall-clock time" style="display:block; max-width:90%; height:auto; margin:2rem auto;" /></p>

<p>With \(N = 50\,000\), choosing \(n = 228\) (0.5% of the terms) runs about \(18\times\) faster at error \(0.23\).</p>

<hr />

<h2 id="5-proof-ideas">5. Proof Ideas</h2>

<p>The proof follows the scalar Chernoff recipe</p>

\[\mathbb{P}\{S \geq t\} \leq e^{-\theta t} \prod_{k} \mathbb{E}[e^{\theta X_k}], \qquad \forall \theta &gt; 0,\]

<p>but the product rule <strong>fails</strong> for matrices, since \(e^{\mathbf{A} + \mathbf{B}} \neq e^{\mathbf{A}} e^{\mathbf{B}}\) in general. The fix runs in four steps: a trace matrix moment generating function, subadditivity in place of the product rule, a moment bound per summand, and optimization over \(\theta\).</p>

<h3 id="step-1-matrix-laplace-transform">Step 1: Matrix Laplace transform</h3>

\[\mathbb{P}\{\lambda_{\max}(\mathbf{S}) \geq t\} \leq e^{-\theta t}\, \mathbb{E}\operatorname{Tr} e^{\theta \mathbf{S}}, \qquad \forall \theta &gt; 0.\]

<p>The proof is as follows
\(\begin{aligned}
\mathbb{P}\{\lambda_{\max}(\mathbf{S}) \geq t\}
&amp;= \mathbb{P}\{e^{\theta \lambda_{\max}(\mathbf{S})} \geq e^{\theta t}\}
&amp;&amp; x \mapsto e^{\theta x} \text{ increasing} \\
&amp;\leq e^{-\theta t}\, \mathbb{E}\big[e^{\theta \lambda_{\max}(\mathbf{S})}\big]
&amp;&amp; \text{Markov's inequality} \\
&amp;= e^{-\theta t}\, \mathbb{E}\big[e^{\lambda_{\max}(\theta \mathbf{S})}\big]
&amp;&amp; \theta \lambda_{\max}(\mathbf{S}) = \lambda_{\max}(\theta \mathbf{S}) \\
&amp;= e^{-\theta t}\, \mathbb{E}\big[\lambda_{\max}(e^{\theta \mathbf{S}})\big]
&amp;&amp; \text{spectral mapping} \\
&amp;\leq e^{-\theta t}\, \mathbb{E}\big[\operatorname{Tr} e^{\theta \mathbf{S}}\big]
&amp;&amp; \lambda_{\max}(\mathbf{M}) \leq \operatorname{Tr}\mathbf{M},\ \mathbf{M} \succ 0
\end{aligned}\)</p>

<p>Bounding the trace in the other direction, \(\operatorname{Tr}\mathbf{M} \leq d\,\lambda_{\max}(\mathbf{M})\), is where the dimensional factor \(d\) enters the final result.</p>

<h3 id="step-2-subadditivity">Step 2: Subadditivity</h3>

<p><strong>Lieb (1973).</strong> The map \(\mathbf{A} \mapsto \operatorname{Tr}\exp(\mathbf{H} + \log \mathbf{A})\) is <em>concave</em> on positive definite \(\mathbf{A}\).</p>

<p>Combined with Jensen’s inequality, this replaces the failed product rule:</p>

\[\mathbb{E}\big[\operatorname{Tr} e^{\theta \mathbf{S}}\big] = \mathbb{E}\Big[\operatorname{Tr} e^{\theta \sum_{k=1}^n \mathbf{X}_k}\Big] \leq
\operatorname{Tr} \exp\Big(\sum_{k=1}^{n} \log \mathbb{E}\big[e^{\theta \mathbf{X}_k}\big]\Big).\]

<h3 id="step-3-one-summand-at-a-time">Step 3: One summand at a time</h3>

<p>For \(\mathbb{E}[\mathbf{X}] = \mathbf{0}\), \(\|\mathbf{X}\| \leq B\) and \(0 &lt; \theta &lt; 3/B\), a scalar bound plus the spectral theorem give</p>

\[\log \mathbb{E}\big[e^{\theta \mathbf{X}}\big] \preccurlyeq \mathbb{E}\big[e^{\theta \mathbf{X}}\big] - \mathbf{I} \preccurlyeq g(\theta)\, \mathbb{E}[\mathbf{X}^2], \qquad
g(\theta) = \frac{\theta^2/2}{1 - \theta B/3}.\]

<p>Only two features of each summand are used: the uniform bound \(B\) and the second moment \(\mathbb{E}[\mathbf{X}^2]\).</p>

<h3 id="step-4-assemble-and-optimize">Step 4: Assemble and optimize</h3>

\[\mathbb{P}\{\lambda_{\max}(\mathbf{S}) \geq t\} \leq
e^{-\theta t}\operatorname{Tr}\exp\Big(g(\theta) \sum_{k} \mathbb{E}[\mathbf{X}_k^2]\Big) \leq
d\, e^{-\theta t} \exp\big(g(\theta) v\big).\]

<p>Choosing \(\theta_\star = t / (v + Bt/3)\) yields the stated inequality</p>

\[\mathbb{P}\{\lambda_{\max}(\mathbf{S}) \geq t\} \leq d \exp\left(\frac{-t^2/2}{v + Bt/3}\right).\]

<hr />

<h2 id="key-messages">Key Messages</h2>

<ul>
  <li>Matrix concentration gives <strong>non-asymptotic</strong> bounds, valid for every \(n\) and \(d\).</li>
  <li>One independent-sum model covers covariance estimation, empirical Hessians, and randomized matrix multiplication.</li>
  <li>The proof runs through the matrix moment generating function, with Lieb’s theorem replacing the scalar product rule.</li>
</ul>

<p><strong>Further reading.</strong> Tropp, J. A. (2015). <em>An introduction to matrix concentration inequalities.</em> Foundations and Trends in Machine Learning.</p>

<!-- *Source: [Tutorial.tex](../Slides/Tutorial.tex)* -->]]></content><author><name>Zhiyu He</name></author><summary type="html"><![CDATA[Random Matrix Theory]]></summary></entry><entry><title type="html">何必觅闲愁：斯佳丽的幻梦</title><link href="https://zyhe.github.io/2026/05/30/gone-with-the-wind.html" rel="alternate" type="text/html" title="何必觅闲愁：斯佳丽的幻梦" /><published>2026-05-30T08:00:00+00:00</published><updated>2026-05-30T08:00:00+00:00</updated><id>https://zyhe.github.io/2026/05/30/gone-with-the-wind</id><content type="html" xml:base="https://zyhe.github.io/2026/05/30/gone-with-the-wind.html"><![CDATA[<h1 id="何必觅闲愁斯佳丽的幻梦">何必觅闲愁：斯佳丽的幻梦</h1>

<p>何志宇 2026年5月30日</p>

<p>贯穿斯佳丽·奥哈拉从骄纵轻狂的青春年华到坚强隐忍的成长岁月的，是她对艾希礼·威尔克斯精心构筑却徒劳无益的幻梦。</p>

<p>谁不会青睐那位风度翩翩又才华横溢的贵公子呢？他是最理想的南方绅士的代表：家境优渥，气质优雅，一举一动尽合乎礼数。在他俊朗的皮囊下，更有一颗醉心于音乐、艺术和书籍的高雅灵魂。凡此种种，虽不能为青涩、现实而未经世故的斯佳丽所理解，却强有力地攫取住她天性飘摇不定的内心。是啊，没有一位少女不会想与那最值得托付的人同赏花开花谢，藉他宽阔的臂膀共担风雨。</p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2026-5-30-gone-with-the-wind/scarlett-ashley.jpg" alt="Scarlett and Ashley" style="width: 70%; display: block; margin: 0 auto;" />
</div>

<p>即使清楚地明白自己永不会是如母亲艾伦般的大家闺秀，斯佳丽也从未改变她对文质彬彬的南方贵族的倾心。艾希礼承载了她无忧无虑的青春时代的幻想、悸动和情思，成为了长久以来萦绕于她心头的寄托。她自认为对艾希礼的爱是如此深沉而坚贞，甚至使她将其作为一根坚强的情感支柱，以此为倚仗度过了南北战争的兵荒马乱和战后重建时期的步履维艰。</p>

<p>斯佳丽从未真正走进过艾希礼的内心。他们在战前传统的南方社会里的关系，完全是发乎情而止乎礼的。对于艾希礼常常谈论的荣誉、梦想、生活方式和其它各种抽象事物，斯佳丽也未曾、想必也不能清晰理解。作为极少数不为斯佳丽的魅力攻势所动的青年，艾希礼的形象又被添上了一层源自“得不到的永远在骚动”的完美滤镜，使斯佳丽更加看不真切。</p>

<p>如果关于斯佳丽和艾希礼的故事线在南北战争爆发前分道扬镳，那不过是又一个关于一厢情愿而无果而终的言情篇章罢了。假如于艾希礼在战后返乡的那一刻戛然而止，仍不失为一段辗转悱恻的记录，至少青山仍在，佳影依旧。然而，时间拥有缓慢、冷酷却不可阻挡的力量，剥蚀了艾希礼华美的外表，也消解了斯佳丽深沉的情愫。</p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2026-5-30-gone-with-the-wind/tara.jpeg" alt="Tara and Scarlett (i)" style="width: 70%; display: block; margin: 0 auto;" />
</div>

<p>南北战争彻底粉碎了南方原有的社会体系和生活方式，将所有人都置于全新的生存斗争中。绝大多数曾经的南方贵族都变得一贫如洗，各自须寻各自门。随着时间的推移，艾希礼的完美形象出现了令人不安的缝隙，而这也被旁人发现得一清二楚。他缅怀着旧有的悠哉游哉的生活方式，因深知永无失而复得的可能而挣扎不已；他愿意做出改变，却无法适应全新的生活方式和角色。用方丹老奶奶的话说，他“像是个四脚朝天的甲鱼一样，一筹莫展”。这一点在筹集保住塔拉庄园的款项一事上展现得格外分明。</p>

<p>为了避免塔拉被战后临时政府没收，斯佳丽需要在短时间内筹取一笔巨大的款项，以支付高昂税金。她满心期待艾希礼能提供建议或支持，却落得一场空。她最终决定只身前往亚特兰大敌营，希望以情分或肉体说服被关押的白瑞德，以从他那里获得金钱。所有人都清楚这是下下策，于情于理都并不正当。艾希礼虽隐隐明白，仍后知后觉，除了敬配佩斯佳丽无畏的勇气，却只有自责、叹息、沉默和无助。也正是在此刻，斯佳丽第一次隐约地意识到，她想象中那个能承担一切的臂膀，或许连为自己遮风挡雨都做不到。</p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2026-5-30-gone-with-the-wind/tara-2.webp" alt="Tara and Scarlett (ii)" style="width: 70%; display: block; margin: 0 auto;" />
</div>

<p>战后重建时期的生活在不断铺展。无论是斯佳丽和媚兰，或是白瑞德和艾希礼，还是其他形形色色的旧贵族和新来客，都经历着种种探索、改变和浮沉。“世间好物不坚牢，彩云易散琉璃脆”。再晶莹剔透的幻想，再熠熠生辉的珈饰，再一意孤行的坚信，再矢志不渝的深情，都会有烟消云散的时候。在故事的最后，斯佳丽终于醒悟过来。支撑着她走出重重迷雾，成长为今天独立、坚强而富有生命力的自己的，不是与艾希礼暗密的联结，而是媚兰无条件的偏爱和信任，是白瑞德对她持续不断、不容怀疑且值得信赖的情爱。她的自白也让笔者产生了强烈的情感共鸣：</p>

<blockquote>
  <p>“他根本并不真正存在，只是存在于我的想象之中 … 我爱的是我自己虚构的东西，它现在跟媚利一样没有生命。我做了一套漂亮的外衣，我爱上了它。艾希礼骑马走过来，他那么漂亮，那么出众，我把那套外衣穿在他身上，不管对他是不是合身。而且我也不管看到他到底是个什么样子。我始终爱着那套漂亮的外衣——根本没有爱他。</p>
</blockquote>

<blockquote>
  <p>现在她能重新回顾一下多年前的情景。那时她穿着绿花布薄棉衣，站在塔拉的阳光下，为那年轻的骑手，为他的一头光闪闪似头盔的金发而倾倒。现在她能看得很清楚，她那时只不过是一种幼稚的空想，就跟哄杰拉尔德给她买一副蓝宝石耳环的情况差不多。耳环到了手，它的价值也就没有了。任何东西，除了钱以外，只要她一弄到手，马上没多大价值了。如果当初艾希礼跟其他男孩子一样，对她先是满怀激情，继而纠缠不休，为她争风吃醋，郁郁不乐，终而对她苦苦哀求，把自己置于她的掌握之中，而她则可以从拒绝他的求婚中得到满足。倘若是那样的话，她对他的醉心早就会成为过去。只要她身边出现另一个新人，他便会像阳光下的薄雾与微风一样很快就被吹散了。…</p>
</blockquote>

<blockquote>
  <p>半小时以前，她还以为在这世界上，除了钱以外，她已经失去一切，失去她生活中值得留恋的一切东西——埃伦、杰拉尔德、邦尼、嬷嬷、媚利和艾希礼。可是她非得等到失去这一切后才能明白过来，她是爱着白瑞德的——她爱白瑞德，因为他强壮、狂妄、热情、现实，跟她自己一样。”</p>
</blockquote>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2026-5-30-gone-with-the-wind/mist-sunrise.jpg" alt="Mist" style="width: 70%; display: block; margin: 0 auto;" />
</div>

<p>告别了，那曾一往情深的爱恋，那似镜花水月的期待。斯佳丽终于看清，这场纵贯她半生的激荡不已的深情，不过是她为自己所觅得的无端闲愁罢了。华美的外衣破碎，飘忽的美梦终醒。在下一个凄风苦雨的夜晚，别再为伊伤神憔悴；在又一次风和日丽的时节，为真正的知音佳人守候。明天，又是新的一天。</p>

<!-- ---- -->
<!-- 不是所有人都是生来坚强，最大的不同在于对待困境的态度。如果是随波逐流并放弃努力，那再大的初期优势到最后也会微不足道。反之，如果奋力主宰自己的命运，那才会是千金散尽还复来。 -->]]></content><author><name>Zhiyu He</name></author><summary type="html"><![CDATA[何必觅闲愁：斯佳丽的幻梦]]></summary></entry><entry><title type="html">谁主沉浮：东周兴衰札记</title><link href="https://zyhe.github.io/2025/07/19/eastern-zhou.html" rel="alternate" type="text/html" title="谁主沉浮：东周兴衰札记" /><published>2025-07-19T08:00:00+00:00</published><updated>2025-07-19T08:00:00+00:00</updated><id>https://zyhe.github.io/2025/07/19/eastern-zhou</id><content type="html" xml:base="https://zyhe.github.io/2025/07/19/eastern-zhou.html"><![CDATA[<h1 id="谁主沉浮东周兴衰札记">谁主沉浮：东周兴衰札记</h1>

<p>何志宇 2025年7月19日</p>

<p>乱世出英雄，东周是正统秩序土崩瓦解的混乱时代，却也为君相大展雄才和名士纵横捭阖提供了广阔舞台。人们颂扬着春秋五霸的功绩，称道着战国七雄的争锋，也着迷于王侯将相的英雄传说。</p>

<p>我对东周这段历史时期的兴趣始于《战国策》，并受《东周列国志》强化，这些都促使我进一步在《史记》中挖掘纷繁复杂的往事。“以古为镜，可以知兴替”。我们固然能从列国的生命轨迹里总结国家生存、发展和竞争的教训，但在本文中，我更愿意聚焦于历史断面内那些鲜活的个体，梳理得失体悟，以资借鉴。</p>

<h2 id="靡不有始鲜克有终">靡不有始，鲜克有终</h2>

<p>盛衰荣辱是不可抗拒的自然规律。小到个体，大到国家，都会经历萌芽、发展、兴盛和衰败的完整周期。《桃花扇》中的“眼看他起朱楼，眼看他宴宾客，眼看他楼塌了”，很好地刻画了这种急速盛衰的过程。绝大多数人都向往处于顶峰的地位，少数人能凭借努力和机遇真正到达山巅，其中只有更小一部分人才懂得有始有终，居安思危，接受终有一天要体面地告别峰顶的事实。</p>

<p>作为春秋五霸之首，齐桓公的伯业不可谓不煊赫。桓公作为襄公庶子，在与公子纠的继位争夺中惊险胜出。他有足够的度量，能够抛下管仲一箭中钩的深仇，尊管仲为仲父，充分信任他施展治国才干。桓公也广泛任用贤能，推行改革，促使齐国走上国富民强的道路。</p>

<p>桓公“九合诸侯，一匡天下”，风光一时五两。他高举尊王攘夷的旗号，既借匡扶周室来深明大义，凝聚人心，又通过征伐夷狄和扶助弱国来巩固齐国的领导地位。在葵丘之盟中，桓公陶醉于无以复加的崇高声望，以至生发了于泰山封禅的念头。</p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-7-19-eastern-zhou/mountain.jpg" alt="" style="width: 70%; display: block; margin: 0 auto;" />
</div>

<p>然而，月满则亏，水满则溢，霸业之下潜藏着深刻的危机，并在桓公晚年演变成了巨大的灾难。随着贤相管仲和鲍叔牙相继谢世，桓公忘却他们的忠言，转而偏信竖刁、雍巫和开方等佞臣。他也许在想，“我已经如此成功，人生苦短，多享受今天的快乐又有何妨？”因此便志气昏堕，流连于声色犬马，任由五位公子争宠，迟迟未选定下一任国君。等到桓公病重，竖刁和雍巫违逆将桓公封锁在宫中，任其活活饿死。待到群公子混战结束，嗣君终于想到殡殓桓公之时，他却已“身死不葬，虫流出户”。经此一乱，齐国也就此堕入二流国家的行列。可惜桓公英雄一世，却有始无终，令人扼腕。</p>

<p>面对兴亡盛衰的周期律，我们是否真的无能为力，只能接受宿命论的结局？并不尽然。如曹雪芹借秦可卿之口所言，“荣辱自古周而复始，岂人力所能常保的？但如今能于荣时筹画下将来衰时的世业，亦可以常远保全了。”我们要能未雨绸缪，争取挺过严酷的衰败时节，为开启下一个蓬勃发展的窗口期做好准备。</p>

<h2 id="大器晚成大音希声">大器晚成，大音希声</h2>

<p>假如一个人在六十二岁时还漂泊无依，那他或她会如何？</p>

<p>从现代人的视角看，在花甲之年，人的精力已不可与年轻时同日而语，更遑论在物质条件微薄、“人到七十古来稀”的古典时代。或许这样一个人的命运就此盖棺定论？</p>

<p>晋公子重耳便有这番际遇。在他人生的前四十三年，他先是作为公子居住在国都绛城，后奉命在外驻守蒲城。此时他虽谈不上养尊处优，生活却也算波澜不惊。然而，随着骊姬惑乱晋献公剪除异己，他不得不携随从出奔他国。随后因献公薨逝，晋国内部变动频仍，重耳在外持续流亡十九载，先后遍历八国。</p>

<p>在重耳一行经卫国往齐国的路上，他们因饥渴落魄到向田夫乞食的地步。一片低声下气却换来田夫用土器戏谑做食的嘲弄。他们总算到达齐国，并获得晚年齐桓公的礼遇，开始憧憬借助齐国的力量返晋即位。然而天不遂人愿，桓公不久便故去，齐国陷入内乱，回归晋国的希望也烟消云散了。</p>

<p>此前重耳已得桓公许配宗室女齐姜为妻，当下的鸳鸯之乐和未来的重重迷雾，自然而然地使他萌发了长久留在齐国的念头。他说：“人生安乐，孰知其他！必死于此，不能去”。是啊，既然返国即位无望，而当下岁月静好，那为何要愚蠢地抛下这一切呢？</p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-7-19-eastern-zhou/king.png" alt="" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<p>然而，也是重耳有幸被深谋远虑和深明大义的人们环绕，也是上天注定要重耳成就一番非凡的事业。齐姜和重耳的随从赵衰和狐偃商议，趁重耳宿醉时用车马载他出城去往他国。“要成鸿鹄志，生割鸾凤情”。重耳惊觉后也只能接受，踏上了前往列国寻求扶助的道路。</p>

<p>“要为天下奇男子，须历人间万里程。”重耳终于赢得了秦穆公的强有力支持，在六十二岁的年纪彻底结束了流亡生涯，即位成为晋文公。在接下来的九年里，他接过了齐桓公尊王攘夷的大旗，成为春秋时代新一位霸主，奠定了晋国在随后近一个世纪里支配中原事务的根基。</p>

<p>晋文公可谓是大器晚成。他能在花甲之年开启华丽的新篇章，既依赖于他本人的坚韧，也极大受益于他身边贤士的不离不弃和鼎力相助。最忌讳的，便是安于享乐与半途而废！无论是个体还是组织，都必须要有自己的坚定使命和宏大愿景，一以贯之，或早或晚定能实现高远理想。</p>

<!-- ## 说服的艺术：聚焦痛点

范睢说秦昭襄王。
茅焦说秦王政。

秦王政关注天下一统，而非儒家伦理。 -->

<h2 id="快意恩仇一雪前耻">快意恩仇，一雪前耻</h2>

<p>古今中外，人们都倾心于跌宕起伏的复仇故事。无论是莎翁笔下的哈姆雷特，还是大仲马塑造的爱德蒙·唐泰斯，都以其鲜明的个性和曲折的经历牵动着万千读者的心弦。“永别了，善良、人道和感激。永别了，所有使心灵之花绽放的情感！我已经代天主酬报了好人，现在让我代复仇之神去惩罚恶人吧！”基督山伯爵的这一席话让人心潮澎湃，也令他的复仇更显雷霆万钧。</p>

<p>中国文化里没有上帝作为最终的主宰，我们的英雄更愿意秉持世俗公义自行做出最好的裁决。伍子胥伐楚掘墓鞭尸，孙膑马陵计陷庞涓，都是脍炙人口的复仇案例。我想讲述另一则同样跌宕却鲜有提及的故事。</p>

<p>试想一下，一个地位低微的门客，因为莫须有的缘故引起主人猜忌，进而招致相国捶击投厕的羞辱。他九死一生，苟延残喘已是侥幸，又如何能东山再起并洗刷耻辱呢？</p>

<p>上述情形不是小说家的杜撰，而是魏人范睢的真实经历。他受主人须贾忌恨，蒙魏相魏齐羞辱，不得不化名张禄，逃亡秦国以图后策。他腹有良谋，胸怀韬略，抓住机会以远交近攻的国策赢得了秦昭襄王的信任，升为秦国相国，获封应侯。</p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-7-19-eastern-zhou/qin-prime-minister.jpg" alt="" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<p>魏国当然无人知晓，如今位高权重的秦相张禄竟是当年轻于鸿毛的门客范睢。在秦将伐魏之际，须贾奉命出使秦国，期望与秦相言和。范睢明白复仇机会已到，却并没有急于揭开当下的真实身份。反之，他装作是为佣糊口的下人，在与须贾相逢时，察觉他良心尚存，对幸存的范睢有些许哀怜之意。随后，他在丞相府显露真实身份，令须贾心胆俱裂，并在随后的宴会上以“两黥徒夹而马食莝豆”来羞辱他。范睢厉言“为我告魏王，急持魏齐头来！不然者，我且屠大梁”，最终迫使魏齐在穷途末路时自尽。</p>

<p>后人评范睢“一饭之德必偿，睚眦之怨必报”。也许他淋漓尽致的复仇确实有些过度，但他先受尽屈辱后一雪前耻的故事，可谓是更久远的东方版基督山恩仇记。</p>

<p><br /></p>

<p>中国历史源远流长，其独特魅力在于那些熠熠生辉的人物。“太阳底下没有新鲜事”，那些历史人物的荣辱、浮沉和悲欢也同当代人的体验共通。愿我们能汲取足够的智慧，在时代的洪流中掌控自己的生活。</p>]]></content><author><name>Zhiyu He</name></author><summary type="html"><![CDATA[谁主沉浮：东周兴衰札记]]></summary></entry><entry><title type="html">Beauty of Mathematics: Hats, Coins, and Paradoxes</title><link href="https://zyhe.github.io/2025/05/05/puzzles.html" rel="alternate" type="text/html" title="Beauty of Mathematics: Hats, Coins, and Paradoxes" /><published>2025-05-05T09:00:00+00:00</published><updated>2025-05-05T09:00:00+00:00</updated><id>https://zyhe.github.io/2025/05/05/puzzles</id><content type="html" xml:base="https://zyhe.github.io/2025/05/05/puzzles.html"><![CDATA[<h1 id="beauty-of-mathematics-hats-coins-and-paradoxes">Beauty of Mathematics: Hats, Coins, and Paradoxes</h1>

<p>Zhiyu He, May 5, 2025</p>

<p>How can you make a table of engineering researchers, each with diverse interests, all concentrate on a single topic? Ask intriguing mathematical questions.</p>

<p>During our lab retreat in Stoos in April 2025, as the dinner conversation gradually converged to intelligence, I threw some mathematical puzzles that I stumbled upon earlier this year. Surprisingly, they quickly captured everyone’s attention. The next hour solely revolved around proposing various questions, contemplating attacking points, sharing reasoning, and challenging each other.</p>

<p>Let me go through some of these interesting questions to give the reader a glimpse of the beauty of mathematics.</p>

<h2 id="derangement">Derangement</h2>

<p>Here is the famous hat-check problem, which I learned from Klaus.</p>

<blockquote>
  <p>Suppose that $N$ people check their hats into a restaurant. The hat-checker is unfortunately absent-minded and forgets which hat belongs to whom. Instead, the hats are randomly redistributed to people. What is the probability that nobody gets his or her own hat back?</p>
</blockquote>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-5-5-puzzles/hat.avif" alt="Hats" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<p>The number of arrangements for redistributing hats is $n!$. The problem is thus reduced to finding the number of permutations $D(N)$ such that everybody receives a hat belonging to another person.</p>

<p>We apply a recursive approach, akin to dynamic programming in computer science, by breaking into sub-problems. The base cases are easy, and we can show by enumeration that $D(1)=0, D(2)=1$. We proceed to analyze the general case. Let us focus on the first person $P_1$, who collects the hat $h_i$ of person $P_i$, and $i=2,\ldots,N$. We distinguish two cases.</p>

<ul>
  <li>
    <p>First, person $P_i$ happens to grab the hat $h_1$. Then, we exclude persons $P_1$ and $P_i$ from the population and address a reduced problem with $N-2$ people. When $i$ is given, the number of desirable arrangements is $D(N-2)$.</p>
  </li>
  <li>
    <p>Second, person $P_i$ does not receive the hat $h_1$. This scenario, albeit appearing complicated, also allows a similar analysis. Let us exclude person $1$ and consider the remaining population involving persons $2, \ldots, i, \ldots, N$. We need to redistribute the hats $h_1, \ldots, h_{i-1}, h_{i+1}, \ldots, h_N$. Since $h_1$ cannot be assigned to person $P_i$, we relabel this person as $\hat{P}_1$ and conclude that this setting is equivalent to the scenario with $N-1$ people. The number of arrangements for a given $i$ is therefore $D(N-1)$.</p>
  </li>
</ul>

<p>Since there are $N-1$ choices of $i$, we have the following recurrence formula:</p>

\[D(N) = (N-1) * (D(N-1) + D(N-2)),\]

<p>with the base cases $D(1)=0$, $D(2)=1$. A closed-form expression of $D(N)$ can be further derived from the above formula. The probability that nobody receives the original hat is $D(N)/N!$.</p>

<p>The technical word summarizing this problem is derangement, i.e., a permutation of the elements in a set such that no element is allocated to its original position. See the Wikipedia <a href="https://en.wikipedia.org/wiki/Derangement">page</a> and this <a href="https://www.math.emory.edu/~rg/derangements.pdf">note</a> for more details.</p>

<h2 id="make-a-fair-coin-from-a-biased-coin">Make a fair coin from a biased coin</h2>

<p>We have a biased coin, whose probability of coming up with a tail or a head is different. Of course, if we flip the coin sufficiently many times, the number of tails will almost always differ from the number of heads. Now the question is:</p>

<blockquote>
  <p>How to make a “fair coin” out of this biased coin?</p>
</blockquote>

<p>By a “fair coin”, we mean that the probabilities of achieving either of two outcomes equal.</p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-5-5-puzzles/coin.avif" alt="Coin" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<p>The key idea is to exploit pairs of coins to extract fair outcomes. It is clear that flipping this coin only once is hopeless. However, we can make several flips and leverage the collective results.</p>

<p>Without loss of generality, consider discrete random variables $X_1, X_2$ with the following distribution</p>

\[\Pr(X_i = 0) = p, \quad \Pr(X_i = 1) = 1 - p, \quad p\in (0,1),\]

<p>where $X_i$ denotes the variable at the $i$-th flip, and $i=1,2$. Let $Z=f(X_1, X_2)$ be a new random variable, and the mapping $f$ is given by</p>

\[f(01) = 0, ~ f(10) = 1, ~ f(00) = \Lambda, ~ f(11) = \Lambda,\]

<p>where $\Lambda$ represents an empty value. Then,</p>

\[\Pr(Z=0|X_1 \neq X_2) = \frac{\Pr(Z=0, X_1 \neq X_2)}{\Pr(X_1 \neq X_2)} = \frac{p(1-p)}{2p(1-p)} = \frac{1}{2}.\]

<p>Analogously, $\Pr(Z=1|X_1 \neq X_2) = 1/2$. Hence, this new variable $Z$ has equal probabilities of taking the value of $0$ or $1$, thereby mimicking the outcome of a “fair coin”. Intuitively, since 01 and 10 are equally likely, this method filters out the bias.</p>

<p>In a similar vein, we define a mapping from multiple $X_i$ to multiple $Z_j$ to replicate a sequence of fair coin flips. The above idea of integrating multiple random variables is also present in <em>block codes</em> analyzed by information theorists.</p>

<h2 id="st-petersburg-paradox">St. Petersburg paradox</h2>

<p>We are offered an opportunity to play a game of flipping a fair coin. That is, the probability that a head or a tail appears at each round is 0.5. The game terminates whenever a head comes up, and afterward, we win $2^k$ euros, where $k \geq 1$ denotes the total number of times of flipping the coin. For example, if we get a head right the first time, then we obtain $2$ euros and are no longer allowed to continue; if we have a tail and then a head, then we collect $4$ euros; if we obtain two tails and finally a head, then we accumulate $8$ euros.</p>

<blockquote>
  <p>How much money are you willing to pay to participate in this game?</p>
</blockquote>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-5-5-puzzles/coin-2.avif" alt="Coins" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<p>At first glance, the reasonable price for this game seems low. After all, it is highly unlikely that we observe tails for more than $5$ consecutive rounds, translating to a return larger than $64$ euros. What does the math say, though?</p>

<p>The (classical) mathematics suggests examining the expected return. Our return is essentially a discrete random variable, which takes the values of $2, 4, 8, \ldots$ with probabilities of $1/2, 1/4, 1/8, \ldots$. Hence, its expectation is</p>

\[\sum_{k=1}^{\infty} 2^k * \frac{1}{2^k} = 2*\frac{1}{2} + 4*\frac{1}{4} + \ldots = \infty.\]

<p>In other words, the expected return is arbitrarily large. What is wrong with our intuition?</p>

<p>The key reason is that we do not have a precise understanding of the tail probability corresponding to an extreme event. In this problem, the diminishing probability of the extremely lucky case (where we have tails consecutively) is compensated by the exponential growth of the return. This balance contributes to the infinite expected return.</p>

<p>There are many follow-up theories to fix the mismatch between our gut feeling and theoretical analysis. For instance, researchers have developed the expected utility theory and probability weighting to account for our tendency of risk aversion, indicating an objective different from the expected value in decision-making, see also this Wikipedia <a href="https://en.wikipedia.org/wiki/St._Petersburg_paradox#Expected_utility_theory">page</a>.</p>

<h2 id="summary">Summary</h2>

<p>We discussed several intriguing mathematical problems and clarified how they illuminate ideas across statistics, decision science, and information theory. Next time when you sit with engineers and scientists, tell these problems to them!</p>]]></content><author><name>Zhiyu He</name></author><summary type="html"><![CDATA[Beauty of Mathematics: Hats, Coins, and Paradoxes]]></summary></entry><entry><title type="html">情思万端：析李商隐《无题》诗</title><link href="https://zyhe.github.io/2025/02/23/li-shangyin.html" rel="alternate" type="text/html" title="情思万端：析李商隐《无题》诗" /><published>2025-02-23T20:00:00+00:00</published><updated>2025-02-23T20:00:00+00:00</updated><id>https://zyhe.github.io/2025/02/23/li-shangyin</id><content type="html" xml:base="https://zyhe.github.io/2025/02/23/li-shangyin.html"><![CDATA[<h1 id="情思万端析李商隐无题诗">情思万端：析李商隐《无题》诗</h1>

<p>何志宇 2025年2月23日</p>

<p>有唐一代，李商隐因诗精致朦胧而独树一帜。后人评其诗“一唱三叹，余音嫋嫋”，深深着迷于其绚丽的语言、精美的意象和深邃的情感。除却“一篇《锦瑟》解人难”，李商隐也留下了众多缠绵悱恻而寓意飘渺的《无题》诗。“无题”彷佛是他与读者所玩的一种特别的游戏。他像是有意隐藏自己原想表达的确切含义，却深信真正的知音自会参透诗中的奥秘。</p>

<p>从表面上看，这些《无题》诗大多诉说相思之深和无缘之苦。诗歌是内心情感世界的反映，有人因而以李商隐本人的情感境遇为参照来鉴赏。自屈子以降，中国文人都有以男女情事寓君臣遇合的传统，故也有人谈李商隐因诗自托身世，抒宦海浮沉之慨。</p>

<p>我更愿意将《无题》诗视为李商隐对生活中与追求、遗憾和思念有关的普遍情感体验的探讨。上至公子王孙，下至布衣黎庶，都会经历未满、无缘和残缺。这些要素是生活的底色，是艺术美的来源，也是悲剧之所以摄人心魄的根基。李商隐将这些错综复杂的感触生动地呈现在我们的面前，让古往今来的读者们都能捕捉到发生共鸣的音符。</p>

<!-- 如果只读一位诗人的精致朦胧的作品，那一定非李商隐莫属。李商隐留下了众多无题诗，他们都以精致朦胧的意象和缠绵悱恻的情感为特色。 -->

<p><br /></p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-23-li-shangyin/moon-sunset.avif" alt="新月" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<blockquote style="text-align: center;">
来是空言去绝踪，月斜楼上五更钟。<br />
梦为远别啼难唤，书被催成墨未浓。<br />  
蜡照半笼金翡翠，麝熏微度绣芙蓉。<br /> 
刘郎已恨蓬山远，更隔蓬山一万重！<br />  
</blockquote>

<p>君言当归，却是虚言一句，自相别后便杳无音信。只听得报晓之钟于五更响起，残月西照在楼阁屋宇间。这种情思并非无端而起，乃是夜来梦得相会、醒后悲戚寂寞所致。梦中的重逢愈是真实动人，现实中的分隔便愈是无助断肠。情到深处，便急急下笔以寄相思之意，待到书罢却才发现连墨都没有磨浓。</p>

<p>红烛映衬着有金翡翠装饰的绮丽帷帐，麝香隐隐穿度而来，弥漫在华美的芙蓉褥周围。正似“翡翠衾寒谁与共”，此间独缺心上人的踪影。尾联直言相隔邈远，缘会难期，更凸显情意之绵长。</p>

<p>所谓“日有所思，夜游所梦”。在线性叙事结构里，应是先有思念，后有梦境，最后梦醒。诗人却打破了这一既定框架，将迷离的梦境和朦胧的现实交织在一起，并糅合了悠远绵长的情思，赋予了全诗深邃别致的意味。</p>

<!-- 说归来是一句虚言，自从离开后便音信全无。夜来入梦，仍是别离时的凄楚场景。因急于书写千般思绪，书成才发觉墨都没有磨浓。 -->
<!-- 心上人在飘渺无际的所在，更难缘会。 -->

<p><br /></p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-23-li-shangyin/lotus-pond.avif" alt="芙蓉塘" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<blockquote style="text-align: center;">
飒飒东风细雨来，芙蓉塘外有轻雷。<br />  
金蟾啮锁烧香入，玉虎牵丝汲井回。<br />  
贾氏窥帘韩掾少，宓妃留枕魏王才。<br />  
春心莫共花争发，一寸相思一寸灰。<br />  
</blockquote>

<p>人说“春风化雨”，东风携细雨传来了春日的气息，莲花池外有些微的雷音，也似马车辚辚而过。春日既催生着万物蓬勃生长，也拨动了主人公隐秘的情丝。颔联写重帏深闺的幽寂。闭锁的香炉传度来氤氲缭绕的冷香，状似玉虎的辘轳发出汲取井水的声响。“香”谐“相”，“丝”同“思”，暗合思念牵挂之意。</p>

<p>在这样一种春日融融而情思袅袅的氛围里，我们自然在期待主人公会经历何种情感体验。诗人在颈联连用两处典故，好似为主人公的境遇做镜像。上联援引了一则圆满的故事：晋人韩寿仪表堂堂，为权臣贾充征为司空掾，他引得贾充之女爱慕，两人暗通款曲，也获贾充首肯而终成眷属。下联的故事则以无尽的遗憾告终：后人附会曹植倾心于甄妃而不得，他在甄妃离世后收到了其遗物金镂玉枕，因睹物思人而凄楚不堪，后在洛河因景生情而作《洛神赋》。</p>

<p>在诉说了这些缠绵悱恻的故事后，诗人以一联奇句作结。这是主人公的大胆自白，也似诗人在看破纠葛后的谆谆劝诫：不要让情思随春花恣意生长，因为再炽热而深刻的情感，也逃不过化为灰烬的悲情命运。</p>

<p>世间有千般无可奈何，也有万种求而不得。有人会说这些遗憾都是生活的底色，有人会说要主动放下而向前继续。诗人却用“春心莫共花争发”这一似嗔非嗔的言语，将遗憾的情思书写到了极致。</p>

<p><br /></p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-23-li-shangyin/candle.avif" alt="灯烛" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<blockquote style="text-align: center;">
凤尾香罗薄几重，碧文圆顶夜深缝。<br />  
扇裁月魄羞难掩，车走雷声语未通。<br />  
曾是寂寥金烬暗，断无消息石榴红。<br />  
斑骓只系垂杨岸，何处西南待好风？<br />  
</blockquote>

<p>开篇将镜头陈设在闺阁绣户内，写凤尾绫罗绸缎，写碧纹圆顶罗帐，更写那个在深夜忙于缝制的主人公。她心怀好合之意，却是独守空闺。主人公与心上人有过相遇，她曾着忙用团扇遮面，想尽力掩饰内心的娇羞。马车急急驶过，他们终未搭上一言片语。</p>

<p>红烛销尽，长夜漫漫，主人公因思念未眠，也为意中人消得憔悴。直等到春日已尽，石榴花红，怎奈别后仍是杳无音信。他身在何方？他会知道她正为他执着守候吗？但无论如何，她“愿为西南风，长逝入君怀”，总在盼望能有再度相会的时节。</p>

<p><br /></p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-23-li-shangyin/flower.avif" alt="花蕊" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<blockquote style="text-align: center;">
重帷深下莫愁堂，卧后清宵细细长。<br />  
神女生涯原是梦，小姑居处本无郎。<br />  
风波不信菱枝弱，月露谁教桂叶香？<br />  
直道相思了无益，未妨惆怅是清狂。<br />  
</blockquote>

<p>在寂寥清冷的重帷深闺中，主人公难以成眠，正细细回味过去的境遇。也曾幻想拥有如巫山神女般浪漫浓烈的际遇，实际上却是一厢情愿，正如青溪小姑一般独处，怎会拥有这样美好的可能。</p>

<p>身世浮沉，雨打风吹，虽似细弱的菱枝，却愈遭风雨催折。即使如质美的桂叶，也难有月露滋润使之飘香。哪怕相思全然没有益处，也不妨碍一往情深。</p>

<p><br /></p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-23-li-shangyin/light.avif" alt="红檐" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<blockquote style="text-align: center;">
昨夜星辰昨夜风，画楼西畔桂堂东。<br />  
身无彩凤双飞翼，心有灵犀一点通。<br />  
隔座送钩春酒暖，分曹射覆蜡灯红。<br />  
嗟余听鼓应官去，走马兰台类转蓬。<br />  
</blockquote>

<p>诗人将思绪聚焦于昨夜的风物，为叙述添上了一层温情脉脉的滤镜。昨夜星光闪烁，凉风习习，在琼楼玉宇内共同度过的时光成为了回忆中清晰的断面。那里想必是高朋满座，宾主尽欢，但到底发生了怎样的故事？诗人话锋一转，未写宴饮之景，直抒相思之意。由于时空阻隔或规则桎梏，有情人无缘比翼齐飞，却仍心心相印，互有默契。</p>

<p>颈联描摹宴会觥筹交错的喧闹场景。大家猜钩嬉戏，共饮春酒，那手中的藏钩和乘酒的杯盘，一定留有意中人触过的余温。在共玩分组猜物的游戏时，画堂内灯烛萤煌，更映衬着佳人的如花笑靥。《红楼梦》第六十二回也有射覆的雅集，此情此景用于宝玉和黛玉倒也未为不可。诗人在尾联由回忆转至现实。时已五更，将不得不赴秘书省当差，正似身不由己的孤蓬，只好将温存的记忆隐遁，独自消化那离愁别恨。
<!-- 但更重要的，是那个意中人也在座中。虽然他们并不是比翼齐飞的伴侣，却互有默契，好像总在不经意间为彼此留下深情的一瞥。 --></p>

<!-- <br> -->

<!-- > 相见时难别亦难，东风无力百花残。  
> 春蚕到死丝方尽，蜡炬成灰泪始干。  
> 晓镜但愁云鬓改，夜吟应觉月光寒。  
> 蓬山此去无多路，青鸟殷勤为探看。  -->

<p><br /></p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-23-li-shangyin/river.avif" alt="河流" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<p>读罢这些有着千般情思和万种滋味的《无题》诗，我不禁想到了沈从文在《边城》的结尾写下的句子：</p>

<blockquote>
  <p>可是那个在月下唱歌，使翠翠在睡梦里为歌声把灵魂轻轻浮起的年青人，还不曾回到茶峒来。… 这个人也许永远不回来了，也许“明天”回来！</p>
</blockquote>

<p>翠翠和傩送还能否重逢？也许。假如真的重逢，还能保有过去的情谊吗？未必。那就珍藏着回忆，放下郁结的情思，让一切都随风飘远。</p>]]></content><author><name>Zhiyu He</name></author><summary type="html"><![CDATA[情思万端：析李商隐《无题》诗]]></summary></entry><entry><title type="html">西游之悟：灵山只在汝心头</title><link href="https://zyhe.github.io/2025/02/02/journey-to-the-west.html" rel="alternate" type="text/html" title="西游之悟：灵山只在汝心头" /><published>2025-02-02T20:00:00+00:00</published><updated>2025-02-02T20:00:00+00:00</updated><id>https://zyhe.github.io/2025/02/02/journey-to-the-west</id><content type="html" xml:base="https://zyhe.github.io/2025/02/02/journey-to-the-west.html"><![CDATA[<h1 id="西游之悟灵山只在汝心头">西游之悟：灵山只在汝心头</h1>

<p>何志宇 2025年2月2日</p>

<p>《西游记》以唐僧师徒西往灵山求取真经为主线，融奇绝瑰丽的想象和幽微细致的洞察于一体，书写了一部异彩纷呈的史诗。这一传世名著既拥有丝丝入扣的故事情节，又有对人性和社会的细腻反映，更因其深邃内涵而兼具多种角度的解读。</p>

<p>通过西天取经这一主线，作者流畅地串起了唐僧师徒所经历的多元化的冒险经历。这些跌宕起伏的情节也成为了后世众多流行文化作品的重要基础，如周星驰的《大话西游》系列，以及《黑神话：悟空》游戏。天庭也有层次化的权力结构，其中的各路神仙既为取经团队提供护佑，也直接或间接地制造了各类困难；呈多元化构成的取经团队里也有分歧、合作、牺牲和互利。古往今来，人们从佛道、讽喻、玩世、劝学和心学等诸多视角来剖析西游宇宙，阐发其深邃意蕴。</p>

<p>我想以第八十五回中悟空的“佛在灵山莫远求，灵山只在汝心头”一语为统摄，就修炼成长的角度来分析西游记提供的智慧。无论是个体还是群体，都会在一段或长或短的阶段里，发愿做一番可大可小的事业。为实现目标，需要发挥各方面的潜能，逢山开路，遇水搭桥，必要时寻求内外部帮助，从而克服横亘在前行路上的重重难关，最终才能功成圆满，见性明心。但这所有的一切，都依赖于坚定、强大、平和而沉稳的内心。</p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-2-Journey-to-the-west/hills.avif" alt="目标" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<!-- 名著往往触及了最本质的方面，提供了最深刻的描绘。同时，由于他们触及了重要的母题，因此具有多种解读角度。例如，天庭是对一般化的权力结构的抽象；各路妖魔鬼怪中，也有亲疏之分，他们的结局也各不相同。我想就取经本身，探讨修炼的意义。
西游记可被视为个人/团队修行或长之书。初为发展做一番前无古人（后亦难有来者）的大事业，继而为寻找帮手、亦可视为锻炼各方面的能力，随后便是逢山开路，遇水搭桥，克服一个又一个的困难。最后才是功成圆满，见性明心。 -->

<h2 id="独自修行不可替代的成长之路">独自修行：不可替代的成长之路</h2>

<p>唐僧的各位弟子都是“谪仙”，能够腾云驾雾，各有神通。唐僧本人虽贵为如来之徒金蝉子转生，无奈因肉体凡胎而力犹未逮。既然悟空往返灵山只是弹指一瞬，为何不由他携唐僧直至佛祖座前求取真经呢？</p>

<p>作者在第二十二回借八戒之问和悟空之答做了巧妙的回应，既让整个取经故事变得自洽，也阐发了坚持修炼的意义。八戒问道：“哥啊，既是这般容易，你把师父背著，只消点点头，躬躬腰，跳过去罢了，何必苦苦的与这怪厮战？”悟空作答：</p>

<!-- 在取经团队中，唐僧的三个弟子都能腾云驾雾，大有神通。那为什么不由悟空直接携唐僧从东土大唐到佛祖座前取得真经呢？八戒也有这般疑问，他说：“哥啊，既是这般容易，你把师父背著，只消点点头，躬躬腰，跳过去罢了，何必苦苦的与这怪厮战？”悟空所做的回答，既让整个取经故事变得自洽，也点出了修炼的意义。他说： -->

<blockquote>
  <p>“但只是师父要穷历异邦，不能够超脱苦海，所以寸步难行也。我和你只做得个拥护，保得他身在命在，替不得这些苦恼，也取不得经来；就是有能先去见了佛，那佛也不肯把经善与你我。正叫做‘若将容易得，便作等闲看’。”</p>
</blockquote>

<p>原来，漫漫取经之路是唐僧师徒为获真经而必经的修炼，既无从逃避，亦无捷径可循。唐僧固然可借助弟子的神通来省却无限烦恼，径往西天。然而，若缺失了这一路上必要的磨练，便会与真经缘悭一面，无法真正开悟。</p>

<p>有一种诱惑是“如果”：如果能借助别人的力量来直接做成，那能有多好！然而，这种唾手可得的收获会随时间推移而被我们快速忽视。此外，我们不总有倚仗他人的机会，也没有人能代为克服属于我们自己的各项挑战。反之，若能经过奋斗而实现足够有挑战性的目标，我们才会从中获得极大的慰藉。</p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-2-Journey-to-the-west/buddha.avif" alt="如来" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<!-- 人们常说“目标不能轻易实现”。一方面，足够有挑战性的目标才能激励我们为之奋斗；另一方面，我们也会忽视。 -->

<!-- 每个人都要独自面对生活中的各项挑战，别人替代不得。 -->

<h2 id="信念的力量挺过至暗时刻">信念的力量：挺过至暗时刻</h2>

<p>修炼不仅仅是独自面对挑战，坚定的信念同样至关重要，也是激励我们持续前行的关键。不然，各类困难会令人未战先怯，以至不能充分发挥潜能，甚至半途而废。</p>

<!-- 这也是为什么毛泽东能在抗日战争处于初期的不利阶段摒弃速胜论、速败论等极端论调、而得出清醒的论持久战的原因。 -->

<p>值得引以为戒的是，即便强如悟空，也会有信念动摇的至暗时刻。悟空降妖除怪，从未有丝毫畏惧或退却的念头，直到他在狮驼岭中三魔大鹏的诛心之计时。</p>

<p>当狮驼岭众魔擒获唐僧师徒而走脱悟空后，心狠手辣的大鹏提出先将唐僧藏匿，再放出唐僧已被吞食的谣言在狮驼城中传播，从而让悟空死心塌地放弃营救唐僧。果不其然，悟空在误以为唐僧已死后心如刀绞，万念俱灰。因为误以为取经之梦将永远幻灭，他叹到“西方胜境无缘到，气散魂消怎奈何？”，便至如来前哭诉，甚至打算重回水帘洞。好在经如来点醒和施展法力，唐僧一众终于获救。悟空等也重新找回了失去的信念，继续踏上了前往灵山的路途。</p>

<!-- 在后半部中，作者用四回详细叙述了唐僧师徒挺过狮驼岭重重磨难的遭遇。这里有一位强劲的敌手：从辈分上说要数如来之舅的大鹏。在利用计谋擒得除悟空外的其它师徒后，为应对悟空威胁，他使出了一条诛心之计： 
“我这皇宫里面有一座锦香亭子，亭子内有一个铁柜。依著我，把唐僧藏在柜里，关了亭子；却传出谣言，说唐僧已被我们夹生吃了，令小妖满城讲说。那行者必然来探听消息，若听见这话，他必死心塌地而去。待三五日不来搅扰，却拿出来，慢慢受用。如何？” -->
<!-- 果然，悟空心如刀绞，万念俱灰，他想到“西方胜境无缘到，气散魂消怎奈何？”，最终打定主意要见一见如来，如若不能取得真经，便重回水帘洞。 -->
<!-- 当狮驼岭师徒四人均落难时，大鹏的毒计使悟空误以为唐僧已经遇害，他心神恍惚，极度不平，总算在如来的指点下才看清真相，找回了失去的信念。 -->

<p>假如我们有外部的肯定和支持，那当然最好。但在更多时候，要用坚定的信念来提供持续的动力，在任何时候都不要放弃希望。当然，这种信念不应是盲目的，而是在经审时度势、缜密分析和去芜存菁后所做出的准确判断。
<!-- 智若能在审时度势后提取关键事信息，做出准确判断。 --></p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-2-Journey-to-the-west/mountain-beast.png" alt="狮驼岭" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<h2 id="戒骄戒躁心如止水">戒骄戒躁，心如止水</h2>

<p>纵使笃定如唐僧，也难免会在山高路远时为目标难及而骄躁。当望见崇山峻岭时，他会道“悟空，你说得几时方可到？”，也会说：“休言无事。我看那山峰挺立，远远的有些凶气，暴云飞出，渐觉惊惶，满身麻木，神思不安。” 悟空所答颇有禅机，他说：</p>

<blockquote>
  <p>“你自小时走到老，老了再小，老小千番也还难；只要你见性志诚，念念回首处，即是灵山。”</p>

  <p>“心净孤明独照，心存万境皆清。差错些儿成惰懈，千年万载不成功。但要一片志诚，雷音只在眼下。似你这般恐惧惊惶，神思不安，大道远矣，雷音亦远矣。且莫胡疑，随我去。”</p>
</blockquote>

<p>须知，浮躁时产生的想法和做出的决定往往是不可靠的。要在大多数时候争取心态平和，也定要能戒骄戒躁，脚踏实地。
<!-- 当然，在冲刺等重要时刻，需调动精神，全力以赴。   --></p>

<div style="text-align: center; margin-top: 20px; margin-bottom: 30px;">
  <img src="/img/posts/2025-2-2-Journey-to-the-west/mountain.avif" alt="山峰" style="width: 60%; display: block; margin: 0 auto;" />
</div>

<!-- 当然，就故事情节的连贯性和多样性而言，西游记并不如水浒传那般精彩。在西游记中，各回得故事相对独立，且形式单一，一般是唐僧（或与八戒及沙说）一起蒙难，仰仗悟空解救，而悟空常力犹未逮，便需天庭的其它神仙一并助力。 -->

<h2 id="结语灵山只在汝心头">结语：灵山只在汝心头</h2>

<p>西游记不仅仅是一部情节跌宕的文学巨著，也因其探讨的修炼成长的主题而引人深思。梦在前方，路在脚下，就让坚定而沉稳的内心引领我们到达灵山胜境。</p>]]></content><author><name>Zhiyu He</name></author><summary type="html"><![CDATA[西游之悟：灵山只在汝心头]]></summary></entry><entry><title type="html">Gittins Index Theorem and Multi-Armed Bandits</title><link href="https://zyhe.github.io/2024/12/05/Gittins-index.html" rel="alternate" type="text/html" title="Gittins Index Theorem and Multi-Armed Bandits" /><published>2024-12-05T11:00:00+00:00</published><updated>2024-12-05T11:00:00+00:00</updated><id>https://zyhe.github.io/2024/12/05/Gittins-index</id><content type="html" xml:base="https://zyhe.github.io/2024/12/05/Gittins-index.html"><![CDATA[<h1 id="gittins-index-theorem-and-multi-armed-bandits">Gittins Index Theorem and Multi-Armed Bandits</h1>

<p>Zhiyu He, December 5, 2024<br />
Adapted from</p>

<ul>
  <li>Gittins, John, Kevin Glazebrook, and Richard Weber. Multi-armed bandit allocation indices. John Wiley &amp; Sons, 2011.</li>
  <li>Sébastie Bubeck and Nicolò Cesa-Bianchi. Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. Foundations and Trends in Machine Learning, 2012.</li>
</ul>

<h2 id="introduction">Introduction</h2>

<ul>
  <li>Goal: solve sequential decision-making problems featuring exploration-exploitation trade-offs</li>
  <li>Salient feature: to find an optimal policy, it suffices to focus on an index corresponding to each choice and encompassing immediate rewards and future benefits</li>
</ul>

<h3 id="motivating-example">Motivating Example</h3>

<ul>
  <li>Two-armed bandit<br />
Consider two slot machines that give the following reward sequences
    <ul>
      <li>Slot machine 1: $4, 5, 2, 6, 10, 1, \ldots$</li>
      <li>Slot machine 2: $8, 3, 10, 12, 1, 2, \ldots$</li>
    </ul>

    <p>If not pulled, then the reward to be given is fixed.<br />
How to sequentially select a machine (i.e., an arm) to maximize the discounted cumulative reward?<br />
A myopic solution: $8 + 4a + 5a^2 + 3a^3 + 10a^4 + \ldots$<br />
An arguably better solution $8 + 3a + 10a^2 + 12a^3 + 4a^4 + \ldots$<br />
(For instance, if the discount factor is $a=0.9$, then the latter discounted sum is larger than the former)</p>
  </li>
  <li>Other examples
    <ul>
      <li>Single machine scheduling</li>
      <li>Managing research projects</li>
    </ul>
  </li>
</ul>

<h3 id="three-formulations-of-the-bandit-problem">Three formulations of the bandit problem</h3>

<ul>
  <li>Criterion: the nature of the reward sequence
    <ul>
      <li>stochastic (drawn from some unknown distribution)
-&gt; upper confidence bound strategies<br />
idea: construct upper confidence bound estimates on the means of arms and choose the arm that looks best under these estimates<br />
principle: optimism in face of uncertainty</li>
      <li>adversarial (arbitrarily set after we select the arm)
-&gt; hedge algorithm<br />
idea: maintain and iteratively update a probability distribution of selecting different arms</li>
      <li>Markovian (features known state transition)<br />
-&gt; Gittins index</li>
    </ul>
  </li>
</ul>

<h2 id="problem-formulation">Problem Formulation</h2>

<ul>
  <li>
    <p>Simple family of alternative bandit processes<br />
$n$ independent bandit processes $B_i, i=1,\ldots,N$</p>
  </li>
  <li>two possible actions
    <ul>
      <li>continue: produces reward and causes state transition</li>
      <li>freeze: no reward and the state is fixed</li>
    </ul>
  </li>
  <li>
    <p>state $\xi_i$ lies in a countable state space $E_i$</p>
  </li>
  <li>
    <p>At decision time $t$: apply control to a specific $B_i$</p>
  </li>
  <li>Results
    <ul>
      <li>obtain a reward $r_i(\xi_i)$ from the bandit process $B_i$</li>
      <li>the state of $B_i$ transits from $\xi_i$ to $y$ with probability $P_i(y|\xi_i)$</li>
      <li>other bandits are frozen: their states do not change, and no rewards are obtained</li>
    </ul>
  </li>
  <li>
    <p>Goal: maximize the infinite-horizon expected discounted sum of rewards (i.e., the payoff)</p>

\[\sup_{\pi} \mathbb{E} \left[ \sum_{t=0}^{\infty} a^t r_{i_t} (\xi_{i_t} (t))\right]\]

    <p>$a \in (0,1)$: discount factor   $i_t$: the index of the bandit process chosen at time $t$</p>
  </li>
</ul>

<h3 id="why-not-alternative-methods">Why not alternative methods?</h3>

<ul>
  <li>Dynamic programming: the number of stationary policies (or the size of the state space) grows exponentially with the number of bandits
    <ul>
      <li>
        <p>Value function</p>

\[\quad V(\xi) = \sup_{\pi} \mathbb{E} \left[ \sum_{t=0}^{\infty} a^t r_{i_t} (\xi_{i_t} (t)) \mathrel{\Big|} \xi(0) = \xi \right]\]
      </li>
      <li>
        <p>Bellman’s equation</p>

\[V(\xi) = \max_{i \in \{1, \ldots, n\}} \left\{ r_i(\xi_i) + a \sum_{y \in E_i} P_i(y|\xi_i)V(\xi_1, \ldots, \xi_{i-1}, y, \xi_{i+1}, \ldots, \xi_n) \right\}\]
      </li>
    </ul>

    <p>number of states $\prod_i k_i \triangleq k$   for each state, we need to choose from $n$ actions</p>

    <p>number of stationary policies: $k^n$</p>
  </li>
</ul>

<h2 id="gittins-index-theorem">Gittins Index Theorem</h2>
<!-- ### Intuitions from single machine scheduling -->

<ul>
  <li>
    <p>Key idea: we aim to find an index indicating the behavior of a bandit process. This behavior blends immediate rewards and future benefits.</p>
  </li>
  <li>
    <p>Point of attack: construct a simple reference bandit process</p>
  </li>
</ul>

<h3 id="calibrating-bandit-processes">Calibrating bandit processes</h3>

<ul>
  <li>
    <p>Consider a discrete-time standard bandit process $\Lambda$, which gives a known reward $\lambda$ each time it is continued</p>

    <p>If it is selected at every time, then the cumulative discounted reward is</p>

\[\lambda(1+a+a^2+\ldots) = \frac{\lambda}{1-a}\]

    <p>We want to compare a bandit process $B_i$ and $\Lambda$. Consider the following setting: we continuously apply control to $B_i$ and then follow an optimal policy.</p>

    <p>This policy can switch to $\Lambda$ at some future time $\tau(\tau&gt;0)$</p>

    <ul>
      <li>
        <p>Observation: if an optimal policy causes us to switch at time $\tau$, we will never switch back</p>
      </li>
      <li>
        <p>Reason: the information on $B_i$ at time $\tau + 1$ is the same as at time $\tau$. If applying control to $\Lambda$ is optimal at time $\tau$, then it is also optimal at time $\tau + 1$ and afterwards.</p>
      </li>
    </ul>

    <p>The maximal payoff is</p>

\[\sup_{\tau &gt; 0} \mathbb{E} \left[ \sum_{t=0}^{\tau-1} a^t r_i(x_i(t)) + a^{\tau} \frac{\lambda}{1-a} \mathrel{\Big|} x_i(0) = x_i \right]\]

    <p>We want to find $\lambda$ such that it is equally optimal to apply control to either of the two bandit processes initially, i.e.,</p>

\[0 = \sup_{\tau &gt; 0} \mathbb{E} \left[ \sum_{t=0}^{\tau-1} a^t r_i(x_i(t)) - (1 - a^{\tau}) \frac{\lambda}{1-a} \mathrel{\Big|} x_i(0) = x_i \right]\]

    <p>We note that $\sum_{t=0}^{\tau-1} a^\tau = \frac{1-a^\tau}{1-a}$. Therefore, $\lambda$ satisfies the following equation</p>

\[0 = \sup_{\tau &gt; 0} \mathbb{E} \left[ \sum_{t=0}^{\tau-1} (a^t r_i(x_i(t)) - a^t \lambda) \mathrel{\Big|} x_i(0) = x_i \right]\]

    <p>The right-hand side is the supremum of decreasing linear functions of $\lambda$. Hence, it is convex and decreasing in $\lambda$. The root exists and is unique. We obtain the following expression of $\lambda$</p>

\[\lambda = \sup_{\tau &gt; 0} \frac{\mathbb{E} \left[ \sum_{t=0}^{\tau-1} a^t r_i(x_i(t)) \right | x_i(0)=x_i ]}{\mathbb{E} \left[ \sum_{t=0}^{\tau-1} a^t | x_i(0)=x_i \right] }\]

    <p>Another equivalent formulation</p>

\[\sup\left\{\lambda : \sup_{\tau &gt; 0} \mathbb{E} \left[ \sum_{t=0}^{\tau-1} a^t [r_i(x_i(t)) - \lambda] \mid x_i(0) = x_i \right] \geq 0\right\}\]

    <p>Reason: the supremum $\lambda$ ensures that the expectation equals zero.</p>

    <p>Interpretation of $\lambda$</p>
    <ul>
      <li>fixed reward that makes us indifferent to which of $B_i$ and $\Lambda$ to continue initially</li>
      <li>fair charge (highest rent) that we are willing to pay to continue $B_i$ for at least one round and then optimally switch</li>
    </ul>

    <p>Observation</p>
    <ul>
      <li>with the supreme $\lambda$ and the optimal stopping time $\tau$, the cumulative discounted rewards collected from these bandit processes are the same</li>
    </ul>

    <p>This $\lambda$ is the so-called Gittins index</p>
  </li>
</ul>

<h3 id="key-statement">Key statement</h3>

<p>We maximize the expected discounted reward obtained from a simple family of alternative bandit processes by always continuing the bandit having greatest Gittins index. Namely, for</p>

\[v(B_i, x_i) = \sup_{\tau &gt; 0} \frac{\mathbb{E} \left[ \sum_{t=0}^{\tau-1} a^t r_i(x_i(t)) \right | x_i(0)=x_i ]}{\mathbb{E} \left[ \sum_{t=0}^{\tau-1} a^t | x_i(0)=x_i \right] }\]

<p>we always select the bandit process $i = \operatorname{argmax}_i v(B_i,x_i)$.</p>

<ul>
  <li>Numerator: discounted reward up to $\tau$ (stopping time)</li>
  <li>Denominator: discounted time up to $\tau$</li>
</ul>

<p>Note that the Gittins index only depends on the corresponding bandit process (or more specifically, $a, r_i, p_i$) but not on other processes.</p>

<h3 id="proof-via-prevailing-charges-arguments">Proof via prevailing charges arguments</h3>

<ul>
  <li>
    <p>High-level idea<br />
(i) Construct an upper bound; (ii) show that this upper bound can be attained<br />
Abstract example<br />
Suppose that we have two functions $f,g$ such that $f(\pi) \leq g(\pi), \forall \pi$. If we find a policy \(\pi^*\) such that \(f(\pi^*) = g(\pi^*)\) and \(g(\pi) \leq g(\pi^*), \forall \pi\), then $\pi^*$ maximizes $f(\pi)$</p>
  </li>
  <li>
    <p>Prevailing charge $g_i(x_i)$: fixed charge that we pay to continue playing the bandit $i$ at state $x_i$<br />
If $g_i(x_i)$ is too large, then we will quit playing this bandit</p>
  </li>
  <li>
    <p>Fair charge $\lambda_i(x_i)$: level of prevailing charge when we are indifferent at state $x_i$ between stopping and continuing for at least one more step and then optimally stopping<br />
We give $\lambda_i$ in the previous ratio</p>
  </li>
</ul>

<p>Processes</p>

<ul>
  <li>Initially, $g_{i,0} = \lambda_i(x_i(0))$</li>
  <li>A point will be reached when it would be optimal to stop continuing the bandit $i$. This point corresponds to the stopping time $\tau$.
At this time, $\lambda_i(x_i(\tau)) &lt; g_{i,0} = \lambda_i(x_i(0))$
This is also the first time that the prevailing charge is higher than the fair charge</li>
  <li>We reduce the prevailing charge and set $g_{i,\tau} = \lambda_i(x_i(\tau))$</li>
  <li>In general, $g_{i,t} = \min_{s\leq t} \lambda_i(x_i(s))$<br />
$g_{i,t}$ is a nonincreasing function of $t$ and only depends on the states of bandit process $i$</li>
</ul>

<p>Consider a simple family of $n$ alternative bandit processes $B_1, \ldots, B_n$. The gambler not only collects $r_{i_t}(x_{i_t}(t))$, but also pays the prevailing charge $g_{i_t,t}$ of the bandit $B_{i_t}$ chosen at time $t$. We have the following observations</p>

<ul>
  <li>
    <p>The gambler cannot do better than break even (i.e., zero expected profit)<br />
reason: a strictly positive profit is only possible if this were to happen for at least one bandit. However, we define the prevailing charge in a way (i.e., always equals the fair charge) such that no profit can be gained from any selected arm.</p>

    <p>Implications</p>

\[\mathbb{E}_\pi \left[ \sum_{t=0}^{\infty} a^t \left( r_{i_t}(x_{i_t}(t)) - g_{i_t}(x_{i_t}(t)) \right) \middle| x(0) \right] \leq 0,\]

    <p>which implies that</p>

\[\mathbb{E}_\pi \left[ \sum_{t=0}^{\infty} a^t r_{i_t}(x_{i_t}(t) \middle| x(0) \right] \leq \mathbb{E}_\pi \left[ \sum_{t=0}^{\infty} a^t g_{i_t}(x_{i_t}(t)) \middle| x(0) \right]\]
  </li>
  <li>We maximize the expected discounted sum of prevailing charges by always continuing the bandit with the greatest prevailing charge<br />
Reason
    <ul>
      <li>Each sequence of prevailing charges is nonincreasing</li>
      <li>The above interleaving strategy maximizes the discounted sum</li>
    </ul>

    <p>Example</p>
    <ul>
      <li>$g_1: 10, 10, 9, 5, 5, 3, \ldots$</li>
      <li>$g_2: 20, 15, 7, 4, 2, 2$</li>
      <li>optimal sequence: $20, 15, 10, 10, 9, 7, 5, 5, 4, 3, 2, 2$</li>
    </ul>

    <p>Implication: the Gittins index policy $\pi^*$ maximizes the right-hand side</p>
  </li>
  <li>The Gittins index policy $\pi^*$ of always continuing the bandit with the greatest index allows us to break even.<br />
Reason: the Gittins index policy enables us to earn as much as those indicated by fair charges
Note: if an arm is not pulled, then the state is fixed, and hence the corresponding Gittins index is unchanged.<br />
Implication: the Gittins index policy attains equality</li>
</ul>

<h3 id="alternative-proof-via-interchanging-bandit-portions">Alternative proof via interchanging bandit portions</h3>

<ul>
  <li>Intuition: let a bandit process with a higher reward run first<br />
If in an optimal policy there is a point where we deviate from the arm with the greatest Gittins index, then by interchanging the order we can increase the reward. Hence, this contradiction implies that the optimal policy requires pulling the arm with a high index as early as possible.</li>
</ul>

<h2 id="computational-example">Computational Example</h2>

<ul>
  <li>
    <p>Setup<br />
We have $M$ independent bandit processes. Each process $i$ is endowed with a state $s_i$ belonging to a set $\mathcal{S} = {1, \ldots, N}$ and a known state transition matrix $P_i \in \mathbb{R}^{N \times N}$. At each state $s_i$, there is an associated reward $r_i \in \mathbb{R}$.</p>

    <p>At every time $k$, we will select a bandit process and receive a reward. The state of the selected process transits to a new value, whereas the states of the remaining processes keep fixed.</p>

    <p>Our goal is to maximize the cumulative discounted rewards.</p>
  </li>
  <li>Comparison
    <ul>
      <li>Strategy based on selecting the maximum Gittins index</li>
      <li>Myopic strategy that selects the arm giving the maximum instantaneous reward</li>
    </ul>
  </li>
  <li>
    <p>Codes<br />
See the Colab link <a href="https://colab.research.google.com/drive/1ekJT80VxjemaFC45LdUuIo78Wj_l8b4_?authuser=0#scrollTo=uPGh6ukeiJay&amp;forceEdit=true&amp;sandboxMode=true">here</a></p>
  </li>
  <li>Results<br />
The strategy based on the Gittins index yields a higher reward.
    <div style="text-align: center;">
  <img src="/img/posts/2024-12-5-Gittins/image.png" alt="Convergence curves" style="width: 70%; display: block; margin: 0 auto;" />
</div>
    <!-- ![alt text](/img/posts/2024-12-5-Gittins/image.png) -->
  </li>
</ul>

<h2 id="summary--extension">Summary &amp; Extension</h2>

<ul>
  <li>
    <p>Summary<br />
The Gittins index theorem provides an elegant characterization of the optimal policy for the multi-armed bandit problem with Markovian reward sequences. Instead of dealing with a vast Markov Decision Process, we only need to calculate the Gittins indexes of states of each arm and then obtain the optimal policy.</p>
  </li>
  <li>
    <p>Control<br />
Setup: an unknown dynamical system and a set of candidate controllers</p>

\[\begin{aligned}
&amp;x(t+1) = Ax(t) + Bu(t), \quad u(t) = -K_{i_t}x(t),  \\
&amp;\{K_i|i=1,\ldots,N\} : \mathrm{set~of~candidate~controllers}
\end{aligned}\]

    <p>Challenges</p>
    <ul>
      <li>dependent bandit processes</li>
      <li>the reward is not fixed if the corresponding bandit is left unchosen</li>
    </ul>
  </li>
</ul>]]></content><author><name>Zhiyu He</name></author><summary type="html"><![CDATA[Gittins Index Theorem and Multi-Armed Bandits]]></summary></entry></feed>