For a function \(f:\mathbb{R}\to \mathbb{C}\), define the translation operator
\[T(a)f(x) = f(x-a)\]where \(a\in\mathbb{R}\) represents the amount we shift our function. These operators form a Lie group under composition, since \(T(a)T(b)=T(a+b)\) and \(T(0)=I\). The natural representation-theoretic question is: what are the irreducible modes of this group action?
Since translations vary continuously with \(a\), they should have an infinitesimal generator. Expanding around \(a=0\), we get
\[\begin{align*} T(a)f(x) &= f(x-a) \\ &= f(x) - a\frac{d}{dx}f(x) + O(a^2) \end{align*}\]which suggests that the translation operator can be written as
\[T(a) = \exp\left(-a\frac{d}{dx}\right)\]so \(\frac{d}{dx}\) is, up to sign convention, the generator of the translation group. Now that we have the generator, to understand how translations operate, it suffices to understand the eigenfunctions of \(\frac{d}{dx}\). These are exactly (using the arbitrary factor of \(ik\) instead of a different constant, because I like physics)
\[\frac{d}{dx}e^{ikx} = ik e^{ikx}\]and therefore
\[T(a)e^{ikx} = e^{-ika}e^{ikx}\]meaning each Fourier mode (with eigenvalue \(e^{-ika}\)) is an eigenfunction of every translation operator. Notably, you immediately derive through the eigendecomposition (since translations are an \(L^2\) operator) that the Fourier modes are an orthogonal basis.
With this new basis, for a generic function \(f\), the Fourier transform is just computing its coordinates in this diagonal basis:
\[\hat{f}(k) = \int_{\mathbb{R}} f(x)e^{-ikx}dx\]with the inverse transform reconstructing the original function by summing over translation eigenmodes:
\[f(x) = \frac{1}{2\pi}\int_{\mathbb{R}}\hat{f}(k)e^{ikx}dk\]The representation-theoretic framing comes when we look at other operators that respect translation symmetry. Suppose \(L\) is some linear operator acting on functions and it commutes with translations:
\[\left[L, T(a)\right] = 0\]for every \(a\). As a consequence of Schur’s lemma, an operator that commutes with a group action preserves the irreducible pieces of that representation (via isomorphism) or degenerately vanishes. On each irreducible piece, it acts by a scalar; when multiple equivalent copies appear, it can only mix those multiplicities. So once we’ve decomposed into translation eigenfunctions, \(L\) acts by scalar multiplication on each mode, up to the usual degeneracies.
This is why translation-invariant operators are diagonalized by the Fourier basis. If \(L\) commutes with all \(T(a)\), and \(e^{ikx}\) spans an irreducible translation mode, then \(L e^{ikx}\) must transform under translations in the same way as \(e^{ikx}\). So \(L e^{ikx}\) is proportional to \(e^{ikx}\):
\[L e^{ikx} = \lambda(k)e^{ikx}\]This principle more broadly applies to any form of operator symmetry. The irreducible representations form subspaces over which we can cleanly diagonalize and understand the operator’s behavior over. For example, in quantum mechanics, you frequently utilize the symmetries of the system to decompose the Hamiltonian over the corresponding irreducible representations, which lets us more tangibly understand dynamics within those subspaces.
We can apply these insights to the Laplacian, an operator that’s frequently found in common PDEs:
\[\nabla^2 = \sum_i \frac{\partial^2}{\partial x_i^2}\]which commutes with translations in Euclidean space. Shifting a function and then taking its second derivatives gives the same result as taking its second derivatives and then shifting. So by the symmetry argument above, the Laplacian must be diagonal in the translation eigenbasis.
In E&M we learn that plane waves are non-normalizable (translation symmetries!) and represented by \(e^{ik\cdot x}\), with which we have
\[\nabla^2 e^{ik\cdot x} = -\vert k\vert^2 e^{ik\cdot x}\]which is exactly the diagonalization. In Fourier space, differential equations involving the Laplacian become algebraic equations involving \(-\vert k\vert^2\). This is why harmonic analysis, PDEs, and Fourier transforms are so tightly bound together. The method works because the operator and the geometry share the same symmetry.
]]>For discrete functions \(f: \mathbb{Z} \to \mathbb{R}\) the derivative is pointwise defined as
\[\Delta f(x) = f(x) - f(x-1)\]Like \(\frac{d}{dx}\) the \(\Delta\) is linear, \(\Delta(af + bg) = a\,\Delta f + b\,\Delta g\), and kills constants, \(\Delta c = 0\).
Looking at monomials, we see that \(\Delta x^n = x^n - (x-1)^n\)
expands out into an ugly sum of terms through binomial expansion. This isn’t that nice to work with, and so instead we can look at a better set of monomials that behave more like the continuous versions. These are the rising factorials:
\[x^{\overline{n}} = x(x+1)(x+2)\cdots(x+n-1) \quad (n \text{ factors})\]which when we apply \(\Delta\) we get
\[\Delta x^{\overline{n}} = x^{\overline{n}} - (x-1)^{\overline{n}} = \big[x(x+1)\cdots(x+n-2)\big]\big[(x+n-1) - (x-1)\big] = n\, x^{\overline{n-1}}\]just like with \(\frac{d}{dx}\). There’s a product rule too, with one discrete wrinkle, just requiring a shift on the second factor: \(\Delta(uv) = v\,\Delta u + u(x{-}1)\,\Delta v\). Summing both sides gives summation by parts, the analogue of integration by parts: \(\sum_{x=a}^{b} v\,\Delta u = uv\big\rvert_{a-1}^{b} - \sum_{x=a}^{b} u(x{-}1)\,\Delta v\), which resolves terms like \(\sum x\,2^x\) exactly the way \(\int x e^x\,dx\) yields to parts.
The chain rule is the one place the analogy breaks: \(\Delta f(g(x)) = f(g(x)) - f(g(x-1))\) won’t factor unless \(g\) is affine (e.g., shifting \(g(x) = x+c\) just reindexes, \(\Delta f(x+c) = (\Delta f)(x+c)\)) since differences care about integer steps and most \(g\) don’t preserve them.
Suppose we want \(\sum_{x=a}^{b} g(x)\). The trick is to find an antidifference, a function \(F\) with \(\Delta F = g\), just as we’d hunt for an antiderivative. The sum telescopes:
\[\sum_{x=a}^{b} \Delta F(x) = \sum_{x=a}^{b} \big[F(x) - F(x-1)\big] = F(b) - F(a-1)\]Every interior term is added once and subtracted once; only the ends survive, an analogue of the fundamental theorem of discrete calculus:
\[\sum_{x=a}^{b} g(x) = F(b) - F(a-1), \qquad \text{where } \Delta F = g\]And we already have the antidifference of a rising factorial. Reading the power rule backwards:
\[\Delta\!\left(\frac{x^{\overline{n+1}}}{n+1}\right) = x^{\overline{n}}\]which is the discrete twin of \(\int x^n\,dx = \frac{x^{n+1}}{n+1}\).
Once you have the power rule and the fundamental theorem, a surprising amount falls out with no cleverness required.
Power sums. For \(\sum_{x=1}^N x\) we have \(g(x) = x = x^{\overline{1}}\), so the antidifference is \(x^{\overline{2}}/2 = x(x+1)/2\) and
\[\sum_{x=1}^{N} x = \frac{x(x+1)}{2}\Big|_{x=0}^{N} = \frac{N(N+1)}{2}\]For higher powers, we just rewrite the monomial in the rising-factorial basis. e.g. for \(k=2\) we get \(x^{\overline{2}} = x^2 + x\), and \(x^2 = x^{\overline{2}} - x^{\overline{1}}\), and antidifferencing each piece gives
\[\sum_{x=1}^{N} x^2 = \left(\frac{x^{\overline{3}}}{3} - \frac{x^{\overline{2}}}{2}\right)\Bigg|_{x=0}^{N} = \frac{N(N+1)(N+2)}{3} - \frac{N(N+1)}{2} = \frac{N(N+1)(2N+1)}{6}\]The same recipe handles \(\sum x^k\) for any \(k\): expand \(x^k\) in rising factorials (the coefficients turn out to be the Stirling numbers) and integrate term by term.
Geometric series. In ordinary calculus the exponential is special because it’s nearly its own derivative; the same is true here. For a constant \(c\), \(\Delta c^x = c^x - c^{x-1} = c^x\!\left(\frac{c-1}{c}\right)\), so \(c^x\) is its own difference up to a constant, with antidifference \(c^{x+1}/(c-1)\). The fundamental theorem then hands you the geometric series for free:
\[\sum_{x=1}^{N} c^x = \frac{c^{x+1}}{c-1}\Bigg|_{x=0}^{N} = \frac{c^{N+1} - c}{c-1} = \frac{c\,(c^N - 1)}{c-1}\]Recurrences, and solving Fibonacci. Recurrence formulas are the discrete analogues of ODEs: \(\Delta a_n = g(n) \quad\Longleftrightarrow\quad a_n = a_{n-1} + g(n),\)
To solve a constant-coefficient ODE in school you’re taught to guess \(e^{\lambda x}\) and solve a characteristic polynomial; since \(c^x\) is the discrete exponential, the discrete move is to guess \(a_n = r^n\). Take Fibonacci, \(F_n = F_{n-1} + F_{n-2}\). Substituting \(r^n\) and dividing by \(r^{n-2}\) leaves the characteristic equation
\[r^2 = r + 1 \quad\Longrightarrow\quad r = \frac{1 \pm \sqrt{5}}{2},\]the golden ratio \(\varphi\) and its conjugate \(\psi\). The general solution is \(F_n = A\varphi^n + B\psi^n\), and the initial conditions \(F_0 = 0\), \(F_1 = 1\) pin down \(A = -B = 1/\sqrt{5}\) to give Binet’s formula
\[F_n = \frac{\varphi^n - \psi^n}{\sqrt{5}}\]]]>The value of information is tied to its asymmetry of access: knowledge is counterfactually worth something only insofar as not everyone has access to it in an efficient market hypothesis world.
Today’s access asymmetry isn’t randomly distributed. Excutive insiders know about the acquisition first. The donor families hear about the policy shifts in real time. The 20B quant fund has the earned access from colocated serveris and proprietary data feeds. Access to information has always been a function of where you sit in the existing hierarchy of capital, power, and connection.
So “the value of information is its asymmetry” is also saying that the returns to information accrue to people who are already networked into the existing hierarchy, implying that when information becomes more valuable, by expanding markets volume and liquidity, can entrench existing power rather than democratize it. The frictions we have in markets both regulated (insider trading rules, halts, leverage caps, market cap mins) and emergent (pace of information exchange, capital access, etc.) are actually desirable as they regularize the net value extraction possible from information asymmetry.
The idea that markets should be the primary channel for disemmination is a recent phenomena within the last fifty year. Historically, for most things that matter to people’s lives like local news, economy, foreign policy etc., the marginal value of a real-time price over ordinary reporting and news outlets is small. Fundamentally, equity markets are a growing asset class and so there’s value creation in expanding access: we’re willing to have retail pay to play (via lower information) because fundamentally the pie is growing and so over a sufficiently long time horizon of investment the EV is positive.
Prediction markets don’t allow for this. Fundamentally these assets are zero sum and the EV of retail trades is always negative, just like gambling, bringing along all of the other severe negative social effects and behaviors associated with it as well. Its hard for me to see how pursuing liquidity here is justified or worth any investment in as a society.
]]>The ceiling on selection power is … low. There’s fundamental constraints on what you can do with \(N\) samples from a highly polygenic distribution.
In a typical egg harvest, you get maybe 10-15 viable embryos if you’re lucky, rarely more than 50-100 even in aggressive scenarios.
With IVF selection effectively choosing the max of \(N\) samples based on some objective function, and given our sample sizing that translates to just 3-7 bits of selection power.
But it gets worse when you account for how these traits actually work genetically. Height is controlled by thousands of genetic variants. IQ is even more polygenic. They’re sums of thousands of small effects with few if any big hitters. We can model each embryo’s genetic potential distribution \(X\) for a polygenic trait as a sum of \(K\) i.i.d variables contributing to the phenotype effect, which by the Central Limit Theorem implies \(X\) can be modeled as a normal distribution. Depending on the polygeneity of the phenotype, the variance of \(X\) can already be quite small across samples, as \(\sigma \sim O(\frac{1}{\sqrt{k}})\).
The question then becomes, can we improve on \(E[X]\)? How much better can we do by selecting \(N\) samples from \(X\)?
Rescaling \(X\sim N(0,1)\), take \(X_1, X_2, \ldots, X_N \stackrel{\text{i.i.d}}{\sim} N(0,1)\), we want to understand \(\mathbb{E}[Z]\) where \(Z = \max(X_1, \ldots, X_N)\).
To bound \(E[Z]\) we can use the CDF:
\[\begin{align*} P(Z \leq z) &= P(X_1 \leq z, \ldots, X_N \leq z) \\ &= \Phi(z)^N \end{align*}\]where \(\Phi\) is the standard normal CDF. For the tail bound, note that for large \(z\):
\[P(X_i > z) \approx \frac{1}{\sqrt{2\pi}z}e^{-z^2/2}\]By union bound: \(P(Z > z) \leq N \cdot P(X_1 > z)\). Setting \(z = \sqrt{2\log N}\):
\[\begin{align*} P(Z > \sqrt{2\log N}) &\leq N \cdot \frac{1}{\sqrt{4\pi \log N}} e^{-\log N} \\ &= \frac{1}{\sqrt{4\pi \log N}} \to 0 \end{align*}\]Similarly, we can show concentration below this threshold is exponentially small, giving us that \(Z\) concentrates around \(\sqrt{2\log N}\) and therefore \(\mathbb{E}[Z] \approx \sqrt{2\log N}\).
The math here is unforgiving. Selection power scales incredibly slowly as \(\sqrt{2\log N}\), and you’re at best getting a tiny constant factor improvement on \(\sigma\) which is low from \(\frac{1}{\sqrt{K}}\).
Selection upside is fundamentally limited by information theory and biology. The traits you sample are highly concentrated due to polygenic effects and you can’t improve on it by scaling sampling.
The technology right now shines at screening out rare genetic diseases, which are extremley sparsely distributed, requiring very few bits of selection power to remove.
For polygenic trait optimization – smarter, taller, more athletic kids – the ceiling is low and already here.
]]>The article takes a different basis approach, representing functions as vectors of polynomial coefficients, which simplifies the picture. However, with a cute reframing you can generalize this to the standard positional basis, where functions are infinite-dimensional vectors with coordinates representing evaluations \(f_n = f(x_n), x_n \in \mathbb{R}\).
To start, consider a circular permutation matrix of dimension \(N\):
\[P_N = \begin{pmatrix} 0 & 1 & 0 & 0 & \cdots & 0 \\ 0 & 0 & 1 & 0 & \cdots & 0 \\ 0 & 0 & 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & 1 \\ 1 & 0 & 0 & 0 & \cdots & 0 \end{pmatrix}\]As a permutation matrix of size \(N\), the eigenvalues are the \(N\) roots of unity \(\varphi_a\) and the eigenvectors from the single permutation cycle are represented by the \(N\) periodic Fourier Modes \(v_a = \frac{1}{N}\left(1, \omega^a, \omega^{2a}, \ldots, \omega^{(N-1)a} \right)\) where \(\omega = \exp\left(\frac{2\pi i}{N}\right)\).
Now, let’s view the derivative as an infinite dimensional operator
\[\begin{align*} A = \frac{d}{dx} = \frac{1}{\epsilon}\begin{pmatrix} 1 & -1 & 0 & 0 & \cdots \\ 0 & 1 & -1 & 0 & \cdots \\ 0 & 0 & 1 & -1 & \cdots \\ 0 & 0 & 0 & 1 & \cdots \\ \vdots & \vdots & \vdots & \vdots & \ddots \end{pmatrix} \end{align*}\]where, in the linearized view, “rows” correspond to discrete evaluations of a functional at a point, spaced \(\epsilon\) apart as we take \(\lim \epsilon \to 0\).
Next, note that we can decompose \(A\) into the following:
\[\begin{align*} A = \lim_{\epsilon\to 0}\frac{1}{\epsilon}\left(I - P_\infty \right) \end{align*}\]where \(P_\infty\) is the operator of all ones above the diagonal, and satisfies convergence in operator norm \(\lim_{N\to\infty}\|P_\infty - P_N\| = 0\) to the permutation operator in \(\mathbb{L^2}\).
Then, it becomes clear that the eigenvectors of \(A\) are exactly the eigenvectors of \(P_\infty\) which map exactly onto the Fourier Modes now continuously indexed by \(a\), which directly corresponds to the eigenfunction basis \(e^{ax}\).
]]>Like in any industry, you drive by scale by finding broad horizontal problems with sufficient pain to warrant pricing, or select verticals with large enough scope to integrate into. Large verticals that are currently viable for robotics mostly include manufacturing, warehouse logistics, big agriculture, precision surgical/medicine, and fast-food restaurants. This list is constantly expanding as the tech improves and pushes COGs lower, increasing the surface area of accessible markets.
At a high level, you need broadly replicable pain points that are also not viewed by companies as core competencies. Core competencies will be brought in-house eventually (case studies: Scale AI AV labeling, Applied Intuition simulation infra sales, Zendesk AI agents, Tecton, and other marginal ML ops providers) and will be resilient to durable sales relations.
I’ve sorted the following list by my personal opinion of most to least solved and labeled whether it’s primarily a vertical or horizontally solved problem:
Distribution (Vertical)
For most companies, this is likely to be viewed as a core competency and remain so in the future. The playbook here feels similar to the wave of vertical SaaS OpenAI wrapper companies, where distribution and domain expertise are the primary moats and value drivers.
An additional differentiator to AI SaaS providers is that the hardware stack for robotics is not unified across use-cases, which prevents distribution from scaling horizontally well in general.
Hardware (Vertical now, Horizontal in ~5-10 years)
As better general use-case hardware develops, this will decrease in relative importance from a core function (“we need custom hardware to solve the problem”) to a competitive operational advantage (“we need custom hardware to reduce costs or drive efficiency compared to competitors”). Similar to the open-source platform and AI model communities, once more effective general use-case hardware hits the markets, the cost scaling benefits of standardized manufacturing will eventually dramatically drive down the marginal value of custom hardware.
Perception (Horizontal)
Vision models have gotten surprisingly good at the edge over the last three years, and edge compute and specialized chips are really pushing the frontier on what robots can now do in real time.
Other modalities of perception (LiDAR, radar, ultrasonic sensors, and infrared cameras) have less accessible public data, which might be limiting useful adoption from new players.
Outside of niche applications and modalities, perception is a pretty well-understood problem, and the majority of recent focus has been on tail edge cases. Finding instances of serviceable edge-cases seems like a promising approach to sourcing customers. For example, real-time vision safety modules to help prevent heavy machinery/robotics accidents in factory environments.
Currently, perception seems primarily internalized as it’s heavily coupled with hardware choice and power/latency tradeoffs and linked to safety, which has severe downside risks if not appropriately managed internally. I expect this to change over the next couple of years as horizontal providers reach acceptable thresholds/tradeoffs in both, as it’s clearly not viewed as a defendable moat.
Data (Vertical but with horizontal infra plays)
As LLMs have started to dominate media, it’s growing awfully apparent that data will become a core competency and moat for traditional platforms.
In robotics, data has significant barriers to access. It’s coupled to your hardware sensors and on-policy control procedures, and can require time-consuming physical interactions to extract realistic environments.
The real2sim gap still exists but is slowly getting better. The reality of long-tail edge cases, however, limits the effectiveness of iterating too reliantly on simulations as your ability to safely extrapolate to real-world deployments is dubious at best.
Simulations are clearly horizontal infrastructure, and I’m optimistic that in the near term new players can exist somewhere in-between to augment or curate physically gathered data (i.e., through data augmentation, scene analysis filtering) in a way that doesn’t compete with the existing bloated field of traditional ML ops providers.
Control (Vertical now, Horizontal in ~5-10 years)
Control is by far the least resolved component in the technical stack and might better be referenced as environment modeling in its current state. In the short term, this is a strongly differentiated capability (Waymo vs. everyone else?), but I suspect like with most software/algorithmic progress eventually this will change or centralize like in the case of foundation model labs.
Traditional control methods work extremely well as functional primitives, supplied with environment model inputs and called with intent from an intelligent orchestrator (often a tokenized action transformer). A primary bottleneck is gathering sufficient data to test automated control on the long tail of edge cases. For this reason, controlled environments or those with limited human interaction are currently the primary targets.
Compared to the other components, this is the purest “algorithmic” problem with low capital barriers to access. The physical economies of scale associated with distribution, hardware development/production, and data gathering might not necessarily apply here if algorithmic unlocks can be found.
There’s a ton to expand on here across the stack, including a lot of key components I’m leaving out, but I’m hoping that others interested in starting or evaluating robotics companies will find this useful. While I’m not currently involved or actively looking right now, I’m still very excited and interested in the five-year horizon, and would love to hear any pushback or new insights you might have.
]]>Arrow’s Theorem1 is the most famous impossibility theorem applied to voting systems – lots of articles and papers have spent time introducing and discussing its implications. In essence, Arrow’s Theorem states that a voting process which ranks candidates in an absolute total order cannot simultaneously satisfy:
Non-Dictatorship - no single voter dominates the output preferences
Pareto Efficiency - if all voters rank candidate A over B, then the resulting rankings support \(A \succcurlyeq B\)
Independence of Irrelevant Alternatives - if \(A \succcurlyeq B\) and a new candidate C is introduced, then the new ranking still requires \(A \succcurlyeq B\)
While this applies to discrete rankings and voter preferences, one might wonder if it’s a unique property of its discrete nature in how candidates are only ranked by ordering. Instead of discrete rankings, could a continuous preference ranking satisfy similar conditions? Unfortunately, a similarly flavored impossibility result holds even in the continuous setting! It seems there’s no getting around the fact that voting is pretty hard to get right.
The Chichilnisky Theorem 2 extends Arrow’s Theorem to the continuous setting, but with slightly different constraints.
Suppose that you have a set of \(K\) voters choosing between \(N\) candidates. Considering only relative preferences, we can represent the preference profile of each voter over the candidates \(p_k \in S^{n-1}\) as a unit vector on the sphere, with each coordinate \((p_k)_i\) representing the allotted relative preference for candidate \(i\). Denoting \(P = S^{n-1}\) as our preference space, then our voting function \(\phi\) becomes a map \(\phi: P^K \to P\) taking some set of preferences \(\phi(p_1,\ldots, p_K) \to u\) to a resulting unit vector preference profile.
The Chichilnisky Theorem states the following cannot be jointly satisfied:
\(\phi\) is smooth 3 - small changes in voter preferences should result in small changes to \(u\)
\(\phi\) respects anonymity - e.g \(\phi(p_1, \ldots, p_K) = \phi(p_{\sigma(1)}, \ldots, p_{\sigma(K)})\)
\(\phi\) respects unanimity - if \(p_1 = \cdots = p_K= u\) then \(\phi(p_1,\ldots,p_K) = u\)
We’ll prove the simplest case of two voters and two candidates, which can be naturally generalized. In this scenario our preference space \(P = S^1\) forms a unit circle and \(\phi: S^1\times S^1 \to S^1\) maps the torus to a circle. The high level idea, is to form two different paths along the torus that have different degrees modulo 2 under \(\phi\) and show that these paths can actually map onto one another forming a contradiction.
For the first path, consider the diagonal \(D = \left \{ (\alpha, \alpha) : \alpha \in S^1 \right\}\) which visually loops around the torus at an angle, such that it rotates along the inner and outer loops exactly once by the time it closes. If we restrict \(\phi \vert_D\), then by the unanimity condition we know that \(\phi\vert_D(\alpha,\alpha)=\alpha\) is one-to-one which satisfies \(\deg \phi\vert_D = 1\).
For our second path, let’s take \(A = \{\alpha\} \cup S^1\) and its symmetric counterpart \(B = S^1 \cup \{\alpha\}\) for some \(\alpha\in S^1\) which form an orthogonal figure-eight, with \(A,B\) representing circles rotating directly on the outer and inner loop respectively. If we look at the restriction \(\phi \vert_{A\cup B}\), then again by symmetry, we see that \(\deg \phi \vert_{A\cup B} = 0\ \text{mod}\ 2\) at any regular value by pairing points in the preimage from both components of the figure-eight. Clearly \(A\) and \(B\) intersect at just the one point \((\alpha,\alpha)\in D\) non-smoothly but otherwise form a connected path. So if we take a small \(\epsilon\) sized smooth deformation at \(D\cap (A\cup B)\) to shift away from the intersection at \((\alpha,\alpha)\) to form a non self-intersecting loop \(L\), then \(\deg \phi \vert_L = 0 \ \text{mod}\ 2\) at all of its regular points as well.
Finally, we can find a homotopy to smoothly deform \(L\) into \(D\) (they’re both just loops now) which produces a contradiction as \(\deg \phi\vert_D \neq \deg\phi\vert_L\)!
This notion of taking the diagonal and paths in \(P^K\) naturally generalizes when you look at degree mod 2. Here, we had to rely on smoothness for our path homotopy invariants to apply, but some heavy machinery homological approaches also extend this result to continuous settings.
Only continuity is actually required, but you can approximate the continuous map arbitrarily closely with a smooth one, and this simplifies the proof ↩
I’ll attempt to convince you, based on a simplified mathematical model, that seeking counterfactual impact implies prioritizing fields which have opportunities for heavy-tailed contributions regardless of the talent distribution.
Let’s start with a simple model of the world. For a given altruistic field \(F\) (e.x medicine), assume talent is distributed from some underlying distribution \(\mathcal{D}\), and that \(F\) is supplied constrained with a fixed number \(N\) of contributors.
Let \(X_1, \ldots, X_N \stackrel{\text{i.i.d}}{\sim} \mathcal{D}\) represent the impact of the \(N\) contributors to \(F\), and being humble for now, we assume that in expectation we’re identical to everyone else throwing their hat in the ring, letting \(X_{N+1}\sim \mathcal{D}\) represent our own impact. What is the counterfactual impact in this scenario?
Well, taking the difference between the worlds where you do and don’t contribute, we see that the counterfactual impact \(I\) is expressed as:
\[\begin{align*} I &= \mathbb{E}[ \sum_{i=1}^{N+1} X_i - \min(X_1, \ldots, X_{N+1}) ] - \mathbb{E}[ \sum_{i=1}^{N} X_i ] \\ &= \mathbb{E}[ X_{N+1} - \min(X_1, \ldots, X_{N+1}) ] \end{align*}\]When is this value large?
Well, if the distribution \(\mathcal{D}\) has a heavy left tail, then the difference between the average and minimum values will be large, and so the counterfactual impact will be large. But, generally speaking, impact is a non-negative quantity, so the counterfactual impact here is upper bounded by \(\mathbb{E}[X_{N+1}]\), which naively represents a maximum relative change of \(\frac{1}{N}\) percent to the field!
For any significantly sized field, this implies that your relative counterfactual impact is negligible if you don’t expect to significantly outperform the average. For many, relative impact is also a proxy for how much pride and value you can derive from your work, so this is a fairly pretty depressing result when the peer group is \(F\) itself.
Well now, of course no self-respecting ambitious altruist aims to achieve just an average impact. So, let’s graciously assume that we’re the top of the top, best of the pack, and implicitly condition on \(X_{N+1} = \max(X_1,\ldots, X_{N+1})\). What does this imply for our counterfactual impact?
Again, taking the difference between the worlds where we do and don’t contribute we find:
\[\begin{align*} I &= \mathbb{E}[ \sum_{i=1}^{N+1} X_i - \min(X_1, \ldots, X_{N+1}) ] - \mathbb{E}[ \sum_{i=1}^{N} X_i ] \\ &= \mathbb{E}[ \sum_{i=1}^{N+1} X_i - \min(X_1, \ldots, X_{N+1}) ] - \mathbb{E}[\sum_{i=1}^{N+1} X_i - \max(X_1,\ldots, X_{N+1})] \\ &= \mathbb{E}[ \max(X_1,\ldots, X_{N+1}) - \min(X_1,\ldots, X_{N+1})] \\ &\leq \mathbb{E}[\max(X_1, \ldots, X_{N+1})] \quad \text{since } X_i \geq 0 \end{align*}\]As expected, our counterfactual impact in this scenario is strictly greater now that we’re the best. Intuitively, distributions for which this maximum can be large relative to average is what we’re looking for – effectively, heavy right-tailed distributions. Here, we easily see that the relative impact is now instead given by
\[\begin{align*}\frac{\mathbb{E}[\max(X_1, \ldots, X_{N+1})]}{N \mathbb{E}[X_i]}\end{align*}\]Interestingly, for distributions in which
\[\begin{align*} \lim_{N\to \infty} \max(X_1, \ldots, X_{N+1}) - \max(X_1, \ldots, X_N) > \mathbb{E}[X_i] \end{align*}\]we see that it’s actually better to choose fields with larger \(N\), where our assumption of being the best is stronger and utilized with greater effect.
To help give some intuition where this phase transition might apply, consider \(\mathcal{D}\) explicitly as the Pareto Distribution1 with parameter \(\alpha\), which is widely used for modelling talent and real world output distributions. For \(\alpha > 1\) (required for finite mean), a quick lookup indicates that \(\mathbb{E}[X_i] \sim \mathcal{D}_\alpha = \frac{\alpha}{\alpha-1}\) and the CDF and inverse CDF are given by
\[\begin{align*} F(x) &= 1 - \frac{1}{x^\alpha} \\ \implies F^{-1}(x) &= \frac{1}{(1-x)^\alpha} \quad \text{Convex on } [0,1] \end{align*}\]Now since we can view \(X_1,\ldots,X_{N+1}\sim \mathcal{D}_\alpha\) as first sampling uniformly in percentile space, and then applying the inverse CDF \(F^{-1}\), this gives us a very simple route to determining the counterfactual impact for a given \(N\) and \(\alpha\). From properties of the uniform distribution, we know that \(\mathbb{E}[\max(F(X_1),\ldots, F(X_{N}))] = 1 - \frac{1}{N}\), and so applying some harmless trickery (to avoid real math), we see that
\[\begin{align*} I &= \mathbb{E}[ \max(X_1, \ldots, X_{N+1}) - \max(X_1,\ldots, X_{N})] \\ &= \mathbb{E}[ \max(F^{-1}(F(X_1)),\ldots, F^{-1}(F(X_{N+1}))) - \max( F^{-1}(F(X_1)),\ldots, F^{-1}(F(X_{N})))] \\ &= \mathbb{E}[ F^{-1}(\max(F(X_1)),\ldots, F(X_{N+1})) - F^{-1}(\max( F(X_1),\ldots, F(X_{N})))] \\ &\geq F^{-1}(\mathbb{E}[ \max(F(X_1)),\ldots, F(X_{N+1})])- F^{-1}(\mathbb{E}[\max( F(X_1),\ldots, F(X_{N}))]) \quad \text{ by Jensen's} \\ &= F^{-1}(1 - \frac{1}{N+1}) - F^{-1}(1 - \frac{1}{N}) \\ &= (N+1)^\alpha - N^\alpha \\ &> \alpha N \end{align*}\]Surprisingly, even for the Pareto Distribution we see that for \(N \geq \frac{\alpha}{\alpha-1}\) that it becomes better for us to choose fields with larger \(N\) in terms of relative impact! Notably, for most reasonable values of \(\alpha\) this phase transition begins at surprisingly low values of \(N\).
Now, sticking with the Pareto Distribution, let’s consider a slightly more realistic scenario . Suppose that we’re not the best, but instead the \(z\)-th percentile of the field, and we enter regardless of whether or not we’re worse than everyone else. What does this imply for our counterfactual impact?
To start, note that this implies \(F(X_{N+1}) = z\). Manipulating our previous result to use \(F^{-1}(z)\) instead of \(\max(X_1,\ldots,X_{N+1})\), it’s easy to see that the counterfactual impact is now given by
\[\begin{align*} I &= F^{-1}(z) - \mathbb{E}[ \max(X_1,\ldots, X_{N})] \\ &\geq \frac{1}{(1-z)^\alpha} - F^{-1}(1-\frac{1}{N}) \\ &= \frac{1}{(1-z)^\alpha} - N^\alpha \end{align*}\]So in terms of \(N\), the phase transition at which it becomes better to choose fields with larger \(N\) for maximizing relative impact is specified by
\[\begin{align*} \frac{1}{(1-z)^\alpha} - N^\alpha &> \frac{\alpha}{\alpha-1} \\ \implies z &> 1 - \left( \frac{1}{N^\alpha + \frac{\alpha}{\alpha - 1}} \right)^\frac{1}{\alpha} \\ &> 1 - \frac{1}{N} \quad \text{ for } \alpha > 1 \end{align*}\]which effectively equates to being the best in the field. This has the surprising conclusion, that unless you really are going to be the best at what you do, the relative impact of your work will be maximized in smaller, less competitive fields, even for common heavy tailed distributions.
These are pretty simplified models, but still further advance my intuition towards seeking tail upside opportunities (confirmation bias, possibly?) to maximize net impact, and pursuing less competitive, smaller contexts to maximize relative impact. Reality is, however, that most of the time, tail opportunities and performance aren’t particularly accessible to most and aren’t nearly as random as I’d like, and so these insights should be taken with a grain of salt.
In the future, some simple extensions of this model to that would be fun to look into include: sampling \(K\) people applying to join field \(F\) and only choosing the top \(N\), introducing multiple competitive fields to choose from, and adding uncertainty to your own performance estimates.
Since at any given time, I found myself optimizing between these categories simultaneously, I’ll present a quarter by quarter rating of each with some brief commentary and use (*) to highlight particularly high quality experiences or activities.
Rating:
Coursework (20): Econ 1, CS 106X, Math 113*, Freshman Writing Seminar (ESF)
Adjusting to Stanford was fairly difficult my first quarter. As a natural introvert and lifetime cynic, finding social groups outside of dorm activities or in clubs was tough, and I ended up having large blocks of free time by myself which was undesirable.
Highlights:
Mistakes:
Rating:
Coursework (22): Math 62CM*, CS 161, CS 107, CS 43, CS 22, PHIL 120
Highlights:
Mistakes:
Rating:
Coursework (22): Math 120, Math 147, Physics 65, CS 166*, CS231N*, CS142
One of my goals throughout freshman year was to find my intellectual/productive limits, which I aimed to accomplish by upping my course load difficulty each quarter until I broke. This quarter was by far my most taxing and difficult throughout college, due to the quantity of work and my own lacking academic maturity, which firmly had me reach the goal of breaking. Most of my memories from this quarter are whirlwinds of psetting in the dark on limited sleep with friends or alone, with little sunlight or external social activity outside of the first week of classes.
Highlights:
Mistakes:
Rating:
Internship: Worked at a tech startup (now defunct) as a SWE intern with one of my closest friends. I did a hybrid internship, commuting for a week at a time between home in SoCal and Redwood City (staying at my friend’s place). This was a great, non-demanding summer, where I really got the chance to relax outside, review material and learn new things for fun, and hang with friends.
Highlights:
Mistakes:
Rating:
Coursework (22): Math 171, Math 220, Physics 110, CS 221, EE 263, Sophomore Writing Seminar (PWR 2)
Another fairly academic quarter, where I tried to bias more towards applied subjects (Physics and EE), after realizing in Spring that competition in pure theory subjects was brutal. Notably, this was the last quarter I took at Stanford where I was primarily invested in academics – school took less of a priority after I realized math/physics PhD wasn’t necessarily the route I wanted, and actively engaged more in social life, and light exploration of several other academic fields/career choices.
Highlights:
Mistakes:
Rating:
Coursework (15): Math 121, Physics 230*, EE 364A*, CS 255, ENGLISH 146A
After realizing my health had suffered during the past Fall/Spring, I decided to take a lighter course load. This proved to be a great decision, and I enjoyed spending the free time socializing with my roommate and other friends. At this point, I also took some of that free time and joined my first club, Christian Intervarsity (IV), where I met a lot of phenomenal people and friends I’ve still kept in touch with. Coincidentally, the courses I took ended up being very high quality, with Physics 230 being my favorite individual course at Stanford, and EE 364A being one of the most useful applied courses taken.
Highlights:
Mistakes:
Rating:
Coursework (19): Physics 231, CS 272, CS 199, CS 224U, CS 229*
This quarter had a ton of free time which I spent socializing and meeting new people – which has had pretty high returns on my current social life quality. Started my first ever relationship, which I learned a ton about myself from.
Highlights:
Mistakes:
Rating:
Internship: Quant Trading in London with my roommate and continued part-time research in Genomics.
Highlights:
Mistakes:
Rating:
Coursework (20): CS 273B, CS 330, CS 279, ECON 180*, Math 230A*, CS 199
I continued to lean into the idea of working in Biotech or going to Med School, by exploring some of the computational aspects of genomics/biology/chemistry. This was a really clear example of fruitful exploration, as by the end of the quarter I was fairly well convinced that this wasn’t the right path for me – this may change over time though, and I’m still intellectually curious about the space!
Highlights:
Mistakes:
Rating:
Coursework (17): CS 259Q, CS 224N, CS 261*, Math 143, Phil 151
The first 8 weeks were pre-covid, and consisted of a lot of fun continuation of the themes in Fall, including the increased emphasis on social life.
Highlights:
Mistakes:
Rating:
Coursework (21): Math 228, EE 180*, CS 110, CS 399, Music 25, German 101
Internship: Wealthfront*
Research : RL Research with Dorsa*
Realizing Covid was going to crush any hope of an on-campus experience, I hunkered down and decided to go pretty hard on my work. In hindsight, I really should have taken it easier as I was not in a great mental state (along with everyone else isolated during covid), and really owe it to my family for helping support me throughout.
Luckily, I finally hit the jackpot in terms of mentorship, and my intern manager at Wealthfront was phenomenal despite the pandemic disruptions, and had a really engaging, cool project. Likewise, for my second attempt at academic research I joined a new project in Dorsa’s lab focused on pure RL, and this time I prioritized mentorship and a good team which dramatically improved my experience and output.
Highlights:
Mistakes:
Rating:
Internship: QuantCo
Research: Continued RL Research with Dorsa*
In summer, I prioritized hanging out (socially distanced) with friends, or in organized online gatherings with some consistency. Remote internship ended up being not very time intensive, which I took advantage of to spend more time with research and playing outside.
Highlights:
Mistakes:
Rating:
Coursework (12): Math 116, Math 215A*, Physics 330, Physics 170
Internship: FB
Lived with some friends in a house during a virtual quarter to try and make the best of quarantine. My memories of this time are very physically dark, as the house had very few windows and we rarely went outside (stayed in a bad neighborhood) and were concerned about Covid. At this point, I had enough to graduate with the Bachelor and Coterm, but for lack of motivation to go and work, decided to just stick it out and play essentially through Spring with the rest of my house/classmates – I think staying was potentially one of the best decisions I made during Stanford, as leaving during Covid would have really stifled any opportunity to rekindle relationships.
Highlights:
Mistakes:
Rating:
Coursework (24): Math 236, Physics 220, Physics 113, EE 276*, CS 228, Applied Physics 228, Music 19A
Continued living mostly with the same group plus some other friends in a safer, cheaper location out of state where we could afford a bigger place. Hung out and went outside to hike and eat out more as quarantine restrictions relaxed slowly which drastically improved quality of life. Because I was feeling pretty lethargic from all of my video games and free time in Fall, I decided to go a bit harder on coursework – this backfired halfway through the quarter, as I got really pulled into the whole GME saga and ended up spending a majority of my time learning about finance, staring at flashy charts, reading options theory and textbooks, and other dumb things (classic “Ryan is a sheeple” example).
Highlights:
Mistakes:
Rating:
Coursework (12): EE 378C, Math 159, Math 215C*, Stats 375
Senior Spring was my victory lap quarter with school finally brought back on campus. With no academic or work requirements left, I heavily prioritized social life and outdoor activities. This quarter is likely the most vibrant and happiest I’ve ever been.
Highlights:
Mistakes:
After writing and reflecting on the above, I think it’s pretty clear to me that there are certain patterns that have held up in terms of improving health and happiness. Some obvious ones are the frequency of social interaction, the quality and intellectual engagement of academic course loads, and having solid teams and mentorship to work with.
Some that seem obvious now but clearly were not in retrospect include the quantity of time spent outdoors and the consistency of physical activity. Clearly, I was hanging out with friends and playing games quite a bit, but why didn’t I do those activities outside? Smh.
There are also some things I spent a lot of energy on that, given my current path, seem wasted, but weren’t quite so clear at the time. I spent a solid few quarters on genomics research and learning computational approaches in biology/chemistry, which I’ll likely never use. But, these types of things, I suppose, are only clear in hindsight. Between my FB internship and genomics research, I realize that quitting was certainly the right move, but in the future, it might be better to bias even more towards quitting earlier3. Likewise, I now realize that classes/internships which I expected to be fluff in advance… usually were, in fact, wastes of time (surprising, right?). And the only thing I got out of it was deceiving myself, unfortunately. Finally, recognizing now that herd mentality and paths of least resistance are more of a force in my life than I originally thought, and that I should actively try to combat these biases in how I currently allocate my time and in my plans for the future.
Taking this time to write down thoughts and self-reflect in itself has been quite rewarding, and I hope anyone who reads this may find it as insightful for themselves in terms of approaches to replicate or avoid, as I have.
Best Paper announcement ↩
Ben Kuhn has a great article on optimal control and stopping times in the context of startup options, which illustrates the advantages quitting confers. Generally, quitting earlier than you’d think, is likely closer to the optimal route (at least for myself!) ↩
In python, it’s often the case I find myself wanting the ability to very quickly construct dictionaries with nested structures. Unfortunately, the following use case is not easy to accomplish with python dictionaries used in the standard way:
dct = {}
dct['a'] = 1
dct['deep']['nested']['auto']['inferred']['structure'] = 2
print(dct)
"""
{
'a': 1,
'deep': {
'nested': {
'auto': {
'inferred': {
'structure': 2,
}
}
}
},
}
"""
After a little bit of exploration with the python defaultdict collection, I found that this is easy enough to accomplish with some fun recursive trickery.
from collections import defaultdict
def reflect():
return defaultdict(reflect)
dct = defaultdict(reflect) # Now works perfectly with the previous example