Tech on Basil Labib

Can discrete Ricci flow signal capability transition in Transformers?

gs454236@gmail.com (Basil Labib) — Sun, 11 Oct 2026 18:23:49 +0530

Abstract

Headline result. A: the layer coefficient through grokking, per transition (thin) and mean (thick), against test accuracy. B: the same coefficient for a grokking run and both training controls. C: the calibrated excursion for every run in the study, grouped.

Researchers have shown that the feature geometry of a feedforward neural network evolves as discrete Ricci flow where a node’s curvature anti-correlates with the expansion of its neighbourhood across layers (Hehl et. al. 2025). Apart from establishing a connection between machine learning training dynamics and loss landscapes with differential geometry, this result also presents new heuristical methods for early stopping and efficient model architecture selection informed by geometric feature evolution.

In this blog post, I asked the question: does the same geometric evolution results hold when the techniques are applied to a decoder-only transformer architecture (Vaswani et. al. 2017)? Based on my limited resources and time, I was able to achieve partial success where I show that indeed some aspects of the feature manifold evolve as Ricci flow, whereas some do not. Application of the techniques as presented in the paper presented non-trivial difficulties when applied to the transformer. This post elaborates how I overcome those issues, presents the results with some plausible explanation and notes the limitations and future work.

Introduction

Stylised representation of the evolution of the data manifold and the corresponding discrete graphs. Source: Hehl et. al. 2025

A neural network transforms the input data manifold through complex non-linear transformations. This sequence of transformation may be viewed as an evolution or a flow on the geometry of the data point cloud. Hehl et. al. studied this phenomenon and showed that for feedforward network trained on both real and synethetic datasets on the task of binary classification exhibit manifold transformation consistent with discrete Ricci flow where the surface smoothens into highly curvatured surfaces corresponding to the specific class.

Since the manifold itself is difficult to visualise for a dataset, the authors built discrete graphs at each layer using techniques such as k-nearest neighbour and r-distance graphs. Moreover, they assigned each node a discrete scalar curvature and studied how the curvature changed with respect to the neighbourhood across the layers.

We asked if this phenomenon transfers to transformers also? And if yes, can the geometry move actually signal a capability transition? We used grokking on modular arithmetic as the motivating problem since it is fairly well-understood and is not resource-intensive to learn. Grokking refers to the observation of sharp decline in network loss, representing moments when the network “learns” or achieves “epiphany” in task comprehension and performance.

The transfer from a simple feedforward neural network training for binary classification to auto-regressive transformer blocks was not straightforward. Some of the difficulties faced were:

Simple MLP are represented as complete transformation of the form, $ x \to g(x) $ . Transformers actually “modify” the residual stream instead of transforming it. This is better represented as $ y \to y + h(y) $ which means that consecutive layers carry non-negligible feature remnants from past streams.
Transformer sublayers alternate in kinds, namely attention and MLP blocks $y = \phi_1 \circ \phi_2 \circ \cdots (x)$ rather than simple chained MLPs $y = \phi \circ \phi \circ \cdots (x) $ .
The representations are per-token across the context length. Therefore, interpreting the idea of a scalar curvature at layer $l$ only makes sense if we fix a read-out position along the context.

Key results

We found that the layer Ricci coefficient indeed tracks the transition across layers.
The local Ricci coefficient measured at a node through training stays around zero in transformers at every depth.
Some part of the above discrepancy can be explained by the presence of residual connections in transformer blocks. For this, we observed local $\rho$ differ by +0.13 to +0.21 covering about two thirds of the feedforward-to-transformer gap. This suggests that comparing architectures that differ in skip connections may not be a good idea.

Preliminary background

Discrete curvature on a feature graph

A curve with both large and small curvatures. Source: Gab Khan

As mentioned before, we pick a subset of activation at each layer $ X_l = \{ \textbf{x}_i \}_{i=1}^n $ and build an undirected graph $G_k(X)$ based on the k-nearest neighbour approach. There are many metrics mentioned in Hehl et. al. to formalise the idea of curvature (See Ollivier-Ricci and Forman-Ricci curvatures). We present the augmented Forman-Ricci curvature of an edge which is computationally more efficient to compute:

$$ AF_l(u,v) = 4 = deg_l(u) - deg_l(v) + 3 | N_l(u) \cap N_l(v)| $$

where $N(u)$ represents the set of neighbour nodes of $u$ and the scalar curvature of the node $\kappa$ is just the mean of curvature of its incident edges.

Across layers, as the same neighbourhood of a node expands, it would mean that the nodes are more spread apart. This can be formalised as an average of the differential shortest path distance between two nodes across the two graphs at layers $l+1$ and layer $l$.

$$ \eta_l(x) = \frac{1}{deg_l(x)} \Sigma_{y \sim x_l} [d_{l+1}(x, y) - d_{l} (x, y)] = \frac{1}{deg_l(x)} \Sigma_{y \sim x_l} d_{l+1}(x,y) - 1 $$

since y in same layer as x implies $d_l(x,y) = 1$. A node’s $\eta$ is large when its layer $l$ neighbours are pushed far apart in layer $l+1$.

The local and layer Ricci coefficients

Ricci flow contracts positively curved regions and expands negatively curved regions so there is an anti-correlation between $\eta$ and $\kappa$. This can either be observed as a Pearson coefficient between $\kappa$ and $\eta$ at each layer $l$, namely $\rho_{layer}(l)$ or at a specific node across $T$ depth transitions, namely $\rho_{local}(x)$.

For feedforward networks, the authors found that $\rho_{local} \in [-0.40, -0.76]$ and $\approx 0$ for untrained networks.

Grokking to represent capability transition

We study the problem of modular addition $c = (a+b) \mod p$ where a small transformer is trained with training set fraction pairs over few hundred steps. Nanda et. al. showed that the network abruptly generalizes and that the underlying mechanism for grokking involves principled circuits that can be cleanly interpreted as algorithmic steps. This problem is suitable for studying capability transition due to its fairly well-understood nature.

Where to read a transformer’s “layers”?

Unlike a MLP, the transformer’s output is a residual stream per-token. Our sequence is a, b, = We read the prediction at the final position and log the residual stream at each sublayer boundary, namely

resid_pre_0, resid_mid_0, resid_post_0, resid_pre_1, ..., resid_post_L-1

Experimental setup

Task: We used modular addition $c = a + b \mod p$, the input sequence was $a, b, =$, and the final position was read as model output.

Model: We used a decoder-only transformer, 4 heads, $d_{head} = 32$, $d_{mlp} = 512$, no Layer Norm, full batch AdamW, $lr = 10^{-3}$, weight decay 1.0, $\beta = (0.9, 0.98)$, 30,000 steps. $d_{model} \in (64, 128, 256) at L=4$. Total parameter counts were 0.41M / 0.82M / 1.64M. $L = (1, 2, 4, 8)$. We used various seed configurations across runs and swept accordingly.

Hardware: All experiments on a M4 Max (36 GB) using MPS.

Results

A: the layer coefficient through grokking, per transition (thin) and mean (thick), against test accuracy. B: the same coefficient for a grokking run and both training controls. C: the calibrated excursion for every run in the study, grouped.

Curvature against mechanistic progress measures computed from the same run’s weights. A: the three phases read off the weights - accuracy, the embedding’s Fourier concentration on the five key frequencies, and the weight norm. B: the layer coefficient (faint: raw; bold: 7-checkpoint rolling median) with the four events marked. Most of the drop precedes saturation; the coefficient then deepens further through the cleanup window and bottoms out just after it.

One of the surprisings results is the above figure which shows that the mean layer coefficient would lead the rise in test accuracy but it actually lags behind by around 900 steps. We found that about half of the geometric reorganization happens when test accuracy has fairly stabilized. Across 15 runs, a median of 50% of the coefficient changes after 99% test accuracy is reached.

Coefficients should be interpreted differentially

A: the local coefficient post-saturation for every well-conditioned configuration, with leave-one-transition-out ranges; the grey band is the range reported for trained feedforward networks. B: matched plain versus residual MLPs at initialization.

Hehl et. al. reported the local coefficient as near zero for untrained networks. However, this does not hold for our setting and therefore, the numbers above should be interpreted differntially rather than on absolute terms.

Configuration	Architecture	$ \rho_{local} $ at init	$ \rho_{layer} $ at init
transformer, $L=4$	transformer	$-0.158$	$-0.322$
transformer, $L = 8$	transformer	$ -0.232$	$-0.358$
transformer, $L=4, d=64$	transformer	$-0.255$	$-0.355$
MLP, $L=4$	MLP	$-0.435$	$-0.224$
MLP, $L=8$	MLP	$-0.637$	$-0.312$

Effects of residual connections

Why is there a feedforward-to-transformer gap in the first place? Can we explain it? The obvious differing mechanism is the residual stream so we constructed similar MLPs with identical depth, width, init scheme, and only differing whether the block transforms input as $h \leftarrow \sigma(Wh)$ or as $h \leftarrow h + \sigma(Wh)$.

Depth	Transitions	$ \rho_{local} $ plain	Residual	Shift	$ \rho_{layer} $ plain	Residual	Shift
4	3	$-0.450 \pm 0.055$	$-0.318 \pm 0.031$	$\mathbf{+0.133}$	$-0.293 \pm 0.042$	$-0.226 \pm 0.018$	$+0.067$
8	7	$-0.588 \pm 0.034$	$-0.361 \pm 0.030$	$\mathbf{+0.226}$	$-0.343 \pm 0.031$	$-0.237 \pm 0.016$	$+0.106$
16	15	$-0.638 \pm 0.018$	$-0.430 \pm 0.026$	$\mathbf{+0.208}$	$-0.387 \pm 0.037$	$-0.296 \pm 0.007$	$+0.091$

Residual connections account for 0.227 of the 0.356 gap (about two thirds). The unexplained part may be transformer specific like attention mixing, single position read-out, no LayerNorm.

The local coefficient represents an absolute level change across transitions at a particular node which is already contaminated at init time due to architectural differences so as expected, it does not transfer. However, the layer coefficient is a quantiy measuring change between layers so the constant init offset cancels. This suggests that architectures differing in skip connections may not make sense to be compared.

Discussion and Limitations

As discussed, $\rho_{local}$ correlates the curvature change around a fixed datapoint across depth. The additive residual stream and skip connections results in a negative result for this metric. The residual MLPs that we train separately confirms that about two-thirds of the unexplained difference can be explained by a constant offset at init time and suggests that measuring absolute local $\rho$ values across incomparable architectures is not a good idea.

Due to how we defined $\rho_{layer}$ as a change between layers, it remains immune to constant offset at init time. Geometric statistics measuring cross-data behaviour can survive residual connections but not cross-depth behaviours.

Even the layer coefficient results are tepid as they lag behind grokking phase for our task. Obviously, this is just one task on a very small model, future work should explore expanding this scope to more architectures, bigger model sizes, and run more ablation studies.

Conclusion

We started out wondering if the feature geometry evolution dynamics transfer to transformers as cleanly as they exhibit in simple MLPs as demonstrated by our motivating paper. The final results are a mix of some positives and some negatives with our main motivation of developing feature geometry based training heuristics renders null as we find that there is a lag in geometry evolution and generalization, conditional on the one task we study in this post.

Moreover, the original framing of an across-depth metric does not make sense for an architecture with skip connections and an additive residual stream. This should motivate more work towards developing alternative metrics to study the feature geometry evolution dynamics in transformers and other sister architectures.

AI usage disclosure

This post is entirely written by me. All experiments were designed, implemented and ran by Claude Code. The figures were generated by Claude. All experiments ran under my oversight.

How to cite this work

Labib, Basil, 2026 October. Can discrete Ricci flow signal capability transition in transformers? Basil Labib's blog. https://basil08.github.io/post/can-discrete-ricci-flow-signal-capability-transition-in-transformers/

References

Hehl, Moritz, Max von Renesse, and Melanie Weber. “Neural feature geometry evolves as discrete Ricci flow.” arXiv preprint arXiv:2509.22362 (2025).
Nanda, Neel, et al. “Progress measures for grokking via mechanistic interpretability.” arXiv preprint arXiv:2301.05217 (2023).
Gabriel Khan, “AN Illustrated Introduction to the Ricci Flow, Third Version” https://differentialgeometri.wordpress.com/2022/01/13/an-illustrated-introduction-to-the-ricci-flow-third-version/

Understanding System One Models Using Interpretability Tools

gs454236@gmail.com (Basil Labib) — Sun, 04 Oct 2026 21:44:41 +0530

Disclaimer: This post is for educational and purely research purposes. I acknowledge that some of the results may be used to construct adversarial attacks on System One models. However, I believe that the builders of these models are already aware of these attacks and a patch or defense would appear sooner than any meaningful attack may occur. If that is not the case, then this post generates evidence towards prioritising that area of defense research building more inclusive guardrails.

In this post, I study non-autoregressive decision models or so-called System One models which are specially designed architectures for cheap, calibrated decision-making, intended as control modules embedded inside larger agentic systems. This is an exploratory study conducted over the weekend applying all forms of interpretability tools to better understand Sys1 model behaviour and to suggest design choices for more efficient or safer deployment practices.

Please reach out to me at gs454236 [AT] gmail [DOT] [COM] for constructive criticisms, questions, or thoughts.

What are System One Models and Why Should You Care?

System one model architecture block diagram. Credits: Unzip.dev

System One models, named so after Daniel Kahnemann’s popularization of the phrase as a subconscious thinking module within the brain as opposed to the more verbalized System two component, are a class of non-autoregressive decision-making language models that output only a calibrated probability distribution over a predefined set of choices given a text input formatted as a predefined schema. The appeal of such models stem from their ability to give sub-second decision making them an order of magnitude faster than conventional language models as well as cheaper (since you can bypass the whole test-time inference cost). These models came into the spotlight with TypeSafe AI’s announcement of Jev-1.13 , however, the claims are disputed as an Indian engineer claims to have built the architecture an year earlier already.

With recent studies on using Jev for LLM-as-judge, an application area notoriously difficult for language models due to uncalibrated decision across batches and prohibitive latency to be useful for current long-running agentic tasks, these models still remain underexplored as a new model architecture.

To the best of my knowledge, this is the first attempt to apply mechanistic interpretability tools developed by researchers across the years and try to pry open the brains of Sys1 models in order to understand them better and in the process, to make them better and safer too.

Setup

In order to facilitate whitebox interpretability, we¹ selected Jev-9B, an open student model of Jev-1.13. It is a frozen Qwen3.5-9B model with a LoRA adapter trained on Jev’s probability outputs. Using this architecture allowed us to compare base Qwen with Jev student with just disabling the adapter. we also used Tev1-4B, Tev1-0.8B, both open models from Together AI. We prioritized open models of varying sizes to have a variant study as well as I wanted to see applicability of interpretability tools on these models and identify blind spots. Some models to explore are CLM-8B, Clef, as well as Jev-1.13 itself.

All the experiments were written and run by Claude Code. I supplied the research questions, provided human oversight as well as guided key design decisions. This blog post is written by me, entirely. The diagrams are generated by Claude. The full experiment took over 12 hours on my Mac M4 Max. I did not rent any cloud GPU.

Methodology

As noted above, we mainly focused on open student models of 0.8B, 4B, and 9B sizes as well as their Qwen base models to serve as control. We constructed four datasets, some from Jev’s training corpus and held-out families as well as some constructed scenarios across various epistemic modalities (ambiguous, missing evidence, etc.), an injection dataset constituting prompt-based attacks as well as pairs with one number change in the prompt state.

We employed pretty much all the tools out there under the sun as the goal was also to see what worked and what didn’t on these seemingly new class of model organisms. I trained linear probes, decision lens, used activation patching, SAEs, crosscoders, and MAttr to quantify model internals, beliefs versus the model output. I used the model’s output confidence as baseline whereever applicable and good ‘ol bag-of-words classifier on raw text.

In particular, I was interested in the following questions (findings and discussions follow in their respective sections):

At which layer does the decision form? Can we use a logit lens approach to find formative layers?
Does the distillation impart new knowledge to student models? What does distillation change exactly?
Is the model internally consistent with its output when it comes to reporting errors?
What are the ASR for common prompt injection techniques on Sys1 models?
Do the model modalities share a common intensity direction? Can it be steered?
Are System One models really easier to interpret?
Can we exploit model internals to build efficient guardrails?
Does the model force a correct answer when there is none?
Are System One Models option shuffle invariant?
How small can a System One adapter be?

When does decision form?

We applied a decision lens (similar to a logit lens) to all 32 layers of the Qwen base model as well as activation patching to find the most causative layer when the decision is flipped.

We found that across the three model decision primitives (choice, score, noul) Jev-9B consistently formed the final answer post layer 24. Across 40 pairs, we found all big models fix the decision gradually over the latter layers – calculated as the median fraction of answer change by patching each layer.

Decision lens on Jev-9B

Activation patching to find the depth at which decision is formed

Does distillation impart new knowledge?

As we focused on distilled models on open base models, I was curious to understand if distilling added new knowledge to the base model or if it was merely learning a transformed output format. Current evidence points to the latter as we trained linear probes across all layers.

Schema output similarity between Jev and Jev-9B

AUROC curve; linear probe on Qwen base

We also performed linear CKA across layers to study how the hidden states differ. We used Matryoskha Attribution or MAttr to find adapter components that restore most of the model behaviour. MAttr learns weight significance by changing weights from base to trained under a random budget.

For Jev-9B, the adapter changes about 3% of the weight norm but rotates the final layer complete (0.23). Tev class models differ in the middle but the final layers are almost identical to initial layers.

linear CKA across layers; Jev-9B, Tev1 0.8B and 4B

MAttr, gradient, size change, and random ranking methods versus components restored; Model: Jev-9B

Why care anyway? I think the biggest takeaway is that adapters can be much smaller (1.6M instead of 40M) making it far easier to train custom adapters on a per-consumer, per-usecase basis. In particular, engineers should prioritize the middle layers. Results on model reformating existing knowledge indicates using a better base model might yield a stronger model.

Can a probe predict mistakes better than the model’s output?

I was curious to understand if a probe on the hidden state can predict the model’s error better than the model’s output probabilities. It turns out that for bigger models, the probabilities are a good proxy and the probes fall short.

Probe on model error probability versus model output

How robust are distilled System one models to prompt injection attacks?

An overview of the injection bench with reference class state object and the five attack types illustrated with an example.

Attack success rate by style and model

Does distillation make the issue worse?

Attack success rate for Qwen base and Jev-9B

Surprisingly, ablating the hijacking features made the ASR slightly worse than baseline. This is most likely because the SAE has selected for the particular scenario at the decision token and not the underlying mechanism.

Effects of ablating hijacking features found via SAEs

We noticed that the model has three decision primitives – score (select a number within a range), choice (select an option from a set), and noul (a binary yes/no probability).

Is it possible to steer one of these answers by putting pressure on another? In other words, does the model make decisions from some “shared context”?

Under the distilled model setup, this seems to be true. Distillation creates a partially shared direction which can influence ratings by steering the “yes” direction.

Cosine similarity between ‘yes’ and ‘high score’ directions. Needs more investigation.

Applying steering pressure to yes direction raises the score by upto 0.27 levels. Random is control.

Are System One models easier to interpret?

One of the appeals of System one models are that they finish their compute in one single pass as opposed to an autoregressive model. Does this single pass nature make them easier to interpret?

We tried learning an SAE reconstructing the decision state as well as try to find a subcircuit using MAttr. The results are mixed. The trained SAE is indeed smaller but the decision circuit is more dense than the base model.

Jev-9B’s decision state is way more compressible. Y-axis measures reconstruction loss for base and Jev-9B. Trained on 375K tokens, BatchTopK SAE.

MAttr components needed to keep the decision for number-flip pairs dataset. Base versus Jev-9B.

Can we exploit model internals to build better guardrails?

Although this sounds like an enticing idea, initial enthusiasm was dissolved when we tested the guardrails on fresh attacks.

Just blanking two-layer field tokens seems to drop ASR from 37% to 3.6%

ASR by token position that was wiped clean.

ASR on new scenarios. Ouch.

Does the model force a correct answer when there is none?

Doing a softmax over the raw logits yield some probability distribution. If we remove the correct answer, does the model shift the mass to another incorrect answer or to “Don’t know”/“Right answer is missing” option?

We trained a small probe on the hidden state as well as noted the raw score. Both agree with each other, meaning the raw max-logit may be interpreted as a fitness score to the output.

Score and probabilities for Jev-9B for ablated correct answer

Are System One models option-shuffle invariant?

Case 1: Prompt1, choose between Opt1, Opt2, Opt3 Case 2: Prompt1, choose between Opt2, Opt3, Opt1

Does the model output the correct option in both cases? We found that under some margin tolerance, the model flips the answer under the shuffling transformation. Moreover, we also patched the decision token to find if the decision binds to a particular letter. We found that initially the model tracks the option and then post layer 22 binds to the option letter after the decision has formed.

Flip rates per shuffle. Jev-9B

Patched decision token across layers.

How small can a System One model adapter be?

I ask this question in our specific setup since we are using a student model. I think there is already some indication in terms of parameter count but in terms of rank, we trained a dedicated rank-16 and rank-1 MLP adapter. Here are some interesting results for a rank-1 adapter with some controls and the assocaited learning curves.

Rank-1 MLP adapter on layer 6-14 (0.44M parameters) is about 67% close to Jev-9B. The rank-16 (40M parameters) adapter closes only 63%.

The KL divergence to Jev-9B on 150 held-out questions as a function of training data size (upto 1600 rows). We only trained on a fraction of what Jev-9B is itself distilled on (608K rows)

Limitations and Future Work

I think the biggest limitation of the above study is that it is only indicative of potential behaviour and does not quantify associated metrics or behaviour. This is due to the intentional small-scale of the experiments conducted on a tight compute budget and more as an intellectual curiosity project.

We did not use the NTU, decision-injection-bench, or Check Point datasets. Those were gated, could have made some effort :P
The training runs were fractional as compared to baselines which means that numbers are only directionally true and needs to be validated.
We only exclusively focused on open weight models for their appeal to be interpreted. This may or may not be a drawback. Other models to explore are the Jev-1.13 model itself, Stanford’s CLM-8B, and Cloudflare’s Clef, kev, any2jev, shgao, among others.
The attacks were synthetic and constructed. Any respectable jailbreaking attempt should cover the standard jailbreaking suites.
The MAttr code was implemented from paper due to lack of an author reference implementation. This may or may not affect the final results and/or correctness of the results.

Conclusion

System one models are definitely going to be around, not on the surface but as a potential driving backbone for many agentic systems that will increasingly continue to dominate most of our lives in the coming days. I think the need to deeply understand these models cannot be understated.

This post summarizes one of the first attempts to apply tools developed by the mechanistic interpretability community to pry open these new class of models. This is not a conclusive study but should inform researchers and industry practitioners towards some helpful directions to pursue as well as to implement better guardrails and integrate these systems so they are safer and more efficient to use.

To cite this work

Basil. "Understanding System One Models Using Interpretability Tools." (October 2026)
*Basil Labib's Blog*, basil08.github.io, 
http://basil08.github.io/post/understanding-system-one-models-using-interpretability-tools/

References

Please note: This is AI-generated section. Please verify before citing.

A. Models (models, data, tooling)

MAttr: Arora et al., Matryoshka Attribution, arXiv 2609.25518 (Sept_2026).
Zoo (HF): autotrust/JEV-9B, autotrust/JEV-27B, togethercomputer/Tev1-{0.8B,4B}-experimental (+ togethercomputer/tev1 repo for the prompt format), Contrastive-LM/CLM-v0.1-8B, jaredpalmer/kev-0.5b, huaweifeng/any2jev-qwen3-0.6b, shgao/rsi-jev-v*-qwen3.5-2b, Cloudflare/clef*
Corpus: SargeDev/jev-distill-corpus-v3 for the 608K distilled Jev-1.13 outputs
Backbone: Qwen Team, Qwen3 Technical Report, arXiv 2505.09388
Gated DeltaNet (the GDN layers at L14/L22): Yang, Kautz, Hatamizadeh, Gated Delta Networks, arXiv 2412.06464

B. Training recipe

Hinton et al., Distilling the Knowledge in a Neural Network, 1503.02531
Hu et al., LoRA, 2106.09685
Shuttleworth et al., LoRA vs Full Fine-tuning: An Illusion of Equivalence, 2410.21228
Epstein (1969), A Scoring System for Probability Forecasts of Ranked Categories, J. Appl. Meteor.
Guo et al., On Calibration of Modern Neural Networks, 1706.04599

C. What fine-tuning changes

Jain et al., Mechanistically Analyzing the Effects of Fine-Tuning on Procedurally Defined Tasks, 2311.12786
Prakash et al., Fine-Tuning Enhances Existing Mechanisms: Entity Tracking, 2402.14811
Lee et al., A Mechanistic Understanding of Alignment Algorithms (DPO/toxicity), 2401.01967
Kornblith et al., Similarity of Neural Network Representations Revisited (CKA), 1905.00414

D. Lenses, probes, patching

nostalgebraist (2020), interpreting GPT: the logit lens, LessWrong
Belrose et al., Eliciting Latent Predictions with the Tuned Lens, 2303.08112
Alain & Bengio, Linear Classifier Probes, 1610.01644
Hewitt & Liang, Control Tasks for Probes, 1909.03368.
Belinkov, Probing Classifiers: Promises, Shortcomings, and Advances, 2102.12452
Meng et al., ROME / causal tracing, 2202.05262
Zhang & Nanda, Towards Best Practices of Activation Patching, 2309.16042
Heimersheim & Nanda, How to Use and Interpret Activation Patching, 2404.15255
Geiger et al., Causal Abstractions of Neural Networks, 2106.02997, plus DAS, 2303.02536
Schuster et al., CALM, 2207.07061, and Din et al., Jump to Conclusions, 2303.09435

E. Uncertainty

Kadavath et al., Language Models (Mostly) Know What They Know, 2207.05221
Burns et al., Discovering Latent Knowledge (CCS), 2212.03827
Azaria & Mitchell, The Internal State of an LLM Knows When It’s Lying, 2304.13734
Marks & Tegmark, The Geometry of Truth, 2310.06824

F. Attribution and circuit discovery

Kramár et al., AtP*, 2403.00745
Syed et al., Attribution Patching Outperforms ACDC, 2310.10348
Conmy et al., ACDC, 2304.14997
Bhaskar et al., Finding Transformer Circuits with Edge Pruning, 2406.16778
Wang et al., IOI circuit, 2211.00593
Braun et al., Attribution-based Parameter Decomposition, 2501.14926, and Bushnaq et al., Stochastic Parameter Decomposition, 2506.20790

G. Dictionaries (SAEs and crosscoders)

Bricken et al. (2023), Towards Monosemanticity, transformer-circuits.pub
Cunningham et al., Sparse Autoencoders Find Highly Interpretable Features, 2309.08600
Templeton et al. (2024), Scaling Monosemanticity, transformer-circuits.pub
Gao et al., Scaling and Evaluating Sparse Autoencoders (TopK, downstream-loss eval), 2406.04093
Bussmann, Leask, Nanda, BatchTopK SAEs, 2412.06410
Bussmann et al., Matryoshka SAEs, 2503.17547 (name and ordering idea shared with MAttr)
Lindsey et al. (2024), Sparse Crosscoders for Cross-Layer Features and Model Diffing, transformer-circuits.pub
Minder et al., Overcoming Sparsity Artifacts in Crosscoders… (latent scaling ν, the L1 artifact), 2504.02922
Kissane et al. (2024), SAEs (Usually) Transfer Between Base and Chat Models, Alignment Forum

H. Steering (RQ3)

Turner et al., Activation Addition, 2308.10248
Rimsky et al., Contrastive Activation Addition, 2312.06681

I. Prompt injection and persuasion

Greshake et al., Not What You’ve Signed Up For, 2302.12173
Yi et al., BIPIA, 2312.14197
Chen et al., StruQ, 2402.06363
Wallace et al., The Instruction Hierarchy, 2404.13208
Zeng et al., How Johnny Can Persuade LLMs…, 2401.06373

J. Multiple-choice and option order (RQ14, S3-D)

Zheng et al., LLMs Are Not Robust Multiple Choice Selectors, 2309.03882
Pezeshkpour & Hruschka, LLMs Sensitivity to the Order of Options, 2308.11483
Wiegreffe et al., Answer, Assemble, Ace, 2407.15018. They also find symbol binding in late layers, which matches your L23 result, so this one is worth citing directly.
Lieberum et al., Does Circuit Analysis Interpretability Scale? (Chinchilla MC), 2307.09458

Footnotes

1: From now onwards, “we” should refer to me and Claude.

The Lament of a Young ML Researcher

gs454236@gmail.com (Basil Labib) — Tue, 08 Sep 2026 00:42:57 +0530

“I wish there was a way to know you’re in the good old days, before you’ve actually left them.”

~Andy, The Office, S9E23

In 2026, around twenty thousand papers were submitted to ICLR , which is a premier (A-star tier in nerd jargon) conference for machine learning and related research topics. Thousands of researchers submit resarch papers, which are peer-reviewed in a double-blind syste (neither the authors know who the reviewers are nor the reviewers know who the authors are) and form the basis for what direction the larger ML community decides to work on next. The conference attracts both veterans and young researchers teeming with energy to publish the next “big thing”. There are thousands of such conferences organized on topics ranging from theory of mind to chaos theory and the next weirdest thing you can think of.

Figure 1: ICLR submission trends. Source: OpenAccept.org

No bots won’t do your research for you

In the past few years, ICLR and other venues have seen an exponential surge in submissions for a number of reasons. Most notorious of them are, of course, language models. Combined with coding agents and end-to-end autoresearch bots, the cost to produce a “paper-looking” pseudo research artifact is practically zero. I call them pseudo-research artifacts because most modern agents are still only good as assistants, not autonomous researchers. AI companies are exaggerating agents’ use for research because that’s what pays their mounting infrastructure capex. AI “influencers” build cheap scaffolds that claim to do autoresearch for as low as $25. The sad truth is, it’s all bullshit. Having used them extensively on multiple projects, I find agents are token burners because they lack epistemic context; they may sample from vast probability distributions over millions of research texts, but they lack “taste”.

That doesn’t stop younglings from bombarding fake-ass “research” to venues like ICLR, which is straining the ICLR review process , leading to suboptimality in the review process and hindering actual research progress. The fact that the coveted “quants” like Jane Street set up stalls and sneakily siphon off the brightest minds from these conferences also adds to the hype and mystery around them.

Like all things in nature, I believe research quality also follows the normal distribution. This means only about the 98th percentile of research is beyond 2 standard deviations from the average. That’s the tiny minority of consequential, moon-shot, maverick research that advances society. This means that there’s a load of crap. One can produce an “evals” paper and make a new benchmark for as little as $100. I don’t even want to comment on how broken and sorry the state of evals is right now, but with each new paper comes new methodology and introduces new metrics. There’s a benchmark for LLMs generating music to LLMs manipulating objects in AutoCAD .

How many papers were accepted? Around five thousand. That’s a 25% acceptance rate .

Cheating review with LLMs

Like all human things, the review process is famously noisy and random . At the same time, it is the backbone of credible ML research. I would argue it’s even more important than the act of writing and submitting a paper itself. Peer review separates performative fluff from actual research that pushes the frontier of human knowledge. To read a paper and deeply engage with it is almost like reading the mind of another researcher, which is intimate in a non-sexual way. Using LLMs breaks the sanctity of the process and discounts the hard work and thought put into the paper. When a researcher knows their merit is as likely to be acknowledged as random chance, merit dies. Hold your horses, young man

Left hanging

Gone are the days when you could hypothetically sit with a stack of 10 papers over an afternoon coffee and be positively assured you’d be intellectually kindled, learn something new, and have your curiosity’s appetite whetted.

The modern young researcher, with all their zeal, is first expected to answer the gargantuan question of “where to begin”? They cannot imagine rekindling their curiosity. They must sift through the noise, adapt, and learn to call bullshit, developing the impossible skill of an “eye” for plausible good research even before they learn a thing or two about the topic at hand. They must master human psychology before they can write an equation or write a piece of code.

The hopelessness of the situation

Ignorance is bliss
~Cypher, The Matrix

The ICLR interface is notoriously bad for browsing and discovering new research, so people have scrapped the platform and built better interfaces around it. I went to one and searched for “theory”. 500 papers turned up in the search. My heart sank. An average researcher like myself would require a day to fully engage with a paper, building intuition, understanding the code, perhaps trying to implement bits and pieces, reviewing past work, and so on. That’s more than a year of just trying to understand what the state of ML is. But wait, the next edition of ICLR will be due in 365 days, and the research debt would keep piling up. You know the fiendish part of this situation? Alongside the 500 papers, it was written “2%”. It meant I was only looking at 2% of the ~20K papers submitted to ICLR this year.

Okay, you must be silly to read all 500 papers, Basil. Apply the rule of the 98th percentile. That’s around 10 papers. Tractable. But wait, that doesn’t give you diversity. Doesn’t build the “research muscles,” so you expand it to 10%, which is around 50 papers. That’s around two months of effort, which is, to euphemistically put it, two months of a polished, cogwork machine of utmost discipline consuming research for upwards of 6-8 hours a day. Long Sigh.

But wait, remember the “2%”. We are only dealing with theory papers here. What about systems? Multimodal training? Continual learning? Spectral theory? evals? What about the 98% of research?

And my dear friend, this is just one conference. There’s NeurIPS, ICML, IROS, CVPR, EVVC, AAMAS, and 5 other “A-tier” conferences.

The elephant in the room. Picture courtesy: Google, meme is by yours truly.

I haven’t even factored in the fact that as a researcher one is supposed to…well, actually do research. We have only been concerned so far with keeping up to speed with the current interests and trends. The fact is, even if we forgo our own research and become cogwork machines, we will only be able to keep up with at most 2% of research at one conference, or about 0.2% of ML research, assuming we only believe the universe of all A-tier conferences consists of 10 elements, which already costs us 3 months or around 25% of the year.

An average researcher will know only a negligible amount of research. Forget about deeply engaging with it.

Ossification of research interests

What does this lead to? Ossification of research expertise and taste. Researchers tend to specialize rather than diversify. Is it good? Is it bad? Who knows. One camp would argue that deeper expertise with decades of experience in a narrow subfield or even topic is required for a breakthrough, while the other camp (to which I belong, btw) would argue that research is a breadth-first search with many cross-edges and benefits when ideas cross-pollinate between seemingly far-away research disciplines. I mean, look at the history of human science and discovery. The legends and the giants were all polymaths – Da Vinci, Gauss, Bernoulli, Lagrange, Leibniz – they studied the natural sciences but were also masters of mathematics, philosophy, chemistry, biology, astronomy, statecraft, linguistics, and whatnot.¹

There is pretty bad news for a modern-day polymath aspirant.

What has the shape of research become? Dendritic protuberances cluster around local regions in the research state space where most researchers now tend to spend their entire lives. Only occasionally does some braveheart push forward, or some old research get rediscovered, and the frontier advances. In my opinion, the most pressing need of the hour is more ambitious, frontier research.

TODO: INSERT ILLUSTRATION OF SHAPE OF RESEARCH

Which begs the question: what should happen to change the incentives that have nurtured this structure? Has the flood of research itself been the Achilles’ heel for fostering good researchers who produce quality work? I don’t think so. Remember the normal. We need to change its shape.

The “paper” black hole

Alright, all this aside, I would want to present a bitter truth to young researchers like myself. Thought of another way: what are the chances that a young paper is actually read? Assuming there are 5K accepted papers from all 10 conferences, that’s around 50K papers. Assume there are 10 subfields and the researcher only publishes in one of them, so that’s around 5000 papers. What are the chances that the paper is read?

So, given that you magically survive the random lucky draw of acceptance with a 25% chance and the normal distribution of “merit” in research quality, here is the cumulative table Gemini generated:

Distance from Center (Mean)	Benchmark Percentage	Unit Allocation	Cumulative Percentile Range
Far Left (Extremely Low: -3σ to -2σ)	2.14%	107 units	Up to 2.27%
Left Side (Below Average: -2σ to -1σ)	13.59%	680 units	2.27% to 15.87%
Center-Left (Average/Low-Mid: -1σ to 0σ)	34.13%	1,707 units	15.87% to 50.00%
Center-Right (Average/High-Mid: 0σ to +1σ)	34.13%	1,707 units	50.00% to 84.13%
Right Side (Above Average: +1σ to +2σ)	13.59%	680 units	84.13% to 97.73%
Far Right (Extremely High: +2σ to +3σ)	2.14%	107 units	97.73% to 99.87%
Outliers (Extreme tails: Beyond ±3σ)	0.27%	13 units	0.00% to 0.13% and 99.87% to 100%

So, to get a reasonable number of peers to discover your research, you’d need to be in the 99.87th percentile+ category. Assuming we swallow our pride and relax the assumptions, the next best thing is to be in the 97th percentile bucket, with around 100 other papers competing for our peers’ attention. Long sigh.

Caveat: This model makes a lot of assumptions about how research propagates and that researchers are rational creatures who only start reading from the top, and completely discounts the fact that search engines exist, which can resurface research from the average and so on. But I think it’s still a decent model to be aware of if one wants to make it, whatever that means.

Solutions

Where do we go now?
~Axl Rose, Sweet Child O’ Mine

The point of research is to seek and discover truth, to ask a non-trivial question about nature and try to seek an answer. I know it’s sounds a whole lot corny but it sounded better in my head, trust me. It should not be Goodharted by paper count to get tenure or bag that million-dollar package or whatever. I know paper quality is hard to quantify and there are no good quantifiable proxies for good research, but oh well.

As a young researcher, one must be ruthless about one’s choices when it comes to consuming research. There’s literally a ton of garbage out there. We must also strive to build better retrieval systems that go beyond TF-IDF, reranking, or even modern generative retrievals.

Incentivize good research. I know it’s easier said than done because such arrangements go against the principle of efficient capital allocation, but we need people who can do maverick, moon-shot research without concerns for tenure or grant cycles.

I haven’t thought much about how to fix the rape of the review system and to maintain the integrity of the process, but some people seem to be considering a credit-based system .

Footnotes

Well not all of them were all of these things but you get the idea.

Day 1 of n: Reading (and Implementing) Ilya's 30 ML Papers

gs454236@gmail.com (Basil Labib) — Wed, 02 Sep 2026 19:01:50 +0530

Source material:

Also supplemented by Essential Deep Learning papers. See https://papers.baulab.info

The lists differ considerably due to no single source of truth and the papers are not arranged in order of attack.

My Approach

My approach will be different in the sense that I will not just read the papers but try to grok them, implement as much as possible (with as little shameless copy-pasting from LLMs) and go into small tree search around the topic anchored by the mother paper. One day one paper, 30 papers. Let’s go.

Group by paper category

Papers are roughly grouped into:

Architecture papers
Theory papers
Engineering papers
Applications papers

Category-wise grouping of papers

### Architecture
1. Attention is all you need
 - Go deeper into types of attention mechanisms. Most critiques don’t like attention; why? Why is it so effective then?
2. NTM
3. RNNs
 - Variations include LSTMs, GRUs
 - See Olah, Karpathy blogposts
 4. Pointer Networks
 5. CNN

### Theory

1. Scaling Laws
2. MDL
3. Komogrov Complexity

### Engineering
1. GPipe

### Applications
1. Seq2Seq
2. Protein folding
3. AI4Science - see GDM research
4. AI4Math
5. Automated research
6. Translation
7. Speech to Text and family of models

Papers arranged by increasing epistemic knowledge (for beginners)

TBD

Day 1: LSTMs, RNNs, and Complexodynamics

Re-read Chris Olah’s excellent exposition of LSTM .

Explore my implementation in this notebook (boiler plate code is LLM-generated).

This time I took the time to implement it using basic PyTorch. I remember my last ambitious (and somewhat stupid) attempt at writing autograd “from scratch” failed marvelously and had to be abandoned with the bitter lesson (pun not intended) that I am, after all, a poor programmer. Anyways, keeping my ego aside, this time, I took the path of least resistance, focused on the main problem - namely implementing the architecture from just the blogpost text - mind you I have never read any paper on LSTM before, and worry about details like dataloaders and training loop later.

This strategy paid off well on average as I quickly ran my code and started seeing loss collapse to zero on the second epoch! The loss was stable and barely moving. Obviously the model was not learning anything. And sure enough, Claude pointed out that I have forgotten to return the variable that is supposed to be trained and moreover, it was a class variable rather than a local variable that holds the hidden state and long-term memory register! Yeah. I felt ashamed. My ego was humbled, but it was only the beginning.

I was asking Gemini for code snippets for “boring” bits and pieces such as DataLoader, normalization, and inference code which accelerated the process considerably and made sure I didn’t waste time on less important aspects. I think this strategy is very good and I will try to use it more in the coming days as I implement other architectures as well. I also noted that large chunks of the ML pipeline is just boilerplate code (training loop, data prep, evals, inference, etc.) and should be packaged into neat reusable modules. Who knows I might build a small library at the end of this learning project.

Anyways, Claude pointed out glaring structural and systemic flaws in my assumptions and design decisions. For example, I was using a word-based tokenizing scheme that split on spaces (I know, don’t roll your eyes; I wanted to start simple, and I was going to implement BPE), and Claude said to use a character-based tokenizing scheme because the corpus was too small. Similarly, it prepped the data into seq_len chunks of a sliding window, essentially blowing up the training dataset. It also suggested (and wrote the code) for an evaluation subroutine for validation. I was following the conventional epoch-based training loops; it suggested using steps since the corpus was too big and there is essentially no new information to learn across epochs. It also did some back-of-the-envelope calculations to tune some key hyperparams and suggested increasing hidden dim and other practical, insightful nuggets. However, at this scale, Claude probably already has the optimal hyperparams memorized lol. I also naively implemented argmax over the softmax-ed logits. Claude suggested I move to temperature-based sampling to avoid getting into repeated phrase loops – very valid point btw.

The model trained alright. It generated text that, funnily, represented Shakespearean writing. Pretty cool stuff. I trained on my Mac. On default dim=128, it took around 30 mins to train for 20K steps across the whole corpus (~5M training pairs).

This exercise made me realise when people say how much of a dark science practically training ML models is. There are so many hyperparams to tune and there is no scientific grounding save a few heuristics and just accumulated industry wisdom. It’s like shooting an arrow in the dark and hoping it lands on the target. In my opinion, people who train models with trillions of parameters across distributed clusters are nothing short of modern-day wizards!

Sample Prompt:

“I know not why I am so sad”

Sample Output: max_chars=100 and temperature=0.5; some hyperparams (dim=256, batch=64, seq_len=128)

Raw: [‘I’, ’ ‘, ‘k’, ’n’, ‘o’, ‘w’, ’ ‘, ’n’, ‘o’, ’t’, ’ ‘, ‘w’, ‘h’, ‘y’, ’ ‘, ‘I’, ’ ‘, ‘a’, ’m’, ’ ‘, ’s’, ‘o’, ’ ‘, ’s’, ‘a’, ’d’, ‘\n’, ‘A’, ’n’, ’d’, ’ ‘, ‘y’, ’e’, ’t’, ’ ‘, ’l’, ‘i’, ‘v’, ’e’, ’d’, ’ ‘, ‘y’, ‘o’, ‘u’, ‘r’, ’ ‘, ’l’, ’e’, ‘a’, ’s’, ’t’, ‘,’, ’ ‘, ‘y’, ’e’, ‘a’, ‘k’, ’ ‘, ‘a’, ‘g’, ‘a’, ‘i’, ’n’, ‘,’, ’ ‘, ‘I’, ’ ‘, ’t’, ‘a’, ‘k’, ’e’, ‘\n’, ’t’, ‘h’, ’e’, ’s’, ’e’, ’ ‘, ’t’, ‘w’, ‘a’, ‘i’, ’n’, ’ ‘, ‘c’, ‘o’, ‘r’, ‘r’, ‘u’, ‘p’, ’t’, ’s’, ’ ‘, ’s’, ‘o’, ‘,’, ’ ‘, ’s’, ‘i’, ‘r’, ‘,’, ’ ‘, ‘w’, ‘i’, ’t’, ‘h’, ’ ‘, ‘a’, ’ ‘, ‘g’, ‘r’, ‘a’, ‘c’, ’e’, ’ ‘, ‘o’, ‘f’, ’ ‘, ‘h’, ‘i’, ’s’, ’ ‘, ’s’, ‘p’, ’e’, ’e’]

“I know not why I am so sad\nAnd yet lived your least yeak again, I take\nthese twain corrupts so, sir with a grace of his spee…”

And then I guess we’ll never know what Shakespearean my model was musing over. Also interestingly note non-English words like “yeak”. Pretty impressed that the model learned English words and basic grammar semantics.

I also skimmed through some parts of the Attention paper but didn’t deep dive today due to lack of time.
I also read through Aaronson’s post on complexodynamics - weird idea but nice to think about. Still need to get my head around the sophistication metric.

Trial by Fire: Tenure as Head of Technology at Rendezvous ‘25

gs454236@gmail.com (Basil Labib) — Wed, 05 Nov 2025 12:41:54 +0530

Disclaimer: This post may contain expletives.

This post chronicles my experiences as head of technology at Rendezvous ‘25 , the annual cultural fest of IIT Delhi and lays out how events unfolded in roughly chronological order, what challenges we faced, how we overcame them, how I managed a team, and finally, delivered to keep the awesome tech culture going at RDV. For those of you who didn’t get a pronite pass and are cursing me right now, please read on.

This post is written as a review document for myself to come to later and revisit the technical challenges, social challenges, and indeed the intense drama of it all. I think the lessons drawn from this experience are timeless and apply to any intersection of humans and technology (More on this later, if you’re confused). This post is a heartfelt appreciation post for those who stood by my side during these times and it is also a warning to those who did not heed my warnings. More importantly, this post is for the thoughtful so they may learn something from this hard-earned wisdom.¹

The beginning

For the uninitiated, Rendezvous or RDV is the annual cultural fest of IIT Delhi and witnesses more than 10K footfall from around the country to the IIT Delhi campus, hosts more than 100+ events, competitions, and is a 4 day mega-event. The entire fest is organised by students. The hierarchy has the CTMs at the top and are generally final year students followed by coordinators and executives. There are over 35 verticals ranging from marketing, publicity, events, and so on. The tech vertical is responsible for maintaining detailed records of registrations, entry, gate passes, event passes, concert night booking and security management. It facilitates the RDV campus ambassador program (CAP) with a task-based tracker system. It also develops and maintains the official RDV website which serves as the face of the fest in cyberspace. As you can imagine, this vertical facilitates tracking, accountability, and complements other verticals like security, publicity, marketing, and just about every other vertical.

Timelines

This year, RDV was scheduled from 27th to 30th Sep. The notification for the CTM applications were floated during last week of July and I filled the form out of curiosity. I wanted to get more hands-on experience in leading a tech team and ship products that people actually use - RDV Tech is one of the best places to get this valuable experience in IIT Delhi. Sure enough, I was selected along with a junior and we started out recruiting the team. We had around 60 days to do everything. Everything.

Building a team

We decided to keep a lean team with a flat hierarchy to reduce communication overhead. In practice, the tech team is involved in a workflow at the very end and other verticals just assume that everything is possible in tech (At least, that’s how it works in college fests). This meant that we needed shorter feedback cycles to deliver and keeping a 3-tiered team just won’t cut it. When other verticals had more than 20 or even 30 people, the tech team finally came down to 5 people. Yes, just five people.

The team mantra

The mantra I followed was to get only A players, pack as much talent as possible in as few people as possible. This required multiple cycles of hiring and firing and I am glad I did this - an excellent team is absolutely non-negotiable. A lot of what we achieved this time was only possible because of my team. I had to fire around five to six people before I settled on a team of five people who showed up everyday, did the work, and more importantly, enjoyed doing the work. Among these five people, I had two “core” juniors who were exceptionally diligent and sincere (You know who you are :)). I often used to notice how people worked - some of them just grudgingly fed the entire spec to GPT and expected a UI out of magic while some genuinely enjoyed what they were doing - looking at network tab and response code to figure out what’s wrong, using GPT to their advantage and not as a barrier, and those who didn’t tell me “This is not working” but who told me “This is not working because of X and I know how to fix it”. These are the kind of people one should vehemently seek and nurture.

I remember quite vividly what one of my team member told me in my first meet (slightly paraphrased): “Waise toh meine socha tha ki RDV ka kaam nahi karunga magar mein log dekh kar kaam karta hun aur aap mujhe acche lage” (I thought I would leave RDV team but I found your personality appealing, that’s why I decided to stay). wink wink

Overview of Systems (Slightly technical)

I have been extolling the tech team a lot but what exactly does the tech team do?

The RDV tech infrastructure is developed from scratch by the RDV team and combines years of experience passed on from previous CTMs. We reused a lot of the infrastructure set up by last year’s team, managed, and modified a lot of the services to cater to the growing needs of the team. The RDV tech infra is a microservices-based design comprising multiple services like the main website, CA portal, admin portal, pronite portal, a URL proxy service, a mailing server, and a centralised OAuth authentication service. We also maintain a mobile app for scanning gate and concert passes for tracking admissions. All services are hosted and managed on AWS and feature multiple layers of rate limits. We use AWS firewall (WAF) as well as cloudflare’s rate limit rules, and involve Google ReCAPTCHA service for selecting high traffic stages in our workflows (such as authentication, and pass booking) to eliminate bot attack attempts.

Some new things we tried this time:

Gate passes and Event level passes
Integration of marketing deliverables in gate pass generation workflows
Integration of additional biometric data as per security guidelines
Integration of an ML pipeline for automated task verification
Complete rehaul of the CA Admin portal
A new mailing server, and a temporary service monitor dashboard
Stress testing of high traffic and mission-critical services using Locust

Early Days - Setting things up

After I got access to the AWS and Github org, I started poking around the existing services. The last CTM had done a fantastic job of organising a lot of these things already such as dockerising services, setting up environments on AWS Beanstalk. This was even my first time managing AWS and I knew I would be managing the entire infrastructure some time in the future so I knew I had to master it. Some loss of sleep hours and I was mostly familiar with the basic tools of the trade for there is no better way to learn how to douse a fire than when there is one on your tail.

If you are connecting AWS RDS using PgAdmin, remember to turn ssl_mode to require. This one configuration took way too long to figure out. And indeed, good DevOps is all a matter of correct configurations. I quickly started making SOPs for deployment and cleanup. See an example SOP below:

[Redacted]

In all this, Perplexity helped a lot in speeding up the search process and retrieving only relevant info in an easy to digest format. I think for devs, Perplexity just makes so much more sense than wading through 10 SO threads before you start getting an idea of what’s going wrong. I got a Cursor Pro subscription and gave it to the team even though they were initially reluctant (coding purists). However, later on, they told me that Cursor had, in fact, helped them quickly prototype designs, especially UI stuff, which just kept changing a lot - a typical feature of fests.

Deploying my first app on AWS

I remember deploying my first environment on EBS took around an hour - creating IAMs with proper roles, checking and double-checking each role’s permissions, setting env variables, configuring VPC subnets, configuring load balancers, configuring firewalls, configuring the database and the service user and double-checking their permissions, configuring third party services like 2Factor for mobile OTP service (Great service at reasonable price and very cooperative person, 10/10 recommend), instance types, and codepipelines for continuous deployment. Once again, I was reminded that there is no room for mistakes and there is no room for assumption.

Along with learning myself, I had to manage communications with the larger team and had to deliver on the CA Portal in a matter of a week since it would be used a month or so leading up to the main fest. I remember the minor exams were also coming up and I had some other prior commitments that I had to tackle as well. As usual, the design team did a sloppy job and we had to improvise on our own to make the design functional. It is partly not their fault as the designer had never used the CA portal or knew about its workflows and what kind of issues the users might face.

The CAP took two days and at least 10 attempts to deploy partly due to poorly understood GPT written code, some overconfidence on part of the developer, and an extra front slash in the environment URL configuration. The path BASE_URL/route should have been used instead of BASE_URL/route/. Eventually, the team figured it out and I’m sure learned a lot in the process. There were countless such issues that forced us to think analytically and often under time constraints.

Work culture

As I already said, the initial days were tumultuous in terms of team composition, I had to remove people almost on a daily basis but eventually the team settled down and the first thing I did was set up a work culture. I signalled it simply by showing up every single day at the exact same time at the same location and asking everyone to do the same. This move meant that the RDV tech team meant business, that the team lead was not messing around, that the team had to actively take up issues and open PR against them. Surprisingly, almost all of my team were sophomores who had little idea about what git even was and so I had to painstakingly (but patiently) teach them about git workflows.

INSERT: meeting at 9pm at sac message

Eventually, the daily 9pm meets at SAC, often extending till 5am in the morning (!) turned out to be great for team bonding as intense coding sessions and discussions gave way to gossip and romance tales, sometimes over regular night mess treats. I think that money was well-spent in establishing trust as I established myself as a senior who was not just interested in getting work done but who was genuinely interested in the flourishing and well-being of my juniors.

Dealing with liabilities

My Co-CTM was turning out to be more of a liability than an asset: not showing up, having to follow up multiple times, no interest in the work, no responsibility or accountability. It turned out to be so bad that work started suffering and I had to report him. The larger team decided to let them go but then I practised compassion and asked them not to. Ironically, my compassion came back to bite me later but more on this later. All I learnt from this experience was that not everyone is meant to be a leader and being too compassionate is probably not okay. Spending even one more word on this is useless just as I decided spending a minute of thought would be back at that time.

Time for reflection

Leading a tech team also exposes an opportunity to understand and discover oneself. I started noticing things about how I manage things, interact with others, and questioning everything. I have an eye for detail, understand people psychology (an indispensable but rare asset for engineers), and am slightly OCD-ed. I remember getting really pissed off and tensed over a single link not working on the mobile version of the footer of the main website.

As the team got busy with assimilating the existing codebase and writing more code, I concerned myself with administrative things and the budget in particular. We needed at least 1000 USD going by last year’s cost and we got exactly 0 USD from the team. Multiple reminders to the concerned verticals were watered down or ignored at worst. Out of my self dignity, I stopped constantly reminding them. We had some leftover credits from last year and I decided to manage with that. It would mean rigorously optimising all services, adopting a lean strategy, and cutting down on frivolous costs (Ironically, we ended up wasting 200 USD on a pilot app that was of no value eventually due to carelessness on my co-CTM’s part).

Good software = Reduce human cognitive load

Designing the CA Portal and the accompanying Admin portal was a masterclass in developing production-grade software that thousands of people used. It also meant that even a single word in the instructions would wreak confusion on thousands. The leverage we had was insane. One thing we learnt was that keeping the target audience in mind while designing the app was most important. We had to ensure each page transition was as simple as possible for cognitive kindness and easier comprehension. If we were careless, users got confused and we had to explain things on a one-on-one basis. We even had to dumb down “clear cache” to “clear browser cookies” and then eventually to “use a different device”. One piece of advice for the future team will be to ensure good cookie management on the frontend. Even with accessibility, mobile-first approaches, there were one-off issues with devices because some had ad-blockers or pop-up blockers installed and just weird edge-cases that we had not anticipated on an average user. That is the thing, right? One is not developing for the average user, one is developing for all users.

Our previous OTP service had some issues so we started searching for a new one. We found 2Factor.in which turned out to be more reliable than I expected. Due to time and resource constraints, we didn’t do DLT registration and went ahead with a default mobile carrier that 2Factor helpfully provided. We did registration trend analysis to find peak demands and last year’s volume data to pick a cost-effective plan.

The first crisis

Inspite of all precautions and SOPs I could write, it is inevitable that things would fail and I got a taste of it right on the day of the first major run of the CA Portal with the internal team. The idea was to onboard the internal team to track their progress on the task and do a makeshift identification system where we tweak their internal IDs to mark them as RDV team members. At this point, we had three types of users on the platform: non-IITD external users - treated as a typical CA, full task management, approval system, and points tracking IITD, non-RDV user - also treated as a typical CA but kinda redundant IITD, RDV users - treated as a typical CA but have to complete specially marked tasks only.

One advice for the future team will be to implement a Task Name and some kind of tagging system for accommodating one-off publicity team requirements.

Anyways, back to the story, the entire team of ~600 have to be onboarded on the platform on Sep 26th and I start getting reports of the authentication system failing. I initially dismiss such reports because there is generally some error on their end so my reaction time is longer to classify reports of a particular kind as a genuine issue. By the time I realised there’s an issue, the team was already gearing up to onboard the platform. I realised there would be massive chaos, so I ran to fix the issue. The issue turned out to be the failing reCAPTCHA service which had exhausted the free trial limit. I laughed so hard. I added my credit card and shifted everything to a paid account. Even last year’s CTM reported that the same thing happened to him two days before the fest.

The big mistake

I was aware that this year’s RDV will be under strict administrative supervision due to fund misappropriation allegations last year. This also meant less funds in general. To help the larger team, I agreed to onboard a third party service (say, DumbCompany) for the pass booking workflow. The pronite pass booking workflow is one of the most critical parts of the tech infra and we have a custom server catering to it. DC was a total black box for me. Their code was closed source and there was no obvious security measure that you expect to see on a ticketing platform. Anyways, we started collaborating and it translated to a lot of grunt work on my end such as copy pasting data from one sheet to another, creating redundant event objects on their platform since they did not have user categorisation. They didn’t even have user access control!!! I approved the risk since I was too blinded by “helping the larger team” inspite of clear instructions from last year’s CTM to not integrate any stupid platform with tech. In any case, their entire system was compromised on day 2 of the fest but we are getting ahead of things here.

One great thing I did, however, was that I approved only a partial collaboration with DC since I didn’t trust them one bit. This meant that the booking step would still be done by RDV tech and only the pass distribution would be handled by DC. This was an attempt to reduce blast radius in case of an emergency. We had no idea what information they had in their QR passes (!). Since this was a first time collaboration, a lot of the things had to makeshift as I highlighted above and all of the burden came down to me which I silently endured. They didn’t even have an API!!!! The entire system was opaque with no documentation, no statistics, no insights into what’s going on. It felt like being built over a weekend by a bunch of sophomores and definitely not a service that you roll out to cater to twenty thousand users.

Marketing team

You guys need to understand that tech is not child’s play. Everything has to be properly configured, designed, tested, and secured before releasing it to people. This is because people love to “hack” websites and claim they hacked “RDV”. The tech RDV has evolved over the years is built to prevent every such nonsense. Adding new tasks during registration workflows at the last minute (increasing average registration time from 30 seconds to 1+ minutes) only meant lower registration rates - a simple psychological point that was lost on the marketing team.

As the days of the fest drew closer, conversations with DC got more intense since they had no infra to cater to RDV. As I was one of the two CTMs from the PG program, I expected feeling left out. But I started feeling treated as an outsider and the vertical at large was being neglected. The idea of not feeling welcomed didn’t bode well with me but I ignored it. I knew what I was building and I didn’t expect non-tech people to appreciate or even understand it at all.

The Insults and Slanders

Due to last year’s fund misappropriation allegations, the admin had mandated that the CTM team sign an undertaking to personally bear the cost of any deficits in the fest’s expenditure. On request for this year’s budget sheet, I was once again ignored by the marketing team. That’s when I decided to post a message on the CTM group saying that I will shut down the entire tech infra if I do not get the sheet by morning. I posted the message at 3 am and went to sleep. The sly thing about this arrangement was that the declaration was given to us at 10pm a day before the fest and our access cards were conditional on signing the declaration. Needless to say, I was furious. After countless nights of toiling fruitlessly for an event, I felt I was being insulted in plain sight now. There were some bitter messages posted about me by certain people in the group. I was hurt but chose to ignore - after all, how can you empty an already hollow soul? Some screenshots of the chat as proof:

[Redacted]

Day 1

I woke up on day 1 and checked my phone and you won’t believe what I saw. A SHATTERED SCREEN! My phone was dead. I had 3 days of the ultimate test for 2 months of my team’s work. I got one of my junior’s phones as an immediate aid but lost all MFA setup on my old phone. Somehow I managed to setup auth for AWS and cloudflare on the new phone. My contacts were gone. It was a mess. I lost all my photos which included some of my most treasured memories. I was heartbroken but too numb to realise the implications back then.

Day 0 pronite bookings were a breeze. I expected this because no major artists were performing so demand were low. This also didn’t push the system enough for me to diagnose that I had actually forgotten to apply the firewall to the load balancer attached to the Pronite portal application instance…Oops.

When a million requests swarmed the system for day 1 bookings and I saw the system struggle to keep up, I quickly realised that all those sleepless nights of thankless work was for this moment, it wasn’t for some dumb POR or ego satisfaction, it was to feel the adrenaline rush when your engineering systems are put to the test in the wild. After initial hiccups with the WAF, it went smoothly.

Due to the dumb as fuck design of DC, I had to send data to the company in google sheets (yes, I know, DUMB). It was time-consuming and error prone and obviously not how things are done. Sure enough, I missed a sheet and a group of MSc students didn’t get any passes. Moreover, an important callout informing the end user about all slots filled was in white text colour and hence didn’t show up on a white background - “white on white bug”. I laughed. Good UI design is so damn important.

“It is working on my machine” is not a valid solution, designing apps for hundreds of user demands thinking laterally - it’s about empathising with conditions and experiences that you can never live by yourself.

What happened on Day 2

DC had just exposed an endpoint to show QR codes on the portal. That dumb endpoint required us to do intensive cryptographic calculation on our end. We pushed the update at 10 am and the pass booking slots were from 12 noon. I noticed that passes were not getting booked. Fuck.

I called the previous CTM and we got on a call to figure out the issue together. It turned out to be a memory hogging issue. Apparently, the new code had saturated all vRAM on the servers. Kudos to Shanky for diagnosing the issue in due time. I throttled the instance RAM by upgrading from small to medium and adjusted the quotas. We were fully booked.

The setup

The entry for day 2 was supposed to start from 5pm. We anticipated around 2000 students and 1000 guests, faculty, staff, sponsors, and so on. That’s a crowd of 3000 people. Delhi Police forces and the IITD security team along with bouncers were deployed. I could feel the adrenaline building. We were in talk with CSC to ensure that the external sender was whitelisted because if people didn’t get the QR code from DC on their emails, it would be a disaster.

Now, how DC’s system worked was that the issuer and scanner had the same login credentials. THERE WAS NO USER ACCESS CONTROL. The security team had decided to share the issuer credentials with freshers who will be scanning the passes at the venue entrance. This, in principle, meant that an inquisitive fresher could gain access to issuing passes if they could enter the OTP before it expired on the issuer portal. That’s exactly what happened.

The discovery

Around 3pm, one of my team members noticed something curious. There were hundreds of emails being sent to the director’s email id. Hundreds. The dumb people at DC marketed their ticketing system as a unique token for each user, then how can there be 600+ tokens issued to the same email id? I was infuriated. But even more than that, I was trying to figure out the implication of this. Clearly, some one has gotten access to the issuer portal, there was no access or event log for tracing, there was only one admin account with all permission. This was a doomed situation.

The countermeasures

We immediately called DC’s representative and he basically raised his hands saying that he has “no control” over the system. I was beyond infuriated. I had to keep calm. I had to navigate this situation. The first thing I did was pull the plug on all DC issued QR codes. We sent out an email via the dean around 4pm saying that all such QRs are invalid. We replaced the DC scanner with our native scanner. We generated our fresh tokens and made it live on the booking portal itself under the Profile section. That would take care of the already booked users but we still had ~1000 people who were given passes due to their privilege like team members, staff, and so on. My fantastic team of 2 people whipped up an endpoint for batch QR generation and we manually distributed the QR codes to these people.

At 5:30 pm, the queues were hundreds of people long. The first scan failed. My heart sank. At that moment, I felt my second adrenaline rush of the day. I knew at the back of my mind that this subsystem was entirely untested on day 1, if this fails at this point there are thousands of people waiting to get entry. Memories of the stampede during my second year of RDV flashed at the back of my mind. Fuck. I checked the credentials and realised that the scanning schedule had to be updated. We did that and the first QR scanned. I cannot express my relief when I saw that green tick show up. Our system was deployed in production without testing and it worked. I was super proud of my team.

What could have gone worse?

If the adversary was a bit more clever, they could have scraped the entire IITD directory and sent an email to all ten thousand members with a QR code. The system wouldn’t have complained because there was no damn upper limit - there was no uniqueness check even. A cleverly orchestrated attack would probably masquerade this as distributed across various user categories so detecting would be harder and by the time it could be figured out, things would be too late. That would mean ten thousand people showing up anticipating an entry into a venue of only three thousand capacity. A sure-shot recipe for a stampede. Thankfully, the attacker was not that sophisticated which makes me think there were either first or second year students.

We got lucky

We were super lucky to discover this attack early enough to invalidate everything from DC and shift to our system in time. I am super proud of my team doing the heavy lifting within one hour. Talk about shipping cycles and saving the day.

Reflections

During this whole time, me and my team were holed up at the PFC studio at SAC along with the marketing coordinator, CTM (the same person who had written bitter words to me in the group a day before the fest), and the overall coordinator. We also had to explain everything to the professors and the deans. I could see life getting sucked out of the coordinator who had onboarded DC as calls started pouring in from the deans. I could sense his inexperience and honestly, I felt pity for him. I was naive and uneducated in the way of the world at some point too. I hope he learnt something that day. My respect for elders also increased manifolds that day. I realised how many times they have knowingly and unknowingly saved our asses when we fucked up. Except in this case, I was the one saving the situation. Retrospecting, that split second decision to invalidate all DC issued QR code feels unprecedented and honestly, unreal. Maybe a bit reckless as well, given we hadn’t tested our native system at all. I felt like Icarus, either flying into the sun or burning to the sea. We soar.

I didn’t get my pass, what should I do?

To everyone who didn’t get a pass, read this carefully. There are 10K members in the IITD community and the venue has a cap of 3000. Even if we were fully booked, 70% of people just won’t get a pass. That’s why there’s at most one criteria to ensure at least 60% of people get to experience at least one concert. This is not tech’s fault. That’s the sad reality of it.

If you have ever taken a computer networks course, you will understand that there are a number of random factors that affect how requests are ordered and go from client to server. The pass booking system is 100% fair in the sense that everyone has a fair chance of requesting for a pass. Whether that request reaches the server depends on your local network topology, your average bandwidth, your local latency which means how many other clients are connected to your router and how many hops it takes from your router to reach the server². In that sense, the pass booking process is essentially a fair lottery for all which is how it should be. So, if you didn’t get a pass, tough luck.

Day 3

The next day, the overall coordinator told me that the guy who had written bitter words about me had apologised. I forgave him.

While I was busy with these, another set of events were unfolding on Reddit. Apparently, people who didn’t get a pass were writing hate messages about me on the r/IITDelhi. I received many concealed hate messages IRL and some of my outside juniors tried to humiliate me by using under-handed scorns for invalidating the QRs because that’s what they saw on the outside. I smiled and shrugged it off. Even if I explained what we had prevented, they probably wouldn’t have the aptitude to understand.

My “co-CTM” who had effectively stopped working but had marketed himself as a RDV CTM started receiving a lot of flak on his social media from people not getting their passes. Karma, I guess.

The final day was a breeze. DC was gone for good. I was confident in my system. My team worked late to incorporate small cosmetic changes like color-coding passes for various categories as per feedback received from the on-ground team. These were good suggestions. In all this chaos, I missed experiencing the fest, of course. My last RDV as a student. I was only looking forward to the qawwali night which I couldn’t attend. Last year, Nizami Bandhu performed and it was surreal. It’s okay. Sacrifice is the cost of leading as I knew from Literati days as well.

My parents were in Delhi and we had a bus booking to go to Ajmer at 10pm on Day 3 (!) for a 5 day trip. I know. Crazy. Moreover, I had to buy a new phone before that. I landed up in Select Citywalk and bought a Nothing 3a from Croma. I also bought my new Mac from the adjacent Apple store since that was the last day for the offer (I got almost 50K worth of value from the offer!). I had tried to buy the laptop two days back but couldn’t due to credit limits. I think it was written that I would buy my first Mac with contributions from my parents and my father’s card. I must be one lucky guy.

I took my parents to the Day 3 concert. Ankit Tiwari was performing. My parents saw the crowd, saw the energy, saw the vibe and I told them “This is what I have been working for”. I forgot the fatigue, the stress, and the adrenaline wearing out. It was all worth it in that moment. I am super grateful I got my parents to experience an RDV concert.

Some numbers (because an engineer’s life is incomplete without it)

What did we deliver exactly? I looked at the Cloudflare numbers after the fest and I was shocked. We served 130K+ unique visitors and served around 13M+ requests. Entry scanning records show more than 10K visitors though this is a lower bound. We have 20K user registrations. All of this within the span of 60 days across 10+ services.

System summary

The following section is copied from my Linkedin post .

Few architecture designs:

Two tiered caching and rate limiting via Cloudflare and AWS. Google RECAPTCHA integration for anti-bot measures.
AWS CloudFront CDN for optimised website experience.
Core OAuth Service featuring multiple social media login options and integration with Kerberos, IITD’s internal auth system.
AWS WAFs, Load balancers, and EBS housed in their own VPC for scalable and secure web applications.
VLM on AWS Bedrock for automated task verification.
Event and Pronite passes scanned via in-house mobile app for real-time update on venue occupancy for crowd management.
Siloed distributed services to reduce blast radius.
Simulations run on Locust to estimate service performance under stress.
CloudPipeline integration for one push builds via Github CI/CD.
RDS Postgres (Database), S3 (Storage), SES (Emails), SNS (SMS), and SEO optimization using sitemaps and Google Rich Results.

Challenges, Advice, and Lessons

If you have made it this far, thank you! What did we learn from all this? Here are my key takeaways for the future team:

Do NOT fuck around with tech - current system is designed taking into account years of learning, no “3rd party integration” unless every single line of their code is audited by the RDV tech team. Even then, just don’t.
Please give a budget to the tech vertical.
Make a capable Tech CTM near year. The system is more than capable to handle RDV workloads (tested for 2 years), it needs to be configured properly though.

Personal challenges

Every vertical had two CTMs except tech. All workload on me.
Thankless working, less bonding with larger CTM team. Most of them were juniors I didn’t know. I knew only 2 or 3 (you know who you are :))
Extreme sacrifices made - I could have published papers in ML conferences, could have spent more time freelancing, could have prepped for placements, could have focused more on learning and upskilling but a word is a word and it must be kept even if it meant dying for it.
It is extremely hard to communicate tech stuff to non-tech people. These people will never understand what goes on to make things work - they just take things for granted (for instance, integrating the Gemini sponsor task which entailed opening a pop-up window but most users had pop-up blockers installed so it “didn’t work”)
I lost my RFID ID card. I had to register a complaint and pay a fine.
Potentially lost chance for a romance. There was this girl during RDV who texted. I couldn’t meet her due to my obligations. Anyways, she blocked me when I texted back after the fest so like whatever.

Resiliency and hard work can overcome all odds. And a good team. It is most crucial to have a good team. This was, in all respects, a trial by fire.

Footnotes

1: I really want to emphasize “hard” here.

2: The quant-minded amongst you might think of reducing this latency by moving your request “closer to the source”. I think this won’t work since all requests are re-routed through a proxy service and shaving a few hops in the IITD network may not be enough to get that edge. But if someone wants to try this out, happy to help. I am actually curious to see someone pull this off and report actual numbers.

Implement a Minimax Agent for Optimal Tic Tac Toe Play

gs454236@gmail.com (Basil Labib) — Wed, 22 Oct 2025 18:46:47 +0530

I hear and I forget. I see and I remember. I do and I understand.

~Confucius

I implemented a (very) simple agent to optimally play the game of tic tac toe (or noughts and crosses). In this post, I will outline the algorithm I used, how I modelled the problem, and walk you through the code. Feel free to mess around with the code here . It is surprising how the core search algorithm is just around 20 lines of code.

Motivation

Well, I am self-learning the AI course offered by my university which is taught by an exceptional professor . It is rare that I find myself wanting to code every algorithm taught in class let alone show up to class. I guess I am just too choosy when it comes to learning¹. Anyhow, I decided to implement the minimax algorithm to solve tic-tac-toe because it seems fun, tractable, and a good way to brush up recursion programming. I spent around 1.5 hours coding and an hour debugging the whole thing (with some help from dear ChatGPT).

A typical Tic Tac Toe game tree

Tic Tac Toe for dummies

This section is probably preposterous but for the sake of completeness, here is how tic-tac-toe is played. It is a 2-player turn based game where each player places their piece “X” or “O” on a 3x3 grid. The first player to make a horizontal, vertical, or diagonal combination wins. It is what is called a deterministic, two-player, zero sum, fully observable game in game theory parlance. These kinds of games can be optimally solved theoretically and can be solved in practice if you just throw enough compute at some naive algorithm. One such naive but foundational algorithm is minimax.

What is the minimax algorithm?

The core abstraction that we will work with is called a game tree . It is an abstract object which formalizes all possible ways that a game can proceed. We represent game states as nodes of this tree and represent reachable game states as children based on some valid move. Of course, some states are better than others from the player’s perspective. How can we formalise this idea? We assign values to each node in the game tree representing how good it is to be in that state. This is convenient because it allows us to apply utility maximization to compute best moves. A little thought will convince you that if you can compute all possible ways a game can be played and pick the best moves based on utility maximization, you have indeed found the optimal game play. Nobody can beat you.

How do we compute these node values? Assume we are thinking from the computer’s perspective. We need to output a move from the available set of moves given the game state. If I give you the value of the game state if you played one of the moves, you can tell me the best move (just pick the move with the max value). We recursively construct this tree implicitly in the code by counterfactually reasoning about “what if? scenarios” - “What if X played at position 3, then O can play at positions 5, 6, 7. But if it plays at 6 then X wins, so it must play at 5 to block X” and similar arguments.

This should give you an idea about the adversarial nature of minimax. The values for the computers are exactly the opposite of the values for the human. If we looked at the game trees constructed by the two players from a given game state, they would be exactly same except that all the values would be inversed. The minimax algorithm formalises this notion by maximizing values for its own actions and minimizing values for the opponent’s actions. We’ll see how it’s implemented in code later. The big assumption here is that your opponent is also a rational agent trying to maximise its own utility.

Modelling TicTacToe as Adversarial search

What do we really need to make this agent? Well, we need to represent the game state somehow. This is implemented as a simple 9 element list of strings in Python where “-” represents an empty cell.

game: list[“X” | “O” | “-”]

We need a function to decide whether we have a winner. This is implemented by the get_winner(game) function which is really just a dumb bunch of if-elses due to the simple and small number of win conditions for TicTacToe.

tictactoe.py

 def get_winner(self):
 # verticals
 if self.game[0] == "X" and self.game[3] == "X" and self.game[6] == "X":
 return "X"
 elif self.game[1] == "X" and self.game[4] == "X" and self.game[7] == "X":
 return "X"
 elif self.game[2] == "X" and self.game[5] == "X" and self.game[8] == "X":
 return "X"
 # horizontals
 elif self.game[0] == "X" and self.game[1] == "X" and self.game[2] == "X":
 return "X"
 elif self.game[3] == "X" and self.game[4] == "X" and self.game[5] == "X":
 return "X"
 elif self.game[6] == "X" and self.game[7] == "X" and self.game[8] == "X":
 return "X"
 # diagonals
 elif self.game[0] == "X" and self.game[4] == "X" and self.game[8] == "X":
 return "X"
 elif self.game[2] == "X" and self.game[4] == "X" and self.game[6] == "X":
 return "X"

We need functions which can transition the game state, namely game.make_move(position, piece) and its inverse game.undo_move(position). Simple and cute functions.

tictactoe.py

 
 def make_move(self, pos, piece):
 if self.game[pos] != "-":
 raise ValueError("Uh oh, cannot make move, cell already occupied!")
 self.game[pos] = piece
 
 def undo_move(self, pos):
 self.game[pos] = "-"

We also need a function which will allow us to evaluate how good a board state is. How can we do this? Well, assume if there are no more moves left, what can happen now? It’s either a draw or we have a winner. Accordingly, we can return 0, 1 (if the computer wins) or -1 (if the human wins). That’s for the leaves, you say, but what about the non-leaf nodes? Well, we simply recurse on the children on the current node and return the best value (minimizing or maximising based on the level parity which represents whose turn it is).

Implementation details and caveats

I’ll not go through every line of code but I’ll note some interesting caveats and bugs I came across while implementing this. There’s one obvious limitation in the code. It doesn’t let you choose your piece. The human is always “X” and always the first to move. Maybe one of you can raise a PR!

Initially, I was always checking for the termination condition for the recursion when there were no available moves left which meant at level 9 but I realised that games can also end at the higher level. That’s when I modified the agent’s compute_move function to check for wins or losses just after it makes a counterfactual move. If it does, then we can just return and save ourselves from recursing!

tictactoe.py

 def compute_node_value(self, level):
 """
 Assumption, we are at max nodes on odd numbered levels
 we are on min nodes on even numbered levels

 Output: (val, pos)
 """
 moves = self.agentgame.get_available_moves()
 
 # pdb.set_trace()
 if len(moves) == 0:
 self.max_depth = level
 self.nodes_explored += 1
 return (self.evaluate_board_state(self.agentgame), -1) # at the leaf, there are no valid moves as there are no children of leaf
 
 values = []
 self.total_branches += len(moves)

 for move in moves:
 self.agentgame.make_move(move, "O" if level % 2 == 1 else "X")
 self.nodes_explored += 1
 if self.agentgame.get_winner() in ("X", "O"):
 values.append((self.evaluate_board_state(self.agentgame), move)) 
 else:
 val, _ = self.compute_node_value(level + 1) 
 values.append((val, move))
 self.agentgame.undo_move(move)
 

 if level % 2 == 1: # odd valued levels are max levels 
 return max(values, key=lambda x: x[0])
 else: # even valued levels are min levels
 return min(values, key=lambda x: x[0])

One more subtle bug which ChatGPT helped me identify was the value function was computing the values and moves of the current node but was somehow returning the moves of the children. Well, you can never be too careful in recursion hell.

Even after reading the code again and again, I still found that the agent was playing stupidly. I suspected that this was due to my stupidity (!) which indeed, it was. I was calling the agent at each game state with an incremental level value which essentially turned it into its own adversary and it was flipping between it’s own friend and foe. LOL.

Results and demo run

I also felt like collecting some statistics to see how large the search trees were. On my laptop, search was instantaneous and the maximum nodes explored were around 60K at the start of the game. The search tree drastically reduces with each level as the number of valid moves decreases.

==== AGENT COMPUTATION SUMMARY ====
Making move 0 with value 0
Max depth explored: 9
Nodes explored: 60112
Total branches: 55504
Average branching factor: 0.9233430928932659
===== END AGENT COMPUTATION SUMMARY ====

Here’s a simplified game run. I chose this because the computer actually beat me as I wasn’t paying attention and it forced me into a game state to guarantee its win. Bewildering stuff!


❯ python tictactoe.py
Move number: 1
- - -
- - -
- - -
Enter move> 1
==== AGENT COMPUTATION SUMMARY ====
Making move 0 with value 0
Max depth explored: 9
Nodes explored: 69088
Total branches: 63904
Average branching factor: 0.9249652616952293
===== END AGENT COMPUTATION SUMMARY ====
Move number: 2
O X -
- - -
- - -
Enter move> 2
==== AGENT COMPUTATION SUMMARY ====
Making move 3 with value 1
Max depth explored: 7
Nodes explored: 1336
Total branches: 1228
Average branching factor: 0.9191616766467066
===== END AGENT COMPUTATION SUMMARY ====
Move number: 3
O X X
O - -
- - -
Enter move> 6
==== AGENT COMPUTATION SUMMARY ====
Making move 4 with value 1
Max depth explored: 7
Nodes explored: 44
Total branches: 44
Average branching factor: 1.0
===== END AGENT COMPUTATION SUMMARY ====
Move number: 4
O X X
O O -
X - -
Enter move> 5
==== AGENT COMPUTATION SUMMARY ====
Making move 8 with value 1
Max depth explored: 7
Nodes explored: 3
Total branches: 3
Average branching factor: 1.0
===== END AGENT COMPUTATION SUMMARY ====
Oops, better luck next time!
O X X
O O X
X - O

Further improvements

There are so many improvements you can try out. TicTacToe with minimax is like the baseline launchpad to explore adversarial games.

Alpha-beta pruning is an obvious optimization although its advantage cannot be felt with such a small game tree (relative to laptop performance).

You can try to look at stochastic games and implement expectiminimax. Or try one of the many Tic Tac Toe variants .

If you’re feeling ambitious, you can model a completely different game like Poker or Backgammon and implement your own agent.

References

COL333 course Prof Mausam IIT Delhi
AIMA Russell et. al. 4th edition Chapter 6 (Adversarial Search)

Footnotes:

Which is a good thing! To actively resist spending time on useless learning is an optimal strategy to acquire most useful knowledge.

Implementing Kerberos Authentication System from Scratch

gs454236@gmail.com (Basil Labib) — Sun, 29 Sep 2024 06:58:24 +0100

Introduction

Sometime back, I started exploring network security systems and stumbled across this nice course page ¹ from my institute. I thought it would be a nice complement to the introductory computer networks course I am taking this semester. I started reading this fascinating short design principles article on the Kerberos system. It was so encapsulating that I spent an entire afternoon drawing pictures in my notebook. I decided to implement the ideas given in the article from scratch. In other words, I decided to build my own Kerberos authentication system.

This blog post outlines the process and the reasoning. We’ll build the system from the ground up, that is, we will propose a system, then point out its flaws, and try to redesign the system to overcome those faults. Along the way, we’ll meet a lot of ideas and end up with a much deeper understanding of why Kerberos is designed the way it is.² (The source code is available on Github).

Authentication model

Motivation

As we live in a world with limited resources, we need to distribute resources fairly among users. Irrespective of the distribution policy, it is imperative to know who is requesting a resource, that is, we need to confidently say that any resource request supposed to come from a user is actually coming from them. Symmetrically, we need to ensure that the resource/service that the user believes it is requesting is really the resource/service.³

Question: How do we authenticate that an entity is really the entity it claims to be?

At the heart of any authentication process is a simple idea of shared secrets. Entity A can authenticate entity B if they share a secret no one knows about. For entity A to verify that an entity is B, A only needs to ask for the secret from that entity. If the set of secrets is sufficiently large, it would be virtually impossible for a rogue entity to impersonate B. This mechanism can be even more foolproof if the secret is changed at every authentication.⁴

Kerberos implements this abstraction for hosts connected through an unreliable network. In an unreliable network, we assume the existence of malicious actors who may observe and capture any data packets shared between hosts.

But before we begin building Kerberos itself, we need to build the network model and define few other primitives.

The user AKA the client

The user interacts with the system through a client program. The client abstracts the details of building data packets, sending/receiving packets from other servers, and parsing those packets while gracefully (well, relatively) handling any errors. The client is implemented in the client.py module.

The resource server

I have implemented a ResourceServer class which represents any resource / service that a user is interested to get / use. Our holy grail for this post will be to get this resource from the resource server as a user.

Cryptography

Kerberos utilises a symmetric key cryptography scheme (at least in this formulation). What this means is that any encryption/decryption happens iff the key is known to both parties beforehand. How is the key mutually agreed upon between the two parties? That’s a question we will repeatedly visit through this post.

Initially, I thought I could use XOR encryption for the sake of this toy application. As it turned out, I spent much of my time debugging annoying unicode boundary errors. Worried that I wasn’t building the actual system, I decided to go old-school. And what’s more old-school than good old Caesar cipher? One funny thing about this algorithm is that it’s laughable and easy to break. If you stare long enough at the ciphertext, you can probably guess what the cleartext might be. Be as it may, it serves our purpose well, so we’ll roll with it. I present the pseudo-code for the encryption and decryption routines below. These can be found in the /caesar.py module.

encryption(clear_text, key):
 for each character in clear_text
 find the character in the position [idx(character) + key] % character_set_size
 add this character to the cipher_text string
 return cipher_text string

decryption(cipher_text, key):
 for each character in cipher_text
 find the character in the position [idx(character) - key] % character_set_size
 add this character to the clear_text string
 return clear_text string

Caeser cipher has this nice symmetry property, as you can see. It can easily be broken using classic cryptanalysis techniques like naive brute-forcing.⁵

Enough of this now. Let’s get to the good part.

Version 1 - The first attempt

What is the most basic secret sharing that you can think of?

We can imagine each resource server maintaining a table of tuples (username, password). If I wish to access a resource, I simply present my password, which the resource server authenticates by looking up the table and matching it. This system works but as you can see, it’s dumb. It doesn’t scale well. If there are 1000 users and 100 resources, we already have 100000 records maintained on the network. Any (user,password) update has to be propagated to every resource server. Moreover, this deeply couples authentication and the actual resource sharing processes. We would ideally want to decouple these two steps so that the resource server is only concerned with serving up the resources. That’s good software engineering!

Naive authentication system

Question: Can we do better?

Yes! Let’s create a dedicated authentication_server which maintains a single (username, password) table. Moreover, it also stores a (servername, password) table. For a client, it is sufficient to authenticate to this auth server. Why? Because this auth server issues an auth token - a “proof of authentication”. How does it work? The auth server first lookups the password for the user in its user table and once its satisfied, it created a data packet containing the username (and some other metadata like a network address) and encrypts it using the servername password. The client takes this auth token and presents it along with its username and network address to the resource server. The resource server attempts to decrypt the auth token and match the username and network address.

What exactly happened here? It is important to note that we established two things through this process. When the resource server was able to successfully decrypt the auth token, it was sure that the auth token was issued by the authentication server. Why? Shared secrets! The server password is only known to the auth server and the resource server!⁶ Matching the packet username and other data with the decrypted data. This ensures that the request was made by the same entity that authenticated itself to the auth server and so, must be one who should access to the resource.

version 1

Question: What are the flaws in this system?

Well, no one is stopping me from capturing these tokens and making a request to the resource server posing as someone else on the network. This is somewhat alleviated by including metadata like network addresses assumed un-editable by system users.
Even more concerning is that the client still has to send the password over the network in cleartext! If a malicious agent can capture the user password then its pretty much game over since this is the only piece of information that the auth server uses to authenticate the user.
Moreover, if a client wishes to access another resource, they must authenticate themselves with the auth server again. We would ideally like the “I am a person who has access to XYZ resources, here is the proof” step to be decoupled from “Hey, can I get resource XYZ?” step.

Question: Can you consider modifying the current system to address these issues?

Version 2 - Streamlining and Decoupling

version 2

Let’s address point 2 first. The client doesn’t send the password along with the username while authenticating with the auth server. Remember that the authentication system is symmetric with respect to the parties, which means it can happen on the client as well. So, we somehow need to make the server send the user password without sending the user password. How is it possible? Well, the auth server just sends back the auth token encrypted with the user password. The client attempts to decrypt the packet (and is successful if its a legit user). No one but the user will be able to decrypt the packet and get the auth token.

To address point 3, we will perform another decoupling. We will have two servers - an AuthenticationTokenGrantor server and a ServiceTokenGrantor server, both of which will maintain their own copies of user and service tables. Moreover, we now make these tokens expirable by attaching a time_of_creation and a lifespan to each token.

The ATG server only issues auth_token which are valid for a certain duration. The STG server issues token specific for any resource server on the network if it gets a valid auth token. As you can see, the user only has to authenticate itself to the ATG server once every reasonable interval of time (say, 8 hours). It can always use the auth_token to get service_token from STG and use the service_token to authenticate itself to the resource server.

Question: What are the flaws with this system?

Actually, there is no guarantee that the request to the resource server that contains the service_token actually comes from the user to which the token was issued. Why? Because there is no shared secret between the ATG and the STG.
It is possible for a malicious actor to gain access to the service_tokens and use them to exploit the resource server as long as the tokens have not expired. This is called a “replay attack” .

Let’s fix these now.

Version 3 - Session Keys and Authenticators

version 3

Question: What kind of secret can the ATG and the STG share? And more importantly, how?

The way this works is by the use of finite-lived session keys. The steps are as follows:

The client authenticates itself to the ATG and gets the auth token. The ATG generates an auth_session_key and includes it in the auth_token (This is the STG’s copy of the session key). Moreover, includes this session key with the packet it sends to the client which is encrypts with the user password first, of course.
The client gets the session key and uses it to encrypt another object called an “authenticator” which is basically another piece of info containing the client username and network address. It encrypts this with the session key it just got.
The client sends both the auth_token and the authenticator to the STG. The STG decrypts the auth_token and uses the newly acquired session key to decrypt the authenticator. It then matches the username that came with the packet to the username in the token AND the username in the authenticator. It then issues the service_token to the client. The STG also generates a service-specific session key and includes it in the service_token and the data packet it sends to the client but encrypts it with the auth_session_key before.
The client decrypts the packet, gets the service session key and uses it to build another authenticator for the resource server.
The resource server performs checks analogous to the STG and finally serves the resource to the client.

Question: What did we achieve here?

Well, the authenticator implements the proof that the bearer of the token is really the entity it claims to be. Can you find the shared secrets in this arrangement? Yes, it’s the session keys!

Although I haven’t implemented this, the authenticator objects also have a lifespan, usually around 1-2 minutes. Thus, it becomes very hard for a malicious actor to replay these attacks.

Elegant, isn’t it? ^_^

Conclusion

If you have made it this far, then I must congratulate you! This wasn’t a trivial post by any means. I have left a lot of the details out of this post for the sake of brevity. If you’re interested, feel free to browse the code and read the article which inspired this post.

All diagrams in this post were produced by the author. Kindly write to me if you wish to use them :)

Ideas for improvement

Make the authenticators finite-lived. This is straightforward to implement and will be analogous to how expiration is implemented for the tokens.
Implement token refreshing. I think this will be a nice add-on to have if this system should ever be put into production.
Abstract the client steps. Right now, the communication with ATG (command gat), STG (command ‘gst’), and the resource server (command get) happens separately and explicitly. Ideally, a nice client would hide these away from the user.
Extend the client to manage tokens and sessions for more than one resource. This demo only focuses on one resource on the network model. We would ideally want more than one resource on the server!
(Advanced) Implement an access control layer on top of this Kerberos. I imagine this would involve creating some role abstraction and storing it in the central database for each user and service. Using this abstraction, the STG may deny issuing service_token to the client.

References

Designing an Authentication System: A dialogue in four scenes

Footnotes

1: Please excuse me if you cannot access the webpage. It’s on IITD’s internal network so it might be accessible outside the network.

2: An additional impetus for me to understand Kerberos is because it is the foundational authentication system used by my institute for authentication and identification.

3: This is called “mutual authentication” and it is generally considered a useful property for general-purpose authentication systems.

4: This closely relates to the idea of a one-time pad which is provably unbreakable. If you’re interested, please check any standard cryptography textbook.

5: Highly recommend reading “The Code Book” by Simon Singh. Highly accessibly and fun read. Or you may also try “Gold Bug” by E. A. Poe.

6: If you’re careful, you may ask but how do we know if the decryption was successful? For this reason, the data packet usually also contains the network identifier of the resource server. So, if the resource server finds its name in the packet, it can be sure that the decryption was successful.

Hacking Wordle Blitz

gs454236@gmail.com (Basil Labib) — Tue, 19 Dec 2023 04:36:28 +0530

Apologies for not blogging in like forever. Funny things have been happening in my life lately, perhaps the most funny is the concentration of all trips and “vacations” in my end-semester holidays. You might call that lucky but I quickly learnt that vacations are supposed to be evenly distributed, atleast from the actually-destressing-and-not-getting-tired-as-hell-from-travelling-and-not-being-able-to-catch-a-breath perspective.

During my entire end-sem holidays, my itinerary allowed me 5 days of staying at home. Just 5. Before you hasten to the conclusion that 5 days is a lot of time to chill and relax, you must understand that there are relatives and friends to meet to maintain “diplomatic relations”. There goes my chill and relax.

Anyways, I’m not pissed. This information is important to put what follows in context (And also, to rant a bit :P)

It cannot be any worse. Right? RIGHT? Guess what, one of my good friends who is the secretary of the Literary club at my college approached me with the request to build a competition portal for a blitz Wordle game as soon as I landed at home. HaHa.

I accepted.¹

This post chronicles how I hacked together the above portal in 2 days (and with a lot of procrastination).

Baby steps

First, I had to find an open-source codebase to start with. Given that the competition was in one week, it would be stupid to attempt to make a Wordle clone from scratch. Luckily, I found this cute and very nicely structured codebase on Github which I forked.

I spent sometime studying and mapping out functionalities, state changes, and components. Steadily, I started commenting out components and looking at the results (to see what breaks). All good, which means I am beginning to understand how the code is wired. Simultaneously, I started designing a flow for the app mentally which would be the path of least resistance with respect to debugging (as any respectable developer will tell you, no such path exists).

It’s a good idea to talk about the structure of this specific competition². It was supposed to be a timed 30 minutes non-stop contest where the puzzle resets each time the current one is solved or all guesses are exhausted. Words would be non-skippable and picked at random from a list of words for each users so that no word list for two users are same.

At first, I thought a very basic token system would do. Just map each registered email against a token and send the token as a query param to some kind of /saveRoundData endpoint on the backend to authorise user. I dismissed this idea quickly due to the possibility of token exchange or other funny things between users or just accidental tampering with the token. There has to be a backend service since the game data would have to be persisted. And there has to be a proper user authorisation with a User model. and what’s a better method to authenticate than my handy JWTs. Also, I have experience with them, no nasty database calls, clean, and simple.³ Sigh, did I just make my life more difficult for an informal event that doesn’t even hold that much value?

It’s `useState()` all the way down

That’s when I decided to put a brake on the user authentication subsystem and work on the actual game flow itself. There were a bunch of state management to hack into and to ensure that the flow was as per the expected business need. Besides, completing this part would just make me happy and motivated to work more.

The code base was in typescript and God only knows why tsc doesn’t like me at all. Made a few edits and a stream of errors pop up. Somehow bash those and new ones pop up. Anyhow, braved my way through this typescript mania to arrive at a somewhat stable (but still buggy) front end. In hindsight, I think typescript is a million types better than javascript and I strongly recommend it over vanilla JS for large projects. Datatype information can save tons of development and debugging time.

Right now, I have the frontend without any backend service feeding it. I move on to the next word on win or lose and words are chosen at random. Moreover, I have made a roundData object that captures metadata about each round’s progress and saves it to localStorage. Neat!

GameData schema

I had originally intended the app to make a server call on every round clearance for each user but then decided against it simply because I would be hosting the app on a free tier vercel instance and I have issues with latency on it. 200 odd disgruntled users would resend the request and soon it will be helltown for my free instance. Although, I think it would still have been able to withstand the anticipated load, I was too lazy to do the math and took the conservative path of saving the roundData on the client side and sending one final request to log the entire array at the end of the countdown.

Now, some of you might have raised your eyebrows high and said “Waaaaiiittt a minute” and I totally get you. I was counting on the fact that no one would be able to open the network and storage tabs, figure this out, and then manually change the array by hand in 30 minutes. The portal link was anyways private and only accessible at the start of the event so I thought this was a reasonable assumption to make.⁴

`passport.authenticate()`

I started fleshing out the auth API using this nice blog post as a reference. It was coming along fine other than a few minor hiccups around Headers and Param types which I identified and resolved. The VS Code Postman extension is such a blessing.

Then, I bumped into a 400 Bad Request on hitting /login with valid credentials. Huh? WHERE IS MY JWT?! I checked and rechecked and banged my head against the wall for a good hour trying different ways to get under the hood of what’s going on. After accepting defeat, I headed over to Stackoverflow and humbly typed my query “passport.authenticate returns bad request” and turns out passport has been a bad boy lately. After scrolling to the end of the thread, I realised maybe I’m messing up the field names (Oh, wow what a revelation) and realised that perhaps passport is opinionated about what goes into its strategy callback function. Fair enough. Problem resolved. For the time being anyways.

Asking the right questions ft. ChatGPT

In my current flow, the countdown is an important component to ensure fairness and a useable app so I set out to refine it. The react-countdown component turned out to be horrible in being unable to retain state across page refreshes and I thought that’s probably okay if I assume people won’t refresh their page during the game which is another sane assumption to make. No JK. But then I noticed timer reset on every keystroke. Bruh.

yarn remove react-countdown after spending another half an hour pouring over countdown’s documentation. How do I implement a simple countdown that retains state across page refresh? This seemed like a common and solved problem.

So, I asked ChatGPT to generate the code for me. And I must say, it did a pretty good job. Updating the timer state in a useEffect block using a pair of setTimeout and clearTimeout and storing timer to localStorage. One fine point though, the initial value of timer was also supposed to be set by first trying to read localStorage and find a stale value which was done conveniently in a callback to useState. Neat again!

At this point, I have pushed my repos to GitHub and setup vercel projects. I could see the project coming to an end and I was elated. Let me just go through the flow one more time, I thought. I generated a new user, took the credentials and tried to login. Worked like a charm! I cleared the token and tried to login with an incorrect password and it…worked like a charm!

Sigh. Back to the drawing board. Bunch of console.logs in and I still couldn’t figure out what’s wrong. This was, by far, the hardest bug to find because I trusted the code from that blogpost I linked earlier. The problem was in the password hash matching middleware that I copied verbatim from the website. I logged the hashes of the input password and the user password (which was saved to the database) and they turned out to be different. All good up to this point. Then I realised that upon failure of bcrpyt.compare, the Promise has to be rejected. This conditional was missing. I added that and bug resolved. Two lines of code.

This bug made me realised that there are probably many more such edge case bugs lying undetected just because I didn’t write tests. I sighed. This was enough. I’m done. The user flow works properly. Authorisation works properly and I’m getting my data logged at the end of the timer which is pretty much fool-proof. I should call it a day (or night, I dont remember).

I pushed to prod, dropped a message and went to sleep. I still had to create those users and send them their credentials. And I had to pack for my flight to Heathrow. Oh well.

Generating user password was a function I copied from Stackoverflow. A 6-character alphanumeric was sufficient for the desired security level. I whipped up a utility ‘/signupmany’ endpoint on the server and fed it a JSON.stringify-ied list of users. 200 OK.

Made few changes to a python mailing script lying around and added my spare account app password. Drafted a beautiful email (Ugh, shouldn’t this be done by the secretary. I thought it’s better to avoid delays anyway) Tested. Got the email. Viola.

Ran a few more tests, added a bunch of try/except blocks (not too many though) and debug statements. Loaded the users.json and ran the script. 227 people were spammed by Purple Santa (my alt account name ^_^)

I shut my laptop and started packing. My name would either go down in history in golden letters or people will curse my name for ages. Moreover, I might have put my friendship in jeopardy. Moreover, I wouldn’t know what happened until 8pm GMT which is around 4 hours after the competition in India. Sometimes, my life decisions surprises even me.⁵

I looked back at the code and thought of some obvious edits to make the user experience better but I let it go. Those state change loaders would have been nice. And perhaps a voluntary quit button for people stuck in the middle of the game and having better things to do.

EDIT: The app turned out to be a success. Hurray! As I connected to the Heathrow WiFi, a stream of notifications and congratulations started pouring in. I was elated. People had used my code and they had loved it. I was fascinated to realise that the line between those messages being of curses and boos was ice thin. It was a bunch of try/catchs at the end of the day that saved me. That’s probably the biggest lesson of this fascinating experience. Who says that one can’t have an adventure sitting in front of a keyboard?

Oh, what a lovely day! The world is saved yet again by untested code pushed to production by a wilful developer who believes in God.

PS: I logged into the database and realised a mismatch between the number of gameEntry objects and gameData objects created. Hmmmm. So, just to be safe, I noted the startTimestamp of each user id upon first login to serve as the baseline in calculating the time deltas for each round completion later. (I have deliberately tried to keep this post jargon free but if you wish to look at the schema or the code, here they are: frontend and backend ). My conjecture is that people logged in but left the game midway/closed the tab. That voluntary quit button would have been nice after all.

PPS: (Dec 21st) Just added a Result page to the app which shows very basic personalised performance report for each user. The metrics shown were: total attempted, wins, loses, average time per solve (basically 1800 seconds divided by the total puzzles seen), and number of tries for each word. I think this additional feature makes for a cute uupgrade in terms of completeness. How the results looked like?

Total 228 participants
Average score of 31.28 (A multiplier table of 10 points for getting it right on the 1st attempt and subsequent decreasing points for more guesses was used)
Average number of attempts is 16.44
Average number of wins is 12.22
Hardest word to solve was ZOOEY (based on the number of guesses required by most people)
Easiest word to solve was ABASH (based on the number of guesses required by most people)

The podium finishes all scored 100+ 🤯 and the first had an insane average time to solve of around 24 seconds with 76% accuracy!

Fin.

Footnotes

1: Partly because I love lit club and would hate to see one more event not organised and also partly because all attempts to pass it on to other people failed 💀

2: If you’re not familiar with classic Wordle, you may check out this video.

3: I hoped Passport library would come in handy.

4: Moreover, I will obviously run basic analysis on the gameData to find out if someone cheated by flagging outliers and extremely exceptional performers.

5: Okay, I must admit. I was enjoying the adrenaline rush. The idea of writing code that people would interact with and the whole thing being setup in a delayed black box fashion made me more excited than nervous. Besides, I somewhat trust the code I write, okay? Somewhat is the operative part here, of course.

6: You might have noticed that I have been chattier than usual. That’s partly because I haven’t written in a while and this feels nice and also because I’m sitting here all alone and bored at LHR airport waiting for a senior to join.

Textile@IITD: Through the eyes of a UG junior student

gs454236@gmail.com (Basil Labib) — Tue, 01 Aug 2023 20:38:11 +0530

INTRODUCTION

I have spent a year in the Textile and Fibre Engineering department of IIT Delhi pursuing a Bachelor of Technology and a minor from the Computer Science department. My designation qualifies me to write this highly opinionated post which is a collection of my experiences and my interpretations for some of them. The value comes from the fact that I make an unbiased review of the department with no filters. This document can serve as a guide for aspiring Textile students at IIT Delhi, current students to guide them in their education, and even professors, administration, and educationists to guide their pedagogical approach.

Note: My experiences are, for better or worse, limited to the UG programme only.

THE PRIOR CONTEXT

One of the first representative experiences for first year students is the subtle branch based discrimination and the nucleation of groups based on it. Although the clustering and formation of impenetrable groups are less prominent in first year as the courses are taken with no regards to department, the bias exists and gets ugly many a times. Casual jokes and snarky remarks about JEE rank and departments are common and even stereotypes based on which department one belongs to (The IITD slang for a Textile student is a Tatta, which means scrotum in English). Just like the natural tendency for humans to hierarchise everything, there exists a well-ordering principle for branches at IITD wherein CS occupies the highest place and Textile the lowest.

This creates a deep-seated psychological dynamic in the minds of Textile peeps which is almost impossible to get rid of unless someone is extremely confident about themselves. Fortunately (and I make this statement based on observations of my batch), students who come to IITD are significantly more “aware” about themselves and smarter on average than the general populace who sit for JEE. This somewhat cushions the dark abyss into what someone can inadvertently fall into if they are exposed to the toxic first year environment very much prevalent in IIT Delhi. I can only imagine the sheer insecurity and hopelessness that a Textile student at a Tier 2 or Tier 3 college would face.

It is not a lie that Textile students lack severe engineering aptitude when it comes to academics. This is confirmed by the well-ordering principle applied to JEE ranks and subsequent CGPAS at the end of year 1 (Again, I don’t have exact data but observations about my batch supports the supposition). One problem that severely aggravates the academic divide between a Textile student and an average IITD student is the role of reservations. Being one of the least sought-after branches at IITD, most students are attracted by the brand value of IITD solely. This allows a larger proportion of not-so-academically-inclined students to get admitted to the department. However, they are expected to compete with every branch and it is simply not possible to get up to speed with a meritorious and sincere student in the first year. This leads to Textile students having some of the lowest mean CGPAs across all branches.

Now let me talk about the other side of the coin - the merit of a Textile student. As noted before, an IITian is and will always be better than the average engineering student. They may not have studied for long hours during JEE prep but most of the students are smart. This is where I’ll introduce the first problem of the department.

THE COURSEWORK

One of the first course Textile students take in their 2nd year is Textile Fibres (TXL111) - a course which is extremely heavy on factual information that one is supposed to rote-learn. There is no other way for a student to get a good grade except to remember a few numerical facts such as density of cotton etc. For an IITian, this is a serious matter of insult. At least, that’s how I felt most students take up the coursework in my experience.

I also understand that course experience is highly professor dependent but the inherent coursework is so unappealing, ancient, and disconnected with modern textile engineering practices that it ends up being heavily resented by students. I remember the course Yarn Manufacturing 1 (TXL221) I took in my 4th semester and the coursework was so disorganised and abrupt. There were very few concepts and no structure to any of them or how they relate to one another. It was impossible to follow the material through lectures and I had to pass that course somehow. The slides were very old and badly developed. It was evident that the same material was recycled each year.

THE PROFESSOR

The course coordinator or the professor is one parameter that is immutable for a student. One can opt or or drop a course but they have no control over the choice of professor at IITD. This creates very interesting dynamics and leads to further fractionalisation of students (Although, I suspect this is not a Textile specific phenomenon, it is, however, definitely a IITD phenomenon). Continuing the example of TXL221, there was a major disconnect between what the professor taught versus what he expected us to regurgitate in the exam. The marking scheme is prof-dependent and is one of the most opaque and harsh in the Textile department. So, there was a lot of surprise when the first minor exam marks came in and a random student got the highest and the class toppers got apparently random marks (myself included although I wasn’t lucky enough to land on the upper side of the class average).

Eventually, a small section of the batch realised how to game the system by learning how to write rather than what to learn. Thus, the optimisation is greatly skewed towards how to maximise marks obtained in an exam rather than to maximise learning. This can lead to a lot of interesting things such as but not limited to, following every single, however ridiculous, whim of the professor, grading policy, and exam pattern. I have seen my friends memorise everything just to regurgitate it on paper for the sake of marks. It is an extremely sad and ugly affair.

The situation is not as hopeless and horrible as it seems. There are sparks of great teaching also. Although I got the lowest possible grade in TXL221, I also got the highest grade possible in another course in the same semester. When I look back, it was a combination of a terse and extremely structured course content and an extremely passionate teacher who evidently put a lot of effort in the course. And by no means was this course “easy”, it was one of the most challenging and tricky courses for me but I knew what was happening and I think that matters a lot to students.

Course content and student expectation mismatch

Textile@IITD courses are not “easy”, they are logical and analytical and expect effort from the students but it still lacks in terms of rigour as compared to an average IITD course. Perhaps it is because of a complete lack of mathematics. Definitions are loosely defined, concepts may feel hollow at times and the course material simply superfluous.

Secondly, professors are forced to implement ridiculous policies ranging from roll call attendance to surprise quizzes just to make sure students are engaged in the class. I can understand how this is a serious pedagogical problem - how to teach something to students who are entirely disinterested in learning? However, this only lowers the reputation of the branch in the eyes of the students who feel like they are being treated as children.

Non-uniform and non-standardised evaluation

Unfairness prevails in the department as students are incentivised to mug up one day before exams, rampantly cheat without shame, and game the system in whatever way possible. A glaring example of this is seen in lab courses (TXPXXX) where marking schemes are geared towards ensuring students turn up rather than ensuring that they get something of value. It is a common practice to allocate 50% of the weightage on attendance and so on. Class participation is valued but that is mostly an eyewash as the process is highly non-institutionalised with very biassed or incorrect assumptions about what “class participation” even means. Vivas are conducted by equally disinterested TAs and are simply decorative in nature and anyone keen to game the system only has to pay a little attention. The tragedy of the situation is that students are not motivated enough to do even that.

A comparison

At the same time, take a CS course where the student-professor dynamic is totally different. The students are inherently interested to learn. The professor takes the role of a facilitator and has to simply point the students in a certain direction. As a result, attendance policies are unheard of in CS/EE courses and students have a say in course policies like evaluation schemes etc. This increases the reputation of the department and the inherent confidence of the students themselves in their eyes.

Most often, it is sufficient to pretend that one is studying than to actually learn in order to secure a good grade. This greatly discourages learning, which discourages interest, which discourages new ideas from the students, which discourages active research, which harms the department’s reputation. It is a vicious cycle. Textile engineering offers a vibrant field for some really amazing research opportunities and for anyone interested it is a gold mine, both from the academic as well as the monetary perspective. I feel this is true for any disciple of engineering simply because people would always demand better goods and services.

THE STUDENT

I began this article with the student in mind and I revisit this part of the story to tie all parts together. There is a strong correlation between the type of students who opt for a department versus the department they choose. For example, the most studious and academically bright opt for CS. A little less opt for Maths. EE consists of even less competent students and so on. Note that the word “competent” is a highly controversial term and I detest using it but it should be interpreted as “competency as measured by the JEE exam”. It is the well-ordering principle at play.

Students who opt for Textile mostly come with a set target in mind even before they begin their first year. This is a sad situation as it completely debars students to explore the liberating academic lifestyle of a university and to effectively identify their interests. Even as I write this, my friends are busy preparing hard for the intern season. As for me, I’m not sure where my interest lies and I am still exploring. I think it’s a good thing to not know. With the textile legacy, it is very difficult to do that.

Psychologically, Textile students are also surprisingly self-confident. Perhaps, this is just the experience of my batch but I find them very proactive in pursuing a number of hobbies from photography to debating and they are pretty good at it. I think it is this self-confidence that enables them to choose textile in the first place, in spite of all the lore and hostility towards it. They approach the department with the prior bias of “This is a dead branch. I have to figure out something else.”

This attitude, fueled by an outdated (or unstructured) curriculum incentivises students to be least bothered about their courses as long as it fetches them a decent CGPA. I am no one to opine if it is right or wrong but as I pointed out, I feel that the world is bigger than the nutshell we choose to live in.

THE OPPORTUNITIES AFTER TEXTILE@IITD

I shall update this section after my third year of the B. Tech programme.

However, I must note one significant experience from my pre-internship experience. Sprinklr is a sought-after tech company that recruits interns from campus. However, this year it did not allow Textile and Material Science students to sit for the pre-intern tests. The reason cited by the company’s HR was: “I don’t think these departments will be able to compete and clear the tests like the other branches.” These are the popular impressions of the department in spite of a significant number of Textile graduates securing tech placements.

WHAT IS THE SOLUTION?

As part of my training as an engineer, I am expected to provide solutions to problems and after presenting this piece, it is only apt that I do so now. Note that this is not substantiated by any data or research on my behalf apart from my experience so take them with a grain of salt.

The individual cogs of the machine work together to make the Textile department what it is today. I don’t know what incentive mechanism drives the faculty or the course development but I can vouch for the fact that the presentation makes a lot of difference simply because I have seen it happen in a Textile course. Textile courses are as important (and perhaps engaging too if they are made to be) as any other course but a combination of above highlighted factors makes it impossible for students to believe otherwise. I think one possible solution would be to reimagine the course material from the ground up and include industry-relevant material with emphasis on application of digital technologies, CAD, connection with business, entrepreneurship etc. Essentially focus on aspects of Textile engineering that are more “usable” for the students. Another solution would be to allow students to craft course policies as they please and introduce more innovative accountability measures than just traditional attendance and exams. This can include projects, reports, mini research work that includes more hands-on application of concepts taught in class. The exams must NEVER be a way for students to test their memory muscles. All this will require a major effort from both the faculty and the student body at the humble Textile department at IIT Delhi.

The Murky Mempool: Making sense of {M|B}EV

gs454236@gmail.com (Basil Labib) — Thu, 27 Jul 2023 23:30:59 +0530

It is human tendency to become unconditionally enchanted by something popular regardless of its intrinsic value. Known by many names like network effect , herd mentality , etc., this psychological effect has a diabolic effect on information dissemination. This is especially true when there is high variance in the educational distribution of the population. Take the room temperature semi-conductor craze for example. Or the plethora of medical news during the 2020 pandemic. Or the invention of the web. You get the point. It’s everywhere.

Something similar has happened to the blockchain technology, made popular by a 2008 anonymous paper . The real information about the tech and its implication is so sparse and shrouded in so much nonsense that the market around it has emerged as a scam/fraud leading to less credibility and trust.

In this post, I wish to discuss how blockchain tech may not be the panacea of decentralised commerce as it has been once been marketed as. I will discuss design flaws inherent in the Ethereum chain and what it means for the properties that enabled Ethereum to differentiate itself from traditional financial systems.

Note: This post assumes the older Ethereum consensus mechanism, namely Proof-Of-Work.

Frontrunning

Front-running (noun): The practice by market-makers of dealing on advance information provided by their brokers and investment analysts, before their clients have been given the information.

Let us take an example. Imagine there are two parties Alice and Bob. Alice somehow gets to know that Bob is about to make a “big” trade, that is, he’s about to purchase a lot of cookies. So, just before Bob’s purchase order, Alice goes to the market and buys a lot of cookies for herself thereby raising the price of cookies. Now, as Bob’s purchase order is processed, he has to pay more money for the same commodity. Alice has managed to extract money from Bob simply by knowing about Bob’s purchase order in advance. Note that there’s also a bunch of other factors, most importantly the design of the trading platform that Alice and Bob are using that affects how big this problem can be. In some ways, frontrunning is a fundamental design flaw in the very act of trading.

Traditional (that is, centralised) financial trading firms use a purchase order book to match up buy and sell orders. If Eve had access to this list of buy and sell orders, she would be able to anticipate the price changes that some of those orders would have and frontrun those transactions to make money out of thin air. Frontrunning is illegal on CeFis and govermental agencies oversee that brokers and trading agencies don’t engage in such activites.

Exploitable opportunities

What kind of purchases can lead to frontrunnable opportunities? Large volume orders are an obvious example. A buy order of 0.5% of all tradable ETH would have a substantial impact on the price of ETH. It would also create arbitrage opportunities across multiple markets that would decay based on the oracle latency between them. Even human errors in placing buy and sell orders can lead to frontrunnable scenarios in DeFi as we’ll see later.

Unlike CeFi, DeFi are designed to be transparent, that is, transactions are public. One cannot be decentralised if one cannot see what’s going on…which makes sense. But now, let us understand how this design feature aggravates frontrunning on blockchain and reduces consensus security.

In the Ethereum protocol, local transactions are broadcasted via a gossip protocol to nearby nodes and are processed by rational miners based on the decreasing order of transaction fees. However, there is always a publicly accessible set of unconfirmed transactions called a mempool . It contains transactions that might potentially fail or be valid but nobody knows. As you can imagine, this presents perfect oppurtunity to frontrunners who can examine the mempool and frontrun profitable transactions without a fuss.

An example

Most DeFi exchanges are based on the purchase orderbook mechanism in which a centralised clearing house matches a list of buy orders with a list of sell orders when they find compatible trades. In a DeFi, the central authority is, of course, the immutable logic of the smart contract behind the exchange. Note that this is a different design than the more innovative automated market maker model introduced by Uniswap. Now, let us see how a frontrunning transaction can exploit an erroneous trade order.

Let’s assume that someone offers to buy an asset at 10x the market rate (perhaps due to a slip while filling the price or a decimal point error). A bot would identify this opportunity and quickly buy that asset from the market and then sell this back to the original buyer. This is possible because of the inherent atomicity of transactions on the blockchain via so-called blocks. Moreover, the incentive mechanism for miners to “mine” a particular trade is assumed to be strictly dependent on the transaction fee they reap out of the order. Hence, in order to prioritise a transaction, one only has to set a marginally higher transaction fee to frontrun a transaction. We’ll see how this leads to an interesting phenomenon.

On the flip side, let us assume that someone offers to sell their token at .1x the market rate (again, an error). A frontrunner might buy those token immediately and sell them in the market and gain 0.9x profit.

Arbitrage and frontrunning situations can arise from a number of factors such as price difference of the same asset across markets and the different response rates for market movements based on platform design, logic bugs, human error, etc. Generalising this, a sophisticated frontrunning bot will replay a number of shortlisted transactions in the mempool in a local environment and try to frontrun one or a sequence which leads to a net positive profit for itself. The exact profit is difficult to calculate simply because it is not possible to anticipate the subset of transactions visible to a bot which depends on a number of factors such as latency, network congestion, and the well-connectedness of the bot.

What I described above is a particular type of exploiting frontrunnable transactions dubbed “pure revenue opportunities” by this paper . Due to the unique public nature of these transactions, a victim can see what happened with their funds. I reproduce few comments made by victims of such bots from the paper linked above.

“I am a single parent trying to make ends meet at a job I hate. Please have some mercy.”

“Please I ask you to send me the ETH back as I really need it to continue my education. I might need to sell my car to pay what is already due for this semester.”

“This was obviously a mistake transaction - any chance you can find it in your heart to send it back?”

Priority Gas Auctions (PGAs)

The mechanism for exploiting an abitrage opportunity is by placing your order before the victim order. In the blockchain paradigm, this translates to assigning a higher transaction fee to your transaction to incentivise a rational miner to include the attack transaction first. But an interesting phenomenon happens if two bots find an exploitable transaction. In this case, a naive strategy bot A would keep increasing their transaction fee in response to bot B, following the same strategy, and trying to nullify A’s attempt to reap a profit. This leads to interesting series of transactions with the same nounce but incrementally higher gas fee by two wallets for the same trade - a pretty conspicuous phenomenon.

Due to the stochastic nature of block inclusion time (which depends on the miner being able to find a solution to a cryptographically hard problem), the transaction to be actually included is not pre-determined.

Source: Flash Boys 2.0

Note this interesting PGA battle where two bots engaged in a biding war. Eventually, the green transaction for 0x6BEcA... was included at 134.02. Interestingly, the bots agreed to raise their fee to ~8K. The other transactions were, of course, discarded from the mempool once the winning transaction was mined.

Bots can adopt different strategies for bidding in such auctions. Two of the simplest and observed strategies are:

Blind raising: Simply increase the bid by a fraction until the order is successful. This is a non-adaptive strategy and doesn’t take into account what the other agent is upto.
Counterbidding: As the name suggests, only raise the bid if the current bid will lead to failure and raise the bid by a certain fraction of the other agent’s bid.¹

Moreover, the winner of a bid war is highly dependent on the latency of the network and the connectedness of the bots to the network, hinting at the development of high-frequency trading-like scenarios on blockchain platforms ; an undesirable and unintended consequence of the current design of the system.

Transaction reordering and unintended optimisation

Once in the mempool, the miners have a lot of freedom in choosing which transactions to include in the block they’ll mine next. The protocol assumes that rational miners would order transactions by decreasing transaction fee which is the intended behaviour. However, if the miners have a greater incentive from outside the system to deviate from this behaviour then it leads to lowered network security. Studies show that at certain thresholds, miners are incentivised to even fork the chain.

If a miner is aware of a transaction sequence that would lead to profitable outcome or even personal connections with the transaction participants, they can reorder transaction orders if the profit from such reordering is greater than the transaction fee. When can such scenarios occur? Once large volume trades are detected in the mempool, miners may identify potential frontrunnable transactions or more sophisticated types called “sandwich attacks” (flank the victim transaction with two attacking transactions to bet in favour of a condition that is determined to occur as an outcome of the victim transaction). Miners can collude with third-party users and reorder transactions to maximise the block reward they can mine. They may also censor transactions of a particular kind, type, demographic, and so on. One quickly realises how this design pattern can fundamentally compromise blockchain security.

Conclusion

While transacting on the application layer, it is often possible that endusers abstract away the complexity of what goes under the hood on the Ethereum protocol which enables them decentralised consensus. But we find that the mempool is not a utopic panacea that will enable blockchain to replace traditional CeFi. This primarily arises from fundamental design flaws that were not originally conceived - an ideal case study to understand why engineering is hard.

I haven’t even scratched the surface of how profitable these “extractable values” have been for adversaries in the past (for numbers, check out the Flash Boys paper) and still continue to be a source of income for sophisticated bots. The original paper Flash Boys 2.0 dubbed this value as MEV or Miner Extractable Value but after the transition to PoS, it was proposed to name it as Maximum Extractable Value. This paper argues that MEV is a subset of total extractable value which it dubs BEV or Blockchain Extractable Value. So, even if the terminologies are not very definite, I hope the problem is.

Well, the above picture might seem more grim than the situation is. Currently there is amazing work going on which will nullify wealth stealing by transferring the incentives to the users and miners. An interesting middle layer is being built - for example, MEV Boost for PoS-enabled blockchains.

References

paper Flash Boys 2.0: Frontrunning, Transaction Reordering, and Consensus Instability in Decentralized Exchanges: Daian P. et. al
paper Quantifying Blockchain Extractable Value: How dark is the forest?: Qin K. et. al

1: I am simplifying a lot of details here. For example, transaction fee increments are not continuous. Most ETH clients require a bid to be atleast 12.5% of the initial bid to be considered for inclusion thus making bidding discrete in practise.

How Bonding Curves Are Crucial to DeFi Protocols

gs454236@gmail.com (Basil Labib) — Thu, 06 Jul 2023 14:26:12 +0530

Source: Google images

After the initial excitement, the current state of DeFi rests with the spawning of a number of speculation and lending markets (such as Compound , AAVE ), complementing their traditional counterparts. For a comprehensive bird-eye view of the history of DeFi, I strongly recommend this post by @njess .

Speculation markets work by providing a platform for traders to bet on the value of any digital asset over time. DeFi is experimenting with devising financial instruments like futures and options for digital assets as evident by a number of perpetual futures protocols like Perpetual , dYdX , Mycelium , etc.

AMMs – a DeFi primitive

However, when the exchange is small and relatively unknown, it is difficult to exchange tokens/assets issued by the exchange for ETH or any stablecoin because one won’t be able to find a buyer or seller. In other words, the exchange doesn’t have enough liquidity. Of course, the smart contract implementing the exchange protocol will provide the liquidity initially but at what price? In other words, what should be the mechanism for price discovery ?

This was the problem that Uniswap protocol designers were grappling with when they launched Unisocks in 2019, a smart contract to exchange socks but equipped with an automated market maker or AMM. Based on a formula, the contract was able to quote a price for an asset on demand. The Unisock protocol used something like the $ x*y = k$ formula to define the number of x and y in circulation. This ensured that there was always enough of x and y in the contract to trade anytime with anyone. As @njess writes,

…very few people realized the power of having a permissionless market always willing to trade. Not an efficient market. An available market.

We’ll dive into the math soon but first, we have to ask a deeper question.

Why should one buy tokens?

For the same reason why one buys any other asset. Because they believe that the asset will appreciate in value overtime. Note that the asset itself doesn’t need to have intrinsic value. Things like paper currency notes, or application of oil on canvas paper are absurdly overvalued if one believes their real value are strictly tied to their intrinsic value. Most of the time, value is conferred by other people and tokens are just another such asset albeit digital.

Infact, web3 protocols are more efficient than traditional finsystems at matching value-valuation levels of an asset if they are well designed as tokens actively incentivise holders towards certain actions. This is an area of active research and people are experimenting with curation markets where they incentivise users to report non-credible, fishy news. Encouraging such positive social behaviour would make such protocols appear valuable and so the tokens would appreciate in value (in other words, get more in demand).

Bonding curves

In finance, a bonding curve is essentially a function to describe the relationship between the price and supply of an asset. Bonding curves are what powers liquidity pools and other exchange protocols. Since exchange is such a fundamental action in DeFi, AMMs are one of the primitives of DeFi.

A typical bonding curve looks like:

$$ y = mx^n $$

where m is a curve parameter (to fine-tune the shape) and n is the order of the curve. y is the token price and x is the token supply.

The power function for various values of m and n. Source: author, plotted on desmos.com

If we take the simplest curve, $ y = x $ which is the bonding curve for our token ABT (Alice-Bob token). Let us see how it affects user behaviour.

The area represents the price for the tokens at different supply levels. Source: author, plotted on desmos.com

If Alice buys token when the supply is low, she has to pay a small amount. She bought 3 tokens when the price was 3 ETH and now the tokens in supply is 6 tokens so the token price rises to 6 ETH. Bob notices that the token has increased in demand so he wants to have some of ABT too. He buys 2 ABT for 6 ETH each. Now, Alice can sell her stock of ABT as the token price is higher than her cost price and reap a profit out of it. Meanwhile, Bob would incur a loss if he sells after Alice has sold her stock since the token price is higher than his cost price. Thus, this curve incentivises users to buy early and rewards early adopters.

I should clarify that the statement “He buys 2 ABT for 6 ETH each” is not correct but a simplification. Since these are continuous curves, the price of assets are continuously changing, hence the price of ABT would increase for even a $ 10^{-18} $ part change. Thus, to compute the cost for 2 ABT, the contract would actually compute the area under curve for the function under appropriate limits. This can be done by good ‘ol integral calculus.

$$ \int mx^n dx = m \int x^n dx = \frac{m}{n+1} x^{n+1} $$

Let’s say that the number of tokens in circulation was t, and we want to buy k more tokens. By putting limits,

$$ p = \left[\frac{m}{n+1} x^{n+1} \right]_t^{t+k} = \frac{m}{n+1}(t+k)^{n+1} - \frac{m}{n+1}t^{n+1}$$ $$ = \frac{m}{n+1}t^{n+1} \left( \frac{(t+k)^{n+1}}{t^{n+1}} - 1 \right) $$ $$ = \frac{m}{n+1}t^{n+1} \left( \left( 1 + \frac{k}{t} \right)^{n+1} - 1 \right) $$

By solving for k, we can find the number of tokens that one can trade for p units of ETH or stablecoin, likewise:

$$ k = t \left( \left( \frac{p(n+1)}{t*s^{n+1}} + 1 \right)^{\frac{1}{n+1}} -1 \right) $$

The problem with the above formula is ofcourse the pesky exponent term which can lead to overflow for large t or n. How can we compute p and k efficiently. We need some math trickery and this is exactly what the bancor formulae are.

Bancor formula

If we define the initial pool balance that is the cost of going from 0 to t as another variable, say b, then the above expressions can be written as follows:

$$ p = b \left( \left( \frac{k}{t} + 1 \right)^{n+1} - 1 \right) $$ and, $$ k = t \left( \left( \frac{p}{b} +1 \right)^{\frac{1}{n+1}} - 1 \right) $$

We still have to compute the exponent but we are getting there, bancor formula introduced another variable called the reserveRatio as follows:

$$ r = \frac{poolBalance}{tokenPrice * tokenSupply} $$ $$ r = \frac{b}{p*t} $$ which is $$ = \frac{\frac{m}{n+1}t^{n+1}}{(m*t^n)*t} = \frac{1}{n+1} $$

Thus, the price formula under the bancor parameterisation becomes:

$$ p = b \left( \left( \frac{k}{t} +1 \right)^{\frac{1}{r}} - 1 \right) $$

If you have had some prior exposure to basic economics then the above bonding curve seems a little disconcerting. Why is the price more when there are more tokens in circulation? Shouldn’t the price be more when the tokens are scarce? But think about it for a minute. More tokens in circulation means more demand. It is not like more tokens implies less scarcity.

Sigmoid functions as bonding functions

People coming from ML are already acquainted with the sigmoid function, a friendly little function with a nice shape.

$$ g(x) = \frac{1}{1-e^{-x}} $$

However, the integral of this function is:

$$ \int\frac{1}{1-e^{-x}}dx = ln(1-e^{-x}) $$

which is a hairy function to compute on the EVM as an infinite power series has to be summed. Smart contracts ideally cannot implement fancier curves as they cost more gas price to compute on-chain. But there is a similar function with a much nicer integral:

$$ f(x) = \frac{x}{\sqrt{1+x^2}} $$

and, more importantly,

$$ \int\frac{x}{\sqrt{1+x^2}}dx = \sqrt{1+x^2} $$

Now, just like m and n in the power bonding function, we can parameterise this function as follows and compute the sigmoid price function:

$$ y = a \left( \frac{(x+b)}{\sqrt{c + (x+b)^2}} -1 \right) $$

which when integrated within proper limits yield the following price function:

$$ p = \int_{t}^{t+k} a \left( \frac{(x+b)}{\sqrt{c + (x+b)^2}} -1 \right)dx $$ $$ = a (\sqrt{c + (t+k+b)^2} + t + k ) - b $$

which is much easier to compute.

Price discovery mechanism

Quantitatively, the value of a token can be defined as the sum of total cash flow available to the bearer. But this raises many questions such as is it always possible to determine the future cash flow from an asset? If yes, then how to compute that? Traditional pricing techniques have evolved tools to price options and futures such as Black-Sholes formula, etc. but I haven’t researched much into that. The precise token design is also affected by the amount of token in circulation. Is it possible to value a token in the future if one allows infinite token minting by design? How to do that? These questions are under active research and no doubt, bonding curves will be an important part of future DeFi protocols.

Problems with bonding curves

Not a problem per se but I just wanted to note that AMMs are useful to provide liquidity to a token when it is in its infancy. But once the token is liquid enough to be traded on the free market at other DEXes, then who sets the price? Well, then the AMM and the free market stay at the same price. Any difference is levelled out by arbitrage.

Frontrunning a transaction

Do you remember the case for ABT token above? Well, what if Eve got to know that Alice is about to buy her tokens, somehow places an exact transaction with a higher gas fee to incentivise miners to mine her transaction as oppose to Alice’s and then sell her tokens to Alice at a higher price (since the token’s price has just increased). One possible solution is limiting the gas fee that one can set for these transactions and always ask buyers/sellers to set it to that value.

Nested bonding curves compounds risks

Imagine that ABT is pegged to ETH with a reserve ratio of 50%. Now, Darlene designs a token and pegs it to ABT with a reserve ratio of 10%. The actual reserve ratio is .5 * .1 = .05 or 5% of ETH. Further nesting can result in a highly volatile asset which basically means huge changes in the value of the asset for tiny changes in the value of the underlying asset (ETH in this case). One possible solution is to limit the number of nesting, of course.

REFERENCES

How to make continuous bonding token models by Slava Balasanov.
Bonding Curves in Depth: with math by Slava Balasanov.
Token Curve Design by Parameters .
Converting between bonding curve and bancor parameters by billy Rennekamp.
Ethereum tokens explained by Linum Labs.
Introduction to bonding curves, shapes, and use cases by Linum Labs.

Playing God: Making sense of CRISPR

gs454236@gmail.com (Basil Labib) — Wed, 22 Feb 2023 16:29:20 +0530

Abstract

Imagine standing on a long corridor with closed doors that stretches to infinity on both ends. One keeps walking along and knocking on each door, maybe trying to pick the lock, peeping through the keyhole, or even trying to break in. But the process is tiresome and hopeless. And yet once in a while, someone manages to open a door and view a world they have never imagined. The story of scientific innovation is roughly similar to the story of the endless corridor.

In 2015, Junjiu Huang and co. at the Sun Yat-Sen university in China modified the gene responsible for beta-thalassaemia , a genetic blood disorder leading to anaemia, in non-viable embryos. Through this experiment, they had opened a new door - a door that led to a utopia of a disease-free humanity and, if I get ahead of myself, of “designer” humans and GMOs.

In this post, we’ll dive deep into CRISPR, the gene-editing tool that has taken the world by storm. The focus of this post will be on the technology rather than the social, political, or governance fallout of the tool. Previous understanding of genetics and biochemistry is recommended.

Adaptive immunity and CRISPR

Adaptive Immunity System in bacteria and archaea, stage 1

Adaptive Immunity System in bacteria and archaea, stage 2

Biological systems are known to be highly resilient towards a changing environment and this resilience stems from their ability to adapt quickly to changing environmental parameters. Bacteria and archaea protect themselves from viruses and plasmids by an adaptive immunity system. The crux of the mechanism is based on recording the genomic fingerprint of the intruder and to deploy proteins that splice future invaders based on the “memory” that it has developed over time. Short sequences of the invader’s genome are integrated into the bacteria or archaea’s genome. These short sequences are called Clustered Regulatory Interspaced Short Palindromic Repeats or CRISPR. These sequences along with CRISPR-associated proteins or Cas proteins form the machinery that enables this feature.

The CRISPR locus have two types of sequences, a highly variable region, called spacer, that constantly gets updated with new incoming invader genomic data and non-contiguous repeat sequences that separate them. Upon transcription, this stretch of DNA is expressed as a special RNA sequence called crRNA (CRISPR RNA) which doesn’t undergo translation but serves as activation director for Cas proteins. The crRNA enabled Cas9 is, then, able to bind to DNA sequences that are complementary to the crRNA and cleave them. The degradation of the foreign DNA renders the attack void and thus, immunity is gained.

CRISPR-Cas9 system: How it works

Block diagram showing the pathway of the crRNA

The interplay between CRISPR and Cas proteins that impart immunity to the bacteria/archaea. Three different types of CRISPR-Cas systems have been identified by researchers: Type I, Type II, and Type III. All systems differ in their exact mechanism to generate crRNA and Cas proteins and eventually catalyse the cleavage of DNA. Our focus will be to understand Type II systems owing to their simplicity and adoption in gene-editing. It uses a single Cas protein called Cas9 and two RNA components. We have met one of these components namely crRNA. The other component is transcribed from a locus upstream of the CRISPR locus on the mother sequence. This sequence is called trans-activating CRISPR RNA (tracrRNA). The tracrRNA has a region complementary to the repeat region of the CRISPR locus and thus can bind to the transcribed crRNA (I’m simplifying details here as there’s something called a pre-crRNA which undergoes epigenetic modifications). The double stranded RNA (tracrRNA and crRNA) undergoes cleavage by a RNA nuclease called RNAIII and the crRNA-tracrRNA complex binds with a Cas9 protein to form a ribonucleo-protein (RNP) complex.

The Cas9 protein can be termed as an RNA-activated endonuclease as it can specifically cleaves DNA that are complementary to the associated crRNA strand. The Cas9 looks for spots along the invading DNA sequence after binding to regions on the invading plasmid DNA called a Proto-spacer Adjacent Motif or PAMs. There are different PAM sites associated with different Cas9 proteins. For example: S. pyogenes Cas9 (SpCas9) recognises a 5’-NGG-3’ PAM site and it is most commonly used for gene editing. Finding different Cas9 proteins to attach to different PAM sites gives the geneticist a versatile toolkit and increases the efficiency of the editing but we’ll get to that later.

In the case of SpCas9, two arginine residues bind to the guanine nucleobase on the PAM site (the two GG from NGG and N could be any nucleotide base). This interaction positions the phosphate of the DNA backbone to a phosphate-lock loop on the Cas9 and facilitates DNA unwinding. After unwinding, the crRNA forms a RNA-DNA hybrid called an R loop and then cleavage occurs. There are two domains on the Cas9 namely: HNH and RuvC-like. They individually nick one strand of the DNA resulting in a double strand break. The HNH nicks the strand complementary to the crRNA and RuvC-like nicks the other strand. This cleavage happens 3 base pairs (bp) upstream of the PAM site in case of SpCas9. As the cleavage is successful, the invading DNA is rendered harmless as it cannot be expressed effectively anymore.

DNA repairs and motivation for editing

Homology-Directed Repair

DNA-RNA systems are highly versatile and robust. In response to a Cas9-induced double break, the organism employs machinery to repair the DNA. There are two types of DNA repair - Non-Homologous End Joining (NHEJ) and Homology-Directed Repair (HDR). NHEJ can be either canonical NHEJ (c-NHEJ) where the two ends are simply “glued” back together or alternative NHEJ (alt-NHEJ) where each strand is cut out to repair the damage. Both these pathways are highly error-prone resulting in unintentional insertions and deletions.

On the other hand, if there is a complementary DNA strand nearby the double break, the homologous DNA can easily bind to repair the break. This is the Homology-Directed Repair pathway. HDR is exploited to introduce engineered sequences into the genome by making the edited strand available near a double break. Thus, to edit a gene, we need to make a cleavage at the desired location which is achieved by the CRISPR-Cas9 duo by making sure the crRNA is transcribed and RNP complex is formed. Further, we check for the availability of a PAM site near the point of interest and finally, to make sure that the engineered strand is available during repair.

A lot of research and study has gone (and is still going) into the role and importance of the various machinery involved in gene editing. The tracrRNA seems to be important for DNA cleavage by Cas9. Additionally, instead of introducing separate tracrRNA and crRNA, researchers can introduce a single guide RNA or sgRNA. This reduces the number of components involved and makes the process simpler.

Steps of a gene edit

Let us investigate the steps needed for an edit in greater detail. The process can be divided into roughly three steps:

The selection of a suitable target and guide.
Introduction of CRISPR-Cas9 and template DNA components into the organism.
Identification of the mutation.

Target Selection

In the case of SpCas9, there is a 20-nucleotide sequence of guide RNA adjacent to the PAM site, 5’-NGG-3’. For best results, the PAM site should be as close as possible to the mutation locus. In fact, the efficiency for a mutation is inversely correlated to the number of base pairs from a PAM site. This is important because if the specificity of the target is not sufficient then the Cas9 can bind to other off-target regions and splice at unintentional locus resulting in edits that we don’t want. Factors that affect RNA efficiency is an active area of research in CRISPR research today. For example: the presence of purine bases (G or A) at the 3’ end of the 20-nucleotide guide is known to increase efficiency.

Introduction of different components

Cleavage of DNA by Cas9

There are primarily three forms in which we can introduce the components:

DNA form

In this pathway, we have to introduce the DNA sequences for sgRNA and Cas9 protein. The point being that we wholly rely on the translation and transcription facilities of the cell for the production of Cas9 protein and sgRNA. Now, as the Cas9 DNA is transcribed into its complementary mRNA, the presence of appropriate 3’ and 5’ untranslated regions (UTR) is ensured for successful translation into the protein. Lastly, a promoter must also be present upstream of the transcription site to bind a RNA polymerase. A promoter must also be present for the transcription of the sgRNA sequence by RNA polymerase III, which is a polymerase which specialises in the expression of ribosomal RNAs and tRNAs. Note that the ultimate product of the sgRNA pathway is RNA and not protein.

RNA

Instead of depending wholly on the cell to produce the required components for an edit, we can ease the burden by synthesising few components outside the cellular system. In this pathway, we introduce sgRNA and mRNA encoding Cas9 instead of DNAs for the same. However, the mRNA needs to be modified post transcription with a 5’ cap and a 3’ poly-A tail.

RNP/Protein complexes

Following the same theme, we can synthesise Cas9 itself and introduce the Cas9 complexed with tracrRNA and crRNA as the RNP complex entity. This pathway is ideal if we do not have the requirement to optimise for the species-specific UTRs and promoters. However, the purification of the protein and complexing in-vitro increases the cost of introduction of the components.

Identification of the genetic mutation

The efficacy of gene editing can only be determined if we are able to identify the desired mutation in the target genome. Identification of genetic modification is a significant bottleneck in the modern CRISPR workflow. There are two types of methods:

Screening
Selection

Screening

In screening for a specific phenotype, one can screen for the corresponding molecular change as a result of the edit. One particular assay is called restriction fragment length polymorphisms (RFLP) analysis. The idea being that the region of interest after a mutation is amplified by PCR and then treated with a restriction endonuclease which cuts a specific DNA sequence. If the enzyme is able to cut, then we still have the unmutated sequence and there is no edit and vice-versa. The DNA fragments can easily be visualised by gel electrophoresis.

However, one disadvantage is that the mutation itself can create another restriction site. There are other endonuclease-based assays such as CEL1 and bacterial T7 endonuclease that recognise and cleave single bases. They have the advantage that they do not rely on creation or destruction of restriction sites. But this process requires a large number of candidates and is thus time consuming.

Selection

Selection relies on the fact that lethal template DNA would kill the cells but the ones with the mutation will not be killed. So, a drug-resistant edit is achieved in the cells with a successful splice whereas other cells are killed by the introduction of the HDR template strand.

Variations of the core CRISPR-Cas9 model

Expression of Green Fluorescent Protein (GFP) in lab mice

In this section, let us look at other exciting applications of the CRISPR-Cas9 system. The CRISPR gene editing technique derives its utility from the specificity of the Cas9 incision. Now, we can increase the specificity of a particular cut by randomly deactivating one of the domains of the Cas9 protein. In this case, Cas9 proteins are only capable of nicking one strand on their own and it would require two Cas9 proteins to make a double strand break. This increases specificity and protects against unintentional splicing.

Another point of interest is the PAM site and its essential presence for the Cas9 to bind. Researchers are looking at other proteins that bind to different regions - one such candidate is Cpf1, which recognises T-rich PAM sites and cuts at a 5’ overhang resulting in a protruding ragged end. This increases the robustness of the process as biologists can now target AT rich sites as well.

Specificity of the CRISPR-Cas9 systems allows gene editing to be successful but the splicing is only the second step of the process of correctly identifying a sequence of interest. If we can deactivate the Cas9 proteins (called dCas9 in the business), and load a custom domain on the Cas9 then we can achieve more varied operations at a particular locus of interest. It’s like having an operating table running along the DNA strand. For example, if we fuse the dCas9 with a domain that represses transcription then we effectively silence the expression of the targeted genes. If the payload is a promoter, then we can activate transcription at the targeted site. We can introduce epigenetics modifiers such as methylators or histone modifying enzymes, all of which modulate transcription at the target site. A fluorescent protein called Green Fluorescent Protein (GFP), derived from jellyfish, has been attached as payload to the dCas9 and used to visualise target genomic sequences complementary to the sgRNA.

Conclusion and ethical concerns

We have certainly come a long way from where we started. If I have been able to even give you a gist of what CRISPR is, then I consider this post a success. All errors and bad explanations are my fault. We began this post with the chronicle of the Chinese scientists who eliminated beta-thalassaemia from non-viable human embryos. Under very close regulation, another research group edited heterozygous embryos for a mutation in the MYBPC3 gene using CRISPR. This mutation resulted in hypertrophic cardiomyopathy, which can cause sudden death.

We have unlocked a truly novel technology, one that allows us to reach out and rewrite who we are but at the same time, we are also accountable for what we do. I guess that is the price we pay for knowledge.

References and Acknowledgements

Molecular Biology at the Cutting Edge: A Review of CRISPR/Cas9 Gene editing for undergraduates - Thurtle, Schmidt, Lo, et. al.
GCR Biosecurity curriculum: Biosecurity Fundamentals
Homology-Directed Repair animation video - Oxford Academics on YouTube

Prometheus Unleashed: Making sense of information hazards

gs454236@gmail.com (Basil Labib) — Wed, 15 Feb 2023 11:18:29 +0530

Intro

As part of my research into global existential risks, I came across the conundrum of information hazard which simply refers to the potential misuse of scientific or technological information by bad actors that might turn into a global risk for everyone.

This phenomenon is most aggravated in the field of biotechnological research where misuse of increasingly openly available genomic data or cheaper protein synthesis techniques can easily be used to develop and mass-produce humanity-threatening super-pathogens (“super” referring to immunity against all known forms of human cure). This is a particularly interesting conundrum as the fine line between openness and secrecy governs the difference between better scientific research and global catastrophe.

It is easy to say that all biotech information and research generated should be classified by labs around the world. But this conservative approach would turn dry the flow of useful information and data that is so important to inspire and facilitate future research. Although this approach guarantees safeguard against information hazard, it also means the end of all future scientific research and technological innovation. Such a scenario will leave humanity helpless in the face of attack by a naturally emerged pathogen.

On the other hand of the spectrum, completely free proliferation of potentially biohazardous information, if accessed by a malicious organisation or even a bad actor with the required expertise and material, can spell doom for humanity.

Thus, it can be concluded that the censorship on scientific information and data should lie somewhere in the grey of the spectrum. Where is that point?

The secrecy point question is more nuanced than it looks. For example, consider this scenario. A biotech lab publishes unsuspecting non-hazardous information. This information is accessed by a malicious group which was lacking some non-hazardous information to bring its “evil project” to fruition. In the light of the newly published (unsuspectful) data, it can now generate the required hazardous information it needs and bring its evil plan to life. Notably, this scenario is quite narrow (one can argue, if the malicious group has the resources to do independent R&D, then it can generate whatever information it needs independently. Or that such expertise is not known to be within the resource constraints of any known terrorist outfit). Sure, but what I’m trying to present is the subtleties and intricacies that surround such censorship and governance issues and truly calls for a departure from the binary of black and white approach taken by governance organisations today.

Case studies

The US government policies on dual-use research is solely restricted to work that fall under any of its seven classes of “experiments of concern” or if it involves a subset of organisms on the Federal Select Agent List. It is not hard to imagine that emerging research may not fall under this classification and yet be of catastrophic concern to humanity.

Or take this second scenario that seems to defy common sense. It is beneficial to publish easy-to-discover information hazardous research as it would benefit good actors and the publication of the work is indifferent for the bad actors as they would discover the information themselves sooner or later. This loudly opens a new can of worms by asking where to draw the line for “easy-to-discover”?

Moreover, keeping useful information hidden may lead to unintentional misuse of currently deployed technology or slowdown preventive measures in case of large-scale risks. An apt commentary on the reassessment to a publication concerning the identification of a subtype of the neurotoxic botulinum toxin reads as follows:

Although it is ethical to identify and mitigate DURC, it is also an ethical imperative to enable others to counter potential harm with good. With critical national security and public health at stake, it is unethical to impede research competitors for personal, professional, or commercial motives. Likewise, excessive government regulation is not helpful if it slows the progress of countermeasure development (Keim, 2016, p. 333).

Incentive games

There are various notorious and perverse incentive games at work too. Take for instance, the decry against biological weaponry in the Geneva protocol during WW2 which informed the Japanese of the potential utility of biological weapons and eventually inspired them to develop and use their own B-weapons (is this a word?). Incentives act on all levels of society. Scaling the academia ladder forces researchers to publish their work, unfortunately often bypassing ethics and common-sense, in order to be promoted or just receive funding. They are often left at the discretion of the policy-makers or fund dispensers who have little to no idea of the potential impact of the research (both good or bad).

The bigger picture

The progress of scientific and technological advancements are asking more existential questions than ever before and at a rate that is simply beyond the mental grasp of the common human and barely understood by a select few. Taking a meta look at the information hazard problem, it can be summarised as the inability to infer the intention of human beings. Take for instance, the open problem in security and blockchain dubbed the “Oracle problem” - which basically asks if it is possible to validate any information generated outside the chain by the chain itself. If one examines the question carefully (the leisure of which I do not exercise in this post), it would be apparent that the fundamental problem lies at the interface between the real world and its equivalent projection in the digital space. For eg: in web2, the identity of a person is closely tied to its account on some database. This abstraction worked because the proof that that person’s existence was guaranteed by some central authority. But with the elimination of the central authority, who backs the data? How do we even define identity in such a case? The premise for the abstraction breaks down. The authentication situation is not as dire as it looks as people are trying to find workarounds but I intend to present the bigger picture here.

Solutions?

So, we understand that this is an important issue. But what is the solution? To be honest, there doesn’t seem to be any fool-proof solution because the problem is on a different plane as noted in the previous paragraph. There are various suggestions such as assessing the bad (and good) impact of an information promulgation beforehand. Or consciously disclosing information in a way that benefits good actors while posing difficulty for bad actors to use it for malicious plans - “Security by obscurity”. Identification of entities who access a particular piece of information may be an option but I only see it eventually dwindling into authoritarian monopolistic control over the information as “someone” has to decide if the information should be made available to a particular entity or not.

Conclusion

In the timeless words of Dickens, I think this is “the best of times, the worst of times”, an exciting time to ask questions, a vibrant time for research,and an amazing time to be alive. The answers we give to ourselves in this century may potentially decide our fate.

Acknowledgements and References

Lewis, et. al - Information Hazards in Biotechnology, 2019

A deep dive into JSON Web Tokens (JWTs)

gs454236@gmail.com (Basil Labib) — Mon, 22 Nov 2021 08:03:03 +0530

What is JWT?

A JSON Web Token or a JWT (pronounced “jot”) is a plaintext, human readable piece of data that enables authentication and trust between two unknown entities.

Why use JWT?

There are alternative competing standards such as SAML (Security Assertion Markup Language) that cater to the same needs like a JWT but JWT has certain advantages that has made it quite common, almost de facto in securing modern web applications.

It is stateless. It means the server or the client don’t have to maintain any piece of data in-memory in between requests for it to work.
It can be delegated. It means the task of generating and issuing a JWT can be completely disjoint from the core application itself, which can be delegated the task of only validated them. Less complexity leads to more robust and bug-free applications.
It is simple. In terms of the complexity of data (and hence the size of a JWT) which makes them ideal for low-bandwidth regions.
It is efficient to compute. A JWT is generated and validated. Both these processes can take place without considerable constraints on resources, making them ideal for hardware critical applications.

Why understand it?

Most developers and builders can get away without truly understanding the underlying mathematics or algorithms that makes JWT works and that’s. . . fine. There are tons of battle-tested, production ready libraries that enable JWT generation and validation, even third-party services like Auth0 , to whom one can delegate the entire process of authenticaing via JWT for their applications.

But, superficial knowledge gets one only so far. A thorough understanding of this technology will equip you to make sound decisions when designing an architecture or troubleshoot elusive JWT errors. It will empower you with a sense of self-assurance and confidence that one gets by mastery.

How do JWTs work?

JWTs combine concepts from cryptography and web technologies to create a standard that allows user authentication and session management among other things. In general, it allows one to say with confidence about the authenticity of a piece of data. This has been adapted to provide security on the modern web.

A JWT is human readable, encoded JSON objects concatenated by periods (.). Here’s an example (newline added for readability):

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.
TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ

There are three pieces of strings:

Header
Payload
Signature

The header and payload are Base64-URL encoded JSON objects. Their contents can be decoded and read. Note: A JWT is not encrypted. However, there are standards for encryption (JWE) RFC 7516 . A JWT with only the header and payload is called an unsecured JWT and is generally not used except to store data on the client side in special use cases.

The signature is used by applications to verify the authenticity of the payload. There are many recommended algorithms to achieve the same but the most popular are HS256 (HMAC and SHA-256) and RS256 (RSA and SHA-256). Signed JWTs are defined in RFC 7515 . We will look at signing and verifying later.

User authentication: Walkthrough

User authentication workflow using JWTs.

In this section, we will walkthrough a process of user authentication using JWT. The outlined process is agnostic of the algorithm used upon which the process might slightly differ.

The user client (browser or app) makes a HTTP request to the authorization server with the user (suppose, Alice) credentials (maybe a login form). This should be secure preferably via TLS (Transport Layer Security).
The authorization server may perform necessary checks to be sure that the user is in fact who it poses to be. This maybe achieved by querying a database or via some other method like Single Sign-On (SSO).
The authorization server generates a signed JWT with the header specifying alg and typ and payload containing some unique identification for this user. The misuse of this identifier shouldn’t pose a secuirty risk, however, it should be adequate for our application to identify the user.
Authorization server issues the token to the client.
User requests resources from the application with authorization header set to Bearer ${token}.
Application verify the token using a secret or a public key based on which algorithm is used (See below). Choosing the corresponding PUB key can be based on the iss field in the payload and maybe an app managed database of public keys.
Application can reject the request or respond with the requested resources based on the result of the validation.

It should be noted that all data flows through the user client in the form of parameters to the request. On a browser, this should be the key-value pairs after ? in the URL bar. Moreover, the transaction is completely stateless. To initiate another request, the client can use the token within its exp period and avail the services. The application will undergo the validation process all over again.

To mitigate many secuirty risks, the exp period is kept as small as possible, however, it is not convenient to login for each single request. This gave rise to the concept of session management and access tokens and refresh tokens.

Refresh tokens have a longer exp time than access tokens and can be used in lieu of user credentials to ask for new access tokens when the previous ones expire.

Structure of a JWT

The header

The header is a plain JSON object having atleast one field alg which specifies which algorithm is to be used to verify it. Another is a typ field which is the media type of the JWT itself. For most practical purposes, its value is JWT. There may be other fields (or “claims”) but these are necessary. A sample header:

{
 "alg": "RS256",
 "typ": "JWT"
}

But wait. This doesn’t look like what we saw earlier. What’s going on? The header and payload are actually encoded by a scheme to the most common subset of UTF-8 characters to ensure portability and avoid garbled text, a sad product of multiple encoding schemes. The chosen method is Base64URL, a URL safe variation of Base64 which swaps / and + characters by _ and - respectively.

The alg, if set to none, can become a security threat for some verification libraries. The reason being, in the past many verification libraries relied on the alg field to pick up the right algorithm. This was the result of ambuiguity in the implementation of verification libraries. To migitate this issue, never use alg: none and never allow unsecured JWTs in your application. Onwards to the payload.

The payload

The payload is another JSON that is the data store of the token. All claims are stored in the payload. There are some registered claims whose field names have been standardized.

iss: The issuer. A URI string identifying the authority issuing the JWT. The data can be application specific.
sub: The subject. A URI string identifying the entity that this JWT carries information about. The data can be application specific.
aud: The audience. A list of URIs intended as recipients for this JWT.
exp: The expiration date. Expressed in number of seconds passed since Epoch.
nbf: not before (time). The opposite of the exp claim. Expressed in number of seconds since Epoch.
iat: The issued at (time). Expressed in number of seconds since Epoch.
jti: The JWT ID. A unique identifier of this JWT. Again, it is application specific.

Apart from the registered claims, there are so-called private claims which are defined by users and add application specific communication to the JWT. Namespaces may be used to avoid key collision. Public claims are either registered with the IANA JSON Web Token registry (central registry to prevent name collisions) or named using collision resistant methods.

Now, let us take a brief interlude to understand hash functions and then examine how signature works.

Hash functions

Hashing is a computational operation that produces a fixed length string from a data of arbitrary size. Moreover, hashing has certain interesting properties which makes them ideal for cryptographic applications.

Irreversibility: Hash functions are one-way functions. The original data cannot be deduced from the output. The only way (known so far) is to brute force each possible data to match the output.
Reproducibility: Hash functions are one-one functions. A piece of data is always guaranteed to produce the same output every single time.
Uniqueness: By choosing proper parameters, it is possible to reduce collisions to practically zero. Collisions happen when two different data match to the same output.
Unpredictability: Even if a single byte is changed in the original data, atleast 50% of the bits changes. This makes it very difficult to guess the data by cunning or intelligent guesses. See Avalanche effect .

Let us examine a popular hash function SHA-256. The Secure Hash Algorithm (SHA) family is defined in FIPS-180 . We are going to use the native node module crypto to generate a SHA256 hash.

// code source: https://nodejs.org/api/crypto.html
const {
 createHash
} = await import('crypto');

const hash = createHash('sha256');

hash.on('readable', () => {
 // Only one element is going to be produced by the
 // hash stream.
 const data = hash.read();
 if (data) {
 console.log(data.toString('hex'));
 // Prints:
 // 6a2da20943931e9834fc12cfe5bb47bbd9ae43489a30726962b576f4e3993e50
 }
});

hash.write('some data to hash');
hash.end();

HMAC + SHA256 = HS256

We have all the components to actually sign and verify a JWT. As it has been noted, the issuing authority can be seperate and oblivious of the identity of the verifier. This is an advantage of JWT. The question is: how do we achieve that? HMAC or Hash-Based Message Authentication Code works this way:

Generation of token:

Take the Header (H), Payload (P), and a secret data (SECRET). Concatenate them together and pass through a hash function.
Let the output be HASH. Append this as the signature or Message Authentication Code (MAC) for our JWT.

Validation of token:

Take H, P, and the same SECRET and produce a hash.
If the contents of the payload had not been tampered with, then MAC and this hash will match.

This scheme is very simple. However, there is an obvious disadvantage: The token issuer and the validation application have to share a common secret which is essentially the crux of security. Requirement of distribution across a large or insecure network makes regular key rotation unfeasible. Moreover, since the SECRET is stored on many nodes in the network, there is more possibility of a SECRET leak or susceptibility to an attack. Public-key cryptography addresses this issue.

RSA + SHA256 = RS256

HS256 suffers from the downside of being unable to distinguish between token generation and token validation. Both are essentially the same process. RSA public key cryptography is a scheme that involves 2 keys: a private key and a public key. The private key can sign a piece of data while a public key can definitively validate if the associated private key had signed the data, hence establishing data integrity. Moreover, compromise of the public key does not diminish the security of the scheme. The private key can be safely stored only with the token generation authority.

Tools like OpenSSL can generate these keys for you or even online services such as this one . An example 1024-bit RSA key pair:

Token generation:

A RSA key pair is generated (PVT and PUB). The PUB is made public.
Header (H), Payload (P) is passed through a hash function (SHA256, in our case). Output is HASH.
The HASH is signed with the PVT key to produce SIGNED.
SIGNED is attached as the JWT signature and issued to a client.

Validation:

H and P are used to generate a hash. Call it HASH2.
Using PUB, it decrypts the SIGNED to produce HASH.
The integrity of the data is established if HASH matches HASH2.

We can deduce authentication because it is guaranteed that just decrypted SIGNED by the PUB key must have been signed by the associated PVT key which is (ideally) only possessed by the issuing server. The issuing server will only issue the token if it is satisfied with the user credentials. Thus, the user possessing the token and posing as the user to our application must be no one else but that user.

One might question: Why not encrypt the header and payload directly and attach that as a signature. That is a valid question. In fact, such an approach would work in theory. It is not employed in practice because the RSA encryption process can be slow for larger payload sizes. Hashing is a relatively faster computational operation and produces fixed length smaller payloads for RSA to consume.

Other algorithms

There are a plethora of other algorithms that may be used. One of them that is specified in the spec and not discussed here is Elliptic Curve Digital Signature Algorithm (ECDSA) leading to ES256. Further, there are variations based on the key or block size in the SHA family or the public-key crypto scheme. For a thorough discussion, see the JWT Handbook .

Common pitfalls and mitigation techniques

If you want to develop secure JWT-enabled applications, you must be aware of the common pitfalls that may allow attackers to exploit the weakness of your system. Moreover, we shall discuss some mitigation techniques to safeguard your application. It is highly recommended to use tested, production-ready libraries and allow oneself to thoroughly understand the nuances of the provided API. It is generally a bad idea to roll out an in-house crypto module.

Unsecured tokens via `alg: none`

You must note that JWTs are unencrypted in general. JWE encrypts only the payload, the header is always in unenrypted form. Thus, due to ambiguity in specification, an attackers crafted JWT having alg: none can be interpreted as valid because of reliance of choosing the algorithm based on that field.

For more, see this paragraph .

Signature stripping

A JWT, as you know, is made of three components – header, payload, and signature. A common method of attack is to strip the signature, manipulate the contents of the payload and mark the token as “unsecured”.

To mitigate signature stripping attacks, make sure your application rejects all signature-less tokens as invalid.

Cross Site Resource Forgery (CSRF)

This attack is effective when JWTs are stored as session cookies on the browser. The attacker embeds a crafted link to the target site on a malicious page. When the browser requests the resource, it sends the cookie too, if the user had previously logged into target. If the target doesn’t implement CSRF mitigation on the server side, the potentially modified token can be deemed valid.

Short-lived tokens, not storing JWTs as cookies, and implementing CSRF mitigation on the application are used to tackle CSRF.

Cross Site Scripting (XSS)

This is similar to SQL injection attacks where the attack takes advantage of improper user data validation on the back end, to inject JavaScript into the trusted sites. It then attempts to harvest JWTs from cookies or local storage. Any data received from a user must be sanitized.

References

Blog post: JWT: The Complete Guide to JSON Web Tokens by Angular University (Good beginner introduction).
Book: The JWT Handbook by Auth0 (In-depth treatment of algorithms, See Chapter 7).

License

All diagrams and code used in this article is produced by the author unless otherwise noted and is licensed under CC BY-NC-SA 3.0 . This means you are free to share, adapt, or modify it but you should provide credit and not use the contents for commercial purposes.

Hacktoberfest and open source

gs454236@gmail.com (Basil Labib) — Thu, 21 Oct 2021 06:22:07 +0530

DigitalOcean, a cloud service provider annually hosts Hacktoberfest , a month-long celebration of open source culture where developers from around the globe contribute to open projects. It is in its 8th iteration this year and has become quite an attraction for programmers, young and experienced.

It is unfortunate that most people don’t really understand what open source is. This post will talk about open source, the people, beliefs, and psychology behind it and how it has changed our lives.

THE PHILOSOPHY OF OPEN SOURCE

Open source is a philosophy, a collection of ideas that have evolved into a culture. That philosophy is grounded in freedom and liberty. The founder of the GNU project and eventually one of the pioneers of open source, Richard Stallman is famously quoted,

“Think free speech, not free beer.”
~Richard Stallman

This succint definition captures the essence of what open or “libre” (from French meaning “freedom”) means. Open source is not a free commodity for anyone to use (though it is that in some plane of thought) but it is much more than that. It is an expression of defiance to corporations and closed systems. Crafty people have, in all eras, attempted to dominate the masses. With software, such ambitions have become more absolute and sustainable because software affects so much of our lives and will continue to do so with increasing fervour. Open source is denial and decentralization of that power back to the people. However, modern corporations have, instead of opposing, embraced open source which is an interesting compromise.

PEOPLE WHO WRITE OPEN SOURCE

As the computing era was underway in the 60s, people obviously realised the potential computing carried. It was one of those mega money-making invention that rarely blessed mankind. However, there was one group who believed freedom should extend to software too. They were romantic cyberpunks who believed that no single group should be allowed to dictate what and how software was created. This was a group of misfits, and troublemakers. They were non-conformists and brilliant. They broke things often and fixed them, often infusing their own ideas and ideals. They were also, what is now popularized as, a “nerd” or a “geek”. But unlike the popular “geek” image painted by media, they were agile and vigilant and receptive to the needs of the society. They were not careless about their expertise and knew what power they yielded. They were responsible, persistent and infinitely curious. These visionaries were called hackers. To be called a hacker is a badge of honour and not of disgust. It is unfortunate that media has rallied this pure word to mark a certain section of malicious actors who use their skills mostly for destruction and chaos. That is a distortion of the real meaning of a hacker.

THE PSYCHOLOGY OF A HACKER

If someone is really interested, they may ask the question: Why do hackers behave the way they do? Why do they see it more befitting to give away their code for free, instead of making money and getting rich of it? Naturally, that is what people in other fields would expect and choose to do.

So, here’s the thing: The peculiarity of Computer Science is that it is a highly specialized field. Most people just “leave it to the experts” and unlike fields like personal finance where people have to “get their hands dirty” and learn a thing or two, they can simply forget about what goes inside the black box of their systems and assume everything works fine. Nobody bothers to understand what programmers and software writers do, let alone appreciate the art of writing code.

This leaves hackers in an awkward position. It’s a psychological response. This group of people seek appreciation from people like them and because they are few in number, they share a special and unique sense of camaraderie which they try to consolidate. Being blissfully unaware of social norms and conventions, they communicate what they are truly passionate about: their code and their visions. The freedom to exchange ideas and distribute code freely is an offshoot of this process. It is the cry of a brilliant dreamer to seek out appreciation from the small community who, they are sure, will understand his visions and his works.

Not for a moment believe that what I have outlined is anything short of a superhuman power. How many professionals would choose to put the good of community above their own selfish gain?

For more on hackers and their beliefs, see here .

WHY IS OPEN SOURCE THE FUTURE?

Open source is the culture that nurtures thinkers, creators, and hackers. As Paul Graham, worked on LISP, founder of Y Combinator (a tech startup incubator), wrote in this interesting essay “Hackers And Painters” ,

You can’t do anything really well unless you love it, and if you love to hack you’ll inevitably be working on projects of your own.
~Hackers and Painters (Essay)

One might scorn in disgust as they use an open software and complain that it is suboptimal or that it doesn’t look good. It is because either 1) That software is fairly new so it doesn’t have much community yet 2) You have completely missed the point. Freedom of expression is most important to hackers. It is the space of infinite creativity where they can realize their wildest dreams. If you don’t like the way a software is built, they give you the opportunity to change it yourself. This choice of having an actionable opinion in what one uses is non-existent in commercial software. This is also one of the reasons why software written by hackers are often choked full of nifty, geeky shortcuts to literally everything, many a times, to the disgust of their corporate employers. It is because they don’t write code to please their employees. Hackers intentionally design their software to be modular like a jigsaw puzzle. It is perhaps because they understand most intimately, of all people, the high of writing great software and the need to communicate ideas.

“Sharing knowledge is the most fundamental act of friendship. Because it is a way you can give something without loosing something.
~R. Stallman

WHY GO OPEN?

Creators and builders understand the need to share their ideas without any hindrance. It leverages different perspectives, minds, and hands which eventually enables software of far superior quality as opposed to a commerical alternative. Take for example, the GNU project, Emacs, Vim, git, Linux, BSD, Blender, Python and so many others. Decentralized fintech such as Bitcoin and Ethereum which are challenging the financial tapestry of our society. Even recent corporate projects have been opened such as Facebook’s React framework.

People believe that if one has to pay for something, then the thing they are paying for must be a valuable product worth the value set by the seller. This is natural herd psychology yet, it’s a fallacy. The price doesn’t determine the value. They are not causative events, in economic jargon.

When code is scrutinized by thousands of peering eyes instead of a couple hundered inside a closed organization, it naturally leads to more bug-free, robust, and versatile systems. Imagine the marvel that is the development story of Linux. Torvalds admits, in his autobiography, to have manually applied the patches (sent as tarballs from strangers over the internet) into his codebase! And somehow it worked.

“Software is like sex: It’s better when it’s free”.
Linus Torvalds.

PERSONAL EXPERIENCE

I wouldn’t have started my fascinating journey into software, had it not been for open source. I started out by literally pulling out and reading source code of repositories on Github! I can see fairly clearly the self-reinforcing structure of open source. It invites not one group, but everyone to contribute back and on the way, enabling them and equipping them with the tools and expertise to do so. And why shouldn’t people have a right to look into the source code of the software they use and depend on every day?

LITTLE HISTORY

(I saved this for the last because I think history doesn’t appeal to many people. But it does to me!)

A basic outline of the history is called for, at this point. It started at AT&T Bell Laboratories in the 60s. This place was the geek haven of the 60s. Most of the foundation of computer science as we know it today was invented here. The transistor was invented at Bell Labs. Dennis Ritchie and Ken Thompson, two prodigious programmers, writers of several CS “bibles”, and visionaries invented UNIX after MULTICS, another operating system failed to materialize out of a contract. Note that this was the pre-personal computing era so most of these research is geared towards militiary or enterprise applications. UNIX was (is?) a revolutionary piece of software. It introduced novel concepts such pipes, user-level programmable shell, hierarchial file system, and the famous UNIX philosophy: “small is beautiful”. But, UNIX was essentially a commercial project. The strategy of AT&T was simple and the gist of it exists in this 1907 tagline,

“One Policy. One System. Universal Service.”

They collected and hired the best, gave them creative and financial autonomy, and reaped the benefits of commercialization by leveraging its monopoly in the telecomm sector.

Richard Stallman independently started implementing a complete system from scratch and called it the GNU project which is a recursive acronym for “GNU’s Not UNIX”. The terms of the GNU Project, launched in 1983, was simple: This source code is free. I won’t charge you any money. But if you modify and make something interesting, you have to publish it with this message. The GNU Public License or GPL was ironclad with no judicial loopholes for exploitation (because Stallman employed lawyers to draw the terms). Though, there were pockets of “hackers” around the globe who were tinkering on their machines, the GNU project was the one that came into the spotlight. The idea wasn’t new, but it captured the public imagination like never before. It spawned the *BSD family of operating systems, all open. And of course, in 1991 a 21 year-old Finnish student started writing a kernel using GNU software and make his hobby project into one of the most successful open source stories in recent times. Linus Torvalds became a sort of poster boy for libre software. Though, Torvalds has recently been known to controversial, I gauge people with what they did. Linux and git are great software. I use them everyday.

The details of Linux is a messy one due to lack of standardization, prolific burst of new microprocessor architectures, birth of WWW, and potential financial gains. For more, I suggest UNIX: a history and a memoir .

BIBLIOGRAPHY

Torvalds, Linux; Diamond, David: Just for Fun - The Story of an Accidental Revolutionary, Harper
Kerninghan W. Brian: UNIX - a history and a memoir, KDP
“How to Become a Hacker” : Essay by Eric S Raymond
“Hackers and painters” : Essay by Paul Graham

Building a Discord bot: A definitive guide

gs454236@gmail.com (Basil Labib) — Sat, 16 Oct 2021 07:56:12 +0530

In this guide, I will be building a Discord bot from scratch using only discord.js and a few open public APIs. I’ll also explain each step and discuss the reason for preference of a particular way of implementation over others. Unlike other “recipes” on the web, this will give you insight into the workflow of building software.

This may seem like a long post because it is. I suggest you to follow along in chunks so you don’t get overwhelmed. The tutorial is divided into chapters. So, let’s get started.

WHAT ARE WE BUILDING?

Mayhem is a simple Discord bot that is aimed for entertainment and fun. It queries external public APIs for images, jokes, facts and definitions. Few commands are meow, cat, whatis, whoami, ls, help, whereami and so on. There is a subtle shoutout to UNIX if you’re familiar ;).

Moreover, these commands are chosen and designed in a way to demonstrate capabilities of the Discord API. Thus, features such as exclusive commands, cooldown periods, pre-fetching etc. are also discussed.

CODE REPOSITORY

The latest version of the codebase can be found here . Instructions for installation on your Discord server can be found in the README.md .

For options to host the bot, see Deployment Options in the README.

JS PRE-REQUISITE CHECK

You should be familiar with JavaScript and computer programming concepts such as loops and conditionals. But JS is a particularly notorious language as it is continuously being modified to suit the industry. You should know at least ES6 or ES7. If not, no worries! Below I cover few JS concepts we’ll be using in this project.

ARROW FUNCTIONS

[1,2,3].map(elem => 
	{ console.log('doubling!');
	 return 2*elem; 
});

A syntactic-sugar to reduct clutter. Nothing more but highly convenient for writing chained promises (If you ever found yourself in such a situation that is).

PROMISES

const data = fetch(URI)
	.then(res => res.text())
	.then(tex => JSON.parse(tex))
	.catch(err => console.log(err));

A fix to execute asynchoronous statements synchoronously. Think of these commands as being executed one after the other and not all together because they can’t as the data are dependent on successful execution of previous steps. I never liked it much.

`await`/`async` FUNCTIONS

async function nano(args){
	const _r = await fs.readFile('stuff.json');
	// more code
}

Successor to Promises. Cleaner, more logical syntax. As opposed to being an API like Promises, async/await are keywords in JS now.

If you want to gain more insights into these constructs, it is recommended to check out MDN JavaScript module .

INSTALLATION

You need Node.js and a package manager such as npm or yarn . npm comes packaged with Node.

We will be using these npm packages:

discord.js@12.4.1 or above (Note: The code is tested with this version only)
node-fetch@2.6.1

BASIC SETUP

To make a new node project, run the following in your terminal:

$ mkdir mayhem
$ cd mayhem
$ npm init -y # -y to say yes to all defaults
$ npm install discordjs@12.4.1 node-fetch@2.6.1
$ touch bot.js

Then, open package.json and add a new script for convenience:

...
 "main": "index.js",
 "scripts": {
 "test": "echo \"Error: no test specified\" && exit 1",
 "start": "node bot.js"
 },
...

CREATE BOT APPLICATION AND GENERATE TOKEN

Now, we have to create a bot application through your Discord account (Make one, we need it!). The steps are as follows:

Login and open the Discord Developer Portal .
Click on the “New Application” button.
Enter a name and click “Create” button to confirm.

Created New Application in Discord Developer Portal

You are free to edit other information as you wish on this page. We’ll leave it to defaults. Then navigate to the “Bots” tab and click “Add Bot” button.

Created new bot and customized with a name and image icon

Clicking on “Reveal Token” will show your bot token. A Token is a unique identifier for your bot application and is used by Discord to authorize your bot to actually log in and appear online in a server.

CAUTION: It is very important that you keep your token safe and secret. You must not mistakenly commit it to a public repo or such. The consequences can be disastrous. If the key got leaked by mistake, immediately click on “Regenerate token” to generate a new token and invalidate all previous tokens.

WARNING: Make sure you copy the “Bot Token” and not the “Client Secret” or the “Client Key” which are on the Applications page.

GENERATE BOT INVITE LINK

Bot invite links explained

A typical invite link looks like this:

https://discord.com/api/oauth2/authorize?client_id=123456789012345678&permissions=0&scope=bot%20applications.commands

The fields that need elaboration are the query parameters which are as follows:

client_id: The client id is the application that is authorized.
permissions: an integers which encodes the bot’s permissions on the server in binary.
scope=bot: Yes, this is a bot invite link.
applications.commands: To enable Discord to allow / (slash) to invoke bot commands.

Head over to the OAuth2 page, scroll down and select applications.commands and bot. Then, choose permissions that you want your bot to have. The page handily generates the link for you.

Permissions for Mayhem

Here, I show the permissions for our bot. As you can see, it is a fairly less-demanding bot in terms of administrative privileges.

Authorizing your bot into a server

Paste the generated link and you will be asked to authorize the server. Note: You can add them to servers in which you have admin access only. If successful, you’re greeted by a confirmation dialog. Congratulations, now you can see Mayhem online with a “BOT” tag!

CHAPTER 2

In Chapter 1 of this tutorial, we had finished logging our bot into our server. In this post, we plan to give it some interactivity with human users. Moreover, we will organize our code in a modular pattern to facilitate modification and addition of more commands easily.

FIRST COMMAND & `bot.js`

// New file: /bot.js
const Discord = require('discord.js');
const {prefix, token} = require('./config.json');

const client = new Discord.Client();

client.once('ready', () => {
	console.log('Ready!');
});

client.on('message', (message) => {
	console.log(`[${message.channel.name}] ${message.author.username}: ${message.content}`);

	// Don't interfere when you are not required and don't be dumb enough to reply to your own msgs.
	if(!message.content.startsWith(prefix) || message.author.bot) return;

	const args = message.content.slice(prefix.length).trim().split(/ +/);
	const cmdname = args.shift().toLowerCase();

	try{
		if (cmdname == 'hi') {
			message.channel.send("Hi ${message.author}! I'm Mayhem. Nice to meet you!");
		}
	} catch(err){
		console.error(err);
		message.reply('Oops. There was an error while executing that command.');
	}
});

client.login(token);

The code is fairly straightforward to understand. We import the required modules and secrets. The prefix is a special character you use to invoke commands on this bot for eg: /, ., -- etc. After instantiating an instance of Discord.Client(), we log simple text when bot is ready i.e. the ready event is triggered.

Then, for every message sent to a channel, the 'message' event is triggered which is handled by the function we supply to client.on(...). The code parses the message and sends a hi message to the user for messages of the form <PREFIX>hi. Finally, we login with the bot token we generated before.

// /config.json 
{
	"prefix": "YOUR-PREFERRED-BOT-PREFIX",
	"token": 0123456789
}

Run npm start or node bot.js and your bot is online!

For now, the bot doesn’t do much and only sends the ‘hi’ message rather annoyingly.

The code structure as such is not very robust or versatile. We’ll organize it to be modular to facilitate that. The idea is we keep code logic for each command in a seperate file in the /commands directory and load them into the client upon instantiation of the client object. Then, adding a new command will be as clean as adding a new file to this directory.

MODULARIZATION

$ cd /path/to/mayhem
$ mkdir commands
$ cd commands

// New file: /commands/hi.js
const {bot_name} = require('../config.json');
const Discord = require('discord.js');

module.exports = {
	name: 'hi',
	description: 'A nice introduction is the gentleman\'s way.',
	usage: '',
	execute(message, args){
		const embed = new Discord.MessageEmbed()
			.setColor('#00ebc7')
			.setTitle(`Hello! ${message.author.username}`)
			.addFields(
				{name: `${bot_name} bot says...`, value: `Yo! Nice to meet you!`},
			)
			.setTimestamp();

		message.channel.send(embed);
	}
}

As you may notice, we export few other handy metadata apart from the function execute(...) which contains the main logic. We will be adding other fields for unique commands as we need them. This approach was highlighted in discordjs.guide .

Now, make the following changes to bot.js.

// /bot.js
const fs = require('fs');
const Discord = require('discord.js');
const {prefix, token} = require('./config.json');

const client = new Discord.Client();

client.commands = new Discord.Collection();

// will be used later
const cooldowns = new Discord.Collection();

const files = fs.readdirSync('./commands').filter(name => name.endsWith('.js'));

for(const f of files){
	const command = require(`./commands/${f}`);
	// According to Discord API, Discord.Collection is a JS Map Object with more convenience methods
	// Set the record with key as the command name and value as the command itself
	client.commands.set(command.name, command);
}

client.once('ready', () => {
	console.log('Ready!');
});

client.on('message', (message) => {
	console.log(`[${message.channel.name}] ${message.author.username}: ${message.content}`);

	// Don't interfere when you are not required and don't be dumb enough to reply to your own msgs.
	if(!message.content.startsWith(prefix) || message.author.bot) return;

	// basic parsing
	const args = message.content.slice(prefix.length).trim().split(/ +/);
	const cmdname = args.shift().toLowerCase();

	// get the cmd object
	const cmd = client.commands.get(cmdname);
	if(!cmd) {
		return message.channel.send('Oops! No such command ;-;');
	}

	try{
		cmd.execute(message, args);
	}catch(err){
		console.error(err);
		message.reply('Oops. There was an error while executing that command.');
	}
});

client.login(token);

Most of the code is intact. The Discord.Collection() is a wrapper around the JS Map Object with few added convenient methods. We read the /commands directory using fs and set the object using client.commands.set(..., ...). Inside the handler, we parse the incoming message, try to get a command object and execute it. At any stage, if we have an exception, gracefully inform the user.

Run the bot again via npm start. Though, we cannot detect any visible changes, we have refactored the code into much more tangible form. Let me talk about one more thing that I have ignored till now.

First command of Mayhem

EMBED MESSAGES

Currently, the messages sent by the bot to the channel are simple text messages which are unattractive and less appealing to our users. If you have experience with Discord bots, you maybe aware of colourful, sprawling messages with many fields, formatted texts and images. The Discord.MessageEmbed object in discord.js allows us to construct such messages. If you see the logic of the hi command, we are indeed constructing one.

// /commands/hi.js
...
	execute(message, args){
		const embed = new Discord.MessageEmbed()
			.setColor('#00ebc7')
			.setTitle(`Hello! ${message.author.username}`)
			.addFields(
				{name: `${bot_name} bot says...`, value: `Yo! Nice to meet you!`},
			)
			.setTimestamp();

		message.channel.send(embed);
	}
...

These are all obvious imperative methods that adds a particular field one at a time. For eg: You should deliberate on a color theme because it conveys your bot’s purpose and “vibe” to your users. The setColor() can be used to set a color. setTitle() adds a bold title to the message. The addFields() take up a JSON object and add all key-value pairs to the message body. The setTimestamp(), as expected, adds a timestamp field to the message.

SUMMARY

In summary, we have executed our first command, modularized our codebase and paved the way to breezy development and we know how discord.js handles channel events (via handlers) and about Discord.Collection(), Discord.Client() and Discord.MessageEmbed().

CHAPTER 3

In the last chapter of this guide, we looked at creating our first command, Discord.MessageEmbed(), and modularization. In this part, we’ll demonstrate API fetching and command aliases, while adding two new commands to our bot!

`meow` & `node-fetch`

The meow command is a simple command that queries a public API of cat pictures and sends one random, cute cat picture down in the channel. We will use the following endpoint:

https://aws.random/cat/meow

NOTE: If you wish to use an API that requires some kind of authorization, then you have to generate a secret key from a dashboard usually and feed this credential to your .env file (this step is more or less platform-dependent. For eg: replit provides a key-value based secret storage feature). Then, you can read that value via process.env.API_SECRET_KEY or likewise. Consult your API documentation and hosting provider documentation.

But to fetch on the server-side, we have to install a fetch-equivalent module on our server. This is achieved by the node-fetch package.

yarn add node-fetch # or npm install node-fetch

Now, let us implement the meow command.

// New file: /commands/meow.js 
const fetch = require('node-fetch');
const {prefix} = require('../config.json');

module.exports = {
	name: 'meow',
	description: 'Coz everyone likes cats!',
	usage: `${prefix}meow`,
	cooldown: 10,
	execute(message, args){
		const file = fetch('https://aws.random.cat/meow')
			.then(file => file.text())
			.catch(err => { 
				console.error(err);
				message.channel.send(`An Error was encountered while executing that command.`);
			});
		message.channel.send(JSON.parse(file).file);

	}
}

The meow command

Once again, the code is obvious and there is no element that requires an explanation as such. The “cooldown” parameter will be dealt with in another command. But, you can probably guess it is to prevent a indirect DDoS attack on the cat server by bad actors and/or bots.

`help`

Good UI/UX don’t require a manual to be used. So, like every good software, we will provide a handy help command for potentially new users of our bot. This command will demonstrate sending personal direct messages and command aliases.

The design of the messages is decoupled from the logic. Feel free to make your bot as snazzy as possible.

// New file: /commands/help.js
const {prefix, version,bot_name, bot_author} = require('../config.json');
const Discord = require('discord.js');

module.exports = {
	name: 'help',
	description: 'Display all available commands',
	aliases: ['h','?'],
	execute(message, args){
		const data = [];
		const {commands} = message.client;

		if(!args.length){
			// show all commands
			data.push(`This is the ${bot_name} Bot version ${version}`);
			data.push(commands.map(command => command.name).join(', '));
			data.push(`Use ${prefix}help [command name] to get details about [command name]`);
			data.push(`All commands are prefixed by the ${prefix}`);
			data.push(`\n\nSend your love to ${bot_author} ;)`);

			// split: true splits the message into separate messages if char count > 2000
			return message.author.send(data, {split: true})
				.then(() => {	// DM as help message can get messy
					if(message.channel.type === 'dm') return;
					message.reply('I\'ve sent you a DM');
				})
				.catch(error => { // the case when user has DM disabled must be gracefully handled
					console.error(`Could not send help DM to ${message.author.tag}.\n`,error);
					message.reply('I can\'t DM you. Do you have DMs disabled?');
				});
		}else{
 ...

As you can see, message.author.send(...) is used to send a message to the original message author’s DM. If a bare help is invoked, we inform the user of our commands personally and ask them to explore more by using the format help [command name].

Once again, the aliases field will come handy in the next part of this function.

...
	}else{
			const name = args[0].toLowerCase();

 // try to make a hit, either the command or its aliases
			const command = commands.get(name) || commands.find(c => c.aliases && c.aliases.includes(name));

			if(!command){
				return message.reply('Not a valid command -_-');
			}

			const embed = new Discord.MessageEmbed()
			.setTitle(`Help for ${command.name}`)
			.setColor('#02fede')
			.addFields(
				{name: 'Aliases', value: (command.aliases) ? `${command.aliases.join(', ')}`: 'There are no aliases for this command!'},
				{name: 'Description', value: (command.description) ? `${command.description}`: 'Currently no description available!'},
				{name: 'Usage', value: (command.usage) ? `${command.usage}`:'Currently no usage available!'},
				{name: 'Cooldown', value: (command.cooldown) ? `${command.cooldown} second(s)`: 'This command has no cooldown!'}
			)
			.setTimestamp();

			message.channel.send(embed);
		}
	}
}

help meow command execution

You can probably understand the workflow that discord.js promulgates. Here it is:

PARSE :right_arrow: EXECUTE :right_arrow: SEND

SUMMARY

In this post, we finally made an API call from our bot and served some cute cat pictures to our users. Now, the sky’s the limit for you! Moreover, we added a help command to be more organized.

CHAPTER 4

In the last part of this guide, we implemented the meow and help commands. Now, we will look at leveraging the API exposed by discord.js to implement cat, whatis, and rm commands.

`cat` & PRE-FETCHING

The cat command makes a query to another free API and displays interesting cat facts to our users. However, the API has only one endpoint and due to performance reasons, we cannot query the entire database each time a user makes request. So, the idea is to fetch the resource at startup time of the boot and save the data locally on the disk. This, though makes our bot more efficient and performant, has obvious disadvantages: What if the data on the API server mutates? But, we shall ignore such concerns as it is basically a database API.

Fire up your text editor and create this new file: fetcher.js.

# New file: /fetcher.js 
const fs = require('fs');
const fetch = require('node-fetch');
const {CAT_API_KEY} = require('./config.json');

async function writeData(){
	const data = await fetch("https://brianiswu-cat-facts-v1.p.rapidapi.com/facts", 
		{
			"method": "GET",
			"headers": {
				"x-rapidapi-key": CAT_API_KEY, 
				"x-rapidapi-host": "brianiswu-cat-facts-v1.p.rapidapi.com"
				 }
		})
		.then(response => response.text())
		.then(test_response => JSON.parse(test_response))
		.catch(error => {
			console.log(error);
		});

	let raw_data = JSON.stringify(data, null, 2);
	console.log(raw_data);
	fs.writeFileSync('./static/data/cat_data.json', raw_data);

	console.log('written.');
}
writeData();

To get your API Key, go here and follow the steps and put the key in your config.json likewise.

// FILE: /config.json
{
 "CAT_API_KEY": "kjsalkjwRlkjVNLKKJASHLl2l49SF2qlkjS",
}

The steps are obvious. We request the cat server with a /GET request and fetch all the data. If succeeded, parse and write that to a JSON file on the disk using fs.writeFileSync(). For convenience, we can update the yarn scripts to run this file before running bot.js on yarn start.

// FILE: /package.json
...
 "start": "node fetcher.js && node bot.js",
...

Now, we will add a new command cat to /commands. The basic structure remains the same. We export a JSON object with a execute(...) function and few other metadata.

// NEW FILE: /commands/cat.js
const Discord = require('discord.js');
const fs = require('fs');
const {prefix} = require('../config.json');

module.exports = {
	name: 'cat',
	description: 'An enlightening piece of cat fact that will change your life.',
	usage: `${prefix}cat`,
	cooldown: 5,
	async execute(message, args){
		let data_file_name = '\\cat_data.json';
		let data = JSON.parse(fs.readFileSync(__dirname + data_file_name));

 // choose a random cat fact from data
		let index = Math.floor(Math.random() * data.all.length);
		const factText = data.all[index].text;
		const upVotes = parseInt(data.all[index].upvotes);

		const payload = new Discord.MessageEmbed()
		.setTitle('Who wants a cat fact?')
		.setColor('#ffa500')
		.addFields(
			{name: 'Fact', value: factText},
			{name: '\u200b', value: '\u200b'},
			{name: 'Upvotes', value: upVotes}
		)
		.setTimestamp();

		message.channel.send(payload);

	}
}

Restart your app and test your new command!

Testing the cat command

`whatis` & COOLDOWNS

Urban Dictionary provides a free API to query its database. Furthermore, there is no need to generate a key or authorization token. We will be using this API to give our bot a sarcastic nature (you’ll soon see what I mean :winking_face:). The overall pattern remains the same. We construct the request using the querystring package. Just do yarn add querystring. The rest of the execute function handles exceptions such as zero-element list, network error during fetching, etc. Here is the code:

// NEW FILE: /commands/whatis.js 

const querystring = require('querystring');
const fetch = require('node-fetch');
const Discord = require('discord.js');
const {prefix} = require('../config.json');

module.exports = {
	name: 'whatis',
	description: 'Your own dictionary utility on Discord.\n\nNo googling or changing tabs :)',
	cooldown: 10,
	usage: `${prefix}whatis <search-term>`,
	aliases: ['def','define'],
	async execute(message, args){
		if(!args.length){
			message.reply(`fatal! You need to specify a search term.`);
			return;
		}

 // See: https://www.npmjs.com/package/query-string
		const query = querystring.stringify({term: args.join(' ')});
		
 // helper function to trim message to sizable chunks
		const trim = (str, max) => ((str.length > max) ? `${str.slice(0, max-3)}...`: str);

		const {list} = await fetch(`https://api.urbandictionary.com/v0/define?${query}`)
			.then(response => response.text())
			.then(textres => JSON.parse(textres))
			.catch(err => {
				console.error(err);
				message.channel.send(`An Error was encountered while executing that command!`);
			});
		
		if(!list.length){
			message.channel.send(`No results found for "${args.join(' ')}"`);
			return;
		}

		// just the first element
 // See: Array Destructuring
		const [answer] = list;

		const embed = new Discord.MessageEmbed()
			.setColor('#EFFF00')
			.setTitle(answer.word)
			.setURL(answer.permalink)
			.addFields(
				{name: 'Description', value: (answer.definition) ? trim(answer.definition, 1024): 'No description available!' },
				{name: 'Example', value: (answer.example) ? trim(answer.example, 1024) : 'No example available' },
				{name: 'Rating', value: (answer.thumbs_up && answer.thumbs_down) ?`${answer.thumbs_up} thumps up. ${answer.thumbs_down} thumbs down`: 'No rating available' }
			);

		message.channel.send(embed);
	}
}

Testing the whatis command. Aliases include: def and define

Cooldowns

You may have noticed the cooldown field. (In fact, it was introduced much earlier but I asked you to ignore it back then). Here, we will use it to add cooldown to our commands. As it is obvious, Indirect DDoS on external servers by repeatedly executing this command is obvious so we must prevent that for susceptible commands such as this.

Update your bot.js with the following code:

// FILE: /bot.js

...
 // use command.usage to improve UI/UX
	if(cmd.args && !args.length){
		let reply = 'You must provide arguments for this command';

		if(cmd.usage){
			reply += `\nThe correct usage of this command is:\n${prefix}${cmd.name} ${cmd.usage}`;
		}
		message.channel.send(reply);
		return;
	}

 // cooldown
	if(cmd.cooldown){
 // keep track of recent commands in cooldown period
		if(!cooldowns.has(cmd.name)){
			cooldowns.set(cmd.name, new Discord.Collection());
		}

		const now = Date.now();
		const timestamps = cooldowns.get(cmd.name);
		const cdtime = (cmd.cooldown) * 1000;

		if(timestamps.has(message.author.id)){
			const expirationDate = timestamps.get(message.author.id) + cdtime;
			
 // calculate how much time to wait
			if(now < expirationDate){
				const timeLeft = (expirationDate - now) / 1000;
				return message.reply(`You have to wait ${timeLeft.toFixed(1)} more second(s) to use the ${cmd.name} command`);
			}
		}

		timestamps.set(message.author.id, now);
 // clear timeout
		setTimeout(() => timestamps.delete(message.author.id), cdtime);
	}
...

Testing cooldown for whatis command

`rm` & EXCLUSIVE COMMANDS

Finally, I will show you how to implement exclusive commands using basic Discord ID and some workarounds.

We will write the rm command which (obviously) deletes messages from the channel. It expects an integer argument.

Each Discord user has a UserID. We will allow the command to succeed if the message author is authorized to use the command, else it will result in a failure.

// NEW FILE: /commands/rm.js
const {prefix} = require("../config.json");

module.exports = {
	name: 'rm',
	description: 'Prunes that many messages from current channel. (Reserved)',
	args: true,
	usage: `${prefix}rm <number>`,
	cooldown: 5,
	aliases: ['remove','prune','del','clear','cls'],
	permissions: [ '<AUTHORIZED_USER_ID>' ],
	execute(message, args){
		const count = parseInt(args[0]);

		if(isNaN(count)){ 
			message.reply(`${args[0]}: That doesn't look like a number to me`);
		} 
		else{
			// +1 to also delete the delete command
			message.channel.bulkDelete(count + 1, true).catch(err => {
				console.log(err);
				message.channel.send('Sorry, there was a problem encountered while pruning this channel');
			});
		}
	}
}

Finally, add a check in bot.js to use the permissions array.

// FILE: /bot.js
// permissions
// Uses the message.author.id to determine if author has privileges to use this command
	if(cmd.permissions){
		if(!(cmd.permissions[0] === message.author.id)){
				message.channel.send(`fatal: you can't use this command.\nDo you have enough privileges?`);
				return;
		}
	}

Phew, that has been a lot of concepts and coding! I hope you learnt something interesting and more importantly, got motivated to try out ideas of your own. Below are few commands that I added to my bot. None of them require any special explanation so I have only provided the code. You can also check this in the bot’s Github repo.

`ls` | LIST ALL COMMANDS

// NEW FILE: /commands/ls.js 
const {version, bot_name, bot_author, prefix} = require('../config.json');
const Discord = require('discord.js');
		const data = [];
		const {commands} = message.client;

		if(!args.length){
			const embed = new Discord.MessageEmbed()
			.setTitle(`This is the ${bot_name} Bot version ${version}`)
			.setColor('#fa5246')
			.addFields({name: 'Commands available', value: commands.map(command => '\t\t'+command.name).join('\n')})
			.setTimestamp();

			return message.channel.send(embed);
		}
		else {
			return message.channel.send(`ls doesn't support arguments now`);
		}
	}
}

`whereami` | BASIC INFO (practically useless)

// NEW FILE: /commands/whereami.js
const Discord = require('discord.js');

module.exports = {
	name: 'whereami',
	description: 'Tells you where you currently are. Immensely useful after a booze party ;)',
	usage: '',
	execute(message, args){
		
		const embed = new Discord.MessageEmbed()
		.setTitle('Where are you?')
		.setColor('#ff5470')
		.addFields(
			{name: `Channel name`,value: `${message.channel.name}`},
			{name: `Server name`, value: `${message.guild.name}`}
		)
		.setTimestamp();

		message.channel.send(embed);
	}
}

`whoami` | BASIC INFO (practically useless)

// NEW FILE: /commands/whoami.js
const Discord = require('discord.js');

module.exports = {
	name: 'whoami',
	description: '...Well, if it was a particularly cool booze party (For context: see whereami)',
	usage: '',
	execute(message, args){
		// message.channel.send(`Your username is: ${message.author.username}\nYour ID: ${message.author.id}`);

		const embed = new Discord.MessageEmbed()
		.setTitle('Who are you?')
		.setColor('#ff5470')
		.addFields(
			{name: `Your username`,value: `${message.author.username}`},
			{name: `Your ID`, value: `${message.author.id}`}
		)
		.setTimestamp();

		message.channel.send(embed);
	}
}

FURTHER DEVELOPMENT IDEAS

One can, in principle, go on and on, adding commands and polishing their bot but we have to stop somewhere. I think this will be a suitable time for me to stop. I built this Discord bot and thought it would be fun sharing the learning / building process. It suffices to say the bot is anything from “complete” or “presentable”. Here are few ideas that you can explore:

Use dotenv package to hide API keys in .env instead of config.json.
Implement an add/delete command to modify permissions.
Implement a modifier command to control cooldowns.
Explore Sockets (persistant TCP connections as in music bots, etc).
Polish the user interface.
Use better APIs.
Use a database somewhere in a creative manner.

DEPLOYMENT OPTIONS

Initially, I hosted the bot on heroku free tier and configured a ping to keep the instance alive but heroku stopped that so I had to find a solution quickly. replit is an innovative and exicting platform for young creators and developers. It is a delight to work in a repl, they have group programming, an in-house database, and so much more. Most importantly, they allow their instances to be pinged so my bot is basically online since a year with as little downtime as ~1 day! If you are passionate about your bot and want more power, consider joining their Hacker plan.

You can program directly in the replit web UI or connect a Github repo and import your code from there.

ACKNOWLEDGEMENTS

This is my first tech guide / tutorial on the web. What things worked for you? What did you not like? I would like to hear from you !

For the development of Mayhem, I heavily relied upon discordjs.guide , a well-explained document and the official Discord API docs .

Thoughts: UNIX - History and Memoir (Book)

gs454236@gmail.com (Basil Labib) — Tue, 30 Mar 2021 00:00:00 +0000

It’s always good to know about the things you love the most. It somehow makes you love them even more. I am glad non-living things are not stupidly self-conscious and self-doubting like us and I can understand them more deeply than I can understand other humans.

Some mind(ful|less) blabberings

With that said, I enjoyed this book. In Brian’s succint prose, he recollects the history of the mother of all operating systems: UNIX. And along the way, describes the sideways development that it encouraged such as the development of C (the “lingua franca of computing”), various timeless tools like grep, awk, sed, yacc, lex, make, document formatters and so on. He talks about what lead to the development of UNIX and ruminates on the kind of environment contagious to such breakthroughs. Unfettered management, fun atmosphere, great minds, stable funding and the freedom and willingness to solve real-world problems. He also points out and clear a lot of the admittedly confusing history that is the legacy of Unix as the (licensed) source code of Unix reached Berkeley. As he correctly pointed out, AT&T tried to “lock the barn door after the crown jewels had flown the coop”. And then we had BSD, Minix, Linux, Xenix and yada yada.

He talks about the revered “Unix philosophy” and spells it out in its entirety which is what I was looking for. He talks about the genius of Ken Thompson (May the Nerd God bless him), the technical eloquence of Dennis Ritchie, the management of Doug and so on. Who wouldn’t love to work at a place where people respect and rever things they loves?

He kind of skims very gently on the technical innovation that Unix brought to the table such as pipes, hierarchial file system, user-level shell programs along with the basic good design decisions and principles of system design.

He talks about pranks on other colleagues, defiance of the management and other geeky things that sound wholesome and naive (It makes me feel that Gods do sometimes play). He points out rather coyly the Unix system in use in the Jurassic Park which controlled the security locks and latches and how Lex navigated it and saved the day. He talks about the infamous comment in the Unix source code that apparently described code responsible for context-switching between processes and it went like:

/** You are not expected to understand this. **/

I think I can safely attribute the code it described to Ken :P.

And he did all that in his usual mix of humour and nostalgia which is heart-warming and I guess will be to anyone who had helped build something that has literally changed the world.

The masters have shown the way, it is time we lead.

Logs

This was a (relatively) short book and I think I completed it in three to four sittings. I finished it on Mar 30, 2021 though I’m sure to dig into the myriad of resources Brian has linked to. I also want to get my hands on the Unix src (6th edition) preferably with Lions’ commentary.

Thoughts: Design for Hackers (Book)

gs454236@gmail.com (Basil Labib) — Sun, 14 Mar 2021 00:00:00 +0000

Design. An idea which I admit rather shamefully to not have paid any serious attention to (until now).

Design of a system happens before building not after.

This, it seems, is a general attitude towards design by builders and engineers or people who work on the guts or so called backend of a system as was coyly pointed out by the author himself.

Unfortunately, it is true. Design is something I have always thought of as an after thought in my projects at least uptil now. But no more!

The more I dwelled into David Kadavy’s excellent book on Design, I began to appreciate design at a much, much deeper level. This book is exactly what it says on the tin and I like it for how it tries to make a seemingly abstract idea like design accessible in a concrete manner to a “hacker” (ie one who is functionality minded) by employing lots of examples.

Kadavy took care to pick out real world good design examples and explain what makes them good. He points out the underlying design principles that generally governs the interface of a design. Seriously though, how on Earth would I have expected to understand why Manet’s “Luncheon on the Garden” was the beginning of Impressionism or what makes Seurat’s “A Sunday afternoon on the Island of La Grand Jatte” a captivating painting?!

The Luncheon on the Grass by Edouard Manet

A Sunday Afternoon on the Island of La Grand Jatte by George Seurat

How did I get hold of this book?

Ok, this is another one of my web rabbit hole expedition. I was reading ESR’s jargon files where he references a blog by an anon CS student for Linux newbies. Did I need any restraint? Off I went through most of her posts (which were fun and properly written) and found this book in one of her currently readings from perhaps 2016(!). She isn’t active as of today. Last post is dated ~2018. 🤷‍♂️ I enjoyed her writings to be honest.

Design is not definitive

Of course, these principles are not set in stone as the subject it deals with is inherently a creative discipline. However, the ideas presented are a general guide for the uninitiated and they seem to be present in almost all good designs so I conclude they must be good heuristics. For eg: the idea of dominance, or direction, composition etc.

The point is design is not something one gains by reading a book. It is not a place where you can reach and exclaim to others “I have arrived!”

You can read a book summary I prepared for more or better yet read the book (which I highly recommend) if you tinker with stuff.

What next?

I would love to go to art exhibitions and develop a taste for art as I can now appreciate the ideas latent therein with more fervour.

Gemini Protocol

gs454236@gmail.com (Basil Labib) — Thu, 25 Feb 2021 00:00:00 +0000

Gemini is a minimal application layer request-response protocol that aims to make a Geminispace of information to be accessible to anyone even with slow/intermittent internet connection. Think HTTP 0.9 or a less restraining Gopher.

TLDR

If you haven’t checked out Gemini yet, you totally should . This is a HTTP proxy to the actual gemini site.

Why Gemini?

I really value your time and understand you have better things to do. So, I’ll cut down on formalities and introductions and preludes and yada yada. My format might be quirky but my motive is always to be efficient and effective and non-conformant.

Gemini appeals to me simply because of it’s lack of noise. Of course, noise is a very generic term and doesn’t make anything anymore clearer so hold on.

The WWW has evolved. It is not anymore the “collaborative space where you can communicate through sharing information” but has become so much more. It’s like a clamoring, dazzling marketplace. Which is….good…and bad. Good because there is something for everyone. The space Tim dreamt of certainly exists (and is thriving) and I admire that. WWW and HTTP has empowered man like nothing before.

Ironically, that is the bane too. The sheer amount of information that the modern woman seems to drowning in, she suffers from lack of clarity. It is so damn easy to get lost in the noises and forget your original motives. And that, my friend, is something I cannot do. Clarity of thought cannot be bargained with.

Gemini is cool precisely because it doesn’t offer much (but don’t be fooled to believe it can’t do much). It is designed deliberately to be non-extensible and thus, only serve the real purpose for any network - pure diffusion of information. Gemini is text-based (however, you can use any of the MIME formats to serve data). A big upshot is because it lacks images and other media (in a pure sense), there is virtually no ads to distract you. No banners that crept up and goddamn blink!

How does it work?

Gemini servers are applications listening on remote hosts implementing the protocol. They, by default, listen on port 1965 (the year the first manned Gemini flew as the specification so humbly states). One can have multiple domains being served from the same IP address. The protocol defines a new MIME type text/gemini to serve gemini files (analogous to text/html over HTTP) which is a light-weight markdown format and renders like HTML.

The render process is not precisely laid out in the specification and is upto the client to implement. The resultant .gemini or .gmi files are served over a regular TCP connection which is meant for exactly one request-response cycle. Note that however it increases network latency, it serves the greater good of protecting end users from noise by preventing clients from making multiple requests for unneccessary jargon. And I care about that!

The protocol can serve text or binary data and let the client handle it. And what’s better? If you can’t find a “rational” client, you can write one because the protocol is so simple. I care for this empowerment. It is most crucial for me to know precisely how the system works. Ignorance is an eyesore.

You can link to other resources in the Geminispace or even link to HTTPS or Gopher resources! Besides, Gemini takes security very seriously (unlike Gopher or HTTP) by enforcing a TLS handshake for every connection.

Check out the spec: Gemini Protocol Specification .

Gemini clients

Due to the relative simplicity of the protocol, there is a number of clients written in a number of languages. However, I could test drive only one which is AV-98 , a client written in Python by the original designer of the protocol itself.

It’s a nice little application though not cross-platform. I tried it inside a VM box running Debian. I can use this thing if it can serve me daily news in my terminal, though I am not sure how many local sites here allow scraping. Would RSS be better? Yeah, I think it’d work but who lives in the past?

The repo is here . Show it some love.

I’m fiddling with writing a client myself but shhh…

More resources

This is a great repo which is comprehensive enough to get you running (wild): Awesome Gemini .
This is a gemlog site I found, that is hosting a site in gemini space and providing a nice proxied HTTP interface to users (possibly for convenience or to attract less tech-savvy people) which IMO is very cool. It doesn’t ask for all my details and does what it should: maintain my anonymity and let me log. gemlog.blue . Definitely check it out!

Tech on Basil Labib

Can discrete Ricci flow signal capability transition in Transformers?

Abstract

Introduction

Key results

Preliminary background

Discrete curvature on a feature graph

The local and layer Ricci coefficients

Grokking to represent capability transition

Where to read a transformer’s “layers”?

Experimental setup

Results

Coefficients should be interpreted differentially

Effects of residual connections

Discussion and Limitations

Conclusion

AI usage disclosure

How to cite this work

References

Understanding System One Models Using Interpretability Tools

What are System One Models and Why Should You Care?

Setup

Methodology

When does decision form?

Does distillation impart new knowledge?

Can a probe predict mistakes better than the model’s output?

How robust are distilled System one models to prompt injection attacks?

Do the model modalities share a common intensity direction? Can it be steered?

Are System One models easier to interpret?

Can we exploit model internals to build better guardrails?

Does the model force a correct answer when there is none?

Are System One models option-shuffle invariant?

How small can a System One model adapter be?

Limitations and Future Work

Conclusion

To cite this work

References

Footnotes

The Lament of a Young ML Researcher

No bots won’t do your research for you

Cheating review with LLMs

Left hanging

The hopelessness of the situation

Ossification of research interests

The “paper” black hole

Solutions

Footnotes

Day 1 of n: Reading (and Implementing) Ilya's 30 ML Papers

My Approach

Group by paper category

Papers arranged by increasing epistemic knowledge (for beginners)

Day 1: LSTMs, RNNs, and Complexodynamics

Trial by Fire: Tenure as Head of Technology at Rendezvous ‘25

The beginning

Timelines

Building a team

The team mantra

Overview of Systems (Slightly technical)

Early Days - Setting things up

Deploying my first app on AWS

Work culture

Dealing with liabilities

Time for reflection

Good software = Reduce human cognitive load

The first crisis

The big mistake

Marketing team

The Insults and Slanders

Day 1

What happened on Day 2

The setup

The discovery

The countermeasures

What could have gone worse?

We got lucky

Reflections

I didn’t get my pass, what should I do?

Day 3

Some numbers (because an engineer’s life is incomplete without it)

System summary

It’s `useState()` all the way down

`passport.authenticate()`