Home on Ruoke Zhang

Text-to-Image System Design

Sun, 31 May 2026 00:00:00 +0000

https://medium.com/pinterest-engineering/building-pinterest-canvas-a-text-to-image-foundation-model-aa34965e84d9

https://arxiv.org/abs/2603.06453?utm_source=chatgpt.com

https://github.com/junfanz1/Awesome-AI-Review/blob/main/System%20Design/GenAI%20System%20Design.md#5-text-to-image

Pinterest Canvas

Text-To-Image

Generate images conditioned on both text prompts and reference images.

1. text-to-image

Inference
1. Embed the text prompt to obtain the text embedding
2. Feed the noisy target latent and the text embedding into the DiT together to obtain the clean target latent
3. The clean target latent passes through the VAE decoder to produce the image
Training
- Training data: (text prompt, real image)
- Only the DiT is trained. Both the text encoder and the VAE use pretrained weights, and are then frozen
- The real image passes through the VAE encoder to obtain the real latent. During training, x_t is generated from the real latent

2. multimodal image editing tasks

Stage 2 starts from the trained Stage 1 text-to-image model.

Visual Search

Wed, 27 May 2026 00:00:00 +0000

bytebytego tutorial

dataset

A query image
(n-1) negative images
A positive image, i.e., the image most similar to the query image.
The index label of the positive image among all the images.

build dataset

human judgement: humans annotate the most similar image
user interaction
- noisy
- sparse: may not have click data available for lots of the images
generate similar images

Offline metrics

Mean reciprocal rank (MRR)

MRR=1/m * sum(1/rank_i)

downside

Deep Learning

Mon, 25 May 2026 00:00:00 +0000

Neural Network

terminology

For example, suppose there are 10,000 samples with a batch size of 100:

epoch: all 10,000 samples go through forward + back prop once. After every epoch, the data should be shuffled to make sure each batch is made up of a different combination of samples each time.
iteration: one iteration means 100 samples go through one forward pass (computing the loss) + backpropagation (computing the gradients), and then optimizer.step() updates the weights.

AI Models

Sun, 24 May 2026 00:00:00 +0000

CLIP

paper

# image_encoder - ResNet or Vision Transformer
# text_encoder - CBOW or Text Transformer
# I[n, h, w, c] - minibatch of aligned images
# T[n, l] - minibatch of aligned texts
# W_i[d_i, d_e] - learned proj of image to embed
# W_t[d_t, d_e] - learned proj of text to embed
# t - learned temperature parameter
# extract feature representations of each modality
I_f = image_encoder(I) #[n, d_i]
T_f = text_encoder(T) #[n, d_t]
# joint multimodal embedding [n, d_e]
I_e = l2_normalize(np.dot(I_f, W_i), axis=1) #[n, d_e]
T_e = l2_normalize(np.dot(T_f, W_t), axis=1) #[n, n]
# scaled pairwise cosine similarities [n, n]
logits = np.dot(I_e, T_e.T) * np.exp(t)
# symmetric loss function
labels = np.arange(n)
loss_i = cross_entropy_loss(logits, labels, axis=1) # image-to-text loss
loss_t = cross_entropy_loss(logits, labels, axis=0)
loss = (loss_i + loss_t)/2

architecture

image encoder

VisionTransformer

Input: [B,3,H,W]
Output: [B,output_dim]

class VisionTransformer(nn.Module):
    def __init__(self, input_resolution: int, patch_size: int, width: int, layers: int, heads: int, output_dim: int):
        super().__init__()
        self.input_resolution = input_resolution
        self.output_dim = output_dim
        self.conv1 = nn.Conv2d(in_channels=3, out_channels=width, kernel_size=patch_size, stride=patch_size, bias=False) #width is the hidden dimension of each token in the Transformer.
        # grid=input_resolution/patch_size
        # after conv1, becomes [B, width, grid, grid]
        scale = width ** -0.5
        self.class_embedding = nn.Parameter(scale * torch.randn(width)) #class token, a special token dedicated to "collecting information from the whole image"
        self.positional_embedding = nn.Parameter(scale * torch.randn((input_resolution // patch_size) ** 2 + 1, width)) # [(input_resolution // patch_size) ** 2 + 1, width]
        self.ln_pre = LayerNorm(width)

        self.transformer = Transformer(width, layers, heads)

        self.ln_post = LayerNorm(width)
        self.proj = nn.Parameter(scale * torch.randn(width, output_dim))

    def forward(self, x: torch.Tensor):
        x = self.conv1(x)  # shape = [*, width, grid, grid]
        x = x.reshape(x.shape[0], x.shape[1], -1)  # shape = [*, width, grid ** 2]
        x = x.permute(0, 2, 1)  # shape = [*, grid ** 2, width]
        x = torch.cat([self.class_embedding.to(x.dtype) + torch.zeros(x.shape[0], 1, x.shape[-1], dtype=x.dtype, device=x.device), x], dim=1)  # shape = [*, grid ** 2 + 1, width]
        x = x + self.positional_embedding.to(x.dtype)
        x = self.ln_pre(x)

        x = x.permute(1, 0, 2)  # NLD -> LND
        x = self.transformer(x)
        x = x.permute(1, 0, 2)  # LND -> NLD

        x = self.ln_post(x[:, 0, :])

        if self.proj is not None:
            x = x @ self.proj

        return x

text encoder

embedding space

Computer Vision

Sun, 24 May 2026 00:00:00 +0000

ViT

Funnel ViT

paper

Object Detection

slides

Single Object

The label is: Cat + (x’, y’, w’, h')
x, y = the position of the box’s top-left corner, w = box width, h = box height; these are manually annotated
The CNN turns the image into a 4096-dim vector
- classification head: Fully connected, 4096 -> 1000, outputs scores for 1000 classes, then softmax
- localization head: Fully connected: 4096 -> 4, outputs predicted box = (x, y, w, h); these are continuous values, using L2 loss
- Total Loss = Softmax Loss + L2 Loss

R-CNN

R-CNN uses Selective Search to generate around 2000 region proposals, warps each proposal to a fixed 224×224 image, then runs a CNN on each region to classify it and later refine the box.

Machine Learning Basics

Wed, 20 May 2026 00:00:00 +0000

https://github.com/andrewekhalel/MLQuestions https://www.1point3acres.com/bbs/forum.php?mod=viewthread&tid=998257&page=1&extra= https://longxingtan.gitbook.io/mle-interview/02_ml

supervised learning

training data includes both the input features and corresponding target labels

classification

Logistic Regression

linear classification, binary classification loss function: Binary Cross-Entropy Loss

Does logistic regression have a closed-form solution?

No, there is no closed-form solution. Although the loss function is convex, taking its derivative yields a nonlinear equation containing an exponential term, so we cannot set the derivative to 0 and solve for closed-form expressions of w and b; we can only use gradient descent.

PyTorch

Sun, 12 Apr 2026 00:00:00 +0000

Module

The parent class of all models; every network inherits from it

init(self)
forward(self, x)

conv

Bilibili

`1`	`output=F.conv2d(input, kernel, stride=1) # performs the convolution computation, producing the feature matrix`

Conv2d

import torch
import torch.nn as nn

class CNN(nn.Module):
    def __init__(self):
        super().__init__()

        self.conv1 = nn.Conv2d(
            in_channels=3,
            out_channels=6,
            kernel_size=3,
            stride=1,
            padding=0
        )

        self.maxpool1 = nn.MaxPool2d(
            kernel_size=3,  # Pooling window size: 3×3
            ceil_mode=True  # Ceiling mode, keeps incomplete edge windows
            # stride defaults to kernel_size=3; you can also explicitly write stride=3, with exactly the same effect
        )

    def forward(self, x):
        # Forward pass: **first convolve to extract features, then pool to downsample** (standard CNN order)
        # x: [b, 3, h, w]
        x = self.conv1(x)       # Step 1: pass through the conv layer, producing a 6-channel feature map, [b, 6, h', w']
        # h'=(h+2*p-k)/s+1
        x = self.maxpool1(x)    # Step 2: pass through the pooling layer, downsampling the feature map
        return x

Linear

self.linear = nn.Linear(
    in_features=1024,
    out_features=512,
    bias=True        # Default True; whether to add the bias b
)

weight = (512, 1024) bias = (512) y = x @ W.T + b

Distributed Systems

Thu, 08 Jan 2026 00:00:00 +0000

RPC Basics

protobuf IDL

Define the .proto file: define Request/Response formats and function signatures.

Protobuf generates CalculatorStub automatically.

channel = grpc.insecure_channel("server-ip:port")

# Client stub (a local proxy for server-side methods)
stub = CalculatorStub(channel)

# Call Stub.Add as if it were a local function
request = AddRequest(num1=2, num2=3)
response = stub.Add(request)

Implement the actual logic for add on the server side.

Semantics

At most once

File Systems and I/O Models

Mon, 05 Jan 2026 00:00:00 +0000

File Systems

Inodes

An inode identifies a file within one file system. It stores metadata such as:

Permissions, owner UID, and group GID
File size and timestamps: ctime for metadata changes, mtime for content changes, and atime for access
Pointers to data blocks
A link count, which tracks hard links to the inode

To read a regular file, the OS resolves each component of its path through directory entries, finds the file’s inode, and uses its block pointers to locate the data. A symbolic link has its own inode and contains a target path, which the OS resolves in turn.

Java RPC: Transport, Discovery, and Reliability

Tue, 30 Dec 2025 00:00:00 +0000

RPC and HTTP

In this project, the RPC protocol uses a small, customizable message header. Providers register themselves when they start, and consumers discover their addresses through a registry. The client handles load balancing, retries, and circuit breaking; the server applies rate limiting.

Serialization

Kryo

Kryo serializes Java objects compactly and quickly, but a Kryo instance is not thread-safe. Registering a class assigns it an ID, so a serialized message can refer to that ID rather than repeatedly writing the full class name:

About

Sun, 07 Sep 2025 00:00:00 +0000

About Me

I am transitioning from software engineering to machine learning.

Current interest: Agentic AI
I enjoy learning on my own and exploring new tools.

2025-2027: Master of Science, Electrical and Computer Engineering, University of Washington

2021-2024: Dual Bachelor of Science in Mathematics and Computer Science, The Ohio State University

2019-2021: B.S. in Mathematics (transferred out), Hefei University of Technology

2016-2019: Wuhan Foreign Languages School

Contact

GitHub: RuokeZhang
LinkedIn: Ruoke Zhang

Bit Manipulation

Fri, 05 Sep 2025 00:00:00 +0000

Common Operations

x.bit_count(): Returns the number of 1s in the binary representation of x. Can also be interpreted as the minimum number of powers of 2 needed to represent x.

Brain Teasers

2749. Minimum Operations to Make the Integer Zero

Let x = num1 - k * num2, then the problem transforms to: Can x be represented using k powers of 2 (duplicates allowed)?

Find the upper and lower bounds of x and enumerate. Plot x = num1 - k * num2 with k as x-axis and x as y-axis. The answer lies in some points on this line in the first quadrant. Analyze the equation x=2^(i1)+…2^(ik) to find possible x values.

Math

Mon, 01 Sep 2025 00:00:00 +0000

Prime Numbers

Sieve of Eratosthenes

# Time complexity O(MX * log log MX)
MX = 1_000_001
is_prime = [False] * 2 + [True] * (MX - 2)  # 0 and 1 are not prime
primes = []
for i in range(2, MX):
    if is_prime[i]:
        primes.append(i)
        for j in range(i * i, MX, i):
            is_prime[j] = False  # j is a multiple of prime i

204. Count Primes

p[i] represents the number of primes less than or equal to i. Starting from i=2, p[i]==0 <=> i is prime. Then, starting from ii, mark multiples of i as composite (by setting p[ik]=-1). And, p[i] is assigned p[i-1]+1.

String

Mon, 18 Aug 2025 00:00:00 +0000

KMP

https://www.zhihu.com/question/21923021/answer/37475572

28. Find the Index of the First Occurrence in a String

Pattern: abababzababab

Step 1: Construct next array while traversing pattern.

next[i] represents maximum common prefix-suffix length for s[0:i+1]
Initialize j=0, i traverses from 1 to n-1. j position indicates current common prefix-suffix s[:j]
During traversal, if common prefix-suffix exists, continuously compare s[i] and s[j]. If different, in loop backtrack j until s[:j] becomes common prefix-suffix. Backtrack to second longest common prefix-suffix, move j to next[j-1]
Compare s[i] and s[j] again. If same, move j and assign next[i]

Step 2: Compare two strings abeababeabf abeabf

Network

Sat, 29 Mar 2025 00:00:00 +0000

TCP Retransmission

Timeout Retransmission: A timer is set, which should be slightly longer than the Round-Trip Time (RTT). If the timer expires before an ACK is received, the segment is immediately retransmitted.
Fast Retransmission: When three consecutive duplicate ACKs are received, the segment is retransmitted immediately.
SACK (Selective Acknowledgment): The receiver can inform the sender which segments have been received and which are missing.
DSACK (Duplicate SACK): If the receiver gets two identical segments (e.g., two 2s), it uses DSACK to inform the sender that it has received duplicates, preventing unnecessary retransmissions due to network latency.

TCP Flow Control

Slow Start

When transmission begins, the sender doubles its congestion window for each ACK received (exponential growth). The initial window size is 1.

Greedy

Fri, 28 Mar 2025 00:00:00 +0000

Ling Shen Problem List

300. Longest Increasing Subsequence

Ling Shen Video

Define g[i] as the minimum end element of an increasing subsequence of length i+1.

Proof by contradiction: g is a strictly increasing array.

659. Split Array into Consecutive Subsequences

tail[i] is used to represent the number of subsequences ending with i. This way, when i+1 is encountered, it can be placed.

If it cannot be placed, open a new sequence and check if cnt[i+1]>0 and cnt[i+2]>2. If not satisfied, return False.

MySQL Logs

Wed, 26 Mar 2025 00:00:00 +0000

All DML operations (INSERT/UPDATE/DELETE) are first completed in the Buffer Pool

redo log

Video Explanation

Implements transaction durability. redo log is flushed to disk first, then dirty pages are flushed to disk

Physical Storage Characteristics

File Structure
- Composed of fixed-size files (such as ib_logfile0, ib_logfile1)
- Circular write (ring buffer) design
- Physical writes are always appended to the end of the file
Write Mode

1
2

// Pseudo code representing physical write
write(log_file, log_record, log_size); // Always append write

Write-Ahead Logging (WAL):

Update data pages in Buffer Pool, these data pages become dirty pages.
Write physical changes to data pages (such as “page number X, offset Y write data Z”) sequentially to redo log buffer.
When executing COMMIT, behavior is determined by configuration parameter innodb_flush_log_at_trx_commit (usually default value is 1)

Parameter Value	Behavior	Durability Guarantee
1 (default)	1. Write PageCache 2. Immediate fsync() flush to disk	Complete durability
0	Batch write + flush to disk every second	May lose 1 second of data
2	Write PageCache but delay flush to disk	Only guarantees OS doesn’t crash

Asynchronous dirty page flush to disk, background threads flush data from BufferPool to hard disk

Redis Interview Notes

Tue, 25 Mar 2025 00:00:00 +0000

Sorted Sets (`ZSET`)

A sorted set maps each unique member to a score and orders members by score. Range queries and updates are typically O(log N).

Compact Encoding

Small sorted sets can use a compact listpack representation. Redis converts to its general representation when the set exceeds limits such as zset-max-listpack-entries or zset-max-listpack-value. Older Redis versions used a ziplist and correspondingly named settings. Check the settings for the version you run.

Common MySQL Interview Questions

Mon, 24 Mar 2025 00:00:00 +0000

Storage Engine Comparison

In InnoDB, primary key index and data are stored together, with the primary key index B+ tree containing complete data rows; in MyISAM, index and data are stored separately, with the B tree storing index values and physical addresses of rows
InnoDB supports ACID transactions, MyISAM does not
InnoDB supports row-level locking, MyISAM does not

What is Page Split

When inserting new data into a full index page (usually 16KB), the storage engine will split the current page into two pages to accommodate the new data

Java Reflection & Dynamic Proxy

Tue, 18 Mar 2025 00:00:00 +0000

Reflection

Reflection is Java’s ability to dynamically obtain class information (such as class name, methods, fields, constructors, etc.) and manipulate classes at runtime.

API

Getting a Class Object

String str="test";
Class<?> clazz = String.class;

String str = "test";
Class<?> clazz = str.getClass();

Class<?> clazz = Class.forName("java.lang.String");

Creating Objects via Reflection

Class<?> clazz = Class.forName("com.example.User");
Object obj = clazz.getDeclaredConstructor().newInstance();

Constructor<?> constructor = clazz.getConstructor(String.class, int.class);
Object obj = constructor.newInstance("Alice", 25);

Getting Fields

getDeclaredField("fieldName")

MySQL Locks

Mon, 17 Mar 2025 00:00:00 +0000

Global Lock

Generally used when doing full database backup. It blocks write operations and only allows read requests. Therefore, it’s usually chosen during business low-peak periods for full database backup.

`1`	`flush tables with read lock`

Table-Level Locks

Table Lock

Table shared read lock. Multiple threads can acquire this lock simultaneously. Threads with this lock can read the table but cannot write to it. MyISAM acquires table shared read locks when processing read requests. InnoDB defaults to row locks and handles read requests through MVCC.
Table exclusive write lock. Only one thread can acquire this lock and perform write operations on the table. No other threads can read this table.

`1`	`lock tables t_student read;`

Table locks are triggered by: ALTER/DROP/TRUNCATE TABLE

Redis High Availability

Sat, 15 Mar 2025 00:00:00 +0000

References: Redis replication, Sentinel, and Redis Cluster.

Primary–Replica Replication

Replication provides copies of data and can serve reads from replicas. A primary accepts writes and sends updates to its replicas. Replication is asynchronous, so a replica can lag behind a successful write.

Full Synchronization

A replica connects to a primary and requests synchronization with PSYNC.
If a partial sync is impossible, the primary responds with FULLRESYNC, including its replication ID and offset.
The primary produces an RDB snapshot and sends it to the replica while buffering subsequent writes.
The replica loads the snapshot and applies the buffered updates, then continues receiving the replication stream over a long-lived connection.

To configure a replica, use REPLICAOF <primary-host> <primary-port>. A full sync consumes network and CPU resources; replicas can also replicate from other replicas to reduce load on the primary.

Binary Search

Thu, 13 Mar 2025 00:00:00 +0000

https://leetcode.cn/discuss/post/3579164/ti-dan-er-fen-suan-fa-er-fen-da-an-zui-x-3rqn/

Closed Interval Approach

Find Target

class Solution:
    # Standard binary search framework, returns index of target element, -1 if not found
    def search(self, nums: List[int], target: int) -> int:
        left = 0
        right = len(nums) - 1

        while left <= right:
            mid = left + (right - left) // 2
            if nums[mid] == target:
                return mid
            elif nums[mid] < target:
                left = mid + 1
            elif nums[mid] > target:
                right = mid - 1
        return -1

Find Left Boundary

def search(self, nums: List[int], target: int) -> int:
    left = 0
    right = len(nums) - 1

    while left <= right:
        mid = left + (right - left) // 2
        if nums[mid] == target:
            right=mid-1
        elif nums[mid] < target:
            left = mid + 1
        elif nums[mid] > target:
            right = mid - 1
    return left

Left-Closed Right-Open

Find Left Boundary

def left_bound(nums: List[int], target: int) -> int:
    left = 0
    right = len(nums)
    
    while left < right:
        mid = left + (right - left) // 2
        if nums[mid] == target:
            right = mid
        elif nums[mid] < target:
            left = mid + 1
        elif nums[mid] > target:
            right = mid

    return left

If target doesn’t exist, left-bound binary search returns index of smallest element > target.

BlackHorse Rating Login Design

Thu, 13 Mar 2025 00:00:00 +0000

User enters phone number on login page and clicks “Get Code”. Backend validates phone format, generates a verification code, stores phone:code in Redis with 2-minute expiry.
User enters code and clicks “Login”. Backend compares submitted code with stored value.
If user doesn’t exist, create and save to database.
Generate random token as login credential; convert User object to Hash and store as token:userHash in Redis.

Requests carry token in header. RefreshTokenInterceptor intercepts all requests (passes through if no token).
Extract token from header, retrieve user Map from Redis using token.
If user exists, convert Map to UserDTO, store in ThreadLocal, and refresh token expiry.

LoginInterceptor intercepts all paths except specified public ones. If UserHolder is empty (not logged in), request is rejected.

Exceptions, Generics

Thu, 13 Mar 2025 00:00:00 +0000

Exceptions

Throwable

Throwable has two subclasses: Exception and Error.

Error class represents serious errors, which are generally not recommended to be caught by catch in our programs. We can only optimize Errors by optimizing the code. When an Error occurs, JVM generally chooses to terminate the thread.

Exception class has two subclasses: Checked Exception and Unchecked Exception. Checked Exception means that the exception will be checked at compile time. If the exception is not caught by catch or not throws, it will not pass compilation. This includes IOException, ClassNotFoundException. Unchecked Exception means that it will not be checked at compile time. It includes RuntimeException and its subclasses, such as NullPointerException, ArrayIndexOutOfBoundsException, and ClassCastException that may occur during forced type conversion.

Graph

Thu, 13 Mar 2025 00:00:00 +0000

Graph Problem List

Grid Graph Problem List

Graph

DFS

Find connected components, check for cycles

261. Graph Valid Tree

Undirected graph, check if it’s acyclic and connected.

Using vis[i]=True is sufficient. However, since it’s an undirected graph, when traversing neighbors, you need to skip the parent node, so it needs to be passed as a parameter.

def dfs(i, parent):
    vis[i]=True
    for nxt in g[i]:
        if nxt==parent:
            continue
        if vis[nxt]:
            return False
        if not dfs(nxt, i):
            return False
    return True

For example, 1-3-6, 1-5-6. If connected this way, there must be a cycle, so three colors are not needed.

Java Collections

Tue, 11 Mar 2025 00:00:00 +0000

Collection Interface

List

ArrayList

Basic Concepts

Allows null values.
Memory is contiguous, saving memory space.
Insertion time complexity: O(n) for insertion at the head because all subsequent elements need to be shifted; O(1) for insertion at the tail if no resizing is needed, otherwise O(n) to copy the original ArrayList and then O(1) for insertion. O(n) for insertion in the middle.
Deletion time complexity: Corresponds to insertion time complexity.
Not thread-safe.

Underlying Implementation

Implemented by dynamic array. Random access to a node only requires calculating its address, so random access is O(1). Implements the RandomAccess marker interface.
If no initial capacity is specified, the default initial capacity is 0. When the first element is added, the capacity becomes 10. Each dynamic expansion increases the capacity to 1.5 times the original.

Differences between ArrayList and Array

ArrayList can be dynamically resized; Array cannot, its length is fixed and needs to be specified when initializing the array.
ArrayList supports generics.
ArrayList only supports objects; for primitive data types, their wrapper classes like Integer, Double are needed. Array supports both primitive data types and objects.

LinkedList

Underlying Implementation

Implemented by a doubly linked list. Insertion/deletion at the head/tail are both O(1).
Does not support random access. Time complexity for accessing a specific element is O(n).
Performance is not as good as ArrayList, rarely used in actual development.
Not thread-safe.
Allows null values.
Occupies more space than ArrayList because it stores pointers in addition to data.

HashMap

Implementation Principle

Implemented by hash table + linked list/red-black tree. Hash collision resolution methods include chaining/probing, usually chaining.
How is the hash value calculated? Get the hashCode() of the key. Perform an XOR operation between the original hashcode and the hashcode shifted right by 16 bits.

`1`	`int hash = (key == null) ? 0 : (key.hashCode()) ^ ((key.hashCode()) >>> 16);`

Calculate array index. hash & (length - 1), equivalent to hash % length. Binary operations are faster than modulo operations.
If a collision occurs, check if the key exists in the linked list or red-black tree. If it exists, replace the value directly. If not, add it. When the linked list length is 8 and the hash table length is greater than 64, it is converted to a red-black tree.
If the number of red-black tree nodes is less than 6, it reverts to a linked list.
Initial capacity is 16, load factor is 0.75, expands to twice the original size.
Not thread-safe. Resizing may cause a circular linked list.

Time Complexity

Ideally, the average time complexity for HashMap’s insertion, deletion, and lookup operations is O(1). If converted to a red-black tree, the time complexity of operations will stabilize at O(log n).

Distributed Training

Sat, 08 Mar 2025 00:00:00 +0000

Communication Methods

Intra-Machine Communication

NVLink: Technology for direct communication between GPUs. NVLink 3.0 has a unidirectional bandwidth of up to 50 GB/s.
PCIe: A universal high-speed bus standard used to connect GPUs, CPUs, and other peripherals. Under the PCIe architecture, communication between GPUs requires going through the CPU and system memory, with bandwidth and latency not as good as NVLink. PCIe 4.0 x16 has a unidirectional bandwidth of 31.5 GB/s. PCIe Switch is used to expand PCIe lanes, allowing multiple devices to share bandwidth.

Inter-Machine Communication

TCP/IP: Communication rate is limited by hardware and is slower than intra-machine communication.

Process

Fri, 07 Mar 2025 00:00:00 +0000

Process

A process uniquely owns:

PC
Stack pointer (stores return addresses)
Address space
File descriptors

To manipulate a process, user code must invoke system calls to enter kernel mode. Always check syscall return values; -1 indicates an error.

Kernel Space

Kernel space is at the top of the entire virtual address space and is shared by all user processes. User processes occupy the lower portion.
As long as the system is running, the kernel remains active. It doesn’t start/stop like ordinary processes; it initializes at boot and runs until shutdown.

Process termination

A process can terminate via three paths:

Redis Persistence: AOF and RDB

Fri, 07 Mar 2025 00:00:00 +0000

References: Xiaolin Coding video and Redis persistence documentation.

Append-Only File (AOF)

AOF records write operations after Redis applies them in memory. Redis can replay the log at startup to reconstruct the dataset. The log uses the Redis protocol format, so it is generally larger than an RDB snapshot.

Why Log After Executing a Command?

Redis records a write only after it has accepted and executed it. Appending and syncing the log still have a cost; the chosen appendfsync policy determines the durability and performance tradeoff.

Virtualization

Wed, 05 Mar 2025 00:00:00 +0000

Referenced technical blog
OSTEP slides
Reference

Virtual Memory

Virtualization: each process believes it has its own address space, and different processes’ address spaces are isolated from each other.

In practice, when the CPU gets a virtual address, the MMU inside the CPU translates the virtual address into a physical address (an address in main memory). Each memory address refers to a single byte.

Segmentation

Each segment corresponds to a logical entity in the address space, for example the stack or data. Each segment can:

Troubleshooting

Mon, 03 Mar 2025 00:00:00 +0000

Reference video (Chinese)

Causes of OutOfMemoryError

Allocating too many objects at once → use pagination/streaming
Memory leak: reachable but no-longer-needed objects still retained → find and release them
Insufficient resources → use to inspect the heap

jmap is a JDK CLI tool to generate a heap dump. A heap dump is a binary snapshot of all objects on the heap. Or add -XX:+HeapDumpOnOutOfMemoryError so the JVM dumps the heap automatically on OOM.

Transaction

Sun, 02 Mar 2025 00:00:00 +0000

Reference: Xiaolin Coding

Transaction Properties

Atomicity: Implemented via undo log.

Consistency: Achieved through the other three properties. Before and after a transaction, the database transitions from one consistent state to another.

Isolation: Implemented via MVCC/locking. The execution of one transaction does not affect the execution of others.

Durability: Implemented via redo log. Once a transaction is committed, its changes to the database are permanent, even in the event of a system failure or crash.

Indexes

Fri, 28 Feb 2025 00:00:00 +0000

Clustered Index, Secondary Index

Clustered Index

In InnoDB, each table has a clustered index that determines the physical order of data storage on disk.

If there is a primary key, it serves as the clustered index
If there is no primary key, choose a unique non-null index as the clustered index
If there still isn’t one, InnoDB implicitly creates an auto-increment column as the clustered index

B+ Tree

By default, InnoDB uses B+ Tree storage for both clustered and secondary indexes.

ThreadLocal & ThreadPool

Thu, 27 Feb 2025 00:00:00 +0000

Reference: JavaGuide

ThreadLocal

The Thread class has a variable threadLocals, which can be considered a HashMap used to store thread-private content.

1
2
3

/* ThreadLocal values pertaining to this thread. This map is maintained
    * by the ThreadLocal class. */
ThreadLocal.ThreadLocalMap threadLocals = null;

The ThreadLocal class has an instance method set, which actually stores the content in threadLocals. The key is the ThreadLocal object itself.

Skill Problems

Wed, 26 Feb 2025 00:00:00 +0000

31. Next Permutation

Consider the last permutation: 5-4-3-2-1, all in descending order.

From right to left, find the first ascending pair nums[i] and nums[i+1], e.g., 12947653, found nums[4]=4.
From right to left, find the first number nums[j] greater than nums[i], which is 5.
Swap nums[i] and nums[j], resulting in 12957643. At this point, all numbers after nums[i] are in descending order.
Sort the numbers after nums[i] in ascending order, resulting in 12953467.

128. Longest Consecutive Sequence

[100,4,200,1,3,2]

Array

Sat, 22 Feb 2025 00:00:00 +0000

LC 188 Rotate Array

Input: nums = [1,2,3,4,5,6,7], k = 3 Output: [5,6,7,1,2,3,4]

Method 1: Slicing operation; take last k elements: nums[-k%n:]

Method 2: Reverse entire array first, then reverse parts. Implement reversal separately without slicing to avoid extra space.

Two Pointers

Opposite Direction Two Pointers

LC 75: Sort Colors

Use pointer i to traverse array, left and right pointers point to next position to fill 0 or 2. First continuously fill 2s, then fill 0s when encountered. Loop condition: i<=r. When i>r, nums[i] must be 2.

JMM

Fri, 21 Feb 2025 00:00:00 +0000

Reference: JavaGuide

Concurrency Issues

CPU Cache

The value of a variable in the Cache may be inconsistent with the value in Main Memory, leading to calculation errors.

Instruction-Level Reordering

Compiler Optimization Reordering

JVM and JIT do this. For Java programs, instruction reordering is performed during compilation without changing its semantics, similar to g1, g2 optimization reordering.

Solution: Prohibit compiler reordering

Instruction Parallel Reordering

CPU does this, related to CPU pipeline. It means a thread can run in parallel on different CPU hardware.

Sorting

Fri, 21 Feb 2025 00:00:00 +0000

Merge Sort

LC 148: Sort a Linked List

Use fast and slow pointers to divide into two halves, break in the middle, recursively sort each half, and finally merge the two sorted linked lists in ascending order. Boundary conditions: head==None or head.next==None.

LC 23. Merge k Sorted Lists

Pay attention to edge cases.

Quick Sort

Using an element of the array (usually the first element) as the pivot, move all elements smaller than the pivot to its left, and all elements larger than the pivot to its right. Python Template

Common Python Operations

Tue, 18 Feb 2025 00:00:00 +0000

intervals.sort(key=lambda p: p[0])

LC 189 [:] slicing operation

dict

# Dictionary with initial key-value pairs
filled_dict = {'key1': 'value1', 'key2': 'value2'}

# Using dict() constructor
constructed_dict = dict(key1='value1', key2='value2')

valueList=list(filled_dict.values())

my_dict = {'apple': 1, 'banana': 2, 'cherry': 3}
for key in my_dict.keys():
    print(f"Key: {key}")

LC 49

Java Threads

Tue, 18 Feb 2025 00:00:00 +0000

Reference: JavaGuide

Threads

Thread Implementation

A Java program runs as a process and is actually multi-threaded concurrently; the main function is the main thread. Each instance of java.lang.Thread that has called start() and not yet finished is a thread. Java threads are the concurrent execution units of Java programs.

One-to-One Thread Model✨

Each Java thread corresponds to a kernel thread in the operating system, fully utilizing multi-core processors. (Now Windows and Linux use this.)

Prefix Sum

Tue, 18 Feb 2025 00:00:00 +0000

Prefix Sum Problem List

One-dimensional Prefix Sum

LC 1749

Maximum absolute sum of any subarray: Find the minimum and maximum values in the prefix sum array, and the absolute difference between them is the answer.

1
2
3

def maxAbsoluteSum(self, nums: List[int]) -> int:
    s = list(accumulate(nums, initial=0))  # prefix sum of nums
    return max(s) - min(s)

2055. Plates Between Candles

Find the first candle at or to the right of a
Find the first candle at or to the left of b
Use prefix sum to count the number of plates between them

Steps 1 and 2 can use binary search, but it will time out. It’s best to pre-process and record the position of the first candle to the left/right of each position.

Sliding Window

Tue, 18 Feb 2025 00:00:00 +0000

reference: Lingchashan Aifu

Fixed-size Sliding Window

Entry: The element at index i enters the window, update relevant statistics. If i<k-1, repeat the first step.

Update: Update the answer. Generally, update the maximum/minimum value.

Exit: The element at index i-k+1 leaves the window, update relevant statistics.

LC 1456 LC 1052 Grumpy Bookstore Owner: Answer = number of customers during all non-grumpy times + number of grumpy customers within the window.

1297. Maximum Number of Occurrences of a Substring ✨

High-frequency problem, greedy approach.

Garbage Collection

Mon, 17 Feb 2025 00:00:00 +0000

References: Understanding the JVM (3rd Edition, Chinese), G1 & ZGC deep dive (Chinese)

Is an Object Dead?

Reference Counting

Drawback: they reference each other, so neither count goes to 0; they won’t be collected.

Reachability Analysis

Start from GC Roots and traverse references downward, marking all reachable objects. The traversal paths are “reference chains.”

GC Roots include:

Objects referenced from local variable tables in JVM stack frames (method params, temps, and the this of instance methods)
Class static variables (in the method area)
Constants referenced from the method area
Native references from the native method stack
All objects held by synchronized
JVM-internal refs: Class objects for primitives, permanent exception objects, system class loader

Even if first marked “unreachable,” an object isn’t necessarily collected immediately.

Binary Tree Patterns

Wed, 12 Feb 2025 00:00:00 +0000

Problem list

Bottom-Up DFS: Postorder Traversal

Use postorder when the computation at a node needs results from its children.

1110, Delete Nodes and Return Forest: Process children before deciding which remaining subtrees become new roots.
1123, Lowest Common Ancestor of Deepest Leaves: Return both the deepest depth beneath a node and the common ancestor of leaves at that depth.
1530, Number of Good Leaf Nodes Pairs: Return the distances from a node to leaves in its subtree. Combine distances from opposite children to count valid pairs.

Preorder Traversal

1026, Maximum Difference Between Node and Ancestor: Carry the smallest and largest ancestor values down the path. A postorder solution is also possible.

Dynamic Programming

Wed, 12 Feb 2025 00:00:00 +0000

Ling Shen Problem List

Linear DP

10. Regular Expression Matching

dp[i][j] indicates if s(0,i-1) matches p(0,j-1)

Base case: s empty, p empty → True; s empty, p xx type → True Compare s[i], p[j]: if not match and p[j] is *, and s[i] matches p[j-1], three possibilities: match one letter, zero letters, multiple letters. If not match and p[j] is *, and s[i] doesn’t match p[j-1], check zero letter option.

Linked List

Sun, 09 Feb 2025 00:00:00 +0000

Reverse Linked List

def reverseBetween(self, head: Optional[ListNode], left: int, right: int) -> Optional[ListNode]:
    #Answer head might change, so use dummy
    dummy=ListNode(next=head)
    pre=dummy
    #Move pre, cur to positions 1 and 2
    for _ in range(left-1):
        pre=pre.next
    cur=pre.next
    #Record current pre
    p0=pre
    #Reverse from 2 to 4, essentially changing next pointers
    for _ in range(right-left+1):
        nxt=cur.next
        cur.next=pre
        pre=cur
        cur=nxt
    #New pre,cur are 4,5
    #Connect 2 and 5
    p0.next.next=cur
    #Connect 1 and 4
    p0.next=pre
    return dummy.next

Reverse linked list focuses on pre and cur positions before start and after end. In this case, old pre.next becomes new pre, old cur.next becomes new cur.

Influencer Exploration

Fri, 07 Feb 2025 00:00:00 +0000

Like Design

Use Redis sorted set: key is post ID, set stores user IDs who liked, score is timestamp of like.

When frontend views post details, backend needs to:

Fetch content from database by post ID
Check Redis to determine if current user liked this post
Add liked field to Blog object and return to frontend

@PutMapping("/like/{id}")
public Result likeBlog(@PathVariable("id") Long id) {
    return blogService.likeBlog(id);
}

Frontend doesn’t send current like status; backend checks if user liked by checking sorted set for user ID.

Interfaces and Abstract Classes

Fri, 07 Feb 2025 00:00:00 +0000

Interface

An interface is an abstraction of behavior.

Abstract methods: Before Java 8, interfaces could only have abstract methods.
Default methods: Provide method bodies, available since Java 8.
Static methods: Provide method bodies, available since Java 8, can be called via Interface.method().

public interface MyInterface {
    // Abstract method
    void abstractMethod();

    // Default method
    default void defaultMethod() {
        System.out.println("This is the default method of the interface");
        privateHelperMethod();  // Call private helper method (Note: private methods are a Java 9 feature, shown here for example only)
    }

    // Static method
    static void staticMethod() {
        System.out.println("This is the static method of the interface");
    }

    // Private helper method (Java 9 feature, but included for completeness)
    private void privateHelperMethod() {
        System.out.println("This is the private helper method of the interface");
    }
}

public class MyClass implements MyInterface {
    @Override
    public void abstractMethod() {
        System.out.println("Implementing the abstract method of the interface");
    }

    // Can choose to override the default method
    @Override
    public void defaultMethod() {
        System.out.println("Overriding the default method of the interface");
        MyInterface.super.defaultMethod();  // Call the default method of the interface
    }
}

public class Main {
    public static void main(String[] args) {
        MyClass myClass = new MyClass();

        // Call the implemented abstract method
        myClass.abstractMethod();

        // Call the default method
        myClass.defaultMethod();  // Note that the overridden version is called here

        // Call the static method of the interface
        MyInterface.staticMethod();
    }
}

Abstract Class

Abstract classes typically serve as base classes, such as Person, Animal. Because the body of certain methods may differ among various subclasses, these methods are placed as abstract methods in the abstract class.

New Features in Java 8

Fri, 07 Feb 2025 00:00:00 +0000

Functional Interface

A functional interface is defined as an interface that must contain only one abstract method. It can have many static or default methods (with implementations), but there must be only one abstract method.

This abstract method indicates that it can perform a certain behavior, but the specific action is determined by the caller through overriding this abstract method. This can be achieved using anonymous inner classes or lambda expressions.

Concurrency

Wed, 05 Feb 2025 00:00:00 +0000

Goals

Mutual exclusion (e.g., A and B don’t run at the same time) — process mutual exclusion
- solved with locks
Ordering (e.g., B runs after A does something) — process synchronization
- solved with condition variables OR semaphores

CS 537 Videos 小林coding

Locks

You can think of a lock as a mutex. It’s just a variable with two states: available/acquired. Each thread can acquire and release it. Reference: OSTEP-Locks

BlackHorse Rating Merchant Cache

Mon, 03 Feb 2025 00:00:00 +0000

Consistency

Cache Update Strategies

Memory Eviction: Redis automatically evicts data when memory is low; cache updated on next request.
TTL Expiry: Set TTL on cached data; deleted automatically when expired; updated on next request.
Active Update: Update cache simultaneously with database modification.

Cache-Database Consistency

Concurrent requests can cause inconsistency regardless of update order (cache first or database first).

Use Cache Aside strategy: cache only updated during read operations.

Write strategy: When modifying data, update database and delete cache.
Read strategy: When reading data, if cache miss, read from database and update cache.

Update-Then-Delete (Delayed Double Delete)

Thread A reading, Thread B updating simultaneously:

Coupon Seckill

Sat, 01 Feb 2025 00:00:00 +0000

Global ID Generator: Redis Auto-increment

Disadvantages of database auto-increment IDs:

May expose business metrics: users could infer daily coupon sales.
Large coupon volumes lead to oversized IDs; sharding causes ID conflicts.

// Sign bit 0 + 31-bit timestamp + 32-bit sequence number
public long nextId(String keyPrefix){
    // Generate timestamp
    LocalDateTime now=LocalDateTime.now();
    long nowSecond=now.toEpochSecond(ZoneOffset.UTC);
    long timestamp=nowSecond-BEGIN_TIMESTAMP;
    // Generate sequence number. Same key per day
    String date=now.format(DateTimeFormatter.ofPattern("yyyy:MM:dd"));
    long count=  stringRedisTemplate.opsForValue().increment("icr:"+keyPrefix+":"+date);
    // Combine and return
    return timestamp<<COUNT_BITS|count;
}

keyPrefix represents object type, e.g., coupon order.

Redis Caching Patterns and Failure Modes

Fri, 31 Jan 2025 00:00:00 +0000

References: Redis query caching, caching patterns, and Redis key eviction.

Caching Patterns

Cache-Aside (Lazy Loading)

On a read, the application checks Redis first. A hit returns immediately. On a miss, it queries the database, stores the result in Redis, and returns it. The cache is filled only for requested data, which suits read-heavy workloads and keeps the cache smaller.

Read-Through

The application asks a cache provider for data. The provider performs a database read on a miss and manages filling the cache, so the application does not contain the miss-handling logic.

JVM Basic

Thu, 30 Jan 2025 00:00:00 +0000

ClassFile

After a Java source file is compiled by javac, it becomes a .class bytecode file and then runs on the JVM. A ClassFile is a structured specification; all .class files must conform to this format.

Basic Info

magic number: 0xCAFEBABE. Every .class must start with these 4 bytes.
Version number, representing the compiler (javac) version.
access flags

Constant Pool

There are 14 kinds of constants, grouped into literals and symbolic references. Literals are strings, final constants, etc. Symbolic references are things like method names and field names; they point to a string entry, i.e., a literal.

Mon, 01 Jan 0001 00:00:00 +0000

Heap

1. Reorganize String (Roblox)
1. Task Scheduler (Roblox, Snowflake)
1. Top K Frequent Words (Adobe)
1. Top K Frequent Elements (Adobe, Bloomberg)
1. Maximum Number of Events That Can Be Attended (Snowflake)
1. Find K Pairs with Smallest Sums (Linkedin, Google)
1. Last Stone Weight (Nvidia)
1. Find Median from Data Stream (Snowflake)
Merge k Sorted Lists (Bloomberg, Snowflake, Nvidia, Microsoft)
1. Stock Price Fluctuation (Google)
1. Meeting Rooms III

Prefix Sum

1. Subarray Product Less Than K (Airbnb)
1. Max Consecutive Ones III (Microsoft, Expedia)
1. Subarrays with K Different Integers (Expedia)
1. Range Sum Query 2D - Immutable (Snowflake)
Subarray Sum Equals K (Bloomberg)
Product of Array Except Self (Bloomberg, Microsoft)
Meeting Rooms II (Microsoft)

Trie

1. Add and Search Word - Data structure design (Docusign, LinkedIn, Snowflake, Rubrik, Tiktok)
1. Longest Common Prefix (Bloomberg)
1. Find the Length of the Longest Common Prefix (Databricks)
Implement Trie (Prefix Tree) (Snowflake)
1. Design In-Memory File System (Snowflake)
1. Remove Sub-Folders (Snowflake, Google)
1. Search Suggestions System (Docusign, Amazon)
1. Word Search II (Microsoft, Google, Airbnb)

Stack

Basic Calculator II (Tesla)
Max Stack (Linkedin)
Nested List Weight Sum II (Linkedin)
Valid Parentheses (Nvidia, Expedia)
1. Evaluate Reverse Polish Notation (Microsoft)
1. Basic Calculator (Microsoft)
1. Valid Parentheses (Microsoft)
1. Longest Valid Parentheses (Tiktok)
1. Maximal Rectangle (Tiktok)
1. Largest Rectangle in Histogram (Microsoft)
1. Simplify Path (Snowflake, Tiktok)
1. Remove Duplicate Letters (Expedia)
1. Number of Visible People in a Queue (Expedia)
1. Minimum Number of Swaps to Make the String Balanced (Expedia)
1. Sum of Subarray Ranges (Tiktok)

Queue

1. Design Circular Queue (Tesla, Cloudflare)
1. Design Hit Counter (Snowflake, Cloudflare, Databricks)
1. Longest Continuous Subarray With Absolute Diff Less Than or Equal to Limit (Google)

Common Mistakes

1. Design Circular Queue (Tesla, Cloudflare)

Hash

1. First Missing Positive (Tesla)
1. Missing Number (Meta, Microsoft, Bloomberg)

Sort

1. Kth Largest Element in an Array (Microsoft)

Sliding Window

1. Max Consecutive Ones III (SAP, Salesforce, LinkedIn, Meta, Google)
1. Longest Repeating Character Replacement (Adobe, Amazon, Bytedance, Bloomberg)
1. Maximum Number of Occurrences of a Substring (Hubspot, Salesforce)
1. Longest Substring with At Most Two Distinct Characters (Ebay, Google, Tiktok)
1. Longest Substring with At Most K Distinct Characters (Microsoft)
1. Subarray Product Less Than K (Paypal, Oracle, Salesforce)
1. Smallest Range Covering Elements from K Lists (Lyft, Databricks)
1. Number of Subarrays of Size K and Average Greater than or Equal to Threshold (LinkedIn)

Home on Ruoke Zhang

Text-to-Image System Design

Pinterest Canvas

Text-To-Image

1. text-to-image

2. multimodal image editing tasks

Visual Search

dataset

build dataset

Offline metrics

Mean reciprocal rank (MRR)

Deep Learning

Neural Network

terminology

AI Models

CLIP

architecture

image encoder

VisionTransformer

text encoder

embedding space

Computer Vision

ViT

Funnel ViT

Object Detection

Single Object

R-CNN

Machine Learning Basics

supervised learning

classification

Logistic Regression

PyTorch

Module

conv

Linear

Distributed Systems

RPC Basics

Semantics

File Systems and I/O Models

File Systems

Inodes

Java RPC: Transport, Discovery, and Reliability

RPC and HTTP

Serialization

Kryo

About

About Me

Contact

Bit Manipulation

Common Operations

Brain Teasers

2749. Minimum Operations to Make the Integer Zero

Math

Prime Numbers

Sieve of Eratosthenes

204. Count Primes

String

KMP

28. Find the Index of the First Occurrence in a String

Network

TCP Retransmission

TCP Flow Control

Slow Start

Greedy

300. Longest Increasing Subsequence

659. Split Array into Consecutive Subsequences

MySQL Logs

redo log

Physical Storage Characteristics

Write-Ahead Logging (WAL):

Redis Interview Notes

Sorted Sets (ZSET)

Compact Encoding

Common MySQL Interview Questions

Storage Engine Comparison

What is Page Split

Java Reflection & Dynamic Proxy

Reflection

API

Getting a Class Object

Sorted Sets (`ZSET`)