Memory layout: the same algorithm, ten times slower, is usually cache

One algorithm written two ways can differ tenfold. The cause is rarely instruction count; it is the memory access pattern. The CPU pulls 64 bytes (one cache line) from main memory, so using only 8 of them wastes the rest of the bandwidth.

The value of sequential access

// slow: outer loop over columns
let sum = 0;
for (let c = 0; c < N; c++)
  for (let r = 0; r < N; r++)
    sum += matrix[r][c];

A 2D array is stored row by row. The version above jumps a whole row each step, hitting a fresh cache line almost every time. Swap the loops:

// fast: outer loop over rows
for (let r = 0; r < N; r++)
  for (let c = 0; c < N; c++)
    sum += matrix[r][c];

The same number of additions, but the sequential version serves 8 elements per cache line (for 8-byte elements). At large N the gap reaches 5 to 10 times.

Size of the data structure

// 32 bytes per element
type Node = { value: number; a: number; b: number; c: number };

// 8 bytes per element
const values = new Float64Array(n);

A million numbers in a typed array costs 8 MB; an array of objects costs 32 MB or more and chases a million pointers while iterating. Removing indirection is itself an optimization.

AoS versus SoA

Splitting fields into parallel arrays (SoA) usually beats one array of objects (AoS):

Layout	Suits
AoS	you read all fields of one object at a time
SoA	you read few fields per pass, or vectorize

Summing a field with SoA never pulls unused fields into cache.

When not to bother

• Data small enough to stay in cache (a few KB)
• Access is not hot (a few dozen rows per request)
• The bottleneck is the network or the database

Measure first. Claiming “cache unfriendly” without a profile is guesswork. In real performance work, memory layout often pays less than removing one network round trip.

Modern CPUs do arithmetic instantly and wait on memory forever. The first-order optimization question is usually how data is arranged, not how many steps the algorithm takes.