Hash and checksum: why CRC32 and MD5 have to be written by hand

Two algorithms you have to write yourself

crypto.subtle.digest covers SHA-1/256/384/512, and it is fast enough that you never think about bit twiddling. It does not cover CRC32 or MD5, which are exactly the two most common in download verification, archive formats and shard identifiers.

So CRC32 uses a table: precompute 256 entries, then XOR and look up byte by byte. That is a few dozen lines. MD5 follows RFC 1321, and the awkward part is endianness. Message padding and the final output are little-endian while the internal state is big-endian, and getting it backwards produces no error at all, only a plausible-looking wrong hex string.

Files should not be read whole

Reading an entire file as a string works for small inputs and is a disaster for a few hundred megabytes. The right shape is chunked: take slices of an ArrayBuffer, feed them into the same hash instance, and read the digest once at the end.

The SHA family supports that pattern natively, so the hand-written CRC32 and MD5 have to be written to update incrementally as well, not as one-shot functions. That is a decision to make when the implementation is designed, not afterwards.

Paste the expected value and let the tool compare

The real scenario is: a vendor publishes a checksum, and you want to confirm the file arrived intact. Comparing 64 hex characters by eye is tedious and unreliable, since case, spacing and a missing digit all slip through.

So there is an input for the expected value. Paste it and the tool tells you which algorithm produced it, or that none match. That turns verification from a visual chore into a statement of fact.

One necessary warning

MD5 and SHA-1 are no longer suitable for security purposes; collisions are far too cheap. They are here for integrity checking, confirming that a transfer did not corrupt, not for signatures or passwords. If the threat model is an active attacker, use SHA-256 or stronger.

Covered by the self-test

A hand-written crypto implementation needs known vectors, otherwise emitting 32 hex characters and being correct are two different statements. The test suite checks standard vectors one by one, for example the MD5 of “abc” is 900150983cd24fb0d6963f7d28e17f72, so a refactor cannot quietly break it.