Why it matters
DataNode design decisions affect the four qualities operators care about most: throughput, cost per terabyte, mean time between failures, and time to recover from a failed disk. A DataNode that has too few disks starves the cluster of parallelism; too many, and a single controller becomes a bottleneck. Networks that are too slow throttle write pipelines; networks that are too fast just cost money. Understanding these trade-offs quantitatively is what separates a competent capacity planner from someone who over-orders hardware.
DataNode behavior also drives observable client latency. A single slow disk on one DataNode can drag the p99 of every client read that touches that node. Detecting and evicting a slow node is a routine operational task that requires knowing how DataNode internal scanning and health reporting work.
The architecture
Each DataNode runs a single Java process that manages multiple local storage directories, one per physical disk. Blocks are stored as regular files on the local filesystem, named with the block ID and a version number, and each block file has a companion metadata file holding a CRC checksum. The DataNode exposes a native block transfer protocol over TCP, distinct from the general HDFS RPC that the NameNode uses.
The DataNode sends a heartbeat to the NameNode every three seconds. This heartbeat is small: it tells the NameNode which storage volumes are healthy, current disk utilization, and pending replication work. Roughly once an hour the DataNode also sends a block report, which is a full list of every block it holds and which volume each block lives on. The NameNode uses this block report to reconcile its in-memory block map against actual storage. Deviation between the two triggers replication or deletion decisions.
How it works end to end
When a client writes a block, the write goes through the DataNode's block server. The client opens a TCP socket, tells the DataNode which block it is writing, and streams bytes. The DataNode splits the incoming stream into packets, writes each packet to its local disk, computes a CRC per packet, and forwards the packet to the next DataNode in the pipeline. This forwarding is what makes replication cheap: bytes cross the network exactly once per replica, not once per client-to-replica pair.
Reads are simpler. The client sends a block read request, the DataNode opens the block file from local disk, and streams it back over the socket with checksum verification on each packet. The block file lives in the OS page cache after the first read, so subsequent reads for hot blocks come straight from RAM without a disk seek.
The background block scanner walks every block on every disk over a rolling three-week window, reading each block and comparing its checksum. If it finds corruption, it reports the block as bad to the NameNode, which schedules replication from another replica and then deletes the corrupt copy. This is why HDFS can tolerate slow disk decay: the scanner catches bit rot before it becomes data loss.