HDFS

HDFS, the Hadoop Distributed File System, stores large files across machines. A NameNode manages names and file-to-block mappings; DataNodes hold block data. Clients obtain locations from the NameNode and transfer data directly with DataNodes.

Block size and redundancy are configurable. A short final block need not occupy a full logical block, and empty files need no data blocks. Replica placement across failure domains reduces correlated-loss risk.

Repair copies a surviving readable replica. If every readable copy is lost, metadata cannot reconstruct the contents. Replication also propagates intended changes; it is not a separate backup against mistaken deletion.

Monitor metadata health as well as storage capacity. High availability, namespace federation, and erasure coding solve different problems. Record the deployed policy and test recovery rather than treating three replicas as a universal guarantee.

See What Hadoop Left Behind for worked examples.

Reference: Apache documentation.