As we've discussed previously, over time, a large number of HFiles gets created for each region. Each additional HFile that is created increases the read latency, as it's one more disk seek to query whether there is data for a given key in that HFile. To avoid an unbounded increase in read latencies, HBase performs a background operation called a compaction.

The idea behind compactions is simple. A set of HFiles for a given region are consolidated into a single HFile. This has the effect of merging record fragments for a given key into a single record fragment. During this merge operation, multiple updates are squashed and delete markers/tombstones are purged, often resulting in a more compact record footprint. Since the individual HFiles are sorted, the compaction process itself is similar to a disk-based merge sort.

The only remaining question is what HFiles should be merged and when to do so. This is governed by the compaction policy, which can be configured by the operator. There are different compaction policies that one could set depending on query patterns and data characteristics for a given table. An in-depth discussion of these compaction policies is beyond the scope of this chapter.