Why does it work this way? Why is there a single WAL per RS? What is the point of recording the edits in a WAL rather than applying the updates in place?

The original motivation from a decade ago was that spinning disks were slow and SSDs were too expensive to be practical. In mechanical spinning disks, the seek costs were high, but if the disk head was already in the right position, writes could be performed with relatively low latency. In other words, the latency of an append operation is less than a random write. The lower latencies translate to higher throughput.

The LSM structure allows a sequence of random writes to be handled as append-only operations, as random writes are buffered in memory and then written out as a single batch.

The disk writes are only for persisting edits in the commit log itself. By having a single commit log for all regions in the RegionServer, writes for different keys are converted into a stream of append-only edits to a singular commit log.

Since HBase and other such LSM databases only needs a file system that supports append-only operations, it can be layered on top of HDFS, which provides only append-only operations. Hence, HBase and HDFS are a unique fit in terms of capabilities exposed (by HDFS) and capabilities required (by HBase).

Over the past decade, there's been a dramatic lowering in the cost and increase in the capacity of SSD devices. They are now cost-effective enough that these days they are the preferred storage medium in many enterprises, even for big data workloads.

SSDs don't have a mechanical disk head. In SSDs, the latency of sequential and random writes is the same. So, given this change in technology, is there any benefit in a database that converts random writes into append-only operations?

In SSDs, the reading and writing of data happens at page level, typically 512 bytes or more. So, if you need to write a 100-byte value into a page on disk, SSDs need to read 512 bytes of data, merge the 100 bytes into this 512-byte page, and then write the 512 bytes back to disk. So, a 100-byte write has become amplified into a write for 512 bytes. This is called write amplification. The lifetime of an SSD device is measured in terms of how many times a page can be rewritten, after which the error rates tend to go up.

An append-only database, such as HBase, has a write amplification of one (which means, each logical write from the database translates to exactly one physical write within the SSD). Hence, append-only databases put less stress on SSD devices, leading to longer device lifetimes.