What is a distributed system?

Distributed systems are just that - systems whose components are distributed in space. This seemingly trivial quality has important implications. It means that components of distributed systems do not share a global clock. There is, in fact, no such thing as a global clock - clocks tick on their own. There is no universal time to keep them “in sync”.

Because components in distributed systems don’t share a clock, ordering events becomes a tricky problem. You can’t simply use timestamps - just because the timestamp of event a on computer A is numerically less than the timestamp of event b on computer B, a didn’t necessarily happen before b, because the clocks that provide the timestamps drift.

The inability to order events based on timestamps has huge implications. When two events should result in the modification of the same data, and you can’t tell which one happened first, how do you know what the value of the data should be? There are protocols you can use to implement varying levels of event ordering, but they each come with their costs in runtime performance and complexity.

Another problem you must confront in distributed systems is that communication relies on inherently non-instantaneous and error-prone networks. This, too, has huge implications. If you can’t trust your means of communication, how do you know if you just haven’t received a message yet, or if it was never sent?

There aren’t any tricks here - you have to work within the theoretical bounds implied by the fact that distributed systems are built in a world where clocks aren’t shared and communication is slow and faulty. Lamport clocks, vector clocks, consistency models, distributed locks, etc. are the beautiful fruits of solutions that have grown from the gnarly branch of software engineering that concerns distributed systems. As application engineers working in these systems, it is our privilege to get to work with these elegant solutions, and our responsibility to learn them in order to build sound systems that meet our business goals.