Managing the crowd: Designing distributed systems
Some people, when confronted with a scalability problem, think "I know, I'll deploy a cluster of servers." Now they have multiple problems.
Websites are becoming both increasingly transactional and increasingly distributed. Many different servers and systems can be at play in powering today's content-, community- and commerce-heavy websites.
With this increased decentralization comes multiple problems that we cannot ignore. Networks have a tendency of being unreliable (especially in cloud infrastructures), the latency between components is non-trivial, servers can fail in multiple different ways, among other commonly overlooked problems.
All the components we are using in the web stack (from the low-level infrastructure all the way up to Drupal) attempt to deal with some of those problems at their level. Some succeed better than others.
This session will provide a framework for understanding the hard problems at play in distributed systems, so that the advantages and limits of each tool can be more easily understood.
We will attempt to bridge the gap between recent (and not-so-recent) research in distributed computing and the web community. We will talk about concurrency control, scalability, fault tolerance, consensus protocols, synchronization... without taboo, but always in a way that is understandable without a degree in Computer Science*.
*that the speaker doesn't have, anyway