Fallout: Distributed Systems Testing as a Service

11 Oct 2021  ·  Guy Bolton King, Sean McCarthy, Pushkala Pattabhiraman, Jake Luciani, Matt Fleming ·

All modern distributed systems list performance and scalability as their core strengths. Given that optimal performance requires carefully selecting configuration options, and typical cluster sizes can range anywhere from 2 to 300 nodes, it is rare for any two clusters to be exactly the same... Validating the behavior and performance of distributed systems in this large configuration space is challenging without automation that stretches across the software stack. In this paper we present Fallout, an open-source distributed systems testing service that automatically provisions and configures distributed systems and clients, supports running a variety of workloads and benchmarks, and generates performance reports based on collected metrics for visual analysis. We have been running the Fallout service internally at DataStax for over 5 years and have recently open sourced it to support our work with Apache Cassandra, Pulsar, and other open source projects. We describe the architecture of Fallout along with the evolution of its design and the lessons we learned operating this service in a dynamic environment where teams work on different products and favor different benchmarking tools. read more

PDF Abstract
No code implementations yet. Submit your code now

Categories


Databases Distributed, Parallel, and Cluster Computing

Datasets


  Add Datasets introduced or used in this paper