Event-Driven End-to-End Testing For Containerized Applications

Abstract

With the complexity of emerging systems rapidly multiplying, it is important to evolve our testing infrastructures required to better understand how our distributed systems deal with scaling, failover and fatal tolerance. Compared to random testing the test of ”deep” failure paths requires different methods for deriving test cases and for running the test. This paper introduces Frisbee, a platform for the automated testing of distributed applications over Kubernetes. Frisbee leverages static and dynamic runtime instrumentation to spin-up the dependency stack and perform execution-driven testing actions, while automating the collection of performance metrics and the assertion of system’s behavior. This technique enables the controlled injection of realistic software faults while the target system executes, ensuring a predetermined faultload distribution throughout the experiment, regardless of the particular system or workload. Our evaluation demonstrates that Frisbee significantly enhances the precision and controllability of prior tools with only modest memory and performance overhead during fault-free execution.