End to End Acceptance Testing with Hydra Platform

Introduction

Introduction by James about Dan Shearer, a key figure in end-to-end acceptance testing with the Hydra platform.

Dan's Background:

  • Joined Adaptive in 2019 for the AerisX project, bringing extensive experience in the tech industry.

  • Worked closely with a team of ex-LMAX staff to successfully deploy a new cryptocurrency exchange in just 9 months, demonstrating leadership and technical skills.

Overview of Hydra Platform

Purpose of Hydra Platform:

  • The Hydra Platform is designed to simplify the application development process by minimizing the need for additional components necessary for efficient application delivery, which accelerates time-to-market for new features.

Key Components:

  • Execution Engine: Manages the execution of trading algorithms and processes, ensuring reliability and performance.

  • Trading Services: Provides a robust set of APIs for order management, market data, and trade execution.

  • Web UI: User-friendly interface for traders and users to interact with the system.

  • FIX Gateways: Facilitate the integration with external trading systems and brokers for seamless order routing.

  • High-Performance Messaging System (Aero): Connects all components efficiently, enabling fast communication and data transfer between services.

Features of Hydra:

  • Persistence and Fault Tolerance: Ensures that data is reliably stored and can withstand unexpected failures, critical for financial applications.

  • Disaster Recovery Options: Provides mechanisms to recover from failures, ensuring business continuity.

  • Focus on Testing Quality: Utilizes Cucumber, a powerful tool for building end-to-end acceptance tests, to ensure high-quality deliverables.

Understanding End-to-End Acceptance Tests

Definition of End-to-End Tests:

  • These tests evaluate the complete flow of an application, reflecting actual business requirements rather than just technical implementation details, ensuring the software meets end-user needs.

Comparison with Component Testing:

  • While component tests focus on the functionality of individual parts, end-to-end tests examine how these components work together, ultimately validating business outcomes over mere implementation specifics.

Example in Context:

  • For instance, after an account is created, verifying that it can be queried serves as a reflection of a key business requirement, showcasing the system's overall integrity.

Testing Strategies and Their Trade-offs

Benefits of End-to-End Testing:

  • Validates interactions among multiple components, ensuring that they function correctly together in real-world scenarios.

Downsides of Acceptance Tests:

  • Can be slow and complex, often requiring significant scaffolding to set up.

  • May not be practical for certain non-functional requirements or complex scenarios due to their nature.

  • Tests can be prone to intermittent failures, which can lead to frustration and inefficiencies in the testing process.

Integration of Diverse Testing Types:

  • It is vital to utilize various testing methodologies based on specific project requirements, such as Test-Driven Development (TDD) coupled with unit tests for comprehensive coverage and reliability.

Cucumber for Expressing Requirements

Cucumber's Role:

  • Cucumber serves as a framework that allows teams to articulate business requirements in a format that can be executed as tests, promoting collaboration between technical and non-technical stakeholders.

  • It emphasizes the creation of readable tests that can be understood by all team members, enhancing communication.

Example Case: Echo Service

  • Illustrates the writing of tests in a Gherkin format, which aids in understanding component interactions and facilitates easy validations of outcomes using straightforward assertions.

Structured Testing Steps:

  • Encourages improving tests through practical steps to manage and control application components effectively while specifying what needs to be tested.

  • Utilizing named sessions offers clarity in scenarios involving multiple users, streamlining the testing process.

  • Expressing data inputs in tables enhances organization and management of test data, simplifies modifications, and boosts reusability.

Executing and Running Tests in Hydra

Execution Models:

  • Embedded Mode: Runs production code within the same Java Virtual Machine (JVM) as tests, providing fast feedback and rapid iteration.

  • Existing Mode: A testing approach against a live system already running, which mimics real-world conditions but may result in slower feedback.

Real vs. Direct Modes Details:

  • Clarifies differences in testing business logic, network interactions, and wiring processes, adding a layer of realism to testing scenarios.

Strategies for Optimizing Tests:

  • Advocates running tests in direct mode for quicker feedback while later validating in existing mode for more accurate, real-world results.

Scaffolding and Utilities in Testing

Utilities and Scaffolding Provided by Hydra:

  • Offers a comprehensive framework for creating and managing test cases, promoting code reusability and efficiency.

  • Step Definitions: These are tied directly to Java code, allowing for seamless implementation and verification of test steps.

Deployment Management:

  • Facilitates identifying, starting, and stopping application components during tests, making management straightforward and manageable.

Using Proxies and Drivers:

  • Enable effective interactions with various application components during testing, ensuring that all flows can be effectively validated.

Matching Messages and Assertions

Framework for Message Matching:

  • Tools such as expector and recorder are used for asserting message integrity and response accuracy, which is crucial in a trading environment.

Data Table Assertions:

  • Incorporates concise and structured responses; complex data structures are handled through hierarchical approaches to maintain clarity and functionality.

Regex Matching:

  • Leverages regular expressions to capture patterns and validate responses rather than seeking exact values, adding flexibility to the testing process.

Handling Test Failures

Debugging Failed Tests:

  • Emphasizes the importance of reviewing failure messages, application logs, and sequence diagrams to trace issues back to their source effectively.

Color-coded Assertions:

  • Utilizes color coding to help quickly identify differences in data tables, simplifying the troubleshooting process.

Utilizing Full Observations:

  • Provides comprehensive insights into failures, allowing for a holistic view of test outcomes and their contexts.

Sequence Diagram Visuals:

  • Generates flow diagrams of interactions, which aid in diagnosing failures by visualizing the sequence of events.

Conclusion

  • The presentation underscores the importance of practical acceptance testing and encourages developers and testers to fully utilize the Hydra framework and its powerful utilities to naturally integrate robust Cucumber tests into their development workflow, aiming for high-quality software delivery.