This activity attempts to create a fully fledged dynamic test environment for new and existing federated software products to enable quick and easy integration testing.

It is currently very hard to test new releases of IdP and SP software against other IdP/SP products. For example, the fix for the latest SATOSA/PySAML security vulnerability turned out to break logins from a number of IdP products, which was not discovered until the patches were tested against real-life IdPs. One solution to this problem would be a dynamic test platform, which could automatically test 'all' combinations of IdPs and SPs.

The aim of the activity is to design such a test environment, identify the necessary technologies and implement a working prototype.

One way to solve this problem is to create an automated test environment. To make this successful a generic platform is needed on which different SP and IdP products can be plugged in. The platform would need to run a CI-like matrix testing (regularly or whenever a new product in added). A number of standard products (SSP, SATOSA, ADFS, etc) would need to be configured for automatic testing, and the platform would need to allow local development teams (for example SSP developers, or local federations which develop their own federation production) to add their own products and versions to the platform.

This activity attempts to design such a test environment. A suitable architecture for an IdP/SP continuous integration platform is to be created. This involves creating use cases and identifying technologies that support them. An iterative approach, in which use cases are successively implemented and products are integrated one after another, is intended to quickly create a functional platform. If the CI approach is successful, ready-to-use software packages can be provided using container technologies such as Docker, to deploy the entire software locally by NRENs or other parties.

The envisioned integration platform would enable developers across the GÉANT community to not only preempt compatibility issues with new software releases, but also flush out incompatibilities between existing IdPs/SPs that are already in eduGAIN. As a result, the quality of the software in federations and eduGAIN as well as the time to market of new products can be significantly improved.

• The automated combination of all possible tools may prove to be too complex
• We do not know exactly how well continious integration works in this scenario

The activity does not affect data protection or privacy.

• Testbed requirements and use cases are analysed and documented
• A design and architecture for the test environment was created
• A prototype of test environment was created and tested with various software products
• Scripts and guidelines for automated deployment are available

• The right strategy needs to be identified depending on the design of the testbed
• To ensure long-term support of the supported products, the development teams responsible should be involved to maintain their own products.

• Documentation: Testbed Design
• Source Code: https://gitlab.geant.org/matteo/idp-sp_testbed 
• Sample screencast: Testbed - deployment demo