These four scenarios outline diverse approaches to SAML SP testing, each tailored to its respective context and purpose and requiring a different type of deployment.
Self-testing by SP for production readiness
Fully internal.
Deploy a test ISP and configure the tested SP for it.
Testing of SP deployment by FedOps during onboarding
Options
- Initiated upon SP's request
- Potentially automated (the SP has to register anyway)
It probably needs to be integrated into the federation's policy and operational guidelines. However, it can be easily communicated among other requirements after the SP requests onboarding.
Periodic testing of SP deployments by FedOps
Options
- Triggered by SPs themselves, with each SP required to invoke it in regular intervals within policy-defined periods.
- Or it could be automatically invoked by FedOps in line with predefined rules.
Must be aligned with the federation's policy and operational A part of the federation's policy and operational rules.
Client institution testing for compliance
Conducted by a client institution for contracted services, possibly as part of its internal compliance reviews (e.g., GDPR audits, ISO 27001 security controls).
How is the practical arrangement of the test to be coordinated between the client institution and the SP?
Practical differences and requirements in comparison to self-testing:
- This form of testing may involve specific compliance criteria dictated by the client institution.
- It often integrates with broader compliance assessments, introducing additional requirements.
- The use of the test by the client institution may necessitate distinct procedures and reporting.
- Could it be done without direct approval or involvement of the SP?
It probably needs to be included in the SLA.