<Incomplete, but the existing information is correct>
Children Display all true
| all | true |
|---|
...
Environments
- Production - domain is seamlessaccess.org
- Beta & staging - domain is thiss.io
...
The agreegated picture is here Seamless Access Deployment Architecture
Production
We have four sites in production currently.
- SUNET's own infrastructure in Stockholm
- Safespring's infrastructure in Stockholm
- Amazon Web Service in Frankfurt, Germany
- Amazon Web Service in North California, USA
We have one Load balaner, two thiss-js servers, two MDQ servers and one Medata Agreegator & Publisher server per site. They are all virtual machines managed by SUNET but has both geographic and network redundancy.
The relationship between these servers in combination with the services in Fastly are described in below diagrams. More details follow for each of these components further in this documentation.
service.seamlessaccess.org
...
| draw.io Diagram | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Beta
The number of servers and sites in Beta enviornment are limited but has the same relationship between them as Production.
use.thiss.io
| draw.io Diagram | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
- Write access to the ops repostiory https://github.com/TheIdentitySelector/thiss-ops/tree/master/global
https://wiki.geant.org/spaces/gn43wp5/pages/1228374096/Seamless+Access+Component+Description+TroubleshootingKnowledge of SUNET's puppet & cosmos structure https://wiki.sunet.se/display/sunetops/Cosmos
...
Fastly is CDN (content delivery network) provider. We use CDN to provide greater rechability accross the world, by take advantage of their cache nodes spread all over the world.
Services that are hosted in Fastly are
...
The servers with the name meta.*seamlessaccess.org run PyFF (https://pyff.io) in production environment. In Beta & Staging they are named a-*.thiss.io.
PyFF is short for python Federation Feeder - is a simple SAML metadata aggregator
In this environment, PyFF aggregates metadata from 3 federations - SWAMID, EduGAIN, InCommon & OpenAthens and publish them under /var/www/html/ using the script /usr/local/sbin/run-pyff running as a cronjob.
...
The servers also runs Apache in a docker container service called sunet-md_publisher to expose and publish the metadata JSON files on port 443 which are accisible only by the servers running MDQ (md-*.seamlessaccess.org) belonging to the same site.
Mointoring
We monitor ages of all the metadata files in https://monitor.seamlessaccess.org/nagios3/. They are
...
- After commiting and bump-taging the changes, run cosmos in the concerned servers, better to do it one at a time & check that the service is working.
- If PyFF is upgraded, run the aforementioned cronjob for PyFF to see that it doesn't show any error.
- You have to restart
sunet-md_publisherif you have upgraded the metdata publishing service. - Check https://monitor.seamlessaccess.org/nagios3/ for any alarms.
- The MDQ servers (mdwith the name m
d-*.seamlessseamlessaccess.org)should should be able to fetch the metadata from the Aggregator & Publisher servers. Make sure it is all 'green' for those servers too. - You can log in to the MDQ servers and run
/usr/local/bin/get_metadata.shand see that they are able to fetch metadata files without any issues. - As a last & final check, visit any SP for example wiki.sunet.se and see that it is possible to login using SA discovery service or check the login using https://demo.seamlessaccess.org/ for production upgrades and https://demo.beta.seamlessaccess.org for Beta upgrades.
...
Descripton & Troubleshooting
The servers with the name md-.*seamlessaccess.org run thiss-mdq in production environment. In Beta & Staging they are named md-*.thiss.io.
The code for thiss-mdq lives here https://github.com/TheIdentitySelector/thiss-mdq.
The aggregator servers running PyFF expose their port 443 to MDQ servers. They run a cronjob that checks to see if there are changes in the metadata JSON files in the Aggregator servers, if there is, it updates the file in the docker container running thiss-mdq and restart it. The script also does a pre-check to see if the fetched metadata files are empty or not, if empty, it won't restart and risk exposing empty metadatas.
| Code Block |
|---|
# Puppet Name: thiss__mdq_prod_fetch_metadata
*/5 * * * * /usr/local/bin/scriptherder --mode wrap --syslog --name thiss__mdq_prod_fetch_metadata -- /usr/local/bin/get_metadata.sh |
The MDQ is a REST-like API for requesting and receiving arbitrary metadata. It exposes the metadata in the URL https://md.seamlessaccess.org/entities. The URL for beta is https://md.thiss.io/entities.
The servers run the MDQ service on port 80 which is open to the HAproxy load balancers with names md.*.seamlessaccess.org belonging to the same site.
Mointoring
We monitor the date when the metadata JSON files are last modified in both monitor.seamlessaccess.org and nagiosxi.nordu.net. The check warns if the metadata is 2 days old, it becomes critical if it is 2 days old.
We have this check on MDQ servers, HAproxy Load balancer server and on the top domain which is https://md.seamlessaccess.org.
A simple check on the URLs on each level will show information about the metadata. The one on MDQ server level can only be run locally from that server or from the HAproxy server belonging to the same site.
| Code Block |
|---|
➜ ~ curl https://md.seamlessaccess.org
{"version":"1.5.8","start_time":"2025-11-22T00:35:20.300Z","metadata":{"last_modified":"2025-11-22T14:15:02.808Z","last_created":"2025-11-22T14:15:02.808Z","size":17502},"trust_metadata":{"last_modified":"2025-11-22T14:15:02.952Z","last_created":"2025-11-22T14:15:02.952Z","size":156}}%
➜ ~ curl -k https://md.ntx.sunet.eu.seamlessaccess.org
{"version":"1.5.8","start_time":"2025-11-22T00:35:03.460Z","metadata":{"last_modified":"2025-11-22T14:15:02.125Z","last_created":"2025-11-22T14:15:02.125Z","size":17502},"trust_metadata":{"last_modified":"2025-11-22T14:15:02.212Z","last_created":"2025-11-22T14:15:02.212Z","size":156}}%
: 14:19 root@md-1: ~ # curl -k http://localhost
{"version":"1.5.8","start_time":"2025-11-22T00:35:05.392Z","metadata":{"last_modified":"2025-11-22T14:15:03.281Z","last_created":"2025-11-22T14:15:03.281Z","size":17502},"trust_metadata":{"last_modified":"2025-11-22T14:15:03.409Z","last_created":"2025-11-22T14:15:03.409Z","size":156}} |
Upgrade
Thiss-js
Descripton & Troubleshooting
Mointoring
Upgrade
...
HAproxy Load Balancer
...
Descripton & Troubleshooting
...