...

https://wiki.sunet.se/pages/viewpage.action?pageId=83493119

Internal Components

General Troubleshooting

Almost all services from internal components run in docker containers. They are addes as systemd units. The names start with sunet-*.

• journalctl -fu <service name of the system unit>
• Check /var/log/syslog for older logs
• docker logs -f <docker container name>
• service <service name of the system unit> restart

For deeper troubleshooting knowledge of SUNET's puppet & cosmos structure is needed as mentioned in the Prerequisites section above.

The puppet manifests that deploy and manage the internal components are found here https://github.com/TheIdentitySelector/thiss-ops/tree/master/global. Those who have write acces to it are mentioned here https://wiki.sunet.se/pages/viewpage.action?pageId=83493119

Aggregator & Publisher

Servers

Descripton & Troubleshooting

...

Take help of the 'Description & Troubleshooting' section above to troubleshoot the alarms. Se also GeneralTroubleshooting. 

Upgrade

• Both PyFF and sunet-md_publisher are upgraded by chaging the versions in thiss-ops/global/overlay/etc/puppet/cosmos-rules.yaml. The puppet manifests for production, beta and staging are separate.

...

• After commiting and bump-taging the changes, run cosmos in the concerned servers, better to do it one at a time & check that the service is working.
• If PyFF is upgraded, run the aforementioned cronjob for PyFF to see that it doesn't show any error.
• You have to restart sunet-md_publisher if you have upgraded the metdata publishing service. See GeneralTroubleshooting
• Check https://monitor.seamlessaccess.org/nagios3/ for any alarms.
• The MDQ servers with the name md-*.seamlessaccess.org should be able to fetch the metadata from the Aggregator & Publisher servers. Make sure it is all 'green' for those servers too.
• You can log in to the MDQ servers and run /usr/local/bin/get_metadata.sh and see that they are able to fetch metadata files without any issues.
• As a last & final check, visit any SP for example wiki.sunet.se and see that it is possible to login using SA discovery service or check the login using https://demo.seamlessaccess.org/ for production upgrades and https://demo.beta.seamlessaccess.org for Beta upgrades.

MDQ

Servers

Descripton & Troubleshooting

...

We also have nagios checks on the accisibility of these web links on each level. Chek also GeneralTroubleshooting.

Upgrade

The process is described in below link along with verification for both production and beta environments.

...

Seamless Access Software Deployment Guide#Backend(md.thiss.io)

Thiss-js

Servers

Descripton & Troubleshooting

...

We also have nagios checks on the accisibility of these web links on each level. Chekc also GeneralTroubleshooting.

Upgrade

The process is described in below link along with verification for both production and beta environments.

...

Descripton & Troubleshooting

Mointoring

Upgrade

Use of SUNET INFRA cert

add details

SeamlessAccess SUNET INFRA cert update

Use of Fleetlock

General Troubleshooting

Almost all services run in docker containers. They are addes as systemd units. The names start with sunet-*.

• journalctl -fu <service name of the system unit>
• Check /var/log/syslog for older logs
• docker logs -f <docker container name>
• service <service name of the system unit> restart

For deeper troubleshooting knowledge of SUNET's puppet & cosmos structure is needed as mentioned in the Prerequisites section above.

The puppet manifests that deploy and manage the internal components are found here https://github.com/TheIdentitySelector/thiss-ops/tree/master/global. Those who have write acces to it are mentioned here https://wiki.sunet.se/pages/viewpage.action?pageId=83493119

Use of SUNET INFRA cert

add details

SeamlessAccess SUNET INFRA cert update

...

Firewall Restrictions

Access to Internal Components

...