...
- Production - domain is seamlessaccess.org
- Beta - domain is thiss.io
- staging Staging - domain is thiss.io
SeamlessAccess Services
...
By setting the version parameter in thiss-ops/global/overlay/etc/puppet/cosmos-rules.yaml or in the thiss-ops/global/overlay/etc/puppet/modules/thiss/manifests/demo_sp.pp.
General Troubleshooting
Almost all services run in docker containers. They are addes as systemd units. The names start with sunet-*.
Log
Server
| Name | Location | Env |
|---|---|---|
| log.seamlessaccess.org | STO1v2, Safespring | Prod |
Descripton
The servers runs a syslog application to collect logs from service.seamlessaccess.org. The server is specifically allowed in Fastly configuration, you can check that under Logging for the current version of service.seamlessacces.org configuration running in Fastly.
We have added Enrique Perez's SSH key and IP address so he can fetch the logs from under /var/log with the names sa.log.
This is how it looks in /root/.ssh/authorized_keys of the server.
| Code Block |
|---|
command="/usr/bin/rrsync -ro /var/log/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-ed25519 AAAAB3NzaC1yc2EAAAADAQABAAABAQDWOTGSoPh/+uNglvrLifb4jVhDLzGnAQlH3jagVnWFQKVieUNB2vlhrTtW/89+9uRUtjICa1gevGxICkavgaP8MIvOrgksgR+j+CakbwKe1gGmC5AqFb1kmbUOpeUrGDHYbWp46fOc0zTBxTqT1u93LAw/ZUHUMB3ETnmScrbvxC3JwA0qsU7bw73QCLM24epy8dvstFTLcNPcPC2TOCh86IkZpvJj38Hy5uqanWN6KceOtQBtOORJE6rAsBTpmhiVtE/AsvkEWKNk1g5uArULK/Dd6K7fMxkr0rv+YT9qot/z0xUqHe5RDn3E5w3ojV8x47/0V9l3eh9jrEf3l6u9 -var-log--command_key |
There is a configuration in logrotate so the sa.log(s) are rotated for 30 days and will be removed afterwards.
Mointoring & Troubleshooting
- Check
/var/log/syslogif there's any issue with access for Enrique or any issue with rsyslog functionality. - Take help of applicable puppet manifests to understand the configuration and troubleshoot further.
- Check in Fastly if there's any warning message in the service configuration for Logging for
service.seamlessaccess.org.
General Troubleshooting
Almost all services run in docker containers. They are addes as systemd units. The names start with sunet-*.
journalctl -fu <service name of the system unit>- Check
/var/log/syslogfor older logs docker logs -f <docker container name>servicejournalctl -fu <service name of the system unit>- Check
/var/log/syslogfor older logs docker logs -f <docker container name>service <service name of the system unit> restart
...
| Server type | Rules |
|---|---|
| All | SSH via SUNET's designated jump hosts |
| All | NRPE to monitor.seamlessaccess.org & nagiosxi.nordu.net |
| All | Egress/ougoing packets from all ports |
| HAproxy Load Balancer for thiss-js | HTTPS to internet TCP 8404 (HAproxy stats port) to vpn1.sunet.se & monitor.seamlessaccess.org |
| HAproxy Load Balancer for thiss-mdq | HTTPS to internet TCP 8404 (HAproxy stats port) to vpn1.sunet.se & monitor.seamlessaccess.org |
| thiss-js | HTTP to HAproxy Load Balancer for |
| thiss-mdq for Production & Beta | HTTP to HAproxy Load Balancer for |
| thiss-mdq for staging | HTTPS and HTTP to SUNET Load Balancers |
| Aggregator & publishers for Production & Staging | HTTPS to |
| Aggregator & publishers for Beta | HTTPS to |
| Monitor | HTTPS to vpn1.sunet.se HTTP to internet (for ACME challenges to renew Let's Encrypt certificate) |
| Demo Application | HTTPS to internet |
| Log | SSH access to Enrique Perez Arnaud & TCP 514 (syslog) to internet |
Staging Metadata Service
...