So, the time has come to retire our old Windows 2003 server. This box runs bookkeeping software (Exact Globe 2003, and BCS Delta), and is exclusively used internally by the accounting staff.
The first thought that came to mind: since we're running several Linux servers already on IPv6-only, would it be possible to run a Windows server exclusively on IPv6?
Last year my intern Joris Claassen has proved that it was possible to run a basic VMware VSphere cluster on IPv6 only, and that included a Windows box as well. So things look bright.
While Windows Server 2012 has been released recently, we can not use it (yet), because the latest version of Windows that our current VMware setup (ESXi 4.1) supports as guest OS is Server 2008 R2.
So I'll go for 2008 R2 and give it a shot.
Basics
IPv6-only VLAN
A new dedicated VLAN was created on our Cisco 3750X for IPv6-only systems, so that building, testing and configuring would not interfere with any production networks. This IPv6-only VLAN allows us to start from scratch using only two IPv6 Access Control Lists (ACLs):
interface Vlan9 description IPv6_only_Servers no ip address no ip proxy-arp ipv6 address 2001:610:148:BAD::1/64 ipv6 nd prefix 2001:610:148:BAD::/64 ipv6 traffic-filter ipv6_only_servers_out in ipv6 traffic-filter ipv6_only__servers_in out end
I started out with ACLs that disallow everything, then started configuring the server, monitor the log files for any dropped traffic, and then gradually opened up the necessary things.
Addressing
Obviously, the first thing to do is to uncheck IPv4 in the interface configuration . For IPv6 addressing I choose autoconfigured EUI64, no privacy extensions. The reason is that this is a server, and it will not be used for any web browsing activities. Static addresses also help configuring the (empty) IPv6 ACLs. The idea is that during configuration and testing the ACLs will be constructed, based on stuff that does not work. Eventually, when everything works, it might be an option to use privacy extensions.
Also, I disabled all tunnelling stuff (ISATAP, Teredo, etc). Combined script:
REM RFC 4941 privacy extensions (i.e. temporary address for outgoing connections) netsh interface ipv6 set privacy state=disabled store=active netsh interface ipv6 set privacy state=disabled store=persistent REM Don't use random identifiers. This will result in EUI64 based adddresses netsh interface ipv6 set global randomizeidentifiers=disabled store=active netsh interface ipv6 set global randomizeidentifiers=disabled store=persistent REM disable unused tunneling protocols netsh interface ipv6 6to4 set state disabled netsh interface ipv6 isatap set state disabled netsh interface ipv6 set teredo disabled
DNS
The DNS server addresses are statically assigned, and are picked from the SURFnet DNSSEC-validating resolvers.
The "Register this connection's addresses in DNS" option has been deselected, because this causes DNS registration requests to go out, which we do not want. Eventually things look like this:
NTP
At the time of writing, the default time server that is used by Windows 2008 R2 to sync its clock, time.windows.com, unfortunately is not (yet) reachable over IPv6.
But again SURFnet comes to the rescue, because several of their NTP servers are IPv6-enabled. I picked chime3.surfnet.nl, which, according to the web interface, appears to be a Meinberg NTP server. Windows digs it all-right:
RDP
To manage the server, Remote Desktop Protocol (RDP) is used, which supports IPv6 without any configuration.
Microsoft Update
According to Microsoft, Windows 8 will use IPv6, if available, to download Windows Updates. They will works with Content Delivery Networks (CDNs) to extend IPv6 support later. So at this time, for Windows Server 2008 R2, the update servers are accessible only through IPv4.
As a solution I set-up a dual-stack limiting HTTP proxy server, and added a number of regular expressions to allow Microsoft Updates to happen.
Details of the proxy and the Windows configuration process are described on a separate page.
As can be seen from the lists, several other URLs also need whitelisting, such as CRLs and OCSPs.
Windows Activation
Same story here. Activating Windows is done by several HTTP POSTs to URLs in the go.microsoft.com domain, which is not (yet) accessible through IPv6:
http://go.microsoft.com/fwlink/?LinkID=88342 | Processor Certificate URL |
http://go.microsoft.com/fwlink/?LinkID=88343 | Machine Certificate URL |
http://go.microsoft.com/fwlink/?LinkID=88345 | Use License URL |
http://go.microsoft.com/fwlink/?LinkID=88344 | Product Key Certificate URL |
Since the proxy server was already in place, I just needed to add this domain to the proxy whitelist, and then the Windows activation succeeded.
Secunia CSI
This tool monitors installed software for vulnerabilities. We have the corporate version CSI 6. This is a service that registers itself at Secunia to download patch lists, upload results, etc.
Unfortunately the host that is used for communication (agent.csi6.secunia.com
)is only reachable via IPv4.
But when configured to use the HTTP proxy everything works as expected and CSI is able to phone home again .
Access from IPv4 networks
Users that need to access this server from IPv4 networks (teleworking etc) use Cisco Secure Mobility Client to connect to our Cisco ASA, which gives out IPv6 addresses.
Exact Software
I phoned up Exact Software Netherlands to see if Exact Globe would support IPv6, but the help desk could not provide me with a definitive answer. So then, time for some testing.
Exact Globe consists of a network share containing an installer, and an SQL Server. Clients map the network drive and install the software on their computer.
File sharing
Windows 2008 R2 does SMB (File sharing) over IPv6-only without any problems:
Windows Internet Name Service (WINS) does not work with IPv6. But because this was a piece of junk anyway, nothing is lost here.
SQL Server
Once installed, the Exact software then initiates a connection to TCP port 1433 of an SQL Server. In our case this is the same server.
The old (Windows 2003) server runs SQL Server 2000, but starting from release 404 (July 2012), the Exact software does not support this any more. So I had to upgrade, preferably to the latest supported version, which is SQL Server 2012. Because such an upgrade needs an intermediate step, I decided it was OK to use SQL Server 2008 R2 for now.
This supports IPv6 just fine. All users in accounting run Windows 7, so the operating system itself shouldn't be a problem, but I was a bit wary of potential client side application issues. While testing I quickly found out that the Exact application was using ODBC, which did support IPv6 right away, so the application worked on IPv6 .
Delta software
This is proprietary software made by Dutch company BCS HRM en Salarisadministratie B.V. and is made up of a network share with files, and a database component.
The network share can do IPv6 as mentioned earlier.
Sybase SQL AnyWhere
The database is a Windows Service, based on a file called dbeng11.exe. It's not stated as such in the BCS documentation, but this is an instance of SQL AnyWhere Personal Server:
The BCS support desk had no clue what IPv6 was, so I did some research myself.
There is a nice and very clear PDF that explains SQL AnyWhere networking essentials. According to this document, SQL AnyWhere supports IPv6 from version 10, which date back to 2006. The docs for version 11 and 12 state is even more explicitly with some examples. These docs came in handy because they list all the connection options, which I needed.
Since we only have one instance of SQL Anywhere, which is running in a different network than the clients, I figured it didn't make much sense to properly implement the service discovery. Instead I hardcoded the server FQDN, which worked fine. After opening up TCP port 2638 the software worked
Crypto
I wasn't really sure how safe this SQL Server traffic was, but running Wireshark revealed plain text SQL queries on the wire . I think it's good practice to encrypt traffic containing personal data, even though it's inside our own network. SQL Server 2008 R2 supports SSL encryption, and because we get "free" SSL certificates through SURFnet (which is part of the TERENA Certificate Service), we should be good to go!
I created the cryptographic materials on one of our Ubuntu systems because those already have OpenSSL installed. All of this is also perfectly possible on Windows, but you do need some extra bits such as Microsoft Visual C++ 2008. I just didn't want to pollute my Windows system with extra software that is already installed elsewhere in our network.
openssl req -new -keyout server.key -out server.csr -subj /CN=hayek.terena.org/
I submitted the signing request to the SURFnet web site, and after jumping through the Domain Control Validation hoops, I got a signed certificate (cert-11988-hayek.terena.org.pem
) back, and the chain, which consists of 3 certificates concatenated in one file (chain-11988-hayek.terena.org.pem
).
Then I combined all certificates into one file, and created a PFX from it:
cp cert-11988-hayek.terena.org.pem all.pem cat chain-11988-hayek.terena.org.pem >> all.pem openssl pkcs12 -export -inkey server.key -in all.pem -out server.pfx
After copying the PFX file to the Windows server, I ran mmc and added the Certificates snap-in. When it asked for who to manage certificates, I selected the "Local System" account, because that's what the SQL Server was configured to run as.
Expand: Console Root -> Certificates (Local Computer) -> Personal.
Right-click -> All Tasks -> Import. Navigated to the pfx file and imported it, including all extended properties.
I kept "Mark this key as exportable" unchecked, as I already have the key materials in a different place. I figured that since it is not needed on this machine, we might as well disable it, so any future malicious export attempts will be impossible (or at least more difficult).
Configuring MS SQL Server to use the certificate
Run the SQL Server Configuration Manager, expand the SQL Server Network Configuration, and right-click Protocols for MSSSQLSERVER (or whatever your instance is called).
On the Certificate tab you should be able to see your certificate.
In my case nothing would show up
According to this Howto, the name of the certificate must be the fully qualified domain name (FQDN) of the computer. Since the TCS certificates we use can only contain a valid FQDN as the Subject's Common Name (CN), this had to be correct. I checked permissions and those seemed to be OK as well.
It turned out that the FQDN is called Full Computer Name in Windows, and indeed that was still set to a single label:
To change this I had to add our domain name at Primary DNS suffix of this computer:
I wasn't aware that Windows did this sort of checks, but I think it makes sense because it prevent you from inadvertently using a wrong certificate .
After fixing the FQDN the certificate showed up in the SQL Server Network Configuration.
Then I forced encryption:
checked again with Wireshark and indeed no more plain text queries
RDP Crypto
Since we now have a nice server certificate in the Windows Certificate Store, I figured I might as well use it to secure more services that run off this machine. The most obvious service is of course the channel by which the machine is managed: Remote Desktop Protocol (RDP). Based on the docs, RDP does support SSL (TLS1.0). The installed certificate can be configured with the Remote Desktop Session Host Configuration. While at it, I also selected to only use SSL (TLS 1.0), and High Encryption level:
Using Wireshark it is again easy to spot that the session is encrypted.
Previously I would always get this warning upon connecting to a Windows server:
After installing a real certificate, this warning isn't shown any more, and a nice little lock is shown on the RDP status bar, which shows the certificate information if clicked:
IPv6 service monitoring with Nagios
All of this stuff is very nice, but what good is a service when it is not being monitored?
We already run a Nagios-3.2 instance on an Ubuntu system in a remote data centre, and my goal is to monitor as much services on my new Windows host as possible.
Host alive
The check-host-alive test is a basic test from the nagios-plugins-basic package, and is based on check_ping
, which can do IPv4 and IPv6, and defaults to IPv4. It is defined in /etc/nagios-plugins/config/ping.cfg
. Interestingly, there are checks to deal with dual stack monitoring:
#### # use these checks, if you want to test IPv4 connectivity on IPv6 enabled systems #### # 'check-host-alive_4' command definition define command{ command_name check-host-alive_4 command_line /usr/lib/nagios/plugins/check_ping -H '$HOSTADDRESS$' -w 5000,100% -c 5000,100% -p 1 -4 }
Since our host's FQDN only has a AAAA DNS records, the default check is OK.
Disk usage
This can be done with the standard check_disk_smb_host_user
, because that relies on the smbclient
binary, which is part from the samba-3.4.7 package, which supports IPv6.
Microsoft SQL Server
There is no Nagios plugins in any Ubuntu package that can check the status of SQL Server, but Nagios Exchange does list a number of them. They are written in various languages (Perl, Python, PHP), and they all rely on FreeTDS, a set of libraries for Unix and Linux that allows programs to natively talk to Microsoft SQL Server and Sybase databases. The bad part is that this library does not support IPv6, hence none of the plugins work with my server .
After asking around on the FreeTDS mailing list, a guy called Peter Deacon wrote a patch that adds IPv6. I did some testing with Perl and after forcefully installing a newer version of DBD::Sybase I was able to successfully connect to my SQL Server .
With the FreeTDS library now supporting IPv6 connections, I went to the Nagios Exchange and picked check_mssql_health, which can do rather elaborate checks on all sorts of SQL Server properties.
SSL certificate expiration date
Now that the server has an SSL certificate, it should be monitored so that the admins get a warning when it is about to expire. The way I usually do this is by using a Nagios plugin to monitor a service that uses the certificate, and then pick the certificate expiration option of the plugin. For example the check_smtp plugin has this option:
-D, --certificate=INTEGER Minimum number of days a certificate has to be valid.
The common way to test this sort of stuff is with OpenSSL's s_client, but unfortunately that does not support IPv6. Off then to GnuTLS.
Connecting to SQL Server on port 1433 yielded nothing, the handshake wasn't establisched, so that's a dead end for now. Connecting to RDP on port 3389 however happens with a nice TLS 1.0 handshake
bofh@nagios:~$ gnutls-cli hayek.terena.org -p 3389 --x509cafile /etc/ssl/certs/ca-certificates.crt Processed 142 CA certificate(s). Resolving 'hayek.terena.org'... Connecting to '2001:610:148:bad:250:56ff:fe86:9:3389'... - Certificate type: X.509 - Got a certificate list of 3 certificates. - Certificate[0] info: - subject `C=NL,O=TERENA,OU=CFO,CN=hayek.terena.org', issuer `C=NL,O=TERENA,CN=TERENA SSL CA', RSA key 2048 bits, signed using RSA-SHA, activated `2013-01-23 00:00:00 UTC', expires `2016-01-23 23:59:59 UTC', SHA-1 fingerprint `513523a823b8d5c15c30a80f2772e58d826605ba' - Certificate[1] info: - subject `C=NL,O=TERENA,CN=TERENA SSL CA', issuer `C=US,ST=UT,L=Salt Lake City,O=The USERTRUST Network,OU=http://www.usertrust.com,CN=UTN-USERFirst-Hardware', RSA key 2048 bits, signed using RSA-SHA, activated `2009-05-18 00:00:00 UTC', expires `2020-05-30 10:48:38 UTC', SHA-1 fingerprint `3a881764472b6441ddb3afdd47c6b8b76ee7ba1d' - Certificate[2] info: - subject `C=US,ST=UT,L=Salt Lake City,O=The USERTRUST Network,OU=http://www.usertrust.com,CN=UTN-USERFirst-Hardware', issuer `C=SE,O=AddTrust AB,OU=AddTrust External TTP Network,CN=AddTrust External CA Root', RSA key 2048 bits, signed using RSA-SHA, activated `2005-06-07 08:09:10 UTC', expires `2020-05-30 10:48:38 UTC', SHA-1 fingerprint `3d4b2a4c64317143f50258d7e6fd7d3c021a529e' - The hostname in the certificate matches 'hayek.terena.org'. - Peer's certificate is trusted - Version: TLS1.0 - Key Exchange: RSA - Cipher: AES-128-CBC - MAC: SHA1 - Compression: NULL - Handshake was completed
This indicates that it possible to monitor the certificate the same way as with HTTPS web sites. I tested this and it works
Concluding, this is what the monitoring looks like in the end: