The recent outages of AWS and the attacks on the DNS network have shown the need of distributing critical infrastructures across multiple cloud providers. This distribution is particularly important for centralized authentication services, which provide users and permissions for various services and organizational offices. Building on the last tutorial, that talked about connecting clients to cloud-based Samba 4 domain controllers, this article will explain how to extend the network by an additional Samba 4 based site server. I will lead you through the step by step process and explain the best practices of running a multi-server installation.

This guide can also be used to connect two on-premises Samba 4 installation or an on-premises installation with a cloud-based one, for example, Univention Corporate Server on Amazon.

Using a multi-server setup protects the network from the failure of a single data center and allows a continued operation even if a single server is unavailable. At the same time, using locations that are, from a network perspective, closer to your workstations, can speed up the login process and enable a more efficient usage of time.

At the same time, an incorrect setup can at best slow down the network. At the worst, it might not replicate the data correctly, and thus an outage of one server might cause an interruption of all systems.

Prerequisites

Server 1

This guide assumes that the first Samba 4 domain controller is already running without any issue.

Univention Corporate Server (UCS) provides a powerful, yet easy to use identity management solution that includes a preconfigured Samba 4. UCS’ unified user and rights management solution is based upon a domain concept that will allow you to skip some steps within this guide. If you are using this guide for a UCS-based system, the server 1 should be a UCS master or backup.

If you are planning on using a Debian or Ubuntu based system, the Samba Wiki has an excellent Getting Started Guide with all the steps needed to get the domain provisioned. The server or virtual machine will need to use a fixed IP for this guide to work flawlessly.

Server 2

We will also assume that you have set up the second server at the target location. If using UCS, it will be considerably easier to finish the installation once the VPN connection has been established. Also disable the automatic join, as we want to change some site settings.

If you are using Debian or Ubuntu, please install Samba4 from the package management system

$ sudo apt-get install samba

VPN Endpoints

For simplicity and security, this guide assumes that the VPN is running on two dedicated servers, thus reducing the load on the domain controllers. It, however, is possible to run OpenVPN on the domain controllers.

VPN Connection

Samba uses multiple ports and protocols to connect two or more servers, including LDAP, Kerberos, and DNS. Using a VPN allows reducing the number of ports and protocols exposed to the Internet to two, thus making securing the connection considerably easier. As previously, we will use OpenVPN to connect the two systems.

If not installed already, you can install it with the following command:

$ sudo apt-get install openvpn

Considering that in most cases port 1194 is used for client-server connections, this example will use 1195 for the connection, which consequently needs to be opened in the firewall.

On UCS based systems the configuration registry can be used to open the port

$ sudo ucr set security/packetfilter/udp/1195/all=ACCEPT
$ sudo service univention-firewall restart

On Debian and Ubuntu, you can manually add the port to your IP tables configuration

$ sudo iptables -A INPUT -p "udp"  --dport 1195 -j ACCEPT

Configuration

First, a secret key is needed to connect the two sites. OpenVPN can create it for you using the command 

$ sudo openvpn --genkey --secret /etc/openvpn/static.key

Both sides will need nearly identical configuration files saved in /etc/openvpn

## Topology and protocol settings
dev tun
proto udp
management /var/run/management-udp unix

## the shared secret for the connection
secret /etc/openvpn/static.key

## Encryption Cypher to use for the VPN
cipher AES-256-CBC

## Compression algorithm to use
comp-lzo

## The port on which the VPN Server should listen on
port 1195

## The address used internally by OpenVPN
ifconfig 10.255.255.10 10.255.255.11

## Route traffic to remote network
## The network should be the one, used by the remote server
route 10.200.10.0 255.255.255.0

## Additional server configuration
keepalive 10 120
persist-key
persist-tun

## Configure the logfile and the verbosity
verb 1  
mute 5
status /var/log/openvpn-status.log

For the second server, the route has to match the respectively other network and the addresses in ifconfig statement have to be switched. Additionally, the keyword remote has to be used to denote the endpoint. The full resulting config file thus looks like this:

## Topology and protocol settings
dev tun
proto udp
management /var/run/management-udp unix

## the shared secret for the connection
secret /etc/openvpn/static.key

## Encryption Cypher to use for the VPN
cipher AES-256-CBC

## Compression algorithm to use
comp-lzo

## The external DNS name or IP of the other VPN
remote vpnserver.univention.com 1195

## The address used internally by OpenVPN
ifconfig 10.255.255.11 10.255.255.10

## Route traffic to remote network
## The network should be the one, used by the remote server
route 10.200.10.0 255.255.255.0

## Additional server configuration
keepalive 10 120
persist-key
persist-tun

## Configure the logfile and the verbosity
verb 1  
mute 5
status /var/log/openvpn-status.log

Completing the Connection

Once the static key and configuration are copied to the correct location, establish the VPN connection by restarting VPN on both systems.

$ sudo service openvpn restart

Changes to Server 1

Once the VPN is established, it is time to join the second server to the domain. For this, some small changes are needed on the first server.

First, server one should use server two as the backup name server. In this case, even when there is an issue with DNS on the first server, the domain should still function correctly.

To set the name resolution on UCS execute

$ sudo ucr set nameserver2=10.200.10.11

On Debian/Ubuntu add the following line to /etc/resolve.conf

nameserver  10.200.10.11

Server 2 Join

Preparations

Similar to the server 1, server 2 should also use the other system as DNS fallback.

Again on UCS execute the following with the proper IP:

$ sudo ucr set nameserver2=10.210.237.171

On Debian/Ubuntu add the following line to /etc/resolve.conf:

nameserver  10.210.237.171

You also need to ensure that the NTP is getting the time from server 1 to guarantee that both systems have a synchronized clock.

On UCS, using the right IP, run:

$ sudo ucr set timeserver=10.210.237.171

On Debian/Ubuntu edit /etc/ntp.conf and add:

server 10.210.237.171

On a Debian or Ubuntu system, you will need to configure Kerberos before trying to join the domain. Overwrite /etc/krb5.conf with the following settings, changing the default_realm as needed:

[libdefaults]
    dns_lookup_realm = false
    dns_lookup_kdc = true
    default_realm = KORTE.UNIVENTION.COM

You can test the settings by running:

$ kinit administrator

Domain Join

Once all the previous steps have been taken, it is time to join the Samba domain, including setting a new AD-site. Sites give a preference for a certain client to prefer a particular DC or group of DCs. If no site is configured, the server will join the default site.

To define the site on UCS run:

ucr set samba4/join/site=my_secondsite

replacing my_secondsite with the actual name of your site

The last step is to execute the Univention join.

$ sudo univention-join

On Debian/Ubuntu Systems, you will need to issue the following command to join the server to a site:

$ sudo  samba-tool domain join kevin.univention.com DC -U"kevinadministrator" --dns-backend=SAMBA_INTERNAL --site=secondsite

Additional Considerations on Debian/Ubuntu

If you are using UCS, please skip this section. The UCS domain join takes care of the following two tasks.

Verify the DNS Records

Some versions of Samba 4 do not create all needed DNS records when joining a second DC. Thus you need to verify that the host record and objectGUID record have been created.

First, verify the host record with the following command on server 1:

$ host -t A server2.$(hostname -d)

Replace backup with the actual name of your server.

If you do not get a result, you can create the entry on server 1 with the following command:

$ sudo samba-tool dns add server1 $(hostname -d) server2 A 10.200.10.11
-Uadministrator

Then determine the objectGUID using the Samba database:

$ sudo ldbsearch -H /var/lib/samba/private/sam.ldb '(invocationId=*)' --cross-ncs objectguid

Results will look similar to this:

# record 1
dn: CN=NTDS Settings,CN=SERVER2,CN=Servers,CN=second,CN=Sites,CN=Configuration,DC=kevin,DC=univention,DC=com
objectGUID: 1b6f180e-5bc2-471f-a029-8c078e58c656

# record 2
dn: CN=NTDS Settings,CN=SERVER1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=kevin,DC=univention,DC=com
objectGUID: d7e12d36-2588-4d2c-b51c-3c762eab046b

# returned 2 records
# 2 entries
# 0 referrals

Select the entry where the DN contains the name of your server and us the objectGUID in the following command:

$ host -t CNAME 1b6f180e-5bc2-471f-a029-8c078e58c656
._msdcs.$(hostname -d)

If not found, you can add it with the following command:

$ sudo samba-tool dns add server1 _msdcs.$(hostname -d) 1b6f180e-5bc2-471f-a029-8c078e58c656 CNAME server2.$(hostname -d) -Uadministrator

Please remember to replace the server names and objectGUID in all of these commands whenever appropriate.

Sysvol Synchronization

Lastly, the group policies need to be synchronized. This job can be done in many different ways. From using a simple cron job to sophisticated multi-server synchronization. The samba wiki has an overview over the different approaches.

Conclusion

Setting up multi-location server systems can be a daunting task that requires some consideration and planning. However, the result of having a more robust and faster network is often enough to justify placing domain controllers off-site or in the cloud. Additionally, an additional cloud component in your identity management solution, no matter whether a second cloud DC or your first one, can often serve as the point to connect third-party services in a resilient manner.

The right tools, such as UCS, can accelerate many of the more complex tasks of setting up a multi-server domain. A professional and integrated domain ensures compliant and fail-safe authentication and policy services across locations and clouds, such as AWS.