DNS

The preferred method of assigning an IP address and hostname, in order of preference:

1. Assign IP address and domain name using DHCP
2. Assign static IP address and domain name in /etc/hosts
3. Assign loopback address and local hostname

There are expectations for how each name should be resolved:

• Domain names should resolve to the associated IP address, if available (source).
• Hostname must always be resolvable.
• localhost must always resolve to 127.0.0.1.
  • This implies that "127.0.0.1 localhost" ideally should remain as the first line in /etc/hosts.

Now consider the following scenarios:

• If there is no network device, the hostname should still resolve somewhere.
  • Hostname should by default resolve to the loopback interface (127.0.0.0/8).
• The hostname lookup should not resolve to "localhost" as the canonical name (source).
  • This happens when both "localhost" and "<hostname>" resolves to the same address, with the earlier line being prioritized.
  • Debian resolves the hostname to 127.0.1.1 instead.
• Some applications expect the hostname to resolve to the domain name.
  • This was true eons ago, e.g. GNOME expecting the hostname to be an FQDN.
  • This can be resolved with "... <domain_name> <hostname>"
• If a static IP address is assigned, and domain names are not assigned via DHCP, this can be added as an entry in /etc/hosts.
  • The domain name, and hence the hostname, should map to this static IP.

This yields, in the case of electric hostname and pikachu domain name (ideally these two should be equivalent, by using assignment by DNS):

/etc/hosts

127.0.0.1 localhost
192.168.1.3 pikachu.pyuxiang.com pikachu electric

# If no domain name is needed/assigned:
# 127.0.1.1 electric

Some useful commands to test NS resolution behaviour: hostname -f and nslookup / dig.

One method to control how DNS flows is using Unbound. Alternatively, if using the stub listener, refer to this instead. Some conflicting information, but main takeaways:

• The DNS set in "/etc/systemd/resolved.conf" are global configurations. To make it transient, tie it to whichever service is being initialized, e.g. (I haven't managed to successfully execute this though)

systemd-networkd.service

[Match]
Name=wg0
 
[Network]
DNS=10.253.253.253
Domains=~pyuxiang.com

A perhaps more illustrative description how name lookup is performed (src, src2):

• /etc/nsswitch.conf specifies how hostname resolution should be performed, amongst others.
  • e.g. hosts: files mdns4 dns, which specifies the file /etc/hosts should be looked up first, followed by mDNS (multicast DNS), then lastly the DNS nameservers in /etc/resolv.conf.
  • Note that applications may trigger other name resolution services, e.g. NBNS (NetBios), LLMNR (link-local multicast name resolution)
• /etc/resolv.conf lists the nameservers to query for DNS.
  • This used to be a static file, but usually needs to be dynamically updated, e.g. with VPNs.
  • This file is used by C-routines for name resolution (see man 3 resolver).
• systemd-resolved, /etc/systemd/resolved.conf is part of the systemd suite of services.
  • The resolvectl binary can be used to interface with the systemd-resolved service.
  • DNS can be set, e.g. 192.168.1.1#pyuxiang.com with custom domain resolution Domains=~.pyuxiang.com.
• resolvconf, /etc/resolvconf.conf, /etc/resolvconf/* seems to be another mechanism using OpenResolv...?
• NetworkManager and systemd-networkd
• Unbound

This section is not complete yet... (2024-11-05)

• Ubuntu 24.04:
  • Uses the systemd-resolved service. Configuration file found at /etc/systemd/resolved.conf.
  • Seems possible to add via NetworkManager as well, likely relying on systemd-resolved as the backend.
  • A modified resolvconf is packaged together with systemd-resolved for compatibility reasons.

root:~# vim /etc/systemd/resolved.conf
[Resolve]
DNS=192.168.1.100
DNSStubListener=no
root:~# systemctl restart systemd-resolved

• Debian:
  • resolvconf seems to be installed by default(?)
  • If using Debian as a stable system with very minimal configuration changes, simply editing /etc/resolv.conf is viable.
• openSUSE 15.6:
  • Do not modify /etc/resolv.conf directly - edit via NetworkManager.
  • YaST delegates to NetworkManager, which in turn internally uses netconfig (a tool used by openSUSE and RHEL) to merge and update DNS configuration.
  • systemd-resolved service is not installed by default.

root:~# nmtui
root:~# nmcli device reapply eth0

Why use dehydrated instead of certbot?

A quick rundown of the ACME process helps with context: ACME (Automated Certificate Management Environment) is fundamentally a protocol between clients and CA that automates certificates issuance, which used to require human intervention. Developed as part of Let's Encrypt (an ACME-supported CA), and formalised in RFC8555.

The ACMEv2 protocol is as follows:

1. Client registers identity with server (optional depending on server, but usually provides useful things like expiry email reminders)
2. Client generates CSR and sends an ACME HTTP request (one of HTTP-01, DNS-01, TLS-01).
3. Server issues a string (challenge token) which it expects to read by accessing a URI or DNS record.
4. Client interacts with the webserver to expose a URL path with file (.well-known), or modifies DNS TXT records (.acme-challenge). Then tells server it is ready.
5. Server verifies the challenge tokens, and signs the CSR.
6. Client uploads the certificate.

The server in question is a CA (e.g. Let's Encrypt, ZeroSSL, or self-hosted ACME-supported step-ca), while the ACME client is a script that ideally automates the steps above. Since the protocol is language-agnostic, many ACME clients exist: a few popular ones that commonly pop up include lego (Go), certbot (Python), acme.sh (Bash) and dehydrated (Bash). Just pick a favorite and go.

dehydrated is a single Bash script (~3k lines) that accepts three pieces of configuration located in /etc/dehydrated:

1. domains.txt: A list of domain/aliases to generate certificates for.
2. config: Configuration for dehydrated.
3. hook.sh: Hook configuration.

Certficate generation/renewal is simply dehydrated -c.

The ACME client can automate certificate/key management and communicate with the ACME server for challenge tokens. Additional functionality of (1) modifying DNS records, (2) uploading certificates to webserver, needs to be patched in using hooks/plugins. In the case of dehydrated, the hooks are implemented as an external client Bash script, which receives a command verb and relevant arguments. The documentation provides a limited list of commands and an hook boilerplate example.

An example set of commands to write for, using ACMEv2 DNS-01:

• "startup_hook": Start ACME-DNS server (if using delegation).
• "deploy_challenge": Write DNS TXT records.
• "clean_challenge": Delete DNS TXT records after verification.
• "deploy_cert": Write certificates to webserver.
• "exit_hook": Stand down ACME-DNS server.

If using joohoi's acme-dns service, there is no need to start a DNS server, nor clean challenges. If running webserver on the same machine, one can also reference the certificate symlink directly, at /etc/dehydrated/certs/$DOMAIN/fullchain.pem, so the script effectively reduces to just "deploy_challenge":

/etc/dehydrated/hook.sh

#!/usr/bin/env bash
 
# Register with:
# curl -s -X POST https://auth.acme-dns.io/register
# then create CNAME DNS record: '.acme-challenge.$DOMAIN CNAME $DNS_SUBDOMAIN.acme-dns.io'
DNS_ENDPOINT="https://auth.acme-dns.io/update"
DNS_SUBDOMAIN="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
DNS_USER="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
DNS_PASS="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
HANDLER="${1}"
 
if [ "$HANDLER" = "deploy_challenge" ]; then
  TOKEN="${4}"
  curl -X POST $DNS_ENDPOINT -H "X-Api-User: $DNS_USER" -H "X-Api-Key: $DNS_PASS" \
    --data "{\"subdomain\": \"$DNS_SUBDOMAIN\", \"txt\": \"$TOKEN\"}" >/dev/null
 
elif [ "$HANDLER" = "deploy_cert" ]; then
  KEYFILE="${3}" CERTFILE="${4}" FULLCHAINFILE="${5}"
  scp {$KEYFILE,$CERTFILE,$FULLCHAINFILE} webserver:/var/www/certs/
fi

Note that the domain associated with the certificate is the first domain in each line of domains.txt, i.e. certificate deployment requires matching to the correct domain.

Upon certificate deployment, the webserver may require a manual reload, which can be achieved via:

# If PermitRootLogin on sshd is enabled, and ssh configured to login as root...
local:~$ systemctl --host webserver reload nginx
 
# ...otherwise, allow user to run specific command as root with sudo
# Note there is rule precedence: <https://unix.stackexchange.com/a/67488>
pyuxiang@remote:~$ sudo visudo -f /etc/sudoers.d/nginx
pyuxiang  ALL=(root) NOPASSWD: /usr/bin/systemctl reload nginx
pyuxiang@remote:~$ sudo chmod 440 /etc/sudoers.d/nginx
local:~$ ssh webserver 'sudo systemctl reload nginx'

Most of the defaults generally work fine. The full config is available, though I only needed to populate these variables:

/etc/dehydrated/config

# dehydrated configuration
BASEDIR=/etc/dehydrated
HOOK=/etc/dehydrated/hook.sh
DEHYDRATED_USER=xxxxxx
DEHYDRATED_GROUP=xxxxxx
 
# ACME server configuration
# Use "letsencrypt-test" staging server to avoid rate limits during testing
CA="letsencrypt"
CHALLENGETYPE="dns-01"
CONTACT_EMAIL=xxxxxx

For a single wildcard certificate, the domains file is straightfoward: sign for the base domain, with the wildcard as a SAN. The exact file I use is attached below:

/etc/dehydrated/domains.txt

pyuxiang.com *.pyuxiang.com

First register with ./dehydrated --register --accept-terms, then put ./dehydrated -c into a cron job and be done!

acme-dns is a simple DNS server that serves TXT records for the ACME challenges, and also supports basic authentication for updating said records over HTTP.

First sign up for the service:

user:~$ curl -sX POST https://auth.acme-dns.io/register
{
    "subdomain": "9d2c7b32-4af4-482c-9e46-718acf50539e",
    "username": "3d97e467-dd67-41d4-871f-5590b3d03c05",
    "password": "cLzdpV031ieuZzAE7jVNnX08uMqV0OsyIbf6Cqfm",
    ...
}

Push ACME challenge tokens:

user:~$ cat header.txt
X-Api-User: 3d97e467-dd67-41d4-871f-5590b3d03c05
X-Api-Key: cLzdpV031ieuZzAE7jVNnX08uMqV0OsyIbf6Cqfm
 
user:~$ cat body.txt
{
    "subdomain": "9d2c7b32-4af4-482c-9e46-718acf50539e",
    "txt": "w3LreTPDfo3GbaoRmoneFgKbmpGdweWbrlpg04-1xY0"
}
 
user:~$ curl -sX POST -H @header.txt --data @body.txt https://auth.acme-dns.io/update
{
    "txt": "w3LreTPDfo3GbaoRmoneFgKbmpGdweWbrlpg04-1xY0"
}

Verify challenge tokens are updated. Note that up to two TXT challenges can be cached in the "auth.acme-dns.io" service (typically needed for simultaneous root domain and wildcard domain validation).

user:~$ dig _acme-challenge.pyuxiang.com TXT
...
;; QUESTION SECTION:
;_acme-challenge.pyuxiang.com.  IN      TXT
 
;; ANSWER SECTION:
_acme-challenge.pyuxiang.com. 3430 IN   CNAME   9d2c7b32-4af4-482c-9e46-718acf50539e.auth.acme-dns.io.
9d2c7b32-4af4-482c-9e46-718acf50539e.auth.acme-dns.io. 1 IN TXT "w3LreTPDfo3GbaoRmoneFgKbmpGdweWbrlpg04-1xY0"
...

For subdomains, extend the name of the record:

user:~$ dig +noall +answer +multiline -t CNAME '_acme-challenge.www.pyuxiang.com'
_acme-challenge.www.pyuxiang.com. 3600 IN CNAME	9d2c7b32-4af4-482c-9e46-718acf50539e.auth.acme-dns.io.