Deploying httpd with acme-client with Ansible

Having the ability to rebuild a server/router from scratch in minutes with confidence, versus slaving over all your configs, trying to get everything working is life changing. I can't remember how many times I've rebuilt a computer, only to run into an issue that I KNOW I've fixed before... over a year ago. With ansible, all the work goes into the first deployment, giving you the ability to redeploy a server at a moments notice.

OpenBSD does require some extra options to work properly, as ansible seems to work best with Linux. Hopefully my struggles can help some of you.

The first thing I do is create a quick file and folder structure. For this example, I'll show how to write a playbook to setup httpd with acme-client for ssl certs, along with a cronjob to ensure that those certs remain active.

findelabs
|-- ansible.cfg
|-- bootstrap.sh
|-- update.yml
|-- group_vars
|   `-- all
|--roles  
   |-- httpd
   |   |-- tasks
   |   |   `-- main.yml
   |   |-- templates
   |   |   `-- httpd.conf
   |   `-- handlers
   |       `-- main.yml
   |-- acme-client
   |   |-- tasks
   |   |   `-- main.yml
   |   `-- templates
   |       `-- acme-client.conf
   `-- cron
       `-- tasks
           `-- main.yml

findelabs/ansible.cfg

First things first, we need to have an ansible.cfg.

[defaults]
internal_poll_interval = 0.01

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=60s

I specify 0.01 for my internal poll interval. This is just the speed that seemed to work best for my VM's. I've read of people reducing that number down to 0.001. So please experiment to find your own best value.

I enable ssh multiplexing with my ssh_args, so that re connections are sped up. This setting simply keeps a socket open so that subsequent authentications are skipped after the initial connection.

Here is a good article to read about optimizing your ansible deployments.

findelabs/bootstrap.sh

Next up is the bootstrap script. OpenBSD doesn't have the software needed by ansible to initially run. This script is used to quickly install the necessary packages for your system to run ansible playbook properly.

release=$(uname -r)
arch=$(uname -p)
echo "http://ftp.openbsd.org/pub/OpenBSD/" > /etc/installurl

export PKG_PATH=http://ftp.openbsd.org/pub/OpenBSD/${release}/packages/${arch}
pkg_add -I git ansible

This script simply creates the installurl file, then installs git and ansible. While git is not required, I always use git to keep track of changes to my playbooks.

findelabs/group_vars/all

We also need a group_vars folder, that will apply to all servers. This is assuming that the servers will be OpenBSD, mind you.

+++
ansible_python_interpreter: /usr/local/bin/python2.7

This file is needed to ensure that ansible knows the correct path to its interpreter.

findelabs/update.yml

Now we need the playbook that will call the correct roles.

+++
- hosts: 127.0.0.1
  roles:
    - { role: httpd, tags: httpd }
    - { role: acme-client, tags: acme-client }
    - { role: cron, tags: cron }

I use tagging to identify the individual roles, so that I can always run individual roles. For instance, if I have a massive playbook with dozens of roles, I won't always want to wait 20 minutes for the entire playbook to complete; I'd rather just run the one role that I updated.

httpd role

Let's delve into the first role. The httpd role should be able to give you a complete web server running with a proper config at its completion.

findelabs/roles/httpd/tasks.yml

These tasks will simply deploy the required httpd.conf file and ensure that the service is started and enabled.

+++
- name: Deploy httpd.conf
  template:
    src: httpd.conf
    dest: /etc/
    owner: root
    group: wheel
    mode: 0644
    backup: no
  notify:
    - restart httpd

- name: Ensure httpd is started and enabled
  service:
    name: httpd
    state: started
    enabled: yes

findelabs/roles/httpd/handlers/main.yml

This handler is only called if the httpd.conf was updated during the role's execution.

- name: restart httpd
  service:
    name: httpd
    state: restarted

findelabs/roles/httpd/templates/httpd.conf

Here we have the actual httpd config. Luckily OpenBSD's standard services use easy to understand configuration styles, so hopefully this config is pretty straight forward. You can read more about this file here.

chroot "/var/www"
ext_addr="*"

prefork 2

server "www.example.com" {
    listen on $ext_addr tls port 443
    alias "example.com"
    root "/htdocs/public"
    tls {
        certificate "/etc/ssl/www.example.com.pem"
        key "/etc/ssl/private/www.example.com.key"
        ticket lifetime default
        ciphers "secure"
    }

    hsts max-age 16000000
    hsts preload
    hsts subdomains

    location "/.well-known/acme-challenge/*" {
        root "/acme"
        request strip 2
    }

}
server "www.example.com" {
    listen on $ext_addr port 80
    alias "example.com"
    block return 301 "https://www.example.com$REQUEST_URI"
    location "/.well-known/acme-challenge/*" {
        root "/acme"
        request strip 2
    }

}

acme-client role

Alright, now on to acme-client. This role is extremely simple, as it only contains one task, to deploy the configuration, and one template, which is the configuration itself.

findelabs/roles/acme-client/tasks/main.yml

+++
- name: Deploy acme-client.conf
  template:
    src: acme-client.conf
    dest: /etc/
    owner: root
    group: wheel
    mode: 0644
    backup: no

findelabs/roles/acme-client/templates/acme-client.conf

authority letsencrypt {
  api url "https://acme-v01.api.letsencrypt.org/directory"
  account key "/etc/acme/letsencrypt-privkey.pem"
}
domain www.example.com {
    alternative names { example.com }
    domain key "/etc/ssl/private/www.example.com.key"
    domain certificate "/etc/ssl/www.example.com.crt"
    domain full chain certificate "/etc/ssl/www.example.com.pem"
    sign with letsencrypt
}

That's all there is to this role. Super simple.

cron role

Now for the last part. This role will create a cronjob that will ensure that your certs never expire on accident. Just so that you are aware, acme-client will exit 0 if the certificates have been updated, 1 on failure, and 2 if the certificates were not within the one month window that acme-client will update certs within.

findelabs/roles/cron/tasks/main.yml

+++
- name: Download and refresh certs
  cron:
    name: "Download current certs"
    hour: "0"
    job: "/usr/sbin/acme-client www.findelabs.com && rcctl reload httpd && logger 'Updated ssl certs'"
    user: root

Running the playbook

Once all these directories and contents have been created, running the bootstrap script, followed by the playbook itself, then by the command to initialize your certs for the first go-round, should leave you with a complete and secure webserver.

cd findelabs
./bootstrap.sh
ansible-playbook update.yml
doas acme-client -vD www.example.com

Has been tested on OpenBSD 6.4