asdf and terraform (or vault or packer)

Using the asdf version manager to manage versions of various binaries like terraform, vault or packer is easily done by following the instructions below:

  1. Download and install asdf if you haven’t got it already (https://asdf-vm.com/#/core-manage-asdf)
  2. If you want to install the latest version of terraform for example do the following:

Get/Set platform information

To get or set in environment variables the platform information of a system you can use the following:

Taken from the installation instructions of the Operator SKD here: https://master.sdk.operatorframework.io/docs/installation/

Using port number in Terraform output

Problem

You would like to output some additional information (ie port number) in addition to the variable provided by Terraform.

Solution

Use interpolation of the Terraform variable using ${var} together with the rest of the string you would like to output, like:

More information about the language https://learn.hashicorp.com/tutorials/terraform/outputs?in=terraform/configuration-language

Failed to start docker.service: Unit is masked

Problem

Trying to start the docker service after some upgrades fails with the following message:

Solution

It turns out that after upgrading or more specifically removing and then upgrading the docker installation in ubuntu (in this particular case in raspberry 4 with Ubuntu 20.04 installed), results in this error.

A search brings up the following:

https://forums.docker.com/t/failed-to-start-docker-service-unit-is-masked/67413

and from that the following bug post:

https://bugs.launchpad.net/ubuntu/+source/docker.io/+bug/1844894

So the solution is to run the following to be able to start the docker service (described in the first link above):

HAProxy load balancing with ssl pass_through in front of Vault

Problem

You would like to set up HAProxy as the load balancer in your Vault cluster configuration. Vault is set up with internal storage (Raft), and you would also like to be able to pass the connection encrypted from the client to the Vault cluster member, without having HAProxy decrypt/encrypt the message.

Solution

Looking and searching around in the internet provides a few examples of how to do this. During this process there are different errors that you can see happening. There are certain things that you will need to do in order to debug and find the correct setup.

One of the main points is that HAProxy’s frontend and backend mode should be tcp instead of http.

Another thing that will help with debugging would be having access to both HAProxy’s and Vault (active) member logs.

Using curl with the -v flag is also very helpful.

One of the attempts of having HAProxy and Vault server communicate through HTTP/2 was resulting in the following errors:

on an external client (not HAProxy, or Vault).

the following on the HAProxy log:

and the following on the Vault log

Changing the configuration on HAProxy to use http1.1 solves the issue (/etc/haproxy/haproxy.cfg

Which results in the correct response now by using the HAProxy as a ssl pass_through load balancer.

SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch

Problem

Trying to configure and use an nginx server that uses intermediate certifcates, you get the error about values mismatch and nginx does not start

Solution

It seems that this is a common mistake and it is mentioned here: https://nginx.org/en/docs/http/configuring_https_servers.html. In order to fix this you will need to change/reverse the original order that you have concatenated the chain, as in:

Tail Vault auth logs

Problem

You would like to tail the Vault auth log files, but if you use the vault tag for auditing and your server is also named vault, you cannot filter out the vault auth logs only.

Solution

Use the client_login as the filter in grep:

Check haproxy configuration file for errors

Problem

You would like to check your haproxy.cfg file for errors when making changes.

Solution

Use the following command to validate/check your HAProxy configuration file: