Validating yaml to json conversion

When using yaml to json conversion as it happens when using Terraform’s helm provider with a helm chart’s values.yaml file, it won’t be possible to check and get any useful validation errors, even when your terraform plan is run.

In order to check before running your plan or if you have any errors you can use yq passing as below using the values.yaml file

yq -p yaml -o json values.yaml

googlecloudsdk.calliope.exceptions.HttpException: ResponseError: code=400, message=Autopilot clusters must be regional clusters.

Problem

Trying to create an auto-cluster either using terraform or gcloud cli, and specifying the region name returns the error that ‘Autopilot clusters must be regional clusters.

So with gcloud this is the command and output

kosmas: (master %%)$ gcloud container clusters create-auto test-cluster --region=europe-west6-b
Note: The Pod address range limits the maximum size of the cluster. Please refer to https://cloud.google.com/kubernetes-engine/docs/how-to/flexible-pod-cidr to learn how to optimize IP address allocation.
ERROR: (gcloud.container.clusters.create-auto) ResponseError: code=400, message=Autopilot clusters must be regional clusters.

Solution

Using the actual region name (that can be taken from the list of available zones/regions)

gcloud compute zones list

NAME                       REGION                   STATUS  NEXT_MAINTENANCE  TURNDOWN_DATE
us-east1-b                 us-east1                 UP
us-east1-c                 us-east1                 UP
...
europe-west6-b             europe-west6             UP
...

And using the correct region name (without the b)

gcloud container clusters create-auto test-cluster --region=europe-west6 --verbosity debug

...
Created [https://container.googleapis.com/v1/projects/gitlab-runner-343714/zones/europe-west6/clusters/test-cluster].
...
NAME          LOCATION      MASTER_VERSION   MASTER_IP     MACHINE_TYPE  NODE_VERSION     NUM_NODES  STATUS
test-cluster  europe-west6  1.21.6-gke.1503  xxx.xxx.xxx.xxx  e2-medium     1.21.6-gke.1503  3          RUNNING

asdf and terraform (or vault or packer)

Using the asdf version manager to manage versions of various binaries like terraform, vault or packer is easily done by following the instructions below:

  1. Download and install asdf if you haven’t got it already (https://asdf-vm.com/#/core-manage-asdf)
  2. If you want to install the latest version of terraform for example do the following:
asdf plugin add terraform
asdf install terraform latest
asdf global terraform 0.14.9
asdf list
terraform
  0.14.9
terraform --version
Terraform v0.14.9

Using port number in Terraform output

Problem

You would like to output some additional information (ie port number) in addition to the variable provided by Terraform.

Solution

Use interpolation of the Terraform variable using ${var} together with the rest of the string you would like to output, like:

output "kibana_endpoint" {
  value       = "http://${aws_instance.elasticsearch.public_ip}:5601"
  description = "The Kibana endpoint"
}

More information about the language https://learn.hashicorp.com/tutorials/terraform/outputs?in=terraform/configuration-language

Decode access secret key from Terraform IAM user creation

Problem

You want to use the pgp encryption when using Terraform to create an AWS IAM user, and you have the secret access token returned as an output but encoded.

Solution

Use the following to get the actual secret key decoded (after copying your encoded key to a file encrypted_key.txt:

$ cat encrypted_key.txt | base64 --decode | gpg -d

Terraform Cloud – error when trying to initialize

Problem

You would like to start using Terraform Cloud and when trying to initialize it with the new remote backend (app.terraform.io), you get the following error:

Error: Required token could not be found

Make sure you configured a credentials block for app.terraform.io in your CLI
Config File.

Solution

Follow the instructions here https://www.terraform.io/docs/cloud/free/index.html and create a file ~/.terraformrc (in linux) with an API token.

Getting the slug name for Digital Ocean’s images with doctl

Problem

You would like to get a list of the available images in Digital Ocean, in order to be able to use them in creating your Terraform IAC script.

Solution

By installing the command line tool doctl, (instructions here), according to the documentation (after authenticating) you can run:

doctl compute image list

But the available images are not listed. In order to be list all the images you have to add the –public option (as described here) :

doctl compute image list --public