googlecloudsdk.calliope.exceptions.HttpException: ResponseError: code=400, message=Autopilot clusters must be regional clusters.

Posted on by
Reply

Problem

Trying to create an auto-cluster either using terraform or gcloud cli, and specifying the region name returns the error that ‘Autopilot clusters must be regional clusters.

So with gcloud this is the command and output

kosmas: (master %)$ gcloud container clusters create-auto test-cluster --region=europe-west6-b
Note: The Pod address range limits the maximum size of the cluster. Please refer to https://cloud.google.com/kubernetes-engine/docs/how-to/flexible-pod-cidr to learn how to optimize IP address allocation.
ERROR: (gcloud.container.clusters.create-auto) ResponseError: code=400, message=Autopilot clusters must be regional clusters.

Solution

Using the actual region name (that can be taken from the list of available zones/regions)

gcloud compute zones list

NAME                       REGION                   STATUS  NEXT_MAINTENANCE  TURNDOWN_DATE
us-east1-b                 us-east1                 UP
us-east1-c                 us-east1                 UP
...
europe-west6-b             europe-west6             UP
...

And using the correct region name (without the b)

gcloud container clusters create-auto test-cluster --region=europe-west6 --verbosity debug

...
Created [https://container.googleapis.com/v1/projects/gitlab-runner-343714/zones/europe-west6/clusters/test-cluster].
...
NAME          LOCATION      MASTER_VERSION   MASTER_IP     MACHINE_TYPE  NODE_VERSION     NUM_NODES  STATUS
test-cluster  europe-west6  1.21.6-gke.1503  xxx.xxx.xxx.xxx  e2-medium     1.21.6-gke.1503  3          RUNNING

Adding a LOKI report in Grafana counting number of http_user_agents from NGINX logs

Posted on by
Reply

To add a Loki report in Grafana, as for example having a count of the different http_user_agents that have logged in, you can use the following query:

count by (http_agent) (rate({namespace="ingress-nginx",stream="stdout"} |= "https://domain.name.com/session/new" |~ "GET\\s/\\s" | pattern "<ip> - - [<timestamp>] \"<method> <path> <version>\" <result> <_> \"<url>\" \"<http_agent>\" <_>" [$__interval]))

You select the source (Loki) and then use the {namespace=”ingress-nginx”,stream=”stdout”} to get the log files from Nginx.

Then you can filter by two conditions:

  1. the instance login
|= "https://domain.name.com/session/new"

and

  1. use the regular expression to only select the GET / calls
|~ "GET\\s/\\s"

Then you can use the pattern parser for the log file in order to get the label

pattern "<ip> - - [<timestamp>] \"<method> <path> <version>\" <result> <_> \"<url>\" \"<http_agent>\" <_>"

<_> is used if we are not interested in getting the field, so everything after http_agent is not analyzed in this case.

Finally use the

count by(http_agent)

to get the data (note that you also need to use the rate with the [$__interval])

Convert a single line string ‘a b c’ to a multiple line string with double quotes and commas like [“a”, “b”, “c”] using bash and sed

Posted on by
Reply

Problem

You have single line output that contains multiple strings (could be output from a kubectl that gives all the instance names) and you want to convert it to text that can be used as the a variable enclosed by double quotes, one string on one line and separated by commas.

So having the string in a file called input.txt:

a b c

you want to convert it to the following and save it in another file called output.txt

"a",
"b",
"c"

Solution

You can use the following:

cat input.txt | tr ' ' '\n' | sed 's/^/"/g' | sed 's/$/",/g' > output.txt

which will first replace the spaces separating the strings to newlines, and then use two passes with sed, in the first adding the first quote, and the second adding the second quote and the comma.

Required ‘compute.networks.create’ permission Google/Terraform Cloud

Posted on by
Reply

Problem

You want to create a GKE cluster in Google using Terraform Cloud and the instructions provided https://registry.terraform.io/providers/hashicorp/google/4.3.0/docs/guides/getting_started#using-terraform-cloud-as-the-backend

You are creating a service account in Google Service Accounts and you have set up the environment variable GOOGLE_CREDENTIALS (after removing the new lines from the json file with tr -d '\n' < original_google_file.json > no_newlines_google_file.json.

But when trying to create the GKE cluster you get the following error in Terraform

Required 'compute.networks.create' permission

Solution

The default permissions when creating the Service Account are not enough.

Add the Editor role to your service account, by going to IAM, Permissions for Project, as it does not seem possible to add this permission by editing the Service Account.