Find the current AMI image (i.e. Ubuntu 16.04) in AWS using command line

Posted on 17 January 2021 by kosmas

If you would like to programatically find the current image of a specific distribution in AWS for your region, with the use of the aws cli, use the following (in this example looking for Ubuntu 16.04).

aws ec2 describe-images \
    --owners 099720109477 \
    --filters &quot;Name=name,Values=ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-????????&quot; &quot;Name=state,Values=available&quot; \
    --query &quot;reverse(sort_by(Images, &amp;CreationDate))[:1].ImageId&quot; \
    --output text

				
					
				1
2
3
4
5

						aws ec2 describe-images \
    --owners 099720109477 \
    --filters &quot;Name=name,Values=ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-????????&quot; &quot;Name=state,Values=available&quot; \
    --query &quot;reverse(sort_by(Images, &amp;CreationDate))[:1].ImageId&quot; \
    --output text

					

			

This is from the examples in the documentation here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/finding-an-ami.html

Getting detailed information about your Linux distribution

Posted on 16 December 2020 by kosmas

To be able to get a more detailed information about your current linux distribution, including the code names use the following:

kosmas:$ cat /etc/os-release
NAME=&quot;Ubuntu&quot;
VERSION=&quot;18.04.5 LTS (Bionic Beaver)&quot;
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME=&quot;Ubuntu 18.04.5 LTS&quot;
VERSION_ID=&quot;18.04&quot;
HOME_URL=&quot;https://www.ubuntu.com/&quot;
SUPPORT_URL=&quot;https://help.ubuntu.com/&quot;
BUG_REPORT_URL=&quot;https://bugs.launchpad.net/ubuntu/&quot;
PRIVACY_POLICY_URL=&quot;https://www.ubuntu.com/legal/terms-and-policies/privacy-policy&quot;
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13

						kosmas:$ cat /etc/os-release
NAME=&quot;Ubuntu&quot;
VERSION=&quot;18.04.5 LTS (Bionic Beaver)&quot;
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME=&quot;Ubuntu 18.04.5 LTS&quot;
VERSION_ID=&quot;18.04&quot;
HOME_URL=&quot;https://www.ubuntu.com/&quot;
SUPPORT_URL=&quot;https://help.ubuntu.com/&quot;
BUG_REPORT_URL=&quot;https://bugs.launchpad.net/ubuntu/&quot;
PRIVACY_POLICY_URL=&quot;https://www.ubuntu.com/legal/terms-and-policies/privacy-policy&quot;
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

					

			

Displaying timestamps in dmesg output

Posted on 10 December 2020 by kosmas

If you would like to be able to see the dmesg output with timestamps instead of time in seconds since booting up, use the -T like:

dmesg -T | less

				1

						dmesg -T | less

Using strace for analyzing performance

Posted on 6 December 2020 by kosmas

There is a very nice example on the O’reilly ‘Linux Under The Hood’ training about using strace to compare performance of two commands that provide the same output but with different calls. The two different commands were: ls and echo *, that they both provide the listing of the current directory.

First using the strace with ls:

kosmas:$ strace -c ls
abi-4.13.0-45-generic      config-4.4.0-31-generic        initrd.img-4.4.0-31-generic   System.map-4.15.0-123-generic         vmlinuz-4.15.0-123-generic
abi-4.13.0-46-generic      efi                            memtest86+.bin                System.map-4.15.0-124-generic         vmlinuz-4.15.0-124-generic
abi-4.4.0-31-generic       grub                           memtest86+.elf                System.map-4.15.0-126-generic         vmlinuz-4.15.0-126-generic
config-4.13.0-45-generic   initrd.img-4.13.0-45-generic   memtest86+_multiboot.bin      System.map-4.4.0-31-generic           vmlinuz-4.4.0-31-generic
config-4.13.0-46-generic   initrd.img-4.13.0-46-generic   retpoline-4.13.0-45-generic   vmlinuz-4.13.0-45-generic             vmlinuz-4.4.0-31-generic.efi.signed
config-4.15.0-123-generic  initrd.img-4.15.0-123-generic  retpoline-4.13.0-46-generic   vmlinuz-4.13.0-45-generic.efi.signed
config-4.15.0-124-generic  initrd.img-4.15.0-124-generic  System.map-4.13.0-45-generic  vmlinuz-4.13.0-46-generic
config-4.15.0-126-generic  initrd.img-4.15.0-126-generic  System.map-4.13.0-46-generic  vmlinuz-4.13.0-46-generic.efi.signed
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 15.81    0.000095          12         8           write
 14.64    0.000088           7        12           mprotect
 13.48    0.000081           5        17           mmap
  9.98    0.000060           7         9           openat
  7.82    0.000047           4        11           close
  7.32    0.000044          22         2           getdents
  6.82    0.000041           6         7           read
  5.66    0.000034           3        10           fstat
  4.99    0.000030           4         8         8 access
  3.16    0.000019          10         2         2 statfs
  2.00    0.000012          12         1           munmap
  1.83    0.000011           4         3           brk
  1.66    0.000010           5         2           ioctl
  1.50    0.000009           5         2           rt_sigaction
  0.67    0.000004           4         1           rt_sigprocmask
  0.67    0.000004           4         1           arch_prctl
  0.67    0.000004           4         1           set_tid_address
  0.67    0.000004           4         1           set_robust_list
  0.67    0.000004           4         1           prlimit64
  0.00    0.000000           0         1           execve
------ ----------- ----------- --------- --------- ----------------
100.00    0.000601                   100        10 total

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

						kosmas:$ strace -c ls
abi-4.13.0-45-generic      config-4.4.0-31-generic        initrd.img-4.4.0-31-generic   System.map-4.15.0-123-generic         vmlinuz-4.15.0-123-generic
abi-4.13.0-46-generic      efi                            memtest86+.bin                System.map-4.15.0-124-generic         vmlinuz-4.15.0-124-generic
abi-4.4.0-31-generic       grub                           memtest86+.elf                System.map-4.15.0-126-generic         vmlinuz-4.15.0-126-generic
config-4.13.0-45-generic   initrd.img-4.13.0-45-generic   memtest86+_multiboot.bin      System.map-4.4.0-31-generic           vmlinuz-4.4.0-31-generic
config-4.13.0-46-generic   initrd.img-4.13.0-46-generic   retpoline-4.13.0-45-generic   vmlinuz-4.13.0-45-generic             vmlinuz-4.4.0-31-generic.efi.signed
config-4.15.0-123-generic  initrd.img-4.15.0-123-generic  retpoline-4.13.0-46-generic   vmlinuz-4.13.0-45-generic.efi.signed
config-4.15.0-124-generic  initrd.img-4.15.0-124-generic  System.map-4.13.0-45-generic  vmlinuz-4.13.0-46-generic
config-4.15.0-126-generic  initrd.img-4.15.0-126-generic  System.map-4.13.0-46-generic  vmlinuz-4.13.0-46-generic.efi.signed
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 15.81    0.000095          12         8           write
 14.64    0.000088           7        12           mprotect
 13.48    0.000081           5        17           mmap
  9.98    0.000060           7         9           openat
  7.82    0.000047           4        11           close
  7.32    0.000044          22         2           getdents
  6.82    0.000041           6         7           read
  5.66    0.000034           3        10           fstat
  4.99    0.000030           4         8         8 access
  3.16    0.000019          10         2         2 statfs
  2.00    0.000012          12         1           munmap
  1.83    0.000011           4         3           brk
  1.66    0.000010           5         2           ioctl
  1.50    0.000009           5         2           rt_sigaction
  0.67    0.000004           4         1           rt_sigprocmask
  0.67    0.000004           4         1           arch_prctl
  0.67    0.000004           4         1           set_tid_address
  0.67    0.000004           4         1           set_robust_list
  0.67    0.000004           4         1           prlimit64
  0.00    0.000000           0         1           execve
------ ----------- ----------- --------- --------- ----------------
100.00    0.000601                   100        10 total

					

			

And using the same with echo *

kosmas:$ strace -c echo *
abi-4.13.0-45-generic abi-4.13.0-46-generic abi-4.4.0-31-generic config-4.13.0-45-generic config-4.13.0-46-generic config-4.15.0-123-generic config-4.15.0-124-generic config-4.15.0-126-generic config-4.4.0-31-generic efi grub initrd.img-4.13.0-45-generic initrd.img-4.13.0-46-generic initrd.img-4.15.0-123-generic initrd.img-4.15.0-124-generic initrd.img-4.15.0-126-generic initrd.img-4.4.0-31-generic memtest86+.bin memtest86+.elf memtest86+_multiboot.bin retpoline-4.13.0-45-generic retpoline-4.13.0-46-generic System.map-4.13.0-45-generic System.map-4.13.0-46-generic System.map-4.15.0-123-generic System.map-4.15.0-124-generic System.map-4.15.0-126-generic System.map-4.4.0-31-generic vmlinuz-4.13.0-45-generic vmlinuz-4.13.0-45-generic.efi.signed vmlinuz-4.13.0-46-generic vmlinuz-4.13.0-46-generic.efi.signed vmlinuz-4.15.0-123-generic vmlinuz-4.15.0-124-generic vmlinuz-4.15.0-126-generic vmlinuz-4.4.0-31-generic vmlinuz-4.4.0-31-generic.efi.signed
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  0.00    0.000000           0         1           read
  0.00    0.000000           0         1           write
  0.00    0.000000           0         5           close
  0.00    0.000000           0         4           fstat
  0.00    0.000000           0         6           mmap
  0.00    0.000000           0         4           mprotect
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         3           brk
  0.00    0.000000           0         3         3 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         1           arch_prctl
  0.00    0.000000           0         3           openat
------ ----------- ----------- --------- --------- ----------------
100.00    0.000000                    33         3 total

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

						kosmas:$ strace -c echo *
abi-4.13.0-45-generic abi-4.13.0-46-generic abi-4.4.0-31-generic config-4.13.0-45-generic config-4.13.0-46-generic config-4.15.0-123-generic config-4.15.0-124-generic config-4.15.0-126-generic config-4.4.0-31-generic efi grub initrd.img-4.13.0-45-generic initrd.img-4.13.0-46-generic initrd.img-4.15.0-123-generic initrd.img-4.15.0-124-generic initrd.img-4.15.0-126-generic initrd.img-4.4.0-31-generic memtest86+.bin memtest86+.elf memtest86+_multiboot.bin retpoline-4.13.0-45-generic retpoline-4.13.0-46-generic System.map-4.13.0-45-generic System.map-4.13.0-46-generic System.map-4.15.0-123-generic System.map-4.15.0-124-generic System.map-4.15.0-126-generic System.map-4.4.0-31-generic vmlinuz-4.13.0-45-generic vmlinuz-4.13.0-45-generic.efi.signed vmlinuz-4.13.0-46-generic vmlinuz-4.13.0-46-generic.efi.signed vmlinuz-4.15.0-123-generic vmlinuz-4.15.0-124-generic vmlinuz-4.15.0-126-generic vmlinuz-4.4.0-31-generic vmlinuz-4.4.0-31-generic.efi.signed
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  0.00    0.000000           0         1           read
  0.00    0.000000           0         1           write
  0.00    0.000000           0         5           close
  0.00    0.000000           0         4           fstat
  0.00    0.000000           0         6           mmap
  0.00    0.000000           0         4           mprotect
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         3           brk
  0.00    0.000000           0         3         3 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         1           arch_prctl
  0.00    0.000000           0         3           openat
------ ----------- ----------- --------- --------- ----------------
100.00    0.000000                    33         3 total

					

			

So the difference in time is 0.00000 for the echo and 0.000601 for the ls.

Similarly the number of calls for the echo is 33, but ls is using 100.

Brennan B2 ssh login

Posted on 18 November 2020 by kosmas

Problem

You would like to login to your Brennan B2 using your ssh key.

Solution

Copy your public ssh key to /root/.ssh/authorized_keys file in your Brennan B2

Make multiple directories with one command in Linux

Posted on 5 October 2020 by kosmas

Use the following to create multiple directories with one command in Linux.

mkdir -p main_folder/{one,two,three,four}

				1

						mkdir -p main_folder/{one,two,three,four}

HAProxy load balancing with ssl pass_through in front of Vault

Posted on 25 September 2020 by kosmas

Problem

You would like to set up HAProxy as the load balancer in your Vault cluster configuration. Vault is set up with internal storage (Raft), and you would also like to be able to pass the connection encrypted from the client to the Vault cluster member, without having HAProxy decrypt/encrypt the message.

Solution

Looking and searching around in the internet provides a few examples of how to do this. During this process there are different errors that you can see happening. There are certain things that you will need to do in order to debug and find the correct setup.

One of the main points is that HAProxy’s frontend and backend mode should be tcp instead of http.

Another thing that will help with debugging would be having access to both HAProxy’s and Vault (active) member logs.

Using curl with the -v flag is also very helpful.

One of the attempts of having HAProxy and Vault server communicate through HTTP/2 was resulting in the following errors:

kosmas:$ curl -v https://proxy_dn:8200/v1/sys/health
*   Trying xxx.xxx.xxx.xxx...
* TCP_NODELAY set
* Connected to proxy_dn (xxx.xxx.xxx.xxx) port 8200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=proxy_dn
*  start date: Sep 15 08:30:43 2020 GMT
*  expire date: Dec 14 08:30:43 2020 GMT
*  subjectAltName: host &quot;proxy_dn&quot; matched cert's &quot;proxy_dn&quot;
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* Using Stream ID: 1 (easy handle 0x55c06f25e580)
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
&gt; GET /v1/sys/health HTTP/2
&gt; Host: proxy_dn:8200
&gt; User-Agent: curl/7.58.0
&gt; Accept: */*
&gt; 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (21):
* TLSv1.3 (IN), TLS alert, Client hello (1):
* Unexpected EOF
* Connection #0 to host proxy_dn left intact
curl: (56) Unexpected EOF

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

						kosmas:$ curl -v https://proxy_dn:8200/v1/sys/health
*   Trying xxx.xxx.xxx.xxx...
* TCP_NODELAY set
* Connected to proxy_dn (xxx.xxx.xxx.xxx) port 8200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=proxy_dn
*  start date: Sep 15 08:30:43 2020 GMT
*  expire date: Dec 14 08:30:43 2020 GMT
*  subjectAltName: host &quot;proxy_dn&quot; matched cert's &quot;proxy_dn&quot;
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* Using Stream ID: 1 (easy handle 0x55c06f25e580)
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
&gt; GET /v1/sys/health HTTP/2
&gt; Host: proxy_dn:8200
&gt; User-Agent: curl/7.58.0
&gt; Accept: */*
&gt; 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (21):
* TLSv1.3 (IN), TLS alert, Client hello (1):
* Unexpected EOF
* Connection #0 to host proxy_dn left intact
curl: (56) Unexpected EOF

					

			

on an external client (not HAProxy, or Vault).

the following on the HAProxy log:

Sep 25 13:07:27 localhost haproxy[33169]: xxx.xxx.xxx.xxx:38196 [25/Sep/2020:13:07:26.985] vault_https~ vault_https_backend/vault1 46/1/57 0 -- 1/1/0/0/0 0/0

				1

						Sep 25 13:07:27 localhost haproxy[33169]: xxx.xxx.xxx.xxx:38196 [25/Sep/2020:13:07:26.985] vault_https~ vault_https_backend/vault1 46/1/57 0 -- 1/1/0/0/0 0/0

and the following on the Vault log

Sep 25 13:07:27 vault1 vault[607]: 2020-09-25T13:07:27.031Z [INFO]  http: TLS handshake error from xxx.xxxx.xxxx.xxxx(proxy_ip):56632: tls: first record does not look like a TLS handshake

				1

						Sep 25 13:07:27 vault1 vault[607]: 2020-09-25T13:07:27.031Z [INFO]  http: TLS handshake error from xxx.xxxx.xxxx.xxxx(proxy_ip):56632: tls: first record does not look like a TLS handshake

Changing the configuration on HAProxy to use http1.1 solves the issue (/etc/haproxy/haproxy.cfg

frontend vault_https
  bind *:443 ssl crt /etc/haproxy/certs/proxy_dn_certificate.pem alpn http/1.1

				1
2

						frontend vault_https
  bind *:443 ssl crt /etc/haproxy/certs/proxy_dn_certificate.pem alpn http/1.1

Which results in the correct response now by using the HAProxy as a ssl pass_through load balancer.

kosmas:$ curl -v https://proxy_dn:8200/v1/sys/health
*   Trying xxx.xxx.xxx.xxx...
* TCP_NODELAY set
* Connected to proxy_dn (xxx.xxx.xxx.xxx) port 8200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=proxy_dn
*  start date: Sep 15 08:30:43 2020 GMT
*  expire date: Dec 14 08:30:43 2020 GMT
*  subjectAltName: host &quot;proxy_dn&quot; matched cert's &quot;proxy_dn&quot;
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
&gt; GET /v1/sys/health HTTP/1.1
&gt; Host: proxy_dn:8200
&gt; User-Agent: curl/7.58.0
&gt; Accept: */*
&gt; 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
&lt; HTTP/1.1 200 OK
&lt; Cache-Control: no-store
&lt; Content-Type: application/json
&lt; Date: Fri, 25 Sep 2020 14:12:59 GMT
&lt; Content-Length: 294
&lt; 
{&quot;initialized&quot;:true,&quot;sealed&quot;:false,&quot;standby&quot;:false,&quot;performance_standby&quot;:false,&quot;replication_performance_mode&quot;:&quot;disabled&quot;,&quot;replication_dr_mode&quot;:&quot;disabled&quot;,&quot;server_time_utc&quot;:1601043179,&quot;version&quot;:&quot;1.5.0&quot;,&quot;cluster_name&quot;:&quot;vault-cluster-123455&quot;,&quot;cluster_id&quot;:&quot;xxxx-xxxx-xxxx-xxx-xxxx&quot;}
* Connection #0 to host proxy_dn left intact

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

						kosmas:$ curl -v https://proxy_dn:8200/v1/sys/health
*   Trying xxx.xxx.xxx.xxx...
* TCP_NODELAY set
* Connected to proxy_dn (xxx.xxx.xxx.xxx) port 8200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=proxy_dn
*  start date: Sep 15 08:30:43 2020 GMT
*  expire date: Dec 14 08:30:43 2020 GMT
*  subjectAltName: host &quot;proxy_dn&quot; matched cert's &quot;proxy_dn&quot;
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
&gt; GET /v1/sys/health HTTP/1.1
&gt; Host: proxy_dn:8200
&gt; User-Agent: curl/7.58.0
&gt; Accept: */*
&gt; 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
&lt; HTTP/1.1 200 OK
&lt; Cache-Control: no-store
&lt; Content-Type: application/json
&lt; Date: Fri, 25 Sep 2020 14:12:59 GMT
&lt; Content-Length: 294
&lt; 
{&quot;initialized&quot;:true,&quot;sealed&quot;:false,&quot;standby&quot;:false,&quot;performance_standby&quot;:false,&quot;replication_performance_mode&quot;:&quot;disabled&quot;,&quot;replication_dr_mode&quot;:&quot;disabled&quot;,&quot;server_time_utc&quot;:1601043179,&quot;version&quot;:&quot;1.5.0&quot;,&quot;cluster_name&quot;:&quot;vault-cluster-123455&quot;,&quot;cluster_id&quot;:&quot;xxxx-xxxx-xxxx-xxx-xxxx&quot;}
* Connection #0 to host proxy_dn left intact

					

			

SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch

Posted on 23 September 2020 by kosmas

Problem

Trying to configure and use an nginx server that uses intermediate certifcates, you get the error about values mismatch and nginx does not start

2020/09/23 09:49:38 [emerg] 20958#20958: SSL_CTX_use_PrivateKey(&quot;/etc/ssl/private/cakey.pem&quot;) failed (SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch)

				1

						2020/09/23 09:49:38 [emerg] 20958#20958: SSL_CTX_use_PrivateKey(&quot;/etc/ssl/private/cakey.pem&quot;) failed (SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch)

Solution

It seems that this is a common mistake and it is mentioned here: https://nginx.org/en/docs/http/configuring_https_servers.html. In order to fix this you will need to change/reverse the original order that you have concatenated the chain, as in:

cat certs/cacert.pem intermediate/certs/intermediate.cacert.pem &gt; intermediate/certs/ca-chain-bundle.cert_new.pem

				1

						cat certs/cacert.pem intermediate/certs/intermediate.cacert.pem &gt; intermediate/certs/ca-chain-bundle.cert_new.pem

Tail Vault auth logs

Posted on 21 September 2020 by kosmas

Problem

You would like to tail the Vault auth log files, but if you use the vault tag for auditing and your server is also named vault, you cannot filter out the vault auth logs only.

Solution

Use the client_login as the filter in grep:

ubuntu@vault1:~$ sudo tail -f /var/log/auth.log | grep 'client_token'

				1

						ubuntu@vault1:~$ sudo tail -f /var/log/auth.log | grep 'client_token'

Check haproxy configuration file for errors

Posted on 17 September 2020 by kosmas

Problem

You would like to check your haproxy.cfg file for errors when making changes.

Solution

Use the following command to validate/check your HAProxy configuration file:

ubuntu@haproxy:~$ sudo haproxy -c -V -f /etc/haproxy/haproxy.cfg 

				1

						ubuntu@haproxy:~$ sudo haproxy -c -V -f /etc/haproxy/haproxy.cfg