Adding a LOKI report in Grafana counting number of http_user_agents from NGINX logs

To add a Loki report in Grafana, as for example having a count of the different http_user_agents that have logged in, you can use the following query:

count by (http_agent) (rate({namespace="ingress-nginx",stream="stdout"} |= "https://domain.name.com/session/new" |~ "GET\\s/\\s" | pattern "<ip> - - [<timestamp>] \"<method> <path> <version>\" <result> <_> \"<url>\" \"<http_agent>\" <_>" [$__interval]))

You select the source (Loki) and then use the {namespace=”ingress-nginx”,stream=”stdout”} to get the log files from Nginx.

Then you can filter by two conditions:

  1. the instance login
|= "https://domain.name.com/session/new"

and

  1. use the regular expression to only select the GET / calls
|~ "GET\\s/\\s"

Then you can use the pattern parser for the log file in order to get the label

pattern "<ip> - - [<timestamp>] \"<method> <path> <version>\" <result> <_> \"<url>\" \"<http_agent>\" <_>"

<_> is used if we are not interested in getting the field, so everything after http_agent is not analyzed in this case.

Finally use the

count by(http_agent) 

to get the data (note that you also need to use the rate with the [$__interval])

SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch

Problem

Trying to configure and use an nginx server that uses intermediate certifcates, you get the error about values mismatch and nginx does not start

2020/09/23 09:49:38 [emerg] 20958#20958: SSL_CTX_use_PrivateKey("/etc/ssl/private/cakey.pem") failed (SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch)

Solution

It seems that this is a common mistake and it is mentioned here: https://nginx.org/en/docs/http/configuring_https_servers.html. In order to fix this you will need to change/reverse the original order that you have concatenated the chain, as in:

cat certs/cacert.pem intermediate/certs/intermediate.cacert.pem > intermediate/certs/ca-chain-bundle.cert_new.pem

Using nginx to redirect to a different port

Problem

You have a docker container with an application running in a port different than port 80 (ie port 3000), and would like to access it without specifying the port, by using the domain name only.

Solution

Use the nginx proxy_pass as follows:

server {
  listen 80;

  server_name your_server_domain_name.com;
  location / {
    proxy_set_header  X-Real-IP $remote_addr;
    proxy_set_header  Host $http_host;
    proxy_pass        http://127.0.0.1:3000;
  }
}