Category Archives: DevOps

Connection refused when using http monitoring with blackbox_exporter and prometheus

Posted on by
Reply

Problem

You are trying to set up http endpoint monitoring with prometheus and blackbox_exporter by specifying something like the following:

...
 job_name: 'blackbox'
    metrics_path: /probe
    params:
      module: [http_2xx]
    static_configs:
      - targets: 
        - 'localhost:8080'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 127.0.0.1:9115
...

but you are getting the error (in prometheus targets) that connection was refused.

Solution

First specify the ip protocol as ip4 if you are not using ip6 like:

modules:
  http_2xx:
    prober: http
    timeout: 5s
    http:
      preferred_ip_protocol: "ip4"
      valid_status_codes: [200]
      method: GET

And then make sure that you use the container IP address in the replacement field if you have started blackbox_exporter as a docker container (tip taken from here), which you can find by looking for the container IP address (docker inspect blackbox_exporter | grep IPAddress).

So your prometheus configuration should look like the following (different replacement IP):

 job_name: 'blackbox'
    metrics_path: /probe
    params:
      module: [http_2xx]
    static_configs:
      - targets: 
        - 'localhost:8080'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 172.17.0.3:9115

Getting the slug name for Digital Ocean’s images with doctl

Posted on by
Reply

Problem

You would like to get a list of the available images in Digital Ocean, in order to be able to use them in creating your Terraform IAC script.

Solution

By installing the command line tool doctl, (instructions here), according to the documentation (after authenticating) you can run:

doctl compute image list

But the available images are not listed. In order to be list all the images you have to add the –public option (as described here) :

doctl compute image list --public

Bash script to add environment and date in Symfony appversion.yml file

Posted on by
Reply

Problem

You are using CI/CD to deploy a Symfony application that has the appversion.yml configuration file, and you want to add the environment and a timestamp to it after deployment.

Solution

You can achieve this by creating and running a bash script like the following and passing the environment as a variable (ie script_name env) :

#!/bin/bash
# Script for adding the environment and time in deployment
# Needs the environment as a parameter
# DATE_CREATED=2017.06.07
# DATE_UPDATED=2017.06.08
# VERSION=1.01

CONF_PATH=/var/www/analyse/app/config

sed '/app.version/s/\(.*$\)/\1'" (${1}) $(date +%Y%m%d-%H%M)"'/g' $CONF_PATH/appversion.yml > $CONF_PATH/appversion_new.yml
mv $CONF_PATH/appversion.yml $CONF_PATH/appversion_bak.yml
mv $CONF_PATH/appversion_new.yml $CONF_PATH/appversion.yml

FATAL -> Failed to fork – Digital Ocean – Debian – Docker

Posted on by
Reply

Problem

When you are trying to install a package inside a debian Docker container you get the following error message:

FATAL -> Failed to fork.

Solution

This is an indication of ‘out of memory’ so add a swap partition if it does not exist, on the host with the following:

fallocate -l 4G /swapfile
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
echo "/swapfile swap swap sw 0 0" >> /etc/fstab

(Thanks Valent)

Docker – Debian – Kernel panick – Automatic restart

Posted on by
Reply

Problem

You have a remote server (Debian) running some Docker containers and you want to make sure that they all restart in the event of a kernel panick.

solution

  • Add an automatic restart after a kernel panick by running: 
    sysctl kernel.panic=20
  • Emulate a kernel panick by running the following (SysRq more info here : 
    echo c > /proc/sysrq-trigger
  • Check that the containers have restarted again with docker ps (NOTE: the containers should have been enabled to autorestart)

Remove line from bash history

Posted on by
Reply

Problem

You want to remove a certain line from your server’s bash history, if for example you pasted a password in the wrong place.

Solution

Use the two steps:

1. history -d line_number to remove from memory
2. history -w to write in-memory history to the history file ~/.bash_history

as described here

When the shell is not bash (sh) you can remove them from ~/.ash_history and then source the file.