Tag Archives: bash

Print the total number of commits per day containing a pattern in git

Posted on by
Reply

Problem

You want to print a summary of commits in git that contain a certain pattern (‘number’) ordered by date and with the total number of commits per day

Solution

You can use the following for the patter ‘number’

git log --pretty=format:'%h %ad | %s%d [%an]' --date=short | grep 'number' | awk '{print $2}' | sort -r | uniq -c | awk '{ print $2,$1}'

The last awk command is to swap the date with the total number

The output should be something like the following

2024-06-18 2
2024-06-17 3
2024-06-14 1
2024-06-13 1
2024-06-12 1
2024-06-11 4
2024-06-10 3

Removing lines that are in fileB from fileA producing fileC

Posted on by
Reply

Problem

You have two different files fileA and fileB containing similar records (one record per line), and you would like to remove the records that fileB contains from fileA (subtract) producing a new file (fileC) that contains only records that are contained in fileA with records from fileB removed.

Solution

Use the following grep command

grep -vxf fileB fileA > fileC

asdf and terraform (or vault or packer)

Posted on by
Reply

Using the asdf version manager to manage versions of various binaries like terraform, vault or packer is easily done by following the instructions below:

  1. Download and install asdf if you haven’t got it already (https://asdf-vm.com/#/core-manage-asdf)
  2. If you want to install the latest version of terraform for example do the following:
asdf plugin add terraform
asdf install terraform latest
asdf global terraform 0.14.9
asdf list
terraform
  0.14.9
terraform --version
Terraform v0.14.9

Automatic emails using bash and sendgrid

Posted on by
Reply

Problem

You would like to be able to automatically send some emails (ie reports) from a linux server, using a bash script that runs in cronjob, without installing an email server on the linux server.

Solution

  • Create an account in Sendgrid and follow the directions for using WebAPI with curl
    here
  • Set up your bash script to use the API key and have a script like the following. If you want to use big or multiple files you will need to use a temporary file for the base64 encoding, as in the example below, as there is a limit in curl. 
    #!/bin/bash

# Email setup
SENDGRID_API_KEY="your_sendgrid_api_key"
FILENAME_ATTACH="title_of_your_attachment"
FILENAME_ZIP="the_path_to_your_zip_file"
FILENAME_BASE64_TMP="the_path_to_temporary_base64_encoding"
EMAIL_TO="email_to_address"
EMAIL_SUBJECT="email_subject"
EMAIL_FROM="email_from_address"
EMAIL_MESSAGE="your_email_message"

function email_exports()
{
  FILENAME_BASE64=$(base64 -w0 $FILENAME_ZIP);

  REQUEST_DATA='{"personalizations": [{
                        "to": [{ "email": "'"$EMAIL_TO"'" }],
                        "subject": "'"$EMAIL_SUBJECT"'"
                }],
                "from": {
                        "email": "'"$EMAIL_FROM"'"
                        },
                "content": [{
                        "type": "text/plain",
                        "value": "'"$EMAIL_MESSAGE"'"
                        }],
                "attachments": [{
                        "content": "'"$FILENAME_BASE64"'",
                        "filename": "'"$FILENAME_ATTACH"'"
                        }]
  }';

  # We need to store the base64 locally as the text
  # is too big for sending directly with curl
   echo $REQUEST_DATA > $FILENAME_BASE64_TMP

   curl -X "POST" "https://api.sendgrid.com/v3/mail/send" \
        -H "Authorization: Bearer $SENDGRID_API_KEY" \
        -H "Content-Type: application/json" \
        -d "@$FILENAME_BASE64_TMP";

   rm $FILENAME_BASE64_TMP
}

email_exports
  • Add your script to crontab

    • Bash getting the parent folder name from a bash defined variable

    Posted on by
    Reply

    Problem

    You have defined a variable in a bash script (ie MY_FOLDER) but you also want to dynamically get the parent folder (ie MY_FOLDER can have multiple values).

    Solution

    Use the dirname as in the following

    MY_FOLDER=/home/user/first_folder/second_folder
MY_PARENT_FOLDER=$(dirname $MY_FOLDER)

echo $MY_FOLDER
echo $MY_PARENT_FOLDER

.......
/home/user/first_folder/second_folder
/home/user/first_folder