Deleting old releases from rails app server using puppet

Problem

You would like to automatically delete old releases from your rails app server using a puppet crontab resource.

Solution

Add the following to your modules/crontab/manifests/init.pp file and modify some of the values to what you need:

 # use // to suppress warning: Warning: Unrecognised escape sequence '\;'
  cron { remove_old_releases:
    command   => "test $(find /var/www/your/path/to/ror/app/ -maxdepth 1 -type d | wc -l) -gt 10 && find /var/www/your/path/to/ror/app/ -maxdepth 1 -type d -mtime +14 -exec rm -rf '{}' \\;",
    user      => ubuntu,
    hour      => 0,
    minute    => 15
  }

You should need to change the following:

/var/www/your/path/to/ror/app/ to your application’s release path
-gt 10 this is the number of minimum old releases you want to keep
-mtime +14 releases older than 14 days are deleted

UPDATE

There is chance that when you do not calculate the frequency of the releases right the above script can potentially delete all of your releases, so you end up with no releases 🙁

A better solution to keep the -n number of releases (based on the solution from here:

find /var/www/your/path/to/ror/app/* -type d -printf '%T@ %p\n' | sort -nr | tail -n+6 | cut -f 2- -d " "  | xargs -i rm -rf {}

and the explanation of each step:

  • find /var/www/your/path/to/ror/app/* -maxdepth 0 -type d -printf ‘%T@ %p\n’ find all the directories (-maxdepth 0 -type d) in the search path (/var/www/your/path/to/ror/app/*) excluding the . and .. directories (/*) and add to them the time information (‘%T %p\n’)
  • sort -nr sort them in numeric and reverse order – newer first, oldest last
  • tail -n+6 keep only the first 5 directories, starting from line 6 (tail -n+6)
  • cut -f 2- -d ” “ Remove the first field (ie the date information) keeping the file from the second field using the space as the delimiter (-d ” “)
  • xargs -i rm -rf {} Pass the rm -rf command to delete the directory for each line produced in the previous steps

So the puppet script should be as follows (including the escape characters \ for the % and the additional \):

 # use // to suppress warning: Warning: Unrecognised escape sequence '\;'
  cron { remove_old_releases:
    command   => "find /var/www/your/path/to/ror/app/* -maxdepth 0 -type d -printf '\\%T@ \\%p\\n' | sort -nr | tail -n+6 | cut -f 2- -d ' ' | xargs -i rm -rf '{}' \\;",
    user      => ubuntu,
    hour      => 0,
    minute    => 15
  }

Error in AWS when trying to include puppet modules (Error: Could not find class apache2 for …)

Problem

You are trying to work with puppet modules in AWS, after installing puppet as a gem, but when you try to put the modules and manifests outside the main manifests/site.pp file you get an error similar to following:

Error: Could not find class apache2 for ip-xx-xx-xx-xx.eu-west-1.compute.internal on node ip-xx-xx-xx-xx.eu-west-1.compute.internal

Solution

Create a file for the puppet configuration in /etc/puppet/puppet.conf and add the paths to your own manifests and modules, like :

$aws sudo vi /etc/puppet/puppet.conf

and add your paths:

[main]
manifest = /home/ubuntu/puppet/manifests/site.pp
modulepath = /etc/puppet/modules:/usr/share/puppet/modules:/home/ubuntu/puppet/modules

Can’t install RMagick 2.13.1. Can’t find Magick-config

Problem

You are trying to install rmagick in an Ubuntu 12.04 virtual box, but you get the following error:

Can't install RMagick 2.13.1. Can't find Magick-config

Solution

Make sure you install both the imagemagick and the libmagickwand-dev packages with either apt-get :

sudo apt-get install imagemagick libmagickwand-dev

or in a puppet script as:

package {
  "imagemagick":
    ensure => installed
}

package {
  "libmagickwand-dev":
    ensure  => installed
}

sudo: puppet: command not found – when trying to use puppet with rvm in aws

When trying to use the

rvmsudo puppet apply --verbose manifests/site.pp

command in a aws instance to follow the instruction from ‘Deploying Rails’, the following error complains that the puppet command is not found, like:

sudo: puppet: command not found

After some searching and having a look at the blog post here and using the rvm notes in the aws instance and looking at this:

  * On some systems (like Ubuntu) rvmsudo requires following changes to work properly:
      http://stackoverflow.com/questions/257616/sudo-changes-path-why

It seems that for security reason the sudo command resets the path in the system.

You can actually see that this the case by running the command to display the path of the normal user:

aws$ echo 'echo $PATH' | sh

and the corresponding one for the sudo:

aws$ echo 'echo $PATH' | sudo sh

So a workaround to make it work is to add your group into the exemptions group in the visudo file, which means editing the visudo file:

sudo visudo

and adding the group that your user you are working with belongs to:

Defaults        env_reset
Defaults        exempt_group=name_of_your_group(or admin in some systems)

Writing a Passenger Module for Puppet – Vagrant

Following the steps described in the “Deploying Rails Applications” book and using the currently latest puppet installation described here, the actual puppet files had to be modified slightly in order to make it work as expected.

The main issues requiring the changes were as follow:

  • System wide rvm installation – different paths, and the use of the rvmsudo command instead of sudo
  • Use of latest puppet gem (3.0.1) – ruby 1.9.3 is supported
  • Use of Debian/Ubuntu system – different way of configuring passenger with apache2

So the full file for the passenger module (puppet/modules/passenger/manifests/init.pp) is:

class passenger {

  # required for passenger gem
  package {
    "apache2-prefork-dev":
      ensure => installed
  }

  # required for passenger-apache-module
  package {
    "libcurl4-openssl-dev":
      ensure => installed
  }

  exec  {
    "gem install passenger -v=3.0.18":
      user    => vagrant,
      group   => vagrant,
      alias   => "install_passenger",
      require => Package["apache2"],
      unless  => "ls /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/bin/passenger",
      before  => Exec["passenger_apache_module"]
  }

  exec  {
    "passenger-install-apache2-module --auto":
      user    => root,
      group   => root,
      alias   => "passenger_apache_module",
      require => Package["apache2-prefork-dev","libcurl4-openssl-dev"],
      unless  => "ls /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/gems/passenger-3.0.18/ext/apache2/mod_passenger.so",
      before  => File["passenger_conf"]
  }

  file  {
    "/etc/apache2/mods-available/passenger.conf":
      mode    => 644,
      owner   => root,
      group   => root,
      alias   => "passenger_conf",
      source  => "/home/vagrant/puppet/modules/passenger/files/passenger.conf",
      before  => File["passenger_load"]
  }

  file {
    "/etc/apache2/mods-available/passenger.load":
      mode    => 644,
      owner   => root,
      group   => root,
      alias   => "passenger_load",
      notify  => Service["apache2"],
      source  => "/home/vagrant/puppet/modules/passenger/files/passenger.load",
      before  => Exec["enable_passenger"]
  }

  exec {
    "a2enmod passenger":
      user    => root,
      alias   => "enable_passenger"
  }
}

Going through the file one of the first things to notice is the declaration about the “apache2-prefork-dev” package installation

 package {
    "apache2-prefork-dev":
      ensure => "installed"
  }

The reason for this package, is that when trying to compile the passenger apache module (passenger-install-apache2-module), the script will complain that the following packages are missing:

apache2-prefork-dev
libapr1-dev
libaprutil1-dev

By installing the first of the packages (apache2-prefork-dev) the other two are installed automatically as dependencies.

NOTE In trying to install it in an Ubuntu AMI image in AWS EC2 there is also an additional package requirement because the installer complains about:

To install Curl development headers with SSL support:  Please run apt-get install libcurl4-openssl-dev or libcurl4-gnutls-dev, whichever you prefer.

So there is the additional package added as:

# required for the passenger_apache_module
package {
  "libcurl4-openssl-dev":
    ensure  => installed
}

Further down there are two file resources named “passenger_conf” and “passenger_load”.
Looking at the structure in debian for the apache2 configuration it seems that apache2 in debian needs two different files for each module that can be loaded one with named as module.conf and another as module.load to be placed into /etc/apache2/mods-available.

So we can create these two files and placed them in the files folder, as:

puppet/modules/passenger/files/passenger.conf


  PassengerRoot /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/gems/passenger-3.0.18
  PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rails3_2_9/ruby
  PassengerUseGlobalQueue on
  PassengerMaxPoolSize  5
  PassengerPoolIdleTime 900
  PassengerMaxRequests 10000

and, puppet/modules/passenger/files/passenger.load

LoadModule passenger_module /usr/local/rvm/gems/ruby-1.9.3-p327/gems/passenger-3.0.11/ext/apache2/mod_passenger.so

Which you can actually see if you compile manually the passenger apache2 module as they are described in the output on successful compilation:

The Apache 2 module was successfully installed.

Please edit your Apache configuration file, and add these lines:                                                                                                                                                                                

   LoadModule passenger_module /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
   PassengerRoot /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/gems/passenger-3.0.18
   PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rails3_2_9/ruby

After you restart Apache, you are ready to deploy any number of Ruby on Rails                                                                                                                                                                   
applications on Apache, without any further Ruby on Rails-specific                                                                                                                                                                              
configuration!                                                                                                                                                                                                                                  

Press ENTER to continue.

Finally the last section of the init.pp file enables the passenger module with the a2enmod apache command:

  exec {
    "a2enmod passenger":
      user    => root,
      alias   => "enable_passenger"
  }

A couple of useful apache2 commands when testing are:

  • To test what modules are availble for loading:
    a2enmod

    If run after creating the two passenger files passenger.conf and passenger.load, then the passenger module should appear in the list

  • To see the currently enabled modules:
    apache2ctl -M

    Which again if it is run after the passenger modules has been enabled with a2enmod passenger then it should show an entry with

    passenger_module (shared)

Could not find puppet (>= 0) amongst [] (Gem::LoadError)

Problem

When you try to apply the puppet configuration into a vagrant box and using systemwide rvm installation (if you follow the instructions here), you have the following error:

vagrant@precise64:~$ sudo puppet apply --verbose puppet/manifests/site.pp
/usr/local/rvm/rubies/ruby-1.9.3-p327/lib/ruby/site_ruby/1.9.1/rubygems/dependency.rb:247:in `to_specs': Could not find puppet (>= 0) amongst [] (Gem::LoadError)
from /usr/local/rvm/rubies/ruby-1.9.3-p327/lib/ruby/site_ruby/1.9.1/rubygems/dependency.rb:256:in `to_spec'
from /usr/local/rvm/rubies/ruby-1.9.3-p327/lib/ruby/site_ruby/1.9.1/rubygems.rb:1231:in `gem'
from /usr/local/rvm/gems/ruby-1.9.3-p327/bin/puppet:18:in `
' from /usr/local/rvm/gems/ruby-1.9.3-p327/bin/ruby_noexec_wrapper:14:in `eval' from /usr/local/rvm/gems/ruby-1.9.3-p327/bin/ruby_noexec_wrapper:14:in `
'

Solution

Use rvmsudo instead of sudo to make sure that the paths are correct:

vagrant@precise64:~$ rvmsudo puppet apply --verbose puppet/manifests/site.pp