Customize Your Environment with Chef Recipes

You can use Chef recipes to customize your Engine Yard Cloud environments. You can edit cookbooks that are provided by Engine Yard or you can make your own cookbooks.

This page describes how to work with custom Chef recipes on the v2 and v4 versions of the EngineYard stack.  For custom Chef on stack v5, please refer to this article

Contents:

Set up the Chef environment

In order to be able to develop Chef recipes that build (and rebuild each time you start an instance) your environment customizations, you need to install the engineyard gem in your local environment on your development machine.

Even if you already have the engineyard gem installed, it is good practice to run "install engineyard" to make sure that you have the latest version of the gem.

To install the engineyard gem

  1. Type:

    $ sudo gem install engineyard 
  2. When prompted, enter the password for your Engine Yard account.

Clone the ey-cloud-recipes repository

To work with the ey-cloud-recipes repository, you need to fork and clone it to your development environment on your local machine. Clone a local copy of the ey-cloud-recipes repository in the directory that you'll work in when writing custom Chef recipes for your environment (the same environment where you installed the engineyard gem).

Here is the standard procedure for cloning a GitHub repository:

To fork and clone the ey-cloud-recipes repository

  1. Browse to the ey-cloud-recipes site on GitHub.
  2. Click Fork to fork the repository.
    This creates a fork under your user account on GitHub.
  3. Copy the URL of your forked repository to your clipboard.
  4. Clone the repository to your local development machine:
    Note: Unless you are using submodules in git, do not put the ey-cloud-recipes repository in the same directory as your application repository. By default, git does not support nesting.

About updating the ey-cloud-recipes repository

From time to time, you might want to or need to refresh your ey-cloud-recipes repository to keep up with changes made by Engine Yard. A good reason to refresh the repository is if something isn't working correctly or if there is a new feature that you want to take advantage of. You can review the GitHub Commit History for the repository to find out about recent changes.

Important! Be sure to test carefully after updating the repository and before applying new Chef recipes to a production environment.

About the file structure of ey-cloud-recipes

Become familiar with the file structure of the ey-cloud-recipes repository.

Base files and directories

README.md 
Rakefile
cookbooks/
main/
attributes/
definitions/
libraries/
recipes/

Cookbooks directory

In the cookbooks directory are directories of the self-contained recipes for various "components" (such as Sphinx, MongoDB, Redis, Varnish) that you can enable and customize for your environment.

Each directory under a cookbook has a number of sub-directories. On this page, we use the Sphinx recipe as the example:

cookbooks/ 
sphinx/
files/
default/
sphinx.logrotate
recipes/
default.rb
templates/
default/
sphinx.monitrc.erb
sphinx.yml.erb

Recipes directory

For each cookbook `recipes/default.rb is the main definition file that prescribes how Chef performs each of its actions to achieve the customization. View the Sphinx example.

Files directory

In the Sphinx cookbook, the recipes/default.rb creates a remote_file resource (that's a Chef term). That resource is found in the files/default/sphinx.logrotate location and corresponds to the source value in the code block below.

remote_file "/etc/logrotate.d/sphinx" do 
owner "root"
group "root"
mode 0755
source "sphinx.logrotate"
backup false
action :create
end

The sphinx.logrotate is a file that has no variables. You use the templates and ERB to insert the variable data into the sphinx.logrotate file.

Templates directory

Also in the the Sphinx cookbook, the recipes/default.rb creates a template and passes variables to the template.

In the recipes/default.rb file:

template "/etc/monit.d/sphinx.#{app_name}.monitrc" do 
source "sphinx.monitrc.erb"
owner node[:owner_name]
group node[:owner_name]
mode 0644
variables({
:app_name => app_name,
:user => node[:owner_name],
:flavor => flavor
})
end

The variables above are passed to the template (sphinx.monitrc.erb), and Chef renders the static file (/etc/monit.d/sphinx.myapp.monitrc).

In the sphinx.monitrc.erb file:

check process sphinx_<%= @app_name %>_3312 
with pidfile /var/run/sphinx/<%= @app_name %>.pid
start program = "/engineyard/bin/<%= @flavor %>_searchd <%= @app_name %> start" as uid <%= @user %> and gid <%= @user %>
stop program = "/engineyard/bin/<%= @flavor %>_searchd <%= @app_name %> stop" as uid <%= @user %> and gid <%= @user %>
group sphinx_<%= @app_name %>

Turn on a cookbook

In order to turn on a cookbook and have that set of recipes run when you deploy your application, you need to uncomment the recipe in the cookbooks/main/recipes/default.rb file.

To turn on an existing cookbook

  1. Select the cookbook that you want to use.

  2. In your local environment, open cookbooks/main/recipes/default.rb for editing.

  3. Uncomment the "include_recipe" line for the recipe that you want to turn on.

  4. Follow any additional instructions in the comments.
    For example, for Sphinx, you need to edit cookbooks/sphinx/recipes/default.rb; for details, see Implement full text search with Sphinx on Engine Yard Cloud.

  5. Save cookbooks/main/recipes/default.rb and commit your changes locally and push them to your remote repository.

  6. Use the ey recipes commands to upload and apply the recipes:

    ey recipes upload -e environment_name 
    ey recipes apply -e environment_name

Create a cookbook

If there isn't a ready-made cookbook to suit your needs, you can create your own cookbook from scratch.

The procedure below shows how to create a new Chef recipe called nginx_logrotate. This recipe changes the retention time of Nginx logs from 30 days (the default) to 60 days.

To generate a new Chef recipe:

  1. In your local environment, in the ey-cloud-recipes repository directory, type:

    $ rake new_cookbook COOKBOOK=nginx_logrotate 

    This creates the file: cookbooks/nginx_logrotate/recipes/default.rb.

  2. Edit cookbooks/nginx_logrotate/recipes/default.rb to add the following code:

    remote_file "/etc/logrotate.d/nginx" do 
    owner "root"
    group "root"
    mode 0755
    source "nginx.logrotate"
    backup false
    action :create
    end
  3. Create a file called files/default/nginx.logrotate with the following content:

    /var/log/engineyard/nginx/*.log { 
    daily
    missingok
    compress
    rotate 60
    dateext
    notifempty
    sharedscripts
    extension gz
    postrotate
    [ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
    endscript
    }
  4. Edit cookbook/main/recipes/default.rb to enable the recipe:

     include_recipe "nginx_logrotate" 
  5. Test the syntax of your new recipe:

     $ rake test 
  6. Commit your changes locally and push them to your remote repository.

  7. Use the ey recipes commands to upload and apply the recipes:

        ey recipes upload -e environment_name 
    ey recipes apply -e environment_name

Report to the dashboard from custom recipes

You can have messages appear when your custom Chef recipes run. These appear on the Environment page under the "Instances" heading; and allow you to see when the custom portions of your Chef recipes are running. These messages also appear in the chef.custom.log file. (See Report to a log file from custom Chef recipes below.)

To report to the dashboard from custom Chef recipes

  1. Edit the Ruby file (for example, recipes/default.rb) file with code like this:

    ey_cloud_report "recipe_name" do 
    message "message text"
    end

    Where more message text is the message that you want to appear on dashboard when that part of the code is executed.

    For example:

    ey_cloud_report "nginx" do 
    message "custom logrotate for nginx"
    end

There is an example of the ey_cloud_report method in the Sphinx recipe.

Report to a log file from custom Chef recipes

Custom Chef recipes are logged to /var/log/chef.custom.log. (Default Engine Yard Cloud recipes are logged to /var/log/chef.main.log.)

To report to a log file from custom Chef recipes

  1. Edit the RB file (for example, recipes/default.rb) file with code like this:

    Chef::Log.info "message text" 
    For example:
    Chef::Log.info "Doing step 1." 
    writes a line like this:
    [Sun, 22 Jan 2012 22:29:00 +0000] INFO: Doing step 1. 

    to the /var/log/chef.custom.log file.

Specify which instance roles run a recipe

In a clustered environment, you have multiple instances, each instance playing a different role. In most cases, you want the recipe to run on only one type of instance; for example, to run on the application master, but not on the application slave, database, or utility instances.

To specify which instance (role) runs a recipe

  1. Add an if statement around the recipe code:
    if node[:instance_role] == 'instance_role' 
    Where instance_role is one of the following:
    • app_master (for the application master)
    • app (for an application slave)
    • solo (for a single instance)
    • db_master (for a database master)
    • db_slave (for a database slave)
    • util (for a utility instance)

Examples

In the ssh_tunnel recipe, to have the recipe run on only the application master in a clustered environment:

  if node[:instance_role] == 'app_master' 
...
end

However, to have the same recipe run in either a clustered environment or a single-instance environment, write the condition this way:

  if ['app_master', 'solo'].include?(node[:instance_role])
...
end

About utility servers

If you want a recipe to run on a particular utility server, you can specify it by name instead of by instance role.

For example,

  if node[:name] == 'myutility' 
...
end

Where myutility is the name of a utility instance.

More information

For more information about... See...
the engineyard gem                                                                          Engine Yard CLI User Guide.
the ey recipes commands Engine Yard CLI User Guide.
running custom Chef recipes during deploy Custom Chef Recipes During Deploy blog by Dr. Nic Williams.

If you have feedback or questions about this page, add a comment below. If you need help, submit a ticket with Engine Yard Support.

Comments

Article is closed for comments.