You can use Chef recipes to customize your Engine Yard Cloud environments. You can edit cookbooks that are provided by Engine Yard or you can make your own cookbooks.
This page describes how to work with custom Chef recipes on the v2 and v4 versions of the EngineYard stack. For custom Chef on stack v5, please refer to this article
Contents:
- Setting up a Chef environment
- Cloning the ey-cloud-recipes repository
- The file structure of ey-cloud-recipes
- Turning on a cookbook
- Creating a cookbook
- Reporting to the dashboard from custom Chef recipes
- Reporting to a log file from custom Chef recipes
- Specifying which instance roles run a custom Chef recipe
Set up the Chef environment
In order to be able to develop Chef recipes that build (and rebuild each time you start an instance) your environment customizations, you need to install the engineyard gem in your local environment on your development machine.
Even if you already have the engineyard gem installed, it is good practice to run "install engineyard" to make sure that you have the latest version of the gem.
To install the engineyard gem
-
Type:
$ sudo gem install engineyard
-
When prompted, enter the password for your Engine Yard account.
Clone the ey-cloud-recipes repository
To work with the ey-cloud-recipes repository, you need to fork and clone it to your development environment on your local machine. Clone a local copy of the ey-cloud-recipes repository in the directory that you'll work in when writing custom Chef recipes for your environment (the same environment where you installed the engineyard gem).
Here is the standard procedure for cloning a GitHub repository:
To fork and clone the ey-cloud-recipes repository
- Browse to the ey-cloud-recipes site on GitHub.
- Click Fork to fork the repository.
This creates a fork under your user account on GitHub. - Copy the URL of your forked repository to your clipboard.
- Clone the repository to your local development machine:
Note: Unless you are using submodules in git, do not put the ey-cloud-recipes repository in the same directory as your application repository. By default, git does not support nesting.
About updating the ey-cloud-recipes repository
From time to time, you might want to or need to refresh your ey-cloud-recipes repository to keep up with changes made by Engine Yard. A good reason to refresh the repository is if something isn't working correctly or if there is a new feature that you want to take advantage of. You can review the GitHub Commit History for the repository to find out about recent changes.
Important! Be sure to test carefully after updating the repository and before applying new Chef recipes to a production environment.
About the file structure of ey-cloud-recipes
Become familiar with the file structure of the ey-cloud-recipes repository.
Base files and directories
README.md
Rakefile
cookbooks/
main/
attributes/
definitions/
libraries/
recipes/
Cookbooks directory
In the cookbooks directory are directories of the self-contained recipes for various "components" (such as Sphinx, MongoDB, Redis, Varnish) that you can enable and customize for your environment.
Each directory under a cookbook has a number of sub-directories. On this page, we use the Sphinx recipe as the example:
cookbooks/
sphinx/
files/
default/
sphinx.logrotate
recipes/
default.rb
templates/
default/
sphinx.monitrc.erb
sphinx.yml.erb
Recipes directory
For each cookbook `recipes/default.rb is the main definition file that prescribes how Chef performs each of its actions to achieve the customization. View the Sphinx example.
Files directory
In the Sphinx cookbook, the recipes/default.rb
creates a remote_file
resource (that's a Chef term). That resource is found in the files/default/sphinx.logrotate
location and corresponds to the source value in the code block below.
remote_file "/etc/logrotate.d/sphinx" do
owner "root"
group "root"
mode 0755
source "sphinx.logrotate"
backup false
action :create
end
The sphinx.logrotate
is a file that has no variables. You use the templates and ERB to insert the variable data into the sphinx.logrotate file.
Templates directory
Also in the the Sphinx cookbook, the recipes/default.rb
creates a template and passes variables to the template.
In the recipes/default.rb file:
template "/etc/monit.d/sphinx.#{app_name}.monitrc" do
source "sphinx.monitrc.erb"
owner node[:owner_name]
group node[:owner_name]
mode 0644
variables({
:app_name => app_name,
:user => node[:owner_name],
:flavor => flavor
})
end
The variables above are passed to the template (sphinx.monitrc.erb
), and Chef renders the static file (/etc/monit.d/sphinx.myapp.monitrc
).
In the sphinx.monitrc.erb file:
check process sphinx_<%= @app_name %>_3312
with pidfile /var/run/sphinx/<%= @app_name %>.pid
start program = "/engineyard/bin/<%= @flavor %>_searchd <%= @app_name %> start" as uid <%= @user %> and gid <%= @user %>
stop program = "/engineyard/bin/<%= @flavor %>_searchd <%= @app_name %> stop" as uid <%= @user %> and gid <%= @user %>
group sphinx_<%= @app_name %>
Turn on a cookbook
In order to turn on a cookbook and have that set of recipes run when you deploy your application, you need to uncomment the recipe in the cookbooks/main/recipes/default.rb
file.
To turn on an existing cookbook
-
Select the cookbook that you want to use.
-
In your local environment, open
cookbooks/main/recipes/default.rb
for editing. -
Uncomment the "include_recipe" line for the recipe that you want to turn on.
-
Follow any additional instructions in the comments.
For example, for Sphinx, you need to edit cookbooks/sphinx/recipes/default.rb; for details, see Implement full text search with Sphinx on Engine Yard Cloud. -
Save
cookbooks/main/recipes/default.rb
and commit your changes locally and push them to your remote repository. -
Use the ey recipes commands to upload and apply the recipes:
ey recipes upload -e environment_name
ey recipes apply -e environment_name
Create a cookbook
If there isn't a ready-made cookbook to suit your needs, you can create your own cookbook from scratch.
The procedure below shows how to create a new Chef recipe called nginx_logrotate. This recipe changes the retention time of Nginx logs from 30 days (the default) to 60 days.
To generate a new Chef recipe:
-
In your local environment, in the ey-cloud-recipes repository directory, type:
$ rake new_cookbook COOKBOOK=nginx_logrotate
This creates the file: cookbooks/nginx_logrotate/recipes/default.rb.
-
Edit cookbooks/nginx_logrotate/recipes/default.rb to add the following code:
remote_file "/etc/logrotate.d/nginx" do
owner "root"
group "root"
mode 0755
source "nginx.logrotate"
backup false
action :create
end -
Create a file called files/default/nginx.logrotate with the following content:
/var/log/engineyard/nginx/*.log {
daily
missingok
compress
rotate 60
dateext
notifempty
sharedscripts
extension gz
postrotate
[ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
endscript
} -
Edit cookbook/main/recipes/default.rb to enable the recipe:
include_recipe "nginx_logrotate"
-
Test the syntax of your new recipe:
$ rake test
-
Commit your changes locally and push them to your remote repository.
-
Use the ey recipes commands to upload and apply the recipes:
ey recipes upload -e environment_name
ey recipes apply -e environment_name
Report to the dashboard from custom recipes
You can have messages appear when your custom Chef recipes run. These appear on the Environment page under the "Instances" heading; and allow you to see when the custom portions of your Chef recipes are running. These messages also appear in the chef.custom.log file. (See Report to a log file from custom Chef recipes below.)
To report to the dashboard from custom Chef recipes
-
Edit the Ruby file (for example, recipes/default.rb) file with code like this:
ey_cloud_report "recipe_name" do
message "message text"
endWhere more message text is the message that you want to appear on dashboard when that part of the code is executed.
For example:
ey_cloud_report "nginx" do
message "custom logrotate for nginx"
end
There is an example of the ey_cloud_report method in the Sphinx recipe.
Report to a log file from custom Chef recipes
Custom Chef recipes are logged to /var/log/chef.custom.log. (Default Engine Yard Cloud recipes are logged to /var/log/chef.main.log.)
To report to a log file from custom Chef recipes
-
Edit the RB file (for example, recipes/default.rb) file with code like this:
Chef::Log.info "message text"
For example:Chef::Log.info "Doing step 1."
writes a line like this:[Sun, 22 Jan 2012 22:29:00 +0000] INFO: Doing step 1.
to the /var/log/chef.custom.log file.
Specify which instance roles run a recipe
In a clustered environment, you have multiple instances, each instance playing a different role. In most cases, you want the recipe to run on only one type of instance; for example, to run on the application master, but not on the application slave, database, or utility instances.
To specify which instance (role) runs a recipe
- Add an if statement around the recipe code:
if node[:instance_role] == 'instance_role'
Where instance_role is one of the following:-
app_master
(for the application master) -
app
(for an application slave) -
solo
(for a single instance) -
db_master
(for a database master) -
db_slave
(for a database slave) -
util
(for a utility instance)
-
Examples
In the ssh_tunnel recipe, to have the recipe run on only the application master in a clustered environment:
if node[:instance_role] == 'app_master'
...
end
However, to have the same recipe run in either a clustered environment or a single-instance environment, write the condition this way:
if ['app_master', 'solo'].include?(node[:instance_role])
...
end
About utility servers
If you want a recipe to run on a particular utility server, you can specify it by name instead of by instance role.
For example,
if node[:name] == 'myutility'
...
end
Where myutility is the name of a utility instance.
More information
For more information about... | See... |
---|---|
the engineyard gem | Engine Yard CLI User Guide. |
the ey recipes commands | Engine Yard CLI User Guide. |
running custom Chef recipes during deploy | Custom Chef Recipes During Deploy blog by Dr. Nic Williams. |
If you have feedback or questions about this page, add a comment below. If you need help, submit a ticket with Engine Yard Support.
Comments
Article is closed for comments.