Overview

Engine Yard, a Rails application framework, operates with the concept of database instances for its applications. Understanding the key components of these instances and knowing where to find the necessary information is crucial for addressing any issues related to this type of item.

This article serves as a guide to understanding the roles, types, directory structure, and main logs of DB instances.

Contents:

The database instance
- Important directories
- Relevant logs
Summary

The database instance

Database instances in Engine Yard work as the database backend of the Rails application that is deployed, and have both a DB engine and the required DB management tools, which include backup, monitoring, and alerting tools.

Some notable characteristics are:

They run an SQL DB engine, which can come in PostgreSQL and MySQL flavors.
Unlike application and utility instances, the codebase from GitHub is not deployed in the DB instance.
Instances can run master or slave DBs for replication and takeover. Only one master exists per application.
For environments with multiple applications, the DB will contain a separate schema per application.

All the information provided in this article applies to the EC2 version of the DB instance. Please refer to Working with Amazon RDS in Engine Yard for information specific to RDS.

Important directories

When working with Engine Yard, the following directories will normally need to be accessed when investigating and addressing issues in the database instance:

MySQL directories & files

The following directories and files are relevant to a MySQL deployment of the DB instance.

Subdirectories & files	Description
/db/mysql/<engine_version>/data	Data directory of the DB deployment.
/etc/mysql/my.conf	Config file of the DB engine.
/etc/mysql/percona-server.conf	Additional config files. Percona is the distribution of MySQL that Engine Yard deploys.

PostgreSQL directories & files

The following directories and files are relevant to a PostgreSQL deployment of the DB instance.

Subdirectories & files	Description
/db/posgresql/<engine_version>/data	Data directory of the DB deployment.
/db/posgresql/<engine_version>/data/postgresql.conf	Config file of the DB engine.
/db/posgresql/<engine_version>/custom_pg_hba.conf	File for configuring the allowed network connections to the DB. Will need to be modified to allow for additional connections.

Other directories and files

Directories & files	Description
/lib/systemd/system/	Contains the systemd configuration files. systemd is used in place of Monit in newer versions of the stack, and is a native OS utility.
/etc/monit.d/	Contains the relevant Monit configuration files relevant to Engine Yard.
/etc/init.d/	Contains service initialization scripts for Nginx, MySQL, etc.
/etc/chef/recipes/	Contains the installed cookbooks. In the v4 stack, etc/chef-custom/recipes exists as well for custom recipes.
/etc/chef/dna.json	This file contains all information about the instance type and use: When the instance is created or 'Apply' is clicked, triggering a Chef run, AWSM creates a new version of the file based on the existing configuration and pushes it to the instance. The file contains data such as instance role, environment type, Ruby version, etc. Variables contained here are used by Enzyme as an input when running Chef.
/etc/engineyard/	Contains configuration files: dracul.yml: Contains the URLs from which recipes are downloaded, among other data. instance_api.yml: Contains an instance ID and token for talking back to AWSM. Used by collectd to send alerts to the UI. collectd.conf: Contains the collectd configuration file.
/engineyard/bin/	Contains the control scripts for DB tools.

Relevant logs

The DB engine and other applications in the EY stack log their operations and errors to a variety of logs. This is a list of the logs that should be accessed during troubleshooting:

MySQL logs

Logs specific to the MySQL DB engine.

Subdirectories & files	Description
/db/mysql/<engine_version>/log	Log directory of the DB deployment.
mysql.err	DB engine logs.
slow_query.log	Log with slow queries. By default, this means queries over 2 seconds. While it can be reduced temporarily, it is recommended that changes are reverted, as this may cause performance issues.

PostgreSQL logs

Logs specific to the PostgreSQL DB engine.

Subdirectories & files	Description
/db/postgresql/<engine_version>/data/pg_log	CSV log with engine logging and slow queries.

Configuration & Chef-related logs

Logs related to instance configuration and Chef runs.

Version	Location
v4	/var/log/chef.main.log /var/log/chef.custom.log /var/log/enzyme.log
v5	/var/log/chef.log /var/log/ey-enzyme.log /var/log/ey-primer.log
v6 & v7	/var/log/chef.log /var/log/ey-enzyme.log /var/log/ey-mnml.*.log

Note: Chef runs work slightly differently depending on the version of the stack:

v4: AWSM SSHs into the instance and executes Enzyme, and Enzyme runs Chef.
v5: AWSM SSHs into the instance and runs the EY Primer, Primer runs Enzyme, and Enzyme runs Chef.
v6 & v7: AWSM SSHs into the instance and runs MNML, MNML runs Enzyme, and Enzyme runs Chef.

System logs

Other logs concerning OS operations.

Location	Description
/var/log/collectd.log	Lists all activities from the monitoring and alerting stack, corresponding to the alerts shown in the web UI. collectd can be configured as required.
/var/log/monit.log	Internal Monit logs, showing its actions.
/var/log/messages.log	OS log with info from applications run in the user's space. While Engine Yard rarely uses it directly, applications in utility instances and other apps may write to it.
/var/log/syslog	OS-related logs, logging information for cronjobs, collectd errors, Monit activities, kernel messages, etc.
/var/log/auth.log	Shows user access to the system, including all SSH attempts.
/var/log/engineyard	Secondary system-related files and logs

Summary

This article provides an overview of Engine Yard DB instances, including their roles, types, and where to find necessary information. Understanding these aspects is crucial for troubleshooting and effectively working with these instances.

Database Instance Cheat Sheet