Rails applications in Engine Yard utilize utility instances for additional services or compute power. This article provides an introduction to utility instances, their roles and types, directory structure, and the location of main logs for investigation.
- The utility instance
The utility instance
Utility instances in Engine Yard provide either additional infrastructure or compute power to the application. Like application instances, the codebase is also deployed in utility instances; this is because their work is related to application code: either use the code directly or provide a related service.
While, theoretically, any required application can be deployed in a utility instances, there are some prebuilt options, which differ depending on the stack:
Deployed through environment variables in v6 and v7:
Deployed through recipes in V7:
- Resque Scheduler
- Delayed Job
v6 offers everything in v7 plus:
- Thinking Sphinx
v5 offers all of the above recipes, plus:
Another important way to split the supported apps is between services and compute:
- Services: Redis, Memcached, Elasticsearch, Thinking Sphinx, Tinyproxy, and Solr.
- Compute: Sidekiq, Resque, Resque Scheduler, and Delayed Job.
When deployed by Engine Yard, the applications listed above are managed similarly to an application server in the application instance:
- They are managed by Monit or systemd (Sidekiq, Redis, and Memcached in v7 only).
- All have a control script that manages the application.
- Chef will install and overwrite configuration files under /data/<app_name>/shared/config
Note: Customers may run anything else that they wish, such as Mongo, any Python apps, Docker, etc. These are considered not to be in the scope of support. While we can provide basic help, it is suggested that PS services are offered and escalation considered.
In addition, the two following utility instance types are considered PS customizations and issues should be escalated accordingly:
- IDS (Wazuh)
In Engine Yard, the following directories will normally need to be accessed when investigating and addressing issues with utility instances:
Utility application deployment: /data/<app_name>/
The application deployment directory is used in the deployment process, and contains the following relevant elements:
|Subdirectories & files
|Contains the directories for successful deployments made on the instance.
|Contains the directories for failed deployments made on the instance.
|It's a symlink to the current 'release' folder.
|Contains the deploy hooks, which allow for customizing the deployment process and are managed by the customer.
|Contains files used by all versions of the code, to prevent duplication. These are managed by Chef.
|Contains shared config files. Env files within the folder contain the environmental variables set in the Engine Yard Web UI.
|Symlink identical to that in /data/<app_name>/current/log/, pointing at /var/log/engineyard/apps/<app_name>.
|Used by the Rails app to store temporary files.
|Contains the control files used by the deployment process.
|Has the PIDs of the applications being managed by Monit.
|Contains a copy of the codebase, so pulling the code from GitHub does not require downloading all files during deployment.
|Contains the installed Ruby libraries ("Gems"), so that only new and modified files are installed on deployment.
Other directories and files
|Directories & files
|Contains the systemd configuration files. systemd is used in place of Monit in newer versions of the stack, and is a native OS utility.
|Contains the relevant Monit configuration files relevant to Engine Yard.
|Contains service initialization scripts for Nginx, MySQL, etc.
Contains the installed cookbooks.
In the v4 stack, etc/chef-custom/recipes exists as well for custom recipes.
This file contains all information about the instance type and use:
Contains configuration files:
|Contains the control scripts for application servers.
The existing flows and applications in the EY stack log their operations and errors to a variety of logs. This is a list of the logs that should be accessed during troubleshooting:
Logs related to the deployment of an application's codebase to the instance.
Contains the last deployment log; for older logs, the Engine Yard web UI must be used instead.
The application master contains all deploy-related logs, while other applications only contain the deploy hooks.
Configuration & Chef-related logs
Logs related to instance configuration and Chef runs.
|v6 & v7
Note: Chef runs work slightly differently depending on the version of the stack:
- v4: AWSM SSHs into the instance and executes Enzyme, and Enzyme runs Chef.
- v5: AWSM SSHs into the instance and runs the EY Primer, Primer runs Enzyme, and Enzyme runs Chef.
- v6 & v7: AWSM SSHs into the instance and runs MNML, MNML runs Enzyme, and Enzyme runs Chef.
Utility application logs
Logs related to the execution of the utility application and the relevant web stack.
Contains logs for the rails application under <environment_framework>.*.log.
Contains, potentially, logs used by the application: Passenger, PHP-FPM, etc.
Other logs concerning OS operations.
Lists all activities from the monitoring and alerting stack, corresponding to the alerts shown in the web UI.
collectd can be configured as required.
|Internal Monit logs, showing its actions.
|OS log with info from applications run in the user's space. While Engine Yard rarely uses it directly, applications in utility instances and other apps may write to it.
|OS-related logs, logging information for cronjobs, collectd errors, Monit activities, kernel messages, etc.
|Shows user access to the system, including all SSH attempts.
This article provides an overview of utility instances in Engine Yard, their roles, types, and directory structure. It also provides a guide on how to work with these instances and the location of the main logs for investigation.