You may encounter an error message that indicates "No space left on device" even if, upon further investigation, you determine that there is more than enough free space.
The main goal of this article is to discuss that the error message can be misleading considering that the issue may be related to a lack of available inodes on the disk, rather than actual disk space. Inodes are data structures used by Linux file systems to store metadata information about files and directories, such as permissions, ownership, and timestamps. Each file or directory on a Linux file system consumes one inode. If the file system runs out of available inodes, it can cause errors like the one you encountered.
While this article's overview assumes you already found the underlying issue, here are some commands that might help you. We will assume that the error is related to the /data volume so that more specific commands can be shared.
Verify disk space usage
df -h /data
Check inode usage
df -i /data
Display the inode count and directory path for the 10 directories with the highest inode counts sorted in descending order.
du -i /data | sort -rn | head -n 10
You needed to identify and remove files or directories that were consuming excessive inodes on the disk. Once identified, you can remove unnecessary files or directories using commands like
Also, cache files are often the culprit of unexpected Inode overloads. So, for example, if there is an issue with the cache used by a Rails application, the
Rails.cache.clear command can be used to clear the errors while you investigate the application.
Alternatively, you may consider increasing the instance's volume since space increments will also increase the number of available Inodes. However, this should not be your first resource.