As per this advisory:
Netflix has identified several TCP networking vulnerabilities in FreeBSD and Linux kernels.
The vulnerabilities specifically relate to the minimum segment size (MSS) and TCP Selective Acknowledgement (SACK) capabilities. The most serious, dubbed “SACK Panic,” allows a remotely-triggered kernel panic on recent Linux kernels.
The ones currently affecting Engineyard's instances are CVE-2019-11477: SACK Panic, CVE-2019-11478: SACK Slowness and CVE-2019-11479: Excess Resource Consumption Due to Low MSS Values.
In this article you will find information about:
AWS Action
As per the security bulletin: https://aws.amazon.com/security/security-bulletins/AWS-2019-005/
Linux-based EC2 instances using Elastic Load Balancing (ELB) Classic Load Balancers, Application Load Balancers, or Network Load Balancers with TLS Termination (TLS NLB) do not require any customer action. ELB Classic and ALB will filter incoming traffic to mitigate any potential DoS concerns of these issues.
Engine Yard recommends applying the fixes for each case below since lots of customers are using HAproxy instead of xLBs or no load balancer at all.
V5 Stack Instructions
Update 18/7/2019: Stack stable-v5-3.0.61 comes with kernel 4.4.182 which fixes these vulnerabilities. Consider replacing your instances with new ones using this stack version.
CVE-2019-11477
To address CVE-2019-11477 on our V5 stack we have introduced an overlay of our sysctl recipe disabling SACK using workaround #2 (Our V4 stack is currently being worked on and this thread will be receiving an update when the work has completed).
More information on how overlay recipes work can be found here.
To address the issue follow the steps outlined below:
- Grab `tune.rb` file from V5 Cookbooks repository: https://github.com/engineyard/ey-cookbooks-stable-v5/blob/next-release/cookbooks/sysctl/recipes/tune.rb
- At the root of your cookbooks directory issue the following:
mkdir -p sysctl/recipes
- Copy `tune.rb` file that you grabbed on previous step under `sysctl/recipes/tune.rb` and edit it:
vi sysctl/recipes/tune.rb
- add the following lines at the end of file
sysctl "net.ipv4.tcp_sack" do
variables 'net.ipv4.tcp_sack' => 0
only_if "sysctl -a 2>/dev/null | grep 'net.ipv4.tcp_sack = 1'"
end
- upload and apply the recipes. To verify that the workaround is currently in place issue the following which should return a value set to 0 (zero):
sudo sysctl -a 2>/dev/null | grep 'net.ipv4.tcp_sack'
CVE-2019-11478
CVE-2019-11478 is addressed using the steps outlined above. If you have already followed those you have successfully addressed this one also.
CVE-2019-11479
To address CVE-2019-11479 we have introduced a custom chef recipe that uses iptables to limit MSS's lowest size. To accomplish this follow the steps outlined below:
- At the root of your cookbooks directory issue the following:
mkdir -p custom-mss/recipes
vi custom-mss/recipes/default.rb
- add the following lines:
#
# Cookbook Name:: custom-mss
# Recipe:: default
#
execute "block low MSS using iptables" do
command "iptables -A INPUT -p tcp -m tcpmss --mss 1:500 -j DROP"
action :run
not_if "iptables -C INPUT -p tcp -m tcpmss --mss 1:500 -j DROP"
end
- continue by editing custom-mss's metadata.rb
vi custom-mss/metadata.rb
- add the following line:
name 'custom-mss'
- continue by editing ey-custom's metadata.rb
vi ey-custom/metadata.rb
- add the following line:
depends 'custom-mss'
- finally edit after-main.rb
vi ey-custom/recipes/after-main.rb
- add the following line:
include_recipe 'custom-mss'
- upload and apply recipes. To verify that the workaround has been successfully applied issue the following on any of the instances:
sudo iptables -S
- it should return the following line amongst others:
-A INPUT -p tcp -m tcpmss --mss 1:500 -j DROP
V4 Stack Instructions
CVE-2019-11477
To address CVE-2019-11477 on our V4 stack we have introduced a custom sysctl recipe disabling SACK using workaround #2
To address the issue follow the steps outlined below:
- At the root of your cookbooks directory issue the following:
mkdir -p custom_sysctl/recipes
vi custom_sysctl/recipes/default.rb
- add the following lines
#
# Cookbook Name:: sysctl
# Recipe:: default
#
# There are some kernel parameters that default to values which are less than
# ideal for our typical use cases. This recipe is used to tune those defaults.
#
sysctl "net.ipv4.tcp_sack" do
variables 'net.ipv4.tcp_sack' => 0
only_if "sysctl -a 2>/dev/null | grep 'net.ipv4.tcp_sack = 1'"
end
- edit your main/recipes/default.rb
vi main/recipes/default.rb
- add the following lines at the end of the file:
#uncoment to include the custom_sysctl recipe
include_recipe "custom_sysctl"
- upload and apply the recipes. To verify that the workaround is currently in place issue the following which should return a value set to 0 (zero):
sudo sysctl -a 2>/dev/null | grep 'net.ipv4.tcp_sack'
CVE-2019-11478
CVE-2019-11478 is addressed using the steps outlined above. If you have already followed those you have successfully addressed this one also.
CVE-2019-11479
To address CVE-2019-11479 we have introduced a custom chef recipe that uses iptables to limit MSS's lowest size. To accomplish this follow the steps outlined below:
- At the root of your cookbooks directory issue the following:
mkdir -p custom_mss/recipes
vi custom_mss/recipes/default.rb
- add the following lines:
#
# Cookbook Name:: custom_mss
# Recipe:: default
#
execute "block low MSS using iptables" do
command "iptables -A INPUT -p tcp -m tcpmss --mss 1:500 -j DROP"
action :run
not_if "iptables -C INPUT -p tcp -m tcpmss --mss 1:500 -j DROP"
end
- edit your main/recipes/default.rb
vi main/recipes/default.rb
- add the following lines at the end of the file:
#uncoment to include the custom_mss recipe
include_recipe "custom_mss"
- upload and apply recipes. To verify that the workaround has been successfully applied issue the following on any of the instances:
sudo iptables -S
- it should return the following line amongst others:
-A INPUT -p tcp -m tcpmss --mss 1:500 -j DROP
Comments
Article is closed for comments.