Image for post
Image for post

Responding to typical breaches on AWS

Alt-Title: “Your AWS account has been mining bitcoin!”

You have received a scary email from AWS Abuse. Common subject lines may include:

  • “Your instances have been port scanning remote hosts on the Internet”
  • “Your AWS account is compromised”

You may have leaked an AWS key.

Leaked Root or IAM credentials are extremely common and are likely a top cause of AWS abuse emails. Here are a few occurrences of this in the wild, as examples:

You may have server exposed with a widely known vulnerability.

If IAM credentials were strictly not involved in the breach, another common scenario may be exploitation of servers you have exposed to the internet. A significant amount of exploitation on the internet is automated, scanning the internet for known vulnerabilities to exploit.

  • What servers are running? Can they be reached by anyone on the internet?
  • If unknown, can you identify these exposed services with an nmap scan?
  • Search the version number of these exposed applications and see if there are there known vulnerabilities, or patches for security issues.
  • Can you reduce your servers exposure to the internet, if they aren’t publicly used?
  • Are these services misconfigured? (IE, a wide open HTTP proxy, an SMTP open relay, or something usable for DNS/NTP amplification attack?)

Collect information in case abuse within your account resurfaces.

The above scenarios are common, but not exhaustive. You’ll want to enable the following configurations to make future troubleshooting much easier.

  • VPC Flowlogs will help you troubleshoot unknown traffic coming to / from your EC2 hosts that might resemble the abuse. You’ll want to plan for, and monitor costs in CloudWatch as this can get noisy.
  • The CloudWatch Logs agent can help you centralize /var/log/syslog and troubleshoot malicious access from abused SSH credentials, abuse of sudo, or other unknowns.

Discovering a breach does not exclude other breaches.

When you have discovered the root cause of an issue, it’s important to consider that you might not have a lone adversary who discovered and exploited it. In fact, these class of issue can attract multiple attackers due to their ease of execution.

When in doubt, blow it all up.

Sometimes it can be too expensive or time consuming to completely understand a root cause and a scorched earth approach is your only option.

Conclusion

AWS key leakages and internet exposed services with remote code execution are responsible for the most abuse email type problems I get involved with.

@magoo

I write incident response stuff on Medium.

Written by

Writing about risk, security, and startups.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store