A common concern of our customers using cloud platforms like AWS is the horror tales about a negligent (or uninformed) developer inadvertently exposing AWS API keys online, only for hackers to find those keys, penetrate the account and cause massive damage.
But how common are these breaches really? Are they a legitimate threat, or just an urban myth for sleep-deprived IT personnel? And what, if anything, can be done against such exposure?
How One AI-Driven Media Platform Cut EBS Costs for AWS ASGs by 48%
The Problem of API Access Key Exposure
The problem of AWS API access key exposure refers to incidents in which developer’s API access keys to AWS accounts and cloud resources are inadvertently exposed and found by hackers.
AWS – and most other infrastructure-as-as-service (IaaS) providers – provides direct access to tools and services via Advanced Programming Interfaces (APIs). Developers leverage such APIs to write automated scripts to help them configure cloud-based resources. This helps developers and DevOps save much time in configuring cloud-hosted resources and automating the roll-out of new features and services.
In order to make sure that only authorized developers are able to access those resources and execute commands on them, API access keys are used to authenticate access. Only code containing authorized credentials will be able to connect and execute.
It Happens All the Time
The problem, however, is that such access keys are sometimes left in scripts or configuration files uploaded to third-party resources, such as GitHub. Hackers are fully aware of this, and run automated scans on such repositories, in order to discover unsecured keys. Once they locate such keys, hackers gain direct access to the exposed cloud environment, which they use for data theft, account takeover, and resource exploitation.
A very common use case is for hackers to access an unsuspecting cloud account and spin-up multiple computing instances in order to run crypto-mining activities. The hackers then pocket the mined cryptocurrency, while leaving the owner of the cloud account to foot the bill for the usage of computing resources.
Examples, sadly, are abundant:
- A Tesla developer uploaded code to GitHub which contained plain-text AWS API keys. As a result, hackers were able to compromise Tesla’s AWS account and use Tesla’s resources for crypto-mining.
- WordPress developer Ryan Heller uploaded code to GitHub which accidentally contained a backup copy of the wp-config.php file, containing his AWS access keys. Within hours this file was discovered by hackers, who spun up several hundred computing instances to mine crypto-currency, resulting in $6,000 of AWS usage fees overnight.
- A student taking a Ruby on Rails course on Udemy opened up a AWS S3 storage bucket as part of the course, and uploaded his code to GitHub as part of the course requirements. However, his code contained his AWS access keys, leading to over $3,000 of AWS charges within a day.
- The founder of an internet startup uploaded code to GitHub containing API access keys. He realized his mistake within 5 minutes and removed those keys. However, that was enough time for automated bots to find his keys, access his account, spin up computing resources for crypto-mining and result in a $2,300 bill.
- Embers.js published an npm code package in their code release containing access keys to their S3 storage buckets.
And examples go on and on…
The problem is so widespread that Amazon even has a dedicated support page to tell developers what to do if they inadvertently expose their access keys.
How to Protect Yourself
One of the main drivers of cloud migration is the agility and flexibility that it offers organizations to speed-up roll-out of new services and reduce time-to-market. However, this agility and flexibility frequently comes at a cost to security. In the name of expediency and consumer demand, developers and DevOps may sometimes not take the necessary precautions to secure their environments or access credentials.
Such exposure can happen in a multitude of ways, including accidental exposure of scripts (such as uploading to GitHub), misconfiguration of cloud resources which contain such keys , compromise of 3rd-party partners who have such credentials, exposure through client-side code which contains keys, targeted spear-phishing attacks against DevOps staff, and more.
Nonetheless, there are a number of key steps you can take to secure your cloud environment against such breaches:
1. Be That Paranoid
There’s no way around this: securing your credentials, as much as possible, is paramount. However, since credentials can leak in such a multitude of ways, and from a multitude of sources, you should therefore assume your credentials are already exposed, or can become exposed in the future.
Adopting this mindset will help you channel your efforts not (just) to limiting this exposure to begin with, but to how to limit the damage caused to your organization should this exposure occur.
2. Limit Permissions
As we pointed out earlier, one of the key benefits of migrating to the cloud is the agility and flexibility that cloud environments provide when it comes to deploying computing resources. However, this agility and flexibility frequently comes at a cost to security.
Once such an example is granting promiscuous permissions to users who shouldn’t have them. In the name of expediency, administrators frequently grant blanket permissions to users, so as to remove any hindrance to operations. The problem, however, is that most users never use most of the permissions they have granted, and probably don’t need them in the first place.
This leads to a gaping security hole, since if any one of those users (or their access keys) should become compromised, attackers will be able to exploit those permissions to do significant damage.
Therefore, limiting those permissions, according to the principle of least privileges, will greatly help to limit potential damage if (and when) such exposure occurs.
3. Act for Early Detection
The final step is to implement measures which actively monitor user activity for any potentially malicious behavior. Such malicious behavior can be first-time API usage, access from unusual locations, access at unusual times, suspicious communication patterns, exposure of private assets to the world, and more.
We can help implement detection measures which look for such malicious behavior indicators, correlate them, and alert against potentially malicious activity will help ensure that hackers are discovered promptly, before they can do any significant damage.
