ZAPESCAPE: Vulnerability Disclosure

Date: March 16th 2022

Severity: High

Security impact: Privilege Escalation, Data Leakage, Data Manipulation

Intro

This document is the vulnerability disclosure report once the vulnerability was discovered. For a description of the vulnerability, its impact and what should you do next, please refer to ZAPESCAPE: Organization-wide control over Code by Zapier

Summary

The Code by Zapier app allows users to execute Python or JavaScript code as part of their zap. Our research exposes that this feature relies on an AWS Lambda instance shared by the entire Zapier account, and demonstrates how any user in that account, no matter their privileges, can gain full control over that Lambda instance, granting them access to code, input and secrets that belong to other users within that account.

Reproduction steps

Step 1: Set Up an Endpoint to Exfiltrate Data

The endpoint will receive data, code and secrets exfiltrated from Zapier. For ease of use, we recommend using https://webhook.site/, which provides a dedicated endpoint viewable in the browser.

The website will generate a unique webhook for you to use. In this case, we got the unique url: https://webhook.site/0f340ac4-fd2b-4a4e-a0d4-0d973b2057a3.

Step 2: Create a Zap and add Code by Zapier

Create a new Zap. You can use a private Zap to avoid anyone including admins from detecting your malicious Zap. Choose any trigger. We used “Schedule by Zapier” and configured it to run hourly. Test your trigger.

When prompted to create an Action, choose “Code by Zapier” with Action Event “Run Python”.

The full Zap:

Step 3: Use a Malicious Payload to Take Control Over the Account’s Lambda instance

Insert the exfiltration URL to the code below. We have already done so with our unique URL from step 1 to the code.

Replace the “code” section with the payload below. The code is explained in detail later on.

In short, the code above replaces the Lambda runtime with our own malicious runtime version, and then invokes the Lambda Runtime API to process the current event and any future event, until Lambda’s cache gets recycled. Every time the code above is executed, the attacker gains full control over the entire account’s Lambda instance for several minutes. Of course, an attacker could set up a Zapier schedule to run this code every 10-15 minutes, thus gaining semi-persistency over the account’s Lambda instance.

Step 4: Zapier will Send Data, Code and Secrets to your Exfiltration URL

While our malicious runtime is handling events instead of AWS’s native runtime, we can do whatever we want. To demonstrate this capability, we introduce a malicious runtime which calls the method before handling any new event, and then continues to execute as a normal AWS’s runtime:

This produces a stream of data sent to our exfiltration URL by Zapier. The data contains the entire Lambda event, including

• Code by Zapier input
• The code used by the Code by Zapier action, including any secrets used, in particular those used by “Store by Zapier” or its Python client “StoreClient”.

On the Exfiltration Endpoint, we see:

Step 5: Run a Benign Zap

To see the attack in action, log in to Zapier with another user of that same Zapier account. Run any Zap that uses “Code by Zapier”, and watch the exfiltration URL for data leaked from Zapier. Once you trigger the Zap, Zapier would immediately send the Action input and code to the exfiltration URL.

Step 6: Automate to Maintain Access

Lambda environments are short-lived. Once the malicious code at step 4 is executed, the malicious runtime will process any event instead of the AWS runtime for as long as Lambda’s cache is not invalidated. Note that this could include several minutes where the original Lambda instance which processed the malicious payload has already terminated, but its cache is still valid. This means that each time the malicious code is executed, the attacker gains full access for several minutes. Our experiments show this to be 10-15 minutes.

To maintain access, an attacker could set up the malicious Zap to run every 10 or even 5 minutes, via a native schedule trigger or a webhook trigger to be triggered with another scheduler.

This allows the attacker to maintain full access for the vast majority of events processed by “Code by Zapier”.

Further details

Vulnerability Scope

This vulnerability enables gaining full control over the “Code by Zapier” runtime environment for the entire account. It is executable by ANY account user, no matter their role. The exploit could be introduced through a private Zap, which would ensure the attacker would not be detected, since admins would have no visibility into private Zaps other than knowing that they exist (even not the names).

Our research shows that “Code by Zapier” runs on an AWS Lambda function shared between the entire Zapier account with ARN “arn:aws:lambda:us-east-1:996097627176:function:prd-mngd-lmbd_paidcodeapipy37_ac”. We were able to demonstrate this vulnerability in all Zapier tiers. Note that in the Free Tier, the Lambda ARN is slightly different, where paidcodeapipy37 is replaced with codeapipy37.

What can attackers do with this vulnerability? Really, anything they want within the scope of the Lambda function. The attacker is practically the function for the time of the attack. This includes:

• Exfiltrating Code by Zapier input and code (which most likely contains secrets, including those used by “Store by Zapier” or its Python client “StoraClient”).
• Steal AWS credentials – AWS passes an invoked role to Lambda via environment variables. In this case, Zapier’s lambda handler (stored at “/var/task/lambda_handler.py”) is actively deleting the following environment variables to avoid leakage: ’AWS_ACCESS_KEY_ID’, ‘AWS_SECURITY_TOKEN’, ‘AWS_SESSION_TOKEN’, ‘AWS_SECRET_ACCESS_KEY’. Of course, an attack with full control over the lambda runtime can replace Zapier’s lambda handler to disable deletion of those secrets, and steal them anyway.
• Manipulate the output of “Code by Zapier” to whatever the attacker would like.
• Use Zapier infrastructure for malicious purposes (e.g. DDOS).
• Steal secrets passed to “StoreClient” by replacing from store_api import StoreClient a malicious wrapper that sends the secret to an exfiltration endpoint. See Root Cause Analysis for context of this import command.

Note that since Zapier provides a separate Lambda function instance for each Zapier account, this vulnerability is scoped to intra-account privilege escalation.

Root Cause Analysis

Our research exposes that “Code by Zapier” runs input code with the following method:

Using the exec command without any limitation is extremely dangerous, and allows the attacker to, among other actions, spawn subprocesses, query the OS (as we did), communicate with the Lambda Runtime API (as done by the malicious code in step 2), and more.

Mitigation

To mitigate the issue, we identify two options. One solution would be for Zapier to provide a separate Lambda function instance for each user, thus avoiding one user gaining control over another user’s Lambda environment. Another solution would be to avoid using exec entirely, or more realistically to limit the scope of exec so it cannot spawn subprocesses or query the entire OS. We also recommend denying usage of the Python inspect module, which lets an attacker analyze the Python call stack for reconnaissance.

Additional Information – Recon

In order to demonstrate these capabilities, we first needed to use a few different commands for reconnaissance of the “Code by Zapier” environment. Using the Python inspect module, we explored to call stack to discover /var/task/lambda_handler.py, Zapier’s Lambda handler and /var/task/secret_store.py, Zapier’s wrapper around https://store.zapier.com. We were also able to peek into the caller stack, namely /var/runtime/bootstrap.py, to query the event and context objects, which are supposedly hidden from the exec command as seen in lambda_handler.py code above.

References

For more information about using RCE to gain full access over a Lambda function instance, see Gaining Persistency on Vulnerable Lambdas.