The Cross-Tenant Power Platform Connectors Vulnerability – Are You Safe Now?

What Happened

Last week, on March 31st, NetSPI researchers announced that they found a cross-tenant Azure vulnerability in the Microsoft Power Platform connectors infrastructure, which allowed them to then access “at least 1,300 secrets/certificates in 180+ vaults”. In this article, we set out to analyze the root cause behind this vulnerability, explain its impact, and provide our own recommendations for Power Platform users and administrators. 

Power Platform And Gateways

To understand the vulnerability and how it was exploited, we first need to explain how the Power Platform cloud communicates with the On-prem gateway that was exploited.

Here’s an overview of the Power Platform connectors underlying Azure infrastructure:

 

Every connector in Power Platform is invoked through the Azure API Management service (APIM). To demonstrate, see what happens after I created a new Custom Connector called “RCE” that interacts with an On-prem gateway, and when we try to run an action.

1. The first thing that happens is Power Automate invokes our connector’s dedicated Azure APIM endpoint, with a dedicated authorization token. This step initiates a roundtrip communication sequence between our connector and On-prem gateway.
2. Next, the request is forwarded through a dedicated Azure App Service instance, which is a machine that runs any additional logic or code that this connector needs to run.
3. This is the stage where the gateway related Azure services (Gateway Cloud Service, Azure Relay) comes into play. Let’s look at this diagram that dives in into the Azure cloud/gateway architecture:
4. The request is then forwarded to the Gateway Cloud Service, which is the service responsible for routing messages securely from the cloud to the On-prem machines and vice-versa.
5. Then it goes to the Azure Relay – which is the way your On-prem gateway can get inbound requests without the firewall dropping these requests.
6. In the final step, the request arrives at the On-prem machine gateway, which deserializes the JSON message using vulnerable code. Once it finishes running the command it got from Power Automate (i.e. fetching data from an On-prem SQL server), a response is sent back all the way to Power Automte, as can be observed in the Power Automate UI:

This message is received from the Azure App Service instance with content which originates from the On-prem machine. For example, you can see the entire .NET gateway code exception stack trace:

This effectively completes a round-trip between Power Automate and the On-prem gateway.

What Went Wrong Here?

The Power Platform connector’s underlying Azure App Service instance and the On-prem data gateway share the same code to deserialize incoming requests in JSON format.

This code is vulnerable to a “JSON deserialization attack.” These attacks occur when an attacker sends a specially crafted JSON object to a vulnerable application, causing it to execute malicious code during the deserialization process. This vulnerability arises when an application receives JSON data from an untrusted source, and the data is automatically converted into an object without proper validation and sanitization.

NetSPI researchers reconstructed and used a malicious gateway using code chunks from the gateway’s decompiled executable to run code on the cloud connector infrastructure. They then went on exploring the underlying service using the code execution capabilities they gained in phase 1.

Is My Network at Risk?

The original article briefly discussed the implications on the gateway itself, but mostly focused on the potential risk for the cloud side. Let’s dig deeper in the opposite direction, by asking… Can a malicious actor with access to Power Platform take over the On-prem machines running gateways?

As we mentioned earlier, the vulnerable code block is shared across the cloud and the gateway executable. This means that the gateway itself is a candidate for exploiting the vulnerability. 

If an attacker could control the cloud side requests, they might use this vulnerability to run arbitrary code on the victim’s On-prem machine by abusing the gateway’s JSON deserializer. If the gateway was granted local admin, the attacker has a double jackpot.

To address this issue, we have decided to see what happens when you can alter the request from the cloud side. We used the custom connector from previous sections, to see what part of the payload we control.

First, we ran the gateway executable on our local machine and attached to the process with a debugger. We set up a breakpoint in the deserialization method call:

And we’ll trigger it by running our custom connector action:

Now, we can see the plain-text payload string from Power Automate, Let’s see what’s in it:

The payload is formatted as JSON, with deserialization instructions given as each object’s “$type” field. If we could control one of these fields, we can exploit the vulnerability and run code on the machine.

The bytes blob has our connector’s action contents. Let’s use Base64 decoding to see what’s in it exactly:

We have tried to change and manipulate the contents, but they were all bundled together into the Base64 blob, which is serialized as a byte array, and the deserialization code won’t acknowledge it as another type.

So exploiting the gateway from the cloud side seems harder than the other way around, but it doesn’t assure an exploit couldn’t be crafted.

To Answer the Question – Is My Network at Risk? 

1. Targeting the gateway executable through the custom connector attack vector is low risk.
2. At the end of the day, the gateway is still deserializing inputs using vulnerable code. Thus, in the general scenario of exploiting the gateway through the cloud is a medium/high risk. Vulnerable gateways still need to be updated to mitigate this threat. 

Verifying Microsoft’s Mitigation

According to NetSPI, the vulnerability was disclosed to Microsoft in September 2022, and a fix was fully deployed by Microsoft in December 2022 across Azure regions.

Microsoft fixed the issue by remodeling the deserialization binder to be stricter with the types it allows to be deserialized. Claims were also made that the cloud side and the gateway side now have different deserialization binders, which might stop attackers from inferring cloud deserialization vulnerabilities by reverse engineering the On-prem gateway executable. This was possible, due to the fact that previously the cloud and gateway shared the same deserialization logic.

We decided to take a look at the latest On-prem gateway releases to find out exactly when the patch was introduced to the On-prem gateway executables.

We analyzed the gateway code to understand which gateway version was the first to include the patch. Turns out it is from the November 22 release (3000.150.11).

The gateway uses 2 executables – EnterpriseGatewayConfigurator.exe and Microsoft.PowerBI.EnterpriseGateway.exe. With dnSpy we can decompile the .NET code from Microsoft.PowerBI.EnterpriseGateway.exe. Look at this piece of vulnerable code from October 22 release, 3000.146.10 (it is in the September release as well):

As of the November 22 release, this code is different. Now, Microsoft created a dictionary that contains the exact allowed types to be deserialized which is more strict:

It is much more robust than the September release, but the dictionary still has some tens of allowed types which is not ideal.

What Should I Do Now?

According to NetSPI, a fix to the Power Platform Azure infrastructure was fully deployed by Microsoft in December 2022 across Azure regions. This means that your Power Platform cloud instance is covered, and no further actions are required this point.

Zenity’s research concludes that Microsoft mitigated the issue in the On-prem gateway executables as of the November 22 release. Our advice is to make sure your gateways are up to date at least to version 3000.150.11, as older versions contain the deserialization vulnerability. It is important to state that we have not found any theoretical or in-the-wild exploit to this vulnerability, targeting the gateway executable up to this point.

Zenity customers can use this query to find all gateways in their organization in need of update.

Recommendations for Power Platform Users and Administrators

It is always important to know what versions of software you are running, and knowing is always the first step. This is how you can get your own gateways’ versions: Go to Power Automate -> Data -> Gateways -> Select a gateway -> Details

Now look at the gateway status:

If there’s a new version available, update it using the “Get it now” link. Download the gateway installation and follow along the instructions.

For administrators, we recommend using the official Microsoft documentation to select a gateway and review its version.Notice that in case when you are not the owner of the gateway, you will need to reach out to the gateway owner and ask them to initiate the update process on their own.

As always, feel free to get in touch with us to learn more!