Azure Server-side Encryption (SSE) and Azure Disk Encryption (ADE) – Part 1 of 2

Security has become one of the most important requirements with clients as no company wants to be the next front page news for a data breach. I’ve been fortunate to have worked with clients who passed on application forms they’ve received when obtaining cyber & data breach insurance and found that every iteration dives deeper into how well their data is protected. This has lead to many conversations around explaining to technical and non-technical audiences about how Azure protects their data. I have to admit that I’m not an encryption expert but have made a conscious effort to keep up with the technologies available to be a better solutions architect so this post serves to provide an overview of the various options of how data can be encrypted in Azure.

Two of the types of encryption you’ll see most while searching through Azure documentation will be:

1. Server-side Encryption (SSE)
2. Azure Disk Encryption (ADE)

For the purpose of narrowing the scope, I will discuss how these apply to IaaS virtual machines in Azure and leave Storage Accounts in another post for the future. Due to the amount of content required for SSE and ADE, this post will be separated into two parts.

Server-side Encryption (SSE) – Part #1
Azure Disk Encryption (ADE) – Part #2

Let me begin by providing the following links to the documentation provided by Microsoft:

Server-side encryption of Azure Disk Storage
https://docs.microsoft.com/en-us/azure/virtual-machines/disk-encryption

Azure Data Encryption at rest
https://docs.microsoft.com/en-us/azure/security/fundamentals/encryption-atrest

Use the Azure portal to enable server-side encryption with customer-managed keys for managed disks
https://docs.microsoft.com/en-us/azure/virtual-machines/disks-enable-customer-managed-keys-portal

What is encryption at rest?
https://docs.microsoft.com/en-us/azure/security/fundamentals/encryption-atrest#what-is-encryption-at-rest

Azure Storage encryption for data at rest
https://docs.microsoft.com/en-us/azure/storage/common/storage-service-encryption

Preview: Server-side encryption with customer-managed keys for Azure Managed Disks
https://azure.microsoft.com/en-ca/blog/preview-server-side-encryption-with-customer-managed-keys-for-azure-managed-disks/

Data encryption models
https://docs.microsoft.com/en-us/azure/security/fundamentals/encryption-models

I’ve found all of the documentation very information and highly suggest the read but it is also a bit difficult to tie all of the information together and impractical to send these to a client so the following is usually how I present the information either through a PowerPoint or discussion with Visio diagrams.

Azure Server-side Encryption (SSE)

By default, managed disks are encrypted with Azure Storage encryption, which uses server-side encryption (SSE) with a platform-managed key to protect the data on OS and data disks. The data on the disks are encrypted transparently using 256-bit AES encryption, one of the strongest block ciphers available, and is FIPS 140-2 compliant. Microsoft also clearly states that Azure Storage encryption does not impact the performance of managed disks. The one note that is worth calling out is that temporary disks are not managed disks and are therefore not encrypted by SSE, unless you enable encryption at host.

What SSE provides is data at rest encryption where if someone where to successfully break into an Azure datacenter, physically take the hard drives, they will not be able to read them upon attaching them to another computer because they would not have the encryption key. SSE is comprised of several components and there are two choices when determining how encryption keys are managed. The two types are:

1. Platform Managed Keys (PMK)
2. Customer Managed Keys (CMK)

Platform Managed Keys (PMK)

PMK is essentially using keys that Microsoft manages for the encryption. This is the default offering and setting when you create a managed disk. When you navigate to a virtual machine, click on the Disks of the VM, you will notice that the Encryption header states: SSE with PMK

Hovering over the information icon will display the following:

SSE with PMK is server-side encryption with a platform-managed key. This is enabled by default on all managed disks. SSE with CMK is server-side encryption with a customer-managed key. ADE is Azure disk encryption.

For many organizations, the essential requirement is to ensure that the data is encrypted whenever it is at rest. Server-side encryption using service-managed Keys enables this model by allowing customers to mark the specific resource (Storage Account, SQL DB, etc.) for encryption and leaving all key management aspects such as key issuance, rotation, and backup to Microsoft. Most Azure services that support encryption at rest typically support this model of offloading the management of the encryption keys to Azure. The Azure resource provider creates the keys, places them in secure storage, and retrieves them when needed. This means that the service has full access to the keys and the service has full control over the credential lifecycle management.

The following diagram provides a visualization of the components of how a managed disk is encrypted at rest with PMK:

Let’s have a more detailed look at each component:

Managed Disk

Microsoft recommends managed disks for deployments as compared to unmanaged ones and the difference between the two is that the legacy unmanaged disks are stored in a storage account with a page blob that stores one or more VHDs. This is similar to the concept of a SAN with aggregates/volumes and LUN in VMware but this also means the Azure administrator would need to continue to assess how many disks are in the page blob, how performance is affected by each disk and many other aspects. The new managed disks abstracts the components so the administrator doesn’t need to place the disks into a storage account and manage them manually. Unmanaged disks arguably provides the administrator more control but the overhead it creates can be laborious.

Key Hierarchy

More than one encryption key is used to encrypt managed disks that are encrypted at rest. The Key Encryption Key is the encryption key stored in a Microsoft key store (equivalent to an Azure Key Vault) and used to encrypt/decrypt the Data Encryption Key that is used to encrypt the actual data. The reason why there are two keys in this process is because having to continuously access the Microsoft key store (Azure Key Vault) and retrieve the KEK would be very inefficient for large amounts of data operation. Using DEK to provide service local access to encryption keys is more efficient for bulk encryption an decryption for data operations and also allows for stronger encryption and better performance. Having different DEKs to encrypt different data reduces the risk of having a key compromised and the cost/effort to re-encrypt with a new key.

Data Encryption Keys (DEK)

The DEK is a symmetric AES256 key used to encrypt a partition or block of data and the foundational Azure resource providers will store the Data Encryption Keys in a store that is close to the data and quickly available and accessible. A single resource can have many partitions and many DEKs, which makes crypto analysis attacks more difficult.

Another advantage of having multiple DEKs encrypting blocks of data is that when a DEK is replaced with a new key, only the data in its associated block must be re-encrypted with the new key.

Key Encryption Key (KEK)

The purpose of a KEK is to encrypt the DEK. Having the KEK that is stored and never leaves the key vault allows the DEK to be securely encrypted and controlled by a centrally managed key store where the KEK resides. The entity has access to the KEK may be different than the entity that requires the DEK. An entity may broker access to the DEK to limit the access of each DEK to a specific partition. Since the KEK is required to decrypt the DEKs, the KEK is effectively a single point by which DEKs can be effectively deleted by deletion of the KEK.

More information about the key hierarchy be found at:

Azure Data Encryption at rest
https://docs.microsoft.com/en-us/azure/security/fundamentals/encryption-atrest

Customer Managed Keys (CMK)

The architecture for CMK forks out into two paths:

1. Server-side encryption using customer-managed keys in Azure Key Vault
2. Server-side encryption using customer-managed keys in customer-controlled hardware

Server-side encryption using customer-managed keys in Azure Key Vault

Server-side encryption using customer-managed keys in Azure Key Vault is essentially the same as Server-side encryption using service-managed keys PMK and the difference is that the keys are stored in a customer managed key vault rather than a Microsoft managed key store. This option typically store the root Key Encryption Key in the customer managed Azure Key Vault and store the encrypted Data Encryption Key in an internal location closer to the data. CMK offers organizations that have the requirement to manage the encryption keys themselves the ability to bring their own keys to Key Vault (BYOK – Bring Your Own Key), or generate new ones, and use them to encrypt the desired resources. While the Resource Provider performs the encryption and decryption operations, it uses the configured key encryption key as the root key for all encryption operations.

As the key encryption keys is used to encrypt and decrypt the data encryption key that is used to encrypt and decrypt the data, loss of the KEK directly means loss of data given how there would be no way to decrypt the DEKs. The recommended best practice is that KEKs should never be deleted. Rather, they should be backed up whenever newly created or rotated. The customer was required to rotate their own keys until recently when a new Automatic key rotation of customer-managed keys (preview) was made available as stated here: https://docs.microsoft.com/en-us/azure/virtual-machines/disk-encryption#automatic-key-rotation-of-customer-managed-keys-preview and here: https://azure.microsoft.com/en-ca/updates/public-preview-automatic-key-rotation-of-customermanaged-keys-for-encrypting-azure-managed-disks/. The feature is still limited to a small set of regions at the time of this writing (May 2021):

Automatic key rotation is in preview and only available in the following regions:

• East US
• East US 2
• South Central US
• West US
• West US 2
• North Europe
• West Europe
• France Central

Additionally, soft-delete and purge protection must be enabled on any vault storing key encryption keys to protect against accidental or malicious cryptographic erasure. Instead of deleting a key, it is recommended to set enabled to false on the key encryption key, which disables the ability to use an out of rotation and retired key.

An additional resource that is required when using CMK is the concept of a Disk Encryption Set, which is a new resource introduced for simplifying the key management for managed disks by placing managed disks into a group that will then be encrypted by DEKs, then KEKs and finally have the KEKs stored in a customer-managed Azure Key Vault. You can place disks as they are newly created into a Disk Encryption Set or you can move existing ones in there provided that the disk is detached from a VM or the VM it is attached to it is deallocated. Those who have attempted to create a managed disk and set the Encryption type to Encryption at-rest with a customer-managed key will see that another drop down list named Disk encryption set will be presented:

Note the warning message that is presented to indicate you can’t revert back to PMK once CMK is used for this disk:

Once a customer-managed-key is used, you can’t change the selection back to a platform-managed key.

The following is also displayed if you hover over the information icon:

A disk encryption set stores the customer key that a disk or snapshot will use for encrypting its data. You must choose an existing disk encryption set during this step. Disk encryption sets require access to key vault and keys.

You can’t create a disk encryption set within the previous managed disk wizard so you’ll need to create one beforehand:

With disk encryption sets described, the following diagram provides a visualization of the components of how a managed disk is encrypted at rest with CMK:

The authorization and authentication of the customer-managed keys is provided by Azure AD to grant permissions to use the keys stored in Azure Key Vault. Azure Active Directory accounts can be permissioned to manage or access keys for Encryption at Rest encryption and decryption.

Microsoft’s official documentation about how the CMK process works is presented here: https://docs.microsoft.com/en-us/azure/virtual-machines/disk-encryption#full-control-of-your-keys

As stated in the documentation above, the following are details about access, disabling keys and auditing:

• You grant access to managed disks in the Azure Key Vault to use keys for encrypting and decrypting the DEK used to encrypt the disks in the disk encryption set
• Keys can be disabled and access to managed disks can be revoked at any time
• Encryption key usage in the Azure Key Vault can be audited (what is accessing the managed disks)

Disabling or deleting keys:

• Working with premium SSDs, standard SSDs, and standard HDDs – When the keys used to encrypt these disks are disabled, any virtual machines with the disks using the key will automatically shutdown and become unusable unless the key is enabled again or a new key is assigned.
• Working with ultra disks – When the keys used to encrypt these disks are disabled, any virtual machines with the disks using the key will continue to run until it is deallocated or restarted as either of those operation will not allow the virtual machines to come back online until the key is enabled again or a new key is assigned.

Microsoft provides a great diagram that shows how managed disks use Azure AD and Azure Key vault to make requests using customer-managed key:

I won’t attempt to rewrite the official document that describes the above diagram as I find it quite clear:

1. An Azure Key Vault administrator creates key vault resources.
2. The key vault admin either imports their RSA keys to Key Vault or generate new RSA keys in Key Vault.
3. That administrator creates an instance of Disk Encryption Set resource, specifying an Azure Key Vault ID and a key URL. Disk Encryption Set is a new resource introduced for simplifying the key management for managed disks.
4. When a disk encryption set is created, a system-assigned managed identity is created in Azure Active Directory (AD) and associated with the disk encryption set.
5. The Azure key vault administrator then grants the managed identity permission to perform operations in the key vault.
6. A VM user creates disks by associating them with the disk encryption set. The VM user can also enable server-side encryption with customer-managed keys for existing resources by associating them with the disk encryption set.
7. Managed disks use the managed identity to send requests to the Azure Key Vault.
8. For reading or writing data, managed disks sends requests to Azure Key Vault to encrypt (wrap) and decrypt (unwrap) the data encryption key in order to perform encryption and decryption of the data.

To revoke access to customer-managed keys, see Azure Key Vault PowerShell (https://docs.microsoft.com/en-us/powershell/module/azurerm.keyvault/?view=azurermps-6.13.0) and Azure Key Vault CLI (https://docs.microsoft.com/en-us/cli/azure/keyvault?view=azure-cli-latest). Revoking access effectively blocks access to all data in the storage account, as the encryption key is inaccessible by Azure Storage.

As mentioned earlier in the post, CMK traditionally required the customer to rotate their own keys but a new feature currently in previous provides this service:

Automatic key rotation of customer-managed keys (preview)

You can choose to enable automatic key rotation to the latest key version. A disk references a key via its disk encryption set. When you enable automatic rotation for a disk encryption set, the system will automatically update all managed disks, snapshots, and images referencing the disk encryption set to use the new version of the key within one hour. The feature is currently available in limited regions in preview. For regional availability, see the Supported Regions (https://docs.microsoft.com/en-us/azure/virtual-machines/disk-encryption#supported-regions) section.

Server-side encryption using customer-managed keys in customer-controlled hardware

Server-side encryption using customer-managed keys in customer-controlled hardware is fairly self-explanatory where the customer will not use the Azure Key Vault to store their keys but rather a solution outside of Microsoft Azure (e.g. on-premise data center or another cloud). I won’t attempt to rewrite the documentation so I’ll simply paste it here:

https://docs.microsoft.com/en-us/azure/security/fundamentals/encryption-models#server-side-encryption-using-customer-managed-keys-in-customer-controlled-hardware

Some Azure services enable the Host Your Own Key (HYOK) key management model. This management mode is useful in scenarios where there is a need to encrypt the data at rest and manage the keys in a proprietary repository outside of Microsoft’s control. In this model, the service must retrieve the key from an external site. Performance and availability guarantees are impacted, and configuration is more complex. Additionally, since the service does have access to the DEK during the encryption and decryption operations the overall security guarantees of this model are similar to when the keys are customer-managed in Azure Key Vault. As a result, this model is not appropriate for most organizations unless they have specific key management requirements. Due to these limitations, most Azure services do not support server-side encryption using server-managed keys in customer-controlled hardware.

Encryption at host – End-to-end encryption for your VM data

A new feature that is currently being made available in Azure (it isn’t available through my test subscription) is Encryption at host. I find this to be an exciting feature because it would eliminate the need for Azure Disk Encryption (ADE), which I will be writing more about in part #2. What Encryption at host does is essentially provide end-to-end encryption between the disk as rest and when the disk is allocated to and ran on a host. As mentioned earlier, Azure Server-side Encryption (SSE) encrypts data at rest and not in transit, when running on a host, and not the temporary disk a VM is configured with.

With Encryption at host, traffic between the host and storage service and all the disks including the temporary disk will be encrypted. Lastly, the most common question I get asked when I start raving about this is whether this will impact the performance of the host or VM and the answer from Microsoft’s document is always follow: Encryption at host does not use your VM’s CPU and doesn’t impact your VM’s performance.

Reference: https://docs.microsoft.com/en-us/azure/virtual-machines/disk-encryption#encryption-at-host—end-to-end-encryption-for-your-vm-data

The following is a diagram outlining the process:

• Whatever is used to encrypt the disk encryption set at rest (PMK or CMK) will be used to encrypt the in transit / in flight data from the disk to the host
• The same PMK or CMK used for the at rest disk will be used to encrypt the cache disk
• The temp disk will always be encrypted by a PMK

Note that as great as the feature sounds, there are restrictions and they are as follow (as of mid-2021):

• Does not support ultra disks.
• Cannot be enabled if Azure Disk Encryption (guest-VM encryption using bitlocker/VM-Decrypt) is enabled on your VMs/virtual machine scale sets.
• Azure Disk Encryption cannot be enabled on disks that have encryption at host enabled.
• The encryption can be enabled on existing virtual machine scale set. However, only new VMs created after enabling the encryption are automatically encrypted.
• Existing VMs must be deallocated and reallocated in order to be encrypted.
• Supports ephemeral OS disks but only with platform-managed keys.

Double encryption at rest

The last SSE encryption feature is the Double encryption at rest. High security sensitive organizations who are concerned of the risk associated with any particular encryption algorithm, implementation, or key being compromised can opt for additional layer of encryption using a different encryption algorithm/mode at the infrastructure layer using platform managed encryption keys. This new layer can be applied to persisted OS and data disks, snapshots, and images, all of which will be encrypted at rest with double encryption. The following is a diagram depicting the double encryption using both CMK and PMK:

With all the theory out of the way, let’s proceed onto demoing how all this looks during the configuration.

Demo – Platform Managed Keys (PMK)

As mentioned earlier in this post, SSE with PMK is turned on by default for managed disks:

Attempting to create a new data disk for a VM will automatically turn on PMK as well as state that a Disk encryption set is not required:

Demo – Customer Managed Keys (CMK)

To set up SSE with CMK, begin by creating an Azure Key Vault that will store the keys:

Depending on what this vault is used for, you would enable the following:

• Azure Virtual Machines for deployment
• Azure Resource Manager for template deployment
• Azure Disk Encryption for volume encryption

For the purpose of this demonstration, I will enable all of them as I will be using this to demonstrate ADE later on as well.

The Permission model default is set to Vault access policy and Azure role-based access control (RBAC) was later introduced but for the purpose of this example, I will leave it as Vault access policy. More information about using Azure RBAC can be found here: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/disk-encryption-key-vault

Proceed to create the vault:

The vault should be created fairly quickly and once complete, proceed into the vault, select Keys and then Generate/Import towe can generated for SSE with CMK:

This is where we can generate, import or restore from backup keys:

Proceed to generate a key with the default Key type as RSA and RSA key size as 2048:

The new key should be created fairly quick:

Proceed to set up the disk encryption set:

—————————————————————————————————————————-

Note that if the account attempting to create the disk encryption set does not have permissions to the subscription then the following error would be thrown:

If you don’t have Owner permissions then this error would be thrown:

[ForbiddenByRbac (Forbidden)] Caller is not authorized to perform action on resource. If role assignments, deny assignments or role definitions were changed recently, please observe propagation time. Caller: name=KeyVault/ManagementPlane;appid=…

To display the error above, I removed my account as an Owner to the subscription.

—————————————————————————————————————————-

The disk encryption set should be created fairly quickly but notice the banner with the message:

To associate a disk, image, or snapshot with this disk encryption set, you must grant permissions to the key vault Test-SSE-CMK:

Make sure you click on the banner so permissions for the disk encryption set is configured in the Access policies of the key vault:

The following are the permissions the above process places into the Azure Key vault:

Key Management Operations: Get
Cryptographic Operations: Unwrap Key, Wrap Key

With the disk encryption set created and permissions to the key vault granted, we can either create a new VM or move an existing disk into it. Let’s start with creating a new VM:

Once the virtual machine has created, proceed to view the properties of the OS disk and you should see SSE with CMK listed:

From here, you can add an additional data disk to a VM and turn on SSE with CMK at the same time.

However, if you want to turn on SSE with CMK on an existing VM’s OS disk, that VM will need to be stopped and deallocated.

Demo – Encryption at host

My Azure subscription unfortunately does not have the encryption at host available so I will refer you to the following documentation:

Use the Azure portal to enable end-to-end encryption using encryption at host

https://docs.microsoft.com/en-us/azure/virtual-machines/disks-enable-host-based-encryption-portal

—————————————————————————————————————————-

I hope this post has been informative and provides the reader an idea of how Azure Server-side Encryption (SSE) works. Microsoft has provided plenty of documents on this topic but I’ve found that information can be scattered so I hope I’ve been able to capture the important sections to combine into this post.

There was quite a bit of writing involved so I will separate Azure Disk Encryption (ADE) to my next post.