The server currently supports Shared Key and AAD Service Principal for ADLS Gen2. Both require storing secrets, which is problematic in hardened Azure environments where account keys and SAS are disabled.
This requests adding a managed_identity auth mode using Azure IMDS, covering both delta log reads (Hadoop MsiTokenProvider) and User Delegation SAS vending for clients.
New config keys: delta.sharing.azure.auth.type and delta.sharing.azure.managed.identity.client.id.
Uses direct REST to IMDS and Blob Storage APIs rather than azure-identity SDK, due to the Jackson 2.6.x pin required by delta-standalone (BlobServiceClientBuilder triggers HttpLoggingPolicy which needs Jackson 2.9+). Related: delta-io/delta#5598, #666.
The server currently supports Shared Key and AAD Service Principal for ADLS Gen2. Both require storing secrets, which is problematic in hardened Azure environments where account keys and SAS are disabled.
This requests adding a
managed_identityauth mode using Azure IMDS, covering both delta log reads (Hadoop MsiTokenProvider) and User Delegation SAS vending for clients.New config keys:
delta.sharing.azure.auth.typeanddelta.sharing.azure.managed.identity.client.id.Uses direct REST to IMDS and Blob Storage APIs rather than azure-identity SDK, due to the Jackson 2.6.x pin required by delta-standalone (BlobServiceClientBuilder triggers HttpLoggingPolicy which needs Jackson 2.9+). Related: delta-io/delta#5598, #666.