Problem

You are trying to create a new storage credential for connecting to AWS S3. You copy the existing IAM role trust relationship policy and don’t change the storage credential’s external ID in it.

Because you copied the policy, you assume the storage credential’s external ID will remain the same, but you encounter a failure to validate the storage credential.

Example IAM trust relationship policy for Databricks on AWS

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::414351767826:role/unity-catalog-prod-UCMasterRole-14S5ZJVKOTYTL",
          "arn:aws:iam::<your-aws-account-id>:role/<this-role-name>"
        ]
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<storage-credential-external-id>"
        }
      }
    }
  ]
}

Cause

The external ID for storage credentials can be different depending on your role.

• If you are a Databricks account admin (excluding workspace admins), the external ID is populated with your Databricks account ID.
• If you are not a Databricks account admin, a new external ID will be created as a random ID when you create a storage credential from Credentials in Catalog Explorer. For more details, review the Give Databricks the IAM role details documentation.

Solution

You need to verify the role of the user that creates the storage credential and update the external ID in the copied IAM role trust relationship policy.

Account admin role

When an account admin creates the storage credential, you need to use the Databricks account ID as the sts:ExternalId.

Example trust relationship policy for Databricks on AWS

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::414351767826:role/unity-catalog-prod-UCMasterRole-14S5ZJVKOTYTL",
          "arn:aws:iam::<your-aws-account-id>:role/<this-role-name>"
        ]
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<databricks-account-id-is-external-id>"
        }
      }
    }
  ]
}

Non-account admin role

When a non-account admin creates the storage credential, the sts:ExternalId is the new external ID you created in the Credentials UI within Catalog Explorer.

Example trust relationship policy for Databricks on AWS

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::414351767826:role/unity-catalog-prod-UCMasterRole-14S5ZJVKOTYTL",
          "arn:aws:iam::<your-aws-account-id>:role/<this-role-name>"
        ]
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<new-storage-credential-external-id>"
        }
      }
    }
  ]
}