Problem

When attempting to migrate from Databricks Runtime 10.4 LTS to 14.3 LTS, you encounter an error message. 

PERMISSION_DENIED: User does not have READ FILES on External Location '<external-table-name>'. 

Cause

Permissions may not be set correctly, causing you to miss the required READ FILES permission. Permissions for external locations are strictly enforced when using the Hive metastore. 

Alternatively, permissions in Databricks Runtime 14.3 LTS may not be inherited as they were in Databricks Runtime 10.4 LTS. For example, even if you have permission on a parent directory, you still need explicit permissions on the child directory. 

Solution

First, ensure you have the necessary READ FILES permission on the external location indicated in the error message. Grant permissions using the following command.

%sql
GRANT READ FILES ON EXTERNAL LOCATION '<external-table-name>' TO '<username>';

For more information, refer to the External locations documentation.

Then, check if permissions are being inherited correctly in Databricks Runtime 14.3 LTS. If not, grant permissions explicitly on the table <external-table-name>. Follow the instructions in the “Grant permissions on an external location” section of the Manage external locations documentation.

For more information on privileges, review the Unity Catalog privileges and securable objects documentation.

Preventative measures

• Regularly review and update permissions on external locations and other resources.
• Monitor changes in Databricks Runtime versions and test jobs thoroughly before migrating to a new version.
• Stay up-to-date with Databricks documentation and forums to learn about any changes or best practices that can help you optimize your Databricks environment.