Problem
When attempting to start a SQL warehouse after upgrading it from pro to serverless, you encounter an error that prevents the warehouse from launching.
"Clusters are failing to launch. Cluster launch will be retried. Request to create a cluster failed with an exception: Serverless warehouse cannot start since warehouse `<warehouse-id>` has runtime version overrides. Please remove the overrides to continue using the warehouse."
Cause
You’re using a custom Databricks runtime version with your SQL warehouse. When you change the warehouse from pro to serverless, any existing runtime version overrides become incompatible with the new serverless configuration.
Serverless SQL warehouses do not support runtime version overrides, resulting in the error.
Solution
Important
You may want to set test_overrides
to NULL
to solve this problem. NULL
overrides all custom Apache Spark configurations for the warehouse. The following solution shows you how to indicate specific Spark configurations using key-value pairs.
- In your SQL warehouse configuration, indicate the runtime version overrides to remove as a key-value pair within the
spark_conf
field. The following code provides an example.
{
"confs":
{
"test_overrides": {
"cluster_attributes": {
"spark_conf": {
key: <key>
value: <value>
}
}
}
}
}