You can set up a Databricks cluster to use an embedded metastore. You can use an embedded metastore when you only need to retain table metadata during the life of the cluster. If the cluster is restarted, the metadata is lost.

If you need to persist the table metadata or other data after a cluster restart, then you should use the default metastore or set up an external metastore.

This example uses the Apache Derby embedded metastore, which is an in-memory lightweight database. Follow the instructions in the notebook to install the metastore.

You should always perform this procedure on a test cluster before applying it to other clusters.

Set up an embedded Hive metastore notebook

Review the embedded Hive metastore notebook.