Symlink format manifest fails when trying to enable liquid clustering on a table

Disable deletion vectors or avoid using Symlink format manifest, as appropriate for your context.

Written by avi.yehuda

Last published at: March 27th, 2025

Problem

When you try to enable liquid clustering on a table using the Symlink format manifest, it fails with the following error. 

DELTA_VIOLATE_TABLE_PROPERTY_VALIDATION_FAILED.PERSISTENT_DELETION_VECTORS_WITH_INCREMENTAL_MANIFEST_GENERATION.

 

Cause

When enabling liquid clustering, deletion vectors are enabled by default. Deletion vectors and Symlink format manifest cannot be used together when using liquid clustering, so the Symlink format manifest fails. 

 

Context

Symlink format manifest allows querying a Delta table with external engines like Presto or Athena.

Deletion vectors enhance Delta table performance by optimizing update operations. 

 

Solution

Disable deletion vectors or avoid using Symlink format manifest, as appropriate for your context.  

 

If you need to query your tables using external engines, you need to use Symlink format manifest. Disable deletion vectors for your table.

ALTER TABLE <your-catalog>.<your-db>.<your-table>
SET TBLPROPERTIES ('delta.enableDeletionVectors' = false);

 

Otherwise, disable Symlink format manifest instead. Deletion vectors help performance so are often a stronger choice. 

ALTER TABLE <your-catalog>.<your-db>.<your-table> 
SET TBLPROPERTIES ('delta.enableSymlinkManifestGeneration' = false);

 

For more information, review the Use liquid clustering for Delta tables (AWS | Azure | GCP) documentation.