Dropping and recreating Delta tables results in a DeltaVersionsNotContiguousException error

Problem If you frequently drop and recreate Delta tables, you may encounter a  DeltaVersionsNotContiguousException error. com.databricks.sql.transaction.tahoe.DeltaVersionsNotContiguousException: Versions (0, 1, 31) are not contiguous. Cause This can happen for two reasons.  Files have been manually removed from the Delta log, which disrupts the seq...

0 min reading time

Query does not skip header row on external table

Problem You are attempting to query an external Hive table, but it keeps failing to skip the header row, even though TBLPROPERTIES ('skip.header.line.count'='1') is set in the HiveContext. You can reproduce the issue by creating a table with this sample code. %sql CREATE EXTERNAL TABLE school_test_score (   `school` varchar(254),   `student_id` varc...

0 min reading time