NullPointerException when reading shapefiles from cloud storage on a Mosaic and GDAL enabled cluster

Zip the entire shapefile and upload to a Unity Catalog volume or DBFS storage.

Written by jessica.santos

Last published at: September 12th, 2024

Problem 

When attempting to read a shapefile from a cloud storage bucket on a cluster with Mosaic and GDAL enabled, you get a java.lang.NullPointerException error.

Cause

This issue occurs when trying to read geospatial files, such as .shp and .geojson, from any cloud-based object storage. It is not possible to use any cloud-based object storage directly with Mosaic GDAL APIs.

Solution

Ensure that the entire shapefile, including all necessary components (.shp, .shx, .dbf, etc.), is zipped and uploaded to a Unity Catalog volume or DBFS storage.

For more information, please review the Mosaic + GDAL Shapefile Example from the Mosaic GitHub repository.