Problem

When attempting to upsert into a Direct Vector Access Index, you may encounter a generic failure message. 

{'status': 'FAILURE', 'result': {'success_row_count': 0, 'failed_primary_keys': ['1', '2']}}.

Cause

The length of the embedding field in the data you are trying to upsert does not match the embedding_dimension parameter defined during the creation of the Direct Vector Access Index.

Example

Create a vector index, defining an embedding_dimension of 1024.

client = VectorSearchClient()
index = client.create_direct_access_index(
    endpoint_name="storage_endpoint",
    index_name="{<catalog-name>}.{<schema-name>}.{<index-name>}",
    primary_key="id",
    embedding_dimension=1024, # <-------- This parameter
    embedding_vector_column="text_vector",
    schema={
     "id": "int",
     "field2": "str",
     "field3": "float",
     "text_vector": "array<float>"}
)

Then, upsert this snippet.

index.upsert([{"id": 1, "field2": "value2", "field3": 3.0,
       "text_vector": [1.0, 2.0, 3.0] # <------- The embedding dimension is 4
},
{"id": 2, "field2": "value2", "field3": 3.0,
"text_vector": [1.1, 2.1, 3.0] # <------- The embedding dimension is again 4.
}])

The following error occurs, indicating a failure status. The rest of the error specifies the input IDs that caused the failure.

{'status': 'FAILURE', 'result': {'success_row_count': 0, 'failed_primary_keys': ['1', '2']}}.

The data in the upsert snippet contains embeddings of dimension 4, but the defined embedding_dimension during the vector index creation was 1024.

Solution

Ensure that the embedding_dimension parameter used during the creation of the Direct Vector Access Index matches the length of the embedding field in the data you are trying to upsert.

If the data's embedding field length is different from the defined embedding_dimension, recreate the index with the correct embedding_dimension value.

For example, if the embedding field length is 3, set embedding_dimension to 3 during index creation.