Connect to Databricks¶
This guide will help you set up a FeatureByte feature store in a Databricks Data Warehouse.
Before You Begin¶
Gather the credentials for connecting to your databricks account, specifically:
- Name of the databricks server you're connecting to
- Sign-in credentials for the databricks server
- Name of DataBricks user group that will have access to the feature store assets
You'll also want to ensure that the user you're connecting with has the relevant privileges that are required. Specifically, the role should have the following privileges:
USAGE
on cluster
Refer to Databricks Cluster ACL for more details.
Why are these privileges needed?
These privileges are needed for featurebyte to write some metadata into your Databricks data warehouse. This is used internally by our application to track some metadata, and perform some optimizations to make your experience better.Setup Guide¶
FeatureByte Installation
Make sure that you have FeatureByte installed. See installation for more details.
Step 1: Test that your connection works¶
We can now try to see if your connection works by trying to create a new feature store. We can do so by running the following commands (either in a notebook, or a python interactive shell).
- If you know that a feature store exists already, we can try to list the existing feature stores on databricks.
- Alternatively, try to create a feature store.
# Name of the feature store that we want to create/connect to feature_store = fb.FeatureStore.get_or_create( name="<feature_store_name>", source_type=fb.SourceType.DATABRICKS, details=fb.DatabricksUnityDetails( host="<host_name>", http_path="<http_path>", catalog_name="hive_metastore", schema_name="<schema_name>", group_name="developers", ), database_credential=fb.AccessTokenCredential( access_token="<access_token>", ), )
Refer to Databricks JDBC Connection Parameters for more details.
Step 2: Connect to your Databricks feature store¶
Congratulations! You have successfully connected to your Databricks data warehouse if you are able to run these commands without any errors!
Next Steps¶
Now that you've connected to your data, feel free to try out some tutorials!