Why Would You Need a Feature Store

Discovery

<aside> 💡

Multiple Data Scientists are trying to solve similar modeling tasks and come up with different difintions of the same features. How can I find the features?

</aside>

Lineage

<aside> 💡

Model governance requires documentation of the features used to train a model, as well as the upstream lineage of a feature to reliably use it. How is it computed, and who owns it?

</aside>

Skew

Online Serving

Workspace FS vs. Unity Catalog FS

Workspace FS

• Specific to a single workspace
• Challenges in sharing feature tables across workspace
• Uses FeatureStoreClient to create feature tables

Unity Catalog FS (Recommended)

• Any Delta table with a primary key can be a feature table (DBR 13.2)
• All UC capabilities are available: discovery, governance, lineage and cross-workspace access.
• Uses FeatureEngineerClient to create feature tables.

Create feature tables in Unity Catalog

%sql
CREATE CATALOG IF NOT EXISTS ml;
USE CATALOG ml;

CREATE SCHEMA IF NOT EXISTS taxi_example;
USE SCHEMA taxi_example;

CREATE TABLE IF NOT EXISTS ml.taxi_example.trip_pickup_time_series_features(
	zip INT NOT NULL,
	ts TIMESTAMP NOT NULL,
	mean_fare_window_1h_pickup_zip FLOAT,
	count_trips_window_1h_pikup_zip INT,
	CONSTRAINT trip_pickup_time_series_features_pk PRIMARY KEY (zip, ts TIMESERIES)
)
COMMENT "Taxi Fares. Pickup Time Series Features";

Write initial feature to feature tables in Unity Catalog