13. Compute historical feature values

Compute historical feature values¶

Historical feature values are needed to train and test Machine Learning models.

Let's take the feature list we just created and compute feature values for a given observation table.

In [1]:

import featurebyte as fb

# Set your profile to the tutorial environment
fb.use_profile("tutorial")

catalog_name = "Grocery Dataset Tutorial"
catalog = fb.Catalog.activate(catalog_name)

import featurebyte as fb # Set your profile to the tutorial environment fb.use_profile("tutorial") catalog_name = "Grocery Dataset Tutorial" catalog = fb.Catalog.activate(catalog_name)

16:45:03 | INFO     | Using configuration file at: /Users/viktor/.featurebyte/config.yaml
16:45:03 | INFO     | Active profile: tutorial (https://tutorials.featurebyte.com/api/v1)
16:45:03 | INFO     | SDK version: 0.6.0.dev121
16:45:03 | INFO     | No catalog activated.
16:45:03 | INFO     | 10 feature lists, 59 features deployed
16:45:03 | INFO     | Using profile: tutorial
16:45:03 | INFO     | Using configuration file at: /Users/viktor/.featurebyte/config.yaml
16:45:03 | INFO     | Active profile: tutorial (https://tutorials.featurebyte.com/api/v1)
16:45:04 | INFO     | SDK version: 0.6.0.dev121
16:45:04 | INFO     | No catalog activated.
16:45:04 | INFO     | 10 feature lists, 59 features deployed
16:45:04 | INFO     | Catalog activated: Grocery Dataset Tutorial

List feature lists in Catalog¶

In [2]:

catalog.list_feature_lists()

catalog.list_feature_lists()

Out[2]:

	id	name	num_feature	status	deployed	readiness_frac	online_frac	tables	entities	primary_entity	created_at
0	6564b959d3f7244b7fc926a0	Customer Simple FeatureList	7	DRAFT	False	0.0	0.0	[GROCERYCUSTOMER, GROCERYINVOICE, INVOICEITEMS...	[customer]	[customer]	2023-11-27T15:44:36.966000

Get Feature List from Catalog¶

In [3]:

simple_feature_list = catalog.get_feature_list("Customer Simple FeatureList")

simple_feature_list = catalog.get_feature_list("Customer Simple FeatureList")

Loading Feature(s) |████████████████████████████████████████| 7/7 [100%] in 0.7s

Get an observation table¶

In [4]:

# List observation tables
catalog.list_observation_tables()

# List observation tables catalog.list_observation_tables()

Out[4]:

	id	name	type	shape	feature_store_name	created_at
0	6564b8b466074d16d2f61078	Preview Table with 10 items	view	[10, 2]	playground	2023-11-27T15:41:42.922000
1	6564b8a56f39a417e1370dbd	1K Customers Spending next 2 weeks at time of ...	observation_table	[1000, 3]	playground	2023-11-27T15:41:29.401000
2	6564b89d66074d16d2f61075	1K Customers at time of purchase 22S2-23S1	view	[1000, 2]	playground	2023-11-27T15:41:20.022000
3	6564b89266074d16d2f61073	Preview Table with 10 Customers	view	[10, 2]	playground	2023-11-27T15:41:10.029000

In [5]:

# Get observation table: '1K Customers Spending next 2 weeks at time of purchase 22S2-23S1'
training_observations = catalog.get_observation_table(
    "1K Customers Spending next 2 weeks at time of purchase 22S2-23S1"
)

# Get observation table: '1K Customers Spending next 2 weeks at time of purchase 22S2-23S1' training_observations = catalog.get_observation_table( "1K Customers Spending next 2 weeks at time of purchase 22S2-23S1" )

In [6]:

training_observations.sample()

training_observations.sample()

Out[6]:

	POINT_IN_TIME	GROCERYCUSTOMERGUID	CUSTOMER_Sum_of_invoice_Amount_next_14d
0	2022-12-22 18:43:31	217d44e9-d85b-4dba-9847-2c4c071f7303	0.00
1	2023-06-10 15:09:47	44d5a090-001a-4899-be0b-a74932ed42dc	141.42
2	2022-11-29 17:08:01	f761a5d1-3b66-4faf-82f1-6cd59e2e28f8	32.69
3	2022-08-01 06:44:40	1e866814-e5a6-475d-87e3-b53377cc005b	15.48
4	2023-03-22 16:07:57	44d5a090-001a-4899-be0b-a74932ed42dc	234.78
5	2022-12-13 09:58:16	df0b0c04-f51b-48a5-b330-772cae5b9283	34.79
6	2022-07-22 15:13:54	f761a5d1-3b66-4faf-82f1-6cd59e2e28f8	9.08
7	2023-04-04 17:37:33	b5a96648-044f-4927-9877-a7d5a1956811	46.65
8	2022-08-02 16:59:08	a67434f3-f3d1-4c68-af80-fcd8fb7c9fc6	0.00
9	2022-11-07 16:58:07	8f029048-c927-4861-809c-9eccec8e4743	50.11

Compute historical features¶

In [7]:

# Create 'Simple Training data 1K Spending n2w 22S2-23S1' historical feature table
training_data_table = simple_feature_list.compute_historical_feature_table(
    training_observations,
    historical_feature_table_name="Simple Training data 1K Spending n2w 22S2-23S1",
)

# Create 'Simple Training data 1K Spending n2w 22S2-23S1' historical feature table training_data_table = simple_feature_list.compute_historical_feature_table( training_observations, historical_feature_table_name="Simple Training data 1K Spending n2w 22S2-23S1", )

Done! |████████████████████████████████████████| 100% in 35.7s (0.03%/s)        

In [8]:

display(training_data_table.to_pandas())

display(training_data_table.to_pandas())

Downloading table |████████████████████████████████████████| 1000/1000 [100%] in|█████████████████████████████▋          | ▂▂▄ 740/1000 [74%] 

	POINT_IN_TIME	GROCERYCUSTOMERGUID	CUSTOMER_Sum_of_invoice_Amount_next_14d	CUSTOMER_Age_band	CUSTOMER_Latest_invoice_Amount	CUSTOMER_Count_of_invoice_14d	CUSTOMER_Avg_of_invoice_Amount_14d	CUSTOMER_Std_of_invoice_Amount_14d	CUSTOMER_Latest_invoice_Amount_Z_Score_to_invoice_Amount_28d	CUSTOMER_vs_OVERALL_item_TotalCost_across_product_ProductGroups_26w
0	2022-08-02 10:50:09	5f18f733-ef27-423b-8fb7-6172948c9255	55.96	75-79	6.00	2	7.120	1.120000	-0.425915	0.819385
1	2022-08-29 13:47:11	2b068f1d-d99b-4c2f-a737-46f619a76cc8	29.29	50-54	22.96	5	23.590	11.680365	-0.206328	0.672926
2	2022-07-12 16:13:42	cc4220ec-16ab-4bb9-991d-deef994bf27a	115.54	35-39	11.86	4	22.830	9.219916	-0.868016	0.728931
3	2022-07-28 14:39:56	9540fe09-bf1c-41b4-847b-35f8a4a79927	40.39	30-34	7.98	6	10.825	4.767972	-0.630101	0.811204
4	2022-10-23 09:59:00	62f833f8-029c-4446-bb85-b90a867d816e	88.74	20-24	1.00	2	1.695	0.695000	-0.669011	0.315181
...	...	...	...	...	...	...	...	...	...	...
995	2022-12-09 03:42:52	5845828c-6c2f-461f-b8cb-16ce5a4ffa73	20.11	25-29	2.68	0	NaN	NaN	-0.964306	0.743137
996	2023-03-07 19:19:20	dba29407-bc25-44ab-853c-3f7c1b78f296	5.55	50-54	31.00	0	NaN	NaN	NaN	0.485408
997	2022-10-29 16:58:50	f2173f69-085f-4264-95c0-f46cbd5834aa	101.79	35-39	25.07	0	NaN	NaN	NaN	0.457425
998	2022-12-22 13:32:44	7e4e042b-6c14-4ba1-a74d-ecb70664fb2f	0.00	15-19	5.18	0	NaN	NaN	NaN	0.212817
999	2022-11-16 19:24:33	09f35825-38ef-4a01-8385-c41822f59de9	41.78	50-54	28.54	0	NaN	NaN	NaN	0.391692

1000 rows × 10 columns

In [9]:

### List historical feature tables from catalog
catalog.list_historical_feature_tables()

### List historical feature tables from catalog catalog.list_historical_feature_tables()

Out[9]:

	id	name	feature_store_name	observation_table_name	shape	created_at
0	6564b9896f39a417e1370e23	Simple Training data 1K Spending n2w 22S2-23S1	playground	1K Customers Spending next 2 weeks at time of ...	[1000, 10]	2023-11-27T15:45:46.646000

Concepts in this tutorial¶

• More on Historical feature serving

SDK reference for¶

• Historical feature table
• FeatureList.compute historical feature table()
• FeatureList.compute_historical_features() to compute directly a data frame

In [10]:

catalog.list_tables()

catalog.list_tables()

Out[10]:

	id	name	type	status	entities	created_at
0	6564b7edbeba6c193e0fe3be	GROCERYPRODUCT	dimension_table	PUBLIC_DRAFT	[product, productgroup]	2023-11-27T15:38:22.276000
1	6564b7ecbeba6c193e0fe3bd	INVOICEITEMS	item_table	PUBLIC_DRAFT	[item, invoice, product]	2023-11-27T15:38:21.489000
2	6564b7ebbeba6c193e0fe3bc	GROCERYINVOICE	event_table	PUBLIC_DRAFT	[invoice, customer]	2023-11-27T15:38:20.150000
3	6564b7eabeba6c193e0fe3bb	GROCERYCUSTOMER	scd_table	PUBLIC_DRAFT	[customer, frenchstate]	2023-11-27T15:38:19.300000

In [ ]: