9. Create Window Aggregate Features

Create window aggregate features¶

Next feature type we will consider is window aggregate feature. These are features generated by aggregating data within specific time frame.

In [1]:

import featurebyte as fb

# Set your profile to the tutorial environment
fb.use_profile("tutorial")

catalog_name = "Grocery Dataset Tutorial"
catalog = fb.Catalog.activate(catalog_name)

import featurebyte as fb # Set your profile to the tutorial environment fb.use_profile("tutorial") catalog_name = "Grocery Dataset Tutorial" catalog = fb.Catalog.activate(catalog_name)

10:47:03 | WARNING  | Service endpoint is inaccessible: http://featurebyte-server:8088/
10:47:03 | INFO     | Using profile: tutorial
10:47:03 | INFO     | Using configuration file at: /Users/gxav/.featurebyte/config.yaml
10:47:03 | INFO     | Active profile: tutorial (https://tutorials.featurebyte.com/api/v1)
10:47:04 | INFO     | SDK version: 2.0.1.dev67
10:47:04 | INFO     | No catalog activated.
10:47:04 | INFO     | Catalog activated: Grocery Dataset Tutorial
16:07:43 | INFO     | Using configuration file at: /Users/gxav/.featurebyte/config.yaml
16:07:43 | INFO     | Active profile: tutorial (https://tutorials.featurebyte.com/api/v1)
16:07:43 | WARNING  | Remote SDK version (1.1.0.dev7) is different from local (1.1.0.dev1). Update local SDK to avoid unexpected behavior.
16:07:43 | INFO     | No catalog activated.
16:07:43 | INFO     | Catalog activated: Grocery Dataset Tutorial

In [2]:

# Set desired windows
windows = ['14d', '28d']

# Set desired windows windows = ['14d', '28d']

Do window aggregation from INVOICEITEMS¶

Let's start with some aggregations from the items view and create features for the interaction between Customer and Product Group.

In [3]:

# Get view from GROCERYPRODUCT dimension table.
groceryproduct_view = catalog.get_view("GROCERYPRODUCT")
# Get view from INVOICEITEMS item table.
invoiceitems_view = catalog.get_view("INVOICEITEMS")
# Join GROCERYPRODUCT view to INVOICEITEMS view.
invoiceitems_view = invoiceitems_view.join(groceryproduct_view, rsuffix="")

# Get view from GROCERYPRODUCT dimension table. groceryproduct_view = catalog.get_view("GROCERYPRODUCT") # Get view from INVOICEITEMS item table. invoiceitems_view = catalog.get_view("INVOICEITEMS") # Join GROCERYPRODUCT view to INVOICEITEMS view. invoiceitems_view = invoiceitems_view.join(groceryproduct_view, rsuffix="")

In [4]:

# Group GROCERYINVOICE view by customer entity (GroceryCustomerGuid) and productgroup entity (ProductGroup).
invoiceitems_view_by_customer_x_productgroup = invoiceitems_view.groupby(
    ['GroceryCustomerGuid', 'ProductGroup']
)

# Group GROCERYINVOICE view by customer entity (GroceryCustomerGuid) and productgroup entity (ProductGroup). invoiceitems_view_by_customer_x_productgroup = invoiceitems_view.groupby( ['GroceryCustomerGuid', 'ProductGroup'] )

In [5]:

# Get Sum of TotalCost for the customer x productgroup over time.
customer_productgroup_sum_of_totalcost_14d_28d = \
invoiceitems_view_by_customer_x_productgroup.aggregate_over(
    "TotalCost", method="sum",
    feature_names=[
        "CUSTOMER_x_PRODUCTGROUP_Sum_of_item_TotalCost"
        + "_" + w for w in windows
    ],
    windows=windows
)

# Get Sum of TotalCost for the customer x productgroup over time. customer_productgroup_sum_of_totalcost_14d_28d = \ invoiceitems_view_by_customer_x_productgroup.aggregate_over( "TotalCost", method="sum", feature_names=[ "CUSTOMER_x_PRODUCTGROUP_Sum_of_item_TotalCost" + "_" + w for w in windows ], windows=windows )

In [6]:

# Get Latest Interaction between Customer and ProductGroup
customer_x_productgroup_latest_timestamp = \
invoiceitems_view_by_customer_x_productgroup.aggregate_over(
    "Timestamp", method="latest",
    feature_names=["CUSTOMER_x_PRODUCTGROUP_Latest_Timestamp"],
    windows=[None]
)["CUSTOMER_x_PRODUCTGROUP_Latest_Timestamp"]

# Get Latest Interaction between Customer and ProductGroup customer_x_productgroup_latest_timestamp = \ invoiceitems_view_by_customer_x_productgroup.aggregate_over( "Timestamp", method="latest", feature_names=["CUSTOMER_x_PRODUCTGROUP_Latest_Timestamp"], windows=[None] )["CUSTOMER_x_PRODUCTGROUP_Latest_Timestamp"]

In [7]:

# Create recency feature: Time Since Latest Interaction between Customer and Product Group
customer_x_productgroup_time_since_latest_timestamp = (
    fb.RequestColumn.point_in_time()
    - customer_x_productgroup_latest_timestamp
).dt.hour
customer_x_productgroup_time_since_latest_timestamp.name = \
"CUSTOMER_x_PRODUCTGROUP_Time_Since_Latest_Timestamp"

# Create recency feature: Time Since Latest Interaction between Customer and Product Group customer_x_productgroup_time_since_latest_timestamp = ( fb.RequestColumn.point_in_time() - customer_x_productgroup_latest_timestamp ).dt.hour customer_x_productgroup_time_since_latest_timestamp.name = \ "CUSTOMER_x_PRODUCTGROUP_Time_Since_Latest_Timestamp"

Do window aggregation from GROCERYINVOICE¶

Now, let's do some aggregations on the invoices view for the Customer entity.

In [8]:

# Get view from GROCERYINVOICE event table.
groceryinvoice_view = catalog.get_view("GROCERYINVOICE")

# Get view from GROCERYINVOICE event table. groceryinvoice_view = catalog.get_view("GROCERYINVOICE")

In [9]:

# Group GROCERYINVOICE view by customer entity (GroceryCustomerGuid).
groceryinvoice_view_by_customer = groceryinvoice_view.groupby(['GroceryCustomerGuid'])

# Group GROCERYINVOICE view by customer entity (GroceryCustomerGuid). groceryinvoice_view_by_customer = groceryinvoice_view.groupby(['GroceryCustomerGuid'])

In [10]:

# Get Latest invoice Amount for the customer
customer_latest_invoice_amount = groceryinvoice_view_by_customer.aggregate_over(
    "Amount", method="latest",
    feature_names=["CUSTOMER_Latest_invoice_Amount"],
    windows=[None]
)["CUSTOMER_Latest_invoice_Amount"]

# Get Latest invoice Amount for the customer customer_latest_invoice_amount = groceryinvoice_view_by_customer.aggregate_over( "Amount", method="latest", feature_names=["CUSTOMER_Latest_invoice_Amount"], windows=[None] )["CUSTOMER_Latest_invoice_Amount"]

In [11]:

# Get Count of invoices for the customer
customer_count_of_invoice_14d_28d = groceryinvoice_view_by_customer.aggregate_over(
    value_column=None,
    method="count",
    feature_names=[
        "CUSTOMER_Count_of_invoice"
        + "_" + w for w in windows
    ],
    windows=windows
)

# Get Count of invoices for the customer customer_count_of_invoice_14d_28d = groceryinvoice_view_by_customer.aggregate_over( value_column=None, method="count", feature_names=[ "CUSTOMER_Count_of_invoice" + "_" + w for w in windows ], windows=windows )

In [12]:

# Get Avg of Amount for the customer over time.
customer_avg_of_invoice_amount_14d_28d = groceryinvoice_view_by_customer.aggregate_over(
    "Amount", method="avg",
    feature_names=[
        "CUSTOMER_Avg_of_invoice_Amount"
        + "_" + w for w in windows
    ],
    windows=windows
)

# Get Avg of Amount for the customer over time. customer_avg_of_invoice_amount_14d_28d = groceryinvoice_view_by_customer.aggregate_over( "Amount", method="avg", feature_names=[ "CUSTOMER_Avg_of_invoice_Amount" + "_" + w for w in windows ], windows=windows )

In [13]:

# Get Std of Amount for the customer over time.
customer_std_of_invoice_amount_14d_28d = groceryinvoice_view_by_customer.aggregate_over(
    "Amount", method="std",
    feature_names=[
        "CUSTOMER_Std_of_invoice_Amount"
        + "_" + w for w in windows
    ],
    windows=windows
)

# Get Std of Amount for the customer over time. customer_std_of_invoice_amount_14d_28d = groceryinvoice_view_by_customer.aggregate_over( "Amount", method="std", feature_names=[ "CUSTOMER_Std_of_invoice_Amount" + "_" + w for w in windows ], windows=windows )

Preview a feature group¶

For convenience, we can create a feature group to preview/save all features we just created.

In [14]:

feature_group = fb.FeatureGroup([
    customer_x_productgroup_time_since_latest_timestamp,
    customer_productgroup_sum_of_totalcost_14d_28d,
    customer_latest_invoice_amount,
    customer_count_of_invoice_14d_28d,
    customer_avg_of_invoice_amount_14d_28d,
    customer_std_of_invoice_amount_14d_28d,
])

feature_group = fb.FeatureGroup([ customer_x_productgroup_time_since_latest_timestamp, customer_productgroup_sum_of_totalcost_14d_28d, customer_latest_invoice_amount, customer_count_of_invoice_14d_28d, customer_avg_of_invoice_amount_14d_28d, customer_std_of_invoice_amount_14d_28d, ])

In [15]:

# Check the primary entity of the feature group. It should be the interaction Customer x ProductGroup.
feature_group.primary_entity

# Check the primary entity of the feature group. It should be the interaction Customer x ProductGroup. feature_group.primary_entity

Out[15]:

[<featurebyte.api.entity.Entity at 0x31b6a3880>
 {
   'name': 'customer',
   'created_at': '2024-12-18T02:40:54.536000',
   'updated_at': '2024-12-18T02:40:58.932000',
   'description': None,
   'serving_names': [
     'GROCERYCUSTOMERGUID'
   ],
   'catalog_name': 'Grocery Dataset Tutorial'
 },
 <featurebyte.api.entity.Entity at 0x31b7af470>
 {
   'name': 'productgroup',
   'created_at': '2024-12-18T02:40:55.766000',
   'updated_at': '2024-12-18T02:41:00.663000',
   'description': None,
   'serving_names': [
     'PRODUCTGROUP'
   ],
   'catalog_name': 'Grocery Dataset Tutorial'
 }]

In [16]:

# Get observation table: 'Preview Table with 10 items'
preview_table = catalog.get_observation_table("Preview Table with 10 items")

# Get observation table: 'Preview Table with 10 items' preview_table = catalog.get_observation_table("Preview Table with 10 items")

In [17]:

# Preview feature_group
feature_group.preview(preview_table)

# Preview feature_group feature_group.preview(preview_table)

Out[17]:

	POINT_IN_TIME	GROCERYINVOICEITEMGUID	CUSTOMER_x_PRODUCTGROUP_Time_Since_Latest_Timestamp	CUSTOMER_x_PRODUCTGROUP_Sum_of_item_TotalCost_14d	CUSTOMER_x_PRODUCTGROUP_Sum_of_item_TotalCost_28d	CUSTOMER_Latest_invoice_Amount	CUSTOMER_Count_of_invoice_14d	CUSTOMER_Count_of_invoice_28d	CUSTOMER_Avg_of_invoice_Amount_14d	CUSTOMER_Avg_of_invoice_Amount_28d	CUSTOMER_Std_of_invoice_Amount_14d	CUSTOMER_Std_of_invoice_Amount_28d
0	2022-07-23 17:33:25	849454c5-6640-419d-871d-1f0895a1c3b4	813.648056	NaN	NaN	4.64	1.0	1.0	4.640000	4.640000	0.000000	0.000000
1	2023-01-06 08:49:29	dad86e21-3af4-4b5b-8058-60f946b6dac5	596.949444	NaN	5.99	16.51	7.0	13.0	10.618571	16.290000	3.572868	13.112894
2	2023-06-06 16:13:44	bdc6b6bb-a23a-48c8-bb3a-622d9161c0e8	1007.838611	NaN	NaN	21.27	NaN	4.0	NaN	13.190000	NaN	8.014637
3	2022-09-07 09:16:15	53eac49e-7ecd-4cb6-a1c0-38e3900efd7a	69.813889	18.05	21.74	48.67	5.0	10.0	41.372000	30.554000	7.211342	20.086457
4	2022-08-22 06:10:42	9267d0dd-9685-4667-8f06-1761abe73c4d	181.691667	5.99	18.45	7.77	2.0	5.0	34.075000	29.366000	26.305000	26.319020
5	2023-05-21 13:59:44	989643fe-1377-4f5f-8f38-f349a611da0c	171.889444	4.27	7.05	4.79	4.0	5.0	22.697500	26.470000	15.060992	15.440006
6	2023-03-07 17:53:55	cb478a4e-9266-4523-8ee0-e205881cc5f5	2761.916944	NaN	NaN	24.21	NaN	NaN	NaN	NaN	NaN	NaN
7	2022-12-30 16:42:05	4dd1487a-0379-4eef-b200-97ac1bb1164f	292.786944	0.89	5.35	27.73	8.0	21.0	16.381250	15.061905	9.945394	9.450327
8	2022-09-23 18:09:33	459a1b6e-1239-46d1-9e40-539c7e895483	217.736944	2.69	8.07	41.98	2.0	4.0	65.835000	46.910000	23.855000	27.137845
9	2023-03-21 13:49:48	84225bbb-adb8-451e-98c0-897c83c2fad9	259.306111	7.67	11.73	21.90	2.0	4.0	19.690000	22.205000	2.210000	4.798445

Save features into catalog¶

With feature groups we can do it in one call.

In [18]:

feature_group.save()

feature_group.save()

Done! |████████████████████████████████████████| 100% in 9.1s (0.11%/s)         
Done! |████████████████████████████████████████| 100% in 6.1s (0.17%/s)         
Loading Feature(s) |████████████████████████████████████████| 10/10 [100%] in 0.
Done! |████████████████████████████████████████| 100% in 9.1s (0.11%/s)         
Done! |████████████████████████████████████████| 100% in 6.1s (0.17%/s)         
Loading Feature(s) |████████████████████████████████████████| 10/10 [100%] in 0.

Add description¶

In [25]:

# Add description
customer_x_productgroup_time_since_latest_timestamp.update_description(
    "Time Since Latest interaction between the customer and the product group"
)

# Add description customer_x_productgroup_time_since_latest_timestamp.update_description( "Time Since Latest interaction between the customer and the product group" )

In [26]:

# Add description
customer_productgroup_sum_of_totalcost_14d = \
customer_productgroup_sum_of_totalcost_14d_28d["CUSTOMER_x_PRODUCTGROUP_Sum_of_item_TotalCost_14d"]
customer_productgroup_sum_of_totalcost_14d.update_description(
    "Total spent by the customer on the product group over a 14d period."
)
customer_productgroup_sum_of_totalcost_28d = \
customer_productgroup_sum_of_totalcost_14d_28d["CUSTOMER_x_PRODUCTGROUP_Sum_of_item_TotalCost_28d"]
customer_productgroup_sum_of_totalcost_28d.update_description(
    "Total spent by the customer on the product group over a 28d period."
)

# Add description customer_productgroup_sum_of_totalcost_14d = \ customer_productgroup_sum_of_totalcost_14d_28d["CUSTOMER_x_PRODUCTGROUP_Sum_of_item_TotalCost_14d"] customer_productgroup_sum_of_totalcost_14d.update_description( "Total spent by the customer on the product group over a 14d period." ) customer_productgroup_sum_of_totalcost_28d = \ customer_productgroup_sum_of_totalcost_14d_28d["CUSTOMER_x_PRODUCTGROUP_Sum_of_item_TotalCost_28d"] customer_productgroup_sum_of_totalcost_28d.update_description( "Total spent by the customer on the product group over a 28d period." )

In [27]:

# Add description
customer_latest_invoice_amount.update_description("Latest invoice Amount for the customer")

# Add description customer_latest_invoice_amount.update_description("Latest invoice Amount for the customer")

In [28]:

# Add description
customer_count_of_invoice_14d = customer_count_of_invoice_14d_28d["CUSTOMER_Count_of_invoice_14d"]
customer_count_of_invoice_14d.update_description(
    "Count of invoice for the customer over a 14d period."
)
customer_count_of_invoice_28d = customer_count_of_invoice_14d_28d["CUSTOMER_Count_of_invoice_28d"]
customer_count_of_invoice_28d.update_description(
    "Count of invoice for the customer over a 28d period."
)

# Add description customer_count_of_invoice_14d = customer_count_of_invoice_14d_28d["CUSTOMER_Count_of_invoice_14d"] customer_count_of_invoice_14d.update_description( "Count of invoice for the customer over a 14d period." ) customer_count_of_invoice_28d = customer_count_of_invoice_14d_28d["CUSTOMER_Count_of_invoice_28d"] customer_count_of_invoice_28d.update_description( "Count of invoice for the customer over a 28d period." )

In [29]:

# Add description
customer_avg_of_invoice_amount_14d = customer_avg_of_invoice_amount_14d_28d["CUSTOMER_Avg_of_invoice_Amount_14d"]
customer_avg_of_invoice_amount_14d.update_description(
    "Avg of invoice Amount for the customer over a 14d period."
)
customer_avg_of_invoice_amount_28d = customer_avg_of_invoice_amount_14d_28d["CUSTOMER_Avg_of_invoice_Amount_28d"]
customer_avg_of_invoice_amount_28d.update_description(
    "Avg of invoice Amount for the customer over a 28d period."
)

# Add description customer_avg_of_invoice_amount_14d = customer_avg_of_invoice_amount_14d_28d["CUSTOMER_Avg_of_invoice_Amount_14d"] customer_avg_of_invoice_amount_14d.update_description( "Avg of invoice Amount for the customer over a 14d period." ) customer_avg_of_invoice_amount_28d = customer_avg_of_invoice_amount_14d_28d["CUSTOMER_Avg_of_invoice_Amount_28d"] customer_avg_of_invoice_amount_28d.update_description( "Avg of invoice Amount for the customer over a 28d period." )

In [30]:

# Add description
customer_std_of_invoice_amount_14d = customer_std_of_invoice_amount_14d_28d["CUSTOMER_Std_of_invoice_Amount_14d"]
customer_std_of_invoice_amount_14d.update_description(
    "Std of invoice Amount for the customer over a 14d period."
)
customer_std_of_invoice_amount_28d = customer_std_of_invoice_amount_14d_28d["CUSTOMER_Std_of_invoice_Amount_28d"]
customer_std_of_invoice_amount_28d.update_description(
    "Std of invoice Amount for the customer over a 28d period."
)

# Add description customer_std_of_invoice_amount_14d = customer_std_of_invoice_amount_14d_28d["CUSTOMER_Std_of_invoice_Amount_14d"] customer_std_of_invoice_amount_14d.update_description( "Std of invoice Amount for the customer over a 14d period." ) customer_std_of_invoice_amount_28d = customer_std_of_invoice_amount_14d_28d["CUSTOMER_Std_of_invoice_Amount_28d"] customer_std_of_invoice_amount_28d.update_description( "Std of invoice Amount for the customer over a 28d period." )

Concepts in this tutorial¶

• Aggregate Features
• Window Aggregate Features

SDK reference for¶

• View.groupby()
• Groupby.aggregate_over()
• List of aggregate functions
• FeatureGroup
• FeatureGroup.save()

In [ ]: