12. Use Embeddings

Use embeddings¶

In this tutorial, we'll use product group embeddings to compare a customer's latest invoice with their past purchases from the last 26 weeks.

To learn how to create a SQL Embedding User-Defined Function (UDF), check out the 'Bring Your Own Transformer' tutorials.

For our hosted tutorials, we have pre-configured a SQL UDF using the SBERT Transformer model on our Snowflake data warehouse. We'll register this UDF in the Catalog and apply it to analyze the ProductGroup descriptions.

In [1]:

            
                Copied!
                
import featurebyte as fb

# Set your profile to the tutorial environment
fb.use_profile("tutorial")

catalog_name = "Grocery Dataset Tutorial"
catalog = fb.Catalog.activate(catalog_name)
import featurebyte as fb

# Set your profile to the tutorial environment
fb.use_profile("tutorial")

catalog_name = "Grocery Dataset Tutorial"
catalog = fb.Catalog.activate(catalog_name)

14:27:17 | INFO     | SDK version: 3.0.1.dev45
INFO    :featurebyte:SDK version: 3.0.1.dev45
14:27:17 | INFO     | No catalog activated.
INFO    :featurebyte:No catalog activated.
14:27:17 | INFO     | Using profile: tutorial
INFO    :featurebyte:Using profile: tutorial
14:27:17 | INFO     | Using configuration file at: /Users/gxav/.featurebyte/config.yaml
INFO    :featurebyte:Using configuration file at: /Users/gxav/.featurebyte/config.yaml
14:27:17 | INFO     | Active profile: tutorial (https://tutorials.featurebyte.com/api/v1)
INFO    :featurebyte:Active profile: tutorial (https://tutorials.featurebyte.com/api/v1)
14:27:17 | INFO     | SDK version: 3.0.1.dev45
INFO    :featurebyte:SDK version: 3.0.1.dev45
14:27:17 | INFO     | No catalog activated.
INFO    :featurebyte:No catalog activated.
14:27:17 | INFO     | Catalog activated: Grocery Dataset Tutorial
INFO    :featurebyte.api.catalog:Catalog activated: Grocery Dataset Tutorial
16:09:06 | INFO     | Using configuration file at: /Users/gxav/.featurebyte/config.yaml
16:09:06 | INFO     | Active profile: tutorial (https://tutorials.featurebyte.com/api/v1)
16:09:06 | WARNING  | Remote SDK version (1.1.0.dev7) is different from local (1.1.0.dev1). Update local SDK to avoid unexpected behavior.
16:09:06 | INFO     | No catalog activated.
16:09:06 | INFO     | Catalog activated: Grocery Dataset Tutorial

Register the F_SBERT_EMBEDDING UDF to the Catalog¶

In [2]:

            
                Copied!
                
                    
                    
                
                

        
fb.UserDefinedFunction.create(
    name='embedding', 
    sql_function_name='F_SBERT_EMBEDDING',
    function_parameters=[fb.FunctionParameter(name="x", dtype=fb.enum.DBVarType.VARCHAR)],
    output_dtype=fb.enum.DBVarType.EMBEDDING,
    is_global=False,
)
fb.UserDefinedFunction.create(
    name='embedding', 
    sql_function_name='F_SBERT_EMBEDDING',
    function_parameters=[fb.FunctionParameter(name="x", dtype=fb.enum.DBVarType.VARCHAR)],
    output_dtype=fb.enum.DBVarType.EMBEDDING,
    is_global=False,
)

Out[2]:

User Defined Function

name

embedding

created_at

2025-06-02 06:27:17

updated_at

None

description

None

sql_function_name

F_SBERT_EMBEDDING

function_parameters

	name	dtype	default_value	test_value
0	x	VARCHAR	None	None

signature

embedding(x: str) -> embedding

output_dtype

EMBEDDING

feature_store_name

playground

used_by_features

[]

Apply the embedding UDF instance to ProductGroup¶

In [3]:

            
                Copied!
                
# Get embedding UDF instance.
embedding_udf = catalog.get_user_defined_function("embedding")
# Get embedding UDF instance.
embedding_udf = catalog.get_user_defined_function("embedding")

In [4]:

            
                Copied!
                
# Get view from GROCERYPRODUCT dimension table.
groceryproduct_view = catalog.get_view("GROCERYPRODUCT")
# Apply embedding to ProductGroup column in GROCERYPRODUCT view.
groceryproduct_view["ProductGroup_embedding"] = embedding_udf(groceryproduct_view["ProductGroup"])
# Get view from GROCERYPRODUCT dimension table.
groceryproduct_view = catalog.get_view("GROCERYPRODUCT")
# Apply embedding to ProductGroup column in GROCERYPRODUCT view.
groceryproduct_view["ProductGroup_embedding"] = embedding_udf(groceryproduct_view["ProductGroup"])

Get other views¶

In [5]:

            
                Copied!
                
# Get view from GROCERYINVOICE event table.
groceryinvoice_view = catalog.get_view("GROCERYINVOICE")
# Get view from INVOICEITEMS item table.
invoiceitems_view = catalog.get_view("INVOICEITEMS")
# Get view from GROCERYINVOICE event table.
groceryinvoice_view = catalog.get_view("GROCERYINVOICE")
# Get view from INVOICEITEMS item table.
invoiceitems_view = catalog.get_view("INVOICEITEMS")

Join views¶

In [6]:

            
                Copied!
                
# Join GROCERYPRODUCT view to INVOICEITEMS view.
invoiceitems_view = invoiceitems_view.join(groceryproduct_view, rprefix="product_")
# Join GROCERYPRODUCT view to INVOICEITEMS view.
invoiceitems_view = invoiceitems_view.join(groceryproduct_view, rprefix="product_")

Get the mean vector of an invoice's Product Group descriptions¶

In [7]:

            
                Copied!
                
# Group invoiceitems_view by invoice entity (GroceryInvoiceGuid).
invoiceitems_view_by_invoice = invoiceitems_view.groupby("GroceryInvoiceGuid")
# Group invoiceitems_view by invoice entity (GroceryInvoiceGuid).
invoiceitems_view_by_invoice = invoiceitems_view.groupby("GroceryInvoiceGuid")

In [8]:

            
                Copied!
                
# Mean vector of product_ProductGroup_embedding for the invoice.
invoice_mean_vector_of_item_product_productgroup_embedding = invoiceitems_view_by_invoice.aggregate(
    "product_ProductGroup_embedding", method=fb.AggFunc.AVG,
    feature_name="INVOICE_Mean_vector_of_item_product_ProductGroup_embedding"
)
# Mean vector of product_ProductGroup_embedding for the invoice.
invoice_mean_vector_of_item_product_productgroup_embedding = invoiceitems_view_by_invoice.aggregate(
    "product_ProductGroup_embedding", method=fb.AggFunc.AVG,
    feature_name="INVOICE_Mean_vector_of_item_product_ProductGroup_embedding"
)

Get the mean vector of the Customer's latest invoice¶

In [9]:

            
                Copied!
                
# Add INVOICE_Mean_vector_of_item_product_ProductGroup_embedding feature to the GROCERYINVOICE view as a column.
groceryinvoice_view = groceryinvoice_view.add_feature(
    "INVOICE_Mean_vector_of_item_product_ProductGroup_embedding",
    invoice_mean_vector_of_item_product_productgroup_embedding
)
# Add INVOICE_Mean_vector_of_item_product_ProductGroup_embedding feature to the GROCERYINVOICE view as a column.
groceryinvoice_view = groceryinvoice_view.add_feature(
    "INVOICE_Mean_vector_of_item_product_ProductGroup_embedding",
    invoice_mean_vector_of_item_product_productgroup_embedding
)

In [10]:

            
                Copied!
                
# Group GROCERYINVOICE view by customer entity (GroceryCustomerGuid).
groceryinvoice_view_by_customer = groceryinvoice_view.groupby(['GroceryCustomerGuid'])
# Group GROCERYINVOICE view by customer entity (GroceryCustomerGuid).
groceryinvoice_view_by_customer = groceryinvoice_view.groupby(['GroceryCustomerGuid'])

In [11]:

            
                Copied!
                
                    
                    
                
                

        
# Get Latest Mean vector of item product_ProductGroup_embedding for the customer
customer_latest_invoice_mean_vector_of_item_product_productgroup_embedding =\
groceryinvoice_view_by_customer.aggregate_over(
    "INVOICE_Mean_vector_of_item_product_ProductGroup_embedding", method="latest",
    feature_names=["CUSTOMER_Latest_INVOICE_Mean_vector_of_item_product_ProductGroup_embedding"],
    windows=[None]
)["CUSTOMER_Latest_INVOICE_Mean_vector_of_item_product_ProductGroup_embedding"]
# Get Latest Mean vector of item product_ProductGroup_embedding for the customer
customer_latest_invoice_mean_vector_of_item_product_productgroup_embedding =\
groceryinvoice_view_by_customer.aggregate_over(
    "INVOICE_Mean_vector_of_item_product_ProductGroup_embedding", method="latest",
    feature_names=["CUSTOMER_Latest_INVOICE_Mean_vector_of_item_product_ProductGroup_embedding"],
    windows=[None]
)["CUSTOMER_Latest_INVOICE_Mean_vector_of_item_product_ProductGroup_embedding"]

Get the mean vector for the Customer's Product Group descriptions over past 26 weeks¶

In [12]:

            
                Copied!
                
# Group INVOICEITEMS view by customer entity (GroceryCustomerGuid).
invoiceitems_view_by_customer = invoiceitems_view.groupby(['GroceryCustomerGuid'])
# Group INVOICEITEMS view by customer entity (GroceryCustomerGuid).
invoiceitems_view_by_customer = invoiceitems_view.groupby(['GroceryCustomerGuid'])

In [13]:

            
                Copied!
                
                    
                    
                
                

        
# Get Mean vector of product_ProductGroup_embedding for the customer over time.
feature_group = invoiceitems_view_by_customer.aggregate_over(
    "product_ProductGroup_embedding", method="avg",
    feature_names=[
        "CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w"
    ],
    windows=["26w"],
)
# Get CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w object from feature group.
customer_mean_vector_of_item_product_productgroup_embedding_26w =\
feature_group["CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w"]
# Get Mean vector of product_ProductGroup_embedding for the customer over time.
feature_group = invoiceitems_view_by_customer.aggregate_over(
    "product_ProductGroup_embedding", method="avg",
    feature_names=[
        "CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w"
    ],
    windows=["26w"],
)
# Get CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w object from feature group.
customer_mean_vector_of_item_product_productgroup_embedding_26w =\
feature_group["CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w"]

Derive Similarity between latest invoice and 26 weeks purchases¶

In [14]:

            
                Copied!
                
                    
                    
                
                

        
# Derive Similarity feature from cosine similarity between
# CUSTOMER_Latest_INVOICE_Mean_vector_of_item_product_ProductGroup_embedding
# and CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice =\
customer_latest_invoice_mean_vector_of_item_product_productgroup_embedding.vec.cosine_similarity(
    customer_mean_vector_of_item_product_productgroup_embedding_26w
)
# Give a name to new feature
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice.name = \
"CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w_vs_latest_invoice"
# Derive Similarity feature from cosine similarity between
# CUSTOMER_Latest_INVOICE_Mean_vector_of_item_product_ProductGroup_embedding
# and CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice =\
customer_latest_invoice_mean_vector_of_item_product_productgroup_embedding.vec.cosine_similarity(
    customer_mean_vector_of_item_product_productgroup_embedding_26w
)
# Give a name to new feature
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice.name = \
"CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w_vs_latest_invoice"

Preview feature¶

In [15]:

            
                Copied!
                
# Get observation table: 'Preview Table with 10 items'
preview_table = catalog.get_observation_table("Preview Table with 10 items")
# Get observation table: 'Preview Table with 10 items'
preview_table = catalog.get_observation_table("Preview Table with 10 items")

In [16]:

            
                Copied!
                
# Preview customer_mean_vector_of_item_product_productgroup_embedding_26w
customer_mean_vector_of_item_product_productgroup_embedding_26w.preview(preview_table)
# Preview customer_mean_vector_of_item_product_productgroup_embedding_26w
customer_mean_vector_of_item_product_productgroup_embedding_26w.preview(preview_table)

Out[16]:

	POINT_IN_TIME	GROCERYINVOICEITEMGUID	CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w
0	2023-06-09 16:38:32	6acb20fd-605d-4982-aa39-77054f08103c	[-0.055679653671241, 0.036607537203578004, -0....
1	2023-05-25 05:21:12	cf670b7a-c6bf-4598-b0c0-400378b9cab6	[-0.036810443362098, 0.054307278078735005, -0....
2	2022-11-01 14:32:22	8687e2a4-7f97-4442-873c-5c52d74404f8	[-0.05579970813751801, 0.038974709615827004, -...
3	2023-01-06 14:38:32	e63c0f14-3530-49e9-b73e-f92594e82663	[-0.05125611890158, 0.0256558639652, -0.012342...
4	2023-04-26 19:36:57	f867935a-d33a-43d1-b3bc-02c539769836	[-0.049507673506261, 0.036102647723594006, -0....
5	2023-06-02 14:24:28	0154e4b4-25a4-4276-af72-2826bbc64c31	[-0.07301349714239201, 0.041353363137788, -0.0...
6	2022-07-23 15:32:29	d87d65b8-4f78-41cc-8bd3-0064f83fe4fb	[-0.043172992735109, 0.033448811410606, -0.017...
7	2023-03-20 15:08:44	6f5299d0-fa38-4707-8108-1b66805d84e5	[-0.042076902204549, 0.024071049535537, -0.006...
8	2023-05-04 15:15:25	099cb405-5b2d-4dba-9071-a157ff0dbadc	[-0.052660999834658, 0.030567513644866002, -0....
9	2023-02-28 10:28:24	59b63729-d448-4496-8f36-de26a91e2310	[-0.048510884983021, 0.034143495114781, -0.026...

In [17]:

            
                Copied!
                
# Preview customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice.preview(preview_table)
# Preview customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice.preview(preview_table)

Out[17]:

	POINT_IN_TIME	GROCERYINVOICEITEMGUID	CUSTOMER_Mean_vector_of_item_product_ProductGroup_embedding_26w_vs_latest_invoice
0	2022-07-23 15:32:29	d87d65b8-4f78-41cc-8bd3-0064f83fe4fb	0.580733
1	2023-02-28 10:28:24	59b63729-d448-4496-8f36-de26a91e2310	0.913123
2	2023-05-25 05:21:12	cf670b7a-c6bf-4598-b0c0-400378b9cab6	0.444495
3	2023-04-26 19:36:57	f867935a-d33a-43d1-b3bc-02c539769836	0.636809
4	2023-01-06 14:38:32	e63c0f14-3530-49e9-b73e-f92594e82663	0.869239
5	2023-06-09 16:38:32	6acb20fd-605d-4982-aa39-77054f08103c	0.518619
6	2023-03-20 15:08:44	6f5299d0-fa38-4707-8108-1b66805d84e5	0.875395
7	2023-05-04 15:15:25	099cb405-5b2d-4dba-9071-a157ff0dbadc	0.926910
8	2022-11-01 14:32:22	8687e2a4-7f97-4442-873c-5c52d74404f8	0.533653
9	2023-06-02 14:24:28	0154e4b4-25a4-4276-af72-2826bbc64c31	0.794856

Save feature¶

In [18]:

            
                Copied!
                
# Save feature
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice.save()
# Save feature
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice.save()

Done! |████████████████████████████████████████| 100% in 6.2s (0.16%/s)         
Done! |████████████████████████████████████████| 100% in 6.1s (0.17%/s)

Add description¶

In [19]:

            
                Copied!
                
                    
                    
                
                

        
# Add description
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice.update_description(
	"Compare the customer's 4w Mean vector of item "
	"product_ProductGroup_embedding with the customer's most recent "
	"invoice. This comparison is done using the Cosine Similarity metric to"
	" measure how similar these mean vector embeddings are."
)
# Add description
customer_mean_vector_of_item_product_productgroup_embedding_26w_vs_latest_invoice.update_description(
	"Compare the customer's 4w Mean vector of item "
	"product_ProductGroup_embedding with the customer's most recent "
	"invoice. This comparison is done using the Cosine Similarity metric to"
	" measure how similar these mean vector embeddings are."
)