Skip to content

8. Ideate Features

FeatureByte provides two primary methods for feature creation:

  • Manual Creation: Utilize the SDK declarative framework for complete control over the feature engineering process.
  • Automatic Creation: Accelerate feature engineering through Feature Ideation an AI-powered, automated method that intelligently suggests relevant features for your use case.

How FeatureByte Ideates Features

Feature Ideation adopts an agentic approach that dynamically tailors feature generation to your use case. The process includes:

  1. Analyzing tables and relationships to identify relevant data.
  2. Inferring missing semantic tags based on column metadata.
  3. Recommending column transformations, such as time deltas, ratios, and differences.
  4. Identifying key filters to isolate critical events.
  5. Highlighting key columns for further feature engineering.
  6. Proposing appropriate aggregation time windows.
  7. Analyzing event frequency patterns to detect timing signals.
  8. Recommending and evaluating features for their semantic relevance.
  9. Promoting feature reuse by detecting duplicates in the Catalog.
  10. Conducting EDA on each feature and assigning an individual Predictive Score.
  11. Selecting the optimal feature set based on SHAP value analysis.

Every step is transparently documented to ensure full traceability.

Modes of Operation

Feature Ideation supports two modes:

  • Fully automated mode: Executes the full workflow end-to-end with minimal user intervention.
  • Semi-automated mode: Gives you control to configure ideation settings, review and refine results, and reuse insights from previous runs.

This tutorial focuses on the Semi-Automated Mode.

Tutorial Workflow

  1. Initial Ideation: Run Feature Ideation on a single time window to identify the most informative columns in your dataset.

  2. Refined Ideation: Use the outputs from the first run to prune uninformative columns, enable filter detection, and allow the system to automatically select optimal aggregation windows.

This two-step workflow reduces the number of ideated features in scenarios involving many tables and columns, while preserving accuracy by enabling a more targeted feature engineering process in the second step.

Note

If you want to learn how to manually create features, please consult our SDK tutorials.

Step 1: Select Your Use Case

  1. Navigate to Feature Ideation from the 'Ideate' section of the menu.

  2. Select the use case: "Loan Default by client". Name

Step 2: Initiate a New Feature Ideation Workflow

  1. Click New Ideation Button to start the Feature Ideation process. Name

  2. Edit the Feature Ideation name and description by clicking Edit Button. Name

Step 3: Start Ideation with a single aggregation window (52 weeks)

  1. Begin the automated Feature Ideation workflow by clicking Auto Run Button.

  2. You can stop the process at any stage if needed. For example, you could stop at the Feature Ideation Setup to review the recommendations for feature aggregration windows, select one and then proceed to Feature Selection. Name

  3. You can configure the ideation by clicking Config Button. We will here set the feature aggregation window to 52 weeks in Ideation Setup. Name

  4. Click Config Button to start ideation.

Once the process is initiated, you’ll see confirmation that the run has started:

Name

Step 4: Review the Feature Ideation Report

After the process completes, a feature selection will be displayed for your review.

Name

  1. Access the Detailed Report, describing each step of the ideation process, by clicking Report Button next to the Ideation name "Simple Ideation".

    Name

  2. Visualize the full report with an indexed view in a new tab, by clicking Report tab Button.

    Name

Step 5: Review Features

Select the Features tab to review the suggested features.

Name

Click on a feature to open its details. You can use the filter Filter Button or the search Search Button, to find a specific feature.

Review Feature Relevance

Check Semantic Relevance in the 'About' tab of the feature.

Name

Review Feature Lineage

  1. Explore Feature Lineage by going to the 'Lineage' tab. Name

  2. Click Lineage Button to trace the feature's origin and transformations. Name

Review Feature Distribution

Analyze Feature Distribution and its relationship with the Target in the 'EDA' tab.

Name

Interact with EDA Plots. Within the 'EDA'' tab, click on the plot to activate tooltips for additional insights.

Name

Review SDK Code

For an Individual Feature

Go to the 'SDK Code' tab of the feature.

Name

For a Feature Selection

  1. Clear Filters by clicking Clear Filter Button and close the filter panel by clicking Close Filter Button.

  2. Ensure the feature selection is properly selected.

    Name

  3. Select features in the feature list by clicking Select All Button.

    Name

  4. Download the notebook to inspect the feature declaration code by clicking Download Notebook.

    Name

  5. Open the notebook to review the feature engineering code.

    Name