Skip to content

8. Ideate Features and Models

Accelerate feature engineering through Ideation an AI-powered, automated method that intelligently suggests relevant features and models for your use case.

Ideation Modes of Operation

Ideation supports two modes:

  • Fully Automated Mode – Runs the complete workflow from raw data to model generation with minimal user input.
  • Semi-Automated Mode – Allows you to customize ideation settings, review and refine the generated results, and reuse insights from previous runs.

This tutorial focuses on the Fully Automated Mode, guiding you through how to access and interpret its results:

Note

If you want to learn how to manually create features, please consult our SDK tutorials.

Step 1: Run a New Ideation Workflow

  1. Navigate to Ideation from the 'Ideate' section of the menu.

  2. Select the use case: "Loan Default by client". Name

  3. Click New Ideation Button to start the Ideation process. Optionally, you can choose to run Ideation with the Development Dataset we created in the previous tutorial. Name

    Why use a Development Dataset ?

    A Development Dataset helps speed up experimentation by working on smaller, representative data samples instead of full production tables. It is especially useful when dealing with very large tables where only a subset of the data is necessary for analysis.

  4. Edit the Ideation name and description by clicking Edit Button. Name

  5. Configure the ideation by clicking Config Button. We will here set the training and validation observation tables to train models in Modeling Setup. Once done, click Save Button. Name

  6. Begin the automated Ideation workflow by clicking Auto Run Button.

Once the process is initiated, you’ll see confirmation that the run has started:

Name

After the process completes, a list of models will be displayed for your review.

Name

How Ideation Works

Ideation dynamically tailors feature generation to your use case. The process includes:

  1. Analyzing tables and relationships to identify relevant data.
  2. Inferring missing semantic tags based on column metadata.
  3. Recommending column transformations, such as time deltas, ratios, and differences.
  4. Identifying key filters to isolate critical events.
  5. Highlighting key columns for further feature engineering.
  6. Proposing appropriate aggregation time windows.
  7. Analyzing event frequency patterns to detect timing signals.
  8. Recommending and evaluating features for their semantic relevance.
  9. Detecting existing features in the Catalog to promote feature reuse.
  10. Conducting EDA on each feature and assigning an individual Predictive Score.
  11. Selecting the optimal feature set based on SHAP value analysis.
  12. Running Machine Learning Models on the feature set.

Every step is transparently documented to ensure full traceability.

Step 2: Review the Ideation Report

  1. Access the Detailed Report, describing each step of the ideation process, by clicking Report Button next to the Ideation name "Fully Automated".

    Name

  2. Visualize the full report with an indexed view in a new tab, by clicking Report tab Button.

    Name

Step 3: Review Best Performing Model

  1. Go back to Model Training step to access the ideation leaderboard and click on More Info for an overview.

    Name

  2. Click on one Model to access more details.

    Name

  3. Go to Evaluate tab for interactive plots and insights into model's separability, ranking quality, calibration, and decision-threshold trade-offs.

  4. Select ROC Curve to assess ranking quality.

    Name

  5. Select Precision–Recall Curve to handle imbalanced classification.

    Name

  6. Analyze top-k performance via KS / Gain Curve, Lift Chart or Gain Report.

    Name

    Name

    Name

  7. Select Predicted Vs Actual Per Bin for a visual calibration check.

    Name

  8. Select Distribution to check class separability.

    Name

  9. Select Confusion Matrix to select a decision threshold and compare metrics and trade-offs interactively.

    Name

  10. Review Feature Importance. Select Per Feature Key for a deeper analysis when dictionary features are used.

    Name

  11. Go to Model Graph to review pre-processing steps and estimator.

    Name

Step 4: Review Feature Selection

Go to Feature Selection step to access suggested/manual feature selection and click on a feature selection (in our example, only one is available) to get an overview how this selection was generated, the signals captured and table columns used.

Name

Step 5: Review a Single Feature

  1. Select the Features tab of the Feature Selection step to review the suggested features. Name

  2. Click on a feature to open its details. You can use the filter Filter Button or the search Search Button, to find a specific feature. Name

  3. Check Semantic Relevance in the About tab of the feature. Name

  4. Explore Feature Lineage by going to the Lineage tab and click Lineage Button to trace the feature's origin and transformations.

    Name

    Name

  5. Analyze Feature Distribution and its relationship with the Target in the EDA tab. Name

  6. Interact with EDA Plots. Within the EDA tab, click on the plot to activate tooltips for additional insights. Name

  7. Go to the 'SDK Code' tab of the feature. Name