Skip to content

System Architecture

FeatureByte is developed to integrate seamlessly with data warehouses, enhancing security and efficiency by bypassing large-scale outbound data transfers. This integration allows feature calculations to be performed within the data warehouse, leveraging scalability, stability, and efficiency.


FeatureByte Architecture

Component Packaging Purpose Comments
Featurebyte SDK Python Package Connects to the API service to provide feature authoring and management functionality through python classes and functions.
API Service Docker Container REST-API service that validates and executes requests, queries data warehouses, and stores data. Horizontally scalable for high availability.
Worker Docker Container Executes asynchronous or scheduled tasks. Horizontally scalable for high availability.
MongoDB Docker Container Store metadata for created assets.
Redis Docker Container Broker and queue for workers, messenger service for publishing progress updates.
Query Graph Transpiler Python Package Construct data transformation steps as a query graph, which can be transpiled to platform-specific SQL.
Source Tables Data Warehouse Tables used as data sources for feature engineering.
Feature Store Data Warehouse Database that store data used to support feature serving.