Skip to main content
Version: 0.4

Materializing Features

Feature Views define features as a view on top of your underlying data sources. Feature Views with materialization enabled will precompute feature values in Tecton's online and offline stores to enable fast access to the feature values during training and inference.

Overview​

When materialization is enabled in a Feature View, Tecton will orchestrate jobs that precompute the results of queries defined by a Feature View and write the results to Tecton-managed stores: the Online and the Offline Feature Store:

  • Online Store: A low-latency key-value store containing the most recent version of feature data. This data is used to serve features to models for low-latency predictions.

  • Offline Store: A high volume, low cost store containing historical feature values. This data is typically used for batch processes such as creating training data sets.

Types of Materialization​

Tecton handles backfill and steady-state materialization for batch and stream features based on your Feature View configuration.

Steady-state Materialization​

Steady-state Materialization refers to materialization being performed on new data arriving in real-time. Steady-state Materialization continuously occurs on all FeatureViews where Materialization is enabled.

When a FeatureView has materialization enabled, Tecton will schedule steady-state materialization jobs on an ongoing basis in order to maintain fresh feature values. The frequency of steady-state materialization is controlled by the batch_schedule parameter.

Backfill materialization​

Backfill refers to any materialization operations performed on data in the past. There are two Backfill operations.

The initial materialization of a Feature View is referred to as a bootstrap backfill. During a bootstrap materialization, existing raw data is processed into feature values.

When materialization is initially enabled for a Feature View, Tecton performs a bootstrap materialization. The amount of data materialized during a bootstrap is controlled by the feature_start_time parameter.

Enabling Feature View materialization​

Every Batch and Stream Feature Views can enable materialization to the online and/or offline store by setting online=True and/or offline=True in the Feature View decorator parameters. These options are available for the following types of Feature Views:

On-Demand Feature Views cannot be materialized since they are calculated only at request-time.

Determining if materialized feature data is being used when reading feature data​

When reading feature data using get_historical_features(), get_online_features(), or the GetFeatures endpoint of the HTTP API, materialized feature data is used if all of the following are true:

  • Your feature service is running in a live workspace

  • The constituent feature views have the option offline=True (when using get_historical_features()) or online=True (when using get_online_features() or the GetFeatures endpoint of the HTTP API)

  • (Applies to get_historical_features() only): You omitted the from_source option or set it to False

danger

Using get_online_features() is not recommended in production. It's much slower than the GetFeatures endpoint of the HTTP API, and is not designed for production workloads.

When reading feature data using get_historical_features() or get_online_features(), materialized feature data is not used if any of the following are true:

  • Your feature service is running in a development workspace

  • Any of the constituent feature views have the option offline=False (when using get_historical_features()) or online=False (when using get_online_features() or the GetFeatures endpoint of the HTTP API)

  • (Applies to get_historical_features() only): You specified from_source=True

Monitoring​

Tecton provides tools to monitor and debug production Feature Views via the Web UI, SDK, and CLI. More information on monitoring is available in Monitoring Materialization.

Was this page helpful?

Happy React is loading...