How to Stop Your AI From Blocking Change!

Change is the only constant. It drives our development, adaptation, and progression toward a better tomorrow. Yet modern AI systems are based on machine learning algorithms that assume that past patterns will continue into the future. Clever feature engineering can help an AI system identify changes and adapt.

Stability is the consistency or similarity of behaviors or events across time. For example, a mobile phone customer may always phone their family for an hour on Sunday afternoons. A network router may always hit its bandwidth limits late in the afternoon on weekdays. A grocery customer may always buy wholemeal bread, half a dozen eggs, and skim milk.

Things become interesting when stable historical patterns cease. When a customer changes their behaviors or changes their residential address, there is the opportunity to sell them new products. When banking transactions change, it may be an indicator of fraud or increased credit risk. If a production line’s instrumentation metrics vary, it could signal that the manufacturing equipment requires maintenance.

Data scientists use similarity metrics to create stability signals, including:

• The cosine similarity of event labels from a recent period versus a longer and older historical period
• Ratios of the latest numeric value to the average or maximum of a historical period
• Flagging values and labels that are outside the range of historical data
• Flagging or counting the number of times a slowly changing attribute (e.g. address) has changed 

Here are some tips to become world-class at feature engineering for stability signals:

• Use a metric that standardizes for the number of events e.g. entropy
• Use a metric that standardizes for the historical magnitude of a value e.g. the ratio of new to old values
• Use a time window that contains enough historical examples for a robust statistic, and to include seasonal variations e.g. 14 days works better than 10 days

The entropy calculations used within stability metrics are computationally intensive. We’ve built an open-source feature engineering library that makes it easy to calculate entropy at scale inside your database. Click here to access our open-source SDK, with worked examples in Python.