AWS Machine Learning published a post detailing how to build an explainable next-best-product (NBP) recommendation system for banking using Amazon SageMaker AI and PyTorch. The system predicts which financial product a customer is likely to purchase next, addressing challenges in capturing temporal patterns in customer adoption journeys. Traditional rule-based and collaborative filtering methods often fall short, but a multi-tower neural network with learned attention provides both accuracy and per-customer explainability, crucial for regulatory compliance.
Architecture Overview
The solution uses a multi-tower deep learning architecture with four specialized towers processing different aspects of customer data. These towers are fused via a learned attention mechanism, enabling explainable predictions. The tech stack includes AWS Glue for ETL, PyTorch for deep learning, Amazon S3 for storage (Snappy-compressed Parquet), and SageMaker AI for training and inference. Training uses ml.g5.12xlarge GPU instances.
Data Pipeline
The data pipeline has two stages: AWS Glue ETL for data unification (normalizing schemas, mapping transaction types, creating chronological records) and Amazon SageMaker Processing for ML-specific feature engineering (product adoption sequences, time-windowed aggregations over 7, 30, 60, 180, and 365 days). For large-scale data, a parallel chunked processing strategy with PyArrow and ProcessPoolExecutor is recommended.
Key Components
- ETL: AWS Glue (PySpark) for serverless, auto-scaling data processing.
- Deep Learning: PyTorch for dynamic computation graphs and variable-length sequence handling.
- Storage: Amazon S3 with Parquet format for column pruning and compression.
- Orchestration: Amazon SageMaker Pipelines for end-to-end ML pipeline management.
- Monitoring: Amazon CloudWatch for training metrics, inference latency, and model drift.
Security and Prerequisites
Prerequisites include an AWS account with permissions for SageMaker AI, S3, AWS Glue, and CloudWatch, plus familiarity with Python 3.11+ and PyTorch. The post emphasizes least-privilege IAM policies and recommends scanning dependencies for vulnerabilities. Deploying the solution creates billable resources; cleanup instructions are provided to avoid ongoing charges.