Senior Data Engineer – Risk & Compliance (CNP Protect)

About This RoleAI processing…

Key Responsibilities

• 1
  Design, build, and operate production-grade, Python-based data pipelines that power ML workflows, including real-time and near-real-time data processing and aggregations.
• 2
  Own end-to-end data quality and reliability by implementing validations, automated testing, monitoring, and alerting — with clearly defined SLAs and ownership boundaries.
• 3
  Strengthen and document our data foundations and governance, covering architecture, data lineage, dataset definitions, and dependency management.
• 4
  Partner closely with the Risk Platform, ML Data Platform, and cross-functional teams — including data scientists, software engineers, and analysts — to align on requirements and deliver changes safely to production.
• 5
  Drive Feature Store governance, improve usability, and standardize our Feature Store setup.

Requirements

• 4+ years of experience as a Data Engineer, focusing on building and deploying scalable data solutions.
• Strong proficiency in Python and PySpark, with solid software development engineering skills for designing computationally effective solutions in large-scale production environments.
• Experience with feature stores and feature engineering pipelines (building and maintaining enriched attributes for online/offline use).
• Experience building and maintaining data pipelines, with a good understanding of API design, containers (Docker), and version control (Git).
• Experience with AWS technologies (S3, EKS, Keyspaces, Athena) or equivalent cloud provider.
• Experience with streaming or event-driven architectures (e.g., Kafka) and familiarity with open table formats (e.g., Apache Iceberg, Delta Lake).

Perks & Benefits