Data Engineer
2RP Net
Nov 2021 - Aug 2024 (2 years 10 months)
Developed and maintained PySpark and SQL pipelines for credit, fraud, and collections analytics in a large-scale financial environment.Built and orchestrated batch data workflows using Apache NiFi, Airflow, Hive, and Impala in a Cloudera-based environment on OCI.Supported data ingestion and API integrations, handling authentication, JSON payloads, and production incident resolution.Participated in end-to-end data projects, collaborating with analysts, stakeholders, and client teams in Agile environments.Contributed to a large-scale cloud migration to GCP, focusing on data validation, consistency checks, and pipeline reliability.