V
Vanaja
About
Detail
New Jersey, United States
Résumé
Jobs
verified_user
0% verified
Data EngineerKeyBankOct 2023 - Current (3 years 1 month)As a Data Engineer, I developed high-throughput ETL pipelines using Azure Data Factory, ingesting IoT and telemetry data from industrial facilities to power real-time energy intensity insights. I built scalable data ingestion workflows with PySpark, handling large daily data volumes to ensure high-performance storage and reliable data availability for energy analytics. Additionally, I designed robust data structures for raw and aggregated datasets in cloud platforms, including Snowflake and Azure, balancing query cost, latency, and data governance. I implemented data quality and anomaly detection layers using PySpark, identifying sensor malfunctions and unusual energy usage patterns to maintain trust in operational dashboards. To ensure ope- HData EngineerHESA ENTERPRISES Pvt Ltd, IndiaJun 2021 - Aug 2022 (1 year 3 months)As a Data Engineer, I designed automated processes to manage Iceberg table lifecycle, handling compaction, partitioning, retention, and storage optimization for transactional payment data. I implemented disaster-recovery and governance features, automating snapshot management and table rollback strategies to ensure compliance and business continuity for payment platforms. My role enabled seamless integration with multiple compute and query engines like Spark and Snowflake, ensuring consistent data access across diverse analytics workflows for payment processing. I developed scalable cloud data pipelines using Azure Data Factory, ingesting diverse raw payment sources into feature stores for real-time analytics. Additionally, I built PySpark
Education
verified_user
0% verified
- UMaster of Science in Computer ScienceUniversity of Bridgeport, ConnecticutSep 2022 - Jul 2024 (1 year 11 months)