N

Naveen Subramanian

About

Detail

Sr. Data Engineer
United States
Contact Naveen regarding: 
work
Full-time jobs

Timeline

work
Job
school
Education

Résumé

Jobs verified_user 0% verified
  • Hearst
    Sr. Data Engineer
    Hearst
    Sep 2025 - Current (1 year 2 months)
  • Nike
    Sr. Data Engineer
    Nike
    Oct 2024 - Sep 2025 (1 year)
    • Implemented framework migration moving EMR Spark scripts to Nike’s Databricks ‘Sole’ platform
    • Developed scalable ETL pipelines in Databricks using PySpark to process terabytes of structured and unstructured data.
    • Implemented Delta Lake for real-time data streaming and batch processing in Databricks, improving pipeline efficiency by 25%.
    • Optimized Databricks cluster configurations to reduce processing costs while maintaining performance.
    • Integrated Airflow with Snowflake to orchestrate data ingestion, transformation, and reporting processes.
  • The Depository Trust & Clearing Corporation (DTCC)
    Sr. Data Engineer
    The Depository Trust & Clearing Corporation (DTCC)
    Jul 2023 - Jun 2024 (1 year)
    • Led the design and development of scalable data pipelines using PySpark and Spark.
    • Created data models, solution designs and data architecture documentation for complex information systems. Owned and managed all changes to the data models.
    • Define standards and procedures for ETL development. carry out source data analysis, data mapping and data integration using ETL tools.
    • Implemented data governance frameworks, increasing data accuracy by 20%.
    • Migrated on-premise Oracle data to AWS and Snowflake using Spark.
    • Built EDI frameworks to ingest and migrate data sources to Snowflake.
    • Utilized Spark for efficient data processing and transformation.
    • Collaborated with cross-functional teams to enhan
  • Nike
    Sr.Data Engineer
    Nike
    Mar 2021 - May 2023 (2 years 3 months)
    - Designed and implemented product features using Pyspark in collaboration with business and Technology business partners - Cleaned, prepared, and optimized the data from amazon s3 for ingestion and consumption - Implemented automated workflows and routines using airflow - Built continuous integration, test-driven development, and production deployment frameworks to seamlessly transform data - Developed architecture and design patterns to process and store high volume data sets on EMR cluster - Resolved data issues and performed root cause analysis to proactively fix product and operational issues - Designed DAGs and automated the dataflow using airflow to ensure daily execution - Develop utility scripts using python to automate data valida
  • Condé Nast
    Big Data Engineer
    Condé Nast
    Oct 2019 - Feb 2021 (1 year 5 months)
    - Build ETL/ELT pipeline in data technologies like pyspark , hive , presto and data bricks- - Extract data from source and transform the same according to business requirements and Load the data in target - Build validation module to summarize the data between source and target using pyspark- - Orchestrate dimension module using Airflow and built dependencies according to business document - Build data pipelines using pyspark to transform data from amazon S3 and stage it in Presto views - Read file from S3 on incremental basis, transform the data and load it in S3 location - Build external table using HQL to match it with the data schema loaded in the target - Analyze and validated the data in Presto using SQL queries - Create and enhanced
  • M
    Data Business Analyst
    Monticello Consulting Group
    Oct 2016 - Jun 2017 (9 months)
    • Integrated and mined large data sets, connecting data from disparate sources
    • Identified insights and patterns using traditional as well as predictive and prescriptive analytics
    • Automated data reconciliation between reports from two different data sources using Python
    • Analyzed and investigated data variances/issues in month-end or ad-hoc reports
    • Created and maintained Dashboards for showing the exact counts of trades with break-up details
  • Goldman Sachs
    Programmer Analyst, Data Engineer
    Goldman Sachs
    Jun 2015 - Aug 2016 (1 year 3 months)
    - Worked closely with traders to develop analytics, analyze data sets using SQL queries, and troubleshoot trading issues - Worked with various domain leads, analysts and IT staff to design, develop and enhance data warehouse models, business intelligence solution, scripts, procedures and metadata for supporting team’s analytic needs - Performed data analysis and data profiling using complex SQL on various sources systems including Oracle and R - Improved, redesigned, and implemented the internal scripting tools that support a global quantitative trading systems - Monitored, supported, and debugged the complex Perl/Linux applications that support trading operations - Created optimal, highly speed-sensitive, and scalable Python scripts to pro
  • VF Corporation
    Data Analytics Engineer Consultant
    VF Corporation
    Jun 2014 - Jun 2015 (1 year 1 month)
    - Developed, refined and scale data management and analytical procedures, systems, workflows, and best practices - Worked closely with external development teams to gather requirements and documented FRDs and BRDs - Developed Python scripts to create interfaces to send data from a data repository to an external SAP system - Generated reports by extracting Purchase order data from the SQL server Database and used Tableau to visualize the data - Interacted with development team to understand design flow, code review, discuss unit test plan - Analyzed and Extracted data by connecting to database using Python and SQL queries - Consulted with the operations team on deploying, migrating data, monitoring, analyzing, and tuning Sybase applications
  • Apollo Education Group
    Intern, Data Analytics Platform and Services
    Apollo Education Group
    Oct 2012 - May 2014 (1 year 8 months)
    • Implemented extensive SQL queries with joins and date functions to validate Data marts
    • Created Test Table Pairs in Informatica DVO to verify data replication between source and target database
    • Involved in execution of test data to test the functionality of the ETL Sources, Mappings and Targets
    • Used Test Management tools like JIRA to raise defects and get quality metrics for testing efforts
  • Syracuse University
    Graduate Student
    Syracuse University
    Aug 2012 - May 2014 (1 year 10 months)
    Completed my Masters in Information Management at Syracuse University
  • Wipro
    PROJECT ENGINEER
    Wipro
    Jul 2010 - May 2012 (1 year 11 months)
    Project 1: Motorola, Telecom Industry
    • Created standard testing procedures to meet user requirements on the then newly launched MOTOROLA DROID mobile
    • Coordinated with external development teams to identify, design and develop test scenarios
    • Documented test cases corresponding with the business rules and other operating conditions
    • Manually performed User interface testing
    • Generated reports and metrics to provide project status to senior management
    • Executed the test cases and raised significant defects which increased the performance of the mobile to an appreciable level Project 2: CVS Caremark, Retail and Pharmaceutical Industry
    • Developed test plans and ensured compliance with current HIPAA rul
Education verified_user 0% verified
  • Syracuse University
    master, Master's Degree Information Management
    Syracuse University
    Jan 2012 - Jan 2014 (2 years 1 month)
  • Anna University Chennai
    bachelor, Bachelor of Technology Information Technology
    Anna University Chennai
    Jan 2006 - Jan 2010 (4 years 1 month)
This is a community-created genome.