Nikhil Reddy Data Engineer | Actively Looking for new opportunities as a Data

Data Engineer

Nike

Jan 2023 - Current (3 years 9 months)

Responsible for requirements gathering and analyzing the data sources.
Developed ETL and ELT pipelines using Data Bricks in AWS environment.
Created databricks notebooks with delta format tables and implemented lake house architecture.
Developed Pyspark and SparkSQL scripts for multiple data products on the business logic.
Worked with various big data file formats such as Parquet, CSV and JSON in different scenarios of building data pipelines
Developed Data Cleaning and Data Validation scripts before applying business transformations on Databricks.
Worked with Visual studio and Repos in AWS DevOps to do code commits and code migrations across dev, test and prod environments.
Created stored procedures

Data Engineer

Airbnb

Nov 2021 - Current (4 years 11 months)

• Implemented Agile Methodology for building an internal application.
• Developed Spark applications using Scala
• Worked in designing tables in Hive and processing data like importing and exporting of databases to the HDFS, involved in processing large datasets of different forms data.
• Developed Spark/Scala, Python for regular expression (regex) project in the Hadoop/Hive environment with Linux/Windows for big data resources.
• Responsible for developing data pipelines with Amazon AWS to extract the data from S3 buckets and store in HDFS
• Querying big data, Data pipeline design and implementation for data extraction, Scheduling and Automation of tasks from data fetching, data cleaning to model testing with DAG i

Data Engineer

MetLife

Sep 2020 - Oct 2021 (1 year 2 months)

• Working with an Agile environments including the Scrum methodology within the cross-functional team and act as a liaison between the business user group and the technical team.
• Prepared scripts to automate the ingestion process using Python and Scala as needed through various sources such as API, AWS S3, Teradata and snowflake.
• Involved in developing and documenting the ETL (Extract, Transformation and Load) strategy to populate the Data Warehouse from various source systems.
• Using Jupyter notebooks to developing, testing & analyzing Spark jobs before Scheduling Customized Spark jobs.
• Designing and developing SSIS Packages to import and export data from MS Excel, SQL Server, and Flat files.
• Analyzing, de

B

Data Engineer

Blue light IT Solutions

Sep 2018 - Jan 2019 (5 months)

• Developed business process models in Waterfall to document existing and future business processes.
• Worked with Python and R Libraries like NumPy, Pandas, Scikit-Learn, SciPy and TensorFlow.
• Used AWS data pipeline for Data Extraction, Transformation and Loading from homogeneous or heterogeneous data sources and built various graphs for business decision-making using Python matplotlib library.
• Designed and developed SSIS Packages to import and export data from MS Excel, SQL Server, and Flat files.
• Developed SQL queries using stored procedures, common table expressions (CTEs), temporary table to support SSRS and Power BI reports.
• Defined and deployed monitoring, metrics, and logging systems on AWS.
• S

Nikhil Reddy

About

Detail

Timeline

Résumé

Nikhil Reddy

About

Detail

Behavioral traits