Nikhil Chinthalapally

• Experienced and proficient Data Engineer with 5+ years of experience in designing, building, and managing scalable data solutions on the Azure platform. • Proven expertise in architecting data pipelines, building data lakes, and leveraging Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake Gen2, Azure Event Hubs and Azure Blob Storageto unlock actionable insights from data. • Spearheaded the development and execution of robust Data Integration and Loading using Azure Data Factory, orchestrating seamless extraction, transformation, and loading (ETL) across diverse data sources. • Integrated Azure Logic Apps into data workflow for seamless automation, allowing for event-driven data processing. • Built ETL data pipelines using PySpark, Spark SQL and Scala to ingest, transform and load large volumes of structured and unstructured data. • Expertise in defining roles and privileges for database access, by implementing Identity and Access Management (IAM) and Role-Based Access Control (RBAC) ensuring data security and appropriate access. • Implemented Azure Functions, Azure Storage, and Service Bus Queries for large enterprise level ERP integration systems. • Experience in streaming applications using Azure Event hubs, Azure Stream Analytics, Azure Synapse Analytics. • Experience in creating and managing Azure DevOps tools for continuous integration and deployment (CI/CD) pipelines. • Optimized data ingestion, data modeling, data encryption and performance by tuning ETL workflows. • Implemented and managed data governance solutions using Azure Key Vault, Azure Active Directory, and Azure Purview to ensure data quality, compliance, and security. • Developed reusable Terraform modules for managing Azure networking, allowing rapid configuration changes and consistent security policies. • Skilled in creating and managing ETL/ELT workflows with Apache Beam or Apache Airflow to optimize data extraction, transformation and loading processes. • Developed data ingestion workflows to read data from various sources and write it to Avro, Parquet, Sequence, JSON, and ORC file formats for efficient storage and retrieval. • Expertise in Database Architecture for OLAP and OLTP Applications, Database Designing, Data Migration, and Data Warehousing concepts with emphasis on ETL. • Experience building and optimizing large scale data pipelines with Apache Hadoop, Java, HDFS, MapReduce, Hive, and Tez. • Experience in using Apache Sqoop to import and export data to and from Relational Database Systems and HDFS. • Optimized Hadoop job execution using Oozie workflows with conditional branching and data dependencies. • Experience in optimizing query performance in Hive using bucketing and partitioning techniques. • Optimized Spark jobs and workflows by tuning Spark configurations, partitioning and memory allocation settings. • Experienced in working with real-time streaming data using Apache Kafka as the data pipeline, and leveraging the Spark Streaming module for data processing. • Well versed in using ETL methodology for supporting corporate-wide solution using Informatica. • Expertise in using various Hadoop infrastructures such as Map Reduce, Pig, Hive, Zookeeper, Sqoop, Oozie, Flume, • Proficient in querying and managing data using both SQL and NoSQL databases for diverse data architectures and mastered distributed database technologies like Cassandra for highly scalable and fault-tolerant data storage. • Extensive experience in developing, maintaining, and implementing Enterprise Data Warehouse (EDW), Data Marts, ODS and Data warehouse with Star schema and Snowflake schema. • Maintained and administered Git source code repository and GitHub enterprise. • Comprehensive knowledge of Software Development Life Cycle and worked on Agile and Waterfall Methodologies. • Collaborated seamlessly with cross-functional teams and stakeholders to implement well-aligned data models, structures, and designs. • Dedicated to keeping up with the latest developments and industry best practices in cloud computing and data engineering technologies.