Srikanth Reddy

• Experienced Data Engineer with a proven track record spanning over 5+ years, specializing in the design, development, and deployment of Azure-based solutions. Expert in leveraging Azure technologies including Azure Data Factory, Azure Databricks, Azure Data Lake Storage Gen2, and Azure SQL Database to enhance business operations. • Demonstrated success in transitioning applications from on-premises to Azure, optimizing cloud storage and database solutions. • Skilled in designing and executing batch processing workflows using Azure Data Factory and Azure Databricks, emphasizing scalability and reliability. • Proficient in managing Databricks environments, including cluster configuration, notebook integration, job scheduling, and implementing auto-scaling features. • Migrated complex ETL workflows from legacy systems to Azure, streamlining data transformation and loading processes using Azure Data Factory. • Strong technical foundation in Big Data ecosystems, with extensive experience in Spark, Hive, MapReduce, Yarn and HDFS, and programming expertise in Python and Java. • Advanced skills in Python for data engineering tasks, including extensive use of PySpark DataFrame APIs and Spark SQL for efficient data manipulation and processing within distributed environments. • Developed and deployed comprehensive big data solutions, employing technologies such as Kafka, Spark, and Hadoop across various distributions including HortonWorks and Cloudera. • Experienced in both real-time and batch data processing, utilizing Spark, Spark Streaming, Databricks, and Delta Lake for high-throughput data handling and real-time data integration. • Highly skilled Data Engineer with extensive expertise in building and maintaining scalable and efficient data pipelines using Delta Live Tables. • Engineered robust data lakes and ETL pipelines, focusing on data integrity, efficiency, and compliance with business analytics requirements. • Expert in implementing and customizing data ingestion and integration workflows using technologies like Sqoop, Kafka, Spark Streaming and Azure Data Factory. • Orchestrated complex data pipelines and automated workflow management using Apache Airflow, enhancing operational efficiency and monitoring. • Migrated database objects and data flows into Snowflake, optimizing data sharing and analytics capabilities. • Demonstrated expertise in utilizing Snowflake's Streams and Tasks to facilitate real-time data processing and automated workflow management. Capable of implementing complex data orchestration solutions that enhance operational efficiency and data throughput. • Experienced in executing advanced data manipulation and management tasks using SnowSQL, optimizing data operations through command-line driven interactions and script automation within Snowflake environments. • Proficient in leveraging Snowpark to enable advanced data processing directly within Snowflake, using Python including creating complex user-defined functions and stored procedures that optimize and automate migration and transformation workflows. • Implemented robust data security practices, utilizing data masking techniques to ensure the confidentiality and integrity of sensitive information across storage and processing stages. Skilled in deploying dynamic and static masking strategies to meet compliance and privacy standards. • Skilled in utilizing Power BI to develop insightful, interactive dashboards and reports that drive data-driven decision-making, leveraging real-time data connectivity to provide stakeholders with up-to-date analytics and visualizations. • Applied Agile practices throughout project lifecycles, ensuring agile delivery and continuous improvement in team and project performance. • Resolved critical performance bottlenecks in data-intensive applications, significantly improving efficiency and system response times.