Sai Yashwanth

• Highly skilled Data Engineer with 5+ years of experience in building and deploying data pipelines for cloud-based data warehouses. • Proven expertise in Snowflake for data ingestion, transformation, and schema creation. Experience with AWS services like S3 and Data Factory for data movement and orchestration. • Successfully migrated legacy data to Snowflake, ensuring data quality and lineage. • Extensive experience in information technology with an expert hand in the areas of Big Data, Hadoop Spark, Hive, Impala, Sqoop, Flume, Kafka, SQL tuning, ETL development, report development, SAS, database development, data modeling and strong knowledge of oracle database architecture. • Proven ability to build efficient data pipelines using Snowpipe for continuous data loading from AWS S3 and other sources. • Experience in Big Data analytics, Data manipulation, using Hadoop Eco system tools Map - Reduce, HDFS, Yarn/MRv2, Pig, Hive, HDFS, HBase, Spark, Kafka, Flume, Sqoop, Oozie, Avro, Sqoop, GCP, Azure, Spring Boot, Spark integration with Cassandra, Avro, Solr and Zookeeper. • Strong experience in migrating other databases to Snowflake. • Managing Database, Azure Data Platform services (Azure Data Lake (ADLS), Data Factory (ADF), Data Lake Analytics, Stream Analytics, Azure SQL DW, HDInsight/Data bricks, NoSQL DB), SQL Server, Oracle, Data Warehouse etc. Build multiple Data Lakes. • Extensive experience in Text Analytics, generating data visualizations using R, Python and creating dashboards using tools like Tableau, Power BI. • Created Snowflake Schemas by normalizing the dimension tables as appropriate, and creating a Sub Dimension named Demographic as a subset to the Customer Dimension. • Utilized Kubernetes and Docker for the runtime environment for the CI/CD system to build, test, and deploy. Experience in working on creating and running Docker images with multiple micro services. • Hands-on experience in developing and deploying enterprise-based applications using major Hadoop ecosystem components like Map Reduce, YARN, Hive, HBase, Flume, Sqoop, Spark MLlib, Spark GraphX, Spark SQL, Kafka. • Proficient with Spark Core, Spark SQL, Spark MLlib, Spark GraphX and Spark Streaming for processing and transforming complex data using in-memory computing capabilities written in Scala. Worked with Spark to improve efficiency of existing algorithms using Spark Context, Spark SQL, Spark MLlib, Data Frame, Pair RDD's and Spark YARN. • Having extensive experience in Microsoft Azure Cloud Computing, GCP and SQL BI Technologies. • Hands-on experience in Azure Cloud Services (PaaS&SaaS), Azure Synapse Analytics, SQL Azure, Data Factory, Azure Analysis Services, Application Insights, Azure Monitoring, Key Vault, Azure Data Lake. • Good experience in tracking and logging end to end software applications build using Azure DevOps. • Used SQL Azure extensively for database needs in various applications. • Experience in development and design of various scalable systems using Hadoop technologies in various environments mostly using Python. Extensive experience in analyzing data using Hadoop Ecosystems including HDFS, MapReduce, Hive & PIG. • In - depth knowledge of Apache Hadoop Architecture (1.x and 2.x) and Apache Spark 1.x Architecture. • Experience building data pipelines using Azure DataBricks and PySpark. • Developed SQL/HSQL queries inside AzureDatabricks. • Extensive knowledge in programming languages such as Scala, Python, Java. • Experience in understanding the security requirements for Hadoop. • Developed Spark applications using PySpark and Scala in Azure DataBricks Environment. • Used Azure Pipelines for CI/CD production devops. • Strong skills in visualization tools Power BI, Confidential Excel - formulas, Pivot Tables, Charts and DAX Commands. • Experienced with Docker and Kubernetes on multiple cloud providers, from helping developers build and containerize their application (CI/CD) to deploying either on public or private cloud.