S
Shruthi P
About
Detail
Data Engineer
United States
Contact Shruthi regarding:
work
Full-time jobs
Résumé
Jobs
verified_user
0% verified
Data EngineerRESMar 2024 - Current (2 years 4 months)• Built and maintained analytics products, optimizing data pipelines using Azure Data Factory and Databricks, improving processing efficiency by 20%. • Led the migration of data from DataStage to Azure cloud platforms, streamlining data integration and enhancing performance. • Configured and optimized Azure Data Factory and Databricks for seamless data ingestion, processing, and transformation. • Designed and implemented data pipelines using Azure Data Factory and Databricks, improving processing efficiency by 20%. • Created high-level design documents to guide the development and implementation of data solutions. • Developed and maintained CI/CD pipelines with Azure DevOps, ensuring continuous integration and deployment of data solut- Data EngineerVerizonJul 2022 - Feb 2024 (1 year 8 months)• Designed and maintained robust data pipelines using Azure Data Factory, enhancing data flow and storage efficiency. • Developed Azure Stream Analytics solutions for real-time data processing and analytics. • Leveraged Azure Synapse Analytics to streamline data warehousing and analytics, improving query performance by 25%. • Implemented data processing frameworks and tools such as Azure Stream Analytics for real-time data analysis. • Automated ETL processes using Azure Synapse Analytics, improving data integration and transformation capabilities. • Leveraged Databricks and Apache Spark for big data processing, enhancing data analytics. • Conducted data quality assurance and validation, ensuring data integrity across all projects. •
- Data EngineerStatefarm InsuranceJan 2020 - Jul 2021 (1 year 7 months)• Developed and automated ETL workflows using AWS Glue, improving data integration and transformation efficiency by 20%. • Utilized AWS services including S3, EC2, Redshift, and Lambda for effective data storage and processing. • Created insightful data visualizations and dashboards using AWS QuickSight, supporting data-driven business decisions. • Created Hive tables on HDFS to store processed data by Spark in Parquet format on Cloudera Hadoop Cluster and used Flume to load log data into HDFS. • Developed Python functions for specific events, automating ETL pipelines from DynamoDB to AWS Redshift. • Configured AWS Data Pipeline to load data from Amazon S3 buckets into Redshift and used JSON schema to define table and column mapping.
- ETL DeveloperUHGJan 2019 - Dec 2019 (1 year)• Assisted in developing data pipelines and ETL processes using Spark, enhancing data flow management. • Supported SQL database optimization tasks, contributing to efficient data retrieval and storage. • Participated in data quality assurance tasks, ensuring high data integrity and reliability. • Provided support in maintaining data systems, learning about system performance optimization and data solution projects. • Performed prescriptive, predictive, and diagnostic analytics using Spark-based ETL solutions, supporting data-driven business decisions. • Actively supported business requirements by developing data solutions aligned with organizational goals. • Collaborated closely with business stakeholders and data analysts to transfo
Education
verified_user
0% verified
- Master's in computer scienceNorthwest Missouri State UniversityAug 2021 - Dec 2022 (1 year 5 months)
- Bachelor's in computer science andJawaharlal Nehru Technological UniversityJun 2016 - May 2020 (4 years)