R
Reddieswara Naidu Kalla
About
Detail
New Paltz, New York, United States
Contact Reddieswara regarding:
work
Full-time jobs
Résumé
Jobs
verified_user
0% verified
Data EngineerDatabricksMay 2024 - Current (2 years 4 months)• Designed and implemented real-time streaming data pipelines using Apache Spark Streaming, Delta Lake, and Lakeflow Declarative Pipelines, enabling continuous ingestion of over 2 TB of data daily with <5 seconds latency for real-time analytics. • Developed and optimized ETL workflows that reduced processing latency by 45% and improved throughput to handle 50K+ events per second, leveraging Spark Streaming and Delta Lake for fault-tolerant, scalable systems. • Automated data ingestion and transformation with Lakeflow Jobs, reducing manual intervention by 90% and ensuring 99.9% uptime for critical streaming applications. • Integrated data from 10+ heterogeneous sources (Kafka, IoT sensors, CRM, web logs), improving real-time visibility a- CProgrammer Analyst – Data EngineerCognizant Technology Solutions LtdJun 2023 - May 2024 (1 year)• Migrated and modeled data from SQL Server, APIs, and flat files into Snowflake Cloud Data Warehouse, reducing compute and storage costs by 20% through zero-copy cloning and automatic clustering. • Built reusable DBT models (staging, intermediate, marts) following star schema and data mesh principles, enabling faster reporting and data governance. • Automated CI/CD deployment of DBT jobs using Git, dbt Cloud, and CLI, ensuring consistency across development, test, and production environments. • Implemented DBT freshness checks and test suites to proactively detect data quality issues, improving data reliability across business reports. • Enabled near real-time analytics by designing Change Data Capture (CDC) pipelines using Snowflake
- CProgrammer Analyst – Azure Data EngineerCognizant Technology Solutions LtdNov 2021 - Jun 2023 (1 year 8 months)• Designed and deployed 30+ ETL pipelines using Azure Synapse, Spark, Python, and SQL to move raw data into curated Lakehouse layers, increasing cross-team data accessibility by 60%. • Developed CDC-based pipelines to efficiently load incremental data, cutting daily processing time by 45% and eliminating redundant data loads. • Diagnosed and resolved 100+ data issues by triaging Spark pipeline failures and collaborating with QA teams, leading to a 35% increase in data quality scores. • Achieved 5x performance improvement in Spark jobs by applying partition pruning, cache optimization, and query tuning in Synapse Spark pools. • Migrated 10+ TB of on-prem SQL Server data into Azure Data Lake Store Gen2, reducing infrastructure costs and
- Data Analyst InternInvestMitraJan 2021 - Oct 2021 (10 months)• Assisted in collecting and integrating customer and claims data from various internal sources, ensuring data was clean, consistent, and ready for analysis. Worked with the data engineering team to ensure the accuracy and reliability of data used for modeling. • Contributed to the engineering of key features such as Customer Lifetime Value (CLV), claim frequency, and time to first claim. Handled missing values, outliers, and duplicates, and standardized data to enhance model performance. • Applied clustering algorithms like K-means and Hierarchical Clustering to segment LIC's customer base into actionable groups based on demographics, behavior, and policy data. Achieved a segmentation accuracy of 90%, enabling more focused marketing str
Education
verified_user
0% verified
- Master of ScienceState University of New York