Krishna Chaitanya Kota

Data Engineer

State Farm

Oct 2022 - Current (2 years 9 months)

• Responsible for designing, developing, validating, and maintaining data driven applications. • Extensively worked in designing and developing API’s using AWS Lambda functions in Python. • Implemented several Terraform modules for deploying, managing, and orchestrating resources in AWS. • Developed Python and SQL used in the transformation process in Matillion. • Designed, performed CERD operations on DynamoDB Tables daily. • Designed weekly backups from DynamoDB to S3. • Used KMS keys for encrypting and decrypting application data. • Transformed S3 data according to business requirements and copied it to Redshift for analysis purposes. • Developed and modified LookML code in Looker business intelligence solution using Matillion. • Create

Data Engineer

Nike

Feb 2022 - Oct 2022 (9 months)

• Responsible for designing, implementing, validating, and maintaining data pipelines using Apache Airflow. • Orchestrated Airflow DAGs to schedule the Ingestions, ETL jobs and business reports using YAML files on a Managed Airflow platform. • Worked extensively on Snowflake cloud data warehouse implementation on AWS. • Full load and delta load of data objects from (AWS S3, SQL Server, Oracle, Teradata) to snowflake cloud using Airflow pipelines. • Used COPY, LIST, PUT and GET commands for validating the internal stage files. • Implemented complex Snow SQL scripts in snowflake cloud data warehouse for business analysis and reporting. • Developed snowflake procedures for executing branching and looping • Implemented Alteryx workflows as Snow

Data Engineer

Genesis

Feb 2021 - Feb 2022 (1 year 1 month)

• Extract, Transform and Load data from on-premises MS SQL Server to Azure Data Lake Gen 2 and Azure Blob Storage using SSIS, Azure Data factory, SQL. • Ingested data from Azure Storage into Azure Databricks using Azure Data Factory, PySpark and Spark-SQL. • Built automated batch processing pipelines that migrates the data from different file formats like JSON, Flat Files to Azure Databricks tables and validated the data using Linked Services/Datasets in ADF. • Developed a script in Databricks notebook using PySpark and Pandas to automatically generate create, insert, delete, and update Spark-SQL scripts. • Developed SSIS packages to parse JSON files, sends user specific notifications every day. • Extensively used transformations like Split

P

AWS Data Engineer

PELITAS

Oct 2019 - Jan 2021 (1 year 4 months)

• Responsible for Designing, implementing, and testing Batch/Streaming data pipelines using AWS services. • Extensively worked on PySpark to read data from S3 Data Lake to preprocess and store it back in S3 to create tables using Athena. • Deployed AWS Lambda functions and other dependencies into AWS using EMR Spin for Data Lake jobs. • Created AWS Lambda functions and assigned IAM Roles to schedule Python scripts using Cloud Watch Triggers to support business needs (SNS, SQS). • Conducted ETL data integration and transformations using AWS Glue Spark scripts. • Created partitioned tables in Athena, also designed data warehouse using Athena external tables, used Athena Queries for data analysis. • Worked with different source and destination

Big Data Engineer

Pearson

Feb 2019 - Oct 2019 (9 months)

• Designed robust and scalable data driven Spark streaming applications to automate the ingestion process using PySpark/Spark-SQL to process massive volumes of data. • Converted SQL queries into Spark transformations using Spark data frames in Pyspark. • Parsed log files to a meaningful structured data using regex, cleansed, and stored into HDFS by partitioning data with different metrics for optimized analysis. • Developed Streaming data pipelines using PySpark and stored intermediate data in Kafka. • Designed and implemented a POC on external No-SQL data base (HBase), enhanced it to production quality and integrated with Spark streaming job. • Worked on compressing the data blocks in HFDS using Snappy, BZip2, LZO. • Used Apache HTTP to qu

C

Data Analyst

CricFantasy

Jun 2016 - Dec 2018 (2 years 7 months)

• Responsible for extraction, transformation, loading (ETL) operational data using SSIS and SQL queries. • Collaborated with cross functional teams to identify the data requirements and delivered the business insights to the teams on time. • Identified, measured, and recommended improvement strategies for Key Performance Indicators across all the business areas with 7 % increase in revenue. • Extracted, compiled, and tracked data and analyzed data to generate reports using SSRS. • Performed daily data queries and prepared reports on daily, weekly, monthly, and quarterly basis. • Accomplished data collecting, cleansing, data modelling, data profiling, data queries to analyze the data from different sources. • Generated Business Intelligence

Krishna Chaitanya Kota

About

Detail

Contact Krishna regarding:

Timeline

Résumé

Languages:

Krishna's behavioral traits
Krishna's likely patterns of behavior, thought, values, and emotion. Based on the HEXACO model, OCP instrument, and Torre’s own research.