My experience covers the complete Software Development Life Cycle (SDLC), including Requirement Gathering, Analysis, Design, Development, Testing, Implementation, and Documentation, and I've worked with both waterfall and Agile methodologies.
I have a good understanding of AWS integration, including Elastic MapReduce (EMR), Simple Storage Service (S3), EC2, and Redshift. I can build deployments on AWS, create scripts with Boto 3 and AWS CLI, and provide automation solutions using Shell and Python.
8 years of experience as a Data Engineer with expertise in the Hadoop Ecosystem, including HDFS, Spark, Apache Cassandra, and HBase. I've worked extensively with Apache Spark, utilizing Scala and Python for tasks such as data cleansing, validation, transformation, and summarization.