A

Archana Ch

About

Detail

Sr Data Engineer
Charlotte, North Carolina, United States
Contact Archana regarding: 
work
Full-time jobs

Timeline

work
Job
school
Education

Résumé

Jobs verified_user 0% verified
  • Fidelity Investments
    Sr Data Engineer
    Fidelity Investments
    Mar 2022 - Current (4 years 7 months)
    • Designed and built Spark applications using Scala for efficient daily batch processing, streamlining critical operations and data transformations.
    • Collaborated closely with cross-functional teams, proactively engaging in data modeling discussions and swiftly resolving data processing challenges.
    • Utilized Spark's powerful capabilities, including Scala API and Spark SQL, to handle intricate data transformations and load structured and semi-structured data into S3 buckets.
    • Employed Spark Data Frame operations to perform data validations and execute comprehensive analytics, enabling data-driven decision-making.
    • Leveraged my expertise across various Spark modules, such as Spark Core, Spark SQL, Spark Streaming, and
  • USDA
    Sr Data Engineer
    USDA
    Jan 2021 - Mar 2022 (1 year 3 months)
    • Ingested Adobe Analytics clickstream data from external FTP locations into GCS data lake.
    • Automated Dataproc cluster launches, optimized resources, and submitted Spark jobs.
    • Utilized Cloud SQL as external Hive metastore for Dataproc clusters.
    • Processed structured and semi-structured files using Spark Data Frame API, loading into GCS.
    • Orchestrated data movement between Big Query stages for streamlined analysis.
    • Collaborated with data science teams, operationalizing machine learning models.
    • Developed custom Python-based UDFs for enhanced Spark-SQL queries.
  • USDA
    Sr Data Engineer
    USDA
    Jan 2021 - Mar 2022 (1 year 3 months)
    • Ingested Adobe Analytics clickstream data from external FTP locations into GCS data lake.
    • Automated Dataproc cluster launches, optimized resources, and submitted Spark jobs.
    • Utilized Cloud SQL as external Hive metastore for Dataproc clusters.
    • Processed structured and semi-structured files using Spark Data Frame API, loading into GCS.
    • Orchestrated data movement between Big Query stages for streamlined analysis.
    • Collaborated with data science teams, operationalizing machine learning models.
    • Developed custom Python-based UDFs for enhanced Spark-SQL queries.
  • Abercrombie Fitch Co
    Big Data Developer
    Abercrombie Fitch Co
    Nov 2019 - Dec 2020 (1 year 2 months)
    • Analyzed the data by performing Hive queries (Hive QL) to study customer behavior.
    • Developed Hive scripts in Hive-QL to de-normalize and aggregate the data.
    • Worked with Apache NiFi to automate the data flow between the systems and managed flow of information between systems
    • Exported the analyzed data to the relational databases using Sqoop, to further visualize and generate reports for the BI team.
    • Worked on cloud deployment using Maven, Docker, and Jenkins.
    • Developed and maintained the data pipelines using Python and Spark.
    • Involved in converting Hive/SQL queries into Spark transformations using python, Spark RDDS.
    • Worked on migrating MapReduce programs into Spark transformations using Spa
  • Wellmark Blue Cross and Blue Shield
    Data Engineer
    Wellmark Blue Cross and Blue Shield
    Jan 2018 - Oct 2019 (1 year 10 months)
    • Experience in Importing and exporting data into HDFS and Hive using Sqoop.
    • Moving Bulk amount of data into HBase using Map Reduce Integration.
    • Worked with Talend for integrating data from different data systems to Hadoop.
    • Experienced in handling different types of joins in Hive like Map joins, bucket map joins, sorted bucket map joins.
    • Optimized the Hive tables using optimization techniques like partitions and bucketing to provide better performance with Hive QL queries.
    • Created tables, partitions, buckets and perform analytics using hive, Ad-hoc queries.
    • Integrated spring schedulers with Oozie client as beans to handle cron jobs.
    • Experience with CDH distribution and Cloudera Manager to m
  • Glencore
    Java Developer
    Glencore
    Jan 2016 - Dec 2016 (1 year)
    • Participated in major phases of software development cycle with requirement gathering, Unit testing, development, and analysis and design phases using Agile/SCRUM methodologies.
    • Interaction with onsite team for change requests and understanding the requirement changes for implementation of the same.
    • Involved in requirements analysis, design, development, and testing of risk workflow systems.
    • Implemented the application with Spring Framework for implementing Dependency Injection and provide abstraction between presentation layer and persistence layer.
    • Developed multiple batch jobs using Spring Batch to import files of different formats like XML.
    • Developed SQL and JPQL queries, triggers, views to intera
Education verified_user 0% verified
  • The University of Kansas
    Masters, Computer and Information Sciences
    The University of Kansas
    Jan 2017 - Dec 2017 (1 year)