Fredy Reyes Sanchez Lead Data Engineer | MSc Applied Artificial Intelligence

Senior Engineer

Ascendion

Nov 2024 - Jan 2025 (3 months)

• Research, evaluate, and recommend generative AI frameworks, libraries, and tools suitable for specific use cases, considering factors such as model complexity, scalability, and integration capabilities with Snowflake. • Collaborate with data scientists and machine learning engineers to design and implement proof of concept projects for generative AI applications, leveraging Snowflake's data infrastructure and machine learning capabilities. • Develop and deploy machine learning models for natural language processing (NLP), text generation, image synthesis, or other generative AI tasks using frameworks like TensorFlow, PyTorch, or Hugging Face Transformers. • Conduct performance testing and benchmarking of generative AI models on Snowfl

Senior Big Data Engineer

Walmart

Apr 2022 - Dec 2023 (1 year 9 months)

• Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Databricks. • Solution designing using the Azure Data Analytics platform Microsoft Azure services. • Development and expert knowledge of Azure Data Lake storage and ingestion using Azure Data Factory. • Proficient in developing and implementing highly performant data ingestion pipelines usin Apache Spark and Pyspark for processing large volumes of data from diverse sources. • Experienced in designing and deploying structured Spark streaming pipelines, leveraging distributed big data technologies and modern architectural approaches. • Experienced in optimizing the performance of Spark jobs by fine-tuning paramet

e

Senior Data Engineer

eCivis Inc

Sep 2021 - Apr 2022 (8 months)

• Implement database schema for ETL and BI reporting solutions in multiple databases including Microsoft SQL Server, Amazon Relational Database Services for SQL Server, AWS Redshift/Postgres, and MySQL. • Design, develop, implement, and optimize ETL processes that merge data from disparate sources into Microsoft SQL Server and AWS Redshift for consumption by enterprise SaaS applications. • Mining administrative data for security and behavioral compliance using AI/ML. • Conduct impact analysis and provide policy recommendations. • Analyze large volumes of structured and unstructured data. • Explore and enhance data models (text, numerical). • Implement security best practices and other compliance requirements.

S

Senior Data Engineer / Scientist

Scrumlink,

Feb 2021 - Sep 2021 (8 months)

• Design, build and unit test applications on Spark framework on Python. • Building Data Pipelines using Databricks and scheduling Databricks jobs. • Airflow scheduling Data Pipelines of Databricks. • Building Databricks notebooks in Scala, PySpark and SQL. • Used Broadcast Join in SPARK for making smaller datasets to large datasets without shuffling of data across nodes. • Designed and implemented Sqoop for the incremental job to read data from DB2 and load to Hive tables and connected to Tableau for generating interactive reports using Hive server2. • Develop and deploy the outcome using spark and Scala code in Hadoop cluster running on GCP.

M

Data Engineer / Scientist

MC1 Win the Market,

Mar 2019 - Feb 2021 (2 years)

• ETL Developer, with extensive knowledge of coding languages and warehouse architecture as SQData. • Integration with cloud applications, real-time business intelligence and legacy data migrations with SQData. • Data replication and Integration process with the most important tools in the marketplace as: Hadoop, SQData, Spark, Kafka. • ETL processes to design and fill a bespoke data warehousing environment for WTM's application. • Developed new storage systems and deploy new cloud instances and testing of new data environments. • Guidance to data scientists and full-stack engineers to build sustainable and high-quality software solutions using the latest advances in data science, data engineer, and cloud-engineering

I

Data Engineer

Insituto Nacional Electoral (INE),

Jan 2018 - Feb 2019 (1 year 2 months)

• Data migration Oracle databases to AWS services. • Migration and replication of multiple databases to AWS cloud infrastructure. • Systems Administrator of Red Hat, Solaris, Windows, and Linux servers. • Member of technical architecture team for migration to AWS cloud services to ensure high availability of PREP website. • Scripted and coded design in Python, Java and Power Shell. • Implementation of technical security configuration guidance to enterprise information systems teams. • Vulnerability scanning, penetration testing, and security assessments in support of risk management and audit activities. Establish configuration of security policies under standard ISO 27001.

Fredy Reyes Sanchez

About

Detail

Timeline

Résumé