Paulo Octavio Araujo de Paula
About
Detail
Data Engineer
São Paulo, Brazil
Contact Paulo regarding:
work
Full-time jobs
Starting at
USD6.5k/month
Résumé
Jobs
verified_user
0% verified
Data Engineer public Remote experienceKITRUMAug 2024 - Current (2 years 3 months)• Implemented and maintained data pipelines to calculate key business metrics for analytics platforms, utilizing tools like PySpark, Airflow, Databricks and SQL. Incorporated data quality checks, monitoring, and retention policies to ensure reliable and efficient metric updates.
• Automated metadata configuration by implementing a schema discovery process and integrated it into CI/CD pipelines, using Scala, SQL, Shell scripting, Databricks, and GitHub Actions. Improving adaptability and scalability across environments.
• Optimized data lifecycle management by implementing a delta-based partition deletion mechanism for databases with Scala, SQL, and Delta Lake. Optimizing export workflows and reducing storage overhead.
• Bui- Data EngineerArtefactSep 2022 - Aug 2024 (2 years)• Automated formerly manual business processes dependent on Excel by implementing a Pyspark data pipeline in Databricks and Data Factory, leading to a monumental reduction in process time from 1 work day to 5 minutes.
• Optimized ETLs for Profit & Loss reporting by refactoring the data model, employing PySpark optimization techniques, and tuning Databricks cluster configurations, reducing execution time from 5 hours to 2 hours.
• Solved dozens of on-demand support tickets regarding the client's data infrastructure, including fixing broken ingestion pipelines, refactoring ETL code for efficiency, improving data models and addressing data inconsistencies in tables.
• Migrated and optimized 40+ legacy on-premise data pipelines t - Data Analytics InternVisagioOct 2021 - Sep 2022 (1 year)• Created machine learning models (scikit‑learn) to redefine the target audience segmentation strategy in marketing campaigns.
• Developed financial dashboards in Power BI for C‑level stakeholders to enable quick and clear monitoring of financial results as well as identification of potential business issues.
• Deployed and analyzed A/B tests to reduce loan default rates using pandas, PySpark and SQL.
• Collaborated with the development and documentation of a Demand Forecast solution to avoid over/understocking and improve supplychain efficiency. - Data Engineer InternAMAROJul 2020 - Mar 2021 (9 months)• Built a streaming ingestion pipeline to make event data (cart, wishlist, etc) of over 500 thousand customers available to business users using Amazon MQ, DynamoDB Stream, and AWS CloudFormation to provision infrastructure resources.
• Collected more than 100 GB of data (jpeg, pdf, and tabular data) via web scraping (Selenium) and stored it on the cloud (S3) for use in rebuilding the company’s PLM system.
• Designed and implemented Python and SQL ETLs pipeline utilizing AWS Lambda and Step Functions to ingest data from external APIs into Snowflake (Data Lake).
• Worked together with stakeholders from Product Management, Marketing and Engineering to identify and to collect business and technical requirements. - Data Science InternAMAROJan 2020 - Apr 2020 (4 months)• Developed from scratch an NLP‑based solution for classifying and conducting sentiment analysis on customers reviews, which saved approx. 1000 hours of manual labor monthly.
• Implemented a data pipeline to gather +5000 customer reviews, preprocess the data for cleanliness, and input it into a machine learning model developed using AWS SageMaker.
• Trained and organized an enterprise‑wide data labeling workforce to label thousands of reviews used to train Machine Learning models. - BSoftware Engineer InternBRAVIUM LATAMMay 2019 - Aug 2019 (4 months)• Optimized the company's legacy CRM system, streamlining daily tasks and significantly reducing operator workload
• Designed complex Microsoft SQL Server queries to efficiently perform CRUD operations
• Collaborated with business users, conducting in-depth interviews to identify and understand product requirements and pain points
Education
verified_user
0% verified
- Exchange Program, Data ScienceRWTH Aachen UniversityApr 2021 - Sep 2021 (6 months)
- Bachelor's Degree, Computer Engineering (Co-operative)Universidade de São PauloJan 2017 - Dec 2022 (6 years)