❗ NOUS LATAM is an intermediary in this recruitment process. The selected candidate will work for another organization, and employment-related matters should be directed to the hiring company.
🟡 Job description:
◽ Our client is looking for a Senior Pipeline Engineer/Team Lead to be responsible for ensuring the reliability of Deckard's data pipelines.
◽ We are a critical part of the Data Engineering organization at Deckard, partnering closely with application engineers, data scientists, and other internal team members to ensure our vast data pipelines run cost-effectively, accurately, and on time.
◽ You have significant experience working with and balancing the needs and priorities of a complex set of users. You get excited about having a hands-on development role, building a solid data foundation for the company, and profoundly impacting Deckard's relationship with data. You are independent and care deeply about people, their goals, and professional development, as well as the potential impact Deckard, can have in helping communities.
🟡 What you should have:
◽ You have at least two years in an engineering management/team lead role.
◽ You can weigh several and often conflicting constraints to make rapid decisions in a dynamic and quickly-growing company.
◽ You've built large-scale data pipelines professionally.
◽ You have excellent communication skills, including identifying data issues and driving them to resolution actively.
◽ You possess proven computer science fundamentals: Data structures, algorithms, and distributed systems.
◽ When things break—and they will—you are eager and able to help fix things.
🟡 Requirements:
◽ 5+ years of Python experience.
◽ Data lake, data catalog, or data warehouse experience.
◽ Hands-on experience with AWS services: Glue, Lambda, S3, SNS, Step Functions, CloudWatch, EMR, Batch.
◽ Experience with MapReduce, Hadoop, or Spark.
◽ Experience with GraphQL.
◽ Advanced English (C1+).
🟡 Desirable experience:
◽ Web scraping experience.
◽ React.js/Typescript.
🟡 What you will be doing:
◽ Own core datasets that power internal and customer-facing products.
◽ Work closely with analysts, product teams, and data scientists to understand current and future data needs.
◽ Define and run SLAs for all data sets.
◽ Create and maintain optimal data pipeline architecture.
◽ Assemble large, complex data sets that meet functional/ non-functional business requirements.
◽ Identity, design, and implement internal process improvements: automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
◽ Build the infrastructure required for optimal ETL of data from various sources using SQL, Spark, and other AWS 'big data' technologies.
◽ Build and cultivate a thriving team of data engineers and managers, focusing on impact, high trust, reliability, strong cohesion, and retention through the entire employee lifecycle.
◽ Architecting a GraphQL data mesh system.
