Design and build a unified cloud data platform that will facilitate the acquisition, storage, processing, and maintenance of large and complex data sources to be used as the single source of truth by the company.
This platform will support business intelligence and advanced analytics, enterprise decision support systems, and machine learning applications.
This role provides the opportunity to shape the strategy and vision for a flexible, reliable, and future-ready data platform that will provide a data-driven competitive advantage to a fast-growing company.
Responsibilities
- Design and build a complete data engineering pipeline
- Master the full life cycle of the data lake and data warehouse implementation process.
- Apply knowledge of software engineering fundamentals, including knowledge of algorithms and data structures
- Bring experience building, deploying, and debugging production systems at scale
- Showcase the ability to trade off the engineering work in terms of value versus cost, maintainability, and generalizability
- Showcase consistency by executing efficiently and proactively against the strategy of complex efforts
- Strive to ensure high-quality outcomes and the best possible user experience
- Exhibit good communication, interpersonal and collaboration skills.
Requirements
- BS or MS degree in Computer Science, Software Engineering, or relevant technical field
- 5+ years of experience with at least one object-oriented/functional language (i.e., Python, Java, Scala)
- 5+ years of SQL experience
Experience with relevant industry tools, architectures, and cloud providers (i.e., AWS, Google Cloud, Microsoft Azure, Databricks)
- Experience implementing cloud analytics platforms (i.e., AWS Redshift, Google BigQuery, Azure Data Warehouse, Snowflake)
- Experience with big data technologies and compute services (i.e., Amazon EMR, Spark, Presto, Hive)
- Knowledge of data pipeline and workflow management tools (i.e., Airflow)
