🚀 We're seeking a data engineer to create and manage identity graphs. An Identity Graph (ID Graph) is a database that stores all identifiers that correlate with individual customers. These identifiers could be anything from usernames to email, phone, cookies, and offline identifiers like loyalty card numbers.
🟣 Requirements:
1. Strong understanding of data modeling and database design:
▪ The candidate should have a solid understanding of database concepts, such as normalization, indexing, partitioning, and replication. In addition, they should be able to design and implement a scalable and performant database system that can handle large volumes of data.
2. Expertise in data integration and ETL:
▪ The candidate should be proficient in extracting, transforming, and loading data from various sources, such as databases, APIs, and file systems. In addition, they should be able to design and implement an ETL pipeline that can handle complex data transformations and ensure data quality.
3. Experience with identity resolution and matching:
▪ The candidate should have experience with identity resolution techniques, such as probabilistic, rule-based, and machine learning-based matching. In addition, they should be able to design and implement a system that can accurately match and merge records from different sources to create a unified view of a customer or user.
4. Knowledge of big data technologies:
▪ The candidate should be familiar with big data technologies like Hadoop, Spark, and Kafka. In addition, they should be able to design and implement a system that can handle large volumes of data in a distributed and fault-tolerant manner.
5. Strong programming skills:
▪ The candidate should have strong programming skills, such as Python or Scala, in one or more languages. In addition, they should be able to write clean, maintainable, and efficient code that can handle complex data processing tasks.
