Vivek Katariya

Principal Data Application Engineer

Streamhub

Aug 2025 - Current (1 year 2 months)

• Lead end-to-end data and product engineering for Japan operations, owning the full development and release lifecycle while ensuring alignment with business goals and high standards of quality, performance, and reliability.
• Design, build, and optimize scalable data platforms and 20+ pipelines (Snowflake, DBT, S3, python and scala), driving system architecture, feature delivery, testing, monitoring, and continuous improvement across production environments which are handling TBs of data per day.
• Implementing AI agents workflow using Claude Code for automated repository analysis and development, reducing manual code review time by 35%
• Enhanced Snowflake query performance by 20% through advanced clustering design and tabl

Data Engineer III

Zinnia

May 2025 - Aug 2025 (4 months)

• Setting up a next-gen data engineering platform to move data from Oracle legacy system to cloud based solution using Apache NIFI, DBT, Snowflake, Bigquery, Airflow etc.
• This platform has reduced the client file generation process from 15 hrs to 1 hrs.
• Completed 30 file generations which processes 13TB data in POC and working on prod deployment for the same.
• Worked as architectural designer and led the end to end process for this migration.
• Implemented SCD 1,2,3 for OLAP and OLTP system depending on file processing logic
• Optimised legacy queries and code operations to reduce the scan size of DB and used effective data loading techniques to optimise the load performance.
• Leveraged Artificial Intelli

Senior Software Engineer

IDFY

Apr 2023 - May 2025 (2 years 2 months)

Data Fabric (Data as service and Insights as Service):
• Creating, managing and maintaining IDfy's Data Fabric platform, processing 900TB of monthly reads and ingesting
40B+ rows each month.
• Leading a data engineering team, overseeing communication, planning, and management of IDfy's data
engineering and analytics sprints across clients, stakeholders, and internal teams.
• Designed/ Developed 60+ ETL/ELT data pipelines using apache beam.
• Developed analytical dashboards using metabase and insights services that act as a single source of truth for
financial data, agent activities, product usage, product health and behavioral insights, serving both internal and external IDfy
clients.
• Developed

O

Data Engineer

Onlinesales.ai

Oct 2021 - Apr 2023 (1 year 7 months)

Monetize & Hyperlocal Products:
• Designed and optimized ETL pipelines to efficiently extract data from cloud platforms and vendor sites, reducing
unnecessary data processing and cutting data extraction costs by 40%.
• Developed automated reports for the business team, eliminating manual aggregation and providing actionable insights
to boost product sales and enhance client retention.
• Successfully migrated the reporting infrastructure from Amazon Redshift to Google BigQuery, improving performance
and scalability.
• Led and managed the L1 support team to handle technical customer queries, root cause analysis (RCA), and fixes for
existing system bugs; served as Scrum Master for sprints in the lead man

Software Engineer

Infosys

Sep 2019 - Jul 2021 (1 year 11 months)

Macys & Bloomingdales (ECS and DAAS):
• Spearheaded developing, managing, analyzing and improving databases using Oracle SQL, MemSQL, Mysql query languages and Tibco Designer tool to increase the sales and make data flow transactions more efficient.
• Implemented automated reports and kibana dashboards for analysis and tracking issues in checkouts details to save 10 Hrs of manual effort/week and made the project more cost efficient.
• Leveraged Python for basic scripting to enhance operational efficiency and automate repetitive tasks within the data engineering workflow.
• Utilized PostgreSQL to meticulously track transactions performed on the e-commerce site, ensuring data integrity and providing a robust audit trail fo

About

Detail

Timeline

Résumé

Vivek Katariya

About

Detail

Behavioral traits