some thoughts of what we need

Source: Notion | Last edited: 2022-10-08 | ID: 21a510ab-84f...

During hiring process, we have to pick from

zixing: experienced software engineer, who have strong coding skills, backend understanding, multi language, good system understanding
yilun:master student
- pros: latest fresh hands-on ml projects, more like junior ds position for him to help me work on some simple tasks, or to
- cons : limited understanding of model improvement (need guidedance), mediate software engineering skills,
ringo: experienced data pipeline builder
- pros: familiar with hadoop or spark
- cons: we dont need big data (so far), no ml exprience, not as strong backend skills as

this is a modern pipeline, we could ask Ringo about which part has he implement in his pipeline and which part he hasn’t

We hire one by one, we don’t need to hire someone with skills that we don’t need right now. if we do need some sophisticated skills in future, we may hire then.

What we need to do next? the most valuable work and lowest cost

for pipeline, what do we need

we do need source and target database comparison for delivery validation

we need real-time data visualization and analytics on changed data