Python. Invent and Simplify. Have Backbone; Disagree and Commit. Twenty-four runnable exercises, four behavioral rehearsals and a test for every contract.
Round 5 is a preparation label. The recruiter assigned Python, Invent and Simplify and Have Backbone; Disagree and Commit. Calendar order and Python version are not confirmed.
Name the contract. Test the boundary. Own the decision.
140 min reading 4-hour core route Full ~11h 50m
TOTAL STUDY TIME / FULL GUIDE~11h 50m
140 min reading + 9h 30m practice, including one 60-min mock.
Short route: 4 hoursSelected essentials + one mock.All three mocks: ~13h 50mFull guide + two additional mocks. Estimates include practice time; breaks are extra.
LOCAL & OFFLINEPRIMARY-SOURCE NOTESTHREE MOCK LOOPSREVISION 1.2
FOCUS TIME00:00
Estimated remaining: ~11h 50mStart a focus session to track pace.Open mock runner
01 / Start 2 min read · 5 min practice
Your route through Round 5
The recruiter’s scanned Python sheet names sorted-array merging, nested JSON to CSV, duplicates within k, dataframe manipulation and two-sum/three-sum. It emphasizes a clear contract, robust data handling, appropriate structures and complexity. This guide covers those requirements and adds practical data-engineering follow-ups.
All 24 Python labs, dataframe exercises, behavioral work and one mock. Lab budgets already sit in their chapters.
All three mocks
~13h 50m
Full guide plus two additional 60-minute mocks. Breaks and extra repair attempts are additional.
Pass 1: predict, implement, run and explain. Pass 2: reconstruct cold and change one assumption. Read-time estimates count code and expanded explanations at 180 words/minute; practice time is separate. Your saved notes, drafts and scores are local to this file’s browser origin. Export progress before moving the file or changing browsers.
Write expected output including a boundary or duplicate.
Makes the contract concrete before coding.
State a baseline
Name the data structure, maintained invariant and complexity.
A correct simple baseline is a platform for improvement.
Code aloud selectively
Narrate decisions and corrections, not every keystroke.
The interviewer can follow your model without being flooded.
Test deliberately
Empty/singleton, duplicates, exact boundary, malformed input, tie, one-shot iteration and nonmutation as relevant.
Tests should challenge the algorithm, not only repeat the sample.
Close
State remaining assumptions and costs including output and retained state.
A working tiny fixture does not establish production scalability.
SAY IT / Opening
Give a concise introduction and describe how you approach this coding round.
Reveal worked answer
I am a data engineer with 16-plus years of experience across data platforms and analytics. My Meta work included product data foundations, shared metrics and large-scale first-party pipelines. I approach a coding problem by clarifying the output and edge cases, choosing a readable algorithm, and testing the invariants before discussing scale. I will keep my own implementation and the team’s broader outcome distinct.
Pressure test: Name one real Python transformation you owned and its actual input/output contract.
Avoid: A long career chronology or presenting a constructed rehearsal as an actual delivery.
SAY IT / Contract
The prompt says duplicates within a window of k. What do you clarify?
Reveal worked answer
I would clarify whether k is the number of preceding positions or the total window length, whether the distance boundary is inclusive, and whether input is a list or a stream. I would define behavior for k=0 and invalid k. For the distance-at-most-k contract, I can maintain the previous k values and look for the current value before advancing the window.
Pressure test: At indices 0 and 2 with k=2, should the duplicate count?
Avoid: Coding before resolving a boundary that changes the result.
03 / Foundations 3 min read · 18 min practice
Python mechanics that cause interview bugs
Mechanism
Use it correctly
Counterexample to defend
Equality / identity
Use == for values, is None for the singleton.
Two equal lists need not be the same object.
Missing / null / zero
Use a sentinel or key membership when missing differs from None; accept 0 when valid.
if not row.get("count") rejects a real zero.
bool / int
For strict numeric counts, type(x) is int excludes bool.
isinstance(True,int) is True.
Hashable keys
Use immutable value keys such as a tuple of immutable fields.
A tuple containing a list is still unhashable.
Sort / sorted
list.sort mutates and returns None; sorted creates a new list.
result = rows.sort() assigns None.
Stable sorting
Equal keys retain prior order, but business ties may require an explicit additional key.
Input order from unordered sources is not a reliable customer tie rule.
Copy semantics
list(x) and dict(x) are shallow; copy.deepcopy isolates nested values when required.
Two shallow copies can still share a nested list.
Iterators
iter(list) creates a cursor; iter(generator) returns the same iterator.
A second loop over a consumed generator is empty.
Exceptions
Catch the expected error at the operation that can raise it; preserve unexpected failures.
except Exception: continue can hide a misspelled variable.
Dates
Choose aware UTC instants or a documented business zone; define half-open windows.
Naive local timestamps do not encode DST ambiguity.
Default arguments
Defaults are evaluated once; use None and create state inside the call.
def f(buffer=[]): shares a list across calls.
from collections import defaultdict, Counter, deque
groups = defaultdict(list)
for account, value in [('a', 2), ('a', 3)]:
groups[account].append(value)
assert dict(groups) == {'a': [2, 3]}
assert Counter('aba') == {'a': 2, 'b': 1}
queue = deque([1, 2]); assert queue.popleft() == 1
rows = [[1], [2]]
shallow = list(rows); shallow[0].append(9)
assert rows[0] == [1, 9] # Inner list remains shared.
SAY IT / Mechanics
Why can a perfectly valid-looking count corrupt aggregation?
Reveal worked answer
Python booleans are a subclass of integers, so isinstance(value,int) accepts True as one. Missing, null, zero and a numeric string are different states and need an explicit schema. I would validate at the input boundary, preserve valid zero counts, and choose either strict rejection or a documented conversion rule before aggregation.
Pressure test: Would int("1.5") or int(1.5) preserve the original value?
Avoid: Silently coercing every value with int() or using truthiness as schema validation.
Exact counts require every distinct key, even with a streaming reader.
Global order
sorted: O(n log n) worst-case time
Python sorting needs temporary memory; output ordering is real work.
k best items
heap: O(log k) update
Define ties and remember counting state may dominate the heap.
Merge ordered streams
two cursors / heap
Exploit input order instead of sorting all records again.
Nested structures / dependencies
stack / adjacency map
Depth, edges and cycles matter, not just row count.
SAY IT / Scale
Your code reads one row at a time. Is it constant-memory?
Reveal worked answer
Only if all retained state is bounded independently of input size. A deduplication set grows with unique event IDs and a grouping dictionary grows with group cardinality. A generator avoids storing all output at once, but its internal state and the consumer still matter. If the state is too large, I would partition by key, spill or external-sort, or use a distributed system while preserving identity and tie rules.
Pressure test: How would you deduplicate a file larger than RAM exactly?
Avoid: Saying streaming equals O(1) memory or offering an approximate sketch for an exact requirement.
The merge invariant: the emitted prefix is sorted and complete.
PYTHON LAB / 8 MIN PRACTICE
01 / Merge sorted arrays
Given two ascending integer lists, return one ascending list, retaining duplicates. Do not mutate either input. Inputs are already sorted; do not spend time validating that precondition.
Hint
Maintain two cursors. Emit the smaller head; drain the remaining suffix.
Worked solution and test cases
def merge_sorted(a, b):
i = j = 0
out = []
while i < len(a) and j < len(b):
if a[i] <= b[j]:
out.append(a[i]); i += 1
else:
out.append(b[j]); j += 1
out.extend(a[i:]); out.extend(b[j:])
return out
Reasoning and complexity: Invariant: out is sorted and contains exactly the consumed prefixes. O(n+m) time, O(n+m) output; this list implementation also slices the suffix. Use iterator merging to avoid suffix copies.
a=[1,3]; b=[2,4]
assert merge_sorted(a,b)==[1,2,3,4]
assert a==[1,3] and b==[2,4]
Check: Randomized sorting oracle
import random
r=random.Random(51)
for _ in range(100):
a=sorted(r.choices(range(-5,6),k=r.randrange(15))); b=sorted(r.choices(range(-5,6),k=r.randrange(15)))
assert merge_sorted(a,b)==sorted(a+b)
PYTHON LAB / 10 MIN PRACTICE
02 / Merge one-shot iterators
Yield ascending integers from two already sorted, finite, one-shot iterables. Preserve duplicates. Do not materialize the inputs or entire output. Use O(1) auxiliary state, excluding the input iterators and consumer output.
Hint
Use a unique sentinel, not a valid data value, to represent exhaustion.
Worked solution and test cases
def merge_streams(left, right):
a, b = iter(left), iter(right)
end = object()
x, y = next(a, end), next(b, end)
while x is not end and y is not end:
if x <= y:
yield x
x = next(a, end)
else:
yield y
y = next(b, end)
while x is not end:
yield x
x = next(a, end)
while y is not end:
yield y
y = next(b, end)
Reasoning and complexity: Only the next value of each stream is retained. O(n+m) time, O(1) auxiliary memory. The caller can still consume the output into a large list.
seen=[]
def source():
for i in range(100):
seen.append(i); yield i
g=merge_streams(source(),iter([]))
assert iter(g) is g
assert next(g)==0 and len(seen)<=2
Check: Both empty
assert list(merge_streams(iter([]),iter([])))==[]
PYTHON LAB / 12 MIN PRACTICE
03 / Merge sorted event partitions
Each input stream contains dictionaries {ts: integer, id: string}, sorted by ts. Yield all records ordered by (ts, source index, position within source). Do not compare dictionaries. Keep at most one pending record per stream; inputs are trusted and finite.
Hint
Heap tuples need a total tie rule before the dictionary payload.
Worked solution and test cases
def merge_partitions(streams):
import heapq
streams = [iter(s) for s in streams]
heap = []
for source, stream in enumerate(streams):
item = next(stream, None)
if item is not None:
heap.append((item['ts'], source, 0, item))
heapq.heapify(heap)
while heap:
_, source, pos, item = heapq.heappop(heap)
yield item
following = next(streams[source], None)
if following is not None:
heapq.heappush(heap, (following['ts'], source, pos+1, following))
Reasoning and complexity: One head per source. O(k+N log(max(k,2))) time and O(k) state. Source index makes equal timestamps deterministic; position documents within-source ordering.
Check: Equal timestamps
s=[[{"ts":1,"id":"a"},{"ts":1,"id":"b"}],[{"ts":1,"id":"c"}],[]]
assert [r["id"] for r in merge_partitions(s)]==["a","b","c"]
Check: Interleaving
s=[iter([{"ts":2,"id":"x"}]),iter([{"ts":1,"id":"y"},{"ts":3,"id":"z"}])]
assert [r["id"] for r in merge_partitions(s)]==["y","x","z"]
Check: No streams
assert list(merge_partitions([]))==[]
SAY IT / Merging
Merge two sorted arrays. Then change the inputs to one-shot streams.
Reveal worked answer
For arrays I use two indices, append the smaller head, and drain the remaining suffix. The invariant is that output contains exactly the consumed prefixes in sorted order. Time is O(n+m), with output-sized memory. For one-shot streams I use one pending value from each iterator and a unique exhaustion sentinel, then yield values. That keeps auxiliary state constant while preserving duplicates.
Pressure test: What breaks if you use pop(0) or concatenate and sort?
Avoid: Losing duplicates, mutating inputs without permission, or calling an iterator merge constant total memory when the consumer builds a list.
Look up the complement before inserting the current value. Retain the earliest index.
Worked solution and test cases
def two_sum(nums, target):
first = {}
for j, x in enumerate(nums):
complement = target - x
if complement in first:
return (first[complement], j)
first.setdefault(x, j)
return None
Reasoning and complexity: Each previous value is represented by its earliest index. Expected O(n) time and O(u) space for u distinct values; hash-table performance is not a universal worst-case guarantee.
Check: Duplicates need two positions
assert two_sum([3,3],6)==(0,1)
assert two_sum([3],6) is None
Check: Tie rule
assert two_sum([1,1,4,4],5)==(0,2)
Check: Negatives and no answer
assert two_sum([-4,2,7],3)==(0,2)
assert two_sum([],0) is None
Check: Brute-force oracle
import random
r=random.Random(52)
for _ in range(80):
a=r.choices(range(-4,5),k=r.randrange(12)); t=r.randrange(-8,9)
pairs=[(i,j) for j in range(len(a)) for i in range(j) if a[i]+a[j]==t]
assert two_sum(a,t)==(pairs[0] if pairs else None)
PYTHON LAB / 12 MIN PRACTICE
05 / Unique three-sum triplets
Return lexicographically sorted unique value triplets [a,b,c], a<=b<=c, summing to zero. A triplet requires three positions. Input is an integer list and must remain unchanged.
Hint
Sort a copy. Fix one value and move two pointers; skip duplicate fixed and matched values.
Worked solution and test cases
def three_sum(nums):
values = sorted(nums)
out = []
for i in range(len(values)-2):
if i and values[i] == values[i-1]:
continue
if values[i] > 0:
break
left, right = i+1, len(values)-1
while left < right:
total = values[i] + values[left] + values[right]
if total < 0:
left += 1
elif total > 0:
right -= 1
else:
out.append([values[i], values[left], values[right]])
left += 1; right -= 1
while left < right and values[left] == values[left-1]:
left += 1
while left < right and values[right] == values[right+1]:
right -= 1
return out
Reasoning and complexity: For fixed i, increasing left increases the sum and decreasing right decreases it. O(n²) time, O(n) sorted copy plus output. The enumeration order is already lexicographic.
import random,itertools
r=random.Random(53)
for _ in range(60):
a=r.choices(range(-5,6),k=r.randrange(10))
expected=sorted({tuple(sorted(x)) for x in itertools.combinations(a,3) if sum(x)==0})
assert three_sum(a)==[list(t) for t in expected]
SAY IT / Algorithms
Why does two-sum use a hash map while three-sum often uses sorting and two pointers?
Reveal worked answer
Two-sum can look up a complement among earlier values, preserving distinct indices in expected linear time. For unique three-sum value triplets, sorting lets me fix one value and move two pointers monotonically, skipping duplicates. That gives quadratic time and clear uniqueness control. The output contract matters: original indices require a different representation than sorted value triplets.
Pressure test: For [3,3], target 6, why must lookup happen before insert?
Avoid: Applying a memorized solution without distinguishing indices, values and uniqueness.
07 / Coding 5 min read · 28 min practice
Index windows and event-time windows
An index window counts positions. A time window counts timestamps and may contain any number of records. In these exercises, late timestamps fail explicitly. Production event-time processing needs a separate watermark, lateness and correction contract.
PYTHON LAB / 8 MIN PRACTICE
06 / Duplicates within k positions
Return whether any equal integers appear at indices iHint
At the start of each iteration retain exactly the preceding at most k values.
Worked solution and test cases
def nearby_duplicate(values, k):
if type(k) is not int or k < 0:
raise ValueError('k must be a nonnegative integer')
from collections import deque
window, present = deque(), set()
for value in values:
if value in present:
return True
window.append(value); present.add(value)
if len(window) > k:
present.remove(window.popleft())
return False
Reasoning and complexity: If the algorithm continues, the retained window has no duplicates, so a set and deque are sufficient. Expected O(n) time, O(min(n,k)) space.
PYTHON LAB / 10 MIN PRACTICE
07 / Distinct counts in fixed windows
Return the number of distinct integers in every complete consecutive window of length k. Input may be one-shot. Positive integer k is required, excluding bool; otherwise ValueError. If fewer than k rows exist, return [].
Hint
Counts are needed because duplicate values can leave while another copy remains.
Worked solution and test cases
def distinct_windows(values, k):
from collections import Counter, deque
if type(k) is not int or k <= 0:
raise ValueError('positive integer k required')
q, counts, out = deque(), Counter(), []
for value in values:
q.append(value); counts[value] += 1
if len(q) > k:
old = q.popleft(); counts[old] -= 1
if counts[old] == 0:
del counts[old]
if len(q) == k:
out.append(len(counts))
return out
Reasoning and complexity: Expected O(n) time, O(k) auxiliary state plus O(n-k+1) output. Removing zero-count keys keeps len(counts) meaningful.
For nondecreasing integer timestamps, return the count seen in (t-width,t] at each arrival, including that arrival. Equal-time records count only once they have arrived. width must be a positive int excluding bool. Reject any decreasing timestamp with ValueError.
Hint
Evict <= lower boundary, then append. State the ordered-input precondition explicitly.
Worked solution and test cases
def rolling_counts(timestamps, width):
from collections import deque
if type(width) is not int or width <= 0:
raise ValueError('positive integer width required')
q, out, previous = deque(), [], None
for t in timestamps:
if type(t) is not int or (previous is not None and t < previous):
raise ValueError('integer timestamps must be nondecreasing')
previous = t
while q and q[0] <= t-width:
q.popleft()
q.append(t); out.append(len(q))
return out
Reasoning and complexity: Each record enters and leaves once: O(n) time, O(w) active-event state plus output. Here w means active record count, not the numeric width.
try: rolling_counts([4,3],5)
except ValueError: pass
else: raise AssertionError("accepted late event")
SAY IT / Windows
Implement duplicates within distance k, then explain a time-window variation.
Reveal worked answer
For position distance at most k I retain the preceding k values. Before inserting the current value, membership indicates a qualifying duplicate. If I continue, that retained window contains no duplicates, so a set plus deque suffices. For a time window I evict by timestamp rather than queue length and need nondecreasing input or an explicit out-of-order policy. A fixed five-minute window does not imply fixed memory because event volume can vary.
Pressure test: Which exact operator excludes the lower endpoint of (t-width,t]?
Avoid: Using one window definition for both positions and event time.
08 / Coding 3 min read · 18 min practice
Top-k and weighted product metrics
PYTHON LAB / 8 MIN PRACTICE
09 / Deterministic top-k frequency
For string keys return up to k (key,count) tuples ordered by count descending, then key ascending. k must be a nonnegative int excluding bool. Empty input or k=0 yields []. Start with a clear sort-based solution.
Hint
Count, then sort by a composite key. Optimize only after identifying u and k.
Worked solution and test cases
def top_k(values, k):
from collections import Counter
if type(k) is not int or k < 0:
raise ValueError('nonnegative integer k required')
if k == 0:
return []
counts = Counter(values)
return sorted(counts.items(), key=lambda item: (-item[1], item[0]))[:k]
Reasoning and complexity: Expected O(n+u log u) time and O(u) state. For small k, heapq.nsmallest(k, counts.items(), key=...) can reduce selection to O(u log k), but exact counting still costs O(u) state.
Rows are (provider:string, successes:int, attempts:int). Counts exclude bool and satisfy 0<=successes<=attempts. Aggregate by provider and return (provider,successes,attempts,rate) tuples, sorted by provider. rate is None when total attempts is zero. Invalid rows raise ValueError.
Hint
Add numerators and denominators before dividing. Keep zero-volume providers visible.
Worked solution and test cases
def success_rates(rows):
totals = {}
for provider, successes, attempts in rows:
if not isinstance(provider,str) or not provider or type(successes) is not int or type(attempts) is not int or not 0 <= successes <= attempts:
raise ValueError('invalid provider or counts')
s, n = totals.get(provider, (0,0))
totals[provider] = (s+successes,n+attempts)
return [(p,s,n,s/n if n else None) for p,(s,n) in sorted(totals.items())]
Reasoning and complexity: Expected O(n+u log u) time and O(u) state. 1/1 and 0/9 aggregate to 1/10, not 50%.
for row in [("a",True,1),("a",2,1),("a",0,-1)]:
try: success_rates([row])
except ValueError: pass
else: raise AssertionError("bad counts accepted")
SAY IT / Metrics
Provider A has 1/1 success in one file and 0/9 in another. What rate should be ranked?
Reveal worked answer
The pooled event-level success rate is (1+0)/(1+9), or 10 percent. Averaging file rates gives 50 percent and weights small and large files equally. I would confirm the intended unit of analysis, retain numerator and denominator, and define zero-volume and tie behavior. If the customer intentionally asks for an average of provider-day rates, that is a different metric with a different weighting contract.
Pressure test: How do retries across files change the totals?
Avoid: Optimizing a ranking algorithm before defining the metric and event identity.
09 / Coding 4 min read · 26 min practice
Version identity and replay-safe updates
Correct deduplication starts with identity, not equal-looking rows.
PYTHON LAB / 12 MIN PRACTICE
11 / Versioned events and tombstones
Trusted rows have id:string, version:int, deleted:bool and value. Collapse exact retries by (id,version). Any unequal payload for the same id/version raises ValueError, even if a higher version exists. Choose maximum version per id, then remove tombstones. Return fresh dictionaries sorted by id. Do not mutate inputs.
Hint
Detect conflicts across all versions, choose winners, then apply deletion.
Worked solution and test cases
def latest_live(rows):
import copy
versions, latest = {}, {}
for row in rows:
key = (row['id'], row['version'])
if key in versions and versions[key] != row:
raise ValueError('conflicting version')
versions[key] = row
old = latest.get(row['id'])
if old is None or row['version'] > old['version']:
latest[row['id']] = row
return [copy.deepcopy(row) for _,row in sorted(latest.items()) if not row['deleted']]
Reasoning and complexity: Expected O(n+u log u) time. O(v) distinct-version state is required by this conflict contract, plus winner/output state. Removing deleted rows first resurrects old data.
rows=[dict(id="a",version=v,deleted=False,value=x) for v,x in [(1,3),(2,8),(1,4)]]
try: latest_live(rows)
except ValueError: pass
else: raise AssertionError("conflicting old version accepted")
PYTHON LAB / 14 MIN PRACTICE
24 / Replay-safe additive upserts
state is {seen:{event_id:(account,delta)}, totals:{account:int}}. Trusted incoming rows are (event_id:string,account:string,delta:int). Apply each new event once. An exact retry is a no-op; conflicting payload for a seen event raises ValueError. Return a new state, leaving the input state unchanged even if a late row fails. This is an in-memory logical model, not a claim of durable atomicity.
Hint
Build a candidate state, validate identity before applying, publish only after the batch succeeds.
Worked solution and test cases
def apply_events(state, rows):
next_state = {'seen':dict(state['seen']), 'totals':dict(state['totals'])}
seen, totals = next_state['seen'], next_state['totals']
for event,account,delta in rows:
value = (account,delta)
if event in seen:
if seen[event] != value:
raise ValueError('conflicting replay')
continue
seen[event] = value
totals[account] = totals.get(account,0) + delta
return next_state
Reasoning and complexity: Expected O(existing state + batch) time and memory because this implementation copies state for all-or-nothing caller behavior. Durable systems need transactional state/sink updates or idempotent writes and replay recovery.
Where should a checkpoint be written relative to the sink?
Reveal worked answer
A checkpoint before a failed sink can lose data; a checkpoint after an unprotected additive write can cause duplicates on retry. I need a mechanism that binds state and output atomically or makes replay idempotent, such as stable event identity with a transactional upsert or committed partition replacement. I would demonstrate a crash after the sink write but before checkpoint persistence and show the recovered logical result.
Pressure test: What happens if the same event ID arrives with a different payload?
Avoid: Calling an offset or retry loop an exactly-once guarantee.
10 / Coding 4 min read · 30 min practice
Nested JSON to valid CSV
PYTHON LAB / 18 MIN PRACTICE
12 / Nested JSON aggregation to CSV
Input is a JSON string containing a list of objects, each with account:string and events:list of {kind:string, count:int>=0}. Required values must have these types; bool counts are invalid. Extra keys are ignored. Aggregate counts by (account,kind), sorted by both, then return CSV text with header account,kind,count and LF line endings. Missing/null required fields and malformed JSON raise ValueError. Empty events create no group. Whitespace inside nonempty strings is preserved.
Hint
Use json.loads and csv.writer. Validate before aggregation; do not join CSV fields with commas.
Worked solution and test cases
def json_to_csv(payload):
import json, csv, io
rows = json.loads(payload)
if not isinstance(rows,list):
raise ValueError('top level must be a list')
totals = {}
for row in rows:
if not isinstance(row,dict) or not isinstance(row.get('account'),str) or not row['account'] or not isinstance(row.get('events'),list):
raise ValueError('invalid account/events')
for event in row['events']:
if not isinstance(event,dict) or not isinstance(event.get('kind'),str) or not event['kind'] or type(event.get('count')) is not int or event['count'] < 0:
raise ValueError('invalid event')
key = (row['account'],event['kind'])
totals[key] = totals.get(key,0) + event['count']
output = io.StringIO(newline='')
writer = csv.writer(output,lineterminator='\n')
writer.writerow(['account','kind','count'])
for (account,kind),count in sorted(totals.items()):
writer.writerow([account,kind,count])
return output.getvalue()
Reasoning and complexity: Parsing materializes the JSON document. O(B+E+g log g) time and O(B+g+output) memory, where B is parsed document size, E event count and g group count. Streaming NDJSON needs a different input contract.
assert json_to_csv("[]")=="account,kind,count\n"
for p in ["oops", "{}", '[{"account":"a","events":null}]','[{"account":"a","events":[{"kind":"x","count":true}]}]']:
try: json_to_csv(p)
except ValueError: pass
else: raise AssertionError("invalid payload accepted")
PYTHON LAB / 12 MIN PRACTICE
21 / Validate NDJSON with explicit rejects
Parse nonblank NDJSON lines. Each must be an object with nonempty id:string and count:int>=0, excluding bool. Return (valid,rejects), where valid preserves accepted objects including extra keys; rejects contains (one_based_line_number,reason) with reason exactly "json" or "schema". Blank lines are skipped but count in line numbers. Catch expected parsing errors only.
Hint
Separate syntax failure from schema failure. Preserve line identity for repair.
Worked solution and test cases
def parse_ndjson(lines):
import json
valid, rejects = [], []
for number,line in enumerate(lines,1):
if not line.strip():
continue
try:
row = json.loads(line)
except json.JSONDecodeError:
rejects.append((number,'json'))
continue
if not isinstance(row,dict) or not isinstance(row.get('id'),str) or not row['id'] or type(row.get('count')) is not int or row['count'] < 0:
rejects.append((number,'schema'))
continue
valid.append(row)
return valid,rejects
Reasoning and complexity: O(bytes) parsing time and O(accepted+rejected payload) memory in this return-list contract. A production sink could stream accepted and rejected records instead.
Aggregate nested JSON into CSV. What are your first decisions?
Reveal worked answer
I would define top-level shape, required fields/types, grouping keys, empty arrays, missing/null behavior, malformed-row policy and output order. I would parse with json, validate schema, aggregate at the requested grain and serialize with csv.writer so commas, quotes and newlines are escaped correctly. For one large JSON array, json.loads materializes the document; NDJSON allows record-wise parsing but grouping state may still grow.
Pressure test: Should a malformed row fail the batch or enter quarantine, and who owns recovery?
Avoid: Using string splitting or silently dropping invalid data.
This extends the older DBSBI provider-ranking exercise to a precise contract. Treat 21 files as one logical dataset: a retry in file 21 can refer to an event in file 1. File count does not change the identity rule. The workspace uses fictional CSV strings so the same transformation is runnable offline.
PYTHON LAB / 25 MIN PRACTICE
13 / Multi-file provider ranking
files is a nonempty list of CSV text strings. Exact ordered header: event_id,provider_id,attempts,on_time. Ignore blank lines; header-only files are valid. Fields must be present, with no extras. IDs must be nonempty and unchanged by strip(). Counts contain ASCII digits only and 0<=on_time<=attempts. Deduplicate exact parsed retries by event_id across all files; conflicting parsed values fail with ValueError. Return (provider_id,on_time,attempts) tuples ordered by weighted rate descending, zero-denominator providers last, then provider_id ascending. Never average row rates.
Hint
Separate parsing, identity deduplication, aggregation and final ranking. Fraction avoids floating-point tie surprises.
Worked solution and test cases
def rank_providers(files):
import csv, io
from fractions import Fraction
if not files:
raise ValueError('no files')
header = ['event_id','provider_id','attempts','on_time']
seen, totals = {}, {}
for content in files:
reader = csv.DictReader(io.StringIO(content,newline=''), strict=True)
try:
actual_header = reader.fieldnames
except csv.Error as exc:
raise ValueError('malformed CSV header') from exc
if actual_header != header:
raise ValueError('unexpected header')
try:
for row in reader:
if None in row or any(row[key] is None for key in header):
raise ValueError('wrong field count')
event, provider = row['event_id'], row['provider_id']
if not event or not provider or event != event.strip() or provider != provider.strip():
raise ValueError('invalid id')
a, o = row['attempts'], row['on_time']
if not a or not o or any(c not in '0123456789' for c in a+o):
raise ValueError('invalid integer')
attempts, on_time = int(a), int(o)
if on_time > attempts:
raise ValueError('on_time exceeds attempts')
value = (provider,attempts,on_time)
if event in seen:
if seen[event] != value:
raise ValueError('conflicting event')
continue
seen[event] = value
s,n = totals.get(provider,(0,0))
totals[provider] = (s+on_time,n+attempts)
except csv.Error as exc:
raise ValueError('malformed CSV') from exc
output = [(p,s,n) for p,(s,n) in totals.items()]
return sorted(output,key=lambda r:(r[2]==0,-Fraction(r[1],r[2]) if r[2] else 0,r[0]))
Reasoning and complexity: O(B+u log u) high-level time for input bytes B and u providers, ignoring big-integer arithmetic cost. Memory O(e+u) for exact event identities and totals, plus input/output. Reading lines incrementally does not make exact global dedup constant-memory.
from pathlib import Path
def rank_directory(directory):
paths = sorted(Path(directory).glob('*.csv'))
if not paths:
raise ValueError('no CSV files')
# This simple adapter materializes file texts. Use a context-managed
# iterator adapter when the input-byte footprint is too large.
contents = []
for path in paths:
with path.open(encoding='utf-8', newline='') as handle:
contents.append(handle.read())
return rank_providers(contents)
SAY IT / Primary mock coding
Write rank_providers for the multi-file contract, then defend retries, ranking and invalid rows.
Reveal worked answer
Start with exact headers and a per-file CSV parser. Validate IDs, field count and integer counts before using them. Maintain event ID to parsed payload across all files: exact retries are skipped and conflicts fail. Aggregate on_time and attempts per provider, then sort by the ratio of sums descending, with undefined zero-volume rates last and provider ID as a deterministic tie-breaker. Fraction provides exact rational comparison for the fixture.
Pressure test: Change file order, repeat a whole file, and add a distinct event with identical counts. Which outputs should change?
Avoid: Deduplicating only within a file, averaging percentages, or treating no files as a successful empty report.
Yield lists of size at most size from a one-shot iterable, with a shorter final list allowed. Positive integer size excluding bool required; otherwise ValueError when iteration begins. Do not pre-read the whole source.
Hint
Create the iterator once, outside the loop. Slice only the next bounded group.
Worked solution and test cases
def batches(iterable, size):
from itertools import islice
if type(size) is not int or size <= 0:
raise ValueError('positive integer size required')
iterator = iter(iterable)
while True:
batch = list(islice(iterator,size))
if not batch:
return
yield batch
Reasoning and complexity: O(n) time, O(size) buffered state. Generator validation executes on next()/iteration, not necessarily at function call.
seen=[]
def gen():
for x in range(20):
seen.append(x); yield x
b=batches(gen(),3)
assert next(b)==[0,1,2] and seen==[0,1,2]
assert list(batches([],2))==[]
Yield all leaves of a finite acyclic nested list in left-to-right order. Only lists are containers; strings, None, dicts and tuples are leaves. Avoid recursive calls and full output materialization. Top level is a list.
Hint
Use a stack of iterators, one for each active nesting depth.
Worked solution and test cases
def flatten(values):
stack = [iter(values)]
while stack:
try:
value = next(stack[-1])
except StopIteration:
stack.pop()
continue
if isinstance(value,list):
stack.append(iter(value))
else:
yield value
Reasoning and complexity: O(nodes+leaves) time and O(depth) auxiliary state. A recursion alternative is readable for bounded depth but can exceed the interpreter recursion limit.
a=[1]
for _ in range(1500): a=[a]
assert list(flatten(a))==[1]
Check: Empty and generator
g=flatten([])
assert iter(g) is g and list(g)==[]
SAY IT / Iteration
What changes when a list-based API becomes a generator API?
Reveal worked answer
Execution and validation can be delayed until iteration, input can be consumed only once, and the consumer determines how much output is retained. I must avoid len(), indexing and a second pass unless the contract supports them. I would test partial consumption and a source with side effects to ensure I am not eagerly reading the entire input.
Pressure test: Why must iter(iterable) be created once in a batching loop?
Avoid: Calling list() at the beginning and still describing the implementation as lazy.
events contains {event_id,account_id}; dimensions contains {account_id,tier}. Dimension keys must be unique and non-null, including identical duplicates: fail ValueError otherwise. Return fresh {event_id,account_id,tier} dictionaries in event order. Null/unmatched event keys retain the event with tier None. Inputs are otherwise trusted.
Hint
Build the unique dimension map while checking duplicates; then preserve the left population.
Worked solution and test cases
def enrich(events, dimensions):
lookup = {}
for row in dimensions:
key = row['account_id']
if key is None or key in lookup:
raise ValueError('dimension key must be unique and non-null')
lookup[key] = row['tier']
return [dict(event_id=r['event_id'],account_id=r['account_id'],tier=lookup.get(r['account_id'])) for r in events]
Reasoning and complexity: Expected O(d+n) time, O(d) auxiliary lookup plus O(n) output. A repeated event key is fine; a repeated dimension key changes join cardinality.
Compare two finite iterables of hashable integer event IDs. Return {missing:[(id,count)],extra:[(id,count)]}, sorted ascending by id. Missing means needed to make actual match expected. Retain multiplicity.
def reconcile(expected, actual):
from collections import Counter
want, have = Counter(expected), Counter(actual)
return {'missing': sorted((want-have).items()), 'extra': sorted((have-want).items())}
Reasoning and complexity: Expected O(n+m+u log u) time and O(u) memory. Counter subtraction removes nonpositive entries, exactly what missing and extra need.
How do you prove an enrichment preserved the event population?
Reveal worked answer
I validate that the dimension key is unique and define null-key behavior before joining. A left many-to-one join should retain one row per input event. I compare event identity multiplicity, not only grand totals, and count unmatched keys separately. If the dimension has multiple historical versions, I need an as-of contract rather than silently taking the last dictionary entry.
Pressure test: Two wrong events cancel in the row count. What catches the problem?
Avoid: Using distinct to conceal fanout or treating equal row totals as full equivalence.
14 / Coding 4 min read · 36 min practice
Sessions, intervals and dependency graphs
PYTHON LAB / 12 MIN PRACTICE
17 / Per-user sessionization
Trusted rows are (user:string, timestamp:int), unordered. Return (user,session_number,start,end,event_count), sorted by user then start. New session begins when gap from the preceding event is strictly greater than gap. Equal timestamps are separate events. gap must be a nonnegative int excluding bool. Do not mutate input.
Hint
Order by user and time, compare with the preceding event, then flush the last session.
Worked solution and test cases
def sessions(rows, gap):
if type(gap) is not int or gap < 0:
raise ValueError('nonnegative integer gap required')
output = []
current = None
number = 0
for user, ts in sorted(rows):
if current is None or user != current[0] or ts-current[3] > gap:
if current is not None:
output.append(tuple(current))
number = number+1 if current is not None and user==current[0] else 1
current = [user,number,ts,ts,1]
else:
current[3] = ts; current[4] += 1
if current is not None:
output.append(tuple(current))
return output
Reasoning and complexity: O(n log n) time, O(n) sorted copy plus output. If upstream supplies grouped ordered rows, the scan can be incremental.
Input pairs [start,end] are closed integer intervals with start<=end. Merge overlaps and touching intervals, return sorted [start,end] pairs. Invalid bounds raise ValueError. Input must not change.
Hint
Sort starts, merge into the last emitted interval, copy pairs before modifying.
Worked solution and test cases
def merge_intervals(intervals):
for start,end in intervals:
if type(start) is not int or type(end) is not int or start > end:
raise ValueError('invalid interval')
output = []
for start,end in sorted(intervals):
if not output or start > output[-1][1]:
output.append([start,end])
else:
output[-1][1] = max(output[-1][1],end)
return output
Reasoning and complexity: O(n log n) time and O(n) memory including sort/output. Closed-interval touching semantics differ from some half-open scheduling contracts.
dependencies maps each string task to a list of prerequisite task names. Referenced tasks must exist; fail ValueError otherwise. Ignore repeated edges. Return a valid order, choosing the alphabetically first currently ready task at every step. Cycles raise ValueError.
Hint
Kahn algorithm: zero-indegree heap, remove edges, verify every task was emitted.
Worked solution and test cases
def dependency_order(dependencies):
import heapq
followers = {task:set() for task in dependencies}
degree = {}
for task, prereqs in dependencies.items():
unique = set(prereqs)
if not unique <= dependencies.keys():
raise ValueError('unknown dependency')
degree[task] = len(unique)
for before in unique:
followers[before].add(task)
ready = [task for task,count in degree.items() if count == 0]
heapq.heapify(ready)
output = []
while ready:
task = heapq.heappop(ready); output.append(task)
for after in followers[task]:
degree[after] -= 1
if degree[after] == 0:
heapq.heappush(ready,after)
if len(output) != len(dependencies):
raise ValueError('dependency cycle')
return output
Reasoning and complexity: Expected O(E+V log V) time and O(E+V) memory. The heap gives deterministic ready-task selection; a queue would be linear but a different tie rule.
for d in [{"a":["b"],"b":["a"]},{"a":["missing"]},{"a":["a"]}]:
try: dependency_order(d)
except ValueError: pass
else: raise AssertionError("invalid graph accepted")
Check: Empty graph
assert dependency_order({})==[]
SAY IT / Boundary reasoning
When does a new session start, and how does that differ from merging intervals?
Reveal worked answer
A session starts when the gap from the previous event exceeds the agreed inactivity threshold, with state reset per user. It is not measured from the session start. Interval merging compares the next interval start with the current merged end and depends on closed versus half-open semantics. In both cases I sort by the necessary keys, state the boundary operator and test an exact-boundary example.
Pressure test: Events at 0,20,40 with a 30-unit threshold: one session or two?
Avoid: Using a remembered comparison operator without stating the customer rule.
15 / Coding 3 min read · 18 min practice
Debugging mutability and decimal precision
PYTHON LAB / 10 MIN PRACTICE
22 / Exact decimal aggregation
Given strings representing finite decimal amounts, sum exactly and round the final total to two decimal places using ROUND_HALF_UP. Return a two-place string. Reject invalid or nonfinite values with ValueError. Fixture amounts have <=12 significant digits and <=6 decimal places; at most 1000 amounts. Round once at the end.
Hint
Construct Decimal from the original string, specify rounding, and size the context for the input contract.
Worked solution and test cases
def total_amount(amounts):
from decimal import Decimal, InvalidOperation, localcontext, ROUND_HALF_UP
with localcontext() as ctx:
ctx.prec = 40
total = Decimal('0')
for text in amounts:
if not isinstance(text,str):
raise ValueError('amount must be a string')
try:
value = Decimal(text)
except InvalidOperation as exc:
raise ValueError('invalid decimal') from exc
if not value.is_finite():
raise ValueError('nonfinite amount')
total += value
return format(total.quantize(Decimal('0.01'),rounding=ROUND_HALF_UP),'.2f')
Reasoning and complexity: Time is linear in number of bounded-size inputs; memory is constant accumulator space at this bounded precision. Arbitrary-size decimals require a broader precision/range policy.
for x in ["NaN","Infinity","bad",1.0]:
try: total_amount([x])
except ValueError: pass
else: raise AssertionError("invalid amount accepted")
PYTHON LAB / 8 MIN PRACTICE
23 / Mutable default and aliasing repair
Implement append_event(event, buffer=None): if omitted, create a new list; if supplied, append to that same list. Store a deep copy of the event dictionary, so later changes to nested input do not alter saved events. Return the buffer.
Hint
Use None as the default and an identity check; distinguish an explicitly supplied empty list.
Worked solution and test cases
def append_event(event, buffer=None):
import copy
if buffer is None:
buffer = []
buffer.append(copy.deepcopy(event))
return buffer
Reasoning and complexity: The function intentionally mutates a supplied buffer, while isolating stored payloads. Copy cost is proportional to the event object graph, not O(1).
A function works once but returns extra events on later calls. How do you investigate?
Reveal worked answer
I would reproduce two calls in the same interpreter and inspect mutable default arguments, module-level state and shared nested references. If a caller supplies an empty buffer, I must distinguish it from None instead of replacing it with buffer or []. I would make ownership explicit, then test repeated calls, object identity and later mutation of the original payload.
Pressure test: What does a shallow dictionary copy fail to isolate?
Avoid: Fixing the sample output without identifying the shared-state mechanism.
The recruiter explicitly mentions dataframe manipulation. Use pandas when the dataset fits a local process and its APIs are available. Use Spark for distributed transformations when the environment and scale justify it. The browser lab embeds standard-library Python only; pandas below was executed locally with assertions. The Spark example is complete and syntax checked, but was not run against a Spark engine here.
Operation
Pandas reasoning
Spark reasoning
Filter/project
Use explicit boolean masks and selected columns; .copy() when you need independent mutation.
Use select/filter with native column expressions.
Latest version
Detect conflicting equal versions, sort, then drop duplicate identities.
Check equal-version conflicts, then row_number in a partition ordered by version.
Join
merge(validate="many_to_one", indicator=True); reject null dimension keys if required.
Validate dimension uniqueness separately; choose left join and inspect unmatched keys.
Nulls
pandas merges can match null keys; groupby needs dropna=False to preserve a null group.
Ordinary equality joins do not match null keys; null-safe matching must be explicit.
Aggregation
Named agg; size counts rows while count excludes missing values.
json_normalize or explode; define empty-array behavior and parallel-array relationships.
explode versus explode_outer changes parent preservation.
Scale
Chunking does not bound global grouping/dedup state. Avoid repeated concat in a loop.
Avoid unbounded collect/toPandas and Python UDFs for operations native functions express.
Order
Sort output explicitly when required.
Distributed row order is not guaranteed without orderBy; global ordering has a cost.
Complete pandas exercise with checked fixture
import pandas as pd
events = pd.DataFrame([
('e1','a',1,2), ('e1','a',1,2),
('e2','a',2,3), ('e3','unknown',1,4), ('e4',None,1,1)
], columns=['event_id','account_id','version','units'])
accounts = pd.DataFrame([('a','pro'),('b','basic')], columns=['account_id','tier'])
# Trusted non-null event IDs/versions; conflicting same-version payloads fail.
unique = events.drop_duplicates()
if unique.duplicated(['event_id','version']).any():
raise ValueError('conflicting event version')
latest = (unique.sort_values(['event_id','version'])
.drop_duplicates('event_id', keep='last'))
if accounts['account_id'].isna().any():
raise ValueError('null dimension key')
enriched = latest.merge(accounts, on='account_id', how='left',
validate='many_to_one', indicator=True)
assert len(enriched) == len(latest) == 4
summary = (enriched.groupby('tier', dropna=False, as_index=False)
.agg(events=('event_id','size'), units=('units','sum')))
assert summary.loc[summary['tier'].eq('pro'), 'units'].iloc[0] == 5
assert summary.loc[summary['tier'].isna(), 'units'].iloc[0] == 5
assert (enriched['_merge'] == 'left_only').sum() == 2
# Explode only the event list; do not accidentally form a Cartesian product.
payload = [{'account':'a','events':[{'kind':'read','count':2},
{'kind':'read','count':3}]},
{'account':'b','events':[]}]
flat = pd.json_normalize(payload, record_path='events', meta='account')
counts = flat.groupby(['account','kind'], as_index=False)['count'].sum()
assert counts.to_dict('records') == [{'account':'a','kind':'read','count':5}]
# b has zero child rows under this contract. Preserve accounts separately if needed.
print(summary.to_string(index=False))
Complete Spark translation for an installed Spark environment
from pyspark.sql import SparkSession, Window, functions as F
spark = SparkSession.builder.master('local[2]').appName('round5-practice').getOrCreate()
try:
events = spark.createDataFrame([
('e1','a',1,2),('e1','a',1,2),('e2','a',2,3),
('e3','unknown',1,4),('e4',None,1,1)
], 'event_id string, account_id string, version long, units long')
accounts = spark.createDataFrame([('a','pro'),('b','basic')],
'account_id string, tier string')
unique = events.dropDuplicates()
conflicts = unique.groupBy('event_id','version').count().filter(F.col('count') > 1)
if conflicts.limit(1).count():
raise ValueError('conflicting event version')
if accounts.filter(F.col('account_id').isNull()).limit(1).count():
raise ValueError('null dimension key')
if accounts.groupBy('account_id').count().filter(F.col('count') > 1).limit(1).count():
raise ValueError('duplicate dimension key')
w = Window.partitionBy('event_id').orderBy(F.col('version').desc())
latest = unique.withColumn('_rn', F.row_number().over(w)).filter('_rn = 1').drop('_rn')
enriched = latest.join(accounts, ['account_id'], 'left')
result = enriched.groupBy('tier').agg(F.count('*').alias('events'),
F.sum('units').alias('units'))
# collect is bounded here by a two-group toy fixture, not a production pattern.
observed = {r['tier']:(r['events'],r['units']) for r in result.collect()}
assert observed == {'pro':(2,5), None:(2,5)}
result.show()
finally:
spark.stop()
SAY IT / Dataframe transfer
Translate the latest-event enrichment into pandas or Spark and identify semantic traps.
Reveal worked answer
I would collapse exact retries, reject conflicting payloads at the same event/version, then select the latest version per event. I would validate unique, nonnull dimension keys and perform a left enrichment, preserving unmatched events. In pandas I can use merge validation and dropna=False during grouping. In Spark I need explicit dimension validation and a window for version selection. I would avoid unbounded collect and distinguish null-key behavior across engines.
Pressure test: How would exploding two independent arrays affect row count?
Avoid: Treating equivalent-looking API calls as proof of identical null, ordering and multiplicity behavior.
Distance exactly k; time exactly lower bound; gap exactly threshold.
Wrong inequalities.
Multiplicity
Same key twice versus two distinct keys with equal values.
Accidental dedup or double counting.
Adversarial shape
Deep list, equal heap priorities, duplicate dimension key.
Recursion, comparison and join assumptions.
Metamorphic
Repeat a retry file; reorder independent files; split a batch.
Idempotency and order invariants.
Independent oracle
Compare two-sum with enumeration on small random arrays.
A second implementation with a different failure mode.
Operational
Crash after write, before checkpoint; partial consumption.
Recovery and laziness claims.
The 24 worked solutions have 76 named checks, including randomized brute-force or sorting comparisons. The runner evaluates fresh solution namespaces per check, so repeated-call behavior is tested explicitly within the relevant check. Passing these checks is evidence for the stated fixtures. Add a changed-input argument before calling the function complete.
SAY IT / Proof
All supplied tests pass. What do you do before finishing?
Reveal worked answer
I restate the invariant and use it to explain the general case, then add an edge case the tests did not lead me toward. I review mutation, exception behavior, deterministic ties and space including output. I separate measured fixture correctness from untested scale behavior. If a requirement remains ambiguous, I state the assumption instead of hiding it.
Pressure test: Give a wrong algorithm that could pass one happy-path fixture.
Avoid: Claiming a finite test suite proves every possible input.
18 / Leadership principle 2 min read · 12 min practice
Invent and Simplify: remove recurring burden
Amazon emphasizes invention, simplification and learning from ideas wherever they originate. Your answer should show a recurring customer problem, a mechanism you changed and evidence that another person could use the result. Novel technology alone does not establish useful invention.
A simplification is complete when the user’s work becomes easier without hiding risk.
Evidence
Strong answer
Weak substitute
Before
Actual steps, duplicated logic, error modes and user pain.
The system was complex.
Choice
What you removed, reused, standardized or deliberately left manual.
We adopted a new framework.
Personal work
Your diagnosis, design, implementation and adoption actions.
The team automated it.
Result
Supported before/after effort, quality and independent use.
It saved millions because the company is large.
Learning
What the first design overcomplicated and how you reduced scope.
Everything worked from the start.
SAY IT / Invent and Simplify
How do you distinguish simplification from moving complexity somewhere hidden?
Reveal worked answer
I compare the total workflow before and after: user decisions, operating steps, exceptions, dependency burden and recovery. A shorter function that moves opaque logic into configuration is not automatically simpler. I would test whether a second engineer can complete a representative task and recover a failure using the documented contract. I would retain explicit exceptions when making them generic would obscure their meaning.
Pressure test: What complexity did your own solution introduce?
Avoid: Equating fewer lines or a fashionable tool with lower customer burden.
19 / Invent and Simplify 4 min read · 15 min practice
One contract instead of repeated pipeline forks
SAY IT / Invent and Simplify
Tell me about: one contract instead of repeated pipeline forks.
Reveal worked answer
In a product analytics program, teams were adding near-identical pipelines for each new partner. The visible problem was delivery time, but the harder customer problem was that the same metric could acquire different definitions in different copies. I owned the common transformation and needed a solution that a small team could maintain.
I compared the existing jobs and separated true business differences from accidental implementation differences. I proposed a small configuration contract for source mapping, entity grain and refresh schedule, backed by shared transformation code. I deliberately kept exception logic in named functions rather than creating a general-purpose language that only I could debug. I borrowed the configuration-driven pattern from an existing internal tool and credited its owner.
I tested the approach on two representative pipelines, including the least convenient exception. I compared keys, values and duplicate counts against the existing outputs, documented the unsupported cases and asked another engineer to add a pipeline without my help. That exposed an unclear configuration field, which I changed before expanding adoption. I migrated incrementally with a rollback path and explicit owners for the shared contract.
In this fictional rehearsal, eight pipeline implementations became three reusable stages plus explicit configuration, and the second engineer added a new supported feed without changing the common code. Those are illustrative results, not verified personal measurements. The useful evidence would be actual reduction in duplicated logic, onboarding effort and recurring incidents over a defined period.
The lesson was that fewer lines are not the goal. A good simplification makes the correct path easier for the next engineer while leaving exceptions and operating responsibility visible.
Pressure test: What was invented? A narrower reusable contract and migration mechanism for a repeated customer problem. The pattern can be borrowed; the valuable work is adapting it and proving it useful.
Avoid: Reciting invented facts, using company scale as personal impact, or avoiding your own trade-off.
Follow-up
Evidence and answer direction
What was invented?
A narrower reusable contract and migration mechanism for a repeated customer problem. The pattern can be borrowed; the valuable work is adapting it and proving it useful.
Why not use one giant generic framework?
The variability was bounded. A general expression engine would transfer complexity into configuration, testing and incident response without a demonstrated need.
How did you measure simplification?
Compare actual duplicated transformations, time to add a supported feed, review effort and incident/recovery burden. Name the measurement window and do not equate lines removed with business impact.
How did you handle the exception?
Keep the exception explicit with a named owner and test. Either extend the supported contract deliberately or leave it outside; do not force it through misleading configuration.
What resistance did you encounter?
A real answer needs a specific concern and the pilot evidence that addressed it. In this rehearsal the maintenance and adoption concern is answered by another engineer using the tool independently.
What did you personally do?
Diagnosed duplication, proposed the contract, implemented the shared stages, designed parity tests and ran the pilot. Separate partner review and rollout work from your ownership.
90-second cut: One sentence on the customer and problem, one on your decision, three on your actions, one supported result and one lesson. Keep the detailed mechanism available for follow-up.
20 / Invent and Simplify 4 min read · 15 min practice
Turning recurring data repair into a bounded replay tool
SAY IT / Invent and Simplify
Tell me about: turning recurring data repair into a bounded replay tool.
Reveal worked answer
A recurring late-data issue required an engineer to identify partitions, reconstruct a command, check dependencies and manually reconcile the output. I noticed that the human effort repeated the same decisions and that copying old commands made repair risky. I owned the repair workflow and wanted to reduce both effort and mistakes.
I first mapped the decisions that required judgment and the steps that were deterministic. I built a small Python tool that generated a repair plan from an explicit date range and source version. It showed the affected partitions, validated dependencies, ran a dry-run comparison and required the operator to choose the execution scope. I kept the write operation separate from planning and used stable run identity so a retry would not create a second logical result.
I rejected a more ambitious automatic remediation design because the tool could not infer whether every source correction was semantically valid. Instead, it produced a clear exception report for those cases. I tested twelve fictional fixtures in this rehearsal, including an empty range, duplicate retry, missing dependency and partial sink failure. Another engineer then completed a repair using only the documented workflow.
The constructed outcome was a repeatable repair process with a visible plan, bounded inputs and a reconciliation record. If I use a real version of this incident, I will replace that outcome with the actual number of repairs, time saved and observed failure behavior. I would not attach public company scale or speculative revenue savings to my personal contribution.
The lesson was to automate the repeatable mechanics while making the remaining judgment clearer. The simplest useful tool was smaller than my first idea because it did one repair workflow reliably.
Pressure test: Why is this invention rather than ordinary scripting? The meaningful contribution is redesigning a recurring workflow and reducing customer/operational burden. Explain the new mechanism and adoption evidence, not the novelty of Python.
Avoid: Reciting invented facts, using company scale as personal impact, or avoiding your own trade-off.
Follow-up
Evidence and answer direction
Why is this invention rather than ordinary scripting?
The meaningful contribution is redesigning a recurring workflow and reducing customer/operational burden. Explain the new mechanism and adoption evidence, not the novelty of Python.
What did you remove?
Repeated command reconstruction, implicit scope and manual comparison steps. The source validity decision remained explicit.
What happens on a partial write?
The real implementation must have a commit/replay mechanism. In the design, use a transactional sink or idempotent partition replacement with a manifest; do not claim exactly-once from a run ID alone.
Why retain human judgment?
The source correction may be syntactically valid but semantically wrong. The tool can expose evidence without pretending to resolve a domain decision it cannot verify.
What failed in your first design?
The initial ambition conflated detecting a problem with knowing its correct repair. Narrowing scope made the supported behavior testable.
How would you prove adoption?
Name who independently used it, which recurring repairs it handled and the actual before/after effort. Sending a link is not adoption.
90-second cut: One sentence on the customer and problem, one on your decision, three on your actions, one supported result and one lesson. Keep the detailed mechanism available for follow-up.
21 / Leadership principle 2 min read · 12 min practice
Disagree with evidence, then commit in actions
The principle asks for respectful challenge when you disagree and full commitment once a decision is made. Prepare one story where your evidence changed the choice and one where you lost the argument. The second story is often where commitment becomes observable.
An engineering disagreement needs a decision and visible follow-through.
Moment
What to show
Reason
Before dissent
Understand the other person’s valid constraint.
Avoid arguing against a caricature.
Challenge
State the risk, evidence, alternative and trade-off.
Conviction should survive a concrete counterexample.
Decision
Identify the accountable owner and what was decided.
Repeated discussion without a decision is not a complete story.
Commit
Change your plan, own implementation and communicate consistently.
Quietly waiting for failure is not commitment.
Reopen
Bring new SLA, recovery or customer evidence.
Repeated preference is not new information.
Boundary
Unresolved security, policy or data-integrity obligations may require escalation.
A legitimate trade-off and an unacceptable misrepresentation are different.
SAY IT / Backbone
You lose an architecture argument. What does commitment look like the next day?
Reveal worked answer
I align the delivery plan with the chosen architecture, own the work needed to make it successful and explain the decision consistently to partners. I retain a concise record of assumptions and measurable triggers for reconsideration. I do not continue the rejected design in parallel without agreement or wait for the chosen option to fail. If new evidence invalidates an assumption, I bring it back through the decision owner.
Pressure test: What evidence would justify reopening rather than simply repairing the chosen implementation?
Avoid: A story where you eventually won every argument and never demonstrate commitment.
22 / Have Backbone; Disagree and Commit 4 min read · 15 min practice
Challenging a metric launch with a concrete counterexample
SAY IT / Have Backbone; Disagree and Commit
Tell me about: challenging a metric launch with a concrete counterexample.
Reveal worked answer
A product partner wanted to launch a dashboard using a joined event table because its aggregate total matched an existing report. I disagreed with the conclusion that the match established correctness. I was responsible for the data model, and a many-to-many identity relationship could duplicate one population while omitting another, leaving the grand total unchanged.
I first made sure I understood the deadline and the decision the dashboard would support. I then brought a small counterexample to the review: two keys with offsetting errors that preserved the total but changed the segment comparison. I explained the customer consequence and proposed a bounded alternative: resolve identity at the intended grain, validate segment coverage and publish the unaffected view while the disputed segment was repaired.
The partner was concerned that I was expanding scope near launch. I separated the must-fix correctness issue from enhancements that could wait and asked the decision owner to choose between the documented options. In this constructed rehearsal, we agreed to the narrower launch with explicit coverage, and I took responsibility for the reconciliation and follow-up delivery instead of merely raising the objection.
The fictional result was a launch whose stated population matched the validated data, followed by the corrected segment once the acceptance cases passed. I would use actual delivery dates and customer consequences in a real answer. I would not claim that my objection alone protected a company-wide revenue figure.
The lesson was to challenge the decision with evidence and an executable alternative. Being technically right would have been insufficient if I had ignored the deadline or left the team with no path forward.
Pressure test: Who disagreed, and what was at stake? State the actual roles, decision and customer consequence. Respect the partner’s deadline concern; do not describe them as careless.
Avoid: Reciting invented facts, using company scale as personal impact, or avoiding your own trade-off.
Follow-up
Evidence and answer direction
Who disagreed, and what was at stake?
State the actual roles, decision and customer consequence. Respect the partner’s deadline concern; do not describe them as careless.
What evidence changed the discussion?
A minimal counterexample showing why matching grand totals could conceal segment error. Walk the records, join multiplicity and affected decision.
Did you escalate?
Only to the accountable decision owner when the trade-off could not be resolved at the working level. A real answer must describe the actual escalation and why it was needed.
What if the owner still chose launch?
Clarify the permitted scope and decision rights, document the risk and support the agreed valid scope. Do not mislabel known incorrect data as correct; escalate unresolved integrity or policy obligations through the appropriate channel.
How did you commit?
Owned the narrower release, reconciliation and next delivery. Commitment is visible work and consistent communication after the decision.
What would change your mind?
Evidence that the identity relationship was truly one-to-one at the relevant time/grain, or that the disputed segment was outside the decision’s population. Name disconfirming evidence.
90-second cut: One sentence on the customer and problem, one on your decision, three on your actions, one supported result and one lesson. Keep the detailed mechanism available for follow-up.
23 / Have Backbone; Disagree and Commit 4 min read · 15 min practice
Losing the streaming argument and owning the batch design
SAY IT / Have Backbone; Disagree and Commit
Tell me about: losing the streaming argument and owning the batch design.
Reveal worked answer
I preferred a streaming design for an analytics pipeline because I expected future consumers to need fresher data. The owner favored scheduled micro-batches because the current report was hourly and the operating team was small. I had a strong technical preference, but the team needed a decision based on the actual customer requirement.
I compared the options on freshness, recovery, operating effort and likely expansion. I explained where streaming would help and asked what evidence supported the hourly requirement. The owner showed that no committed consumer needed sub-hour freshness and that adding a new stateful service would increase on-call responsibility. I still saw future value in streaming, but I recognized that I was giving an uncertain future use case too much weight.
Once the owner chose micro-batches, I supported that decision publicly and took responsibility for making it work. I implemented deterministic batch identity, replay-safe writes, late-data handling and a reconciliation view. I removed the streaming dependency from the delivery plan and wrote down a small set of conditions that would justify reopening the design: a committed lower-latency consumer or measured inability to meet the agreed recovery target.
In this fictional rehearsal, the team met the hourly acceptance window with a simpler operating model. The real version needs the actual observation period, delivery evidence and remaining limitations. I would not say I was vindicated later unless that happened and I could explain it without undermining the team’s earlier decision.
The lesson was that commitment is an engineering behavior. I did not need to pretend my original preference never existed; I needed to implement the chosen design well and let new evidence, rather than frustration, trigger the next debate.
Pressure test: Did you actually disagree or merely offer an option? Name the recommendation you advocated, the evidence you presented and the decision you did not get. Avoid a story with no real tension.
Avoid: Reciting invented facts, using company scale as personal impact, or avoiding your own trade-off.
Follow-up
Evidence and answer direction
Did you actually disagree or merely offer an option?
Name the recommendation you advocated, the evidence you presented and the decision you did not get. Avoid a story with no real tension.
Why were you wrong?
I overweighted a speculative future latency need and underweighted current operating capacity. That is a specific judgment error, not a vague communication issue.
What did commitment look like the next day?
Changed the plan, removed the rejected dependency, implemented replay/recovery controls and represented the decision consistently to partners.
What if the chosen design starts failing?
Diagnose and repair it in good faith. Reopen with measured SLA/recovery evidence and alternatives, not an I-told-you-so narrative.
How is commitment different from silence?
Silence avoids the initial challenge; commitment follows a substantive evidence-based debate and an explicit decision.
Would you always obey a decision?
Decision authority does not erase data integrity, security or policy obligations. Distinguish a legitimate engineering trade-off from an unresolved obligation requiring escalation.
90-second cut: One sentence on the customer and problem, one on your decision, three on your actions, one supported result and one lesson. Keep the detailed mechanism available for follow-up.
24 / Personalize 2 min read · 20 min practice
Your four-story evidence workspace
Use the supplied resume’s programs as anchors. Publicly available company numbers can establish context only; they cannot prove a personal outcome. Confirmed notes are your attestation, not independent verification. Editing notes clears the confirmation. Keep a primary and backup per principle and avoid repeating the same story across every round.
Invent and Simplify
One contract instead of repeated pipeline forks
Invent and Simplify
Turning recurring data repair into a bounded replay tool
Have Backbone; Disagree and Commit
Challenging a metric launch with a concrete counterexample
Have Backbone; Disagree and Commit
Losing the streaming argument and owning the batch design
25 / Under pressure 2 min read · 10 min practice
Interruptions and changing requirements
Interruption
Response
Why
Make it streaming.
Identify which input/output/state is materialized, then change only the needed contract.
A generator wrapper does not remove a global sort.
This input can be malformed.
Agree fail-fast versus quarantine, add schema validation and one failing case.
Do not silently swallow errors.
Why not Spark?
Compare actual data size, execution environment and operational cost.
A big-data tool is not automatically the right answer.
Your result is wrong.
Trace a smallest counterexample and revise the invariant.
Defending a broken result wastes time and trust.
I disagree with your design.
Ask which assumption differs, compare evidence and recognize the decision owner.
Do not turn a technical discussion into a personal contest.
SAY IT / Live correction
You discover your solution fails after confidently explaining it. What do you say?
Reveal worked answer
I would name the specific defect: for example, my set removes a value while another copy remains in the window. I would show the smallest failing input, replace membership-only state with counts, and rerun both the original and failing cases. I would update the complexity explanation and state any remaining limitation. Correcting the mechanism directly is more useful than defending the initial answer.
Pressure test: What if there is no time to finish? State the remaining defect and exact next change.
Avoid: Silently changing code while leaving the old verbal explanation uncorrected.
SAY IT / Closing
Ask two useful questions and close the interview.
Reveal worked answer
I would ask which recurring data-engineering workflows this team most wants to simplify, and how engineers resolve trade-offs between product freshness, correctness and operating burden. I would follow up on the actual answer and connect my experience to one concrete need. I would avoid asking for a score or assuming that the job description reveals the team’s full architecture.
Pressure test: Which answer changes your understanding of this role?
Avoid: A generic question list with no listening or follow-up.
26 / Knowledge 5 min read · 20 min practice
Thirty knowledge checks
Aim for 30/30 cold and explain the tempting wrong option. Reset clears these answers only.
27 / Practice workspace 2 min read · 5 min practice
Write, run and test Python offline
Python 3.12.7 runs locally through pinned Pyodide 0.27.7. The interpreter and standard library are embedded in this single file. Each run starts a disposable worker, with up to 30 seconds to initialize and 5 seconds for code/tests after readiness. Stop terminates the worker. Output is capped at 12,000 characters. These are practice limits, not algorithm benchmarks. Pandas and Spark are covered in the separate transfer chapter.
Engine starts when you run.
Captured outputTest cases / inspect after your attempt
Three complete interviews, with first-person scripts, explicit follow-ups and checked technical answers.
Read the scripts in the next three chapters, then rehearse cold. Use the 60-minute mode for a full interview or the short drill for concise recall. The continuous clock pauses and resumes; moving stages does not reset it.
Nailed: correct mechanism and a defensible follow-up. Wobbled: right direction with a missing condition or weak evidence. Blank: no usable solution. Repair each gap, then repeat cold.
29 / Complete interview 16 min read · 0 min practice
Full mock A / Streaming CSV, bounded duplicates and simpler contracts
Six stages, one 60-minute interview. Every question has a complete first-person answer and two answered follow-ups.
Evidence boundary: Technical cases are worked examples. Personal incidents and their specific outcomes are constructed rehearsals anchored to resume programs, not verified autobiographical claims. The introduction reflects the supplied background.
Read the complete transcript on pass 1. On pass 2, keep answers closed and use the runner to speak, code and defend your decisions. Reading is counted in this chapter; one live mock is counted in the runner section. The all-three-mocks total adds two more hours.
STAGE 1 / 00:00 TO 05:00 / 5 MIN
Shafi, introduce yourself and connect your experience to this interview.
Resume-based introduction
Shafi’s full answer + 2 answered follow-ups
I am a data engineer with 16-plus years of experience across enterprise data platforms and product analytics. My background includes telecom data migration at Sprint, the Citi analytics platform and Meta product and payments data. At Meta, my work included Automotive and Travel data foundations, first-party data pipelines, shared metrics and cross-functional delivery. I have led technical work without treating leadership as a reporting-line title.
I use Python to make data contracts executable. I start with input shape, ordering, identity and error handling, then choose a data structure with a clear complexity bound. I value a solution another engineer can review and recover, not just a short implementation.
For this role, I am interested in turning complex service and product data into reliable decisions. I would bring the same discipline I use in production: define what the data means, make the implementation understandable, test failure cases and own the operating outcome.
Interviewer follow-ups, fully answered
1. What did you personally own rather than the broader team?
I owned data-engineering decisions and implementation within the programs I worked on: defining transformations and models, resolving data-quality questions, coordinating dependencies and validating outputs. Product managers set product priorities and partner engineers owned other systems. I distinguish those contributions when I describe a result; the size of a product does not make all of its revenue my personal impact.
2. Why are you interested in this Data Engineer II role?
The work described in the posting is close to problems I enjoy: scalable data products, analytics foundations and engineering decisions that serve real consumers. I would evaluate the role through its ownership, customer problems and opportunity to contribute. My experience can help me work independently and mentor through technical judgment, while I remain hands-on with the implementation.
STAGE 2 / 05:00 TO 30:00 / 25 MIN
files is a nonempty list of CSV text strings. Exact ordered header: event_id,provider_id,attempts,on_time. Ignore blank lines; header-only files are valid. Fields must be present, with no extras. IDs must be nonempty and unchanged by strip(). Counts contain ASCII digits only and 0<=on_time<=attempts. Deduplicate exact parsed retries by event_id across all files; conflicting parsed values fail with ValueError. Return (provider_id,on_time,attempts) tuples ordered by weighted rate descending, zero-denominator providers last, then provider_id ascending. Never average row rates.
Technical scenario
Shafi’s full answer + 2 answered follow-ups
I parse CSV with the standard csv module so quoted commas and headers are handled correctly. The input is a list of CSV text strings. I parse each file once, deduplicate exact event retries across files and aggregate by provider before ranking. A repeated event ID with a conflicting parsed payload fails. The full implementation below encodes field validation, weighted calculation and deterministic ties.
I would not average row rates because their denominators can differ. I sum on_time and attempts per provider, compare their ratios exactly using Fraction, place zero-denominator providers last and break ties by provider ID. With R rows, U unique events and P providers, the main work is O(R + P log P), treating field sizes and arithmetic as bounded. Auxiliary memory is O(U + P): the deduplication map can grow with the entire input, even though CSV rows are parsed incrementally. The supplied text strings also already occupy memory.
Before considering the answer finished, I run the named checks below, including malformed input and tied results. In production I would keep file provenance with validation failures and define whether a rejected row fails the batch or enters quarantine. The exercise’s failure contract is explicit, so the implementation follows it instead of silently skipping bad rows.
Python 3.12 / standard library
def rank_providers(files):
import csv, io
from fractions import Fraction
if not files:
raise ValueError('no files')
header = ['event_id','provider_id','attempts','on_time']
seen, totals = {}, {}
for content in files:
reader = csv.DictReader(io.StringIO(content,newline=''), strict=True)
try:
actual_header = reader.fieldnames
except csv.Error as exc:
raise ValueError('malformed CSV header') from exc
if actual_header != header:
raise ValueError('unexpected header')
try:
for row in reader:
if None in row or any(row[key] is None for key in header):
raise ValueError('wrong field count')
event, provider = row['event_id'], row['provider_id']
if not event or not provider or event != event.strip() or provider != provider.strip():
raise ValueError('invalid id')
a, o = row['attempts'], row['on_time']
if not a or not o or any(c not in '0123456789' for c in a+o):
raise ValueError('invalid integer')
attempts, on_time = int(a), int(o)
if on_time > attempts:
raise ValueError('on_time exceeds attempts')
value = (provider,attempts,on_time)
if event in seen:
if seen[event] != value:
raise ValueError('conflicting event')
continue
seen[event] = value
s,n = totals.get(provider,(0,0))
totals[provider] = (s+on_time,n+attempts)
except csv.Error as exc:
raise ValueError('malformed CSV') from exc
output = [(p,s,n) for p,(s,n) in totals.items()]
return sorted(output,key=lambda r:(r[2]==0,-Fraction(r[1],r[2]) if r[2] else 0,r[0]))
# Executable checks
# Weighted ranking and cross-file retry
h="event_id,provider_id,attempts,on_time\n"
a=h+"e1,a,1,1\ne2,a,9,0\ne3,b,2,1\n"
b=h+"e1,a,1,1\ne4,z,0,0\n"
assert rank_providers([a,b])==[("b",1,2),("a",1,10),("z",0,0)]
# Header-only and quoted provider
h="event_id,provider_id,attempts,on_time\n"
assert rank_providers([h])==[]
assert rank_providers([h+'e1,"a,b",2,2\n'])==[("a,b",2,2)]
# Conflict, malformed and missing files
h="event_id,provider_id,attempts,on_time\n"
for files in [[],[""],[chr(34)+"unclosed"],[h+"e,a,1,1\ne,a,2,1\n"],[h+"e,a,1\n"],[h+"e,a,1,1,x\n"],[h+"e,a,1.0,1\n"],[h+"e,a,1,2\n"]]:
try: rank_providers(files)
except ValueError: pass
else: raise AssertionError("invalid file set accepted")
# Exact-rate tie
h="event_id,provider_id,attempts,on_time\n"
assert rank_providers([h+"x,b,4,2\ny,a,2,1\n"])==[("a",1,2),("b",2,4)]
Checked result: All 4 named checks pass; each assert above states the expected value or error.
Interviewer follow-ups, fully answered
1. What if there are too many providers to fit in memory?
I would first deduplicate and detect conflicts using external grouping by event ID, then partition the canonical rows by provider for aggregation. Partitioning only by provider is insufficient because a conflicting event might name a different provider. A database or external engine can spill both stages. A top-k heap reduces result selection memory after aggregation, but it does not remove the deduplication or grouping state.
2. How would you avoid counting the same file twice?
I would require a stable batch or object identity and record completed inputs in a durable manifest tied to the output commit. A filename alone may be reused, so the identity should include the agreed object version or content identity. If partial output can be replayed, I need idempotent keys or atomic replacement of the affected batch, not just a Python set that disappears on restart.
STAGE 3 / 30:00 TO 40:00 / 10 MIN
Return whether any equal integers appear at indices i<j with j-i<=k. k must be a nonnegative int, excluding bool; invalid k raises ValueError. k=0 is valid and always false. The input may be a one-shot iterable.
Technical scenario
Shafi’s full answer + 2 answered follow-ups
I keep a deque of the previous at most k values and a set for membership. Before inserting the current value, I check the set; a match proves an equal value exists within k positions. If there is no match, I append the value and expire the oldest item when the deque exceeds k.
The set is safe without counts because I return immediately on the first duplicate in the active window, so retained values are unique on every continuing iteration. At k=0, each inserted value is immediately removed and no pair qualifies. Expected time is O(n), auxiliary space is O(min(n,k)) apart from one transient inserted value, and a one-shot iterable works.
That removes the unnecessary nested loop. The checks below cover the exact k boundary, no qualifying duplicate and invalid k. This is a positional window, not an event-time window. If the requirement changed to timestamps, I would need an ordered time contract and a different expiration rule.
Python 3.12 / standard library
def nearby_duplicate(values, k):
if type(k) is not int or k < 0:
raise ValueError('k must be a nonnegative integer')
from collections import deque
window, present = deque(), set()
for value in values:
if value in present:
return True
window.append(value); present.add(value)
if len(window) > k:
present.remove(window.popleft())
return False
# Executable checks
# Exact distance boundary
assert nearby_duplicate([1,2,1],2) is True
assert nearby_duplicate([1,2,1],1) is False
# Zero and iterator
assert nearby_duplicate([1,1],0) is False
assert nearby_duplicate(iter([1,2,3,1]),3) is True
# Invalid k
for k in [-1,True,1.5]:
try: nearby_duplicate([],k)
except ValueError: pass
else: raise AssertionError("invalid k accepted")
# Pair oracle
import random
r=random.Random(54)
for _ in range(100):
a=r.choices(range(5),k=r.randrange(15)); k=r.randrange(7)
assert nearby_duplicate(a,k)==any(a[i]==a[j] and j-i<=k for j in range(len(a)) for i in range(j))
Checked result: All 4 named checks pass; each assert above states the expected value or error.
Interviewer follow-ups, fully answered
1. Could you solve it with a last-index dictionary instead?
Yes. Before updating the latest index for a value, I would check whether its previous index is at most k positions away. That also gives expected linear time, but a dictionary without expiration can retain every distinct value ever seen. The deque-and-set solution bounds state by the active window, which is useful for a long stream.
2. Does sorting the input help?
Sorting would destroy the original position distances unless I retained and reasoned about the indices separately. It also adds O(n log n) work. The requirement is about nearby original positions, so a hash-based scan directly matches the contract and provides a clearer proof.
STAGE 4 / 40:00 TO 48:00 / 8 MIN
Tell me about simplifying a repeated engineering problem.
Constructed personal incident; resume program anchor
Shafi’s full answer + 2 answered follow-ups
In a product analytics program, teams were adding near-identical pipelines for each new partner. The visible problem was delivery time, but the harder customer problem was that the same metric could acquire different definitions in different copies. I owned the common transformation and needed a solution that a small team could maintain.
I compared the existing jobs and separated true business differences from accidental implementation differences. I proposed a small configuration contract for source mapping, entity grain and refresh schedule, backed by shared transformation code. I deliberately kept exception logic in named functions rather than creating a general-purpose language that only I could debug. I borrowed the configuration-driven pattern from an existing internal tool and credited its owner.
I tested the approach on two representative pipelines, including the least convenient exception. I compared keys, values and duplicate counts against the existing outputs, documented the unsupported cases and asked another engineer to add a pipeline without my help. That exposed an unclear configuration field, which I changed before expanding adoption. I migrated incrementally with a rollback path and explicit owners for the shared contract.
Eight pipeline implementations became three reusable stages plus explicit configuration. A second engineer added a supported feed without changing the common code. The concrete improvement was that the shared stages had one implementation and one test suite, while genuine business exceptions remained visible. I did not use lines removed as a substitute for correct outputs or claim a revenue effect from that refactor.
The lesson was that fewer lines are not the goal. A good simplification makes the correct path easier for the next engineer while leaving exceptions and operating responsibility visible.
Interviewer follow-ups, fully answered
1. How did you avoid building an over-general framework?
I extracted only the stages shared by the existing feeds and kept genuine business exceptions explicit in configuration or separate handling. I tested the abstraction by having another engineer onboard one supported feed. I did not add speculative plugin layers for cases we had not encountered.
2. What evidence showed the simplification worked?
Eight implementations became three common stages with one set of acceptance tests, and another engineer added a supported feed without changing that common code. I also compared outputs against the previous pipelines. The evidence was reduced duplication with preserved semantics, not a claim that fewer lines automatically meant fewer defects.
STAGE 5 / 48:00 TO 56:00 / 8 MIN
Describe disagreeing with a launch decision and following through.
Constructed personal incident; resume program anchor
Shafi’s full answer + 2 answered follow-ups
A product partner wanted to launch a dashboard using a joined event table because its aggregate total matched an existing report. I disagreed with the conclusion that the match established correctness. I was responsible for the data model, and a many-to-many identity relationship could duplicate one population while omitting another, leaving the grand total unchanged.
I first made sure I understood the deadline and the decision the dashboard would support. I then brought a small counterexample to the review: two keys with offsetting errors that preserved the total but changed the segment comparison. I explained the customer consequence and proposed a bounded alternative: resolve identity at the intended grain, validate segment coverage and publish the unaffected view while the disputed segment was repaired.
The partner was concerned that I was expanding scope near launch. I separated the must-fix correctness issue from enhancements that could wait and asked the decision owner to choose between the documented options. In this constructed rehearsal, we agreed to the narrower launch with explicit coverage, and I took responsibility for the reconciliation and follow-up delivery instead of merely raising the objection.
The narrower launch represented the population we had actually validated. We added the corrected segment after its key-level acceptance cases passed. I owned the reconciliation and follow-up delivery, so my disagreement came with the work needed to reach the goal. The result was an accurate scope and a completed correction, rather than an unsupported claim about revenue protected.
The lesson was to challenge the decision with evidence and an executable alternative. Being technically right would have been insufficient if I had ignored the deadline or left the team with no path forward.
Interviewer follow-ups, fully answered
1. How did you make the disagreement constructive?
I brought a concrete key-level counterexample and explained which customer conclusion it invalidated. I proposed a narrower validated population and volunteered to own the reconciliation and missing-segment follow-up. That gave the decision owner a workable route rather than only an objection.
2. Would you have committed if the owner chose the wider launch?
I would first make the unresolved correctness issue and its consequence explicit to the accountable owner. If the decision was a legitimate tradeoff with accurate disclosure, I would implement the agreed scope and controls. I would not represent an unsupported metric as validated or hide a known defect in order to appear committed.
STAGE 6 / 56:00 TO 60:00 / 4 MIN
What questions do you have for me, and how would you close?
Role-focused closing
Shafi’s full answer + 2 answered follow-ups
I have two questions. Which Python workloads create the most operational burden for the team? How do you review scripts for replay safety and data contracts as well as code style?
The reason I ask is that I want to understand the decisions the data supports and where the team’s operating burden sits. My experience is most useful when I can connect a data contract to a customer outcome and then carry the implementation through validation and operation. Thank you for the discussion; the problems we covered reinforce my interest in doing that work here.
Interviewer follow-ups, fully answered
1. What would you focus on during your first month?
I would learn one important consumer workflow end to end, trace its data sources and definitions, and understand the team’s release and incident practices. I would pair with an engineer on a small, useful change rather than propose a redesign before understanding the constraints. By the end of the month I would aim to own a bounded improvement with an agreed acceptance test and a clear operational handoff.
2. What would you do if the team’s approach differed from your previous company?
I would first understand which constraints the approach solves: scale, staffing, correctness, cost or customer expectations. I would compare mechanisms rather than brand names. If I saw a material issue, I would bring a concrete example and an alternative, then work through the accountable owner. Once a legitimate decision was made, I would help implement it well.
30 / Complete interview 15 min read · 0 min practice
Full mock B / Nested JSON, replay safety and disagreeing then committing
Six stages, one 60-minute interview. Every question has a complete first-person answer and two answered follow-ups.
Evidence boundary: Technical cases are worked examples. Personal incidents and their specific outcomes are constructed rehearsals anchored to resume programs, not verified autobiographical claims. The introduction reflects the supplied background.
Read the complete transcript on pass 1. On pass 2, keep answers closed and use the runner to speak, code and defend your decisions. Reading is counted in this chapter; one live mock is counted in the runner section. The all-three-mocks total adds two more hours.
STAGE 1 / 00:00 TO 05:00 / 5 MIN
Shafi, introduce yourself and connect your experience to this interview.
Resume-based introduction
Shafi’s full answer + 2 answered follow-ups
I am a data engineer with 16-plus years of experience across enterprise data platforms and product analytics. My background includes telecom data migration at Sprint, the Citi analytics platform and Meta product and payments data. At Meta, my work included Automotive and Travel data foundations, first-party data pipelines, shared metrics and cross-functional delivery. I have led technical work without treating leadership as a reporting-line title.
My Python work sits inside larger data systems, so I think about what happens after a timeout or a restart as well as what happens on a clean input. I prefer explicit schemas, deterministic outputs and a small recovery surface that another engineer can operate.
For this role, I am interested in turning complex service and product data into reliable decisions. I would bring the same discipline I use in production: define what the data means, make the implementation understandable, test failure cases and own the operating outcome.
Interviewer follow-ups, fully answered
1. What did you personally own rather than the broader team?
I owned data-engineering decisions and implementation within the programs I worked on: defining transformations and models, resolving data-quality questions, coordinating dependencies and validating outputs. Product managers set product priorities and partner engineers owned other systems. I distinguish those contributions when I describe a result; the size of a product does not make all of its revenue my personal impact.
2. Why are you interested in this Data Engineer II role?
The work described in the posting is close to problems I enjoy: scalable data products, analytics foundations and engineering decisions that serve real consumers. I would evaluate the role through its ownership, customer problems and opportunity to contribute. My experience can help me work independently and mentor through technical judgment, while I remain hands-on with the implementation.
STAGE 2 / 05:00 TO 30:00 / 25 MIN
Input is a JSON string containing a list of objects, each with account:string and events:list of {kind:string, count:int>=0}. Required values must have these types; bool counts are invalid. Extra keys are ignored. Aggregate counts by (account,kind), sorted by both, then return CSV text with header account,kind,count and LF line endings. Missing/null required fields and malformed JSON raise ValueError. Empty events create no group. Whitespace inside nonempty strings is preserved.
Technical scenario
Shafi’s full answer + 2 answered follow-ups
I first identify the output grain and the one nested list that should produce rows. I accumulate the required measures by the specified key and write the result with csv.writer rather than concatenating strings. That handles delimiters, quoting and embedded newlines according to CSV rules.
The code below validates the shapes required by the contract, handles the specified missing or empty collections and produces deterministic ordering. I do not recursively explode every list because unrelated arrays would create a Cartesian product. I also do not assume that a nested field being present means it has the correct type. The named checks make those boundary choices visible.
For a very large source, parsing one giant JSON object still requires memory even if the aggregation loop is efficient. I would request NDJSON or a streaming parser for a suitable source format. If the number of distinct output keys is itself too large, I would partition or spill the aggregation. Those are separate memory constraints.
Python 3.12 / standard library
def json_to_csv(payload):
import json, csv, io
rows = json.loads(payload)
if not isinstance(rows,list):
raise ValueError('top level must be a list')
totals = {}
for row in rows:
if not isinstance(row,dict) or not isinstance(row.get('account'),str) or not row['account'] or not isinstance(row.get('events'),list):
raise ValueError('invalid account/events')
for event in row['events']:
if not isinstance(event,dict) or not isinstance(event.get('kind'),str) or not event['kind'] or type(event.get('count')) is not int or event['count'] < 0:
raise ValueError('invalid event')
key = (row['account'],event['kind'])
totals[key] = totals.get(key,0) + event['count']
output = io.StringIO(newline='')
writer = csv.writer(output,lineterminator='\n')
writer.writerow(['account','kind','count'])
for (account,kind),count in sorted(totals.items()):
writer.writerow([account,kind,count])
return output.getvalue()
# Executable checks
# Aggregate nested groups
p='[ {"account":"a","events":[{"kind":"read","count":2},{"kind":"read","count":3}]} ]'
assert json_to_csv(p)=="account,kind,count\na,read,5\n"
# CSV quoting roundtrip
import json,csv,io
p=json.dumps([{"account":"a,b","events":[{"kind":'say "hi"\nnext',"count":0}]}])
assert list(csv.reader(io.StringIO(json_to_csv(p))))==[["account","kind","count"],["a,b",'say "hi"\nnext',"0"]]
# Empty and invalid required fields
assert json_to_csv("[]")=="account,kind,count\n"
for p in ["oops", "{}", '[{"account":"a","events":null}]','[{"account":"a","events":[{"kind":"x","count":true}]}]']:
try: json_to_csv(p)
except ValueError: pass
else: raise AssertionError("invalid payload accepted")
Checked result: All 3 named checks pass; each assert above states the expected value or error.
Interviewer follow-ups, fully answered
1. How would you handle schema evolution?
I would distinguish additive optional fields from changes that alter required keys or measures. I would version the input contract, retain rejected-record context and test representative old and new records. I would not silently coerce a newly nested object into a string when the downstream schema expects a number.
2. How would you verify a CSV consumer reads the intended output?
I would round-trip the generated text through csv.reader or DictReader and compare parsed fields and totals, including a value with a comma, quote or newline. I would specify encoding, newline handling, headers and numeric representation in the contract. Visual inspection of the raw CSV is not enough.
STAGE 3 / 30:00 TO 40:00 / 10 MIN
state is {seen:{event_id:(account,delta)}, totals:{account:int}}. Trusted incoming rows are (event_id:string,account:string,delta:int). Apply each new event once. An exact retry is a no-op; conflicting payload for a seen event raises ValueError. Return a new state, leaving the input state unchanged even if a late row fails. This is an in-memory logical model, not a claim of durable atomicity.
Technical scenario
Shafi’s full answer + 2 answered follow-ups
The central issue is that an additive update is not idempotent by itself. Replaying the same event can add its value twice. The implementation below tracks the stable event identity under the exercise’s contract and applies each new event once, while the checks replay inputs and test conflicting identities.
The in-memory model demonstrates the state transition, but production correctness requires the deduplication record and aggregate change to commit together. If I write the aggregate and crash before recording the ID, a retry can double count. If I record the ID first and crash before the aggregate update, I can lose the change. A transactional sink, atomic batch replacement or a durable ledger with deterministic recomputation must close that gap.
I would also define retention for deduplication state and what happens when a replay arrives after that retention period. I would not call the entire pipeline exactly-once simply because the Python function passes repeated-call tests.
Python 3.12 / standard library
def apply_events(state, rows):
next_state = {'seen':dict(state['seen']), 'totals':dict(state['totals'])}
seen, totals = next_state['seen'], next_state['totals']
for event,account,delta in rows:
value = (account,delta)
if event in seen:
if seen[event] != value:
raise ValueError('conflicting replay')
continue
seen[event] = value
totals[account] = totals.get(account,0) + delta
return next_state
# Executable checks
# Replay twice and negative delta
s={"seen":{},"totals":{}}
a=apply_events(s,[("e","a",4),("f","a",-1)])
assert a["totals"]=={"a":3}
assert apply_events(a,[("e","a",4)])==a
assert s=={"seen":{},"totals":{}}
# Failure leaves original intact
s={"seen":{"old":("a",1)},"totals":{"a":1}}
try: apply_events(s,[("new","a",5),("old","a",9)])
except ValueError: pass
else: raise AssertionError("conflict accepted")
assert s=={"seen":{"old":("a",1)},"totals":{"a":1}}
# Distinct IDs both count
assert apply_events({"seen":{},"totals":{}},[("x","a",2),("y","a",2)])["totals"]=={"a":4}
Checked result: All 3 named checks pass; each assert above states the expected value or error.
Interviewer follow-ups, fully answered
1. What if the same ID arrives with a different amount?
That is not a harmless replay. I would use a versioned correction contract if the source supports it, or reject the conflict for investigation. Reusing an identity with a different payload must not silently overwrite an additive history. The fixture’s conflict checks make that distinction explicit.
2. Where would you store the checkpoint?
I would store the input position or batch manifest in the same durable transaction as the sink changes where the architecture supports that. Otherwise I would make the sink replay-safe and advance the checkpoint only after confirmed publication. A local file on one worker is insufficient for a distributed restart unless ownership and durability are explicitly guaranteed.
STAGE 4 / 40:00 TO 48:00 / 8 MIN
Tell me about automating a manual process without hiding risk.
Constructed personal incident; resume program anchor
Shafi’s full answer + 2 answered follow-ups
A recurring late-data issue required an engineer to identify partitions, reconstruct a command, check dependencies and manually reconcile the output. I noticed that the human effort repeated the same decisions and that copying old commands made repair risky. I owned the repair workflow and wanted to reduce both effort and mistakes.
I first mapped the decisions that required judgment and the steps that were deterministic. I built a small Python tool that generated a repair plan from an explicit date range and source version. It showed the affected partitions, validated dependencies, ran a dry-run comparison and required the operator to choose the execution scope. I kept the write operation separate from planning and used stable run identity so a retry would not create a second logical result.
I rejected a more ambitious automatic remediation design because the tool could not infer whether every source correction was semantically valid. Instead, it produced a clear exception report for those cases. I tested twelve fictional fixtures in this rehearsal, including an empty range, duplicate retry, missing dependency and partial sink failure. Another engineer then completed a repair using only the documented workflow.
The result was a repeatable repair process that another engineer completed from the documented plan. The tested partial-write case recovered through the agreed replay mechanism, and each repair produced a reconciliation record. I had reduced repeated command reconstruction without pretending that the tool could decide whether every source correction was semantically valid.
The lesson was to automate the repeatable mechanics while making the remaining judgment clearer. The simplest useful tool was smaller than my first idea because it did one repair workflow reliably.
Interviewer follow-ups, fully answered
1. Why did you keep a plan and reconciliation step?
The recurring work was reconstructing commands and scope, but deciding whether a source correction was valid still required evidence. The tool made the bounded input, intended changes and resulting reconciliation visible. That reduced manual repetition while retaining the decision an engineer needed to review.
2. What failure did you test before handing it to another engineer?
I tested a partial write followed by the documented replay path and checked that the affected scope reconciled without duplicate effects. I also made the repair plan explicit so the operator could see which partitions would change. A successful clean run would not establish that the repair tool was safe to recover.
STAGE 5 / 48:00 TO 56:00 / 8 MIN
Tell me about a technical disagreement where your preferred option was rejected.
Constructed personal incident; resume program anchor
Shafi’s full answer + 2 answered follow-ups
I preferred a streaming design for an analytics pipeline because I expected future consumers to need fresher data. The owner favored scheduled micro-batches because the current report was hourly and the operating team was small. I had a strong technical preference, but the team needed a decision based on the actual customer requirement.
I compared the options on freshness, recovery, operating effort and likely expansion. I explained where streaming would help and asked what evidence supported the hourly requirement. The owner showed that no committed consumer needed sub-hour freshness and that adding a new stateful service would increase on-call responsibility. I still saw future value in streaming, but I recognized that I was giving an uncertain future use case too much weight.
Once the owner chose micro-batches, I supported that decision publicly and took responsibility for making it work. I implemented deterministic batch identity, replay-safe writes, late-data handling and a reconciliation view. I removed the streaming dependency from the delivery plan and wrote down a small set of conditions that would justify reopening the design: a committed lower-latency consumer or measured inability to meet the agreed recovery target.
The team met the hourly acceptance window during the agreed pilot and retained a simpler operating model. The limitation was explicit: this design did not promise sub-hour freshness. I continued to own its recovery controls and used the documented customer and recovery triggers for future decisions, rather than keeping my rejected architecture alive in the background.
The lesson was that commitment is an engineering behavior. I did not need to pretend my original preference never existed; I needed to implement the chosen design well and let new evidence, rather than frustration, trigger the next debate.
Interviewer follow-ups, fully answered
1. Why did you support batch after arguing for streaming?
The customer requirement was an hourly acceptance window, and the team’s chosen batch approach met that requirement with a simpler operating model. Once the decision was made, I owned recovery controls and the pilot evidence. Continuing to optimize for my preferred architecture would have weakened the agreed plan.
2. When would you reopen the decision?
I would reopen it if the documented triggers changed, such as a real sub-hour freshness need, recovery that no longer fit the window or unacceptable source-load behavior. I would bring measured evidence tied to those triggers, not argue again simply because I still liked streaming.
STAGE 6 / 56:00 TO 60:00 / 4 MIN
What questions do you have for me, and how would you close?
Role-focused closing
Shafi’s full answer + 2 answered follow-ups
I have two questions. How does this team define successful operational simplicity? What would make you choose a streaming implementation over a well-controlled batch pipeline?
The reason I ask is that I want to understand the decisions the data supports and where the team’s operating burden sits. My experience is most useful when I can connect a data contract to a customer outcome and then carry the implementation through validation and operation. Thank you for the discussion; the problems we covered reinforce my interest in doing that work here.
Interviewer follow-ups, fully answered
1. What would you focus on during your first month?
I would learn one important consumer workflow end to end, trace its data sources and definitions, and understand the team’s release and incident practices. I would pair with an engineer on a small, useful change rather than propose a redesign before understanding the constraints. By the end of the month I would aim to own a bounded improvement with an agreed acceptance test and a clear operational handoff.
2. What would you do if the team’s approach differed from your previous company?
I would first understand which constraints the approach solves: scale, staffing, correctness, cost or customer expectations. I would compare mechanisms rather than brand names. If I saw a material issue, I would bring a concrete example and an alternative, then work through the accountable owner. Once a legitimate decision was made, I would help implement it well.
31 / Complete interview 14 min read · 0 min practice
Full mock C / Sorted streams, dataframe contracts and disciplined abstraction
Six stages, one 60-minute interview. Every question has a complete first-person answer and two answered follow-ups.
Evidence boundary: Technical cases are worked examples. Personal incidents and their specific outcomes are constructed rehearsals anchored to resume programs, not verified autobiographical claims. The introduction reflects the supplied background.
Read the complete transcript on pass 1. On pass 2, keep answers closed and use the runner to speak, code and defend your decisions. Reading is counted in this chapter; one live mock is counted in the runner section. The all-three-mocks total adds two more hours.
STAGE 1 / 00:00 TO 05:00 / 5 MIN
Shafi, introduce yourself and connect your experience to this interview.
Resume-based introduction
Shafi’s full answer + 2 answered follow-ups
I am a data engineer with 16-plus years of experience across enterprise data platforms and product analytics. My background includes telecom data migration at Sprint, the Citi analytics platform and Meta product and payments data. At Meta, my work included Automotive and Travel data foundations, first-party data pipelines, shared metrics and cross-functional delivery. I have led technical work without treating leadership as a reporting-line title.
I like to explain an algorithm through the invariant it preserves. That makes it easier to remove unnecessary work, choose the right data structure and test the exact condition that would break the solution. I apply the same reasoning to dataframe joins and production pipeline design.
For this role, I am interested in turning complex service and product data into reliable decisions. I would bring the same discipline I use in production: define what the data means, make the implementation understandable, test failure cases and own the operating outcome.
Interviewer follow-ups, fully answered
1. What did you personally own rather than the broader team?
I owned data-engineering decisions and implementation within the programs I worked on: defining transformations and models, resolving data-quality questions, coordinating dependencies and validating outputs. Product managers set product priorities and partner engineers owned other systems. I distinguish those contributions when I describe a result; the size of a product does not make all of its revenue my personal impact.
2. Why are you interested in this Data Engineer II role?
The work described in the posting is close to problems I enjoy: scalable data products, analytics foundations and engineering decisions that serve real consumers. I would evaluate the role through its ownership, customer problems and opportunity to contribute. My experience can help me work independently and mentor through technical judgment, while I remain hands-on with the implementation.
STAGE 2 / 05:00 TO 30:00 / 25 MIN
Each input stream contains dictionaries {ts: integer, id: string}, sorted by ts. Yield all records ordered by (ts, source index, position within source). Do not compare dictionaries. Keep at most one pending record per stream; inputs are trusted and finite.
Technical scenario
Shafi’s full answer + 2 answered follow-ups
Each input is already sorted, so I only need its next available item to determine the next global item. I keep one head from each nonempty iterator in a min-heap. I pop the smallest head, emit it and replace it with the next item from that same iterator. The implementation includes a deterministic tie-breaker so equal keys do not force comparison of incompatible payload objects.
The invariant is that the heap contains the smallest remaining candidate from each active stream. Therefore its minimum is the smallest item remaining across all streams. With K input streams, H initially nonempty streams and N total items, setup costs O(K) and merging costs O(N log(max(2,H))). The heap uses O(H) state, while the iterator list uses O(K), so total auxiliary state is O(K), excluding input storage and anything the caller retains. This also accounts for many empty streams.
For exactly two sorted arrays, I would normally use two indices and compare the two heads directly, which gives O(n+m) time and avoids heap overhead. For one-shot iterators, I would maintain two current values and exhausted flags. I would not repeatedly sort the growing result or use pop(0) on a Python list, which shifts elements.
Python 3.12 / standard library
def merge_partitions(streams):
import heapq
streams = [iter(s) for s in streams]
heap = []
for source, stream in enumerate(streams):
item = next(stream, None)
if item is not None:
heap.append((item['ts'], source, 0, item))
heapq.heapify(heap)
while heap:
_, source, pos, item = heapq.heappop(heap)
yield item
following = next(streams[source], None)
if following is not None:
heapq.heappush(heap, (following['ts'], source, pos+1, following))
# Executable checks
# Equal timestamps
s=[[{"ts":1,"id":"a"},{"ts":1,"id":"b"}],[{"ts":1,"id":"c"}],[]]
assert [r["id"] for r in merge_partitions(s)]==["a","b","c"]
# Interleaving
s=[iter([{"ts":2,"id":"x"}]),iter([{"ts":1,"id":"y"},{"ts":3,"id":"z"}])]
assert [r["id"] for r in merge_partitions(s)]==["y","x","z"]
# No streams
assert list(merge_partitions([]))==[]
Checked result: All 3 named checks pass; each assert above states the expected value or error.
Interviewer follow-ups, fully answered
1. What if an input is not sorted?
The merge invariant fails. I would either require sorted inputs and validate each source’s monotonic order while consuming it, or sort each bounded input before merging. Detecting a violation after yielding earlier values cannot retract them, so a consumer that needs atomic publication must stage or validate before making the output visible.
2. What if the consumer stops early?
A lazy generator avoids reading the rest of the inputs merely to build a result list. Resource ownership still matters: the caller or an enclosing context manager must close file handles. I would document whether the function owns the iterators, because arbitrary iterators do not all have the same close contract.
STAGE 3 / 30:00 TO 40:00 / 10 MIN
Using the complete fixture below, deduplicate versioned events, enrich them with account tier and aggregate units without losing unknown accounts. Also flatten the nested event list safely.
Technical scenario
Shafi’s full answer + 2 answered follow-ups
I distinguish duplicate delivery from conflicting versions. Exact duplicate rows can be removed; two different payloads for the same event ID and version fail the contract. I then choose the latest version per event. The dimension is required to have a non-null unique account key, and merge validation enforces the many-to-one relationship so enrichment cannot multiply facts.
I use a left join and retain unmatched accounts. Pandas groupby drops missing keys by default, so I set dropna=False to keep the unknown tier in the summary. The checked fixture has five units in the pro tier and five in the unknown tier, with four canonical events and two unmatched accounts.
For the nested payload I expand only the events list and retain its account metadata. Account b has an empty list, so it contributes no child row under this contract. If the report requires a row for every account, I keep an account spine and left join the child aggregate to it. I would not infer that an absent child row means an account does not exist.
The complete pandas program below includes its own inputs and assertions. It runs in local pandas 2.2.3. The guide’s embedded Python runtime provides the standard library only, so this particular dataframe example is for local execution.
Python / pandas 2.2.3 / local execution
import pandas as pd
events = pd.DataFrame([
('e1','a',1,2), ('e1','a',1,2),
('e2','a',2,3), ('e3','unknown',1,4), ('e4',None,1,1)
], columns=['event_id','account_id','version','units'])
accounts = pd.DataFrame([('a','pro'),('b','basic')], columns=['account_id','tier'])
# Trusted non-null event IDs/versions; conflicting same-version payloads fail.
unique = events.drop_duplicates()
if unique.duplicated(['event_id','version']).any():
raise ValueError('conflicting event version')
latest = (unique.sort_values(['event_id','version'])
.drop_duplicates('event_id', keep='last'))
if accounts['account_id'].isna().any():
raise ValueError('null dimension key')
enriched = latest.merge(accounts, on='account_id', how='left',
validate='many_to_one', indicator=True)
assert len(enriched) == len(latest) == 4
summary = (enriched.groupby('tier', dropna=False, as_index=False)
.agg(events=('event_id','size'), units=('units','sum')))
assert summary.loc[summary['tier'].eq('pro'), 'units'].iloc[0] == 5
assert summary.loc[summary['tier'].isna(), 'units'].iloc[0] == 5
assert (enriched['_merge'] == 'left_only').sum() == 2
# Explode only the event list; do not accidentally form a Cartesian product.
payload = [{'account':'a','events':[{'kind':'read','count':2},
{'kind':'read','count':3}]},
{'account':'b','events':[]}]
flat = pd.json_normalize(payload, record_path='events', meta='account')
counts = flat.groupby(['account','kind'], as_index=False)['count'].sum()
assert counts.to_dict('records') == [{'account':'a','kind':'read','count':5}]
# b has zero child rows under this contract. Preserve accounts separately if needed.
print(summary.to_string(index=False))
Checked result: Four canonical events; pro units = 5; unknown-tier units = 5; two left-only accounts; flattened a/read count = 5. All assertions pass.
Interviewer follow-ups, fully answered
1. What differs when moving this to Spark?
Spark is lazy, so transformations do not execute until an action. I would enforce the same keys and conflict rules, use a window for latest-version selection, and inspect the physical plan and shuffle behavior. Spark groupBy retains a null group, while pandas requires dropna=False here. Join null equality also needs explicit treatment rather than assuming identical library defaults.
2. When would you broadcast the account dimension?
Only when its serialized and in-memory sizes fit the workers with enough headroom and the join semantics support it. I would inspect the plan and measurements rather than use row count alone. Broadcasting an unexpectedly large dimension can make every executor fail, so I would retain a safe fallback and monitor dimension growth.
STAGE 4 / 40:00 TO 48:00 / 8 MIN
Tell me about a simplification you deliberately kept small.
Constructed personal incident; shared data-platform anchor
Shafi’s full answer + 2 answered follow-ups
In a shared reporting workflow, engineers had separate scripts for validating source coverage before a release. The scripts disagreed about how to represent a missing partition, and reviewers had to reconstruct the expected input set from notes. I owned one of the pipelines and proposed making that contract reusable.
I started with one explicit manifest: expected source, partition, cutoff and status. I wrote a small validation function that compared observed inputs with the manifest and returned structured missing, late and unexpected entries. I kept publication outside the function so a caller could review the result without accidentally releasing data.
I migrated two scripts first and compared their decisions on complete, missing and late fixtures. A partner engineer then adopted the function in a third workflow using the same documented result format. I rejected an early suggestion to add a general scheduling framework because scheduling was not the repeated defect we were fixing.
The three workflows used the same coverage decision and produced a reviewable exception list. We still had pipeline-specific transformations, but we no longer had three subtly different definitions of complete input. My lesson was that a small shared contract can remove more confusion than a broad abstraction with many extension points.
Interviewer follow-ups, fully answered
1. How did you avoid making one shared bug affect every pipeline?
I used explicit fixtures for the shared decision, migrated two callers first and compared their old and new results before switching the third. I versioned the contract so a change in missing-input semantics could not silently propagate. Centralizing code increases the importance of focused acceptance tests.
2. What was your measure of success?
I checked that each workflow made the same expected decision on complete, missing and late inputs, and that another engineer could interpret the structured exceptions without reading the validator. Adoption by three workflows was useful evidence, but I would not confuse adoption alone with correctness or claim an unmeasured time saving.
STAGE 5 / 48:00 TO 56:00 / 8 MIN
Describe pushing back on a clever solution and then supporting the final decision.
Constructed personal incident; pipeline engineering anchor
Shafi’s full answer + 2 answered follow-ups
During a data-processing review, a proposed solution used a deeply nested comprehension to flatten, filter, transform and aggregate records in one expression. It worked on the happy-path sample, but it mixed validation with transformation and made malformed-input behavior hard to see. I was responsible for reviewing and operating the resulting pipeline.
I raised the concern with a concrete malformed record and an example containing an empty child list. I wrote an alternative with three named steps and the same output contract. I compared both implementations on normal, empty and invalid fixtures rather than argue about personal style.
The owner chose to retain the concise transformation for one well-typed internal step while accepting explicit validation and a named aggregation function around it. That was not exactly my original proposal, but it addressed the operating concern. I helped add the fixtures and documented the error policy so the team could ship the agreed implementation.
The final code had a visible validation boundary and passed the shared cases. I learned to distinguish the mechanism I needed from the style I preferred. The goal was maintainable behavior under bad input, not winning a debate about how many lines Python should use.
Interviewer follow-ups, fully answered
1. Why was this worth disagreeing about?
The code was on an operational path where malformed input could stop a release or be silently misclassified. I showed that the ambiguity affected recovery and output behavior. If the concern had been only my formatting preference, I would have accepted the team convention without escalating it.
2. What if the concise version was measurably faster?
I would compare the performance difference with the actual workload and service target. If it mattered, I would preserve the optimized step behind a clear contract and focused tests. I would still separate input validation and make failure behavior explicit; speed does not require hiding the contract.
STAGE 6 / 56:00 TO 60:00 / 4 MIN
What questions do you have for me, and how would you close?
Role-focused closing
Shafi’s full answer + 2 answered follow-ups
I have two questions. What kinds of Python changes have the highest leverage for this team today? How does the team resolve disagreements about complexity and then check whether the chosen design worked?
The reason I ask is that I want to understand the decisions the data supports and where the team’s operating burden sits. My experience is most useful when I can connect a data contract to a customer outcome and then carry the implementation through validation and operation. Thank you for the discussion; the problems we covered reinforce my interest in doing that work here.
Interviewer follow-ups, fully answered
1. What would you focus on during your first month?
I would learn one important consumer workflow end to end, trace its data sources and definitions, and understand the team’s release and incident practices. I would pair with an engineer on a small, useful change rather than propose a redesign before understanding the constraints. By the end of the month I would aim to own a bounded improvement with an agreed acceptance test and a clear operational handoff.
2. What would you do if the team’s approach differed from your previous company?
I would first understand which constraints the approach solves: scale, staffing, correctness, cost or customer expectations. I would compare mechanisms rather than brand names. If I saw a material issue, I would bring a concrete example and an alternative, then work through the accountable owner. Once a legitimate decision was made, I would help implement it well.
Normal + edge + boundary; invariant; total memory; failure behavior.
Customer problem; your choice; alternatives; evidence; outcome; lesson.
Mastery target: 24 cold lab passes, 30/30 knowledge checks, both passes on every section, four real incident notes, all 18 full-mock stages nailed, readiness gates checked and zero unresolved repairs. These are self-recorded preparation targets. They do not represent Amazon’s internal evaluation.
33 / References 3 min read · 2 min practice
Coverage, sources and revision record
Prepared November 13, 2026 from the supplied recruiter call, scanned Python sheet, current Product BI JD and supplied resume. Cases and data are original fictional practice material. The historical DBSBI cases informed the provider-ranking exercise; they are not evidence of exact future questions.
Requirement / source
Covered here
Boundary
Python Scripting.pdf: sorted arrays
Merging chapter and labs 01-03
Inputs explicitly ordered; stream variant added.
Two-sum/three-sum
Labs 04-05 with enumeration comparisons
Value versus index output made explicit.
Window duplicates
Labs 06-08
Index and time semantics separated.
Nested JSON to CSV
Lab 12 and NDJSON lab 21
Types, quoting, empty arrays and failure policy.
Dataframe manipulation
Executed pandas fixture and complete Spark translation
Spark syntax checked only; no Spark runtime verification.
Recruiter Call for next role.txt
Python + Invent and Simplify + Have Backbone; Disagree and Commit
Round number is a prep label.
MShafi_ResumeDE (1).pdf
Program anchors for four rehearsals
Constructed incident details are labeled and need personal replacement.
Embedded browser Python runtime; teaching environment, not an interview-platform claim.
Two content passes
Pass 1 mapped every recruiter topic to contracts, worked examples and drills. Pass 2 challenged boundaries, multiplicities, mutation, tie rules, retained memory and failure semantics; executed the 24 solutions against 76 checks; and reviewed the dataframe transfer and behavioral evidence labels. Browser QA checks the embedded runtime and interactive controls separately.
Revision log
Version
Date
Changes
1.0
2026-11-13
24 offline Python labs; 76 checks; dataframe transfer; four constructed stories and 24 answered probes; 30 knowledge checks; three full mocks; two-pass progress; visible budgets; persistence; repair queue; print CSS.
Pyodide 0.27.7 is pinned and the downloaded npm archive was checked against its published SHA-512 integrity. It embeds Python 3.12.7; this is not a claim about Amazon’s coding environment. Its license is included below. All fixture data is fictional. Code executes locally; no account or cloud data is connected.
Embedded runtime license
Revision 1.2 / 2026-11-13
Added three complete 60-minute interview transcripts: 18 first-person primary answers and 36 fully answered follow-ups. Added complete technical fixtures and checked solutions where relevant, linked all stages to the existing timer and repair queue, and recalculated sidebar and full-guide study times. Preserved existing saved notes and migrated prior mock ratings into individual practice.