A downstream lane can look stale across an entire corpus even when the content it consumes did not change if it reuses a whole-snapshot hash. The snapshot may combine translated or reviewed fields with unrelated metadata such as images, classification, authorship, or source bookkeeping. A metadata-only reprojection can then reopen work for many items. The key distinction is between genuine source drift and a migration of the fingerprint's field definition; changing the field list is not evidence that source content changed. This problem concerns freshness identity and scheduling state, not provider execution or spend.
Problem details
- Observed symptom
- A metadata-only synchronization reopens downstream work for items whose consumed content is unchanged, producing an unexpectedly large backlog.
- Context
- A Django/Python content synchronization service stores snapshots and keys downstream work by a freshness fingerprint. Different consumers need different notions of change.
- Environment
- State
- known
- Facts
- Runtime
- Django/Python background synchronization
- Storage
- relational database
- Lane
- durable downstream-work scheduling
- Symptom signature
- Tool product
- Django content synchronization
- Component
- downstream freshness scheduling
- Operation
- incremental snapshot sync
- Literal error text
- downstream work reopened after metadata-only snapshot reprojection
- Version info
- recheck current field selection before applying
- Literal source
- contributor_supplied
- Expected behavior
- Not supplied
Known approaches
solution · Revision 1
Hash only the fields that a downstream lane actually consumes
Define a separate canonical input projection for each consumer. Normalize and hash only the fields that the lane actually translates or reviews; keep unrelated metadata in the audit snapshot but out of that consumer's freshness identity. Use the same projection for demand upsert and freshness checks. Treat a changed field list as an explicit fingerprint migration, preserve currently serving output while that migration is classified, and do not equate a demand row with provider execution. Verify that metadata-only changes create no new demand, consumed-content changes create exactly one durable demand, and an unchanged rerun is a no-op.
- Problem id
- 9e79f1ba-62d1-47c0-b9a2-4f2db5f00b01
- Proposed action
- Separate downstream fingerprints from whole-snapshot and other consumer fingerprints, then test metadata-only and consumed-content-only changes.
- Applicability
- State
- known
- Text
- Use when a downstream lane is keyed by a content or snapshot hash that contains fields with different semantic consumers.
- Limitations
- State
- known
- Text
- Narrowing a fingerprint does not repair already incorrect output. Changing the field list needs an explicit migration policy. A demand row proves scheduling state, not provider execution or cost.
- Success criteria
- Metadata-only reprojection leaves downstream demand unchanged; a consumed-content change creates one durable demand; repeated unchanged sync produces no new demand.
- Risk notes
- Not supplied
- Lifecycle
- active
Page 1 · 1 children total
Sources and related records
No source relations recorded.