Two-Way Sync Engineering: Conflicts, Loops, and Recovery
Learn why independent sync pipelines can lose updates and how to coordinate identity, field ownership, repeated writes, recovery, and reconciliation.
- Author
- Ruben Burdin · Founder & CEO
- Published
- Updated
- Read time
- 6 min read
A reverse pipeline needs a shared correctness contract
Two-way synchronization can be built from multiple workers, queues, and directional pipelines. The engineering challenge is to make those components agree on record identity, write ownership, ordering, and recovery. Two independent jobs that copy current rows in opposite directions do not automatically provide that agreement.
A label such as “unified engine” is not proof of correctness, and a distributed implementation is not inherently incorrect. Evaluate observable guarantees: which edits survive, how reflected writes are recognized, what happens after an ambiguous timeout, and whether the systems converge on the business-approved result.
This guide uses illustrative failure cases and acceptance tests. It does not present an unnamed customer benchmark or claim a universal cost reduction. Start with the two-way sync explanation if you need the terminology.
Trace an overlapping edit before designing the fix
Suppose both systems hold status A. A person changes system A to B while another changes system B to C. Each directional job captures its source independently. Without a common rule, a stale snapshot or later delivery can overwrite the other edit. The final result may depend on arrival order rather than the intended owner.
An echo is a different problem: after a worker writes to B, B reports that write as another change, and the reverse worker sends it back. Even if the value stays the same, a receiving workflow might send a second notification or run another action. Test the whole destination behavior, not only whether a row count increased.
The necessary protection depends on the APIs. Origin tracking, version checks, saved prior state, and application-level idempotency can contribute. Do not assume that adding a timestamp field or ignoring every write by an integration user preserves all legitimate changes.
Share the state that determines correctness
| State or policy | Why it matters | Acceptance evidence |
|---|---|---|
| Cross-system record identity | A retry must find the original entity | Repeated create produces one intended entity |
| Field ownership and prior values | A worker can distinguish allowed updates from competing edits | Both edit directions follow the agreed rule |
| Operation identity and outcome | A timeout may happen after commit | Retry does not repeat the business effect |
| Capture position and backfill boundary | Updates can arrive during initial load | Writes during backfill appear in the final dataset |
| Mapping version and validation | A replay may use an obsolete transformation | The intended mapping version is recorded |
| Exceptions and reconciliation state | A completed job can leave business records inconsistent | Affected IDs and values are checked after recovery |
Store only the state needed by the contract and protect it as production data. A key mapping registry is not disposable cache if losing it can create duplicate customers. Define its backup, restoration, retention, and migration behavior alongside the workers that use it.
Field-level edits still need business validation
Merging an email change from one system with a phone change from the other can preserve useful independent edits. It does not follow that different-field edits can never conflict. A country and postal address, currency and amount, or order state and shipment state may need to satisfy a shared constraint.
Define a single owner for a field where possible. Where both sides edit, describe the resolution rule and the conditions that require review. The outcome should be reproducible from the captured evidence. A last-write-wins rule may be acceptable for one field and inappropriate for another; timestamps alone do not express business authority.
Product settings vary. For example, HubSpot documents its Salesforce field mapping rules. A design document should name the supported configuration instead of assuming that any integration can implement arbitrary per-field merge logic.
Change capture and initial load form one boundary
Choose the capture mechanism from the source’s actual capabilities: webhooks, log-based change data capture, application triggers, or polling. Each requires its own treatment of missing events, permissions, deletes, and schema changes. The existence of a webhook does not prove that it covers every relevant object mutation.
The Debezium PostgreSQL connector documentation describes snapshots and change streaming for a concrete CDC implementation. Stacksync’s Salesforce guide documents different capture modes. These mechanisms detect changes; a two-way application integration still needs matching, destination writes, ownership, and recovery.
Test initial loading while the source continues changing. Record the selection filter, the captured boundary or documented handoff, and the final identities and values. Equal total counts are insufficient: one missing record and one duplicate can cancel out numerically.
Separate delivery uncertainty from rejected input
If the destination may already have committed, establish the outcome before repeating a non-idempotent action. A stable request identifier helps only when the receiving contract recognizes it. AWS’s retry guidance explains how operation identity can make repeated calls safe; it is not a promise that every CRM or ERP endpoint supports the same mechanism.
Bound retries, add increasing delays with jitter, and respect destination capacity. Retrying an unchanged invalid enum or missing required relationship consumes quota without repairing the record. Give unresolved records a visible status and owner, and let unrelated healthy work proceed where the design permits.
In Stacksync, Retry to sync reads the latest source values. That is a current-state repair operation, distinct from replaying an original event. A workflow that needs every intermediate transition must verify retained event history separately. Use the recovery deep dive and runbook to rehearse the distinction.
Use a failure-oriented acceptance suite
- 01Repeat a create after an intentionally lost acknowledgment; verify the record and any triggered business actions.
- 02Change the same field on both sides before capture; compare the result with the documented owner.
- 03Change different fields whose combination must satisfy a business rule.
- 04Update and create records during initial backfill, then reconcile by stable ID.
- 05Remove a relationship without deleting either endpoint.
- 06Send a missing parent and an invalid enum; verify an actionable failure.
- 07Recover from a destination outage while new changes continue arriving.
- 08Change a test mapping and confirm how pending work uses the old or new configuration.
Capture the input, expected result, observed state, timestamps, and responder action. Keep the suite small enough to inspect and broad enough to cover the business risks. The field mapping workbook provides a place to define those expectations; the release checklist records their evidence.
Compare operating cost with the same scope
Evaluate native connectors, managed platforms, and custom code against the same objects and acceptance suite. Include infrastructure, API consumption, engineering maintenance, incident response, credential rotation, and schema changes. Use measured workload and an explicit cost model instead of assuming that a particular topology is cheaper.
The integration evaluation worksheet makes those comparisons reviewable. If considering Stacksync, request an engineering integration review with the competing-edit and recovery cases that matter to your application. A working demonstration should establish the promised behavior for your selected configuration.
FAQ
Frequently asked questions
Related articles
