Engineering

As part of this post, we’ll be covering the design of a modern, production-grade Order Management System (OMS) with a focus on multi-fulfillment, cancellations, refunds, inventory synchronization, and multi-region deployment. Let’s first start with the requirements. Requirements # Functional Requirements # Core order lifecycle: Create order with multiple line items, shipping options, and payment methods. Order state machine: Support states such as PENDING → CONFIRMED → PARTIALLY_FULFILLED → FULFILLED → CANCELLED → REFUNDED. Split shipments: Support split shipments and partial fulfillment when items originate from multiple locations or arrive at different times. Cancellations: Allow customer and system-initiated cancellations in various states (pre-fulfillment, mid-fulfillment) with clear rules. Refunds: Support refunds (full and partial), including multi-payment or mixed-method scenarios (card, wallet, store credit). Multi-fulfillment: Route each line item to an optimal fulfillment node (warehouse, store, 3PL, marketplace drop-shipper). Multiple shipments: Track multiple shipments per order with independent tracking IDs and statuses. Backorders and preorders: Support delayed fulfillment while the order remains active. Inventory and payments: Reserve inventory atomically as part of the order creation saga; release on failure or cancellation. Inventory sync: Prevent overselling across channels with near real-time inventory sync and event-driven updates. Payment gateways: Integrate with one or more payment gateways for authorization, capture, and refund. Multi-channel and integrations: Receive orders from internal checkout, marketplaces, and POS; normalize into a canonical order model. Fulfillment updates: Push fulfillment updates and cancellations back to channels and customer notification systems. Multi-region deployment: Deploy OMS in multiple regions, each with a full stack of services fronted by a global load balancer. Data synchronization: Keep critical data (orders, payments, inventory) synchronized across regions using a mix of strong and eventual consistency depending on domain constraints. Non-Functional Requirements # High Availability and resilience: One failure in a downstream flow should not take down the entire order flow. Scalability: Capable of handling peak events such as flash sales and promotions. Consistency: Clear consistency model for orders and inventory (strong vs eventual consistency). Observability: Comprehensive logging, monitoring, and tracing. Extensibility: Easy to add new fulfillment types, payment methods, or regions without major rewrites. High Level Design # Order Lifecycle and Domain Model # Order Lifecycle Stages # A typical e-commerce order lifecycle contains the following high-level stages:

As part of this post, we’ll be covering the design of a mobile wallet payment system that supports - Top-ups (add money to wallet from bank/card) P2P transfers (wallet -> wallet) Basic fraud detection Concurrency with clear trade-offs between strong and eventual consistency at scale. Let’s start with a basic design and then we can scale it up.

In Part 1, Part 2, and Part 3, we covered hazards, primitives, and execution models. In this final part, we will focus on structured concurrency, fan-out/fan-in, fail-fast cancellations, timeout propagation, resource scoping, and observability with thread dumps and Java Flight Recorder (JFR), emphasizing when to choose which pattern.

In Part 1 and Part 2, we covered the fundamentals and building blocks of concurrency. In this part, we will discuss the execution models of concurrency - classic thread pools, task queues, Future/Callable, CompletableFuture, and, from Java 21+, virtual threads and virtual-thread-per-task executors. Choosing among them is ultimately about latency, throughput, and operational simplicity, not about syntax.

In Part 1, we discussed the core concurrency hazards and control concepts in Java: race conditions, visibility, atomicity, deadlocks, starvation, livelock, contention, backpressure, interruption, and cancellation. In this part, we will discuss the coordination primitives - synchronized, volatile, Atomics, Locks, Semaphores, Blocking Queues, and Concurrent Collections.

Introduction # Here, we will discuss the core concurrency hazards and control concepts in Java: race conditions, visibility, atomicity, deadlocks, starvation, livelock, contention, backpressure, interruption, and cancellation. Race Conditions # A race condition occurs when the correctness of a task depends on the relative timing or interleaving of threads accessing shared mutable state. The bug is not “thread A ran before B”, it is “if they interleave in this specific way, invariants break.”