Purpose: If a service is responsible for allowing only a single transaction for a given identity through at once, but should allow concurrent processing of transactions for diverse entities, the arbiter can improve throughput, and potentially reduce resource utilization and transaction latency.

If a service can often receive delayed, out-of-date, or defunct messages that should not be processed, or potentially receive redundant messages that will impact overall throughput, the arbiter pattern can mitigate those requests before they impact performance.

Primarily appropriate for services where the work done per request is significant relative to the gRPC management. Please see the examples directory for benchmarks that demonstrate the relative performance versus a naive mutex based concurrency management approach.

Design Points

Uses two stages of request tracking, the Processing List and the Waiting List. For any given identifier (key), a single in-flight request will be tracked on the Processing list. If another request for that ID comes in while the first is still processing, assuming it is valid, it will be placed on the waiting list. By design, if the Processing list is empty (for an ID), the Waiting list will be empty (Waitling list entries are immediately promoted to Processing list once the prior in-flight request has completed). Currently the Waiting list depth is only one entry per ID (future enhancement to support configurable Waiting list per ID depth).

Single unified channel for begin/end messages eliminates potential race conditions and ensures deterministic processing of all incoming requests. This allows all Processing/Waiting list management to be done atomically before handling subsequent incoming requests.

Currently supports idempotent requests (where each request contains all needed data). Enhancement to support queue depth for same identifier in the works (branch: )

Telemetry interface allows plumbing with adapter to project specific monitoring framework, or using including Prometheus based monitoring subpackage. Similarly logging interface allows plumbing of existing service logging into arbiter.

When the package consumer specifies the interface functions to implement the Arbiter request interface, error formats and attached metadata are the package consumer's choice. This allows them to flow through the supervisor, so all errors generated by the Arbiter supervisor can be matched to the desired error specification (the supervisor does not generate its own errors, other than panic inducing configuration errors, which should be apparent with even cursory testing).

Arbitrary gated functions:

• decide to proceed, or drop

FUTURE:

• Go generics for message key (go 1.18)
• GoDocs
• depth of command queue per key (idempotent = 1)
• rate limit implementation based on configurable weighting
• TLA+ Model of operations (including queuing and rate limits)
• Graceful Supervisor shutdown