Scaled Scheduler in Go

This scheduler implements the protocol used in scaled, making it a drop-in replacement for scaled's Python scheduler.

⚠️ The scaled scheduler protocol is subject to change so there may be compatibility issues.

Building

Go 1.19+ is supported.

go build

Usage

To get the list of command line options:

scaled-scheduler-go --help

If you are ok with using default options, you can simply run the scheduler as follows:

scaled-scheduler-go tcp://0.0.0.0:8786

Todo

• Create an interface for task allocators so multiple implementations can be used interchangeably.
  • Current task allocator uses a priority queue (heap) with each allocation costing O(log n).
• Implement a memory limiter and spill in-memory tasks to a disk-backed cache.
  • An embedded database like BadgerDB could be used for this.