icon peppermint

Peppermint AI

Distributed Scheduler - API design

How do we make the API idenmpotent so that jobs are executed only once? Let's dive deeper to understand how to design an idempotent API

Practice NowTranscript

Summary

Distributed job scheduler APIs require careful design around three fundamental concepts: resource hierarchy, idempotency, and temporal scheduling. The job represents the scheduling definition while runs represent individual executions, creating a parent-child relationship that affects API endpoint design.

Idempotency mechanisms prevent duplicate job creation when clients retry requests due to network failures - critical in distributed environments where partial failures are common. This typically involves idempotency keys that allow servers to recognize and handle duplicate submissions gracefully.

Temporal scheduling requires the API to distinguish between one-time jobs (executed at specific timestamp) and recurring jobs (following cron patterns). The API design must accommodate both scheduling types while maintaining clear request structure and avoiding ambiguity in job definitions.

Designed APIs

  api_name: /v1/jobs,
  {
  method_name POST: 
  {
      description:  Submitting new jobs.,
      body:
      {
      // only one of schedule_recurring and schedule_oneoff can be present
      schedule_recurring:"cron syntax"
      schedule_oneoff: "timestamp"
      job_url: "https://MYJOB_EXECUTION_URL"
      job_payload: {
          // forced to POST
          additional_params
      }
      }
  },
  response: 201,400,403,401
  authentication: OAuth or IAM
  }
    api_name: /v1/jobs/{jobId}/run/{runId}/,
    {
    method_name DELETE: 
    {
        description:  Delete or cancel job
    },
    response_body: {
        // to maintain history it is always a soft delete.
        "status": "CANCELED|DELETE"
    }
    response: 201,400,403,401
    authentication: OAuth or IAM
    }

Continue Reading. Push past the basics.

Login to access the interviewer insights and advanced concepts with our AI learning companion

♡ Drafted by PAI, verified by engineers