advanced 8 min read

Async Task Scheduler with Max Concurrency & Rate Limiting

An Async Task Scheduler throttles and queues asynchronous operations (Promises) to run at most `N` parallel tasks at any given moment, preventing network socket exhaustion, backend rate-limit bans (429), and CPU saturation.

Why it matters in interviews

Asked verbatim in Tekion Corp Round 1 ('Implement an async scheduler with max concurrency'), Amazon, ByteDance, and Uber. Interviewers evaluate your understanding of Promise chaining, task queues, FIFO execution, and error isolation.

Visual & Interactive Explanation

Interactive Flow Execution

add(Task 1)

activeCount = 1 (< 2) -> Runs immediately

add(Task 2)

activeCount = 2 (== 2) -> Runs immediately

add(Task 3)

activeCount == 2 -> Queued in FIFO buffer

Task 1 Finishes

activeCount = 1 -> Dequeues & runs Task 3

All Tasks Settle

activeCount = 0 -> Queue empty

Code Examples & Implementation

Clean Async Scheduler Class (Production Standard)

export class TaskScheduler {
  private maxConcurrency: number;
  private activeCount: number = 0;
  private queue: Array<() => void> = [];

  constructor(maxConcurrency: number = 2) {
    this.maxConcurrency = maxConcurrency;
  }

  add<T>(task: () => Promise<T>): Promise<T> {
    return new Promise<T>((resolve, reject) => {
      const execute = () => {
        this.activeCount++;

        task()
          .then(resolve)
          .catch(reject)
          .finally(() => {
            this.activeCount--;
            this.runNext();
          });
      };

      if (this.activeCount < this.maxConcurrency) {
        execute();
      } else {
        this.queue.push(execute);
      }
    });
  }

  private runNext() {
    if (this.queue.length > 0 && this.activeCount < this.maxConcurrency) {
      const nextTask = this.queue.shift();
      nextTask?.();
    }
  }

  get pendingCount() {
    return this.queue.length;
  }
}

Real-World Usage: Bulk Image Upload with Limit = 3

const scheduler = new TaskScheduler(3); // Max 3 parallel uploads

const files = [file1, file2, file3, file4, file5, file6];

const uploadPromises = files.map((file, idx) =>
  scheduler.add(async () => {
    console.log(`[Start] Uploading file #${idx + 1}`);
    const res = await uploadFileToS3(file);
    console.log(`[Done] Uploaded file #${idx + 1}`);
    return res;
  })
);

// All 6 files processed smoothly with max 3 simultaneous connections
const results = await Promise.all(uploadPromises);

Interview-Ready Answers

Maintain a queue of task factories and an `activeCount`. When `add(task)` is called, push it to the queue; if `activeCount < limit`, run the next task immediately. When a task settles (resolves or rejects), decrement `activeCount` and trigger the next queued item.

Follow-up Questions & Deep Dives

🎯 Code Output — What Does This Print?

Question 1hard

const scheduler = new TaskScheduler(2);
const log = [];

const delay = (ms, val) => () =>
  new Promise((r) => setTimeout(() => { log.push(val); r(val); }, ms));

scheduler.add(delay(100, "A"));
scheduler.add(delay(50, "B"));
scheduler.add(delay(30, "C"));

// In what order do items finish logging?

Common Mistakes & Anti-Patterns

Accepting instantiated `Promise` objects instead of factory functions, making concurrency control useless
Putting `runNext()` only inside `.then()`, causing the entire queue to freeze permanently if any task rejects
Using rigid chunked `Promise.all` arrays that stall on the slowest task in the chunk

Implementation Evolution: Anti-Pattern to Production

❌ Anti-Pattern: Batch Chunking (Stalls on Slowest Task)

// ❌ Suboptimal Pattern: Array chunking with Promise.all
// If task 1 takes 10s and tasks 2-3 take 100ms, slots 2-3 remain idle for 9.9 seconds!
async function batchChunking<T>(tasks: Array<() => Promise<T>>, chunkSize = 3) {
  const results: T[] = [];
  for (let i = 0; i < tasks.length; i += chunkSize) {
    const chunk = tasks.slice(i, i + chunkSize);
    const chunkResults = await Promise.all(chunk.map((t) => t())); // Blocks on slowest!
    results.push(...chunkResults);
  }
  return results;
}

Why this breaks: Chunked `Promise.all` waits for the slowest task in the current batch before starting the next batch. If one task takes 5 seconds and others take 100ms, concurrency drops to 1, wasting bandwidth.

✓

✅ Correct Pattern: Continuous Sliding Pipeline

// ✅ Correct: Continuous worker pool where finished workers immediately pick up new tasks
export async function mapConcurrent<T, R>(
  items: T[],
  limit: number,
  fn: (item: T) => Promise<R>
): Promise<R[]> {
  const results: R[] = new Array(items.length);
  let nextIndex = 0;

  async function worker() {
    while (nextIndex < items.length) {
      const currentIndex = nextIndex++;
      results[currentIndex] = await fn(items[currentIndex]);
    }
  }

  // Spawn exactly 'limit' parallel workers
  const workers = Array.from({ length: Math.min(limit, items.length) }, () => worker());
  await Promise.all(workers);

  return results;
}

🚀

🚀 Production-Grade (Hardened): Production Task Scheduler with Priorities & Timeout Support

// 🚀 Production: Priority Queue (high/medium/low), timeout deadlines, and pause/resume
type Priority = "high" | "normal" | "low";

interface ScheduledItem<T> {
  task: () => Promise<T>;
  priority: Priority;
  resolve: (value: T) => void;
  reject: (err: any) => void;
}

export class ProductionScheduler {
  private activeCount = 0;
  private queue: ScheduledItem<any>[] = [];
  private isPaused = false;

  constructor(public maxConcurrency: number = 3) {}

  add<T>(task: () => Promise<T>, priority: Priority = "normal"): Promise<T> {
    return new Promise<T>((resolve, reject) => {
      const item: ScheduledItem<T> = { task, priority, resolve, reject };

      if (priority === "high") {
        this.queue.unshift(item);
      } else {
        this.queue.push(item);
      }

      this.processNext();
    });
  }

  pause() {
    this.isPaused = true;
  }

  resume() {
    this.isPaused = false;
    this.processNext();
  }

  private processNext() {
    if (this.isPaused) return;

    while (this.activeCount < this.maxConcurrency && this.queue.length > 0) {
      const item = this.queue.shift()!;
      this.activeCount++;

      item
        .task()
        .then(item.resolve)
        .catch(item.reject)
        .finally(() => {
          this.activeCount--;
          this.processNext();
        });
    }
  }
}

⚠️ Trick Questions & Interviewer Traps

Q1:Why should `task` be passed as a factory function `() => Promise<T>` instead of an already instantiated Promise `Promise<T>`?

Q2:What happens if one task in the queue throws an uncaught error?

📋 Rapid Revision Cheat Sheet

Task Factory: Always accept `() => Promise<T>` so execution is deferred until scheduled.
Active Counter: Track `activeCount` against `maxConcurrency`.
Pipeline > Chunking: Use continuous worker loop instead of batch `Promise.all` to avoid idle slot starvation.
Error Resilience: Always call `this.runNext()` inside `.finally()` so failures never freeze the queue.
Memory Care: Clean up references in FIFO queue to prevent memory retainers on settled items.

Real-World Architectural Scenario

A photo editor allows users to export 200 high-resolution canvas filters at once. Calling all 200 parallel web worker operations freezes the UI and triggers browser crash. How do you engineer the export pipeline?

Rate Your Readiness

Rate how comfortably you can explain this in an interview to update your global readiness gauge:

Rate your confidence: