Day 15: Asynchronous Iterators & Generators in Node.js 🚀

When working with streams of data, APIs, or long-running I/O tasks, we often need a way to process items one by one without blocking the event loop. That’s where asynchronous iterators and generators in Node.js shine.

They give us a clean, “for-await-of” style way to consume async data, much like how we use regular iterators (for-of) with arrays.

---

🔹 Why Async Iterators?

Normally, JavaScript iterators work synchronously:

const arr = [1, 2, 3];
for (const val of arr) {
  console.log(val);
}

But what if fetching each value involves an async operation (like reading from a file, streaming from a network, or paginating an API)?

That’s where async iterators come in:

• They return Promises instead of values directly.

• You consume them with for-await-of.

---

🔹 Asynchronous Iterators – Basics

const asyncIterable = {
  async *[Symbol.asyncIterator]() {
    yield "Hello";
    await new Promise(res => setTimeout(res, 1000));
    yield "World";
  }
};

(async () => {
  for await (const val of asyncIterable) {
    console.log(val);
  }
})();

Output:

Hello
(World after 1 second delay)

Here’s what’s happening:

• async * defines an asynchronous generator.

• Each yield can wait on async logic.

• for-await-of ensures values resolve before continuing.

---

🔹 Real Use Case 1: Streaming a File Line by Line

Instead of loading the entire file into memory, you can read it line by line:

const fs = require("fs");
const readline = require("readline");

async function* readLines(filePath) {
  const rl = readline.createInterface({
    input: fs.createReadStream(filePath),
    crlfDelay: Infinity
  });

  for await (const line of rl) {
    yield line;
  }
}

(async () => {
  for await (const line of readLines("data.txt")) {
    console.log("Line:", line);
  }
})();

Great for large logs, CSVs, and streaming analytics.

---

🔹 Real Use Case 2: Async Pagination (APIs)

Many APIs return data in pages. Async generators can abstract pagination:

async function* fetchPaginatedData() {
  let page = 1;
  while (true) {
    const res = await fetch(`https://api.example.com/items?page=${page}`);
    const data = await res.json();

    if (data.items.length === 0) break;

    yield* data.items; // yield each item in the array
    page++;
  }
}

(async () => {
  for await (const item of fetchPaginatedData()) {
    console.log(item);
  }
})();

👉 Instead of manually handling page++, consumers just iterate until exhausted.

---

🔹 Async Generators vs. Sync Generators

---

🔹 Real Use Case 3: Node.js Streams → Async Iterators

In Node.js v10+, streams are async iterable:

const fs = require("fs");

(async () => {
  const stream = fs.createReadStream("data.txt", { encoding: "utf8" });

  for await (const chunk of stream) {
    console.log("Chunk:", chunk);
  }
})();

👉 Cleaner alternative to .on('data') and .on('end').

---

🔹 Async Iteration Flow (Diagram)

Here’s how async iteration works conceptually:

Think of it as a pipeline:

Producer → Async Iterator → Consumer.

🔹 When to Use

• ✅ Handling streams (files, sockets, HTTP requests).

• ✅ Processing APIs with pagination.

• ✅ Consuming large datasets without loading into memory.

• ✅ Writing producer-consumer pipelines.

---

🔹 Key Takeaways

1. for-await-of is the async cousin of for-of.

2. async function* creates an async generator that can yield promises.

3. Node.js streams can be consumed directly as async iterators.

4. Perfect for I/O-heavy, streaming, and paginated workloads.

Will meet with another interesting concept tomorrow!!