Iterators in JavaScript

In this article, we will look at the mechanism of iterators. What they are, how to apply them and how to create your own.

Basic terms

To understand the iteration mechanism, let's take a look at the following interfaces.

interface Iterable {
  [Symbol.iterator]: () => Iterator;
}

interface Iterator {
  next: () => IteratorResult;
  
  return?: (value?: any) => IteratorResult;
  
  throw?: (exception?: any) => IteratorResult;
}

interface IteratorResult {
  done?: boolean;
  value?: any;
}

So, an object is considered iterable if it implements the Iterable interface. In other words, if it contains the [Symbol.iterator] method - a function that directly returns an iterator object.

An iterator is a ordinary object that contains a next() method. The other two methods in the Iterator interface are not mandatory. The next() method is required to return an IteratorResult object.

The essence of iterability is that such an object can be traversed with for..of and for..in loops.

for (const item of [1, 2, 3]) {
    console.log(item);
}
// 1
// 2
// 3

Built-in iterators

The iteration mechanism is one of the most widely used in JavaScript. The language is literally riddled with embedded iterable objects. These include: Array, TypedArray, Map, Set, WeakMap, WeakSet, String.

// Array
for (const item of new Array(1, 2, 3)) {
    console.log(item);
}
// 1
// 2
// 3

// TypedArray
for (const item of new Int8Array(3)) {
    console.log(item);
}
// 0
// 0
// 0

// Map
for (const item of new Map([[1, 'one'], [2, 'two'], [3, 'three']])) {
    console.log(item);
}
// [1, 'one']
// [2, 'two']
// [3, 'three']

// Set
for (const item of new Set([1, 2, 3])) {
    console.log(item);
}
// 1
// 2
// 3

// String
for (const item of "abc") {
    console.log(item);
}
// a
// b
// c

In addition to JavaScript, some Web API objects also implement the Iterable interface. For example, in the DOM standard, examples of such objects are NodeList and DOMTokenList.

// NodeList
for (const item of document.querySelectorAll("div")) {
    console.log(item);
}
// Node
// Node
// ...

// DOMTokenList
const block = document.createElement("div");
block.classList.add("classA", "classB")

for (const item of block.classList) {
    console.log(item)
}
// classA
// classB

Custom iterators

Other than built-in iterable objects, it is possible to create our own. To do this, it is sufficient to simply implement the Iterable interface, following the iteration protocol.

So, as mentioned earlier, an object is considered iterable if its prototype contains the method [Symbol.iterator](). Otherwise, if an attempt is made to iterate over such an object, an exception will be thrown.

const obj = {};

for (const item of obj) {
    console.log(item);
}

// Uncaught TypeError: obj is not iterable

Now let's add an iterator method to our object.

const obj = {
    [Symbol.iterator]: () => {},
};

for (const item of obj) {
    console.log(item);
}

// Uncaught TypeError: Result of the Symbol.iterator method is not an object

Now, formally, the object is considered iterable, but it still cannot be iterated over in its current state, as the iterator method does not return an object that implements the Iterator interface, as required by the protocol.

next()

According to the protocol, an iterator should be a simple object that contains at least one method, next(). The next() method may take arguments, but it's not strictly required. The ECMA-262 specification guarantees that all built-in consumers of iterable objects call this method without arguments. However, in rare cases, arguments may be useful when manually calling the iterator.

const obj = {
    [Symbol.iterator]: () => {
        return {
            next: () => {},
        };
    },
};

for (const item of obj) {
    console.log(item);
}

// Uncaught TypeError: Iterator result undefined is not an object

In the above example, we added the next() method, but this method still doesn't return the correct IteratorResult.

An IteratorResult is also a simple object with two properties, done and value.

The done property is a boolean and expresses the completion of the iteration process. In other words, as long as the object can be iterated further, its iterator returns done: false; once it reaches the last step of iteration, the iterator must return done: true, signaling the user of the iterable object (e.g., a loop) to end the process (exit the loop).

The value property is the actual value associated with a given step of iteration. There are no restrictions on the format of the returned value.

As I just mentioned, if the iterator returns done: true, the iteration will be considered complete, the next iteration will not occur, and therefore, there is no point in the value. Consequently, passing it together with done: true is not necessary in this case.

It is worth noting that the ECMA-262 specification has taken a very liberal approach to the issue of iterators. In fact, it is not mandatory to return either the done property or the value property. Default values are provided for both (false and undefined, respectively). In other words, in order for our iterable object to finally work, the iterator only needs to return an empty object.

const obj = {
    [Symbol.iterator]: () => {
        return {
            next: () => {
                return {}; // { done: false, value: undefined }
            },
        };
    },
};

for (const item of obj) {
    console.log(item);
}

// Caution!!! This code leads to an infinite loop

However, there is little benefit from such an iterator. Moreover, since the iterator will never return done: true, the entire code will go into an infinite loop.

So, let's describe a functional iterator according to all the rules.

const obj = {
    [Symbol.iterator]: () => {
        let i = 0;
        
        return {
            next: () => {
                return {
                    done: i >= 3,
                    value: i++,
                };
            },
        };
    },
};

for (const item of obj) {
    console.log(item);
}

// 0
// 1
// 2

In this example, we sequentially return the numbers 0, 1, 2. Exiting the loop is done when i === 3.

return()

In some cases, the iteration process can be forcibly stopped in one way or another from the outside. In this case, the user of the iterable object must call the return() iterator method, if it exists.

const obj = {
    [Symbol.iterator]: () => {
        let i = 0;
        
        return {
            next: () => {
                return {
                    done: i >= 3,
                    value: i++,
                };
            },
            
            return: () => {
                console.log("Return", i);
                return {
                    done: i >= 3,
                    value: i,
                }
            }
        };
    },
};

for (const item of obj) {
    console.log(item);
    
    if (item === 1) {
        break;
    }
}
// 0
// 1
// Return 2

Like the next() method, the return() method must return an IteratorResult object. This method is, in a way, a callback at the time of forced interruption of the iteration. It is assumed that specific operations necessary to complete the process can be performed here. For example, unbinding listeners or clearing memory.

throw()

This is another callback method similar to return(). Its essence is to allow the user of the iterable object to report a discovered error. There are no built-in JavaScript users that can invoke this method in any situation. However, this callback can be manually invoked.

The protocol allows passing an argument to the throw() method, which is suggested to be used as a reference to the exception, although formally there are no restrictions on the number or types of arguments.

When called, the throw() method should throw the passed exception to halt the further iteration process. Again, such behavior is recommended and anticipated, but not mandatory. The iterator's author has the right to define the further logic.

const obj = {
  [Symbol.iterator]: () => {
    let i = 0;
    
    return {
      next: () => {
        return {
          done: i >= 3,
          value: i++,
        };
      },
      
      throw: (exception) => {
        console.log("Thrown:", exception);
        
        if (exception) {
          throw exception;
        }
        
        return {
          done: true,
        };
      },
    };
  },
};

for (const item of obj) {
  console.log(item);
  
  if (item === 1) {
    obj[Symbol.iterator]().throw(`Reached ${item}`);
  }
}

// 0
// 1
// Thrown: Reached 1
// Uncaught Reached 1

Most often, the throw() method is used in asynchronous iterators in conjunction with generators. More on this later.

Performing the iterator manually

As it has already become clear, the main users of iterable objects are the constructs for..of and for..in. These built-in users independently implement the iteration protocol. However, sometimes there is a need to create your own user, or simply to manually iterate through the process. As we saw in the last example, the iterator can be accessed directly to iterate through the entire loop on its own.

const obj = {
  [Symbol.iterator]: () => {
    let i = 0;
    
    return {
      next: () => {
        return {
          done: i >= 5,
          value: i++,
        };
      },
      
      return: () => {
        return {
          done: true,
          value: i,
        };
      },
      
      throw: (exception) => {
        if (exception) {
          throw exception;
        }
        
        return {
          done: true,
        };
      },
    };
  },
};

// getting the iterator by calling the iterator method
const iterator = obj[Symbol.iterator]();

// first step of the iteration
let result = iterator.next();
console.log(result.value);

// the remaining steps will be called in a loop until
// the iterator returns `done: true`
while (!result.done) {
  const result = iterator.next();
  console.log(result.value);
  
  if (isSuccess(result.value)) {
    // if necessary, we can stop the iteration process
    iterator.return();
    break;
  }
  
  if (isFailed(result.value)) {
    // or throw an exception
    iterator.throw(new Error("failed"));
  }
}

Self-iterating objects

We have already acquainted ourselves with two concepts: iterable object (Iterable) and iterator (Iterator). So far, we have considered both entities separately. The following technique allows us to combine everything into a single self-iterating object.

const obj = {
  i: 0,
  
  [Symbol.iterator]() {
    return this;
  },
  
  next() {
    return {
      done: this.i >= 3,
      value: this.i++,
    };
  },
  
  return() {
    return {
      done: true,
      value: this.i,
    };
  },
  
  throw(exception) {
    if (exception) {
      throw exception;
    }
    
    return {
      done: true,
    };
  },
};

for (const item of obj) {
  console.log(item);
}

In this example, in the iterator method, we returned a reference to the iterable object as the iterator, so obj simultaneously serves as both the iterator and the iterable object.

A similar pattern might be seen in an iterable class.

class Obj {
  i = 0;
  
  [Symbol.iterator]() {
    return this;
  }
  
  next() {
    return {
      done: this.i >= 3,
      value: this.i++,
    };
  }
  
  return() {
    return {
      done: true,
      value: this.i,
    };
  }
  
  throw(exception) {
    if (exception) {
      throw exception;
    }
    
    return {
      done: true,
    };
  }
}

const obj = new Obj();

for (const item of obj) {
  console.log(item);
}

Asynchronous iterators

So far we've only been talking about synchronous iterators. In June 2018, the 9th edition of the ECMA-262 specification was published. This version introduced asynchronous iterators and the AsyncIterator protocol.

In essence, an asynchronous iterator differs little from a synchronous one. Let's take a look at the following interfaces.

interface AsyncIterable {
  [Symbol.asyncIterator]: () => AsyncIterator;
}

interface AsyncIterator {
  next: () => Promise<IteratorResult>;
  
  return?: (value?: any) => Promise<IteratorResult>;
  
  throw?: (exception?: any) => Promise<IteratorResult>;
}

So, the main difference between synchronous and asynchronous iterable objects is that an asynchronous object must have an iterator method [Symbol.asyncIterator] (instead of [Symbol.iterator] as in the case of a synchronous object). As for the return values of the next(), return(), and throw() methods, they are expected to be Promises.

Asynchronous iterators themselves wouldn't be as convenient to use if the specification didn't provide for a new structure - the asynchronous for..await..of loop.

const obj = {
  [Symbol.asyncIterator]: () => {
    let i = 0;
    
    return {
      next: () => {
        return Promise.resolve({
          done: i >= 3,
          value: i++,
        });
      },
      
      return: () => {
        return Promise.resolve({
          done: true,
          value: i,
        });
      },
      
      throw: (exception) => {
        if (exception) {
          return Promise.reject(exception);
        }
        
        return Promise.resolve({
          done: true,
        });
      },
    };
  },
};

(async () => {
  for await (const item of obj) {
    console.log(item);
    
    if (item === 1) {
      await obj[Symbol.asyncIterator]().throw(`Reached ${1}`);
    }
  }
})();

Asynchronous generators

One important addition in the 9th edition of ECMA-262 is the introduction of asynchronous generators closely related to asynchronous iterators.

A simple asynchronous generator may look as follows.

async function* gen() {
  yield 0;
  yield 1;
  yield 2;
}

(async () => {
  for await (const item of gen()) {
    console.log(item);
  }
})();

// 0
// 1
// 2

An asynchronous generator implements the AsyncIterator protocol, so we can use all available iterator methods manually.

async function* gen() {
  yield 0;
  yield 1;
  yield 2;
}

const iterator = gen();

(async () => {
  console.log(await iterator.next());
  console.log(await iterator.next());
  console.log(await iterator.next());
  console.log(await iterator.next());
})();

// {value: 0, done: false}
// {value: 1, done: false}
// {value: 2, done: false}
// {value: undefined, done: true}

This includes pausing the iteration process.

async function* gen() {
  yield 0;
  yield 1;
  yield 2;
}

const iterator = gen();

(async () => {
  console.log(await iterator.next());
  console.log(await iterator.return("My return reason"));
})();

// {value: 0, done: false}
// {value: 'My return reason', done: true}

Or throwing an exception

async function* gen() {
  yield 0;
  yield 1;
  yield 2;
}

const iterator = gen();

(async () => {
  console.log(await iterator.next());
  console.log(await iterator.throw("My error"));
})();

// {value: 0, done: false}
// Uncaught (in promise) My error

Async-from-Sync iterator

An Async-from-Sync iterator is an asynchronous iterator obtained from a synchronous one through the abstract operation CreateAsyncFromSyncIterator. I remind you that an abstract operation is an internal mechanism of the JavaScript language. Normally, such operations are not available within the execution context. However, for example, the V8 engine allows the use of these operations when the --allow-natives-syntax flag is enabled.

> v8-debug --allow-natives-syntax iterator.js

// iterator.js
const syncIterator = {
  [Symbol.iterator]() {
    return this;
  },
  
  next() {
    return {
      done: true,
      value: Promise.resolve(1),
    };
  },
};

const asyncIterator = %CreateAsyncFromSyncIterator(syncIterator);

console.log(asyncIterator);
// [object Async-from-Sync Iterator]

console.log(asyncIterator[Symbol.asyncIterator]);
// function [Symbol.asyncIterator]() { [native code] }

As we can see, this abstract operation converts a synchronous iterator to an asynchronous one. As I mentioned before, we cannot call an abstract operation in the normal execution context. The engine itself handles this where necessary. According to the specification, this is required in one specific case:

The current environment is asynchronous.
The iterable object does not have a [Symbol.asyncIterator] method.
The iterable object has a [Symbol.iterator] method.

Как мы можем видеть, эта абстрактная операция преобразует синхронный итератор в асинхронный. Как я уже говорил, в обычном исполняемом контексте мы не можем вызвать абстрактную операцию. Это делает сам движок в тех местах, где требуется. А требуется это, согласно спецификации, в одном конкретном случае:

Текущее окружение является асинхронным
Итерируемый объект не имеет метода [Symbol.asyncIterarot]
Итерируемый объект имеет метод [Symbol.iterator]

The easiest way to demonstrate this process is with generators.

function* syncGen() {
  yield 1;
  yield 2;
  yield Promise.resolve(3);
  yield Promise.resolve(4);
  yield 5;
}

for (const item of syncGen()) {
  console.log(item);
}

// 1
// 2
// Promise
// Promise
// 5

In this case, a synchronous generator will return unresolved promises at steps 3 and 4. However, an asynchronous user for..await..of can convert these steps into asynchronous ones.

function* syncGen() {
  yield 1;
  yield 2;
  yield Promise.resolve(3);
  yield Promise.resolve(4);
  yield 5;
}

(async () => {
  for await (const item of syncGen()) {
    console.log(item);
  }
})();

// 1
// 2
// 3
// 4
// 5

The same applies to regular synchronous iterators.

const syncIterator = {
  i: 0,
  
  [Symbol.iterator]() {
    return this;
  },
  
  next() {
    const done = this.i >= 3;

    if (this.i === 1) {
      return {
        done: false,
        value: Promise.resolve(this.i++),
      };
    }

    return {
      done,
      value: this.i++,
    };
  },
};

for (const item of syncIterator) {
  console.log(item);
}

// 0
// Promise
// 2

However, when an asynchronous user is applied, the second step will be transformed into an asynchronous one.

const syncIterator = {
  i: 0,
  
  [Symbol.iterator]() {
    return this;
  },
  
  next() {
    const done = this.i >= 3;

    if (this.i === 1) {
      return {
        done: false,
        value: Promise.resolve(this.i++),
      };
    }

    return {
      done,
      value: this.i++,
    };
  },
};

(async () => {
  for await (const item of syncIterator) {
    console.log(item);
  }
})();

// 0
// 1
// 2

My telegram channels:

EN - https://t.me/frontend_almanac
RU - https://t.me/frontend_almanac_ru

Русская версия: https://blog.frontend-almanac.ru/iterators