On this page

稳定性:2 - 稳定

流是 Node.js 中用于处理流式数据的抽象接口。 node:stream 模块提供了实现流接口的 API。

Node.js 提供了许多流对象。例如,对 HTTP 服务器的请求process.stdout 都是流实例。

流可以是可读的、可写的,或两者皆是。所有流都是 EventEmitter 的实例。

要访问 node:stream 模块:

node:stream 模块对于创建新类型的流实例很有用。通常不需要使用 node:stream 模块来消费流。

本文档包含两个主要部分和一个备注部分。第一部分解释如何在应用程序中使用现有流。第二部分解释如何创建新类型的流。

Node.js 内有四种基本流类型:

此外,该模块还包括实用函数 stream.duplexPair()stream.pipeline()stream.finished()stream.Readable.from()stream.addAbortSignal()

const { pipeline } = require('node:stream/promises');
const fs = require('node:fs');
const zlib = require('node:zlib');

async function run() {
  await pipeline(
    fs.createReadStream('archive.tar'),
    zlib.createGzip(),
    fs.createWriteStream('archive.tar.gz'),
  );
  console.log('管道成功。');
}

run().catch(console.error);

要使用 AbortSignal,将其作为最后一个参数传递给选项对象。 当信号被中止时,底层管道将被调用 destroy,并带有 AbortError

const { pipeline } = require('node:stream/promises');
const fs = require('node:fs');
const zlib = require('node:zlib');

async function run() {
  const ac = new AbortController();
  const signal = ac.signal;

  setImmediate(() => ac.abort());
  await pipeline(
    fs.createReadStream('archive.tar'),
    zlib.createGzip(),
    fs.createWriteStream('archive.tar.gz'),
    { signal },
  );
}

run().catch(console.error); // AbortError

pipeline API 也支持异步生成器:

const { pipeline } = require('node:stream/promises');
const fs = require('node:fs');

async function run() {
  await pipeline(
    fs.createReadStream('lowercase.txt'),
    async function* (source, { signal }) {
      source.setEncoding('utf8');  // 使用字符串而不是 `Buffer`。
      for await (const chunk of source) {
        yield await processChunk(chunk, { signal });
      }
    },
    fs.createWriteStream('uppercase.txt'),
  );
  console.log('管道成功。');
}

run().catch(console.error);

记得处理传递给异步生成器的 signal 参数。 特别是在异步生成器是管道的源(即第一个参数)的情况下,否则管道将永远不会完成。

const { pipeline } = require('node:stream/promises');
const fs = require('node:fs');

async function run() {
  await pipeline(
    async function* ({ signal }) {
      await someLongRunningfn({ signal });
      yield 'asd';
    },
    fs.createWriteStream('uppercase.txt'),
  );
  console.log('管道成功。');
}

run().catch(console.error);

pipeline API 提供了 回调版本

const { finished } = require('node:stream/promises');
const fs = require('node:fs');

const rs = fs.createReadStream('archive.tar');

async function run() {
  await finished(rs);
  console.log('流已完成读取。');
}

run().catch(console.error);
rs.resume(); // 排空流。

finished API 还提供了 回调版本

stream.finished() 在返回的 promise 被履行或拒绝后,会留下悬空的事件监听器(特别是 'error''end''finish''close')。 这样做的原因是,意外的 'error' 事件(由于不正确的流实现)不会导致意外的崩溃。 如果这是不需要的行为,则应将 options.cleanup 设置为 true

  • StringsBuffers 是与流一起使用的最常见类型。
  • TypedArrayDataView 允许你使用 Int32ArrayUint8Array 等类型处理二进制数据。当你将 TypedArray 或 DataView 写入流时,Node.js 会处理原始字节。

然而,流实现有可能与其他类型的 JavaScript 值一起工作(null 除外,它在流中有特殊用途)。 此类流被认为是在“对象模式”下操作。

流实例在创建时使用 objectMode 选项切换到对象模式。尝试将现有流切换到对象模式是不安全的。

潜在缓冲的数据量取决于传递给流构造函数的 highWaterMark 选项。对于普通流,highWaterMark 选项指定 字节总数。对于以对象模式操作的流,highWaterMark 指定对象总数。对于操作字符串(但不解码)的流,highWaterMark 指定 UTF-16 代码单元总数。

当实现调用 stream.push(chunk) 时,数据会在 Readable 流中缓冲。如果流的消费者不调用 stream.read(),数据将停留在内部队列中直到被消费。

一旦内部读取缓冲区的总大小达到 highWaterMark 指定的阈值,流将暂时停止从底层资源读取数据,直到当前缓冲的数据可以被消费(即,流将停止调用用于填充读取缓冲区的内部 readable._read() 方法)。

当反复调用 writable.write(chunk) 方法时,数据会在 Writable 流中缓冲。当内部写入缓冲区的总大小低于 highWaterMark 设置的阈值时,对 writable.write() 的调用将返回 true。一旦内部缓冲区的大小达到或超过 highWaterMark,将返回 false

stream API 的一个关键目标,特别是 stream.pipe() 方法,是将数据缓冲限制在可接受的水平,以便不同速度的源和目标不会压倒可用内存。

highWaterMark 选项是一个阈值,而不是限制:它规定了流在停止请求更多数据之前缓冲的数据量。它通常不强制执行严格的内存限制。特定的流实现可以选择执行更严格的限制,但这是可选的。

因为 DuplexTransform 流既是 Readable 又是 Writable,所以它们各自维护 两个 独立的内部缓冲区用于读取和写入,允许每一侧独立操作,同时保持适当且高效的数据流。例如,net.Socket 实例是 Duplex 流,其 Readable 侧允许消费 套接字接收的数据,而其 Writable 侧允许写入数据 套接字。因为写入套接字的数据速率可能比接收数据的速率快或慢,所以每一侧都应独立操作(和缓冲)。

内部缓冲的机制是内部实现细节,可能随时更改。但是,对于某些高级实现,可以使用 writable.writableBufferreadable.readableBuffer 检索内部缓冲区。不鼓励使用这些未记录的属性。

几乎所有的 Node.js 应用程序,无论多么简单,都会以某种方式使用流。以下是在实现 HTTP 服务器的 Node.js 应用程序中使用流的示例:

const http = require('node:http');

const server = http.createServer((req, res) => {
  // `req` 是一个 http.IncomingMessage,它是一个可读流。
  // `res` 是一个 http.ServerResponse,它是一个可写流。

  let body = '';
  // 将数据获取为 utf8 字符串。
  // 如果未设置编码,将接收到 Buffer 对象。
  req.setEncoding('utf8');

  // 可读流一旦添加了监听器就会发出 'data' 事件。
  req.on('data', (chunk) => {
    body += chunk;
  });

  // 'end' 事件表示整个 body 已接收完毕。
  req.on('end', () => {
    try {
      const data = JSON.parse(body);
      // 向用户写回一些有趣的内容:
      res.write(typeof data);
      res.end();
    } catch (er) {
      //  哎呀!JSON 错误!
      res.statusCode = 400;
      return res.end(`error: ${er.message}`);
    }
  });
});

server.listen(1337);

// $ curl localhost:1337 -d "{}"
// object
// $ curl localhost:1337 -d "\"foo\""
// string
// $ curl localhost:1337 -d "not json"
// error: Unexpected token 'o', "not json" is not valid JSON

[可读流][] 流(例如示例中的 [Readable][])暴露了 [push][] 和 [unshift][] 等方法,用于将数据写入流。

[流][] 流使用 [EventEmitter][] API 在数据可供从流中读取时通知应用程序代码。可以通过多种方式从流中读取可用数据。

[可读流][] 和 [可写流][] 流都以各种方式使用 [EventEmitter][] API 来通信流的当前状态。

[双工流][] 和 [转换流][] 流既是 [可读流][] 也是 [可写流][]。

向流写入数据或从流消费数据的应用程序不需要直接实现流接口,并且通常没有理由调用 _write()。

希望实现新类型流的开发者应参考 流实现者 API 部分。

[可写流][] 流的示例包括:

其中一些示例实际上是实现了 [stream.Writable][] 接口的 [流][] 流。

所有 [Writable][] 流都实现了 [stream.Writable][] 类定义的接口。

虽然 [Writable][] 流的具体实例可能在各方面有所不同,但所有 [Writable][] 流都遵循与以下示例中说明相同的基本使用模式:

const myStream = getWritableStreamSomehow();
myStream.write('some data');
myStream.write('some more data');
myStream.end('done writing data');

当流及其任何底层资源(例如文件描述符)已关闭时,会发出 'close' 事件。该事件表示将不再发出更多事件,也不会发生进一步的计算。

如果使用 emitClose 选项创建 Writable 流,它将始终发出 'close' 事件。

如果对 stream.write(chunk) 的调用返回 false,则当适合恢复向流写入数据时,将发出 'drain' 事件。

// 向提供的可写流写入数据一百万次。
// 注意背压。
function writeOneMillionTimes(writer, data, encoding, callback) {
  let i = 1000000;
  write();
  function write() {
    let ok = true;
    do {
      i--;
      if (i === 0) {
        // 最后一次!
        writer.write(data, encoding, callback);
      } else {
        // 看看我们是应该继续,还是等待。
        // 不要传递回调,因为我们还没完成。
        ok = writer.write(data, encoding);
      }
    } while (i > 0 && ok);
    if (i > 0) {
      // 不得不提前停止!
      // 一旦排空,再写入一些。
      writer.once('drain', write);
    }
  }
}

如果在写入或通过管道传输数据时发生错误,则会发出 'error' 事件。调用监听器回调时会传递单个 Error 参数。

除非在创建流时将 autoDestroy 选项设置为 false,否则在发出 'error' 事件时流会关闭。

'error' 之后,应该 不再发出除 'close' 之外的其他事件(包括 'error' 事件)。

在调用 stream.end() 方法且所有数据已刷新到底层系统后,会发出 'finish' 事件。

const writer = getWritableStreamSomehow();
for (let i = 0; i < 100; i++) {
  writer.write(`hello, #${i}!\n`);
}
writer.on('finish', () => {
  console.log('所有写入现已完成。');
});
writer.end('This is the end\n');
Attributes
管道传输到此可写流的源流

当在可读流上调用 stream.pipe() 方法并将此可写流添加到其目的地集合时,会发出 'pipe' 事件。

const writer = getWritableStreamSomehow();
const reader = getReadableStreamSomehow();
writer.on('pipe', (src) => {
  console.log('Something is piping into the writer.');
  assert.equal(src, reader);
});
reader.pipe(writer);
Attributes
取消管道 到此可写流的源流

当在 Readable 流上调用 stream.unpipe() 方法并将此 Writable 从其目的地集合中移除时,会发出 'unpipe' 事件。

如果此 Writable 流在有 Readable 流管道传输到它时发出错误,也会发出此事件。

const writer = getWritableStreamSomehow();
const reader = getReadableStreamSomehow();
writer.on('unpipe', (src) => {
  console.log('Something has stopped piping into the writer.');
  assert.equal(src, reader);
});
reader.pipe(writer);
reader.unpipe(writer);

writable.cork() 方法强制将所有写入的数据缓冲在内存中。当调用 stream.uncork()stream.end() 方法时,缓冲的数据将被刷新。

writable.cork() 的主要目的是适应这种情况:多个小块数据连续快速地写入流。writable.cork() 不会立即将它们转发到底层目的地,而是缓冲所有块,直到调用 writable.uncork(),如果存在,这将把它们全部传递给 writable._writev()。这防止了头阻塞情况,即数据在等待第一个小块被处理时被缓冲。但是,如果不实现 writable._writev() 而使用 writable.cork() 可能会对吞吐量产生不利影响。

另见:writable.uncork()writable._writev()

Attributes
error:<Error>
可选,一个要与  'error' 事件一起发出的错误。

销毁流。可选地发出 'error' 事件,并发出 'close' 事件(除非 emitClose 设置为 false)。在此调用之后,可写流已结束,后续调用 write()end() 将导致 ERR_STREAM_DESTROYED 错误。 这是一种破坏性的且立即销毁流的方式。之前的 write() 调用可能尚未排空,并可能触发 ERR_STREAM_DESTROYED 错误。如果数据应在关闭前刷新,请使用 end() 而不是 destroy,或者在销毁流之前等待 'drain' 事件。

const { Writable } = require('node:stream');

const myStream = new Writable();

const fooErr = new Error('foo error');
myStream.destroy(fooErr);
myStream.on('error', (fooErr) => console.error(fooErr.message)); // foo error

一旦调用了 destroy(),任何进一步的调用都将是无操作,并且除了来自 _destroy() 的错误外,不会再作为 'error' 发出其他错误。

实现者不应覆盖此方法,而应实现 writable._destroy()

在发出 'close' 事件后为 true

在调用 writable.destroy() 后为 true

const { Writable } = require('node:stream');

const myStream = new Writable();

console.log(myStream.destroyed); // false
myStream.destroy();
console.log(myStream.destroyed); // true
Attributes
可选的要写入的数据。对于不在对象模式下运行的流, chunk  必须是 <string><Buffer><TypedArray><DataView> 。对于对象模式流, chunk 可以是除 null 之外的任何 JavaScript 值。
encoding:<string>
如果  chunk 是字符串,则为编码
callback:<Function>
流完成时的回调。

调用 writable.end() 方法表示不再有数据写入 Writable。可选的 chunkencoding 参数允许在关闭流之前立即写入最后一个额外的数据块。

在调用 stream.end() 后调用 stream.write() 方法将引发错误。

// 写入 'hello, ' 然后以 'world!' 结束。
const fs = require('node:fs');
const file = fs.createWriteStream('example.txt');
file.write('hello, ');
file.end('world!');
// 现在不允许再写入!
Attributes
encoding:<string>
新的默认编码

writable.setDefaultEncoding() 方法为 Writable 流设置默认 encoding

writable.uncork() 方法刷新自调用 stream.cork() 以来缓冲的所有数据。

当使用 writable.cork()writable.uncork() 管理流写入的缓冲时,请使用 process.nextTick() 延迟调用 writable.uncork()。这样做允许批处理在给定 Node.js 事件循环阶段内发生的所有 writable.write() 调用。

stream.cork();
stream.write('some ');
stream.write('data ');
process.nextTick(() => stream.uncork());

如果在流上多次调用 writable.cork() 方法,则必须调用相同次数的 writable.uncork() 来刷新缓冲的数据。

stream.cork();
stream.write('some ');
stream.cork();
stream.write('data ');
process.nextTick(() => {
  stream.uncork();
  // 直到第二次调用 uncork() 数据才会被刷新。
  stream.uncork();
});

另见:writable.cork()

如果安全调用 writable.write() 则为 true,这意味着流未被销毁、出错或结束。

返回流是否在发出 'finish' 之前被销毁或出错。

在调用 writable.end() 后为 true。此属性不指示数据是否已刷新,为此请使用 writable.writableFinished

需要调用 writable.uncork() 的次数才能完全打开流。

如果流已因错误被销毁,则返回错误。

'finish' 事件发出之前立即设置为 true

返回创建此 Writable 时传递的 highWaterMark 值。

此属性包含队列中准备写入的字节数(或对象数)。该值提供有关 highWaterMark 状态的内省数据。

如果流的缓冲区已满且流将发出 'drain',则为 true

给定 Writable 流的 objectMode 属性的 getter。

Calls writable.destroy() with an AbortError and returns a promise that fulfills when the stream is finished.

Attributes
Optional data to write. For streams not operating in object mode,  chunk  must be a <string> , <Buffer> , <TypedArray> or <DataView> . For object mode streams,  chunk may be any JavaScript value other than null .
encoding?:<string> | <null>
The encoding, if  chunk is a string. Default: 'utf8'
callback:<Function>
Callback for when this chunk of data is flushed.
Returns:<boolean>
false if the stream wishes for the calling code to wait for the 'drain' event to be emitted before continuing to write additional data; otherwise true .

The writable.write() method writes some data to the stream, and calls the supplied callback once the data has been fully handled. If an error occurs, the callback will be called with the error as its first argument. The callback is called asynchronously and before 'error' is emitted.

The return value is true if the internal buffer is less than the highWaterMark configured when the stream was created after admitting chunk. If false is returned, further attempts to write data to the stream should stop until the 'drain' event is emitted.

While a stream is not draining, calls to write() will buffer chunk, and return false. Once all currently buffered chunks are drained (accepted for delivery by the operating system), the 'drain' event will be emitted. Once write() returns false, do not write more chunks until the 'drain' event is emitted. While calling write() on a stream that is not draining is allowed, Node.js will buffer all written chunks until maximum memory usage occurs, at which point it will abort unconditionally. Even before it aborts, high memory usage will cause poor garbage collector performance and high RSS (which is not typically released back to the system, even after the memory is no longer required). Since TCP sockets may never drain if the remote peer does not read the data, writing a socket that is not draining may lead to a remotely exploitable vulnerability.

Writing data while the stream is not draining is particularly problematic for a Transform, because the Transform streams are paused by default until they are piped or a 'data' or 'readable' event handler is added.

If the data to be written can be generated or fetched on demand, it is recommended to encapsulate the logic into a Readable and use stream.pipe(). However, if calling write() is preferred, it is possible to respect backpressure and avoid memory issues using the 'drain' event:

function write(data, cb) {
  if (!stream.write(data)) {
    stream.once('drain', cb);
  } else {
    process.nextTick(cb);
  }
}

// Wait for cb to be called before doing any other write.
write('hello', () => {
  console.log('Write completed, do more writes now.');
});

A Writable stream in object mode will always ignore the encoding argument.

Examples of Readable streams include:

All Readable streams implement the interface defined by the stream.Readable class.

Readable streams effectively operate in one of two modes: flowing and paused. These modes are separate from object mode. A Readable stream can be in object mode or not, regardless of whether it is in flowing mode or paused mode.

  • In flowing mode, data is read from the underlying system automatically and provided to an application as quickly as possible using events via the EventEmitter interface.

  • In paused mode, the stream.read() method must be called explicitly to read chunks of data from the stream.

All Readable streams begin in paused mode but can be switched to flowing mode in one of the following ways:

The Readable can switch back to paused mode using one of the following:

  • If there are no pipe destinations, by calling the stream.pause() method.
  • If there are pipe destinations, by removing all pipe destinations. Multiple pipe destinations may be removed by calling the stream.unpipe() method.

The important concept to remember is that a Readable will not generate data until a mechanism for either consuming or ignoring that data is provided. If the consuming mechanism is disabled or taken away, the Readable will attempt to stop generating the data.

For backward compatibility reasons, removing 'data' event handlers will not automatically pause the stream. Also, if there are piped destinations, then calling stream.pause() will not guarantee that the stream will remain paused once those destinations drain and ask for more data.

If a Readable is switched into flowing mode and there are no consumers available to handle the data, that data will be lost. This can occur, for instance, when the readable.resume() method is called without a listener attached to the 'data' event, or when a 'data' event handler is removed from the stream.

Adding a 'readable' event handler automatically makes the stream stop flowing, and the data has to be consumed via readable.read(). If the 'readable' event handler is removed, then the stream will start flowing again if there is a 'data' event handler.

The "two modes" of operation for a Readable stream are a simplified abstraction for the more complicated internal state management that is happening within the Readable stream implementation.

Specifically, at any given point in time, every Readable is in one of three possible states:

  • readable.readableFlowing === null
  • readable.readableFlowing === false
  • readable.readableFlowing === true

When readable.readableFlowing is null, no mechanism for consuming the stream's data is provided. Therefore, the stream will not generate data. While in this state, attaching a listener for the 'data' event, calling the readable.pipe() method, or calling the readable.resume() method will switch readable.readableFlowing to true, causing the Readable to begin actively emitting events as data is generated.

Calling readable.pause(), readable.unpipe(), or receiving backpressure will cause the readable.readableFlowing to be set as false, temporarily halting the flowing of events but not halting the generation of data. While in this state, attaching a listener for the 'data' event will not switch readable.readableFlowing to true.

const { PassThrough, Writable } = require('node:stream');
const pass = new PassThrough();
const writable = new Writable();

pass.pipe(writable);
pass.unpipe(writable);
// readableFlowing is now false.

pass.on('data', (chunk) => { console.log(chunk.toString()); });
// readableFlowing is still false.
pass.write('ok');  // Will not emit 'data'.
pass.resume();     // Must be called to make stream emit 'data'.
// readableFlowing is now true.

While readable.readableFlowing is false, data may be accumulating within the stream's internal buffer.

The Readable stream API evolved across multiple Node.js versions and provides multiple methods of consuming stream data. In general, developers should choose one of the methods of consuming data and should never use multiple methods to consume data from a single stream. Specifically, using a combination of on('data'), on('readable'), pipe(), or async iterators could lead to unintuitive behavior.

The 'close' event is emitted when the stream and any of its underlying resources (a file descriptor, for example) have been closed. The event indicates that no more events will be emitted, and no further computation will occur.

A Readable stream will always emit the 'close' event if it is created with the emitClose option.

Attributes
The chunk of data. For streams that are not operating in object mode, the chunk will be either a string or  Buffer . For streams that are in object mode, the chunk can be any JavaScript value other than null .

The 'data' event is emitted whenever the stream is relinquishing ownership of a chunk of data to a consumer. This may occur whenever the stream is switched in flowing mode by calling readable.pipe(), readable.resume(), or by attaching a listener callback to the 'data' event. The 'data' event will also be emitted whenever the readable.read() method is called and a chunk of data is available to be returned.

Attaching a 'data' event listener to a stream that has not been explicitly paused will switch the stream into flowing mode. Data will then be passed as soon as it is available.

The listener callback will be passed the chunk of data as a string if a default encoding has been specified for the stream using the readable.setEncoding() method; otherwise the data will be passed as a Buffer.

const readable = getReadableStreamSomehow();
readable.on('data', (chunk) => {
  console.log(`Received ${chunk.length} bytes of data.`);
});

The 'end' event is emitted when there is no more data to be consumed from the stream.

The 'end' event will not be emitted unless the data is completely consumed. This can be accomplished by switching the stream into flowing mode, or by calling stream.read() repeatedly until all data has been consumed.

const readable = getReadableStreamSomehow();
readable.on('data', (chunk) => {
  console.log(`Received ${chunk.length} bytes of data.`);
});
readable.on('end', () => {
  console.log('There will be no more data.');
});
Type:<Error>

The 'error' event may be emitted by a Readable implementation at any time. Typically, this may occur if the underlying stream is unable to generate data due to an underlying internal failure, or when a stream implementation attempts to push an invalid chunk of data.

The listener callback will be passed a single Error object.

The 'pause' event is emitted when stream.pause() is called and readableFlowing is not false.

The 'readable' event is emitted when there is data available to be read from the stream, up to the configured high water mark (state.highWaterMark). Effectively, it indicates that the stream has new information within the buffer. If data is available within this buffer, stream.read() can be called to retrieve that data. Additionally, the 'readable' event may also be emitted when the end of the stream has been reached.

const readable = getReadableStreamSomehow();
readable.on('readable', function() {
  // There is some data to read now.
  let data;

  while ((data = this.read()) !== null) {
    console.log(data);
  }
});

If the end of the stream has been reached, calling stream.read() will return null and trigger the 'end' event. This is also true if there never was any data to be read. For instance, in the following example, foo.txt is an empty file:

const fs = require('node:fs');
const rr = fs.createReadStream('foo.txt');
rr.on('readable', () => {
  console.log(`readable: ${rr.read()}`);
});
rr.on('end', () => {
  console.log('end');
});

The output of running this script is:

$ node test.js
readable: null
end

In some cases, attaching a listener for the 'readable' event will cause some amount of data to be read into an internal buffer.

In general, the readable.pipe() and 'data' event mechanisms are easier to understand than the 'readable' event. However, handling 'readable' might result in increased throughput.

If both 'readable' and 'data' are used at the same time, 'readable' takes precedence in controlling the flow, i.e. 'data' will be emitted only when stream.read() is called. The readableFlowing property would become false. If there are 'data' listeners when 'readable' is removed, the stream will start flowing, i.e. 'data' events will be emitted without calling .resume().

The 'resume' event is emitted when stream.resume() is called and readableFlowing is not true.

Attributes
error:<Error>
Error which will be passed as payload in  'error' event
Returns:<this>

Destroy the stream. Optionally emit an 'error' event, and emit a 'close' event (unless emitClose is set to false). After this call, the readable stream will release any internal resources and subsequent calls to push() will be ignored.

Once destroy() has been called any further calls will be a no-op and no further errors except from _destroy() may be emitted as 'error'.

Implementors should not override this method, but instead implement readable._destroy().

Is true after 'close' has been emitted.

Is true after readable.destroy() has been called.

Returns:<boolean>

The readable.isPaused() method returns the current operating state of the Readable. This is used primarily by the mechanism that underlies the readable.pipe() method. In most typical cases, there will be no reason to use this method directly.

const readable = new stream.Readable();

readable.isPaused(); // === false
readable.pause();
readable.isPaused(); // === true
readable.resume();
readable.isPaused(); // === false
Returns:<this>

The readable.pause() method will cause a stream in flowing mode to stop emitting 'data' events, switching out of flowing mode. Any data that becomes available will remain in the internal buffer.

const readable = getReadableStreamSomehow();
readable.on('data', (chunk) => {
  console.log(`Received ${chunk.length} bytes of data.`);
  readable.pause();
  console.log('There will be no additional data for 1 second.');
  setTimeout(() => {
    console.log('Now data will start flowing again.');
    readable.resume();
  }, 1000);
});

The readable.pause() method has no effect if there is a 'readable' event listener.

Attributes
destination:<stream.Writable>
The destination for writing data
options:<Object>
Pipe options
End the writer when the reader ends.  Default: true .
The  destination , allowing for a chain of pipes if it is a Duplex or a Transform stream

The readable.pipe() method attaches a Writable stream to the readable, causing it to switch automatically into flowing mode and push all of its data to the attached Writable. The flow of data will be automatically managed so that the destination Writable stream is not overwhelmed by a faster Readable stream.

The following example pipes all of the data from the readable into a file named file.txt:

const fs = require('node:fs');
const readable = getReadableStreamSomehow();
const writable = fs.createWriteStream('file.txt');
// All the data from readable goes into 'file.txt'.
readable.pipe(writable);

It is possible to attach multiple Writable streams to a single Readable stream.

The readable.pipe() method returns a reference to the destination stream making it possible to set up chains of piped streams:

const fs = require('node:fs');
const zlib = require('node:zlib');
const r = fs.createReadStream('file.txt');
const z = zlib.createGzip();
const w = fs.createWriteStream('file.txt.gz');
r.pipe(z).pipe(w);

By default, stream.end() is called on the destination Writable stream when the source Readable stream emits 'end', so that the destination is no longer writable. To disable this default behavior, the end option can be passed as false, causing the destination stream to remain open:

reader.pipe(writer, { end: false });
reader.on('end', () => {
  writer.end('Goodbye\n');
});

One important caveat is that if the Readable stream emits an error during processing, the Writable destination is not closed automatically. If an error occurs, it will be necessary to manually close each stream in order to prevent memory leaks.

The process.stderr and process.stdout Writable streams are never closed until the Node.js process exits, regardless of the specified options.

Attributes
Optional argument to specify how much data to read.

The readable.read() method reads data out of the internal buffer and returns it. If no data is available to be read, null is returned. By default, the data is returned as a Buffer object unless an encoding has been specified using the readable.setEncoding() method or the stream is operating in object mode.

The optional size argument specifies a specific number of bytes to read. If size bytes are not available to be read, null will be returned unless the stream has ended, in which case all of the data remaining in the internal buffer will be returned.

If the size argument is not specified, all of the data contained in the internal buffer will be returned.

The size argument must be less than or equal to 1 GiB.

The readable.read() method should only be called on Readable streams operating in paused mode. In flowing mode, readable.read() is called automatically until the internal buffer is fully drained.

const readable = getReadableStreamSomehow();

// 'readable' may be triggered multiple times as data is buffered in
readable.on('readable', () => {
  let chunk;
  console.log('Stream is readable (new data received in buffer)');
  // Use a loop to make sure we read all currently available data
  while (null !== (chunk = readable.read())) {
    console.log(`Read ${chunk.length} bytes of data...`);
  }
});

// 'end' will be triggered once when there is no more data available
readable.on('end', () => {
  console.log('Reached end of stream.');
});

Each call to readable.read() returns a chunk of data or null, signifying that there's no more data to read at that moment. These chunks aren't automatically concatenated. Because a single read() call does not return all the data, using a while loop may be necessary to continuously read chunks until all data is retrieved. When reading a large file, .read() might return null temporarily, indicating that it has consumed all buffered content but there may be more data yet to be buffered. In such cases, a new 'readable' event is emitted once there's more data in the buffer, and the 'end' event signifies the end of data transmission.

Therefore to read a file's whole contents from a readable, it is necessary to collect chunks across multiple 'readable' events:

const chunks = [];

readable.on('readable', () => {
  let chunk;
  while (null !== (chunk = readable.read())) {
    chunks.push(chunk);
  }
});

readable.on('end', () => {
  const content = chunks.join('');
});

A Readable stream in object mode will always return a single item from a call to readable.read(size), regardless of the value of the size argument.

If the readable.read() method returns a chunk of data, a 'data' event will also be emitted.

Calling stream.read([size]) after the 'end' event has been emitted will return null. No runtime error will be raised.

Is true if it is safe to call readable.read(), which means the stream has not been destroyed or emitted 'error' or 'end'.

Returns whether the stream was destroyed or errored before emitting 'end'.

Returns whether 'data' has been emitted.

Getter for the property encoding of a given Readable stream. The encoding property can be set using the readable.setEncoding() method.

Becomes true when 'end' event is emitted.

Type:<Error>

Returns error if the stream has been destroyed with an error.

This property reflects the current state of a Readable stream as described in the [Three states][] section.

Returns the value of highWaterMark passed when creating this Readable.

This property contains the number of bytes (or objects) in the queue ready to be read. The value provides introspection data regarding the status of the highWaterMark.

Getter for the property objectMode of a given Readable stream.

Returns:<this>

The readable.resume() method causes an explicitly paused Readable stream to resume emitting 'data' events, switching the stream into flowing mode.

The readable.resume() method can be used to fully consume the data from a stream without actually processing any of that data:

getReadableStreamSomehow()
  .resume()
  .on('end', () => {
    console.log('Reached the end, but did not read anything.');
  });

The readable.resume() method has no effect if there is a 'readable' event listener.

Attributes
encoding:<string>
The encoding to use.
Returns:<this>

The readable.setEncoding() method sets the character encoding for data read from the Readable stream.

By default, no encoding is assigned and stream data will be returned as Buffer objects. Setting an encoding causes the stream data to be returned as strings of the specified encoding rather than as Buffer objects. For instance, calling readable.setEncoding('utf8') will cause the output data to be interpreted as UTF-8 data, and passed as strings. Calling readable.setEncoding('hex') will cause the data to be encoded in hexadecimal string format.

The Readable stream will properly handle multi-byte characters delivered through the stream that would otherwise become improperly decoded if simply pulled from the stream as Buffer objects.

const readable = getReadableStreamSomehow();
readable.setEncoding('utf8');
readable.on('data', (chunk) => {
  assert.equal(typeof chunk, 'string');
  console.log('Got %d characters of string data:', chunk.length);
});
Attributes
destination:<stream.Writable>
Optional specific stream to unpipe
Returns:<this>

The readable.unpipe() method detaches a Writable stream previously attached using the stream.pipe() method.

If the destination is not specified, then all pipes are detached.

If the destination is specified, but no pipe is set up for it, then the method does nothing.

const fs = require('node:fs');
const readable = getReadableStreamSomehow();
const writable = fs.createWriteStream('file.txt');
// All the data from readable goes into 'file.txt',
// but only for the first second.
readable.pipe(writable);
setTimeout(() => {
  console.log('Stop writing to file.txt.');
  readable.unpipe(writable);
  console.log('Manually close the file stream.');
  writable.end();
}, 1000);
Attributes
Chunk of data to unshift onto the read queue. For streams not operating in object mode,  chunk  must be a <string> , <Buffer> , <TypedArray> , <DataView> or  null . For object mode streams, chunk may be any JavaScript value.
encoding:<string>
Encoding of string chunks. Must be a valid  Buffer encoding, such as 'utf8' or 'ascii' .

Passing chunk as null signals the end of the stream (EOF) and behaves the same as readable.push(null), after which no more data can be written. The EOF signal is put at the end of the buffer and any buffered data will still be flushed.

The readable.unshift() method pushes a chunk of data back into the internal buffer. This is useful in certain situations where a stream is being consumed by code that needs to "un-consume" some amount of data that it has optimistically pulled out of the source, so that the data can be passed on to some other party.

The stream.unshift(chunk) method cannot be called after the 'end' event has been emitted or a runtime error will be thrown.

Developers using stream.unshift() often should consider switching to use of a Transform stream instead. See the [API for stream implementers][] section for more information.

// Pull off a header delimited by \n\n.
// Use unshift() if we get too much.
// Call the callback with (error, header, stream).
const { StringDecoder } = require('node:string_decoder');
function parseHeader(stream, callback) {
  stream.on('error', callback);
  stream.on('readable', onReadable);
  const decoder = new StringDecoder('utf8');
  let header = '';
  function onReadable() {
    let chunk;
    while (null !== (chunk = stream.read())) {
      const str = decoder.write(chunk);
      if (str.includes('\n\n')) {
        // Found the header boundary.
        const split = str.split(/\n\n/);
        header += split.shift();
        const remaining = split.join('\n\n');
        const buf = Buffer.from(remaining, 'utf8');
        stream.removeListener('error', callback);
        // Remove the 'readable' listener before unshifting.
        stream.removeListener('readable', onReadable);
        if (buf.length)
          stream.unshift(buf);
        // Now the body of the message can be read from the stream.
        callback(null, header, stream);
        return;
      }
      // Still reading the header.
      header += str;
    }
  }
}

Unlike stream.push(chunk), stream.unshift(chunk) will not end the reading process by resetting the internal reading state of the stream. This can cause unexpected results if readable.unshift() is called during a read (i.e. from within a stream._read() implementation on a custom stream). Following the call to readable.unshift() with an immediate stream.push('') will reset the reading state appropriately, however it is best to simply avoid calling readable.unshift() while in the process of performing a read.

Attributes
stream:<Stream>
An "old style" readable stream
Returns:<this>

Prior to Node.js 0.10, streams did not implement the entire node:stream module API as it is currently defined. (See [Compatibility][] for more information.)

When using an older Node.js library that emits 'data' events and has a stream.pause() method that is advisory only, the readable.wrap() method can be used to create a Readable stream that uses the old stream as its data source.

It will rarely be necessary to use readable.wrap() but the method has been provided as a convenience for interacting with older Node.js applications and libraries.

const { OldReader } = require('./old-api-module.js');
const { Readable } = require('node:stream');
const oreader = new OldReader();
const myReader = new Readable().wrap(oreader);

myReader.on('readable', () => {
  myReader.read(); // etc.
});
to fully consume the stream.
const fs = require('node:fs');

async function print(readable) {
  readable.setEncoding('utf8');
  let data = '';
  for await (const chunk of readable) {
    data += chunk;
  }
  console.log(data);
}

print(fs.createReadStream('file')).catch(console.error);

If the loop terminates with a break, return, or a throw, the stream will be destroyed. In other terms, iterating over a stream will consume the stream fully. The stream will be read in chunks of size equal to the highWaterMark option. In the code example above, data will be in a single chunk if the file has less than 64 KiB of data because no highWaterMark option is provided to fs.createReadStream().

An  AsyncIterable<Uint8Array[]> that yields batched chunks from the stream.

When the --experimental-stream-iter flag is enabled, Readable streams implement the Stream.toAsyncStreamable protocol, enabling efficient consumption by the stream/iter API.

This provides a batched async iterator that drains the stream's internal buffer into Uint8Array[] batches, amortizing the per-chunk Promise overhead of the standard Symbol.asyncIterator path. For byte-mode streams, chunks are yielded directly as Buffer instances (which are Uint8Array subclasses). For object-mode or encoded streams, each chunk is normalized to Uint8Array before batching.

The returned iterator is tagged as a validated source, so from() passes it through without additional normalization.

import { Readable } from 'node:stream';
import { text, from } from 'node:stream/iter';

const readable = new Readable({
  read() { this.push('hello'); this.push(null); },
});

// Readable is automatically consumed via toAsyncStreamable
console.log(await text(from(readable))); // 'hello'

Without the --experimental-stream-iter flag, calling this method throws ERR_STREAM_ITER_MISSING_FLAG.

Calls readable.destroy() with an AbortError and returns a promise that fulfills when the stream is finished.

Attributes
options:<Object>
allows destroying the stream if the signal is aborted.
Returns:<Duplex>
a stream composed with the stream  stream .
import { Readable } from 'node:stream';

async function* splitToWords(source) {
  for await (const chunk of source) {
    const words = String(chunk).split(' ');

    for (const word of words) {
      yield word;
    }
  }
}

const wordsStream = Readable.from(['text passed through', 'composed stream']).compose(splitToWords);
const words = await wordsStream.toArray();

console.log(words); // prints ['text', 'passed', 'through', 'composed', 'stream']

readable.compose(s) is equivalent to stream.compose(readable, s).

This method also allows for an <AbortSignal> to be provided, which will destroy the composed stream when aborted.

See stream.compose(...streams) for more information.

Attributes
options:<Object>
destroyOnReturn?:<boolean>
When set to  false , calling return on the async iterator, or exiting a for await...of iteration using a break , return , or throw will not destroy the stream. Default: true .
to consume the stream.

The iterator created by this method gives users the option to cancel the destruction of the stream if the for await...of loop is exited by return, break, or throw, or if the iterator should destroy the stream if the stream emitted an error during iteration.

const { Readable } = require('node:stream');

async function printIterator(readable) {
  for await (const chunk of readable.iterator({ destroyOnReturn: false })) {
    console.log(chunk); // 1
    break;
  }

  console.log(readable.destroyed); // false

  for await (const chunk of readable.iterator({ destroyOnReturn: false })) {
    console.log(chunk); // Will print 2 and then 3
  }

  console.log(readable.destroyed); // True, stream was totally consumed
}

async function printSymbolAsyncIterator(readable) {
  for await (const chunk of readable) {
    console.log(chunk); // 1
    break;
  }

  console.log(readable.destroyed); // true
}

async function showBoth() {
  await printIterator(Readable.from([1, 2, 3]));
  await printSymbolAsyncIterator(Readable.from([1, 2, 3]));
}

showBoth();
Attributes
a function to map over every chunk in the stream.
data:<any>
a chunk of data from the stream.
options:<Object>
aborted if the stream is destroyed allowing to abort the  fn call early.
options:<Object>
concurrency?:<number>
the maximum concurrent invocation of  fn to call on the stream at once. Default: 1 .
highWaterMark?:<number>
how many items to buffer while waiting for user consumption of the mapped items.  Default: concurrency * 2 - 1 .
allows destroying the stream if the signal is aborted.
Returns:<Readable>
a stream mapped with the function  fn .

This method allows mapping over the stream. The fn function will be called for every chunk in the stream. If the fn function returns a promise - that promise will be awaited before being passed to the result stream.

import { Readable } from 'node:stream';
import { Resolver } from 'node:dns/promises';

// With a synchronous mapper.
for await (const chunk of Readable.from([1, 2, 3, 4]).map((x) => x * 2)) {
  console.log(chunk); // 2, 4, 6, 8
}
// With an asynchronous mapper, making at most 2 queries at a time.
const resolver = new Resolver();
const dnsResults = Readable.from([
  'nodejs.org',
  'openjsf.org',
  'www.linuxfoundation.org',
]).map((domain) => resolver.resolve4(domain), { concurrency: 2 });
for await (const result of dnsResults) {
  console.log(result); // Logs the DNS result of resolver.resolve4.
}
Attributes
a function to filter chunks from the stream.
data:<any>
a chunk of data from the stream.
options:<Object>
aborted if the stream is destroyed allowing to abort the  fn call early.
options:<Object>
concurrency?:<number>
the maximum concurrent invocation of  fn to call on the stream at once. Default: 1 .
highWaterMark?:<number>
how many items to buffer while waiting for user consumption of the filtered items.  Default: concurrency * 2 - 1 .
allows destroying the stream if the signal is aborted.
Returns:<Readable>
a stream filtered with the predicate  fn .

This method allows filtering the stream. For each chunk in the stream the fn function will be called and if it returns a truthy value, the chunk will be passed to the result stream. If the fn function returns a promise - that promise will be awaited.

import { Readable } from 'node:stream';
import { Resolver } from 'node:dns/promises';

// With a synchronous predicate.
for await (const chunk of Readable.from([1, 2, 3, 4]).filter((x) => x > 2)) {
  console.log(chunk); // 3, 4
}
// With an asynchronous predicate, making at most 2 queries at a time.
const resolver = new Resolver();
const dnsResults = Readable.from([
  'nodejs.org',
  'openjsf.org',
  'www.linuxfoundation.org',
]).filter(async (domain) => {
  const { address } = await resolver.resolve4(domain, { ttl: true });
  return address.ttl > 60;
}, { concurrency: 2 });
for await (const result of dnsResults) {
  // Logs domains with more than 60 seconds on the resolved dns record.
  console.log(result);
}
Attributes
a function to call on each chunk of the stream.
data:<any>
a chunk of data from the stream.
options:<Object>
aborted if the stream is destroyed allowing to abort the  fn call early.
options:<Object>
concurrency?:<number>
the maximum concurrent invocation of  fn to call on the stream at once. Default: 1 .
allows destroying the stream if the signal is aborted.
Returns:<Promise>
a promise for when the stream has finished.

This method allows iterating a stream. For each chunk in the stream the fn function will be called. If the fn function returns a promise - that promise will be awaited.

This method is different from for await...of loops in that it can optionally process chunks concurrently. In addition, a forEach iteration can only be stopped by having passed a signal option and aborting the related AbortController while for await...of can be stopped with break or return. In either case the stream will be destroyed.

This method is different from listening to the 'data' event in that it uses the readable event in the underlying machinery and can limit the number of concurrent fn calls.

import { Readable } from 'node:stream';
import { Resolver } from 'node:dns/promises';

// With a synchronous predicate.
for await (const chunk of Readable.from([1, 2, 3, 4]).filter((x) => x > 2)) {
  console.log(chunk); // 3, 4
}
// With an asynchronous predicate, making at most 2 queries at a time.
const resolver = new Resolver();
const dnsResults = Readable.from([
  'nodejs.org',
  'openjsf.org',
  'www.linuxfoundation.org',
]).map(async (domain) => {
  const { address } = await resolver.resolve4(domain, { ttl: true });
  return address;
}, { concurrency: 2 });
await dnsResults.forEach((result) => {
  // Logs result, similar to `for await (const result of dnsResults)`
  console.log(result);
});
console.log('done'); // Stream has finished
Attributes
options:<Object>
allows cancelling the toArray operation if the signal is aborted.
Returns:<Promise>
a promise containing an array with the contents of the stream.

This method allows easily obtaining the contents of a stream.

As this method reads the entire stream into memory, it negates the benefits of streams. It's intended for interoperability and convenience, not as the primary way to consume streams.

import { Readable } from 'node:stream';
import { Resolver } from 'node:dns/promises';

await Readable.from([1, 2, 3, 4]).toArray(); // [1, 2, 3, 4]

const resolver = new Resolver();

// Make dns queries concurrently using .map and collect
// the results into an array using toArray
const dnsResults = await Readable.from([
  'nodejs.org',
  'openjsf.org',
  'www.linuxfoundation.org',
]).map(async (domain) => {
  const { address } = await resolver.resolve4(domain, { ttl: true });
  return address;
}, { concurrency: 2 }).toArray();
Attributes
a function to call on each chunk of the stream.
data:<any>
a chunk of data from the stream.
options:<Object>
aborted if the stream is destroyed allowing to abort the  fn call early.
options:<Object>
concurrency?:<number>
the maximum concurrent invocation of  fn to call on the stream at once. Default: 1 .
allows destroying the stream if the signal is aborted.
Returns:<Promise>
a promise evaluating to  true if fn returned a truthy value for at least one of the chunks.

This method is similar to Array.prototype.some and calls fn on each chunk in the stream until the awaited return value is true (or any truthy value). Once an fn call on a chunk awaited return value is truthy, the stream is destroyed and the promise is fulfilled with true. If none of the fn calls on the chunks return a truthy value, the promise is fulfilled with false.

import { Readable } from 'node:stream';
import { stat } from 'node:fs/promises';

// 使用同步谓词。
await Readable.from([1, 2, 3, 4]).find((x) => x > 2); // 3
await Readable.from([1, 2, 3, 4]).find((x) => x > 0); // 1
await Readable.from([1, 2, 3, 4]).find((x) => x > 10); // undefined

// 使用异步谓词,最多同时进行 2 个文件检查。
const foundBigFile = await Readable.from([
  'file1',
  'file2',
  'file3',
]).find(async (fileName) => {
  const stats = await stat(fileName);
  return stats.size > 1024 * 1024;
}, { concurrency: 2 });
console.log(foundBigFile); // 如果列表中的任何文件大于 1MB,则为大文件的文件名
console.log('done'); // 流已完成

稳定性:1 - 实验性

Attributes
一个在流的每个块上调用的函数。
data:<any>
来自流的一个数据块。
options:<Object>
如果流被销毁则中止,允许提前中止  fn 调用。
options:<Object>
concurrency:<number>
同时在流上调用的  fn 的最大并发调用次数。 默认: 1
如果信号被中止,允许销毁流。

此方法类似于 Array.prototype.every,并在流中的每个块上调用 fn 以检查所有 await 的返回值是否都是 fn 的真值。一旦某个块上的 fn 调用的 await 返回值为假值,流将被销毁,并且 promise 将以 false 履行。如果所有块上的 fn 调用都返回真值,则 promise 将以 true 履行。

import { Readable } from 'node:stream';
import { stat } from 'node:fs/promises';

// 使用同步谓词。
await Readable.from([1, 2, 3, 4]).every((x) => x > 2); // false
await Readable.from([1, 2, 3, 4]).every((x) => x > 0); // true

// 使用异步谓词,最多同时进行 2 个文件检查。
const allBigFiles = await Readable.from([
  'file1',
  'file2',
  'file3',
]).every(async (fileName) => {
  const stats = await stat(fileName);
  return stats.size > 1024 * 1024;
}, { concurrency: 2 });
// 如果列表中的所有文件都大于 1MiB,则为 `true`
console.log(allBigFiles);
console.log('done'); // 流已完成

稳定性:1 - 实验性

Attributes
一个用于映射流中每个块的函数。
data:<any>
来自流的一个数据块。
options:<Object>
如果流被销毁则中止,允许提前中止  fn 调用。
options:<Object>
concurrency:<number>
同时在流上调用的  fn 的最大并发调用次数。 默认: 1
如果信号被中止,允许销毁流。

此方法通过将给定的回调应用于流的每个块然后扁平化结果来返回一个新流。

可以从 fn 返回一个流或另一个可迭代对象或异步可迭代对象,结果流将被合并(扁平化)到返回的流中。

import { Readable } from 'node:stream';
import { createReadStream } from 'node:fs';

// 使用同步映射器。
for await (const chunk of Readable.from([1, 2, 3, 4]).flatMap((x) => [x, x])) {
  console.log(chunk); // 1, 1, 2, 2, 3, 3, 4, 4
}
// 使用异步映射器,合并 4 个文件的内容
const concatResult = Readable.from([
  './1.mjs',
  './2.mjs',
  './3.mjs',
  './4.mjs',
]).flatMap((fileName) => createReadStream(fileName));
for await (const result of concatResult) {
  // 这里将包含所有 4 个文件的内容(所有块)
  console.log(result);
}

稳定性:1 - 实验性

Attributes
limit:<number>
要从 readable 中丢弃的块数量。
options:<Object>
如果信号被中止,则允许销毁流。

此方法返回一个新流,其中前 limit 个块被丢弃。

import { Readable } from 'node:stream';

await Readable.from([1, 2, 3, 4]).drop(2).toArray(); // [3, 4]

稳定性:1 - 实验性

Attributes
limit:<number>
要从 readable 获取的块的数量。
options:<Object>
如果信号被中止,允许销毁流。

此方法返回一个新流,其中包含前 limit 个块。

import { Readable } from 'node:stream';

await Readable.from([1, 2, 3, 4]).take(2).toArray(); // [1, 2]

稳定性:1 - 实验性

Attributes
一个在流的每个块上调用的归约函数。
previous:<any>
上一次调用  fn 得到的值,或者如果指定了 initial 值则为该值,否则为流的第一个块。
data:<any>
来自流的一个数据块。
options:<Object>
如果流被销毁则中止,允许提前中止  fn 调用。
initial:<any>
用于归约的初始值。
options:<Object>
如果信号被中止,允许销毁流。

此方法按顺序在流的每个块上调用 fn,将上一个元素计算的结果传递给它。它返回一个归约最终值的 promise。

如果没有提供 initial 值,则使用流的第一个块作为初始值。如果流为空,则 promise 将被带有 ERR_INVALID_ARGS 代码属性的 TypeError 拒绝。

import { Readable } from 'node:stream';
import { readdir, stat } from 'node:fs/promises';
import { join } from 'node:path';

const directoryPath = './src';
const filesInDir = await readdir(directoryPath);

const folderSize = await Readable.from(filesInDir)
  .reduce(async (totalSize, file) => {
    const { size } = await stat(join(directoryPath, file));
    return totalSize + size;
  }, 0);

console.log(folderSize);

reducer 函数逐个元素地迭代流,这意味着没有 concurrency 参数或并行性。要并发执行 reduce,可以将异步函数提取到 readable.map 方法。

import { Readable } from 'node:stream';
import { readdir, stat } from 'node:fs/promises';
import { join } from 'node:path';

const directoryPath = './src';
const filesInDir = await readdir(directoryPath);

const folderSize = await Readable.from(filesInDir)
  .map((file) => stat(join(directoryPath, file)), { concurrency: 2 })
  .reduce((totalSize, { size }) => totalSize + size, 0);

console.log(folderSize);

Duplex 流是同时实现 ReadableWritable 接口的流。

Duplex 流的示例包括:

如果为 false,则当 readable 端结束时,流将自动结束 writable 端。最初由 allowHalfOpen 构造函数选项设置,默认为 true

可以手动更改此项以更改现有 Duplex 流实例的半开行为,但必须在发出 'end' 事件之前更改。

Transform 流是输出以某种方式与输入相关的 Duplex 流。像所有 Duplex 流一样,Transform 流同时实现 ReadableWritable 接口。

Transform 流的示例包括:

Attributes
error:<Error>

销毁流,并可选地发出 'error' 事件。在此调用之后,transform 流将释放任何内部资源。 实现者不应覆盖此方法,而应实现 readable._destroy()Transform_destroy() 的默认实现也会发出 'close',除非 emitClose 设置为 false。

一旦调用了 destroy(),任何进一步的调用都将为空操作,并且除了来自 _destroy() 的错误外,不会再发出任何 'error' 错误。

Attributes
options:<Object>
传递给两个  Duplex 构造函数的值,用于设置缓冲等选项。

实用函数 duplexPair 返回一个包含两项的 Array,每一项都是连接到另一侧的 Duplex 流:

写入一个流的内容可在另一个流上读取。它提供了类似于网络连接的行为,其中客户端写入的数据可由服务器读取,反之亦然。

Duplex 流是对称的;可以使用其中一个或另一个,行为没有任何区别。

一个函数,用于在流不再可读、可写或遇到错误或过早关闭事件时得到通知。

const { finished } = require('node:stream');
const fs = require('node:fs');

const rs = fs.createReadStream('archive.tar');

finished(rs, (err) => {
  if (err) {
    console.error('流失败。', err);
  } else {
    console.log('流已完成读取。');
  }
});

rs.resume(); // 排空流。

在错误处理场景中特别有用,其中流被过早销毁(例如中止的 HTTP 请求),并且不会发出 'end''finish'

finished API 提供 promise 版本

stream.finished()callback 被调用后会在流上留下悬空的事件监听器(特别是 'error''end''finish''close')。这样做的原因是防止意外的 'error' 事件(由于不正确的流实现)导致意外的崩溃。 如果这是不需要的行为,则需要在回调中调用返回的清理函数:

const cleanup = finished(rs, (err) => {
  cleanup();
  // ...
});

一个模块方法,用于在流和生成器之间进行管道传输,转发错误并正确清理,并在 pipeline 完成时提供回调。

const { pipeline } = require('node:stream');
const fs = require('node:fs');
const zlib = require('node:zlib');

// 使用 pipeline API 轻松地将一系列流管道在一起
// 并在 pipeline 完全完成时得到通知。

// 一个高效 gzip 可能巨大 tar 文件的 pipeline:

pipeline(
  fs.createReadStream('archive.tar'),
  zlib.createGzip(),
  fs.createWriteStream('archive.tar.gz'),
  (err) => {
    if (err) {
      console.error('管道失败。', err);
    } else {
      console.log('管道成功。');
    }
  },
);

pipeline API 提供 promise 版本

stream.pipeline() 将在所有流上调用 stream.destroy(err),除了:

  • 已发出 'end''close'Readable 流。
  • 已发出 'finish''close'Writable 流。

stream.pipeline()callback 被调用后会在流上留下悬空的事件监听器。在失败后重用流的情况下,这可能导致事件监听器泄漏和吞掉的错误。如果最后一个流是 readable,悬空的事件监听器将被移除,以便以后可以消费最后一个流。

stream.pipeline() 在引发错误时关闭所有流。 IncomingRequestpipeline 一起使用可能会导致意外行为,因为它会在没有发送预期响应的情况下销毁 socket。 参见下面的示例:

const fs = require('node:fs');
const http = require('node:http');
const { pipeline } = require('node:stream');

const server = http.createServer((req, res) => {
  const fileStream = fs.createReadStream('./fileNotExist.txt');
  pipeline(fileStream, res, (err) => {
    if (err) {
      console.log(err); // 没有这样的文件
      // 一旦 `pipeline` 已经销毁了 socket,就无法发送此消息
      return res.end('error!!!');
    }
  });
});

将两个或多个流组合成一个 Duplex 流,该流写入第一个流并从最后一个流读取。每个提供的流都使用 stream.pipeline 管道到下一个流。如果任何流出错,则所有流都被销毁,包括外部 Duplex 流。

因为 stream.compose 返回一个新流,该流又可以(并且应该)被管道到其他流中,所以它支持组合。相比之下,当将流传递给 stream.pipeline 时,通常第一个流是 readable 流,最后一个是 writable 流,形成一个闭合电路。

如果传递的是 Function,它必须是一个接受 source Iterable 的工厂方法。

import { compose, Transform } from 'node:stream';

const removeSpaces = new Transform({
  transform(chunk, encoding, callback) {
    callback(null, String(chunk).replace(' ', ''));
  },
});

async function* toUpper(source) {
  for await (const chunk of source) {
    yield String(chunk).toUpperCase();
  }
}

let res = '';
for await (const buf of compose(removeSpaces, toUpper).end('hello world')) {
  res += buf;
}

console.log(res); // 打印 'HELLOWORLD'

stream.compose 可用于将异步可迭代对象、生成器和函数转换为流。

  • AsyncIterable 转换为可读 Duplex。不能 yield null
  • AsyncGeneratorFunction 转换为可读/可写转换 Duplex。 必须将 source AsyncIterable 作为第一个参数。不能 yield null
  • AsyncFunction 转换为可写 Duplex。必须返回 nullundefined
import { compose } from 'node:stream';
import { finished } from 'node:stream/promises';

// 将 AsyncIterable 转换为 readable Duplex。
const s1 = compose(async function*() {
  yield 'Hello';
  yield 'World';
}());

// 将 AsyncGenerator 转换为 transform Duplex。
const s2 = compose(async function*(source) {
  for await (const chunk of source) {
    yield String(chunk).toUpperCase();
  }
});

let res = '';

// 将 AsyncFunction 转换为 writable Duplex。
const s3 = compose(async function(source) {
  for await (const chunk of source) {
    res += chunk;
  }
});

await finished(compose(s1, s2, s3));

console.log(res); // 打印 'HELLOWORLD'

为了方便起见,readable.compose(stream) 方法在 <Readable><Duplex> 流上可用作此函数的包装器。

返回流是否遇到错误。

返回流是否可读。

返回流是否可写。

一个用于从迭代器创建可读流的实用方法。

const { Readable } = require('node:stream');

async function * generate() {
  yield 'hello';
  yield 'streams';
}

const readable = Readable.from(generate());

readable.on('data', (chunk) => {
  console.log(chunk);
});

调用 Readable.from(string)Readable.from(buffer) 不会为了性能原因而迭代字符串或缓冲区以匹配其他流语义。

如果传递包含 promise 的 Iterable 对象作为参数,可能会导致未处理的拒绝。

const { Readable } = require('node:stream');

Readable.from([
  new Promise((resolve) => setTimeout(resolve('1'), 1500)),
  new Promise((_, reject) => setTimeout(reject(new Error('2')), 1000)), // 未处理的拒绝
]);

返回流是否已被读取或取消。

一个用于创建双工流的实用方法。

  • Stream 将 writable 流转换为 writable Duplex,readable 流转换为 Duplex
  • Blob 转换为 readable Duplex
  • string 转换为 readable Duplex
  • ArrayBuffer 转换为 readable Duplex
  • AsyncIterable 转换为 readable Duplex。不能 yield null
  • AsyncGeneratorFunction 转换为 readable/writable transform Duplex。必须将 source AsyncIterable 作为第一个参数。不能 yield null
  • AsyncFunction 转换为 writable Duplex。必须返回 nullundefined
  • Object ({ writable, readable })readablewritable 转换为 Stream,然后将它们组合成 Duplex,其中 Duplex 将写入 writable 并从 readable 读取。
  • Promise 转换为 readable Duplex。值 null 被忽略。
  • ReadableStream 转换为 readable Duplex
  • WritableStream 转换为 writable Duplex
  • 返回:<stream.Duplex>

如果传递包含 promise 的 Iterable 对象作为参数,可能会导致未处理的拒绝。

const { Duplex } = require('node:stream');

Duplex.from([
  new Promise((resolve) => setTimeout(resolve('1'), 1500)),
  new Promise((_, reject) => setTimeout(reject(new Error('2')), 1000)), // 未处理的拒绝
]);

将 AbortSignal 附加到可读流或可写流。这允许代码使用 AbortController 控制流的销毁。

在与传递的 AbortSignal 对应的 AbortController 上调用 abort 的行为,与在流上调用 .destroy(new AbortError()) 以及在 webstreams 上调用 controller.error(new AbortError()) 的行为相同。

const fs = require('node:fs');

const controller = new AbortController();
const read = addAbortSignal(
  controller.signal,
  fs.createReadStream(('object.json')),
);
// 稍后,中止操作以关闭流
controller.abort();

或者将 AbortSignal 与可读流一起用作异步可迭代对象:

const controller = new AbortController();
setTimeout(() => controller.abort(), 10_000); // 设置超时
const stream = addAbortSignal(
  controller.signal,
  fs.createReadStream(('object.json')),
);
(async () => {
  try {
    for await (const chunk of stream) {
      await process(chunk);
    }
  } catch (e) {
    if (e.name === 'AbortError') {
      // 操作已取消
    } else {
      throw e;
    }
  }
})();

或者将 AbortSignal 与可读流一起使用:

const controller = new AbortController();
const rs = new ReadableStream({
  start(controller) {
    controller.enqueue('hello');
    controller.enqueue('world');
    controller.close();
  },
});

addAbortSignal(controller.signal, rs);

finished(rs, (err) => {
  if (err) {
    if (err.name === 'AbortError') {
      // 操作已取消
    }
  }
});

const reader = rs.getReader();

reader.read().then(({ value, done }) => {
  console.log(value); // hello
  console.log(done); // false
  controller.abort();
});

返回流使用的默认 highWaterMark。 默认为 65536 (64 KiB),对于 objectMode16

设置流使用的默认 highWaterMark。

const { Writable } = require('node:stream');
const fs = require('node:fs');

class WriteStream extends Writable {
  constructor(filename) {
    super();
    this.filename = filename;
    this.fd = null;
  }
  _construct(callback) {
    fs.open(this.filename, 'w', (err, fd) => {
      if (err) {
        callback(err);
      } else {
        this.fd = fd;
        callback();
      }
    });
  }
  _write(chunk, encoding, callback) {
    fs.write(this.fd, chunk, callback);
  }
  _destroy(err, callback) {
    if (this.fd) {
      fs.close(this.fd, (er) => callback(er || err));
    } else {
      callback(err);
    }
  }
}
Attributes
要写入的  Buffer ,由传递给 stream.write()string 转换而来。如果流的 decodeStrings 选项为 false 或流在对象模式下运行,则 chunk 不会被转换 & 将是传递给 stream.write() 的任何内容。
encoding:<string>
如果 chunk 是字符串,则  encoding 是该字符串的字符编码。如果 chunk 是 Buffer ,或者流在对象模式下运行, encoding 可能会被忽略。
callback:<Function>
当提供的 chunk 处理完成时调用此函数(可选带错误参数)。

所有 Writable 流实现必须提供 writable._write() 和/或 writable._writev() 方法以将数据发送到基础资源。

Transform 流提供它们自己的 writable._write() 实现。

此函数不得由应用程序代码直接调用。它应由子类实现,并仅由内部 Writable 类方法调用。

callback 函数必须在 writable._write() 内部同步调用或异步调用(即不同的 tick),以信号表示写入成功完成或因错误失败。传递给 callback 的第一个参数必须是 Error 对象(如果调用失败)或 null(如果写入成功)。

在调用 writable._write() 和调用 callback 之间发生的所有 writable.write() 调用都将导致写入的数据被缓冲。当调用 callback 时,流可能会发出 'drain' 事件。如果流实现能够一次处理多个数据块,则应实现 writable._writev() 方法。

如果在构造函数选项中将 decodeStrings 属性显式设置为 false,则 chunk 将保持与传递给 .write() 相同的对象,并且可能是字符串而不是 Buffer。这是为了支持对某些字符串数据编码具有优化处理的实现。在这种情况下,encoding 参数将指示字符串的字符编码。否则,encoding 参数可以安全地忽略。

writable._write() 函数以前缀下划线开头,因为它对于定义它的类是内部的,用户程序绝不应直接调用它。

Attributes
chunks:<Object>
[] 要写入的数据。值是一个 <Object> 数组,每个对象代表一个要写入的离散数据块。这些对象的属性是:
包含要写入数据的 buffer 实例或字符串。如果  Writable 创建时 decodeStrings 选项设置为 false 并且字符串传递给 write() ,则 chunk 将是字符串。
encoding:<string>
chunk 的字符编码。如果 chunkBuffer ,则 encoding 将是 'buffer'
callback:<Function>
当提供的 chunks 处理完成时调用的回调函数(可选带错误参数)。

此函数不得由应用程序代码直接调用。它应由子类实现,并仅由内部 Writable 类方法调用。

在能够一次处理多个数据块的流实现中,writable._writev() 方法可以作为 writable._write() 的补充或替代来实现。如果实现了并且存在来自先前写入的缓冲数据,则将调用 _writev() 而不是 _write()

writable._writev() 方法以前缀下划线开头,因为它对于定义它的类是内部的,用户程序绝不应直接调用它。

Attributes
可能的错误。
callback:<Function>
接受可选错误参数的回调函数。

_destroy() 方法由 writable.destroy() 调用。它可以被子类覆盖,但不得直接调用。

Attributes
callback:<Function>
当完成写入任何剩余数据时调用此函数(可选带错误参数)。

_final() 方法不得直接调用。它可以由子类实现,如果是这样,将仅由内部 Writable 类方法调用。

这个可选函数将在流关闭之前调用,延迟 'finish' 事件直到 callback 被调用。这对于在流结束之前关闭资源或写入缓冲数据很有用。

在处理 writable._write()writable._writev()writable._final() 方法期间发生的错误必须通过调用回调并将错误作为第一个参数传递来传播。从这些方法内部抛出 Error 或手动发出 'error' 事件会导致未定义的行为。

如果 Readable 流管道连接到 Writable 流,当 Writable 发出错误时,Readable 流将被取消管道连接。

const { Writable } = require('node:stream');

const myWritable = new Writable({
  write(chunk, encoding, callback) {
    if (chunk.toString().indexOf('a') >= 0) {
      callback(new Error('chunk is invalid'));
    } else {
      callback();
    }
  },
});

以下说明了一个相当简单(且有点无意义)的自定义 Writable 流实现。虽然这个特定的 Writable 流实例没有任何真正的特别用处,但该示例说明了自定义 Writable 流实例的每个必需元素:

const { Writable } = require('node:stream');

class MyWritable extends Writable {
  _write(chunk, encoding, callback) {
    if (chunk.toString().indexOf('a') >= 0) {
      callback(new Error('chunk is invalid'));
    } else {
      callback();
    }
  }
}

解码缓冲区是一项常见任务,例如在使用以字符串作为输入的转换器时。当使用多字节字符编码(如 UTF-8)时,这并不是一个简单的过程。以下示例展示了如何使用 StringDecoderWritable 解码多字节字符串。

const { Writable } = require('node:stream');
const { StringDecoder } = require('node:string_decoder');

class StringWritable extends Writable {
  constructor(options) {
    super(options);
    this._decoder = new StringDecoder(options?.defaultEncoding);
    this.data = '';
  }
  _write(chunk, encoding, callback) {
    if (encoding === 'buffer') {
      chunk = this._decoder.write(chunk);
    }
    this.data += chunk;
    callback();
  }
  _final(callback) {
    this.data += this._decoder.end();
    callback();
  }
}

const euro = [[0xE2, 0x82], [0xAC]].map(Buffer.from);
const w = new StringWritable();

w.write('currency: ');
w.write(euro[0]);
w.end(euro[1]);

console.log(w.data); // currency: €

自定义 Readable必须 调用 new stream.Readable([options]) 构造函数并实现 readable._read() 方法。

Attributes
options:<Object>
highWaterMark:<number>
在停止从基础资源读取之前存储在内部缓冲区中的最大  字节数默认: 65536 (64 KiB),对于 objectMode 流为 16
encoding:<string>
如果指定,则 buffers 将使用指定的编码解码为字符串。 默认: null
objectMode:<boolean>
此流是否应表现为对象流。意味着  stream.read(n) 返回单个值而不是大小为 nBuffer默认: false
emitClose:<boolean>
流在销毁后是否应发出  'close'默认: true
stream._read() 方法的实现。
destroy:<Function>
stream._destroy() 方法的实现。
construct:<Function>
stream._construct() 方法的实现。
autoDestroy:<boolean>
此流在结束后是否应自动调用  .destroy()默认: true
表示可能取消的信号。
const { Readable } = require('node:stream');

class MyReadable extends Readable {
  constructor(options) {
    // 调用 stream.Readable(options) 构造函数。
    super(options);
    // ...
  }
}

或者,当使用 ES6 之前的构造函数风格时:

const { Readable } = require('node:stream');
const util = require('node:util');

function MyReadable(options) {
  if (!(this instanceof MyReadable))
    return new MyReadable(options);
  Readable.call(this, options);
}
util.inherits(MyReadable, Readable);

或者,使用简化构造函数方法:

const { Readable } = require('node:stream');

const myReadable = new Readable({
  read(size) {
    // ...
  },
});

对传入的 AbortSignal 对应的 AbortController 调用 abort 的行为将与在创建的可读流上调用 .destroy(new AbortError()) 相同。

const { Readable } = require('node:stream');
const controller = new AbortController();
const read = new Readable({
  read(size) {
    // ...
  },
  signal: controller.signal,
});
// 稍后,中止操作以关闭流
controller.abort();
Attributes
callback:<Function>
当流完成初始化时调用此函数(可选带错误参数)。

_construct() 方法不得直接调用。它可以由子类实现,如果是这样,将仅由内部 Readable 类方法调用。

这个可选函数将由流构造函数安排在下一个 tick 中,延迟任何 _read()_destroy() 调用直到 callback 被调用。这对于在流可以使用之前初始化状态或异步初始化资源很有用。

const { Readable } = require('node:stream');
const fs = require('node:fs');

class ReadStream extends Readable {
  constructor(filename) {
    super();
    this.filename = filename;
    this.fd = null;
  }
  _construct(callback) {
    fs.open(this.filename, (err, fd) => {
      if (err) {
        callback(err);
      } else {
        this.fd = fd;
        callback();
      }
    });
  }
  _read(n) {
    const buf = Buffer.alloc(n);
    fs.read(this.fd, buf, 0, n, null, (err, bytesRead) => {
      if (err) {
        this.destroy(err);
      } else {
        this.push(bytesRead > 0 ? buf.slice(0, bytesRead) : null);
      }
    });
  }
  _destroy(err, callback) {
    if (this.fd) {
      fs.close(this.fd, (er) => callback(er || err));
    } else {
      callback(err);
    }
  }
}
Attributes
异步读取的字节数

此函数不得由应用程序代码直接调用。它应由子类实现,并仅由内部 Readable 类方法调用。

所有 Readable 流实现必须提供 readable._read() 方法的实现以从基础资源获取数据。

当调用 readable._read() 时,如果资源中有可用数据,实现应开始使用 this.push(dataChunk) 方法将该数据推入读取队列。一旦流准备好接受更多数据,每次调用 this.push(dataChunk) 后将再次调用 _read()_read() 可以继续从资源读取并推送数据,直到 readable.push() 返回 false。只有当 _read() 在停止后再次被调用时,它才应恢复向队列推送额外数据。

一旦 readable._read() 方法被调用,它将不会再被调用,直到通过 readable.push() 方法推送更多数据。空数据(如空 buffers 和字符串)不会导致 readable._read() 被调用。

size 参数是建议性的。对于“读取”是返回数据的单个操作的实现,可以使用 size 参数来确定要获取多少数据。其他实现可能会忽略此参数,并在数据可用时简单地提供数据。在调用 stream.push(chunk) 之前,没有必要“等待”直到 size 字节可用。

readable._read() 方法以前缀下划线开头,因为它对于定义它的类是内部的,用户程序绝不应直接调用它。

Attributes
可能的错误。
callback:<Function>
接受可选错误参数的回调函数。

_destroy() 方法由 readable.destroy() 调用。它可以被子类覆盖,但不得直接调用。

Attributes
要推入读取队列的数据块。对于不在对象模式下运行的流, chunk  必须是 <string><Buffer><TypedArray><DataView> 。对于对象模式流, chunk 可以是任何 JavaScript 值。
encoding:<string>
字符串块的编码。必须是有效的  Buffer 编码,例如 'utf8''ascii'

chunk 是 <Buffer><TypedArray><DataView><string> 时,数据 chunk 将被添加到内部队列以供流的用户消费。传递 chunknull 信号表示流结束 (EOF),之后不能再写入更多数据。

Readable 在暂停模式下运行时,可以使用 readable.push() 添加的数据通过调用 readable.read() 方法在 'readable' 事件发出时读出。

Readable 在流动模式下运行时,使用 readable.push() 添加的数据将通过发出 'data' 事件来交付。

readable.push() 方法旨在尽可能灵活。例如,当包装提供某种暂停/恢复机制和数据回调的底层源时,底层源可以由自定义 Readable 实例包装:

// `_source` 是一个具有 readStop() 和 readStart() 方法的对象,
// 以及一个在有数据时调用的 `ondata` 成员,
// 和一个在数据结束时调用的 `onend` 成员。

class SourceWrapper extends Readable {
  constructor(options) {
    super(options);

    this._source = getLowLevelSourceObject();

    // 每次有数据时,将其推入内部缓冲区。
    this._source.ondata = (chunk) => {
      // 如果 push() 返回 false,则停止从源读取。
      if (!this.push(chunk))
        this._source.readStop();
    };

    // 当源结束时,推送 EOF 信号 `null` 块。
    this._source.onend = () => {
      this.push(null);
    };
  }
  // 当流想要拉取更多数据时将调用 _read()。
  // 在这种情况下,建议的 size 参数被忽略。
  _read(size) {
    this._source.readStart();
  }
}

readable.push() 方法用于将内容推入内部缓冲区。它可以由 readable._read() 方法驱动。

对于不在对象模式下运行的流,如果 readable.push()chunk 参数是 undefined,它将被视为空字符串或缓冲区。有关更多信息,请参阅 readable.push('')

在处理 readable._read() 期间发生的错误必须通过 readable.destroy(err) 方法传播。从 readable._read() 内部抛出 Error 或手动发出 'error' 事件会导致未定义的行为。

const { Readable } = require('node:stream');

const myReadable = new Readable({
  read(size) {
    const err = checkSomeErrorCondition();
    if (err) {
      this.destroy(err);
    } else {
      // 做一些工作。
    }
  },
});

以下是一个基本的 Readable 流示例,它按升序发出从 1 到 1,000,000 的数字,然后结束。

const { Readable } = require('node:stream');

class Counter extends Readable {
  constructor(opt) {
    super(opt);
    this._max = 1000000;
    this._index = 1;
  }

  _read() {
    const i = this._index++;
    if (i > this._max)
      this.push(null);
    else {
      const str = String(i);
      const buf = Buffer.from(str, 'ascii');
      this.push(buf);
    }
  }
}

因为 JavaScript 不支持多重继承,所以扩展 stream.Duplex 类来实现 Duplex 流(而不是扩展 stream.Readable stream.Writable 类)。

stream.Duplex 类原型继承自 stream.Readable 并寄生继承自 stream.Writable,但由于在 stream.Writable 上覆盖了 Symbol.hasInstanceinstanceof 对于两个基类都能正常工作。

自定义 Duplex必须 调用 new stream.Duplex([options]) 构造函数并实现 readable._read()writable._write() 方法 两者

Attributes
options:<Object>
传递给  WritableReadable 构造函数。还有以下字段:
allowHalfOpen:<boolean>
如果设置为  false ,则当可读侧结束时,流将自动结束可写侧。 默认: true
readable:<boolean>
设置  Duplex 是否应可读。 默认: true
writable:<boolean>
设置  Duplex 是否应可写。 默认: true
readableObjectMode:<boolean>
为流的可读侧设置  objectMode 。如果 objectModetrue 则无效。 默认: false
writableObjectMode:<boolean>
为流的可写侧设置  objectMode 。如果 objectModetrue 则无效。 默认: false
readableHighWaterMark:<number>
为流的可读侧设置  highWaterMark 。如果提供了 highWaterMark 则无效。
writableHighWaterMark:<number>
为流的可写侧设置  highWaterMark 。如果提供了 highWaterMark 则无效。
const { Duplex } = require('node:stream');

class MyDuplex extends Duplex {
  constructor(options) {
    super(options);
    // ...
  }
}

或者,当使用 ES6 之前的构造函数风格时:

const { Duplex } = require('node:stream');
const util = require('node:util');

function MyDuplex(options) {
  if (!(this instanceof MyDuplex))
    return new MyDuplex(options);
  Duplex.call(this, options);
}
util.inherits(MyDuplex, Duplex);

或者,使用简化构造函数方法:

const { Duplex } = require('node:stream');

const myDuplex = new Duplex({
  read(size) {
    // ...
  },
  write(chunk, encoding, callback) {
    // ...
  },
});

当使用流水线时:

const { Transform, pipeline } = require('node:stream');
const fs = require('node:fs');

pipeline(
  fs.createReadStream('object.json')
    .setEncoding('utf8'),
  new Transform({
    decodeStrings: false, // 接受字符串输入而不是 Buffers
    construct(callback) {
      this.data = '';
      callback();
    },
    transform(chunk, encoding, callback) {
      this.data += chunk;
      callback();
    },
    flush(callback) {
      try {
        // 确保是有效的 json。
        JSON.parse(this.data);
        this.push(this.data);
        callback();
      } catch (err) {
        callback(err);
      }
    },
  }),
  fs.createWriteStream('valid-object.json'),
  (err) => {
    if (err) {
      console.error('failed', err);
    } else {
      console.log('completed');
    }
  },
);

以下说明了一个简单的 Duplex 流示例,它包装了一个假设的底层源对象,数据可以写入该对象,也可以从中读取数据,尽管使用的 API 与 Node.js 流不兼容。 以下说明了一个简单的 Duplex 流示例,它通过 Writable 接口缓冲传入的写入数据,然后通过 Readable 接口读回。

const { Duplex } = require('node:stream');
const kSource = Symbol('source');

class MyDuplex extends Duplex {
  constructor(source, options) {
    super(options);
    this[kSource] = source;
  }

  _write(chunk, encoding, callback) {
    // 底层源只处理字符串。
    if (Buffer.isBuffer(chunk))
      chunk = chunk.toString();
    this[kSource].writeSomeData(chunk);
    callback();
  }

  _read(size) {
    this[kSource].fetchSomeData(size, (data, encoding) => {
      this.push(Buffer.from(data, encoding));
    });
  }
}

Duplex 流最重要的方面是 ReadableWritable 侧尽管共存于单个对象实例中,但彼此独立运行。

对于 Duplex 流,objectMode 可以分别使用 readableObjectModewritableObjectMode 选项专门为 ReadableWritable 侧设置。

例如,在以下示例中,创建了一个新的 Transform 流(它是 Duplex 流的一种类型),它具有对象模式 Writable 侧,接受 JavaScript 数字,这些数字在 Readable 侧转换为十六进制字符串。

const { Transform } = require('node:stream');

// 所有 Transform 流也是 Duplex 流。
const myTransform = new Transform({
  writableObjectMode: true,

  transform(chunk, encoding, callback) {
    // 如有必要,将 chunk 强制转换为数字。
    chunk |= 0;

    // 将 chunk 转换为其他内容。
    const data = chunk.toString(16);

    // 将数据推入可读队列。
    callback(null, '0'.repeat(data.length % 2) + data);
  },
});

myTransform.setEncoding('ascii');
myTransform.on('data', (chunk) => console.log(chunk));

myTransform.write(1);
// 打印:01
myTransform.write(10);
// 打印:0a
myTransform.write(100);
// 打印:64

输出不必与输入大小相同、块数相同或同时到达。例如,Hash 流将永远只有一个输出块,该块在输入结束时提供。zlib 流将产生比其输入小得多或大得多的输出。

stream.Transform 类被扩展以实现 Transform 流。

stream.Transform 类原型继承自 stream.Duplex 并实现自己的 writable._write()readable._read() 方法版本。自定义 Transform 实现 必须 实现 transform._transform() 方法并 可以 实现 transform._flush() 方法。

在使用 Transform 流时必须小心,因为写入流的数据可能导致流的可写侧暂停,如果可读侧的输出未被消费。

Attributes
options:<Object>
传递给  WritableReadable 构造函数。还有以下字段:
transform:<Function>
stream._transform() 方法的实现。
stream._flush() 方法的实现。
const { Transform } = require('node:stream');

class MyTransform extends Transform {
  constructor(options) {
    super(options);
    // ...
  }
}

或者,当使用 ES6 之前的构造函数风格时:

const { Transform } = require('node:stream');
const util = require('node:util');

function MyTransform(options) {
  if (!(this instanceof MyTransform))
    return new MyTransform(options);
  Transform.call(this, options);
}
util.inherits(MyTransform, Transform);

或者,使用简化构造函数方法:

const { Transform } = require('node:stream');

const myTransform = new Transform({
  transform(chunk, encoding, callback) {
    // ...
  },
});

[finish][] 事件来自 Transform 类。finish 事件在所有数据输出后发出,这发生在 ._flush 中的回调被调用之后。在出错的情况下,不应发出 finish。

[end][] 事件来自 Transform 类。在调用 push(null) 且所有块都由 ._transform 处理后,发出 end 事件。在出错的情况下,不应发出 end。

  • _flush <Function> 当剩余数据已刷新时调用的回调函数(可选带错误参数和数据)。

此函数不得由应用程序代码直接调用。它应由子类实现,并仅由内部 _readableState 类方法调用。

在某些情况下,转换操作可能需要在流结束时发出额外的一点数据。例如,zlib 压缩流将存储用于优化压缩输出的内部状态量。然而,当流结束时,需要刷新该额外数据,以便压缩数据完整。

自定义 [Transform][] 实现可以实现 _flush 方法。当没有更多写入数据要消费时,但在发出信号表示 [Writable][] 流结束的 [finish][] 事件之前,将调用此方法。

在 Transform 实现中,_flush 方法可以调用零次或多次,视情况而定。当刷新操作完成时,必须调用 callback 函数。

_flush 方法以前缀下划线开头,因为它对于定义它的类是内部的,用户程序绝不应直接调用它。

  • chunk <Buffer> | <string> | <any> 要转换的 chunk,由传递给 write 的参数转换而来。如果流的 decodeStrings 选项为 false 或流在对象模式下运行,则 chunk 不会被转换 & 将是传递给 write 的任何内容。
  • encoding <string> 如果 chunk 是字符串,则这是编码类型。如果 chunk 是 buffer,则这是特殊值 buffer。 在这种情况下忽略它。
  • callback <Function> 在提供的 chunk 处理完成后调用的回调函数(可选带错误参数和数据)。

此函数不得由应用程序代码直接调用。它应由子类实现,并仅由内部 _write 类方法调用。

所有 Transform 流实现必须提供 _transform 方法以接受输入并产生输出。_transform 实现处理正在写入的字节,计算输出,然后使用 push 方法将该输出传递给可读部分。

_push 方法可以调用零次或多次以从单个输入块生成输出,具体取决于作为块的结果要输出多少。

有可能不会从任何给定的输入数据块生成输出。

仅当当前块完全消耗时才必须调用 callback 函数。传递给 callback 的第一个参数必须是 Error 对象(如果在处理输入时发生错误)或 null(否则)。如果将第二个参数传递给 callback,它将被转发到 _write 方法,但仅当第一个参数为 falsy 时。换句话说,以下等价:

transform.prototype._transform = function(data, encoding, callback) {
  this.push(data);
  callback();
};

transform.prototype._transform = function(data, encoding, callback) {
  callback(null, data);
};

transform._transform() 方法以前缀下划线开头,因为它对于定义它的类是内部的,用户程序绝不应直接调用它。

transform._transform() 绝不会并行调用;流实现了一个队列机制,要接收下一个块,必须调用 callback,无论是同步还是异步。

stream.PassThrough 类是 Transform 流的简单实现,它简单地将输入字节传递到输出。其主要目的是用于示例和测试,但在某些用例中,stream.PassThrough 可用作新型流的构建块。

下面提供了一些使用 Node.js 流与异步生成器和异步迭代器的常见互操作案例。

(async function() {
  for await (const chunk of readable) {
    console.log(chunk);
  }
})();

异步迭代器会在流上注册一个永久错误处理程序,以防止任何未处理的销毁后错误。

可以使用 Readable.from() 工具方法从异步生成器创建 Node.js 可读流:

const { Readable } = require('node:stream');

const ac = new AbortController();
const signal = ac.signal;

async function * generate() {
  yield 'a';
  await someLongRunningFn({ signal });
  yield 'b';
  yield 'c';
}

const readable = Readable.from(generate());
readable.on('close', () => {
  ac.abort();
});

readable.on('data', (chunk) => {
  console.log(chunk);
});

当从异步迭代器写入可写流时,确保正确处理背压和错误。stream.pipeline() 抽象化了背压和与背压相关错误的处理:

const fs = require('node:fs');
const { pipeline } = require('node:stream');
const { pipeline: pipelinePromise } = require('node:stream/promises');

const writable = fs.createWriteStream('./file');

const ac = new AbortController();
const signal = ac.signal;

const iterator = createIterator({ signal });

// 回调模式
pipeline(iterator, writable, (err, value) => {
  if (err) {
    console.error(err);
  } else {
    console.log(value, '返回的值');
  }
}).on('close', () => {
  ac.abort();
});

// Promise 模式
pipelinePromise(iterator, writable)
  .then((value) => {
    console.log(value, '返回的值');
  })
  .catch((err) => {
    console.error(err);
    ac.abort();
  });
  • 'data' 事件会立即开始发射,而不是等待调用 stream.read() 方法。需要执行一定工作量来决定如何处理数据的应用程序需要将读取的数据存储到缓冲区中,以免数据丢失。
  • stream.pause() 方法是建议性的,而不是强制保证的。这意味着即使流处于暂停状态,仍然需要准备好接收 'data' 事件。

在 Node.js 0.10 中,添加了 Readable 类。为了与旧版 Node.js 程序向后兼容,当添加 'data' 事件处理程序或调用 stream.resume() 方法时,Readable 流会切换到“流动模式”。其效果是,即使不使用新的 stream.read() 方法和 'readable' 事件,也不再需要担心丢失 'data' 块。

虽然大多数应用程序将继续正常运行,但这在以下条件下引入了一种边缘情况:

  • 未添加 'data' 事件监听器。
  • 从未调用 stream.resume() 方法。
  • 流未管道传输到任何可写目标。

例如,考虑以下代码:

// 警告!已损坏!
net.createServer((socket) => {

  // 我们添加了一个 'end' 监听器,但从不消费数据。
  socket.on('end', () => {
    // 永远不会到达这里。
    socket.end('The message was received but was not processed.\n');
  });

}).listen(1337);

在 Node.js 0.10 之前,传入的消息数据会被简单地丢弃。然而,在 Node.js 0.10 及更高版本中,socket 将永远保持暂停状态。

这种情况下的解决方法是调用 stream.resume() 方法来开始数据流:

// 解决方法。
net.createServer((socket) => {
  socket.on('end', () => {
    socket.end('The message was received but was not processed.\n');
  });

  // 开始数据流,将其丢弃。
  socket.resume();
}).listen(1337);

除了新的 Readable 流切换到流动模式外,还可以使用 readable.wrap() 方法将 0.10 之前风格的流包装在 Readable 类中。

如果内部读取缓冲区低于 highWaterMark,并且流当前未读取,则调用 stream.read(0) 将触发低级 stream._read() 调用。

虽然大多数应用程序几乎永远不需要这样做,但在 Node.js 中存在这种情况,特别是在 Readable 流类内部。

将零字节 <string><Buffer><TypedArray><DataView> 推送到非 object 模式的流会产生有趣的副作用。因为它 readable.push() 的调用,该调用将结束读取过程。然而,因为参数是空字符串,所以没有数据添加到 readable 缓冲区,因此用户没有什么可消费的。

通常,当前缓冲区的大小是相对于 highWaterMark字节 测量的。但是,在调用 setEncoding() 后,比较函数将开始按 字符 测量缓冲区的大小。

这在 latin1ascii 的常见情况下不是问题。但是,当处理可能包含多字节字符的字符串时,建议注意此行为。