Learn advanced node js, Readable streams

In the last lesson we discovered why streams are important. In the next several lessons we’re going to take a look at the various types of streams that you can use with nodejs. We’re going to start with readable streams. Now, a readable stream reads data from a source and then feeds it into a pipeline bit by bit. For instance, we can create a readable stream that can read from a source array. I’m inside of chapter two, chapter two lesson one, inside of the start folder, and in your index.js file you’ll find a sample array of strings. These are local mountain peaks here in the Sierra.

So what we’re goanna do is we’re goanna create a particular type of stream that can read the data from the array and then pass it along chunk by chunk. So, let’s see what that would look like. I’m going to go ahead and create a new instance of our peak stream, and we’re goanna use the new StreamFromArray class, and we’ll pass it the peaks. Now streams implement the event emitter, which means they can raise events. So we can listen for particular events. On a read stream we want to listen for data events.

So whenever a data event is raised, a little bit of data is passed to the callback function. We’ll call that a chunk. And what we’ll do is we’ll just take a look at what a chunk of data looks like by logging it. Another important read stream event is end. When the stream is finished reading, an end event will be raised. And when this is over we’ll just go ahead and console that log, done. Great. So now all we have to do is implement the class StreamFromArray.

So, we’re actually going to create a readable stream, so the first thing I’m going to do is get my readable stream from the stream module, and now I’m going to go down here and create our new class, StreamFromArray, and this is going to extend the readable stream. So we are actually going to create a readable stream. It will inherit that interface.

Whenever we extend the readable stream, we want to implement a read function. So the read function is what happens when this stream reads a chunk of data. So within this function, we want to read one line of the array, and then pass it on. So to that we’re going to need to create a constructor, and take in our array, so when we set up our peak stream, we pass in the peaks. I’m going to invoke super, that actually invokes the readable stream, so it sets up our stream object, and then I’m going to go ahead and add some values to this object, this.array is going to be the array that was passed here, and then we’re going to go ahead and make sure that we’re starting off at zero.

So the index, or the first item that we want to read, is zero. So, within the read method, what we want to do, is we actually want to push chunks of data into the stream by invoking this.push. So the first thing I need to do is get a chunk of data. I’m going to go ahead and create a chunk, and my chunk will be this.array at this.index. There we go. And instead of pushing data I’ll go ahead and push this chunk in here. And then, what we also want to do is make sure that we increment this.index.

So now we have a stream that will read from the array, but how do we know when the stream is over? So in order to do that we need to make sure the index isn’t out of bounds, so I’m going to say, if this.index is less than or equal to this.array.length then we know it’s in bounds, and we can use the index to grab an item from the array. Otherwise, it’s out of bounds, which means that we’ve already read all of the data in the array, so we want to notify this readable stream that it’s over.

So I’m going to go ahead and say this.push null, and by pushing null, or nothing, into our stream, that will signal to the readable that the stream is over. So let’s go ahead and save our application, and then we will go out here and start it by running node . So you can see that we’re reading our array, bit by bit, and all of these little buffers are the chunks. That means that this stream is operating in binary mode, so it’s reading each line of this array, and each chunk is showing us the binary data.

Now, streams have two modes, binary mode and object mode, and when they’re in binary mode, we can actually read data as binary or we can read it as a string. So in order to read this data as a string, all we need to do is set the encoding type to UTF-8. I’m going to go ahead and set it here, when I invoke the super readable class. So I can pass options in, and then coding is one of those options, and we can go ahead and pass in UTF-8, there we go, and I’ll go ahead and save this, and let’s go out to our terminal and run this one more time, and there we can see that all of our chunks are now strings.

So, when data is raised, we’re going to go ahead and console log each of these chunks and their strings. Now, there is one other type of mode, and that’s object mode. So we can pass binary data through a read stream, we can also pass string data by sending the encoding type, and I can also specify whether the object mode of our readable stream is on. So, when the stream is in object mode, it can actually pass any type of JavaScript object. So what I’m going to do is I’m going to actually make the chunk a JavaScript object.

So down here inside of the read function, just go ahead and give it a little object, and we’ll say data, and that will be our chunk, and then we’ll also pass forward the index. So now when we invoke this read stream, we’re still reading from the source array, but now we are sending the data out of this readable stream as an object. Let’s go over here to our terminal, and I will clear this, and type node . Whoops, looks like I’m missing something, I accidentally put semicolons here inside of my JSON, silly me, let me go ahead and add commas, there, and then let’s try this again, so from the terminal, I’ll type clear, and then node ., and now you can see that each bit of our stream, each bit of our array, is being read and then passed into the object.

Now, when our object stream is over, notice that that’s handled just a little but differently, pushing null does end the object event, but it also does send us one more object, so just bear that in mind. So readable streams can read data from any type of source and then feed it to anyone who is listening, chunk by chunk.


Please enter your comment!
Please enter your name here