Learn advanced node js, Using readable streams

In the last lesson we saw how we can create classes that extend stream dot readable to build our own readable streams. Now, sometimes you’ll build your own readable streams, but most of the time you’re going to use already existing stream types. You’ve already been doing this. Node JS comes with all types of readable streams. Http requests on the server and a response on the client, they’re readable streams. The file system has readable streams. Zipping and unzipping uses readable streams. TCP sockets, process stdin, not to mention the many MPMs that implement readable streams.

So let’s go ahead and implement our own instance of a readable stream that already exists. We’re going to go ahead and use the create readStream from the file system. So, I’m inside of an empty index dot js. You’ll find this inside of your exercise files for lesson two. Lesson two, chapter three within the start folder. And I’m going to start just by requiring the fs module. So here’s our file system module. And then I’m going to go ahead and create an instance of a new stream. We’ll call it the readStream. And I can use fs.createReadStream to create a readStream to a file.

Now, I just have to put the location of the file in. So I included a video under chapter two within your exercise files. So let me go ahead and back out of the start folder, back out of the chapter two lesson three folder, and load powder-day.mp4. So we’re going to go ahead and create a readStream to a video file. So now that we have a readStream, we want to listen for the same events. readStream.on and when a data event is raised, that’s when a chunk of data is going to be passed to the call back handler.

So we’ll go ahead and console log it. Reading little chunk, we’ll do a little slash n and then I will go ahead and do chunk. So we can see that chunk and see what it looks like. We also use the end event. So again, on any stream instance we can listen for an end event and that will tell us that the stream is over. So I’ll go ahead and console log readStream finish.

Great and another event that we really should be listening for, on read streams, are error events. So, I can also wire that up on error. Then, if an error occurs, it will be passed to this call back handler and we’re just going to go ahead and throw that on the console. Console log an error has occurred. And then we can console error, the error. Alright, so now we’re using an existing readable stream but you know that the interface is exactly like the one that we created.

We’re listening for specific events. The data event, the end event, and the error event. Let’s go out to the terminal and type node dot. And we can see that we’re reading little chunk and all of our chunks are these binary buffers. And all of this is just little binary bits of our video. That’s great. So, if we wanted to see the size of each of these bits instead of the binary tag out there, we can actually just look for chunked out length. So this will actually give us the size. Instead of doing that I’ll change this to size.

And I can come over here and run this again by typing node dot. And now we can see the size of each of these little bits. So here’s our whole video being streamed from beginning to end, bit by bit by bit until the stream has finished. So actually Node JS has read streams everywhere. Standard input or stdin is a read stream. So that’s why we can listen for data events. And with standard input, just like with every other stream, each chunk of data is actually being passed to this call back handler.

So, if a stream is in binary mode and it’s not utf8, don’t worry, we can still get the text of the chunk if the chunk is a string simply by invoking toString. I’m also going to invoke it trim, just to get rid of any leading or trailing spaces and we’ll go ahead and log this chunk just to see what it looks like. I will actually call this echo, and let’s save this. Come out to our terminal, clear everything. Oh, one other thing I’m going to do is comment outline six here just so that we don’t see all of those chunk lengths inside of the terminal.

So node dot to run the file. So our readStream has finished. But, stdin is still open. So if I type hello, you’ll see the echo world, you’ll see the echo. So you’ll notice that this stream is a little bit different. We’re still sending data chunk by chunk, but we have to ask for the next chunk. So this stream is in non-flowing mode. When a stream is in flowing mode, it automatically pushes each chunk of data into the pipeline. Non-flowing streams mean that we have to ask for the data. So we can actually enter any stream into non-flowing or flowing mode.

So let’s take the readStream, this is currently in flowing mode, and by invoking pause on it, we are now creating our readStream as a non-flowing mode stream. So that means we’re going to have to ask for bits of data. I’m actually going to do this right here inside of process standard input. So if I want to ask for a bit of data, I can call readStream dot read. Anytime we have any type of input from standard input, we can actually tell our readStream to read one chunk of data.

So let’s go ahead and comment on comment line six. I’ll come back over here to the terminal. Control c to stop that. Clear and then node dot. And what you’ll notice is, we haven’t read any of the video. We don’t see the chunk links. But as I hit enter, I’m asking for one little bit of the video, one bit at a time. So I have, by pausing the readStream, made it enter into non-flowing mode. So I’m going to hit Control c one more time to stop this. And what we’re going to do is, we’re going to listen inside a process stdin.

Encase the text and we’ll go ahead and call this chunk dot toString dot trim is equal to the word finish. Then we can actually enter any stream back into flowing mode by typing readStream of the name of the stream resume. So pause enters it into non-flowing mode and resume will convert it back into flowing mode. So let’s see what happens. Let’s go over here to our terminal. Type node dot to run the application.

So every time I hit enter, I’m reading one small chunk of the video. But, if I type finish and hit enter, the stream enters back into flowing mode and begins to push the rest of the chunks of the video automatically. Readable streams are everywhere. Learning to work with readable streams is an essential skill for a Node JS developer.

Leave a Reply

Notify of