Purpose

Just a learning note here. There are already tons of glorious posts talking about this topic, so don’t expect to see something super edge, extreme frontier technology here.

What is asynchronous?

Before moving on, knowing what exactly is Asynchronous is essential.

Synchronous, contrary to Asynchronous, means blocking. That means computer can only run a task at one time. Before finishing this one, another will not start.

When we mention about Asynchronous, we are referring to non-blocking, which means, computer runs a tasks without “blocking” other tasks.

And don’t confuse Asynchronous with words Concurrency or Parallel, they are different things.

Parallel usually refers to true multi-tasking, which means computer distributes several tasks among processors. In this way, tasks are executed in real simultaneously.

Concurrent is a concept of having a group of tasks sharing an execution thread with each other to let them run simultaneously. Asynchronous and Synchronous are just descriptions of state between tasks.

How does it work in JavaScript?

JavaScript is an event-driven language and it uses an event-loop to get codes run in asynchronous. JavaScript is also called single-threaded language because it can only run a piece of code at a time.

So, does that mean JavaScript is really a single-threaded language? Hmm, not accurate. No matter what platform JavaScript is on, either client side(browsers) or server side(Node.js), there is actually a thread pool underneath it.

After digging here and there on the web, I realized to explicitly explain how does it work is a huge thing, and it really hurts your head. For the sake of simplicity, here is my simple version:

JavaScript on both client and server side use one thread for code execution, another one for taking care of event loop, and all of others for handling events and triggering handlers. Whenever interpreter runs into async code like button.click()(UI events) or fs.readFile() (I/O events), it pushes callbacks into an event queue, and after all things in main function are done, it pops callbacks in queue to get these executed.

And…that’s it. :P

To get more illustration, visit this awesome site. The website visualizes how event loop works in JavaScript in a very clear manner, and if it still can’t satisfy you, scroll down to References & Resources for further reading.

Callback

So, how do we take advantage of async? Well, the answer is pretty simple:

1
2
3
4
5

setTimeout(function(){
  console.log("Wolrd!");
}, 1000);

console.log("Hello");

When you set an callback on an event, you are using async. Another common practice is using ajax:

1
2
3
4
5
6
7
8
9
10

var ajax = new XMLHttpRequest();
ajax.open("GET", "http://www.example.com");
ajax.addEventListener("readystatechange", function(){
  if(ajax.readyState === 4){
    if(ajax.state === 200){
      console.log(ajax.responseText);
    }
  }
});
ajax.send();

Callback really fulfills our need for async, but when our program grows larger, our callback logic may become complicated as well, which results in some nasty and hard-to-maintained code.

1
2
3
4
5
6
7
8
9
10
11

request("http://www.example.com/api/data1", function(result, status){
  if(status === "OK"){
    request("http://www.example.com/api/data2", function(result, status){
      if(status === "OK"){
        request("http://www.example.com/api/data3", function(result, status){
          console.log(result);
        });
      }
    });
  }
});

What worse than callback hell is that you can’t even catch exceptions which was thrown in callback. This will lead our program unpredictable.

1
2
3
4
5
6
7
8
9
10

try{
  setTimeout(function(){
    throw new Error("you can't catch me!");
  }, 1000);
}
catch(e){
  console.log("Caught one!");
  console.log(e);
}
// Here is actually exception was thrown.

Luckily, there is a new feature called Promise which can save you from this mess.

Promise

Promise is ES6’s feature. With this, you can simplify complicated callbacks into human readable one:

1
2
3
4
5
6
7
8

// if request returns Promises
request("http://www.example.com/api/data")
.then((result) => {
  return result.status === "OK" ? request("http://www.example.com/api/data2") : throw new Error("Something went wrong while requesting data 2");
})
.then((result) => {
  return result.status === "OK" ? request("http://www.example.com/api/data3") : throw new Error("Something went wrong while requesting data 3");
});

This is much easier to read, right? Did I just mention about exception?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

new Promise((resolve, reject) => {
  let result = doSomething();
  if(result){
    resolve(result);
  }
  else {
    reject(result.reason);
  }
})
.then((result) => {
  console.log("Succeed: " + result);
})
cathc((reason) => {
  console.log("Oops, something went wrong: " + reason);
});

In this way you can solve async problem elegantly.

Using Babel like me? Insert require("babel/polyfill") at the head of your code to enable this goodie.

If in your environment ES6 is not allowed, don’t be sad, there are tons of libraries implementing this awesome feature!

• Bluebird
• jQuery - Deferred
• Q
• when
• …I only named a few, google js promise lib for more!

Async/Await

What if you could write async code like it was sync? ES7 provides a new feature called async/await. By combining it with promise, you can write async code in sync style!

1
2
3
4
5
6
7
8
9
10
11
12
13

IamAsync();

console.log("Hello");

async function IamAsync(){
  let result = await new Promise((resolve, reject) => { setTimeout(resolve, 3000, "World!"); });
  return result;
}

// output:
// Hello
// (wait for 3 seconds...)
// World!

Currently async/await is only available in Babel by running with babel source.js -o compiled.js --stage 0 and beware that it is still in experiment, everything is subject to change.

References & Resources

• How to articulate the difference between asynchronous and parallel programming?
• What is the difference between concurrency, parallelism and asynchronous methods?
• How does a single thread handle asynchronous code in JavaScript?
• Get the door, Javes! Understanding Sync & Async in JavaScript
• Philip Roberts: What the heck is the event loop anyway? JSConf EU 2014
• Promises for asynchronous programming
• A Primer on ES7 Async Functions
• ES7 Async/Await
• The Evolution of Asynchronous JavaScript
• 2. What is Node.js?
• Is javascript guaranteed to be single-threaded?
• How does a single thread handle asynchronous code in JavaScript?

Polling, long-polling, websocket and blah blah blah, these tech buzz words have existed for many years, and the shameful thing is, as an frontend engineer(at least I think I am), I can’t even explain to someone else what the heck are they clearly, and that’s why I come up and write it down.

I will use Nodejs as backend with below demonstrations.

Polling

The most common technique when a client wants to fetch data from server.

Client establishes a connection between server, sends request, server responds and closes this connection.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

var http = require("http");
var fs = require("fs");

var index = null;

fs.readFile("./index.html", function(err, file){
  if(err){
    throw err;
  }
  index = file.toString();
});

http.createServer(function(req, res){
  if(req.url === "/"){
    res.writeHeader(200, {
      "content-type": "text/html"
    });
    res.end(index);
  }
  else if(req.url === "/polling"){
    res.writeHeader(200, {
      "content-type": "text/plain"
    });
    res.end("polling~");
  }
}).listen(3000);

console.log("server is running...");

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

<!DOCTYPE html>
<html>
  <head>
    <title>index</title>
  </head>
  <body>
    <div id="text"></div>
    <script>
      var text = document.getElementById("text");
      var ajax = new XMLHttpRequest();
      ajax.addEventListener("readystatechange", function(){
        if(ajax.readyState === 4){
          if(ajax.status === 200){
            text.appendChild(document.createTextNode(ajax.responseText));
          }
        }
      });
      ajax.open("GET", "http://localhost:3000/polling");
      ajax.send();
    </script>
  </body>
</html>

Long-Polling

In some applications like stock market, online games and friends status, you need to get realtime data from server. Before Websocket was born, web programmers usually use long-polling ajax technique to simulate realtime data exchange.

The difference between traditional polling and long-polling is that long-polling sends a request, and waits until server responds and closes connection and finally, opens another one. When repeatedly doing this, we call it keep-alive connection.

The Pros are:

• Server side code remains unmodified in most case. Only need to change client side code.
• Our data updates in a fancy manner, which leads our customers happy, profit!

The Cons are:

• Opening and closing connections repeatedly means unnecessary bandwidth cost.
• Client sometimes gets unexpected results because long-polling fires requests regularly on a fixed time, however server might respond in longer or shorter time than what client expected.

Let’s add following codes to simulate time of processing.

1
2
3
4
5
6
7
8
9
10

...
else if(req.url === "/polling"){
  setTimeout(function(){
    res.writeHeader(200, {
      "content-type": "text/plain"
    });
    res.end("polling~");
  }, 2500);
}
...

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

<!DOCTYPE html>
<html>
  <head>
    <title>index</title>
  </head>
  <body>
    <div id="text"></div>
    <script>
      var text = document.getElementById("text");

      (function longPolling(){
        setTimeout(function(){
          var ajax = new XMLHttpRequest();
          ajax.addEventListener("readystatechange", function(){
            if(ajax.readyState === 4){
              if(ajax.status === 200){
                text.appendChild(document.createTextNode(ajax.responseText));
                longPolling();
              }
            }
          });
          ajax.open("GET", "http://localhost:3000/polling");
          ajax.send();
        }, 3000);
      })();
    </script>
  </body>
</html>

Websocket

Luckily, we don’t need to simulate realtime anymore (unless you need to support old browsers) because now we get the real realtime technique with Websocket!

Websocket lets client and server to communicate with each other asynchronously, which means, client and server can send and get data at the same time without waiting for each other (non-blocking).

Modern browsers (not you IE8 and IE9!) this day all support native Websocket, but for the cross platform compatibility, many people use 3rd party libraries to get rid of handling compatibility themselves.

In this example I use socket.io as our Websocket helper: npm install socket.io

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

var app = require("http").createServer(handler);
var io = require("socket.io")(app);
var fs = require("fs");

app.listen(3000);

function handler(req, res){
  fs.readFile(__dirname + "/index.html", function(err, file){
    if(err){
      res.writeHeader(500, {
        "content-type": "text/plain"
      });
      res.end(filename + " : not found");
    }

    res.writeHeader(200, {
      "content-type": "text.html"
    });
    res.end(file);
  });
}

io.on("connection", function(socket){
  socket.emit("server_says", "Hello Client");
  socket.on("client_says", function(data){
    console.log(data);
  });
});

console.log("server is running...");

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

<!DOCTYPE html>
<html>
  <head>
    <title>index</title>
    <script src="https://cdn.socket.io/socket.io-1.3.7.js"></script>
  </head>
  <body>
    <div id="text"></div>
    <button id="button">Hello Server!</button>
    <script>
      var text = document.getElementById("text");
      var button = document.getElementById("button");
      var socket = io("http://localhost:3000");
      socket.on("server_says", function(data){
        text.appendChild(document.createTextNode(data));
      });

      button.addEventListener("click", function(e){
        socket.emit("client_says", "Hello Server!");
      });
    </script>
  </body>
</html>

As you can see in Chrome developer’s network tool, websocket helps us maintain a persistent connection.

WebRTC

Unlike Websocket’s client-to-server structure, Web Real-Time Communication is peer-to-peer structure which allows browsers to exchange data like video, audio with each other directly. Browser-to-browser, in this fashion, data exchange is more efficient.

So, regarding the tag of “browser-to-browser”, that means we don’t need a server, right? Sadly, you do, but server here merely acts as a connector, not a proxy, as a result, data won’t be passed through server.

WebRTC currently is not being widely supported by all browsers. See here.

Here I use peerjs as 3rd party helper to overcome cross browser compatibility.

npm install peer for our server.

npm install peerjs for client side, but here I use direct link instead:

<script src="http://cdn.rawgit.com/peers/peerjs/master/dist/peer.js"></script>

1
2
3
4

var PeerServer = require('peer').PeerServer;
var server = PeerServer({port: 3000, path: "/"});

console.log("server is running...");

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

<!DOCTYPE html>
<html>
  <head>
    <title>index</title>
    <script src="http://cdn.rawgit.com/peers/peerjs/master/dist/peer.js"></script>
  </head>
  <body>
    <div id="output"></div>
    <script>
      var output = document.getElementById("output");
      var peer1 = new Peer("Mike", {host: "localhost", port: 3000, path: "/"});
      var peer2 = new Peer("Bob", {host: "localhost", port: 3000, path: "/"});

      peer1.on("open", function(){
        appendMessage(peer1.id + " is ready to connect with others.");
      });

      peer1.on("connection", function(c){
        c.on("open", function(){
          c.send("Hello " + c.peer);
        });
        c.on("data", function(data){
          appendMessage(data);
        });
      });

      var c = peer2.connect(peer1.id);
      c.on("data", function(data){
        appendMessage(data);
      });

      setTimeout(function(){
        c.send("Hello " + c.peer);
      }, 1000);

      function appendMessage(message){
        var p = document.createElement("p");
        p.appendChild(document.createTextNode(message));
        output.appendChild(p);
      }
    </script>
  </body>
</html>

Server Push

Unlike Websocket which opens a duplex connection for client and server, server push (aka server sent) only opens a unidirectional connection for client and server. Server, as an message sender, only push message to client. Client, as a message receiver, only accepts message from server.

Since there is a Websocket for duplex connection, why would someone only want one-way connection? In come scenarios like news feeds or friends notifications, client requests only once for page, and waits for following responses from server.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

var http = require("http");
var fs = require("fs");

http.createServer(function(req, res){
  if(req.headers.accept === "text/event-stream"){
    if(req.url === "/events"){
      res.writeHeader(200, {
        "content-type": "text/event-stream"
      });
      setInterval(function(){
        res.write("data: Hello Server Push!\n\n");
      }, 2000);
    }
  }
  else {
    res.writeHeader(200, { "content-type": "text/html"});
    res.write(fs.readFileSync(__dirname + "/index.html"));
    res.end();
  }
}).listen(3000);

console.log("server is running...");

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

<!DOCTYPE html>
<html>
  <head>
    <title>index</title>
  </head>
  <body>
    <div id="output"></div>
    <script>
      var output = document.getElementById("output");
      var es = new EventSource("/events");

      es.addEventListener("message", function(e){
        appendMessage(e.data);
      });

      function appendMessage(message){
        var p = document.createElement("p");
        p.appendChild(document.createTextNode(message));
        output.appendChild(p);
      }
    </script>
  </body>
</html>

References

• stackoverflow: websocket vs long-polling vs webrtc vs server sent events
• simple long-polling with javascript and jquery
• stackoverflow: what are websockets, sse and comet?
• finding the right nodejs websocket implementation
• MDN: writing websocket client applications
• webRTC on wikipedia
• what is webrtc and how does it work
• peejs.com
• Stream Updates with SSE