Fail to Upload Large File to Google App Engine Managed VMs

After deploying a Node.js app with custom runtime to Google App Engine, when uploading file larger than 1MB, the server returns HTTP 413 status code, meaning the server is refusing to process the request because the uploading entity size is too large.

I had the following setup:

  • The App Engine application is Google managed
  • The application is Node.js app with custom Dockerfile
  • The application Docker container runs node command directly on port 8080

After accessing the server (which changed from Google managed to user managed), and checked the Docker containers:

1
2
$ sudo docker ps -a
gcr.io/google_appengine/nginx-proxy:latest      "/usr/sbin/nginx"

It looks like Nginx is running as the proxy to Node.js application. When uploading file that is larger than 1MB, the response is HTTP 413 error:

1
2
3
4
5
6
7
8
9
10
11
12
<html>
    <head>
        <title>413 Request Entity Too Large</title>
    </head>
    <body bgcolor="white">
        <center>
            <h1>413 Request Entity Too Large</h1>
        </center>
        <hr>
            <center>nginx</center>
        </body>
    </html>

And the Nginx access log from the server:

1
2
$ tail /var/log/nginx/error.log 
2015/06/25 12:00:00 [error] 7#0: *101623 client intended to send too large body: 1696805 bytes

By default, Nginx limits uploading size or client_max_body_size to be 1MB. The default size is just too small. A short video upload could easily exceed the limit. So, how to accept file upload larger than 1MB via Google Managed VMs in App Engine? Is there a way to change the value through app.yamlcustom runtime configuration file? Or a custom Nginx Docker container can be deployed along with the application Docker container?

Migrate to Cloud9 IDE

My blog is powered by Hexo:

Hexo is a fast, simple and powerful blog framework. You write posts in Markdown (or other languages) and Hexo generates static files with a beautiful theme in seconds. - https://hexo.io/docs/

To generate static files, I need a Node.js development environment. But as I move around from different computers and operating systems, this becomes inconvenient. I need a cloud development environment where I can access it from any device.

So, I have decided to move my blog to Cloud9 IDE, a development environment in the cloud. And this is the first blog from the environment.

Looking forward!

Require Modules from the Command Line

io.js v1.6.0 introduces a new command line option: -r or --require, which can be used to pre-load modules at startup, see:

https://github.com/iojs/io.js/blob/v1.x/CHANGELOG.md#2015-03-19-version-160-chrisdickinson

I can think of one interesting use case: overriding the default behavior or console.log to perform deep inspection:

1
2
3
4
5
6
7
// console.js
var util = require('util');
var log = console.log;
console.log = function (obj) {
  log(util.inspect(obj, { colors: true, depth: null }));
};

Now I can pre-load this script to have all console log show in better format:

1
$ iojs -r /path/to/console.js app.js

Time to find out more use cases.

Test Execution Order in Mocha

In synchronous tests, both describe and it statements are executed in the order they are laid out.

Test Foo should run before Bar:

1
2
3
4
5
6
7
describe('Suite', function () {
  it('Foo', function () {
  });
  it('Bar', function () {
  });
});

Suite Foo should run before Bar:

1
2
3
4
5
describe('Foo', function () {
});
describe('Bar', function () {
});

In asynchronous tests, the same ordering applies.

Test Foo should run before Bar even Foo takes much longer to execute, and Suite B should also run after Suite A:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
var should = require('should');
describe('Suite A', function () {
  it('Foo', function (done) {
    setTimeout(function () {
      (true).should.equal(true);
      done();
    }, 1000);
  });
  it('Bar', function () {
    (true).should.equal(true);
  });
});
describe('Suite B', function () {
  it('Foo', function () {
    (true).should.equal(true);
  });
});

Result:

1
2
3
4
5
6
7
8
9
Suite A
  ✓ Foo (1002ms)
  ✓ Bar 
Suite B
  ✓ Foo 
3 passing (1s)

This is the great feature in Mocha.

SuperTest: Listen at Random Port

In the example section of SuperTest (v0.15.0), it mentions something about ephemeral port:

You may pass an http.Server, or a Function to request() - if the server is not already listening for connections then it is bound to an ephemeral port for you so there is no need to keep track of ports. - https://github.com/visionmedia/supertest#example

Let’s dissect this sentence. There are a few key concepts to grasp:

  1. The request function will accept either a function or an http.Server object.
  2. When does the server not listening for connections?
  3. What is an ephemeral port?

Taking a peek at the index.js file in SuperTest source code, it is easy to see that it accepts both function and http.Server object, but prefer latter:

1
2
3
4
5
6
7
8
9
10
11
// See `index.js` in [SuperTest] source code.
if ('function' == typeof app) app = http.createServer(app);
// See `lib/test.js` in [SuperTest] source code.
Test.prototype.serverAddress = function (app, path) {
  var addr = app.address();
  if (!addr) this._server = app.listen(0);
  var port = app.address().port;
  var protocol = app instanceof htts.Server ? 'https' : 'http';
  return protocol + '://1270.0.0.1:' + port + path;
});

Why it does this? This has to do with Express:

1
2
3
4
5
6
7
8
> typeof require('http').createServer()
'object'
> typeof require('express')()
'function'
> typeof require('koa')()
'object'
> require('http').createServer(require('express')()).address()
null

Only when initiating an Express app, it returns a function, not an object. And follows up on lib/test.js, when SuperTest detects the created server is yet to bind to any port number, it will invoke app.listen(0), so called ephemeral port. In fact, it is just a random port.

When something is ephemeral, it last for a very short time. When allowing a server to accept connections, we usually do is setting the server to listen on a specific port:

1
app.listen(3001);

What if setting this to 0 like above or omit this port number?

Locate HTTP Port Number from http.Server Instance

This is a bare minimal HTTP server:

1
require('http').createServer().listen(port);

If the port number is evaluated to be falsy, such as 0, null, undefined or an empty string, a random port will be assigned.

Begin accepting connections on the specified port and hostname. If the hostname is omitted, the server will accept connections on any IPv6 address (::) when IPv6 is available, or any IPv4 address (0.0.0.0) otherwise. A port value of zero will assign a random port. - https://iojs.org/api/http.html#http_server_listen_port_hostname_backlog_callback

We can use command line tool such as netstat to find out the port number:

1
2
$ netstat -ltpn | grep node
tcp6     0    0 :::44055             :::*            LISTEN      5624/node

However, it will be very convenient without relying on external tools.

Luckily, we can find out the port number by using the address method:

1
require('http').createServer().listen().address()

The result will be similar to:

1
{ address: '::', family: 'IPv6', port: 44055 }

Furthermore, it works with all popular libraries that create an http.Server instance:

Use Array of Middleware in Koa Router

Koa Router v4.2 does not support an array of middleware, multiple middleware must be entered one by one:

1
app.get('/foo', middleware1, middleware2);

Using an array is much cleaner than multiple arguments.

1
app.get('/foo', [middleware1, middleware2]);

But the above will throw error:

1
Error: GET: `middleware` must be a function, not `object`

This can be easily fixed by using Koa Compose:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
var app = require('koa')();
var router = require('koa-router');
var compose = require('koa-compose');
app.use(router(app));
app.get('/foo', compose([
  function *(next) {
    this.body = {};
    yield next;
  },
  function *(next) {
    this.body.foo = 'foo';
    yield next;
  },
  function *(next) {
    this.body.bar = 'bar';
    yield next;
  }
]));

Hence:

1
app.get('/foo', compose([middleware1, middleware2]));