DevOps

Fail to Upload Large File to Google App Engine Managed VMs

After deploying a Node.js app with custom runtime to Google App Engine, when uploading file larger than 1MB, the server returns HTTP 413 status code, meaning the server is refusing to process the request because the uploading entity size is too large.

I had the following setup:

  • The App Engine application is Google managed
  • The application is Node.js app with custom Dockerfile
  • The application Docker container runs node command directly on port 8080

After accessing the server (which changed from Google managed to user managed), and checked the Docker containers:

1
2
$ sudo docker ps -a
gcr.io/google_appengine/nginx-proxy:latest      "/usr/sbin/nginx"

It looks like Nginx is running as the proxy to Node.js application. When uploading file that is larger than 1MB, the response is HTTP 413 error:

1
2
3
4
5
6
7
8
9
10
11
12
<html>
    <head>
        <title>413 Request Entity Too Large</title>
    </head>
    <body bgcolor="white">
        <center>
            <h1>413 Request Entity Too Large</h1>
        </center>
        <hr>
            <center>nginx</center>
        </body>
    </html>

And the Nginx access log from the server:

1
2
$ tail /var/log/nginx/error.log 
2015/06/25 12:00:00 [error] 7#0: *101623 client intended to send too large body: 1696805 bytes

By default, Nginx limits uploading size or client_max_body_size to be 1MB. The default size is just too small. A short video upload could easily exceed the limit. So, how to accept file upload larger than 1MB via Google Managed VMs in App Engine? Is there a way to change the value through app.yamlcustom runtime configuration file? Or a custom Nginx Docker container can be deployed along with the application Docker container?

Install Docker and then Run Docker Commands without Sudo

Commands:

1
2
3
4
$ curl -sSL https://get.docker.io/ubuntu/ | sudo sh
$ sudo usermod -a -G docker $USER
$ sudo service docker restart
$ logout

Explanation:

  1. One line command installation
  2. Add the user to the group docker
  3. Restart the daemon
  4. Make sure to logout and log back in, in order to sudoless Docker taking effect.

Vagrant

  1. What is Vagrant?
  2. What are the main features?
  3. Why do they build it?
  4. How would it benefit me?
  5. Where does this fit into the stack?

What is Vagrant?

Vagrant is a configuration management tool for creating, configuring, and managing complete development environments in virtual machine.

What are the main features?

Development environment version control

Configuration for setting up development environment is in code (Vagrantfile), so the development environment can be version controlled.

Consistent but distributed development environments

Consistent development environment, build one development environment, and distribute to the rest of the team. Create identical development environment for everyone on the team.

Distributed development environment, but linked by source control. “I like to think of Vagrant as the Git of development clouds. Centralized development and test environments create bottlenecks. Vagrant lets developers work at their own pace and in their own environment, while keeping all the environment synchronized with each other.” (By Jeff Sussna) [1].

Disposable computing resource

“Vagrant lowers development environment setup time, increases development/production parity, and brings the idea of disposable compute resources down to the desktop.” [1]

“At the end of the day, Vagrant can suspend, halt, or destroy the development environment, keeping the overall system clean. Never again can developers forget to shut down a stray server process and waste precious compute resources.” [1]

Simple development environment setup

Forget about README or INSTALL instructions, type vagrant up and you are good to go.

“Say goodbye to ‘works on my machine’ bugs.” [2] For designers, “no more bothering other developers to help you fix your environment so you can test designs. Just check out the code, vagrant up, and start designing.” [2]