Archives

Amazon API Gateway No Integration Defined for Method

Encountered the following problem when deploying with the Serverless Framework (v1.27.3) to Amazon API Gateway:

1
2
3
4
CloudFormation - CREATE_IN_PROGRESS - AWS::ApiGateway::Deployment - ApiGatewayDeployment
CloudFormation - CREATE_FAILED - AWS::ApiGateway::Deployment - ApiGatewayDeployment
An error occurred: ApiGatewayDeployment - No integration defined for method

The problem was not the bug from the Serverless side, but was originated with manually created resource without integration defined for the POST method:

1
2
3
$ aws apigateway get-integration --rest-api-id xxxxxxxxxx --resource-id 0xxxxx --http-method post
An error occurred (NotFoundException) when calling the GetIntegration operation: No integration defined for method

After removing the resource. The deployment works again.

In conclusion, if there is a resource and one of its method has no integration, the REST API cannot be deployed. Either removing the resource or creating an integration will resolve the problem.

The following script will look for a REST API for no integration error:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
#!/bin/sh
#
# Search Amazon API Gateway for resources that have no integration defined for
# method.
API_NAME=example.com
REST_API_ID=$(aws apigateway get-rest-apis --query='items[?name==`'$API_NAME'`].id | [0]' --output=text)
RESOURCES=$(aws apigateway get-resources --rest-api-id=$REST_API_ID --query='items[*].id' --output=text)
for resource in $(echo "$RESOURCES")
do
for method in $(aws apigateway get-resource \
--query='resourceMethods && keys(resourceMethods)' \
--output=text \
--rest-api-id=$REST_API_ID \
--resource-id=$resource)
do
if [ "$method" != "None" ]
then
aws apigateway get-integration \
--rest-api-id=$REST_API_ID \
--resource-id=$resource \
--http-method=$method > /dev/null
if [ $? -ne 0 ]
then
aws apigateway get-resource \
--rest-api-id=$REST_API_ID \
--resource-id=$resource
fi
fi
done
done

References:

jq: Filter JSON Array by Value

Print the JSON object in an array where the object key is equal to the specified value with jq:

1
2
3
4
$ echo '[{"key":"foo"},{"key":"bar"}]' | jq '.[] | select(.key == "foo")'
{
"key": "foo"
}

Reference:

1
2
$ jq --version
jq-1.5

Avoiding Dependencies Reinstall When Building NodeJS Docker Image

When I first started writing Dockerfile for building a NodeJS app, I will do:

1
2
3
WORKDIR /app
COPY . /app
RUN npm install

Sure, this is very simple, but every time a change in the source file, the entire dependency tree needs to be re-installed. The only time you need to rebuild is when package.json changes.

One trick is to move package.json elsewhere, and build the dependencies, then move back:

1
2
3
4
5
WORKDIR /app
ADD package.json /tmp
RUN cd /tmp && npm install
COPY . /app
RUN mv /tmp/node_modules /app/node_modules

However, if there are a lot of depending packages with lots of files, the last step will take a long time.

How about using symbolic link to shorten the time?

1
2
3
4
5
WORKDIR /app
ADD package.json /tmp
RUN cd /tmp && npm install
COPY . /app
RUN ln -s /tmp/node_modules /app/node_modules

Well, it works, but not every NPM package is happy about it. Some will complain about the softlink, result in errors.

So, how can we reuse caches as much as possible, and:

  1. Do not need to reinstall dependencies
  2. Do not use symbolic link

Here is a solution, just move down the COPY statement:

1
2
3
4
5
6
7
8
9
10
11
12
13
# Set the working directory, which creates the directory.
WORKDIR /app
# Install dependencies.
ADD package.json /tmp
RUN cd /tmp && npm install
# Move the dependency directory back to the app.
RUN mv /tmp/node_modules /app
# Copy the content into the app directory. The previously added "node_modules"
# directory will not be overridden.
COPY . /app

With this configuration, if package.json stays the same, only the last cache layer will be rebuilt when source codes change.

Wrapping Line by Line JSON to an Array

Some utilities print out JSON data (object) line by line. Line by line JSON is not really a valid JSON format, need to wrap it in bracket as an array.

For example, here are a few lines of JSON objects:

1
2
3
4
$ for i in $(seq 3); do echo '{"id":'$i'}'; done
{"id":1}
{"id":2}
{"id":3}

To combine them into a single array, we can use jq‘s -s, —slurp option:

1
2
$ for i in $(seq 3); do echo '{"id":'$i'}'; done | jq -cM -s
[{"id":1},{"id":2},{"id":3}]

The option reads the entire input stream into a large array. This is good until the input stream is too large to process, because jq is not producing output while slurping up the input. If the input is line by line JSON, and all you want to do is wrapping it into an array (with or even without the trailing newline), we can simply do the following:

1
2
3
$ for i in $(seq 3); do echo '{"id":'$i'}'; done |\
awk 'BEGIN { printf "["; getline; printf "%s", $0 } { printf ",%s", $0 } END { print "]" }'
[{"id":1},{"id":2},{"id":3}]

This is with the newline printed. And it will solve the problem of a very large input. Data will simply stream out as it comes it. Try it with a forever while loop.

There is another option in jq named —stream, and it seems to be doing the same. But option —slurp overrides —stream, and the option itself is already so complicated.

In conclusion, you can use jq -s to wrap line by line JSON into an array. If you want to be safe, just use the awk example above.

Gitignore: Excluding Everything Except a Specific Directory

I have the following directory structure:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$ tree -aF src/
src/
├── github.com/
│   ├── git/
│   │   └── git/
│   │   └── file
│   └── realguess/
│   ├── git/
│   │   └── file
│   └── hello/
│   └── hello.go
└── .gitignore
6 directories, 4 files

My intention is to keep the projects in src/github.com/realguess/ only. So, I have added the following gitignore file:

1
2
3
$ cat src/.gitignore
/github.com/
!/github.com/realguess/

But the directory is ignored:

1
2
$ git check-ignore -v github.com/realguess/
src/.gitignore:1:/github.com/ github.com/realguess/

It is not possible to re-include a file if a parent directory of that file is excluded. Git doesn’t list excluded directories for performance reasons, so any patterns on contained files have no effect, no matter where they are defined.[^1]

Uh-oh! The directory github.com/ is excluded. Therefore, it is not possible to re-include a file github.com/realguess (directory is a file). To fix it, just need to add the wildcard to the end of the parent directory:

1
2
3
$ cat src/.gitignore
/github.com/*
!/github.com/realguess/

By adding the wildcard character, it matches every file (including hidden ones) or directory under github.com/ directory, the directory itself as the parent of github.com/realguess/ is not excluded. Then, we use the negation pattern to re-include github.com/realguess/.

Double check it:

1
2
$ git check-ignore -v github.com/realguess/
1

The debugging command returns non-zero status code, which indicates that the directory is not ignored.

Registering a Domain with Amazon Route 53 via CLI

Register a domain with Amazon Route 53 via CLI is very straightforward, the only complicate thing is to generate the acceptable JSON structure, then somehow enter the JSON string in the command line without meddling from the shell.

The command is:

1
$ aws route53domains register-domain

The documentation can be found from the AWS CLI Command Reference or issuing:

1
$ aws route53domains register-domain help

Most the command line options require simple string values, but some options like --admin-contact requires a structure data, which is complex to enter in the command line. The easiest way is to forget all other options and use --cli-input-json to get everything in JSON.

First let’s generate the skeleton or template to use:

1
$ aws route53domains register-domain --generate-cli-skeleton

That should output something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
{
"DomainName": "",
"IdnLangCode": "",
"DurationInYears": 0,
"AutoRenew": true,
"AdminContact": {
"FirstName": "",
"LastName": "",
"ContactType": "",
"OrganizationName": "",
"AddressLine1": "",
"AddressLine2": "",
"City": "",
"State": "",
"CountryCode": "",
"ZipCode": "",
"PhoneNumber": "",
"Email": "",
"Fax": "",
"ExtraParams": [
{
"Name": "",
"Value": ""
}
]
},
"RegistrantContact": {
"FirstName": "",
"LastName": "",
"ContactType": "",
"OrganizationName": "",
"AddressLine1": "",
"AddressLine2": "",
"City": "",
"State": "",
"CountryCode": "",
"ZipCode": "",
"PhoneNumber": "",
"Email": "",
"Fax": "",
"ExtraParams": [
{
"Name": "",
"Value": ""
}
]
},
"TechContact": {
"FirstName": "",
"LastName": "",
"ContactType": "",
"OrganizationName": "",
"AddressLine1": "",
"AddressLine2": "",
"City": "",
"State": "",
"CountryCode": "",
"ZipCode": "",
"PhoneNumber": "",
"Email": "",
"Fax": "",
"ExtraParams": [
{
"Name": "",
"Value": ""
}
]
},
"PrivacyProtectAdminContact": true,
"PrivacyProtectRegistrantContact": true,
"PrivacyProtectTechContact": true
}

Fill in the fields, leave out the ones not needed:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
{
"DomainName": "example.com",
"DurationInYears": 1,
"AutoRenew": true,
"AdminContact": {
"FirstName": "Joe",
"LastName": "Doe",
"ContactType": "PERSON",
"AddressLine1": "101 Main St",
"City": "New York",
"State": "NY",
"CountryCode": "US",
"ZipCode": "10001",
"PhoneNumber": "+1.8888888888",
"Email": "[email protected]"
},
"RegistrantContact": {
"FirstName": "Joe",
"LastName": "Doe",
"ContactType": "PERSON",
"AddressLine1": "101 Main St",
"City": "New York",
"State": "NY",
"CountryCode": "US",
"ZipCode": "10001",
"PhoneNumber": "+1.8888888888",
"Email": "[email protected]"
},
"TechContact": {
"FirstName": "Joe",
"LastName": "Doe",
"ContactType": "PERSON",
"AddressLine1": "101 Main St",
"City": "New York",
"State": "NY",
"CountryCode": "US",
"ZipCode": "10001",
"PhoneNumber": "+1.8888888888",
"Email": "[email protected]"
},
"PrivacyProtectAdminContact": true,
"PrivacyProtectRegistrantContact": true,
"PrivacyProtectTechContact": true
}

Now let’s register by running through jq to clean up extra blanks and end of line characters, and then pipe to xargs command by using new line character as the delimiter instead of a blank by default:

1
2
3
4
5
$ cat input.json | jq -cM . | \
xargs -d '\n' aws route53domains register-domain --cli-input-json
{
"OperationId": "00000000-0000-0000-0000-00000000000"
}

Dockerizing Redmine

Migrate an existing Redmine application into a Docker based container. There are two main steps:

  1. Migrate MySQL database
  2. Migrate Redmine application

MySQL

Create an archive from the source data directory, where data to be migrated:

1
$ sudo tar -C /var/lib/mysql -czf ~/mysql.tar.gz .

The directory /var/lib/mysql is the default location where all database related data are stored.

Download the archive:

1
$ scp example.com:~/mysql.tar.gz .

Extract to the directory where we will map the host volume to the guest:

1
$ sudo mkdir -p /srv/mysql && sudo tar -C $_ -xzf mysql.tar.gz

Download the latest (5.7.16 make sure it is the latest, because 8.x branch might not work) MySQL Docker image:

1
$ docker pull mysql && docker pull mysql:5.7.16

Run a MySQL container with the data directory mounted from the host:

1
2
3
4
5
6
$ docker run \
--detach \
--name mysql \
--restart always \
--volume /srv/mysql:/var/lib/mysql \
mysql

Environment variables (MYSQL_ROOT_PASSWORD, MYSQL_DATABASE, MYSQL_USER, or MYSQL_PASSWORD) become unnecessary:

Do note that none of the variables below will have any effect if you start the container with a data directory that already contains a database: any pre-existing database will always be left untouched on container startup.[^1]

Now I should be able to access my existing database:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ docker exec -it mysql sh -c 'mysql -uredmine -p redmine'
Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.7.16 MySQL Community Server (GPL)
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>

But when accessing MySQL database from another host, we have to update the permissions. Let’s log into MySQL with root user:

1
$ docker exec -it mysql sh -c 'mysql -uroot -p mysql'

Issue SQL commands:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
mysql> SELECT Host, User FROM db WHERE Db='redmine';
+-----------+---------+
| Host | User |
+-----------+---------+
| localhost | redmine |
+-----------+---------+
1 row in set (0.00 sec)
mysql> SELECT Host, User FROM user WHERE User='redmine';
+-----------+---------+
| host | user |
+-----------+---------+
| localhost | redmine |
+-----------+---------+
1 row in set (0.00 sec)

Only localhost is allowed for both the database and the user. This is problematic. How does one know the host of the container yet to be created? The easiest way is to allow access from all hosts:

1
2
3
4
5
6
7
mysql> UPDATE db SET Host='%' WHERE Db='redmine';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> UPDATE user SET Host='%' WHERE User='redmine';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

We can restrict to a set of IP addresses such as 172.17.0.% or sub domains %.example.com. Docker networking uses 172.17.0.x range.

The better way is that when linking containers, should be able to update mysql container /etc/hosts file to include a redmine container field. Then, a wild card is not necessary.

But having to allow all hosts to connect is fine, because all hosts do not mean all hosts. The port is not bind or map to any open port in the host machine. Only the host (172.17.0.1) and other Docker containers inside the host can discover the service provided. Other machines or containers outside the host is not able to access at all.

Restart the container to have the update taking effect:

1
$ docker restart mysql

Redmine

For a basic Redmine setup, there aren’t much to migrate, mainly just three directories, config, files, and log, others are application data. Configuration could be done via environment variables, and if logs weren’t important to migrate, we just need to move the files directory, which is used for file upload.

A CLI Method to Check SSL Certificate Expiration Date

I know that browser does this automatically, but it might come in handy if you need to check the expiration date of a SSL certificate through CLI. The key is openssl, OpenSSL command line tool.

1
2
3
$ echo | openssl s_client -connect example.com:443 2> /dev/null | \
openssl x509 -noout -enddate
notAfter=Nov 28 12:00:00 2018 GMT

The command is consisted of two parts:

  • Retrieve SSL certificate from the server
  • Extract the expiration date data

The openssl program is a command line tool for using the various cryptography functions of OpenSSL’s crypto library from the shell. It can be used for[^1]

  • Creation and management of private keys, public keys and parameters
  • Public key cryptographic operations
  • Creation of X.509 certificates, CSRs and CRLs
  • Calculation of Message Digests
  • Encryption and Decryption with Ciphers
  • SSL/TLS Client and Server Tests
  • Handling of S/MIME signed or encrypted mail
  • Time Stamp requests, generation and verification

What we need here is to perform SSL/TLS Client and Server Tests.

s_client is one of the standard commands of openssl command line tool:

This implements a generic SSL/TLS client which can establish a transparent connection to a remote server speaking SSL/TLS. It’s intended for testing purposes only and provides only rudimentary interface functionality but internally uses mostly all functionality of the OpenSSL ssl library.[^1]

Dig deeper into s_client command:

The s_client command implements a generic SSL/TLS client which connects to a remote host using SSL/TLS. It is a very useful diagnostic tool for SSL servers.[^2]

Option -connect host:port:

This specifies the host and optional port to connect to. If not specified then an attempt is made to connect to the local host on port 4433.[^2]

And the format is:

1
$ openssl s_client -connect servername:443 > data

If a connection is established, openssl enters interactive mode:

If a connection is established with an SSL server then any data received from the server is displayed and any key presses will be sent to the server. When used interactively (which means neither -quiet nor -ign_eof have been given), the session will be renegotiated if the line begins with an R, and if the line begins with a Q or if end of file is reached, the connection will be closed down.[^2]

To quit, type Q or <ctr>+d (EOF).

1
2
3
4
5
6
7
$ openssl s_client -connect example.com:443 > /tmp/example.com
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance
Server CA
verify error:num=20:unable to get local issuer certificate
verify return:0
Q
DONE

Dump the session data:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
$ cat /tmp/example.com
CONNECTED(00000003)
---
Certificate chain
0 s:/C=US/ST=California/L=Los Angeles/O=Internet Corporation for Assigned Names and Numbers/OU=Technology/CN=www.example.org
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIF8jCCBNqgAwIBAgIQDmTF+8I2reFLFyrrQceMsDANBgkqhkiG9w0BAQsFADBw
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMS8wLQYDVQQDEyZEaWdpQ2VydCBTSEEyIEhpZ2ggQXNz
dXJhbmNlIFNlcnZlciBDQTAeFw0xNTExMDMwMDAwMDBaFw0xODExMjgxMjAwMDBa
MIGlMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTEUMBIGA1UEBxML
TG9zIEFuZ2VsZXMxPDA6BgNVBAoTM0ludGVybmV0IENvcnBvcmF0aW9uIGZvciBB
c3NpZ25lZCBOYW1lcyBhbmQgTnVtYmVyczETMBEGA1UECxMKVGVjaG5vbG9neTEY
MBYGA1UEAxMPd3d3LmV4YW1wbGUub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
MIIBCgKCAQEAs0CWL2FjPiXBl61lRfvvE0KzLJmG9LWAC3bcBjgsH6NiVVo2dt6u
Xfzi5bTm7F3K7srfUBYkLO78mraM9qizrHoIeyofrV/n+pZZJauQsPjCPxMEJnRo
D8Z4KpWKX0LyDu1SputoI4nlQ/htEhtiQnuoBfNZxF7WxcxGwEsZuS1KcXIkHl5V
RJOreKFHTaXcB1qcZ/QRaBIv0yhxvK1yBTwWddT4cli6GfHcCe3xGMaSL328Fgs3
jYrvG29PueB6VJi/tbbPu6qTfwp/H1brqdjh29U52Bhb0fJkM9DWxCP/Cattcc7a
z8EXnCO+LK8vkhw/kAiJWPKx4RBvgy73nwIDAQABo4ICUDCCAkwwHwYDVR0jBBgw
FoAUUWj/kK8CB3U8zNllZGKiErhZcjswHQYDVR0OBBYEFKZPYB4fLdHn8SOgKpUW
5Oia6m5IMIGBBgNVHREEejB4gg93d3cuZXhhbXBsZS5vcmeCC2V4YW1wbGUuY29t
ggtleGFtcGxlLmVkdYILZXhhbXBsZS5uZXSCC2V4YW1wbGUub3Jngg93d3cuZXhh
bXBsZS5jb22CD3d3dy5leGFtcGxlLmVkdYIPd3d3LmV4YW1wbGUubmV0MA4GA1Ud
DwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwdQYDVR0f
BG4wbDA0oDKgMIYuaHR0cDovL2NybDMuZGlnaWNlcnQuY29tL3NoYTItaGEtc2Vy
dmVyLWc0LmNybDA0oDKgMIYuaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL3NoYTIt
aGEtc2VydmVyLWc0LmNybDBMBgNVHSAERTBDMDcGCWCGSAGG/WwBATAqMCgGCCsG
AQUFBwIBFhxodHRwczovL3d3dy5kaWdpY2VydC5jb20vQ1BTMAgGBmeBDAECAjCB
gwYIKwYBBQUHAQEEdzB1MCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2Vy
dC5jb20wTQYIKwYBBQUHMAKGQWh0dHA6Ly9jYWNlcnRzLmRpZ2ljZXJ0LmNvbS9E
aWdpQ2VydFNIQTJIaWdoQXNzdXJhbmNlU2VydmVyQ0EuY3J0MAwGA1UdEwEB/wQC
MAAwDQYJKoZIhvcNAQELBQADggEBAISomhGn2L0LJn5SJHuyVZ3qMIlRCIdvqe0Q
6ls+C8ctRwRO3UU3x8q8OH+2ahxlQmpzdC5al4XQzJLiLjiJ2Q1p+hub8MFiMmVP
PZjb2tZm2ipWVuMRM+zgpRVM6nVJ9F3vFfUSHOb4/JsEIUvPY+d8/Krc+kPQwLvy
ieqRbcuFjmqfyPmUv1U9QoI4TQikpw7TZU0zYZANP4C/gj4Ry48/znmUaRvy2kvI
l7gRQ21qJTK5suoiYoYNo3J9T+pXPGU7Lydz/HwW+w0DpArtAaukI8aNX4ohFUKS
wDSiIIWIWJiJGbEeIO0TIFwEVWTOnbNl/faPXpk5IRXicapqiII=
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=Los Angeles/O=Internet Corporation for Assigned Names and Numbers/OU=Technology/CN=www.example.org
issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA ---
No client certificate CA names sent
---
SSL handshake has read 3393 bytes and written 421 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: C828441A824CE7B0F6A74BBE890AB23727445EAE8521E19F438E679C39E969B1
Session-ID-ctx:
Master-Key: 38BE4F754FBCB5F41650AD91AA5588ACD88B75D7939487052D9FD2790476E7C6D2512A6451A3FC102958488BF173CB54
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 83 70 c4 28 23 ee 9c 9e-87 1b 96 bf 44 76 ee d3 .p.(#.......Dv..
0010 - 45 c9 be ee a5 c5 42 49-c9 08 35 10 ba 79 03 b4 E.....BI..5..y..
0020 - 46 99 9a f2 d3 7b b5 f2-ad 9e 10 5c 7a 61 c3 0e F....{.....\za..
0030 - e0 09 aa 7a 5e 2a 2e bb-42 6a 08 18 16 ae 56 66 ...z^*..Bj....Vf
0040 - 11 0c 96 1a 4a 20 9f 50-6d f7 e2 53 00 75 6f 07 ....J .Pm..S.uo.
0050 - 7f 94 bf 4a 5f e1 f6 3b-d5 b7 6c 11 bc 33 7b 10 ...J_..;..l..3{.
0060 - 78 e3 81 a0 0b 83 25 d6-e6 a5 64 90 59 24 a6 e9 x.....%...d.Y$..
0070 - 9b b6 4b be 9e 42 1b 03-e0 d7 76 e9 32 87 3e 0d ..K..B....v.2.>.
0080 - 3d 09 09 32 18 fd 04 63-93 fe 33 9f 47 50 d4 c1 =..2...c..3.GP..
0090 - e1 a9 21 cc 67 30 ea 03-7f c1 ee 2a 54 02 c8 11 ..!.g0.....*T...
Start Time: 1475971200
Timeout : 300 (sec)
Verify return code: 20 (unable to get local issuer certificate)
---

To avoid the interactive mode, we can pipe an empty string into the command:

1
$ echo | openssl s_client -connect example.com:443 > /tmp/example.com 2> /dev/null

Now we have retrieved the SSL certificate from the server. Next, extract the expiration date. This is done by using the standard command x509:

Renewing Let's Encrypt SSL Certificate with Docker

Let’s Encrypt CA issues short-lived certificates (90 days). Automated renewal process is preferred, recommended, and encouraged. But in a few situations, automated process is not available, here is how to do it manually when SSL certificate was installed with Docker:

First, update the container to the latest version. The latest version can be found from the release page in GitHub.

The latest is v0.9.1:

1
$ docker pull quay.io/letsencrypt/letsencrypt:v0.9.1

Turn off application (if running as a Docker container) to free up the HTTPS port 443:

1
$ docker stop app

Renew the certificate by issuing renew command:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
$ docker run -it --rm -p 443:443 --name certbot \
-v /etc/letsencrypt:/etc/letsencrypt \
-v /var/log/letsencrypt:/var/log/letsencrypt \
quay.io/letsencrypt/letsencrypt:v0.9.1 renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log
-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/example.com.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for example.com
Waiting for verification...
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0001_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0001_csr-certbot.pem
-------------------------------------------------------------------------------
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/example.com/fullchain.pem
-------------------------------------------------------------------------------
Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/example.com/fullchain.pem (success)

Restart app (it’s start not restart):

1
$ docker start app

Check the expiration date:

1
2
3
4
$ echo | openssl s_client -connect example.com:443 2> /dev/null | \
openssl x509 -noout -dates
notBefore=Oct 9 12:00:00 2016 GMT
notAfter=Jan 7 12:00:00 2017 GMT

For more information on renewing, see the Renewing Certificates section from the Certbot documentation.

Settings:

  • Certbot v0.9.1
  • Docker v1.12.1

Getting Let's Encrypt SSL Certificate with Docker

Let’s Encrypt is a free, open, and automated certificate authority (CA). And its Certbot is a fully-featured, extensible client for Let’s Encrypt CA that can automate the tasks of getting, renewing and even installing SSL certificates.

First, you need to get Certbot. There are a few ways to install Certbot. But with Docker, you don’t need to install, you just need to download the Docker image and run the container. However, the caveat is that this method does not install the certificate automatically respecting to your web server. But if you’re like me, running your server in another Docker container, this might be the way to go.

Let’s start.

First, download the image. You can download the latest version (tag):

1
$ docker pull quay.io/letsencrypt/letsencrypt:latest

But the latest usually is not a stable release:

1
2
$ docker run -it --rm quay.io/letsencrypt/letsencrypt:latest --version
certbot 0.10.0.dev0

Therefore, it’s better to use a specific release, which can be found in Certbot’s GitHub page: https://github.com/certbot/certbot/releases.

The latest one now is v0.9.1. We can pull that from Quay.io:

1
$ docker pull quay.io/letsencrypt/letsencrypt:v0.9.1

Confirm the release version:

1
2
$ docker run -it --rm quay.io/letsencrypt/letsencrypt:v0.9.1 --version
certbot 0.9.1

Let’s take a look at the Docker image:

1
$ docker inspect quay.io/letsencrypt/letsencrypt:v0.9.1

Dropping things don’t care, the output is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
[
{
"RepoTags": [
"quay.io/letsencrypt/letsencrypt:v0.9.1"
],
"ContainerConfig": {
"ExposedPorts": {
"443/tcp": {}
},
"Cmd": [
"/bin/sh",
"-c",
"#(nop) ENTRYPOINT \u0026{[\"certbot\"]}"
],
"Volumes": {
"/etc/letsencrypt": {},
"/var/lib/letsencrypt": {}
},
"Entrypoint": [
"certbot"
]
}
}
]

The ENTRYPOINT is certbot binary:

An ENTRYPOINT allows you to configure a container that will run as an executable.[^1]

And the command line arguments to docker run becomes the arguments to certbot command. As we saw earlier to obtain the release version by using --version.

You can override the ENTRYPOINT instruction using the docker run --entrypoint flag.[^1]

For example, to override and run the container without executing the certbot command:

1
2
$ docker run -it --rm --name certbot --entrypoint /bin/bash \
quay.io/letsencrypt/letsencrypt:v0.9.1

But we are more concerning about others, such as exposed port and mapped volumes. The exposed port is 443, HTTPS port. The most important volume (directory) is /etc/letsencrypt. All generated keys and issued certificates can be found in there. Directory /var/lib/letsencrypt is the default working directory, some backup stuff are stored. I have yet to find it useful. However, the logs directory /var/log/letsencrypt is not being used. This could be useful if things went haywire.