array

Wrapping Line by Line JSON to an Array

Some utilities print out JSON data (object) line by line. Line by line JSON is not really a valid JSON format, need to wrap it in bracket as an array.

For example, here are a few lines of JSON objects:

1
2
3
4
$ for i in $(seq 3); do echo '{"id":'$i'}'; done
{"id":1}
{"id":2}
{"id":3}

To combine them into a single array, we can use jq‘s -s, —slurp option:

1
2
$ for i in $(seq 3); do echo '{"id":'$i'}'; done | jq -cM -s
[{"id":1},{"id":2},{"id":3}]

The option reads the entire input stream into a large array. This is good until the input stream is too large to process, because jq is not producing output while slurping up the input. If the input is line by line JSON, and all you want to do is wrapping it into an array (with or even without the trailing newline), we can simply do the following:

1
2
3
$ for i in $(seq 3); do echo '{"id":'$i'}'; done |\
awk 'BEGIN { printf "["; getline; printf "%s", $0 } { printf ",%s", $0 } END { print "]" }'
[{"id":1},{"id":2},{"id":3}]

This is with the newline printed. And it will solve the problem of a very large input. Data will simply stream out as it comes it. Try it with a forever while loop.

There is another option in jq named —stream, and it seems to be doing the same. But option —slurp overrides —stream, and the option itself is already so complicated.

In conclusion, you can use jq -s to wrap line by line JSON into an array. If you want to be safe, just use the awk example above.

Randomizing an Array with Sort

How to randomize an array? Use the sort command, with the option:

1
2
-R, --random-sort
sort by random hash of keys

For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ seq 1 10 | sort -R
4
2
10
6
3
9
7
5
8
1
$ seq 1 10 | sort --random-sort
9
6
1
3
2
8
7
5
4
10

Use Array of Middleware in Koa Router

Koa Router v4.2 does not support an array of middleware, multiple middleware must be entered one by one:

1
app.get('/foo', middleware1, middleware2);

Using an array is much cleaner than multiple arguments.

1
app.get('/foo', [middleware1, middleware2]);

But the above will throw error:

1
Error: GET: `middleware` must be a function, not `object`

This can be easily fixed by using Koa Compose:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
var app = require('koa')();
var router = require('koa-router');
var compose = require('koa-compose');
app.use(router(app));
app.get('/foo', compose([
function *(next) {
this.body = {};
yield next;
},
function *(next) {
this.body.foo = 'foo';
yield next;
},
function *(next) {
this.body.bar = 'bar';
yield next;
}
]));

Hence:

1
app.get('/foo', compose([middleware1, middleware2]));

CoffeeScript Array of Objects

We frequently find ourselves need to define array of objects. In CoffeeScript we can write in the following way:

1
2
3
4
5
6
7
8
9
10
arr = [
{
name: 'foo'
pass: 'oof'
}
{
name: 'bar'
pass: 'rab'
}
]

Or another format is to insert a comma:

1
2
3
4
5
6
7
arr = [
name: 'foo'
pass: 'oof'
,
name: 'bar'
pass: 'rab'
]

In the spirit of CoffeeScript, we want to avoid using braces as much as possible. Therefore, the second method is preferred.

Repeat Character Serveral Times in JavaScript

In Markdown, a header can be written as:

1
2
Header
======

This is a Setext-style headers, which are ‘underlined’ using equal signs (for
first-level headers) and dashes (for second-level headers)
. In order to
generate those underlined equal signs, a simple loop could be used to repeat
the character = several times. By the way, it is not necessary to have exact
number of the equal signs as the header length, but it just looks better. But
there is another way:

1
Array(heading.length + 1).join('=')

It is using the JavaScript Array global object to construct an empty Array
with the specified array length. Clever indeed!