Skip to content

Commit 2c4721d

Browse files
bketelsenbketelsen
authored
Merge pull request #331 from tebeka/tebeka-cmdline
Writing command line friendly applicaitons
2 parents 86b804a + 38e7e9c commit 2c4721d

File tree

8 files changed

+543
-0
lines changed

8 files changed

+543
-0
lines changed

‎content/advent-2019/cmdline.md‎

Lines changed: 325 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,325 @@
1+
+++
2+
title = "Writing Friendly Command Line Applications"
3+
date = "2019-12-02T00:00:00+00:00"
4+
series = ["Advent 2019"]
5+
author = ["Miki Tebeka"]
6+
linktitle = "flag"
7+
+++
8+
9+
Let me tell you a story...
10+
11+
In 1986 [Knuth](https://en.wikipedia.org/wiki/Donald_Knuth) wrote a program to
12+
demonstrate [literate
13+
programming](https://en.wikipedia.org/wiki/Literate_programming).
14+
15+
The task was to read a file of text, determine the n most frequently used
16+
words, and print out a sorted list of those words along with their frequencies.
17+
Knuth wrote a beautiful 10 page monolithic program.
18+
19+
Doug Mcllory read this and said
20+
`tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q`
21+
22+
It's 2019, why am I telling you a story that happened 33 years ago? (Probably
23+
before some of you were born). The computation landscape has changed a lot...
24+
or has it?
25+
26+
The [Lindy effect](https://en.wikipedia.org/wiki/Lindy_effect) is a concept
27+
that the future life expectancy of some non-perishable things like a technology
28+
or an idea is proportional to their current age. TL;DR - old technologies are
29+
here to stay.
30+
31+
If you don't believe me, see:
32+
33+
- [oh-my-zsh](https://github.com/ohmyzsh/ohmyzsh) having close to 100,000 stars on GitHub
34+
- [Data Science at the Command Line](https://www.datascienceatthecommandline.com/) book
35+
- [Command-line Tools can be 235x Faster than your Hadoop Cluster](https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html)
36+
- ...
37+
38+
Now that you are convinced, let's talk on how to make your Go programs command
39+
line friendly.
40+
41+
## Design
42+
43+
When writing command line application, try to adhere to the [basics of Unix
44+
philosophy](http://www.catb.org/esr/writings/taoup/html/ch01s06.html)
45+
46+
- Rule of Modularity: Write simple parts connected by clean interfaces.
47+
- Rule of Composition: Design programs to be connected with other programs.
48+
- Rule of Silence: When a program has nothing surprising to say, it should say
49+
nothing.
50+
51+
These rules allow you to write small program that do one thing.
52+
53+
- A user asks for support of reading data from REST API? Have them pipe a
54+
`curl` command output to your program
55+
- A user wants only top n results? Have them pipe your program output through
56+
`head
57+
- A user wants only the second column of data? Since you write tab seperated
58+
output, they can pipe your output via `cut` or `awk`
59+
60+
If you don't follow these and let your command line interface grow organically,
61+
you might end up in the following situation
62+
63+
[![](https://imgs.xkcd.com/comics/tar.png)](https://xkcd.com/1168/)
64+
65+
66+
## Help
67+
68+
Let's assume your team have a `nuke-db` utility. You forgot how to invoke it
69+
and you do:
70+
71+
```
72+
$ ./nuke-db --help
73+
database nuked
74+
```
75+
76+
Ouch!
77+
78+
Using the [flag](https://golang.org/pkg/flag/), you can add support for `--help` in 2 extra lines of code
79+
80+
```go
81+
package main
82+
83+
import (
84+
"flag" // extra line 1
85+
"fmt"
86+
)
87+
88+
func main() {
89+
flag.Parse() // extra line 2
90+
fmt.Println("database nuked")
91+
}
92+
```
93+
94+
Now your program behaves
95+
96+
```
97+
$ ./nuke-db --help
98+
Usage of ./nuke-db:
99+
$ ./nuke-db
100+
database nuked
101+
```
102+
103+
If you'd like to provide more help, use `flag.Usage`
104+
105+
```go
106+
package main
107+
108+
import (
109+
"flag"
110+
"fmt"
111+
"os"
112+
)
113+
114+
var usage = `usage: %s [DATABASE]
115+
116+
Delete all data and tables from DATABASE.
117+
`
118+
119+
func main() {
120+
flag.Usage = func() {
121+
fmt.Fprintf(flag.CommandLine.Output(), usage, os.Args[0])
122+
flag.PrintDefaults()
123+
}
124+
flag.Parse()
125+
fmt.Println("database nuked")
126+
}
127+
```
128+
129+
And now
130+
```
131+
$ ./nuke-db --help
132+
usage: ./nuke-db [DATABASE]
133+
134+
Delete all data and tables from DATABASE.
135+
```
136+
137+
## Structured Output
138+
139+
Plain text is the universal interface. However, when the output becomes
140+
complex, it might be easier for machines to deal with formatted output. One of
141+
the most common format is of course JSON.
142+
143+
A good way to do it is not to print using `fmt.Printf` but use your own
144+
printing function which can be either text or JSON. Let's see an example:
145+
146+
```go
147+
package main
148+
149+
import (
150+
"encoding/json"
151+
"flag"
152+
"fmt"
153+
"log"
154+
"os"
155+
)
156+
157+
func main() {
158+
var jsonOut bool
159+
flag.BoolVar(&jsonOut, "json", false, "output in JSON format")
160+
flag.Parse()
161+
if flag.NArg() != 1 {
162+
log.Fatal("error: wrong number of arguments")
163+
}
164+
165+
write := writeText
166+
if jsonOut {
167+
write = writeJSON
168+
}
169+
170+
fi, err := os.Stat(flag.Arg(0))
171+
if err != nil {
172+
log.Fatalf("error: %s\n", err)
173+
}
174+
175+
m := map[string]interface{}{
176+
"size": fi.Size(),
177+
"dir": fi.IsDir(),
178+
"modified": fi.ModTime(),
179+
"mode": fi.Mode(),
180+
}
181+
write(m)
182+
}
183+
184+
func writeText(m map[string]interface{}) {
185+
for k, v := range m {
186+
fmt.Printf("%s: %v\n", k, v)
187+
}
188+
}
189+
190+
func writeJSON(m map[string]interface{}) {
191+
m["mode"] = m["mode"].(os.FileMode).String()
192+
json.NewEncoder(os.Stdout).Encode(m)
193+
}
194+
195+
```
196+
197+
Then
198+
199+
```
200+
$ ./finfo finfo.go
201+
mode: -rw-r--r--
202+
size: 783
203+
dir: false
204+
modified: 2019-11-27 11:49:03.280857863 +0200 IST
205+
$ ./finfo -json finfo.go
206+
{"dir":false,"mode":"-rw-r--r--","modified":"2019-11-27T11:49:03.280857863+02:00","size":783}
207+
```
208+
209+
## Progress
210+
211+
Some operations can take long time, one way to make them faster is not by
212+
optimising the code but by showing a spinner/progress bar. Don't believe me,
213+
here's an excerpt from [Nielsen
214+
research](https://www.nngroup.com/articles/progress-indicators/)
215+
216+
> people who saw the moving feedback bar experienced higher satisfaction and
217+
> were willing to wait on average 3 times longer than those who did not see any
218+
> progress indicators.
219+
220+
### Spinner
221+
222+
Adding a spinner does not require any special packages:
223+
224+
```go
225+
package main
226+
227+
import (
228+
"flag"
229+
"fmt"
230+
"os"
231+
"time"
232+
)
233+
234+
var spinChars = `|/-\`
235+
236+
type Spinner struct {
237+
message string
238+
i int
239+
}
240+
241+
func NewSpinner(message string) *Spinner {
242+
return &Spinner{message: message}
243+
}
244+
245+
func (s *Spinner) Tick() {
246+
fmt.Printf("%s %c \r", s.message, spinChars[s.i])
247+
s.i = (s.i + 1) % len(spinChars)
248+
}
249+
250+
func isTTY() bool {
251+
fi, err := os.Stdout.Stat()
252+
if err != nil {
253+
return false
254+
}
255+
return fi.Mode()&os.ModeCharDevice != 0
256+
}
257+
258+
func main() {
259+
flag.Parse()
260+
s := NewSpinner("working...")
261+
for i := 0; i < 100; i++ {
262+
if isTTY() {
263+
s.Tick()
264+
}
265+
time.Sleep(100 * time.Millisecond)
266+
}
267+
268+
}
269+
```
270+
271+
Run it and you'll see a small spinner going.
272+
273+
274+
### Progress Bar
275+
276+
For a progress bar, you'll probably need an external package such as
277+
`github.com/cheggaaa/pb/v3`
278+
279+
```go
280+
package main
281+
282+
import (
283+
"flag"
284+
"time"
285+
286+
"github.com/cheggaaa/pb/v3"
287+
)
288+
289+
func main() {
290+
flag.Parse()
291+
count := 100
292+
bar := pb.StartNew(count)
293+
for i := 0; i < count; i++ {
294+
time.Sleep(100 * time.Millisecond)
295+
bar.Increment()
296+
}
297+
bar.Finish()
298+
299+
}
300+
```
301+
302+
Run it and you'll see a nice progress bar.
303+
304+
305+
# Conclusion
306+
307+
It's almost 2020, and command line applications are here to stay. They are the
308+
key to automation and if written well, provide elegant "lego like" components
309+
to build complex flows.
310+
311+
I hope that this article will prompt you to be a good citizen of the command
312+
line nation.
313+
314+
# About the Author
315+
316+
Hi there, I'm Miki, nice to e-meet you ☺. I've been a long time developer and
317+
have been working with Go for about 10 years now. I write code professionally as
318+
a consultant and contribute a lot to open source. Apart from that I'm a [book
319+
author](https://www.amazon.com/Forging-Python-practices-lessons-developing-ebook/dp/B07C1SH5MP),
320+
an author on [LinkedIn
321+
learning](https://www.linkedin.com/learning/search?keywords=miki+tebeka), one of
322+
the organizers of [GopherCon Israel](https://www.gophercon.org.il/) and [an
323+
instructor](https://www.353.solutions/workshops). Feel free to [drop me a
324+
line](mailto:[email protected]) and let me know if you learned something
325+
new or if you'd like to learn more.

‎content/advent-2019/cmdline/finfo.go‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
package main
2+
3+
import (
4+
"encoding/json"
5+
"flag"
6+
"fmt"
7+
"log"
8+
"os"
9+
)
10+
11+
func main() {
12+
var jsonOut bool
13+
flag.BoolVar(&jsonOut, "json", false, "output in JSON format")
14+
flag.Parse()
15+
if flag.NArg() != 1 {
16+
log.Fatal("error: wrong number of arguments")
17+
}
18+
19+
write := writeText
20+
if jsonOut {
21+
write = writeJSON
22+
}
23+
24+
fi, err := os.Stat(flag.Arg(0))
25+
if err != nil {
26+
log.Fatalf("error: %s\n", err)
27+
}
28+
29+
m := map[string]interface{}{
30+
"size": fi.Size(),
31+
"dir": fi.IsDir(),
32+
"modified": fi.ModTime(),
33+
"mode": fi.Mode(),
34+
}
35+
write(m)
36+
}
37+
38+
func writeText(m map[string]interface{}) {
39+
for k, v := range m {
40+
fmt.Printf("%s: %v\n", k, v)
41+
}
42+
}
43+
44+
func writeJSON(m map[string]interface{}) {
45+
m["mode"] = m["mode"].(os.FileMode).String()
46+
json.NewEncoder(os.Stdout).Encode(m)
47+
}

‎content/advent-2019/cmdline/go.mod‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
module advent2019
2+
3+
go 1.13
4+
5+
require github.com/cheggaaa/pb/v3 v3.0.2

‎content/advent-2019/cmdline/go.sum‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
github.com/VividCortex/ewma v1.1.1 h1:MnEK4VOv6n0RSY4vtRe3h11qjxL3+t0B8yOL8iMXdcM=
2+
github.com/VividCortex/ewma v1.1.1/go.mod h1:2Tkkvm3sRDVXaiyucHiACn4cqf7DpdyLvmxzcbUokwA=
3+
github.com/cheggaaa/pb v2.0.7+incompatible h1:gLKifR1UkZ/kLkda5gC0K6c8g+jU2sINPtBeOiNlMhU=
4+
github.com/cheggaaa/pb/v3 v3.0.2 h1:/u+zw5RBzW1CxRpVIqrZv4PpZpN+yaRPdsRORKyDjv4=
5+
github.com/cheggaaa/pb/v3 v3.0.2/go.mod h1:SqqeMF/pMOIu3xgGoxtPYhMNQP258xE4x/XRTYua+KU=
6+
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
7+
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
8+
github.com/mattn/go-colorable v0.1.2 h1:/bC9yWikZXAL9uJdulbSfyVNIR3n3trXl+v8+1sx8mU=
9+
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
10+
github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE=
11+
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
12+
github.com/mattn/go-runewidth v0.0.4 h1:2BvfKmzob6Bmd4YsL0zygOqfdFnK7GR4QL06Do4/p7Y=
13+
github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
14+
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223 h1:DH4skfRX4EBpamg7iV4ZlCpblAHI6s6TDM39bFZumv8=
15+
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=

0 commit comments

Comments
 (0)