Just before Thanksgiving last year, my colleague Tim Pettersen wrote an article about building command-line tools with Node.js. While I was reviewing the article, I asked Tim why he likes commander for parsing command-line arguments. As with many programming choices, it came down to what he had used before. I’m sure most programmers have been down this road before, especially for command-line parsing. What works best is something you know. That’s usually fine for me. But, in the world of command-line interfaces, I think there is now just one right way. It’s docopt.
Writing command-line tools is something I’ve done many times. I’ve written them in C, Java, C#, Ruby, and Python. My latest is a Python program that manipulates Bitbucket Snippets. When I started about a year ago, there was an itch I just couldn’t scratch. I looked at the standard argparse library in Python. The code seemed easy enough. It automatically generates command-line help, which helps keep documentation up-to-date with the code. But here I was learning yet-another-argument-parser-library. I wanted something that would keep me consistent between languages, not just within Python.
For web applications, I have been following the growing trend of API Description Languages, such as Swagger, RAML, I/O Docs, and API Blueprints. For some, these tools are reminiscent of heavyweight technologies like SOAP and WSDL. For me, these solve the important need to keep consistency between code and documentation. They are especially helpful because web APIs can be consumed by any language. And so it was that I found myself Googling for “command-line interface description language”. Fortunately for me, I wasn’t the first with the idea. At PyCon UK 2012, Vladimir Keleshev had presented his Python implementation of docopt. Now, there are parser implementations in every language I had ever used, and for every one I might in the future. A docopt description is simple to write. In addition to Vladimir’s video, there are ample documentation and examples to learn from. There’s even a nifty docopt web tool to try out descriptions in a browser. Here’s the description language for my Snippet CLI tool:
A command-line interface for creating, inspecting, and editing
Bitbucket Snippets.
usage:
snippet list [--owner|--contributor|--member]
snippet create [--title=<title>] [--public|--private] [FILES ...]
snippet modify <id> [--title=<title>] [--public|--private] [FILES ...]
snippet delete <id>
snippet info <id>
snippet files <id>
snippet content <id> <filename>
snippet watchers <id>
snippet comments <id>
snippet commits <id>
This description is exactly what the program emits with an implicit --help or -h, or whenever the provided command-line arguments don’t match the syntax. Even though it serves as documentation, it is also a full description of the grammar. Binding the description language to my code is also simple. In Python, I can just place the above as comments in my main.py. (In other languages, it is often just a string variable.) Python lets me access the code comments with the special doc string symbol __doc__, which I pass it into docopt() like this:
def main():
arguments = docopt(__doc__)
When my program runs, arguments will be populated with a dictionary of parsed values. Here’s a JSON representation for the arguments create --title=Example --public example.py:
{
"--contributor": false,
"--member": false,
"--owner": false,
"--private": false,
"--public": true,
"--title": "Example",
"<filename>": null,
"<id>": null,
"FILES": [
"example.py"
],
"comments": false,
"commits": false,
"content": false,
"create": true,
"delete": false,
"files": false,
"info": false,
"list": false,
"modify": false,
"watchers": false
}
From here, my program is responsible for dispatching into the right code. This will vary by language, but you can see how I dispatched in my main.py method for Snippet.
While my example shows how easy it is for a simple case, I have noticed some much more complex examples. One nice thing is the subcommands can be distributed in separate files. In the Python docopt repository, there is a sample using the Git command structure. In a real-world example, the developers at ActiveState wrote a tool with 60 subcommands.
I realize that some developers might object to docopt because their favorite command-line framework does much more than just parse arguments. For example, even the standard Python argparse will automatically dispatch to Python methods. (For what it’s worth, Keleshev has written an additional Python library to perform dispatching.) More extensive libraries like Python’s clint help format text tables, write color output, and collect user input. But I hope for the best of both worlds. Ideally, docopt might be a plugin to a broader CLI framework.
If you found this useful, or have your own tips about command-line tools, tweet me at @devpartisan or my team at @atlassiandev.