Archive for October, 2014

Introducing pyhcl

Wednesday, October 15th, 2014

HCL is a configuration file format created by Hashicorp to “build a structured configuration language that is both human and machine friendly for use with command-line tools”. They created it to use with their tools because they weren’t satisfied with existing solutions, and I think they did a really good job with it.

I also have similar opinions. JSON has parsers in just about any language one can think of, but is a really terrible format for humans to deal with since it doesn’t support comments. YAML is sometimes mentioned as a better choice, but I think it’s generally pretty terrible to use. I’ve only had to use YAML a few times, but in my opinion it’s tricky to get YAML files of any complexity correct without a bit of futzing.

Instead, HCL is super easy to type out and read, and looks quite nice:

variable "ami" {
    # this is a comment
    description = "the AMI to use"

I liked the idea of HCL a lot, so I created a python implementation of the parser using ply, which I’ve called pyhcl. When you read an HCL file, pyhcl turns it into a python dictionary (just like JSON), and the dictionary representation for the above HCL is like this:

  "variable": {
    "ami": {
      "description": "the AMI to use"

To get that result, parsing HCL using pyhcl is super easy, and is pretty much the same as parsing JSON:

import hcl

with open('file.hcl', r) as fp:
     obj = hcl.load(fp)

So far, I’ve been really happy with HCL, and I’ve been using it for some projects with complex configuration requirements, and the end-users of those projects have been quite happy with the simplicity that HCL provides.

pyhcl is provided under the MPL 2.0, just like the golang parser, and can mostly be used in the same places one might use JSON. It’s probably not terribly performant — but parsing files that humans read shouldn’t really require performance. If you do, python’s JSON parser is written in C and should meet your needs quite nicely.

One thing that would be nice is if there was an actual specification for HCL, and in particular it isn’t very well defined on how to convert HCL to/from JSON… but lacking that, pyhcl currently tries to match the golang implementation bug for bug, and its test suite has stolen most of the fixtures from the golang parser to ensure maximum compatibility.

Github site for pyhcl
Pypi site for pyhcl