In this article we will build a program in Golang to parse a JSON file containing a collection held in a named key – without knowing the structure of this object, we will expose the schema for the object including data types and recurse the object for its values.
This example uses a great Go package called
tablewriter to render the output of these operations using a table style result set.
The program has
select verbs as operation types; describe shows the column names in the collection and their respective data types, select prints the keys and values as a tabular result set with column headers for the keys and rows containing their corresponding values.
Starting with this:
We will end up with this when performing a
And this when performing a
Now let’s talk about how we got there…
Support for JSON in Go is provided using the
encoding/json package, this needs to be imported in your program of course… You will also need to import the
reflect package – more on this later.
io/ioutil is required to read the data from a file input, there are other packages included in the program that are removed for brevity:
We will read the data from the JSON file into a variable called
body, note that we are not attempting to deserialize the data at this point. This is also a good opportunity to handle any runtime or IO errors that occur here as well.
We will declare an empty interface called
data which will be used to decode the json object (of which the structure is not known), we will also create an abstract interface called
colldata to hold the contents of the collection contained inside the JSON object that we are specifically looking for:
Next we need to validate that the input is a valid JSON object, we can use the
json.Valid(body) method to do this:
Now the interesting bits, we will deserialize the JSON object to the empty data interface we created earlier using the
Note that this operation is another opportunity to catch unexpected errors and handle them accordingly.
Now that we have serialized the JSON object into the data interface, there are several ways we can inspect the type of the object (which could be a map or an array). One such way is to use reflection. Reflection is the ability of a program to inspect itself at runtime. An example is shown here:
This instruction would produce the following output for our
Another method to decode the type of the data object (and any objects nested as elements or keys within the data object), is to use the type switch, an example of a type switch function is shown here:
The aim of the program is to find a collection (an array of maps) nested in a JSON object. The maps with each element of the array are unknown at runtime and are discovered through recursion.
If we are performing a describe operation, we only need to parse the first element of the collection to get the key names and the data type of the values (for which we will use the same
getObjectType function to perform a type switch.
If we are performing a select operation, we need to parse the first element to get the column names (the keys in the map) and then we need to recurse each element to get the values for each key.
If the element contains a key named id or name, we will place this at the beginning of the resultant record, as maps are unordered by definition.
As mentioned, we are using the
tablewriter package to render the output of the collection as a pretty printed table in our terminal. As wrap around can get pretty ugly an additional
maxfieldlen argument is provided to truncate the values if needed.
Although it is a bit more involved than some other languages, once you get your head around processing JSON in Go, the possibilities are endless!
Full source code can be found at: https://github.com/gamma-data/json-wrangling-with-golang