bash, curl, and jq

The JSON-based REST APIs of AQUARIUS Time-Series can also be easily exercised using standard *NIX command line tools.

This section shows some examples of simple bash command lines, using curl to issue REST API requests to your AQTS app server and jq to manipulate the JSON responses.

All three tools are available on Linux, mac OS, and Windows systems.

Using curl to make API requests

Here's a minimum request to fetch all the parameters in a system. This sends a large-ish 30KB JSON response to the console.

$ curl -f -u admin:admin "http://myappserver/AQUARIUS/Publish/v2/GetParameterList"
{"Parameters":[{"Identifier":"Stage","UnitGroupIdentifier":"Length","UnitIdentifier":"m","DisplayName":"Height of Gauge (River Stage)","InterpolationType":"1","RoundingSpec":"DEC(3)"},{"Identifier":"Ice Thickness","UnitGroupIdentifier":"Length","UnitIdentifier":"cm","DisplayName":"Ice thickness","InterpolationType":"7","RoundingSpec":"SIG(3)"},{"Identifier":"Snow Depth","UnitGroupIdentifier":"Length","UnitIdentifier"

Commonly used curl options for REST API requests

You'll need to set these options for every curl request.

Option	Description
-f	Fail if the response is not an HTTP 2xx status code.
-u username:password	Set your AQUARIUS credentials using HTTP Basic Authentication.

See curl --help for the (huge!) list of supported options.

Making POST, PUT, or DELETE requests

curl makes GET requests by default. To use a different HTTP verb, you'll need to set a few more command line options.

Option	Description
-X POST	Set the HTTP request verb to something other than GET. Useful for POST, PUT, and DELETE requests.
-d '{json data}'	Sets the request payload with a JSON document.
-H "Content-Type: application/json"	Sets the content-type header. Used in conjuction with the -d option to tell the server that the request payload is JSON data.

Here's a PUT request which changes the name of a location tag:

$ curl -f -s -u admin:admin -H 'Content-Type: application/json' -X PUT -d '{"Name":"WebPortal.Engineering"}' "http://doug-vm2012r2/AQUARIUS/Provisioning/v1/tags/location/811181fe3a93418c983e2a3215ff747d"

Useful curl options for debugging your requests

Option	Description
-v	Enables verbose output from curl, showing headers and cookie information.
-x127.0.0.1:8888	Routes the request through the given HTTP proxy, like the awesome Fiddler web proxy. Use this when -v is not enough.
-s	Silent mode, which disables the textual response progress bar on the terminal.

Using jq to process JSON responses

The jq command is a powerful tool for manipulating and transforming JSON response documents.

You typically pipe the output of your curl request into jq for processing.

jq with no other arguments just pretty-prints the received JSON

Here's the first request example, piped into jq. That single-line 30 KB JSON document is now spread out over about 1100 lines.

$ curl -f -u admin:admin "http://myappserver/AQUARIUS/Publish/v2/GetParameterList" | jq
{
  "Parameters": [
    {
      "Identifier": "Stage",
      "UnitGroupIdentifier": "Length",
      "UnitIdentifier": "m",
      "DisplayName": "Height of Gauge (River Stage)",
      "InterpolationType": "1",
      "RoundingSpec": "DEC(3)"
    },
...

Use jq to pull out one item from an array

jq command lines are a set of filters in a single-quoted string, with pipes separating each filter.

We'll find the Discharge parameter by using two filters:

• .Parameters[] to extract the Parameters array from the response
• select(.Identifier = "Discharge") to return the one parameter named "Discharge"

$ curl -f -u admin:admin -s "http://doug-vm2012r2/AQUARIUS/Publish/v2/GetParameterList" | jq '.Parameters[] | select(.Identifier == "Discharge")'
{
  "Identifier": "Discharge",
  "UnitGroupIdentifier": "Volumetric Flow",
  "UnitIdentifier": "m^3/s",
  "DisplayName": "Discharge",
  "InterpolationType": "1",
  "RoundingSpec": "SIG(3)"
}

Extract all the locations in a system

This command line fetches all the locations in a system, pulls out their identifiers, one per line.

$ curl -f -u admin:admin -s "http://doug-vm2012r2/AQUARIUS/Publish/v2/GetLocationDescriptionList" | jq '.LocationDescriptions[] | .Identifier'
"01FB002"
"01AH005"
"03OA003"
...

Each location identifier is still surrounded by quotes, so use the -r option to get raw output instead.

$ curl -f -u admin:admin -s "http://doug-vm2012r2/AQUARIUS/Publish/v2/GetLocationDescriptionList" | jq -r '.LocationDescriptions[] | .Identifier'
01FB002
01AH005
03OA003
...

Transform a JSON array into a CSV stream

jq supports a @csv filter to transforms an array into a CSV line.

jq -r '.someArray[] | [.field1, .field2, .field3] | @csv'

That cryptic filter sequence will:

• select the someArray JSON array from the response
• compose a new array using the 3 fields
• convert it to a CSV
• And spit it out in raw mode

Let's apply that to the list of locations to get a CSV of location names and identifiers.

$ curl -f -u admin:admin -s "http://doug-vm2012r2/AQUARIUS/Publish/v2/GetLocationDescriptionList" | jq -r '.LocationDescriptions[] | [.Identifier, .Name] | @csv'
"01FB002","SOUTHWEST MARGAREE RIVER AT MARGAREE FORKS"
"01AH005","MAMOZEKEL RIVER NEAR CAMPBELL RIVER"
"03OA003","MCPHADYEN RIVER NEAR THE MOUTH"
"01CB004","WILMOT RIVER NEAR WILMOT VALLEY"
"01AO006","PORTOBELLO STREAM DOWNSTREAM OF BURNS BRIDGE"
"01EE007","WHITEBURN BROOK BELOW POLLOCK LAKE"
...