Hi, i've followed all of the instructions to download jQuery, but for the my mac it does not offer me a file to save, it just opens a new tab and fills. Jsp MAC downloads: Advanced Graph and Chart Collection, 3D Stacked Horizontal Bar Graph Software, 2D/3D Line Graph Software, jQuery Dropdown Menu Style 3, jQuery Horizontal Menu Style 05, jQuery Drop Down Menu Style 2, jQuery Menu Style 1. The simplest way to install jq and test that it works is through brew and then using the simplest filter that merely formats the JSON Install brew is the easiest way to manage packages on a mac.
Plus your entire music library on all your devices. With jq, we can easily parse JSON in the Bash shell. And it doesn't matter whether you have to work with well-engineered, elegant JSON, or the stuff nightmares are made of. How to Install jq. We had to install jq on all the Linux distributions we used to research this article. To install jq on Ubuntu type this command: sudo apt-get install jq.
Most of the popular API and data services use the JSON data format, so we'll learn how it's used to serialize interesting information, and how to use the jq to parse it at the command-line.
JSON - a lightweight data format
JSON stands for JavaScript Object Notation and is nearly ubiquitous as a data format, for its lightweight nature and (relatively) ease of human-readability.
Using jq
jq is a program described as 'sed for JSON data':
You can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.
Installing jq
If you've haven't installed the pup HTML parsing tool, or any other tool which required you to edit your ~/.bashrc's' PATH and create the ~/bin_compciv directory, then follow the instructions here: Installing programs for your personal Stanford account.
To install jq:
Check out the tutorial on jq here. The full manual can be found here.
JSON compared to CSV
JSON is more verbose, but easier to read than CSV.
CSV:
JSON:
Objects and attribute-value pairs
Inherent to the JSON data format is the ability to encapsulate data structures known as Objects consisting of attribute-value pairs
The object below has the attributes first_name and last_name, with the values of John and Smith, respectively:
JSON objects can also contain Arrays, which can be thought of as lists of elements. Below, the things_carried attribute has an Array containing apples, hat, and harmonica.
What makes JSON more flexible than flat data structures (such as CSV and other spreadsheet-type formats) is the ability to nest objects and arrays:
Sen. Joni Ernst as a JSON object
The unitedstates/congress-legislators Github repo contains a wealth of useful, serialized information about the U.S. Congress. From their current legislators file, I've extracted the entry for newly-elected Senator Joni Ernst and converted it to a JSON, which you can download here.
The formatted output:
Senator Ernst is new to Congress (and the first woman to represent Iowa in Congress), so her JSON-object representation is brief enough for now to read on one page. The object includes 4 nested objects, under the attributes, id, name, bio, terms.
Use jq to view each of these attributes separately, by prepending a dot to the desired attribute:
Nested objects
To access a nested attribute, for example, the first part of the name object, simply list the attributes in sequence:
Arrays
Think of an array as a list. In JSON, square brackets are used to enclose an array of objects or values. Below, the attribute fruits points to an array of string values, 'apples', 'oranges', 'pears':
In the case of the Congressmember JSON, each member has a terms attribute, pointing to an array of objects that look like this:
Here, we use jq's notation to access Senator Ernst's terms as an attribute: take care to notice the square brackets which denote the array that is contained in terms:
Because Senator Ernst has so far served only one term, there is only one object in the .terms array. However, trying to access the start attribute, using just the dot notation for attributes, will result in an error:
Accessing objects in an array
The individual objects in an array are indexed by number, starting from zero. In other words, if we want to get the start attribute for Senator Ernst's 1st term, we tell jq that we want the 0th element, and we use the square bracket notation when accessing an array, in this case, the terms array.
In the result below, take particular note of the lack of square brackets: by specifying a specific element in terms, we get a single object in return:
Now try to retrieve the start attribute:
Try accessing the 1st element in terms; if Senator Ernst has only served one term, and that term is accessible via the 0th element, think to yourself what the 1st element in the terms array will contain:
Rep. Nancy Pelosi as a JSON object
The terms array for Sen. Ernst is still an array, even with one object. But let's see how an array of multiple objects works: we can use Rep. Nancy Pelosi as an example, as she's been in Congress for multiple terms:
How many terms has Pelosi served, exactly? There may be a specific filter in jq that returns a count, but let's just use the good ol' wc tool and pipe the result of jq into it:
All of Congress
You can continue to practice your JSON selectors. Or just move on to the entire Congress, because what we've learned so far should apply to all Congressmember objects.
With the JSON files for Rep. Pelosi and Sen. Ernst, we were dealing with single Congressmember objects. This legislators.json contains an array of those Congressmember objects. So this query to find the name attribute will fail:
So we have to use the array selector (the []) to access individual objects. To find the name object of the first Congressmember object (again, remember that elements in the array are numbered starting with 0):
Keep playing with jq and check out its manual for more examples. We've just gone over some of the concepts on how JSON is structured. In later exercises, we'll use some of jq's more advanced features.
Testing the Github API
Just another example of JSON data to parse:
Status endpointJq For Mac Download
Check out Github's status page, which has some nice visualizations about their current uptime and response speed.
Here's the API and JSON format that is behind that page: status.github.com/api
To get the current status as raw JSON:
Passing it through jq's parser, with the most basic option, will make it look nicer (with colors):
Do some basic selection:
Select both elements:
JSON stands for JavaScript Object Notation and is nearly ubiquitous as a data format, for its lightweight nature and (relatively) ease of human-readability.
Using jq
jq is a program described as 'sed for JSON data':
You can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.
Installing jq
If you've haven't installed the pup HTML parsing tool, or any other tool which required you to edit your ~/.bashrc's' PATH and create the ~/bin_compciv directory, then follow the instructions here: Installing programs for your personal Stanford account.
To install jq:
Check out the tutorial on jq here. The full manual can be found here.
JSON compared to CSV
JSON is more verbose, but easier to read than CSV.
CSV:
JSON:
Objects and attribute-value pairs
Inherent to the JSON data format is the ability to encapsulate data structures known as Objects consisting of attribute-value pairs
The object below has the attributes first_name and last_name, with the values of John and Smith, respectively:
JSON objects can also contain Arrays, which can be thought of as lists of elements. Below, the things_carried attribute has an Array containing apples, hat, and harmonica.
What makes JSON more flexible than flat data structures (such as CSV and other spreadsheet-type formats) is the ability to nest objects and arrays:
Sen. Joni Ernst as a JSON object
The unitedstates/congress-legislators Github repo contains a wealth of useful, serialized information about the U.S. Congress. From their current legislators file, I've extracted the entry for newly-elected Senator Joni Ernst and converted it to a JSON, which you can download here.
The formatted output:
Senator Ernst is new to Congress (and the first woman to represent Iowa in Congress), so her JSON-object representation is brief enough for now to read on one page. The object includes 4 nested objects, under the attributes, id, name, bio, terms.
Use jq to view each of these attributes separately, by prepending a dot to the desired attribute:
Nested objects
To access a nested attribute, for example, the first part of the name object, simply list the attributes in sequence:
Arrays
Think of an array as a list. In JSON, square brackets are used to enclose an array of objects or values. Below, the attribute fruits points to an array of string values, 'apples', 'oranges', 'pears':
In the case of the Congressmember JSON, each member has a terms attribute, pointing to an array of objects that look like this:
Here, we use jq's notation to access Senator Ernst's terms as an attribute: take care to notice the square brackets which denote the array that is contained in terms:
Because Senator Ernst has so far served only one term, there is only one object in the .terms array. However, trying to access the start attribute, using just the dot notation for attributes, will result in an error:
Accessing objects in an array
The individual objects in an array are indexed by number, starting from zero. In other words, if we want to get the start attribute for Senator Ernst's 1st term, we tell jq that we want the 0th element, and we use the square bracket notation when accessing an array, in this case, the terms array.
In the result below, take particular note of the lack of square brackets: by specifying a specific element in terms, we get a single object in return:
Now try to retrieve the start attribute:
Try accessing the 1st element in terms; if Senator Ernst has only served one term, and that term is accessible via the 0th element, think to yourself what the 1st element in the terms array will contain:
Rep. Nancy Pelosi as a JSON object
The terms array for Sen. Ernst is still an array, even with one object. But let's see how an array of multiple objects works: we can use Rep. Nancy Pelosi as an example, as she's been in Congress for multiple terms:
How many terms has Pelosi served, exactly? There may be a specific filter in jq that returns a count, but let's just use the good ol' wc tool and pipe the result of jq into it:
All of Congress
You can continue to practice your JSON selectors. Or just move on to the entire Congress, because what we've learned so far should apply to all Congressmember objects.
With the JSON files for Rep. Pelosi and Sen. Ernst, we were dealing with single Congressmember objects. This legislators.json contains an array of those Congressmember objects. So this query to find the name attribute will fail:
So we have to use the array selector (the []) to access individual objects. To find the name object of the first Congressmember object (again, remember that elements in the array are numbered starting with 0):
Keep playing with jq and check out its manual for more examples. We've just gone over some of the concepts on how JSON is structured. In later exercises, we'll use some of jq's more advanced features.
Testing the Github API
Just another example of JSON data to parse:
Status endpointJq For Mac Download
Check out Github's status page, which has some nice visualizations about their current uptime and response speed.
Here's the API and JSON format that is behind that page: status.github.com/api
To get the current status as raw JSON:
Passing it through jq's parser, with the most basic option, will make it look nicer (with colors):
Do some basic selection:
Select both elements:
The jq tool outputs formatted text that looks nice, but often we just need it in a raw form that can be filtered into our other command-line tools – notice how the --raw-output option removes the quotation marks from the output:
As per the jq tutorial, you can take a look at what Github records when you push your commits to your compciv repo:
If you're really that interested in Github's API and its other endpoints, check out the documentation here.
Jq For Mac Shortcut
Sometimes you just want to read a JSON config file from Bash and iterate over an array. For example, when seeding some credentials to a credential store. This sometimes can be tricky especially when the JSON contains multi-line strings (for example certificates).
In this blog post I will explain how this can be done with jq and a Bash for loop.
Jq Command Not Found
First we will start with some data:
By using jq --compact-output (or -c) we can get each object on a newline.
We could start iterating of the above with a Bash for loop if our data does not contain spaces or newlines. But since certificates contain newlines we better base64 encode each line. Also, instead of -c, we now use -r to get rid of the extra quotes.
Now let's build our for loop.
