Embedding Gists Inside HTML

To embed inside of an HTML page look for the “Embed this gist” box to the left of a gist. Copy the code listed there (which will look something like <script src="https://gist.github.com/xrd/8923697.js"></script>) and paste it into your HTML.

If you wish to include only a particular file from the gist (if it contains multiple files), then add ?file=hi.rb to the end of the URL specified in the src attribute.

Embedding Inside Jekyll Blogs

Jekyll blogs (explained in Chapter 6) can easily host gists using a special syntax. The shortcut {% gist 8138797 %} will embed a private gist, which would be found at http://gist.github.com/8138797. If you want to use a specific file within the gist, add a filename to the gist code like {% gist 8138797 hi.rb %}. Secret gists can also be embedded. If you use a secret gist, prefix the username of the account holder in the gist like so: {% gist xrd/8138797 hi.rb %}.

Now let’s look at creating gists from outside the GitHub.com site, using the command-line.

Going Deeper into the Gist API

The GitHub API uses hypermedia instead of basic resource-driven APIs. If you use a client like Octokit, the hypermedia details are hidden behind an elegant Ruby client. But there is a benefit to understanding how hypermedia works when you need to retrieve deeper information from the GitHub API.

Most RESTful APIs come with a “sitemap,” generally an API reference document that tells a user which endpoints to use. You view the resources available from that API and then apply some HTTP verb to do something to them. Hypermedia thinks of an API differently. Hypermedia APIs describe themselves inside their responses using “affordances.” What this means is that the API might respond like this:

{
    "_links": {
        "self": {
            "href": "http://shop.oreilly.com/product/0636920030300.do"
        }
    }
    "id": "xrd",
    "name": "Chris Dawson"
}

In this payload, you can see that there is an id ("xrd") and a name ("Chris Dawson"). This particular payload was forked from the HAL explanation at the HAL Primer document, and you can find a more detailed explanation of these concepts there.

The important thing to note about hypermedia APIs is that payloads contain metadata about data itself and metadata about the possible options of operating on the data. RESTful APIs typically provide a mapping outside of the payload. You have to join the API sitemap with the data in an ad hoc way when using RESTful APIs; with hypermedia APIs your client can react to the payload itself correctly and intelligently without knowing anything about a sitemap stored in human-readable documentation.

This loose coupling makes APIs and their clients flexible. In theory, a hypermedia API works intuitively with a hypermedia-aware client. If you change the API, the client, as it understands hypermedia, can react and still work as expected. Using a RESTful API means that clients must be updated (a newer version of the client must be installed) or the client code must be upgraded. Hypermedia APIs can alter their backend, and then the client, as long as it is hypermedia-aware, can automatically and dynamically determine the right way to access information from the response itself. In other words, with a hypermedia client the API backend can change and your client code should not need to.

This is explained in great detail in the book Building Hypermedia APIs with HTML5 and Node (O’Reilly).

Using Hypermedia Data from Octokit

Now that you know a little about hypermedia, let’s navigate it using Octokit:

• Start at a resource, with code like user = Octokit.user "xrd". This begins the initialization of the client.

• user now is an object filled with the actual data of the resource. In this case, you could call a method like user.followers to see a meager follower count.

• user also has hypermedia references. You can see these by calling user.rels. This retrieves the relationships described in the hypermedia links.

• Relationships (found by calling user.rels) include avatar, self, followers, etc.

• Use a relationship by calling the get.data method to retrieve and access the data from the GitHub API (followers = user.rels[:followers].get.data).

• Calling .get.data populates an array of the followers (paged if it exceeds 100 items).

Let’s extend our Sinatra app to retrieve actual data about the user’s gists by using hypermedia references:

require 'sinatra'
require 'octokit'

set :views, "."

helpers do
  def h(text)
    Rack::Utils.escape_html(text)
  end
end

get '/:username' do |username|
  gists = Octokit.gists username, :per_page => 5
  erb :index, locals: { :gists => gists, username: username }
end

The index.erb file contains code to iterate over each gist and pull the content. You can see that our response object is an array of gists, and each has an attribute called fields. This fields attribute specifies the filenames available in each gist. If you reference that filename against the files, the response includes a hypermedia ref attribute. Retrieve the raw content using the Octokit method .get.data:

<html>
<body>

<h2>User <%= username %>'s last five gists</h2>

<% gists.each do |g| %>
<% g[:files].fields.each do |f| %>
<b><%= f %></b>:

<%= h g[:files][f.to_sym].rels[:raw].get.data %>

<br/>
<br/>

<% end %>
<% end %>

</body>
</html>

Now we see the gists and the contents, as in Figure 2-3.

Figure 2-3. Last five gists, with details

Repository Linked Wikis

Any repository on GitHub, public or private, can have an associated Gollum wiki. To create a wiki linked to your repository, visit the repository page and then look in the rightmost colum. You’ll see an icon that looks like a book, next to which will be the word “Wiki,” as in Figure 3-1.

Figure 3-1. Accessing the associated wiki from the sidebar

Clicking this link will bring you to a page where you are asked to create a wiki for the first time. GitHub will ask you to create the “Home” page, which is the starting point in a Gollum wiki (Figure 3-2). GitHub will automatically create a page template with the project name; you can customize this information to suit your own needs. Clicking “Save Page” will save your first page and create the wiki for you.

Figure 3-2. Genesis of a new wiki, creating the home page

Your wiki is now as public as your repository is public. Public repositories have public wikis, accessible to anyone. Private repositories have private wikis, accessible only to those users or organizations that have rights to edit the repository data.

Let’s review the markup options for Gollum wikis now.

Markup and Structure

Gollum files can be written in any of the supported “Github Markup” formats, which include ASCIIdoc, Creole, Markdown, Org Mode, Pod, RDoc, ReStructuredText, Textile, and MediaWiki. The variety of markup languages brings flexibility but it can be difficult to know which one to use. Markdown (and its variant cousins) is the most popular markup language on GitHub, and is well liked on other popular sites like Stack Overflow. If you are unsure about which language to use, Markdown is a safe bet because it is ubiquitous across GitHub. Chapter 6 has a much deeper overview of Markdown.

If you do choose Markdown, in addition to the standard vanilla Markdown language tags, Gollum adds its own set of wiki-specific tags. There are often subtle (or conflicting) differences from other wiki markup so it is worth reviewing the Gollum repository documentation page. We’ll go over the most important ones here.

```ruby
def hello
  puts "hello"
end
```

```ruby:github:xrd/TeddyHyde/blob/master/Gemfile```

```ruby:/lib/gollum/app.rb```

Inserting images

Images are inserted into your document using the same tag format [[ceo.png]]: this adds the correct HTML tags to include an image named ceo.png inside your page. This basic syntax is often extended for additional funtionality. For example, to add a frame and an alt tag, you could use syntax like [[ceo.png|frame|alt=Our CEO relaxing on the beach]]. This creates the proper HTML tags for the same image, and also adds a frame and alt text (helpful for better context and the extra information is used by screenreaders for visually impaired users as well). Review the documentation on the Gollum repository for more details about the breadth of the image options.

You can also add images using the editor on GitHub. But you’ll notice that either way you are adding a link to an image and that there is no way to upload images into GitHub from the editor (Figure 3-3).

Figure 3-3. No image upload, only image URLs

For nontechnical users, this makes Gollum wikis on GitHub almost unusable if they need to add images. Let’s address this problem by building our own customized image-centric Gollum editor that still interoperates with regular Gollum wikis. We can put this editor in front of nontechnical users, allowing them to add images, and then publish the wiki into GitHub as is.

Authentication

Your identity as a user can determine the result set from a search query, so it’s important to know about authentication. We cover GitHub authentication fully in “Authentication”, but this API is also available without logging in. However, there are a few limitations to this approach.

First, you’ll only be able to search public repositories. This is probably fine if you’re primarily working with open source software, but users of your application will probably expect to have access to their private code, as well as that of any organizations they belong to. Also, since all Enterprise repositories are private, anonymous search is completely useless there.

Secondly, authenticating opens up your rate limit. The limits on search are stricter than other APIs anyway, because search is computationally expensive, but anonymous searches are stricter still. As of this writing, and according to the documentation, anonymous searches are limited to 5 per minute, and you can do 20 authenticated queries per minute. Take a look at “GitHub API Rate Limits” for more on how to work with rate limits.

Here’s that same information in tabular form:

Result Format

No matter what you’re searching for, the return value from the API follows a certain format. Here’s a sample result from a query, which has been heavily edited to focus only on the parts you’ll always see:

{
  "total_count": 824,
  "incomplete_results": false,
  "items": [
    {
      …
      "score": 3.357718
    }
  ]
}

Starting from the top: the total_count field represents the total number of search results that turned up from this query. It’s not uncommon for a fairly specific search to turn up thousands of results—remember, there are millions of repositories on GitHub. By default, only the first 30 are returned, but you can customize this with page and per_page query parameters in the URL. For example, a GET request to this URL will return 45 items, starting with the 46th result:

search/repositories?q=foobar&page=2&page_size=45

Page sizes are generally limited to 100.

The incomplete_results field refers to a computational limit placed on the Search API. If your search takes too long, the GitHub API will stop it partway through executing, return the results that did complete, and set this flag to true. For most queries this won’t be a problem, and the total_count will represent all the results from the search, but if your query is complicated, you might only get a partial result set.

Search results are returned in the items array, and each item always has a score field. This field is numeric, but it’s only a relative measure of how well a result matches the query, and is used for the default sort order—highest score first. If you do pay attention to it, remember it only has meaning when compared to other results from the same query; a result with a score of 50 isn’t necessarily ten times “better” than a result scored 5.

Here’s a summary of the important fields:

Search Operators and Qualifiers

Of course, it’s always better if you can avoid pagination altogether, or at least get the best results in the first page. Qualifiers and operators can help narrow your search results to fewer pages, hopefully allowing the right result to float to the top.

With the Search API, all searches are done through a search query, which is encoded and passed in the URL as the q parameter. Most of the query will be free text, but the API also supports some powerful syntax, such as these forms:

• x AND y, as well as OR and NOT

• user:<name>, where name is a user or organization

• repo:<name>

• language:<name>

• created:<date(s)>

• extension:<pattern> matches file extensions (like py or ini)

Numerical values and dates can have ranges:

• 2015-02-01 will match only the given date

• <2015-02-01 will match any date previous to the one given

• 2015-02-01..2015-03-01 will match dates within the given range, including the endpoints

For example, to find code written by the user tpope during July of 2012, you would write "user:tpope created:2012-07-01..2015-07-31" for the query parameter. That would be encoded in a URL like so:

search/repositories?q=user%3Atpope+created%3A2012-07-01..2015-07-31

To constrain this search to only Python code, we could add ` language=python`, URL encoded as +language%3Apython, to the end of the URL.

There are many other options. Check out https://github.com/search/advanced for a UI that can help you construct a query.

Sorting

If search query operators can’t narrow down a result set to just the most important items, perhaps sorting them can. Search results are returned in a definite order, never at random. The default order is “best match,” which sorts your results based on their search score, best score first. If you want to override this, you can pass stars, forks, or updated in the sort query parameter, as in search/repositories?q=foobar&sort=stars.

You can also reverse the sort order using the order parameter, like search/repositories?q=foobar&sort=stars&order=desc. The default is desc (“descending”), but asc is also accepted, and will reverse the order.

Repository Search

The search/repositories endpoint looks in the repository metadata to match your query. This includes the project’s name and description by default, though you can also search the read me file by specifying in:readme in the query. Other qualifiers are documented at https://developer.github.com/v3/search/#search-repositories.

A query such as search/repositories?q=foobar might result in a response that looks something like this:

{
  "total_count": 824,
  "incomplete_results": false,
  "items": [
    {
      "id": 10869370,
      "name": "foobar",
      "full_name": "iwhitcomb/foobar",
      "owner": {
        "login": "iwhitcomb",
        "id": 887528,
        "avatar_url": "https://avatars.githubusercontent.com/u/887528?v=3",
        …
      },
      "private": false,
      "html_url": "https://github.com/iwhitcomb/foobar",
      "description": "Drupal 8 Module Example",
      "fork": false,
      …
      "score": 59.32314
    },
    …
  ]
}

Each item in items is the description of a repository. All sorts of useful information is included, such as a URL to the UI for this repository (html_url), the owner’s avatar (owner.avatar_url), and a URL suitable for cloning the repository using Git (git_url).

Code Search

The search/code endpoint is for searching the contents of a repository. You can try matching the contents of the files themselves, or their paths (using in:path). (For complete documentation on the other available qualifiers, check out https://developer.github.com/v3/search/#search-code.)

This API is subject to several limits that don’t affect the other search endpoints, because of the sheer amount of data the server must sort through to find matches. First, it requires that you provide a general search term (a phrase to match); specifying a query with only operators (like language:python) is valid with other APIs, but not here. Second, any wildcard characters in the query will be ignored. Third, files above a certain size will not be searched. Fourth, it only searches the default branch of any given project, which is usually master. Fifth, and possibly most importantly, you must specify a repository owner using the user:<name> qualifier; you cannot search all repositories with one query.

The JSON returned looks something like this:

{
  "total_count": 9246,
  "incomplete_results": false,
  "items": [
    {
      "name": "migrated_0000.js",
      "path": "test/fixtures/ES6/class/migrated_0000.js",
      "sha": "37bdd2221a71b58576da9d3c2dc0ef0998263652",
      "url": "…",
      "git_url": "…",
      "html_url": "…",
      "repository": {
        "id": 2833537,
        "name": "esprima",
        "full_name": "jquery/esprima",
        "owner": {
          "login": "jquery",
          "id": 70142,
          "avatar_url": "https://avatars.githubusercontent.com/u/70142?v=3",
          …
        },
        "private": false,
        …
      },
      "score": 2.3529532
    },
    …
  ]
}

Each item has some data about the file that turned up, including its name and URLs for a couple of representations of it. Then there’s the blob of data about its repository, followed by a score, which is used for the default “best match” sorting.

Issue Search

Repositories contain more than just code. The search/issues endpoint looks for matches in the issues and pull requests attached to a project. This endpoint responds to a wide variety of search qualifiers, such as:

type

Either “pr” for pull requests, or “issue” for issues (the default is both).

team

Match issues whose discussions mention a specific team (only works for organizations you belong to).

no

Match issues that are missing a piece of data (as in “no:label”).

There are many more; see https://developer.github.com/v3/search/#search-issues for complete documentation.

The result of a call to this endpoint looks like this:

{
  "total_count": 1278397,
  "incomplete_results": false,
  "items": [
    {
      "url": "…",
      "labels_url": "…",
      "comments_url": "…",
      "events_url": "…",
      "html_url": "…",
      "id": 69671218,
      "number": 1,
      "title": "Classes",
      "user": {
        "login": "reubeningber",
        "id": 2552792,
        "avatar_url": "…",
        …
      },
      "labels": [
        …
      ],
      "state": "open",
      "locked": false,
      "assignee": null,
      "milestone": null,
      "comments": 0,
      "created_at": "2015-04-20T20:18:56Z",
      "updated_at": "2015-04-20T20:18:56Z",
      "closed_at": null,
      "body": "There should be an option to add classes to the ul and li...",
      "score": 22.575937
    },
  ]
}

Again, each item in the list looks like the result of a call to the issued API. There are a lot of useful bits of data here, such as the issue’s title (title), labels (labels), and links to information about the pull-request data (pull_request.url), which won’t be present if the result isn’t a pull request.

User Search

All the other Search APIs are centered around repositories, but this endpoint searches a different namespace: GitHub users. By default, only a user’s login name and public email address are searched; the in qualifier can extend this to include the user’s full name as well, with in:fullname,login,email. There are several other useful qualifiers available; see https://developer.github.com/v3/search/#search-users for complete documentation.

Querying the search/users endpoint gives you this kind of response:

{
  "total_count": 26873,
  "incomplete_results": false,
  "items": [
    {
      "login": "ben",
      "id": 39902,
      "avatar_url": "…",
      "gravatar_id": "",
      "url": "…",
      "html_url": "…",
      …
      "score": 98.24275
    },
    {
      "login": "bengottlieb",
      "id": 53162,
      "avatar_url": "…",
      "gravatar_id": "",
      "url": "…",
      "html_url": "…",
      …
      "score": 35.834213
    },
  ]
}

The list of items in this case look like the results from a query of the users/<name> endpoint. Useful items here are the user’s avatar (avatar_url), several links to other API endpoints (repos_url, url), and the type of result (user or organization, in type).

User Flow

That’s the overall goal, but let’s dig in to more detail about how the user experiences the application.

The first thing we’ll do is require the user to log in with GitHub credentials. Why? Partly because the Search API is throttled pretty aggressively, and the rate limits are higher with authenticated access. But also because our user is going to need the ability to search issues in private repositories. To make this easier, our program will try to get GitHub credentials from Git’s credential store, but it’ll fall back to a login form, which looks like Figure 4-4.

Figure 4-4. Login UI

Once the user logs in, they’ll be shown a search box. Typing in a search query and hitting Enter will result in a scrollable list of search results, with titles and the first line of the description. Clicking a search result opens the issue in the user’s browser.

That’s about it. This application only has two main screens from the user’s point of view. It’s a simple, focused tool to solve a very tightly defined problem, so the code shouldn’t be too hard.

AGitHub

The first thing we should mention is the library we’ll use to talk to the GitHub API, which is called agithub. agithub is a very thin layer that converts GitHub’s REST API into method calls on objects, resulting in delightfully readable code.

agithub can be found at https://github.com/jpaugh/agithub, and the “installation” is simply to download a copy of the agithub.py source file and place it alongside your project files.

WxPython

WxPython is how we’ll create the graphical interface for our application. It’s an object-oriented Python layer over the top of a toolkit called WxWidgets, which is itself a common-code adapter for native UI toolkits. WxWidgets supports Linux, Mac, and Windows operating systems with native controls, so you can access all of those platforms with the same Python code.

Information about the WxPython project can be found at http://www.wxpython.org, and you’ll find a download link for your platform on the lefthand side of the page. The next version of WxPython (code-named “Phoenix”) will be installable via PIP, but at the time of this writing Phoenix is still prerelease software, so it’s probably safer to use the stable version.

PyInstaller

PyInstaller will be our distribution tool. Its main function is to read your Python code, analyze it to discover all its dependencies, then collect all these files (including the Python interpreter) and put them in one directory. It can even wrap all of that up in a single package that, when double-clicked, runs your program. It does all this without needing much input from you, and there are only a few configuration options. If you’ve written GUI applications before, you’ll know how hard each of these problems are.

For information on this project, you can visit http://pythonhosted.org/PyInstaller. You can install it using Python’s package manager by running pip install pyinstaller.

Git Credential Helper

That’s how most of the UI code is going to be structured, but before we go any further, we should define a function to help us get the user’s GitHub credentials. We’ll be cheating a bit, by asking Git if it has the user’s login and password.

We’ll leverage the git credential fill command. This is used internally by Git to avoid having to ask the user for their GitHub password every time they interact with a GitHub remote. The way it works is by accepting all the known facts about a connection as text lines through stdin, in the format <key>=<value>. Once the caller has supplied all the facts it knows, it can close the stdin stream (or supply an empty line), and Git will respond with all the facts it knows about this connection. With any luck, this will include the user’s login and password. The whole interaction looks a bit like this:

$ echo "host=github.com" | git credential fill 
host=github.com
username=ben 
password=(redacted)

This passes a single line to git credential and closes stdin, which Git will recognize as the end of input.

Git responds with all the facts it knows about the connection. This includes the input values, as well as the username and password if Git knows them.

One other thing you should know about git-credential is that by default, if it doesn’t know anything about the host, it’ll ask the user at the terminal. That’s bad for a GUI app, so we’re going to be disabling that feature through the use of the GIT_ASKPASS environment variable.

Here’s what our helper looks like:

GITHUB_HOST = 'github.com'
def git_credentials():
    os.environ['GIT_ASKPASS'] = 'true' 
    p = subprocess.Popen(['git', 'credential', 'fill'],
                         stdout=subprocess.PIPE,
                         stdin=subprocess.PIPE) 
    stdout,stderr = p.communicate('host={}\n\n'.format(GITHUB_HOST)) 

    creds = {}
    for line in stdout.split('\n')[:-1]: 
        k,v = line.split('=')
        creds[k] = v
    return creds 

Here we set GIT_ASKPASS to the string 'true', which is a UNIX program that always succeeds, which will in turn cause git-credential to stop trying to get credentials when it gets to the “ask the user” stage.

subprocess.Popen is the way you use a program with stdin and stdout in Python. The first argument is a list of arguments for the new program, and we also specify that we want stdin and stdout to be captured.

p.communicate does the work of writing to stdin and returning the contents of stdout. It also returns the contents of stderr, which we ignore in this program.

Here we process the stdout contents by splitting each line at the = character and slurping it into a dictionary.

So the return value from this call should be a dictionary with 'username' and 'password' values. Handy!

Windowing and Interface

Okay, so now we have something that can help us skip a login screen, but we don’t have a way of showing that login screen to the user. Let’s get closer to that goal by filling in the main frame’s implementation:

class SearchFrame(wx.Frame):
    def __init__(self, *args, **kwargs): 
        kwargs.setdefault('size', (600,500))
        wx.Frame.__init__(self, *args, **kwargs)

        self.credentials = {}
        self.orgs = []

        self.create_controls()
        self.do_layout()
        self.SetTitle('GitHub Issue Search')

        # Try to pre-load credentials from Git's cache
        self.credentials = git_credentials()
        if self.test_credentials():
            self.switch_to_search_panel()

        self.Show()

There’s a bit of syntax here that might be confusing. The *args and **kwargs entries here are ways of capturing multiple arguments into one parameter. For now, just know that we’re only capturing them here so we can pass them to the parent class constructor two lines down.

The __init__ method is the constructor, so this is where we start when the main function calls SearchFrame(). Here’s what’s happening at a high level—we’ll dig into the details in a bit:

1. Set up some layout dimensions and pass to the parent class’s constructor

2. Create the UI controls

3. Retrieve the credentials from the user using the credential helper we described earlier

4. Change the title and display the application to the user

Before we get to how all those things are done, let’s step back a bit and talk about this class’s job. It’s responsible for maintaining the top-level frame (a window with a title bar, a menu, and so on), and deciding what’s displayed in that frame. In this case, we want to show a login UI first, and when we get valid credentials (either from Git or the user), we’ll switch to a searching UI.

Alright, enough background. Let’s walk through the code for getting and checking credentials:

    def login_accepted(self, username, password):
        self.credentials['username'] = username
        self.credentials['password'] = password
        if self.test_credentials():
            self.switch_to_search_panel()

    def test_credentials(self):
        if any(k not in self.credentials for k in ['username', 'password']):
            return False
        g = Github(self.credentials['username'], self.credentials['password'])
        status,data = g.user.orgs.get() 
        if status != 200:
            print('bad credentials in store')
            return False
        self.orgs = [o['login'] for o in data] 
        return True

    def switch_to_search_panel(self):
        self.login_panel.Destroy()
        self.search_panel = SearchPanel(self,
                                        orgs=self.orgs,
                                        credentials=self.credentials)
        self.sizer.Add(self.search_panel, 1, flag=wx.EXPAND | wx.ALL, border=10)
        self.sizer.Layout()

The agithub library always returns two values from every function call. Python lets us bind these directly to variables with this a,b = <expr> syntax.

agithub decodes the JSON from the API call into a Python dictionary. Here we’re only really interested in the names of the organization, so we use a list comprehension, where we tell Python to only keep the value of the "login" field from each dictionary in the data list.

Each of these three methods comes in at a different point during our program’s execution. If our credentials are coming from Git, we proceed straight to test_credentials; if they’re coming from the login panel (see “GitHub Login”), they go through the login_accepted callback first, which then calls test_credentials.

Either way, what we do is try to fetch a list of the user’s organizations, to see if they work. Here you can see the usage pattern for agithub—the URL path is mapped to object-property notation on an instance of the Github class, and the HTTP verb is mapped to a method call. The return values are a status code and the data, which has been decoded into a dictionary object. If it fails—meaning the returned status is not 200—we send the user to the login panel. If it succeeds, we call switch_to_search_panel.

The last method handles the UI switch. The login panel is referenced by two things: the SearchFrame instance (the parent window), and the sizer that’s controlling its layout. Fortunately, calling the Destroy() method cleans both of those up, so we can then create the SearchPanel instance and add it to our sizer. Doing this requires a specific call to the sizer’s Layout() method; otherwise, the sizer won’t know that it needs to adjust the position and size of the new panel:

    def create_controls(self):
        # Set up a menu. This is mainly for "Cmd-Q" behavior on OSX
        filemenu = wx.Menu()
        filemenu.Append(wx.ID_EXIT, '&Exit')
        menuBar = wx.MenuBar()
        menuBar.Append(filemenu, '&File')
        self.SetMenuBar(menuBar)

        # Start with a login UI
        self.login_panel = LoginPanel(self, onlogin=self.login_accepted)

    def do_layout(self):
        self.sizer = wx.BoxSizer(wx.VERTICAL)
        self.sizer.Add(self.login_panel, 1, flag=wx.EXPAND | wx.ALL, border=10)
        self.SetSizer(self.sizer)

create_controls is fairly straightforward. It instantiates a menu that only contains File→Exit, and a login panel, whose implementation we’ll cover a bit later on. Note that when we create a visible control, we pass self as the first parameter to the constructor. That’s because the SearchFrame instance we’re constructing is the parent window of that control.

do_layout uses a WxWidgets feature called sizers to do some automated layout. Sizers are a complex topic, but here’s all you need to know about this snippet:

• A BoxSizer stacks widgets in a single direction, in this case vertically.

• The second parameter to sizer.Add is a scaling factor. If it’s zero, the widget you’re adding will always stay the same size if the parent window resizes; if it’s anything else, all the things the sizer is controlling will adjust to fill their container. There’s only one control in this sizer, but we still want it to take up the full area of the window, so we pass 1.

• The border parameter tells the sizer how much area to leave around the widget as padding.

• The wx.EXPAND flag tells the sizer that we want the widget to expand in the direction the sizer isn’t stacking. In this case, we’re stacking vertically, but we also want this widget to expand horizontally.

• The wx.ALL flag specifies which edges of the widget should have the border area.

Let’s make sure we’re following good practices, and write some tests. There isn’t a lot here we can verify automatedly, but what there is should be covered:

from nose.tools import eq_, ok_, raises 

class TestApp:
    def setUp(self): 
        self.f = None
        self.app = wx.App()

    def tearDown(self):
        if self.f:
           self.f.Destroy()
        self.app.Destroy()

    def test_switching_panels(self): 
        self.f = SearchFrame(None, id=-1)
        # Sub-panels should exist, and be of the right type
        ok_(isinstance(self.f.login_panel, LoginPanel))
        ok_(isinstance(self.f.search_panel, SearchPanel))
        # Already destroyed
        raises(RuntimeError, lambda: self.f.login_panel.Destroy())
        # Not already destroyed
        ok_(self.f.search_panel.Destroy())

Here we’re using a testing tool called Nose. Install it with pip install nose, and invoke it at the command line by typing nosetests app.py. It uses naming conventions to identify tests and fixtures, and is generally nice to work with.

Nose will automatically find these setUp and tearDown methods, and call them before and after each test method is run. In this case, we’re just managing the frames we want to test, as well as an App instance for all of them to belong to.

Here’s a test method that Nose will find and run. We ensure the subpanels are the right type, and that we’ve auto-transitioned to the SearchPanel by finding credentials in Git’s storage.

That’s it! Aside from managing a couple of fields, most of this code is managing the UI, which is almost exactly what we’d want from a UI class. Let’s write the first of the two panels we swap in and out.

GitHub Login

The LoginPanel class is similar in structure to the SearchFrame class, with a couple of key differences, which we’ll describe after the wall of code:

class LoginPanel(wx.Panel):
    def __init__(self, *args, **kwargs):
        self.callback = kwargs.pop('onlogin', None)
        wx.Panel.__init__(self, *args, **kwargs)

        self.create_controls()
        self.do_layout()

    def create_controls(self):
        self.userLabel = wx.StaticText(self, label='Username:')
        self.userBox = wx.TextCtrl(self, style=wx.TE_PROCESS_ENTER)
        self.passLabel = wx.StaticText(self, label='Password (or token):')
        self.passBox = wx.TextCtrl(self, style=wx.TE_PROCESS_ENTER)
        self.login = wx.Button(self, label='Login')
        self.error = wx.StaticText(self, label='')
        self.error.SetForegroundColour((200,0,0))

        # Bind events
        self.login.Bind(wx.EVT_BUTTON, self.do_login)
        self.userBox.Bind(wx.EVT_TEXT_ENTER, self.do_login)
        self.passBox.Bind(wx.EVT_TEXT_ENTER, self.do_login)

    def do_layout(self):
        # Grid arrangement for controls
        grid = wx.GridBagSizer(3,3)
        grid.Add(self.userLabel, pos=(0,0),
                 flag=wx.TOP | wx.LEFT | wx.BOTTOM, border=5)
        grid.Add(self.userBox, pos=(0,1),
                 flag=wx.EXPAND | wx.LEFT | wx.RIGHT, border=5)
        grid.Add(self.passLabel, pos=(1,0),
                 flag=wx.TOP | wx.LEFT | wx.BOTTOM, border=5)
        grid.Add(self.passBox, pos=(1,1),
                 flag=wx.EXPAND | wx.LEFT | wx.RIGHT, border=5)
        grid.Add(self.login, pos=(2,0), span=(1,2),
                 flag=wx.EXPAND | wx.LEFT | wx.RIGHT, border=5)
        grid.Add(self.error, pos=(3,0), span=(1,2),
                 flag=wx.EXPAND | wx.LEFT | wx.RIGHT, border=5)
        grid.AddGrowableCol(1)

        # Center the grid vertically
        vbox = wx.BoxSizer(wx.VERTICAL)
        vbox.Add((0,0), 1)
        vbox.Add(grid, 0, wx.EXPAND)
        vbox.Add((0,0), 2)
        self.SetSizer(vbox)

    def do_login(self, _):
        u = self.userBox.GetValue()
        p = self.passBox.GetValue()
        g = Github(u, p)
        status,data = g.issues.get()
        if status != 200:
            self.error.SetLabel('ERROR: ' + data['message'])
        elif callable(self.callback):
            self.callback(u, p)

There’s some structure that’s similar to the preceding code. We’ll start with the constructor.

Recall that this panel is created with a keyword argument in SearchFrame’s create_controls method, like LoginPanel(self, onlogin=self.login_accepted). In the constructor definition, we pull that callback out and store it for later. Afterward, we just call the two other construction functions and return.

create_controls has more to it than SearchFrame’s version, because this panel has more controls. Every static-text, text-input, and button control gets its own line of code. The wx.TE_PROCESS_ENTER style tells the library we want an event to be triggered if the user presses the Enter key while the cursor is inside that text box.

The next block binds control events to method calls. Every event in WxPython will call the handler with a single argument, an object that contains information about the event. That means we can use the same function to handle any number of different kinds of events, so we do—the ENTER handlers for both text boxes and the BUTTON handler for the button all go through self.do_login.

do_layout uses a different kind of sizer—a GridBagSizer. Again, the topic of sizers is way outside the scope of this chapter, but just know that this kind arranges things in a grid, and you can allow some of the rows or columns to stretch to fill the container. Here we drop all of the controls into their positions with the pos=(r,c) notation (here “rows” come first, which isn’t like most coordinate systems), and cause one control to span two columns with the span parameter. The flags and border parameters mostly mean the same things as before, and the AddGrowableCol function tells the layout engine which parts of the grid should be allowed to stretch.

Then we do something curious: we put the GridBagSizer into another sizer. Sizer nesting is a powerful feature, and allows almost any window layout to be possible—although perhaps not easy or simple. The vertical box sizer also contains some bare tuples; this special form is called “adding a spacer.” In this case, we sandwich the sizer with all the controls between two spacers with different weights, making it float about a third of the way down the window. The effect is like Figure 4-5.

Figure 4-5. Resizing behavior of login UI

Then comes the do_login method, which tests out the given credentials, and if they work, passes them back through the callback set at construction time. If they don’t work, it sets the text of a label, whose foreground color has been set to a nice, alarming shade of red.

Let’s make sure this behavior is tested at least a little bit. Again, there’s not much that it’s doing other than setting up WxPython stuff, but we can validate that a login error is displayed by adding this method to the test class:

    def test_login_panel(self):
        self.f = wx.Frame(None)
        lp = LoginPanel(self.f)
        eq_(lp.error.GetLabelText(), '')
        lp.do_login(None)
        ok_(lp.error.GetLabelText().startswith('ERROR'))

GitHub Search

Once the user has successfully logged in, we destroy the LoginPanel instance and show the SearchPanel:

class SearchPanel(wx.Panel):
    def __init__(self, *args, **kwargs):
        self.orgs = kwargs.pop('orgs', [])
        self.credentials = kwargs.pop('credentials', {}) 
        wx.Panel.__init__(self, *args, **kwargs)

        self.create_controls()
        self.do_layout()

    def create_controls(self):
        self.results_panel = None
        self.orgChoice = wx.Choice(self, choices=self.orgs, style=wx.CB_SORT)
        self.searchTerm = wx.TextCtrl(self, style=wx.TE_PROCESS_ENTER)
        self.searchTerm.SetFocus()
        self.searchButton = wx.Button(self, label="Search")

        # Bind events 
        self.searchButton.Bind(wx.EVT_BUTTON, self.do_search)
        self.searchTerm.Bind(wx.EVT_TEXT_ENTER, self.do_search)

    def do_layout(self):
        # Arrange choice, query box, and button horizontally
        hbox = wx.BoxSizer(wx.HORIZONTAL)
        hbox.Add(self.orgChoice, 0, wx.EXPAND)
        hbox.Add(self.searchTerm, 1, wx.EXPAND | wx.LEFT, 5)
        hbox.Add(self.searchButton, 0, wx.EXPAND | wx.LEFT, 5)

        # Dock everything to the top, leaving room for the results
        self.vbox = wx.BoxSizer(wx.VERTICAL)
        self.vbox.Add(hbox, 0, wx.EXPAND) 
        self.SetSizer(self.vbox)

    def do_search(self, event):
        term = self.searchTerm.GetValue()
        org = self.orgChoice.GetString(self.orgChoice.GetCurrentSelection())
        g = Github(self.credentials['username'], self.credentials['password'])
        code,data = g.search.issues.get(q="user:{} {}".format(org, term)) 
        if code != 200:
            self.display_error(code, data)
        else:
            self.display_results(data['items'])

    def display_results(self, results): 
        if self.results_panel:
            self.results_panel.Destroy()
        self.results_panel = SearchResultsPanel(self, -1, results=results)
        self.vbox.Add(self.results_panel, 1, wx.EXPAND | wx.TOP, 5)
        self.vbox.Layout()

    def display_error(self, code, data): 
        if self.results_panel:
            self.results_panel.Destroy()
        if 'errors' in data:
            str = ''.join('\n\n{}'.format(e['message']) for e in data['errors'])
        else:
            str = data['message']
        self.results_panel = wx.StaticText(self, label=str)
        self.results_panel.SetForegroundColour((200,0,0))
        self.vbox.Add(self.results_panel, 1, wx.EXPAND | wx.TOP, 5)
        self.vbox.Layout()
        width = self.results_panel.GetSize().x
        self.results_panel.Wrap(width)

There’s quite a bit here, but some of it is familiar. We’ll skip the usual walkthrough to point out a couple of interesting features:

When creating the panel, we pass in the user’s credentials and list of organizations as keyword arguments, so they show up in the kwargs dictionary. Here we use pop to make sure the parent class’s constructor doesn’t get confused by them.

Here we capture both the search button’s “click” event, as well as the text box’s “enter key” event. Both should cause the search to be performed.

When we add the search bar to the sizer, we use 0 as a scale factor. This means it shouldn’t expand to fit the available size, but keep its own size instead, to leave room to add a results panel later on.

Here’s where the actual search is being done. We get the search term and organization, and send them to the agithub instance, which returns our results and an HTTP result code.

We pass the search results into another class, then add it to the main sizer with parameters to fill the remaining available space.

If an error is returned from the search call instead, we display it here. There’s some code to adjust the wrap width of the text, based on the laid-out width of the control. This isn’t a great approach, but doing it better is left as an exercise for the reader.

Again, there’s a fair amount of code here, but most of it should look familiar. Here’s the test code that covers the previous code:

    def test_search_panel(self):
        self.f = wx.Frame(None)
        sp = SearchPanel(self.f, orgs=['a', 'b', 'c'])
        eq_(0, sp.orgChoice.GetCurrentSelection())
        eq_('a', sp.orgChoice.GetString(0))
        sp.display_error(400, {'errors': [{'message': 'xyz'}]})
        ok_(isinstance(sp.results_panel, wx.StaticText))
        eq_('xyz', sp.results_panel.GetLabelText().strip())

Displaying Results

So now we have our login panel, and a way for the user to enter a search query, but no way to display results. Let’s fix that.

Whenever search results are retrieved, we create a new instance of SearchResultsPanel, which then creates a series of SearchResult instances. Let’s look at both of them together:

class SearchResultsPanel(wx.ScrolledWindow): 
    def __init__(self, *args, **kwargs):
        results = kwargs.pop('results', [])
        wx.PyScrolledWindow.__init__(self, *args, **kwargs)

        # Layout search result controls inside scrollable area
        vbox = wx.BoxSizer(wx.VERTICAL)
        if not results:
            vbox.Add(wx.StaticText(self, label="(no results)"), 0, wx.EXPAND)
        for r in results:
            vbox.Add(SearchResult(self, result=r),
                     flag=wx.TOP | wx.BOTTOM, border=8)
        self.SetSizer(vbox)
        self.SetScrollbars(0, 4, 0, 0)

class SearchResult(wx.Panel):
    def __init__(self, *args, **kwargs):
        self.result = kwargs.pop('result', {})
        wx.Panel.__init__(self, *args, **kwargs)

        self.create_controls()
        self.do_layout()

    def create_controls(self): 
        titlestr = self.result['title']
        if self.result['state'] != 'open':
            titlestr += ' ({})'.format(self.result['state'])
        textstr = self.first_line(self.result['body'])
        self.title = wx.StaticText(self, label=titlestr)
        self.text = wx.StaticText(self, label=textstr)

        # Adjust the title font
        titleFont = wx.Font(16, wx.FONTFAMILY_DEFAULT,
                            wx.FONTSTYLE_NORMAL, wx.FONTWEIGHT_BOLD)
        self.title.SetFont(titleFont)

        # Bind click and hover events on this whole control 
        self.Bind(wx.EVT_LEFT_UP, self.on_click)
        self.Bind(wx.EVT_ENTER_WINDOW, self.enter)
        self.Bind(wx.EVT_LEAVE_WINDOW, self.leave)

    def do_layout(self):
        vbox = wx.BoxSizer(wx.VERTICAL)
        vbox.Add(self.title, flag=wx.EXPAND | wx.BOTTOM, border=2)
        vbox.Add(self.text, flag=wx.EXPAND)
        self.SetSizer(vbox)

    def enter(self, _):
        self.title.SetForegroundColour(wx.BLUE)
        self.text.SetForegroundColour(wx.BLUE)

    def leave(self, _):
        self.title.SetForegroundColour(wx.BLACK)
        self.text.SetForegroundColour(wx.BLACK)

    def on_click(self, event): 
        import webbrowser
        webbrowser.open(self.result['html_url'])

    def first_line(self, body):
        return body.split('\n')[0].strip() or '(no body)'

The containing panel is simple enough that it only consists of a constructor. This class’s job is to contain the results and present them in a scroll window.

A SearchResult is comprised of two static text controls, which contain the issue’s title and the first line of its body.

We’re not only binding the click handler for this entire panel, but also the mouse-enter and mouse-leave events, so we can make it behave more like a link in a browser.

Here’s how you open the default browser to a URL in Python.

So now you’ve seen the code for a simple WxPython application. Using this library tends to produce code of a certain style, which is kind of verbose. The positive side of this is that nothing is hidden; all the layout for your app is done right in the code, with no “magic,” and the fact that it can run without modification on just about anybody’s computer is a huge plus. WxPython may lack some facilities of newer frameworks, but there’s nothing better for getting a basic cross-platform UI out the door quickly.

That’s all of the code! If you’ve been following along and typing all this code into a file, you can run that file and do issue searches. However, our use case has a nontechnical user running this; let’s see what can be done to make it easier for them to get started.

Raw Statuses

Now that we know how we get access to commit statuses, let’s see what they look like. Commit statuses exist as atomic entities, and each commit can have a practically unlimited number of them (the actual number is in the thousands). You can query for existing statuses by doing a GET request to the API server at /repos/<user>/<repo>/<ref>/statuses, and it will return a list of them that looks like this:

[
  {
    "url": "https://api.github.com/repos/…",
    "id": 224503786,
    "state": "success",
    "description": "The Travis CI build passed",
    "target_url": "https://travis-ci.org/libgit2/libgit2/builds/63428108",
    "context": "continuous-integration/travis-ci/push",
    "created_at": "2015-05-21T03:11:02Z",
    "updated_at": "2015-05-21T03:11:02Z"
  },
  …
]

Most of this is self-explanatory, but a couple of fields need explaining:

This API is useful for getting the raw data involved, but it gets complicated quickly. How do you decide if a given commit is “good?” What if there are three pending statuses, one success, another pending, two failures, and another success, in that order? The context field can help you correlate a single service’s updates, and you can order them by created_at to see how each one turned out, but that’s a lot of work. Fortunately, the API server can do it for you.

Combined Status

If you instead do a GET to /repos/<user>/<repo>/<ref>/status (note that the last word is singular), you’ll instead get a response that looks like this:

{
  "state": "success",
  "statuses": [
    {
      "url": "https://api.github.com/repos/…",
      …
    },
    { … }
  ],
  "sha": "6675aaba883952a1c1b28390866301ee5c281d37",
  "total_count": 2,
  "repository": { … },
  "commit_url": "https://api.github.com/repos/…",
  "url": "https://api.github.com/repos/…"
}

The statuses array is the result of the logic you’d probably write if you had to: it collapses the statuses by context, keeping only the last one. The state field contains an overall status that takes into account all of the contexts, providing a final value based on these rules:

This is probably exactly what you want, but if you find that your use case calls for different rules, you can always use the statuses endpoint to get the raw data and calculate your own combined status.

Creating a Status

Now obviously these statuses have to come from somewhere. This API also includes a facility for creating them. To do this, you simply make a POST request to /repos/<user>/<repo>/statuses/<sha>, and supply a JSON object for the fields you want to include with your status:

Notice how the last component in that URL is <sha>. While you can query for statuses or a combined status using a ref name (like master), creating a status requires you to know the full SHA-1 hash of the commit you want to annotate. This is to avoid race conditions: if you were targeting a ref, it may have moved between when your process started and when it finished, but the SHA of a commit will never change.

Libraries

You’ll be happy to know we won’t be writing an entire HTTP server from scratch in this chapter. There are a number of open source packages that do this work for us, and in this project we’ll be using Nancy. Nancy is a project that started as a CLR port of the Sinatra framework for Ruby (it takes its name from Frank Sinatra’s daughter, Nancy). It’s very capable, but also very succinct, as you’ll see.

We also won’t be directly implementing access to the GitHub API, because GitHub provides a CLR library for that. It’s called octokit.net, and it does all the right things with regard to asynchrony and type safety. This is the same library used by the GitHub client for Windows, so it’ll definitely do the job for our little application. It is, however, the source of a constraint on how we set up our example project: it requires a rather new version of the CLR (4.5) in order to function. If you want some guidance on how to avoid this pitfall and follow along, continue reading the next section. If you’ve worked with Nancy before, and have installed NuGet packages in the past, you might be able to skip to the section labeled “Sending the Request”.

Development Environment

If you’d like to follow along with the code examples, here’s how to set up a development environment with all the necessary elements. The process is different on Windows (using Visual Studio) and any other platforms (using Xamarin tools).

Visual Studio

If you’re running Windows, you’ll want to visit https://www.visualstudio.com/ and download the Community edition of Visual Studio. The installer will present you with lots of options; for this example, we’ll only need the “web developer” components, but feel free to check all the boxes that look interesting to you. (If you have access to a higher tier of Visual Studio, or already have it installed with the web-development packages, you’re all set.)

In order to make things just a little smoother, you’ll want to install a plug-in: the Nancy project templates. Visit https://visualstudiogallery.msdn.microsoft.com/ and search for “nancy.templates.” Choose the search result “Nancy.Templates,” which belongs to the NancyFx organization, and click “Get Now.” This should download a .vsix file that you can double-click to install the templates into Visual Studio.

The next step is to create a new project using one of the newly installed templates. Go to “File→New Project” and select “Visual C#→Web→Nancy Application with ASP.NET Hosting” from the template list (as shown in Figure 5-2). Make sure the path and name settings at the bottom are to your liking, and click OK.

Figure 5-2. Creating a Nancy application in Visual Studio

Next, change the target CLR framework version to something that will work with Octokit. Right-click the project’s node in the Solution Explorer, and select “Properties.” In the Application section, set Target Framework to be .NET 4.5 (or later), and save. You may be prompted to reload the solution.

The very last step is to add NuGet packages for Octokit and Nancy. Right-click the project node in Solution Explorer, and select “Manage NuGet Packages.” Do a search for “Nancy,” and upgrade it if necessary—there’s a chance the Nancy project template specifies an out-of-date version. Then do a search for “Octokit,” and install that. At this point, you should have an empty solution, configured and ready for our example code. To run it with debugging, go to “Debug→Start Debugging,” or hit F5. Visual Studio will start the server under a debugger, and open an IE instance on http://localhost:12008/ (the port might be different), which should serve you the default Nancy “404 Not Found” page.

Xamarin Studio

On OS X and Linux, as of this writing the easiest way forward is to visit http://www.monodevelop.com/ and install MonoDevelop. Mono is an open source implementation of Microsoft’s CLR specification, and MonoDevelop is a development environment that works much like Visual Studio, but is built on Mono, and is completely open source. If you try to download MonoDevelop on a Windows or OS X machine, you’ll be prompted to install Xamarin Studio instead; this is a newer version of MonoDevelop with more capabilities, and will work just as well for these examples.

There are no Nancy-specific project templates for these IDEs, so you’ll just start with an empty web project. Go to “File→New→Solution,” and choose “ASP.NET→Empty ASP.NET Project” from the template chooser, as shown in Figure 5-3.

Figure 5-3. Creating an empty ASP.NET application in Xamarin Studio

The rest of the wizard steps are about the project name and location; feel free to name and locate this project however you like.

Next, update the target framework setting. Control- or right-click the node in the solution explorer that corresponds with your project (not your solution), and select Options from the menu. Under “Build→General,” set the Target Framework to “Mono / .NET 4.5” (or later) and click OK.

Lastly, install the Nancy and Octokit NuGet packages. Go to “Project→Add NuGet Packages” in the menu to open the package manager. Search for Nancy, check the box next to it, search for Octokit, check its box, and click “Add Packages” at the bottom right. Once the process is complete, your project is ready for our example code. To run it under the debugger, go to “Run→Start Debugging,” or type ⌘-Enter. Xamarin will start the server and open a browser window to http://127.0.0.1:80080 (possibly with a different port), which at this point will just show the default “404 Not Found” page.

Sending the Request

Alright, now that we have a project ready for some code, let’s get our Nancy application up and running. Let’s be good engineers, and write our tests first. In order to do this, generate a new unit-test project alongside your existing application project, and add a NuGet reference to the Nancy.Testing package. You can then copy and paste the test examples over the top of the default test module that comes with that template.

The first thing we’re going to write is an endpoint that reports how many followers a user has. In order to test it, we’ll choose a well-known user and make sure their real name is fetched. Here’s what the test code looks like:

using NUnit.Framework;
using Nancy;
using Nancy.Testing;
using Nancy.Bootstrapper;
using System.Collections.Generic;
using Nancy.Session;

namespace NancyApplication1.Tests
{
  [TestFixture ()]
  public class Test
  {
    private Browser browser;

    [SetUp]
    public void Setup(){
      this.bootstrapper =
        new ConfigurableBootstrapper(with => {
        with.Module<Handler>();
      });
      this.browser = new Browser (bootstrapper);
    }

    [Test ()]
    public void FetchesUserDetails ()
    {
      var result = this.browser.Get ("/mojombo", 
        with => with.HttpRequest ());
      Assert.AreEqual (HttpStatusCode.OK, result.StatusCode);
      Assert.IsTrue (result.Body.AsString()
        .Contains("Tom Preston-Werner")); 
    }
  }
}

Here we’re using the Browser class provided by Nancy.Testing to make a request to /mojombo, which should give us the number of likes for that GitHub user.

Here we’re asserting that mojombo’s real name is fetched by the endpoint.

Now that we have a failing test, let’s write the code to implement that endpoint in Nancy. Here’s what the initial version of that file will look like:

using Nancy;
using Octokit;
using System;
using System.Collections.Generic;
using System.Linq;

namespace NancyApp
{
    public class Handler : NancyModule 
    {
        private readonly GitHubClient client =
            new GitHubClient(new ProductHeaderValue("MyHello")); 

        public Handler()
        {
            Get["/{user}", true] = async (parms, ct) => 
                {
                    var user = await client.User.Get(parms.user.ToString()); 
                    return String.Format("{0} people love {1}!",
                                         user.Followers, user.Name); 
                };
        }
    }
}