Simple URL Access with File Functions

An extremely convenient aspect of file access functions like file_get_contents() is that they understand URLs in addition to local filenames. Grabbing a remote URL and putting it into a string is just a matter of handing that URL to file_get_contents().

Example 11-1 uses file_get_contents() to display an interesting fact from the website numbersapi.com about September 27th.

Did you know that <?= file_get_contents('http://numbersapi.com/09/27') ?>

The Numbers API knows a lot of facts about each day, but for me the result of Example 11-1 looked like this:

Did you know that September 27th is the day in 1961 that Sierra Leone 
joins the United Nations.

The http_build_query() function is useful when you need to build an API URL that includes query string parameters. Give http_build_query() an associative array of parameter names and values and it gives you back a string of key=value pairs joined by & and properly encoded—exactly what you need for a URL.

The United States Department of Agriculture has a nifty API on top of its National Nutrient Database. This NDB API is free and easy to use.

define('NDB_API_KEY','273bqhebrfkhuebf');

Example 11-2 uses the NDB search API to find some information about black pepper. This API returns information about each food in the database whose name matches what’s in the q query string parameter.

$params = array('api_key' => NDB_API_KEY,
                'q' => 'black pepper',
                'format' => 'json');

$url = "http://api.nal.usda.gov/ndb/search?" . http_build_query($params);

The $url variable in Example 11-2 is set to something like the following (the actual value depends on your API key):

http://api.nal.usda.gov/ndb/search?
api_key=j724nbefuy72n4&q=black+pepper&format=json

Each key and value in the $params array has been put together with = and &, and special characters, such as the space in black pepper, have been encoded.

Passing such a URL to file_get_contents() makes the API call. In this case, the API returns JSON, so the return value of file_get_contents() is this string:

{
    "list": {
        "q": "black pepper",
        "sr": "27",
        "start": 0,
        "end": 1,
        "total": 1,
        "group": "",
        "sort": "r",
        "item": [
            {
                "offset": 0,
                "group": "Spices and Herbs",
                "name": "Spices, pepper, black",
                "ndbno": "02030"
            }
        ]
    }
}

Because file_get_contents() returns the response from retrieving the URL as a string, it’s a snap to pass that string to other functions that further transform it. For example, pass the API response above to json_decode() to transform the JSON into a PHP data structure you can manipulate. Example 11-3 prints out the NDB ID number for each matching food item.

$params = array('api_key' => NDB_API_KEY,
                'q' => 'black pepper',
                'format' => 'json');

$url = "http://api.nal.usda.gov/ndb/search?" . http_build_query($params);
$response = file_get_contents($url);
$info = json_decode($response);

foreach ($info->list->item as $item) {
    print "The ndbno for {$item->name} is {$item->ndbno}.\n";
}

The json_decode() function turns JSON objects into PHP objects and JSON arrays into PHP arrays. The top-level item in the response is an object. This is the return value from json_decode() and is assigned to $info. That object has a list property that is another object. The list object can be referred to as $info->list. That list object has an array property named item whose elements hold the details about the matching foods. So, the array that foreach() iterates over is $info->list->item. Each $item inside the foreach() loop is one object from that array. Example 11-3 prints:

The ndbno for Spices, pepper, black is 02030.

The NDB API calls made so far return JSON because of the format=json query string parameter. The API also supports specifying the response format by sending a Content-Type header.¹ A header value of application/json tells the server to format the response as JSON.

To add headers to your HTTP request with file_get_contents(), you create a stream context. The PHP engine’s underlying mechanisms for flowing data into and out of your programs are called streams. A stream can be a local file, a remote URL, or another exotic place that produces or consumes data. The first argument to file_get_contents() is the stream’s target: the file or URL to read from or write to. Additional information about the reading or writing operation is expressed through the stream context, created by passing an associative array of the additional information to the stream_context_create() function.

Different kinds of streams support different kinds of options for their contexts. For the http stream, a header option gets a string value containing the names and values of any headers to send with the HTTP request. Example 11-4 shows how to create a stream context that includes an HTTP header and use it with file_get_contents().

// Just key and query term, no format specified in query string
$params = array('api_key' => NDB_API_KEY,
                'q' => 'black pepper');
$url = "http://api.nal.usda.gov/ndb/search?" . http_build_query($params);

// Options are to set a Content-Type request header
$options = array('header' => 'Content-Type: application/json');
// Create a context for an 'http' stream
$context = stream_context_create(array('http' => $options));

// Pass the context as the third argument to file_get_contents
print file_get_contents($url, false, $context);

In Example 11-4, the $options array contains the key/value pairs of options to set. The stream_context_create() function needs to be told which kind of stream it’s creating a context for, so its argument is an array whose key is the stream type (http) and the value is the options to set.

The second argument of file_get_contents() indicates whether the function should pay attention to the PHP engine’s include path when looking for a file. This is irrelevant with HTTP, so false is supplied for a value. The context is the third argument to file_get_contents().

The context is also how you send a POST request with file_get_contents(). The method context option controls the request method, and the content context option contains any request body to send. The content must be formatted properly for the content type you specify as a header.

We can’t use the NDB API for POST, since it just tells us nutrition data via GET. It doesn’t allow us to send new data in. Example 11-5 uses file_get_contents() to send a POST request to an example URL. This request acts just like a form submission sending two form variables: name and smell.

$url = 'http://php7.example.com/post-server.php';

// Two variables to send via POST
$form_data = array('name' => 'black pepper',
                   'smell' => 'good');

// Set the method, content type, and content
$options = array('method' => 'POST',
                 'header' => 'Content-Type: application/x-www-form-urlencoded',
                 'content' => http_build_query($form_data));
// Create a context for an 'http' stream
$context = stream_context_create(array('http' => $options));

// Pass the context as the third argument to file_get_contents.
print file_get_contents($url, false, $context);

In Example 11-5, the method stream context option ensures that this is a POST request. The value you supply here is used verbatim by the PHP engine to make the request, so be sure to make it all capital letters. The value for the Content-Type header is the standard value that web browsers use for regular form data. It corresponds to data formatted like query string parameters but sent in the request body. Conveniently, this lets us use http_build_query() to construct the properly formatted request body.

If you need to send a different kind of data in your POST request, just change the value of the Content-Type header and how you format the request content. For example, to send JSON, change the header option to Content-Type: application/json and change the content option to json_encode($form_data).

More information about the PHP engine’s different stream types and other supported context options is available at http://www.php.net/context.

Although the simplicity of retrieving URLs with built-in file access functions is fantastic, these functions do not make life simple if there is an error when making the request. When that happens, file_get_contents() returns false and the PHP engine generates an error message that looks like failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found. Having more control over what to do when a request is not successful is one good reason to use the cURL functions instead.

Comprehensive URL Access with cURL

The file_get_contents() function, especially when combined with context options, lets you make a wide variety of HTTP requests. But when you really need control over the details of your HTTP requests and responses, turn to PHP’s cURL functions. By using a powerful underlying library, libcurl, these functions give you access to all aspects of your HTTP requests and responses.

<?php

$c = curl_init('http://numbersapi.com/09/27');
// Tell cURL to return the response contents as a string
// rather then printing them out immediately
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
// Execute the request
$fact = curl_exec($c);

?>
Did you know that <?= $fact ?>

// Just key and query term, no format specified in query string
$params = array('api_key' => NDB_API_KEY,
                'q' => 'black pepper');
$url = "http://api.nal.usda.gov/ndb/search?" . http_build_query($params);

$c = curl_init($url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
curl_setopt($c, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
print curl_exec($c);

// A pretend API endpoint that doesn't exist 
$c = curl_init('http://api.example.com');
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($c);
// Get all the connection info, whether or not it succeeded
$info = curl_getinfo($c);

// Something went wrong with the connection
if ($result === false) {
    print "Error #" . curl_errno($c) . "\n";
    print "Uh-oh! cURL says: " . curl_error($c) . "\n";
}
// HTTP response codes in the 400s and 500s mean errors
else if ($info['http_code'] >= 400) {
    print "The server says HTTP error {$info['http_code']}.\n";
}
else {
    print "A successful result!\n";
}
// The request info includes timing statistics as well
print "By the way, this request took {$info['total_time']} seconds.\n";

Error #6
Uh-oh! cURL says: Could not resolve host: api.example.com
By the way, this request took 0.000146 seconds.

The server says HTTP error 404.
By the way, this request took 0.00567 seconds.

$url = 'http://php7.example.com/post-server.php';

// Two variables to send via POST
$form_data = array('name' => 'black pepper',
                   'smell' => 'good');

$c = curl_init($url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
// This should be a POST request
curl_setopt($c, CURLOPT_POST, true);
// This is the data to send
curl_setopt($c, CURLOPT_POSTFIELDS, $form_data);

print curl_exec($c);

$url = 'http://php7.example.com/post-server.php';

// Two variables to send as JSON via POST
$form_data = array('name' => 'black pepper',
                   'smell' => 'good');

$c = curl_init($url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
// This should be a POST request
curl_setopt($c, CURLOPT_POST, true);
// This is a request containing JSON
curl_setopt($c, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
// This is the data to send, formatted appropriately
curl_setopt($c, CURLOPT_POSTFIELDS, json_encode($form_data));

print curl_exec($c);

// Use the value sent in the cookie, if any, or 0 if no cookie supplied
$value = $_COOKIE['c'] ?? 0;
// Increment the value by 1
$value++;
// Set the new cookie in the response
setcookie('c', $value);
// Tell the user what cookies we saw
print "Cookies: " . count($_COOKIE) . "\n";
foreach ($_COOKIE as $k => $v) {
    print "$k: $v\n";
}

// Retrieve the cookie server page, sending no cookies
$c = curl_init('http://php7.example.com/cookie-server.php');
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
// The first time, there are no cookies
$res = curl_exec($c);
print $res;

// The second time, there are still no cookies
$res = curl_exec($c);
print $res;

Cookies: 0
Cookies: 0

// Retrieve the cookie server page, sending no cookies
$c = curl_init('http://php7.example.com/cookie-server.php');
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
// Turn on the cookie jar
curl_setopt($c, CURLOPT_COOKIEJAR, true);

// The first time, there are no cookies
$res = curl_exec($c);
print $res;

// The second time, there are cookies from the first request
$res = curl_exec($c);
print $res;

Cookies: 0
Cookies: 1
c: 1

// Retrieve the cookie server page
$c = curl_init('http://php7.example.com/cookie-server.php');
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
// Save cookies to a 'saved.cookies' file in the same directory
// as this program
curl_setopt($c, CURLOPT_COOKIEJAR, __DIR__ . '/saved.cookies');
// Load cookies (if any have been previously saved) from the
// 'saved.cookies' file in this directory
curl_setopt($c, CURLOPT_COOKIEFILE, __DIR__ . '/saved.cookies');

// This request includes cookies from the file (if any)
$res = curl_exec($c);
print $res;

Cookies: 0

Cookies: 1
c: 1

Cookies: 1
c: 2

Serving API Requests

Your PHP program can serve API requests to clients as well. Instead of generating a plain HTML page, you generate whatever data is appropriate to the API call. Additionally, you may need to manipulate the HTTP response code and response headers you send.

To send an HTTP header along with your response, use the header() function. Example 11-15 is a tiny clock API in PHP. It serves up the current time in JSON format.

$response_data = array('now' => time());
header('Content-Type: application/json');
print json_encode($response_data);

The call to header() in Example 11-15 adds a header line to the HTTP response that the PHP engine generates. The header line is whatever you pass to the function. The code uses the json_encode() function to generate the JSON that makes up the response. This function is the opposite of json_decode(). You give it a PHP data type (string, number, object, array, etc.) and it returns a string containing the JSON representation of what was passed in. Example 11-15 generates an HTTP response that looks like this:

HTTP/1.1 200 OK
Host: www.example.com
Connection: close
Content-Type: application/json

{"now":1962258300}

The first four lines are the response headers. A few get added automatically by your web server. The Content-Type: application/json line is from the call to header(). After the blank line comes the request body. This is the stuff you see in your web browser (or, if it’s HTML, you see the results of your web browser rendering the HTML). In this case, the request body is a JSON object with one property, now, whose value is a timestamp corresponding to the current time. The value you get when you run this code will almost certainly be different because you are almost certainly running this code at a different time than the timestamp printed here. For more details on time handling in PHP, see Chapter 15.

The first line of the response headers contains the response code, which in this case is 200—which is HTTP-speak for “everything was fine.” To send a different response code, use the http_response_code() function. Example 11-16 is like Example 11-15 except it sends back a 403 status code (“Forbidden”) and an error response body unless a query string parameter named key is supplied with a value of pineapple.

if (! (isset($_GET['key']) && ($_GET['key'] == 'pineapple'))) {
    http_response_code(403);
    $response_data = array('error' => 'bad key');
}
else {
    $response_data = array('now' => time());
}
header('Content-Type: application/json');
print json_encode($response_data);

Without the proper query string parameter supplied, Example 11-16’s response looks like this:

HTTP/1.1 403 Forbidden
Host: www.example.com
Connection: close
Content-Type: application/json

{"error":"bad key"}

Example 11-4 used a Content-Type header to tell the NDB API to send back a response as JSON. To access incoming request headers from your PHP program, look in the $_SERVER array. Each incoming header is there. A header’s array key is HTTP_ followed by the header’s name in all uppercase and with any dashes (-) converted to underscores (_). For example, the value of an incoming Content-Type header would be in $_SERVER['HTTP_CONTENT_TYPE']. Example 11-17 examines the value of an incoming Accept header to determine how to format the output data.

<?php 

// Formats we want to support
$formats = array('application/json','text/html','text/plain');
// Response format if not specified
$default_format = 'application/json';

// Was a response format supplied?
if (isset($_SERVER['HTTP_ACCEPT'])) {
    // If a supported format is supplied, use it
    if (in_array($_SERVER['HTTP_ACCEPT'], $formats)) {
        $format = $_SERVER['HTTP_ACCEPT'];
    }
    // An unsupported format was supplied, so return an error
    else {
        // 406 means "You want a response in a format I can't generate"
        http_response_code(406);
        // Exiting now means no response body, which is OK
        exit();
    }
} else {
    $format = $default_format;
}

// Figure out what time it is
$response_data = array('now' => time());
// Tell the client what kind of content we're sending
header("Content-Type: $format");
// Print the time in a format-appropriate way
if ($format == 'application/json') {
    print json_encode($response_data);
}
else if ($format == 'text/html') { ?>
<!doctype html>
  <html>
    <head><title>Clock</title></head>
    <body><time><?= date('c', $response_data['now']) ?></time></body>
  </html>
<?php 
} else if ($format == 'text/plain') {
    print $response_data['now'];
}

If the incoming Accept header is application/json, text/html, or text/plain, then $format is set to the proper format to use. This value gets put in the Content-Type header of the response and is used to generate a format-appropriate output. If no Accept header is supplied, a default of application/json is used. If any other value is supplied in the Accept header, then the program returns an empty body with a 406 error code. This tells the client that an invalid format was asked for.²

The $_SERVER array is also where you look to determine if the current request is a secure request; that is, if it was made with HTTPS. If the current request is secure, then $_SERVER['HTTPS'] is set to on. Example 11-18 checks if the current request was made over HTTPS and redirects to an HTTPS version of the current request’s URL if not.

$is_https = (isset($_SERVER['HTTPS']) && ($_SERVER['HTTPS'] == 'on'));
if (! $is_https) {
    $newUrl = 'https://' . $_SERVER['HTTP_HOST'] . $_SERVER['REQUEST_URI'];
    header("Location: $newUrl");
    exit();
}
print "You accessed this page over HTTPS. Yay!";

In Example 11-18, the first line determines whether the current request was over HTTPS by ensuring two things: first, that there is a value set for $_SERVER['HTTPS'] and second, that the value is on. If both of those things are not true, then an HTTPS version of the current URL is built by combining the right protocol (https://) with the current request’s hostname (the value of $_SERVER['HTTP_HOST']) and the current request’s path (the value of $_SERVER['REQUEST_URI']). If the request included any query string parameters, those are included in $_SERVER['REQUEST_URI'] as well. The Location header, sent by the header() function, redirects a web client to a new URL.

Chapter Summary

This chapter covered:

• Retrieving URLs with file_get_contents()
• Retrieving a URL that includes query string parameters
• Decoding a JSON HTTP response
• Understanding PHP’s stream contexts
• Including additional headers when retrieving a URL
• Retrieving URLs via the POST method with file_get_contents()
• Retrieving URLs with cURL
• Using query string parameters with cURL
• Adding request headers with cURL
• Handling errors from cURL requests
• Retrieving URLs via the POST method with cURL
• Keeping track of HTTP cookies with cURL
• Using cURL securely with HTTPS
• Serving non-HTML responses
• Changing the HTTP response code.
• Using values from HTTP request headers
• Checking if a request is made with HTTPS

Exercises

1. http://php.net/releases/?json is a JSON feed of the latest PHP releases. Write a program that uses file_get_contents() to retrieve this feed and print out the latest version of PHP released.
2. Modify your program from the previous exercise to use cURL instead of file_get_contents().
3. Write a web page that uses a cookie to tell the user when he last looked at the web page (you may find the date-and-time-handling functions described in Chapter 15 useful).
4. A GitHub gist is a snippet of text or code that is easy to share. The GitHub API allows you to create gists without logging in. Write a program that creates a gist whose contents are the program you’re writing to create a gist. Note that the GitHub API requires you to set a User-Agent header in your HTTP API requests. The CURLOPT_USERAGENT setting can be used to set this header.