For this recipe, you will need a list of URLs that you want to test for insecure headers. Save these into a text file called urls.txt, with each URL on a new line, alongside your recipe.

The following code will highlight any vulnerable headers received in the HTTP response from each of the target URLs:

import requests

urls = open("urls.txt", "r")
for url in urls:
  url = url.strip()
  req = requests.get(url)
  print url, 'report:'

  try:
    xssprotect = req.headers['X-XSS-Protection']
    if  xssprotect != '1; mode=block':
      print 'X-XSS-Protection not set properly, XSS may be possible:', xssprotect
  except:
    print 'X-XSS-Protection not set, XSS may be possible'

  try:
    contenttype = req.headers['X-Content-Type-Options']
    if contenttype != 'nosniff':
      print 'X-Content-Type-Options not set properly:',  contenttype
  except:
    print 'X-Content-Type-Options not set'

  try:
    hsts = req.headers['Strict-Transport-Security']
  except:
    print 'HSTS header not set, MITM attacks may be possible'

  try:
    csp = req.headers['Content-Security-Policy']
    print 'Content-Security-Policy set:', csp
  except:
    print 'Content-Security-Policy missing'

  print '----'

This recipe is configured for testing many sites, so the first part reads in the URLs from the text file and prints out the current target:

urls = open("urls.txt", "r")
for url in urls:
  url = url.strip()
  req = requests.get(url)
  print url, 'report:'

Each header is then tested inside a try/except block. This is similar to the previous recipe in which this coding style is needed because the headers are not mandatory. If we attempted to reference a key for a header that doesn't exist, Python would raise an exception.

The first X-XSS-Protection header should be set to 1; mode=block to enable XSS protection in the browser. The script prints out a warning if the header does not explicitly match that format or if it's not set:

try:
    xssprotect = req.headers['X-XSS-Protection']
    if  'xssprotect' != '1; mode=block':
      print 'X-XSS-Protection not set properly, XSS may be possible'
  except:
    print 'X-XSS-Protection not set, XSS may be possible'

The next X-Content-Type-Options header should be set to nosniff to prevent MIME type confusion. A MIME type specifies the content of the target resource, for example, text/plain means the remote resource should be a text file. Some web browsers attempt to guess the MIME type of a resource if it's not specified. This can lead to Cross-site scripting attacks; if a resource contains a malicious script, but it only indicates to be a plain text file, it may bypass content filters and be executed. This check will print a warning if the header is not set or if the response does not explicitly match to nosniff:

try:
    contenttype = req.headers['X-Content-Type-Options']
    if contenttype != 'nosniff':
      print 'X-Content-Type-Options not set properly'
  except:
    print 'X-Content-Type-Options not set'

The next Strict-Transport-Security header is used to force communication over a HTTPS channel, to prevent man in the middle (MITM) attacks. The lack of this header means that the communication channel could be downgraded to HTTP by an MITM attack:

  try:
    hsts = req.headers['Strict-Transport-Security']
  except:
    print 'HSTS header not set, MITM attacks may be possible'

The final Content-Security-Policy header is used to restrict the type of resources that can load on the web page, for example, restricting where JavaScript can run:

  try:
    csp = req.headers['Content-Security-Policy']
    print 'Content-Security-Policy set:', csp
  except:
    print 'Content-Security-Policy missing'

The output from the recipe is shown in the following screenshot: