Previous Chapter

Python for Secret Agents

Table of Contents

Python for Secret Agents

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Our Espionage Toolkit

Getting the tools of the trade – Python 3.3

Windows secrets

Mac OS X secrets

Getting more tools – a text editor

Getting other developer tools

Getting a tool to get more Python components

Confirming our tools

How do we stop?

Using the help() system

Mac OS and GNU/Linux secrets

Windows secrets

Using the help mode

Background briefing – math and numbers

The usual culprits

The ivory tower of numbers

Integer numbers

Rational numbers

Floating-point numbers

Decimal numbers

Complex numbers

Outside the numbers

Assigning values to variables

Writing scripts and seeing output

Gathering user input

Handling exceptions

Looping and trying again

Handling text and strings

Converting between numbers and strings

Parsing strings

Organizing our software

Working with files and folders

Creating a file

Reading a file

Defining more complex logical conditions

Solving problems – recovering a lost password

Reading a word corpus

Reading a ZIP archive

Using brute-force search

Summary

2. Acquiring Intelligence Data

Accessing data from the Internet

Background briefing – the TCP/IP protocols

Using http.client for HTTP GET

Changing our client information

Using FTP in Python

Downloading a file via FTP

Using our FTP get() function

Using urllib for HTTP, FTP, or file access

Using urllib for FTP access

Using a REST API in Python

Getting simple REST data

Using more complex RESTful queries

Saving our data via JSON

Organizing collections of data

Using a Python list

Using list index operations

Using a Python tuple

Using generator expressions with list of tuples

Using a Python dictionary mapping

Using the dictionary access methods

Transforming sequences with generator functions

Using the defaultdict and counter mappings

Using a Python set

Using the for statement with a collection

Using Python operators on collections

Solving problems – currency conversion rates

Summary

3. Encoding Secret Messages with Steganography

Background briefing – handling file formats

Working with the OS filesystem

glob

os

Processing simple text files

Working with ZIP files

Working with JSON files

Working with CSV files

JPEG and PNG graphics – pixels and metadata

Using the Pillow library

Adding the required supporting libraries

GNU/Linux secrets

Mac OS X secrets

Windows secrets

Installing and confirming Pillow

Decoding and encoding image data

Manipulating images – resizing and thumbnails

Manipulating images – cropping

Manipulating images – enhancing

Manipulating images – filtering

Manipulating images – ImageOps

Some approaches to steganography

Getting the red-channel data

Extracting bytes from Unicode characters

Manipulating bits and bytes

Assembling the bits

Encoding the message

Decoding a message

Detecting and preventing tampering

Using hash totals to validate a file

Using a key with a digest

Solving problems – encrypting a message

Unpacking a message

Summary

4. Drops, Hideouts, Meetups, and Lairs

Background briefing – latitude, longitude, and GPS

Coping with GPS device limitations

Handling politics – borders, precincts, jurisdictions, and neighborhoods

Finding out where we are with geocoding services

Geocoding an address

Reverse geocoding a latitude-longitude point

How close? What direction?

Combining geocoding and haversine

Compressing data to make grid codes

Creating GeoRef codes

Decoding a GeoRef code

Creating Maidenhead grid codes

Decoding the Maidenhead grid codes

Creating natural area codes

Decoding natural area codes

Solving problems – closest good restaurant

Creating simple Python objects

Working with HTML web services – tools

Working with HTML web services – getting the page

Working with HTML web services – parsing a table

Making a simple Python object from columns of data

Enriching Python objects with geocodes

Enriching Python objects with health scores

Combining the pieces and parts

Working with clean data portals

Making a simple Python object from a JSON document

Combining different pieces and parts

Final steps

Understanding the data – schema and metadata

Summary

5. A Spymaster's More Sensitive Analyses

Creating statistical summaries

Parsing the raw data file

Finding an average value

Understanding generator expressions

Finding the value in the middle

Finding the most popular value

Creating Python modules and applications

Creating and using a module

Creating an application module

Creating a hybrid module

Creating our own classes of objects

Using a class definition

Comparisons and correlations

Computing the standard deviation

Computing a standardized score

Comparing a sequence and an iterable

Computing a coefficient of correlation

Writing high-quality software

Building a self-testing module and a test module

Creating more sophisticated tests

Adding doctest cases to a class definition

Solving problems – analyzing some interesting datasets

Getting some more data

Further research

Summary

Index

Previous Chapter

Python for Secret Agents