Table of Contents for
Python for Secret Agents

Version ebook / Retour

Cover image for bash Cookbook, 2nd Edition Python for Secret Agents by Steven F. Lott Published by Packt Publishing, 2014
  1. Cover
  2. Table of Contents
  3. Python for Secret Agents
  4. Python for Secret Agents
  5. Credits
  6. About the Author
  7. About the Reviewers
  8. www.PacktPub.com
  9. Preface
  10. What you need for this book
  11. Who this book is for
  12. Conventions
  13. Reader feedback
  14. Customer support
  15. 1. Our Espionage Toolkit
  16. Getting more tools – a text editor
  17. Confirming our tools
  18. Background briefing – math and numbers
  19. Handling text and strings
  20. Organizing our software
  21. Working with files and folders
  22. Solving problems – recovering a lost password
  23. Summary
  24. 2. Acquiring Intelligence Data
  25. Using a REST API in Python
  26. Organizing collections of data
  27. Solving problems – currency conversion rates
  28. Summary
  29. 3. Encoding Secret Messages with Steganography
  30. Using the Pillow library
  31. Some approaches to steganography
  32. Detecting and preventing tampering
  33. Solving problems – encrypting a message
  34. Summary
  35. 4. Drops, Hideouts, Meetups, and Lairs
  36. Finding out where we are with geocoding services
  37. How close? What direction?
  38. Compressing data to make grid codes
  39. Decoding a GeoRef code
  40. Creating natural area codes
  41. Solving problems – closest good restaurant
  42. Summary
  43. 5. A Spymaster's More Sensitive Analyses
  44. Creating Python modules and applications
  45. Creating our own classes of objects
  46. Comparisons and correlations
  47. Writing high-quality software
  48. Solving problems – analyzing some interesting datasets
  49. Summary
  50. Index

Solving problems – analyzing some interesting datasets

What is the correlation coefficient between per-capita cheese consumption and death code W75 (accidental suffocation and strangulation in bed) for the years 2000 to 2009?

Recall that the cheese data came from http://www.ers.usda.gov/datafiles/Dairy_Data/chezcon_1_.xls.

This is a pesky bit of data because it's in a proprietary spreadsheet format. As much as we dislike copying and pasting, there's no other easy way to get this data.

The strangulation in bed data comes from cause of death W75 grouped by year. The data request process starts at http://wonder.cdc.gov/controller/datarequest/D76. There will be some further espionage work required to enter a request for data. For some additional help, check http://wonder.cdc.gov/wonder/help/ucd.html.

How the correlation possibly be so high?

What is it about cheese consumption and death by strangulation in bed that leads to this amazing correlation?

Getting some more data

HQ appears to be working on a theory about dairy products. The cheese spreadsheet also has mozzarella cheese consumption over the same period of time—2000 to 2009.

We've been ordered to get details on civil engineering doctorates awarded during this period.

Some initial espionage turned up this set of data:

http://www.nsf.gov/statistics/infbrief/nsf12303/

This is a tricky table to parse. It's a bit more complex because the years are in columns and the data we're looking for is in a particular row, a row with th.text == "Civil engineering". The heading for the table as a whole is in a row with th.text == "Field". This means that the navigation will be rather complex to locate the Field row and the Civil engineering rows of the proper table on this page.

How does annual per-capita mozzarella cheese consumption correlate with civil engineering doctorates?

How can the correlation possibly be so high?

What is it about cheese, death, and doctorates?

Further research

Is this just a spurious correlation?

Are there other correlations like this?

What else can we learn from http://www.tylervigen.com/?