Table of Contents for
Hands-On Machine Learning for Cybersecurity

Version ebook / Retour

Cover image for bash Cookbook, 2nd Edition Hands-On Machine Learning for Cybersecurity by Sinan Ozdemir Published by Packt Publishing, 2018
  1. Hands-on Machine Learning for Cybersecurity
  2. Title Page
  3. Copyright and Credits
  4. Hands-On Machine Learning for Cybersecurity
  5. About Packt
  6. Why subscribe?
  7. Packt.com
  8. Contributors
  9. About the authors
  10. About the reviewers
  11. Packt is searching for authors like you
  12. Table of Contents
  13. Preface
  14. Who this book is for
  15. What this book covers
  16. To get the most out of this book
  17. Download the example code files
  18. Download the color images
  19. Conventions used
  20. Get in touch
  21. Reviews
  22. Basics of Machine Learning in Cybersecurity
  23. What is machine learning?
  24. Problems that machine learning solves
  25. Why use machine learning in cybersecurity?
  26. Current cybersecurity solutions
  27. Data in machine learning
  28. Structured versus unstructured data
  29. Labelled versus unlabelled data
  30. Machine learning phases
  31. Inconsistencies in data
  32. Overfitting
  33. Underfitting
  34. Different types of machine learning algorithm
  35. Supervised learning algorithms
  36. Unsupervised learning algorithms 
  37. Reinforcement learning
  38. Another categorization of machine learning
  39. Classification problems
  40. Clustering problems
  41. Regression problems
  42. Dimensionality reduction problems
  43. Density estimation problems
  44. Deep learning
  45. Algorithms in machine learning
  46. Support vector machines
  47. Bayesian networks
  48. Decision trees
  49. Random forests
  50. Hierarchical algorithms
  51. Genetic algorithms
  52. Similarity algorithms
  53. ANNs
  54. The machine learning architecture
  55. Data ingestion
  56. Data store
  57. The model engine
  58. Data preparation 
  59. Feature generation
  60. Training
  61. Testing
  62. Performance tuning
  63. Mean squared error
  64. Mean absolute error
  65. Precision, recall, and accuracy
  66. How can model performance be improved?
  67. Fetching the data to improve performance
  68. Switching machine learning algorithms
  69. Ensemble learning to improve performance
  70. Hands-on machine learning
  71. Python for machine learning
  72. Comparing Python 2.x with 3.x 
  73. Python installation 
  74. Python interactive development environment
  75. Jupyter Notebook installation
  76. Python packages
  77. NumPy
  78. SciPy
  79. Scikit-learn 
  80. pandas
  81. Matplotlib
  82. Mongodb with Python
  83. Installing MongoDB
  84. PyMongo
  85. Setting up the development and testing environment
  86. Use case
  87. Data
  88. Code
  89. Summary
  90. Time Series Analysis and Ensemble Modeling
  91. What is a time series?
  92. Time series analysis
  93. Stationarity of a time series models
  94. Strictly stationary process
  95. Correlation in time series
  96. Autocorrelation
  97. Partial autocorrelation function
  98. Classes of time series models
  99. Stochastic time series model
  100. Artificial neural network time series model
  101.  Support vector time series models
  102. Time series components
  103. Systematic models
  104. Non-systematic models
  105. Time series decomposition
  106. Level 
  107. Trend 
  108. Seasonality 
  109. Noise 
  110. Use cases for time series
  111. Signal processing
  112. Stock market predictions
  113. Weather forecasting
  114. Reconnaissance detection
  115. Time series analysis in cybersecurity
  116. Time series trends and seasonal spikes
  117. Detecting distributed denial of series with time series
  118. Dealing with the time element in time series
  119. Tackling the use case
  120. Importing packages
  121. Importing data in pandas
  122. Data cleansing and transformation
  123. Feature computation
  124. Predicting DDoS attacks
  125. ARMA
  126. ARIMA
  127. ARFIMA
  128. Ensemble learning methods
  129. Types of ensembling
  130. Averaging
  131. Majority vote
  132. Weighted average
  133. Types of ensemble algorithm
  134. Bagging
  135. Boosting
  136. Stacking
  137. Bayesian parameter averaging
  138. Bayesian model combination
  139. Bucket of models
  140. Cybersecurity with ensemble techniques
  141. Voting ensemble method to detect cyber attacks
  142. Summary
  143. Segregating Legitimate and Lousy URLs
  144. Introduction to the types of abnormalities in URLs
  145. URL blacklisting
  146. Drive-by download URLs
  147. Command and control URLs
  148. Phishing URLs
  149. Using heuristics to detect malicious pages
  150. Data for the analysis
  151. Feature extraction
  152. Lexical features
  153. Web-content-based features
  154. Host-based features
  155. Site-popularity features
  156. Using machine learning to detect malicious URLs 
  157. Logistic regression to detect malicious URLs
  158. Dataset
  159. Model
  160. TF-IDF
  161. SVM to detect malicious URLs
  162. Multiclass classification for URL classification
  163. One-versus-rest
  164. Summary
  165. Knocking Down CAPTCHAs
  166. Characteristics of CAPTCHA
  167. Using artificial intelligence to crack CAPTCHA
  168. Types of CAPTCHA
  169. reCAPTCHA
  170. No CAPTCHA reCAPTCHA
  171. Breaking a CAPTCHA
  172. Solving CAPTCHAs with a neural network
  173. Dataset 
  174. Packages
  175. Theory of CNN
  176. Model
  177. Code
  178. Training the model
  179. Testing the model 
  180. Summary
  181. Using Data Science to Catch Email Fraud and Spam
  182. Email spoofing 
  183. Bogus offers
  184. Requests for help
  185. Types of spam emails
  186. Deceptive emails
  187. CEO fraud
  188. Pharming 
  189. Dropbox phishing
  190. Google Docs phishing
  191. Spam detection
  192. Types of mail servers 
  193. Data collection from mail servers
  194. Using the Naive Bayes theorem to detect spam
  195. Laplace smoothing
  196. Featurization techniques that convert text-based emails into numeric values
  197. Log-space
  198. TF-IDF
  199. N-grams
  200. Tokenization
  201. Logistic regression spam filters
  202. Logistic regression
  203. Dataset
  204. Python
  205. Results
  206. Summary
  207. Efficient Network Anomaly Detection Using k-means
  208. Stages of a network attack
  209. Phase 1 – Reconnaissance 
  210. Phase 2 – Initial compromise 
  211. Phase 3 – Command and control 
  212. Phase 4 – Lateral movement
  213. Phase 5 – Target attainment 
  214. Phase 6 – Ex-filtration, corruption, and disruption 
  215. Dealing with lateral movement in networks
  216. Using Windows event logs to detect network anomalies
  217. Logon/Logoff events 
  218. Account logon events
  219. Object access events
  220. Account management events
  221. Active directory events
  222. Ingesting active directory data
  223. Data parsing
  224. Modeling
  225. Detecting anomalies in a network with k-means
  226. Network intrusion data
  227. Coding the network intrusion attack
  228. Model evaluation 
  229. Sum of squared errors
  230. Choosing k for k-means
  231. Normalizing features
  232. Manual verification
  233. Summary
  234. Decision Tree and Context-Based Malicious Event Detection
  235. Adware
  236. Bots
  237. Bugs
  238. Ransomware
  239. Rootkit
  240. Spyware
  241. Trojan horses
  242. Viruses
  243. Worms
  244. Malicious data injection within databases
  245. Malicious injections in wireless sensors
  246. Use case
  247. The dataset
  248. Importing packages 
  249. Features of the data
  250. Model
  251. Decision tree 
  252. Types of decision trees
  253. Categorical variable decision tree
  254. Continuous variable decision tree
  255. Gini coeffiecient
  256. Random forest
  257. Anomaly detection
  258. Isolation forest
  259. Supervised and outlier detection with Knowledge Discovery Databases (KDD)
  260. Revisiting malicious URL detection with decision trees
  261. Summary
  262. Catching Impersonators and Hackers Red Handed
  263. Understanding impersonation
  264. Different types of impersonation fraud 
  265. Impersonators gathering information
  266. How an impersonation attack is constructed
  267. Using data science to detect domains that are impersonations
  268. Levenshtein distance
  269. Finding domain similarity between malicious URLs
  270. Authorship attribution
  271. AA detection for tweets
  272. Difference between test and validation datasets
  273. Sklearn pipeline
  274. Naive Bayes classifier for multinomial models
  275. Identifying impersonation as a means of intrusion detection 
  276. Summary
  277. Changing the Game with TensorFlow
  278. Introduction to TensorFlow
  279. Installation of TensorFlow
  280. TensorFlow for Windows users
  281. Hello world in TensorFlow
  282. Importing the MNIST dataset
  283. Computation graphs
  284. What is a computation graph?
  285. Tensor processing unit
  286. Using TensorFlow for intrusion detection
  287. Summary
  288. Financial Fraud and How Deep Learning Can Mitigate It
  289. Machine learning to detect financial fraud
  290. Imbalanced data
  291. Handling imbalanced datasets
  292. Random under-sampling
  293. Random oversampling
  294. Cluster-based oversampling
  295. Synthetic minority oversampling technique
  296. Modified synthetic minority oversampling technique
  297. Detecting credit card fraud
  298. Logistic regression
  299. Loading the dataset
  300. Approach
  301. Logistic regression classifier – under-sampled data
  302. Tuning hyperparameters 
  303. Detailed classification reports
  304. Predictions on test sets and plotting a confusion matrix
  305. Logistic regression classifier – skewed data
  306. Investigating precision-recall curve and area
  307. Deep learning time
  308. Adam gradient optimizer
  309. Summary
  310. Case Studies
  311. Introduction to our password dataset
  312. Text feature extraction
  313. Feature extraction with scikit-learn
  314. Using the cosine similarity to quantify bad passwords
  315. Putting it all together
  316. Summary
  317. Other Books You May Enjoy
  318. Leave a review - let other readers know what you think

Host-based features

Hosting services are internet services that allow users to make a website available to the World Wide Web. Oftentimes, a single server is rented out to multiple websites. Using these services, we will be able to find out the IP addresses of each of the URLs that we inspect. We will also look up the autonomous system number (ASN). These numbers are a collection of IP routing prefixes that are controlled by central network operators. We use the ASN information to look for sites that have already been filed under the bad ASN category (colored in red), as shown in the following diagram:

The following code helps in identifying the ASN. This number is unique, and identifies an autonomous system that exchanges routing details:

def getASN(host_info):
    try:
        g = pygeoip.GeoIP('GeoIPASNum.dat')
        asn=int(g.org_by_name(host_info).split()[0][2:])
        return asn
    except:
         return nf