This is a simple model that stocks in historical data features (the ones listed in the Data parsing section) that are associated with Windows logs. When a new feature parameter comes in, we see whether this is a new one by comparing to the historical data. Historical data could include AD logs with res to the features from over a year ago. The AD event that we will use for this purpose is 4672.

For the purposes of a use case, we will only choose the privilege feature. A list of privileges could be as follows:

• SeSecurityPrivilege
• SeTakeOwnershipPrivilege
• SeLoadDriverPrivilege
• SeBackupPrivilege
• SeRestorePrivilege
• SeDebugPrivilege
• SeSystemEnvironmentPrivilege
• SeImpersonatePrivilege

We store in the historical database all privileges that the user account had in the past year, such as the write privilege and the read privilege. When a new privilege is seen to be invoked by the user account, we raise an anomaly alarm. To score the severity of the anomaly, we check how many other people have access to the newly observed privilege. A rarity score is associated with it accordingly. The total number of privileges used by the user account in the entire day is also counted and the final score is the function of the total and the rarity score:

import sys
import os
sys.path.append('.')
sys.path.insert(0, os.getcwd())
sys.path.append('/usr/lib/python2.6/site-packages')

import math

#history of privileges used

input_path_of_file_hist = "/datasets/historical.data"
data_raw_hist = sc.textFile(input_path_of_file_hist, 12)

#for each privilge a rarity map is present

rarity_map = {}
input_path_of_file_rare = "/datasets/rare.data"
data_raw_rare = sc.textFile(input_path_of_file_rare, 12)
arr = data_raw_rare.split(',')
privilege = arr[0] 
rarityscore = arr[1]
rarity_map[privilege] = rarityscore


priv_hist = {}
FOREACH line in data_raw_hist :
 if line in priv_hist:
 do_nothing = 1
 else:
 priv_hist[line] = 1


input_path_of_file_curr = "/datasets/current.data"
data_raw_curr = sc.textFile(input_path_of_file_curr, 12)

num_lines = sum(1 for line in open(input_path_of_file_curr))

FOREACH line in data_raw_curr :
 if line in priv_hist
 print "i dont care this is privilege is old"
 else:
 print "new activity detected"
 C = computeScore()
 score = C.compute(line,num_lines)

For every new activity detected in the network, we compute the score on a scale of 1 to 10 to measure the degree of malicious behavior. There are two classes that would be doing this. The first script calls the score computation script to generate the final score:

class computeScore:
    def __init__(self,userandkey,rarity):
        self.userandkey = userandkey
        self.anomaly_score = 0


  def compute(line,num_lines)
  total=num_lines
    itemrarity = rarity_map[line]
  T = NoxScoring()
  anomaly_score = T.threat_anomaly_score(int(itemrarity),int(total))
  return anomaly_score

This script is used to generate the relevant score when a newly observed privilege is seen:

class NoxScoring():
    def __init__(self):
        self.item_raririty_table = []
  self.item_raririty_table.append([.8,1,0.1])
        self.item_raririty_table.append([.7,.8,0.2])
        self.item_raririty_table.append([.6,.7,0.3])
        self.item_raririty_table.append([.5,.6, 0.4])
        self.item_raririty_table.append([.4,.5, 0.5])
        self.item_raririty_table.append([.3, .4, 0.6])
        self.item_raririty_table.append([.2, .3, 0.7])
        self.item_raririty_table.append([.1, .2, 0.8])
        self.item_raririty_table.append([.001, .1, 0.9])
        self.item_raririty_table.append([0, .001, 1]) 

    def threat_anomaly_score(self,rarityscore,totalusers):
  if rarityscore is None :
          age = .9
  else :
    age = float(rarityscore) / float(totalusers)

        for row in self.item_raririty_table:
            if (age>=row[0]) and (age<row[1]):
                score = row[2]
                return score
        return score

    def combine_threat_score(self,score,correlationscore):
        combined_score = score * 1
        return combined_score

#if __name__=='__main__':
# T = NoxScoring()
# print T.threat_anomaly_score(43,473)

This simplistic model can be easily used to detect newly observed document (object) access, newly seen servers in the system, or even newly added users.