Chapter 1. Foundation: Understanding the Basics
Chapter 2. Open Source Intelligence and Advanced Social Media Search
Advanced search techniques for some specific social media
Chapter 3. Understanding Browsers and Beyond
Some of the well-known custom browsers
Chapter 4. Search the Web—Beyond Convention
Chapter 5. Advanced Web Searching
Chapter 6. OSINT Tools and Techniques
Why do we need to be anonymous
Chapter 9. Deepweb: Exploring the Darkest Corners of the Internet
Chapter 10. Data Management and Visualization
Data management and analysis tools
Update operating system and other applications
Precautions against social engineering
Chapter 12. Basics of Social Networks Analysis
Chapter 13. Quick and Dirty Python
Chapter 14. Case Studies and Examples
In this chapter we will be setting up the basics clear and understand the working of the internet. After that we will talk about the World Wide Web. Then we will move ahead and learn about some basic technologies of the computer world, which will help us later to get a clear picture of what we are dealing with. Later in this chapter we will be describing about the basic environment setup, so that the examples and exercises discussed in the book could be easily replicated by the readers.
After understanding the basics of the internet in the first chapter, in this chapter we will be learning about the open source intelligence. We will see it from an average user's perspective and will talk about the basic ways in which we access it. Then we will move forward and understand the evolution of the web as we know it today, i.e., Web 2.0 and how it influences us. Afterward we will discuss about Social Media Intelligence. Then we will be dealing with social networks and talk about advanced social media search. In the end we will briefly talk about the web of the future, also termed as Web 3.0
In the last chapter we discussed about open source intelligence and learned about advanced search mechanisms of various social networking sites, now we will be moving on and we will be specifically dealing with browsers in this chapter. First we will talk about some basic information about browsers and their brief history. After that we will move on to the architecture of browsers. Then we will discuss about some of the common features of browsers such as autocomplete, bookmarks, proxy, etc. Further we will learn about possibility and need for custom browsers. Once we are into custom browsers, we will talk about some of the available custom browsers and their functionalities. Later in this chapter we will discuss the term addons and will also learn about some of the interesting addons. Toward the end of this chapter, bookmarks will be discussed and in the end we will briefly talk about the threat that browsers pose toward the users.
We learned about browsers and their advanced usage in last chapter, now we will be moving on to the search engines, but not the conventional ones. In this chapter we will discuss about various different kinds of search engines which are domain specific and allow us to find information which regular search engines don’t. We will discuss about their working, understand their usage, and see how they are different. There will be various categories under which we will discuss the best search engines that belong there. These platforms that we will discuss are not a replacement to the conventional ones but simply complement them.
In last few chapters we learned about searching various social media platforms, unconventional search engines, and better usage of our browsers. In this chapter we will learn how to use conventional search engines to perform better search using various different advanced search operators and techniques. We will check out advanced features of some of the most popular search engines and see how we use them to their full potential so that we can achieve precise results.
Till now we have learned a lot related to searching the web in various different ways just by using the browser. In this chapter we will learn about some applications and web-based services, which automate the process of data extraction. We will learn about different ways to extract data simply by clicking on a graphical interface to executing commands in a command-line interface (CLI). The tools' interface and their usage will be demonstrated in detail so that the users can get a strong hold of and can easily explore them further.
Lately we have learned a lot about the conventional and unconventional ways of extracting information from the internet. In this chapter we will move a step further and understand about metadata, the hidden treasure. It is mostly ignored due to the unawareness or negligence, but sometimes it can play a game changing role during cyber exercises. We will discuss about the basic definition of the term, where it is usually found, how to extract it, and also some defensive measures.
Previously we have covered a lot about finding information related to people, organizations, IT infrastructure etc. But what about our own identity? We use internet on a daily basis for professional as well as personal use, and during this usage we reveal our identity in many ways. In this chapter we will learn how to stay anonymous online using different tools and techniques. Though promising 100% anonymity is very difficult, still using the mentioned methods will help to achieve it up to a certain level.
After learning about being anonymous on the web, in this chapter we will be moving on to explore some of the deepest and darkest corners of the internet. We will learn about the places which are hidden from a casual internet user. Methods to access as well as creating this secret space in the internet will be discussed. We will find out some of the most common and frequently visited parts of this area and learn what can be expected there.
Till now we have learned to extract data from various sources and also about anonymity. In this chapter we will discuss about data management and visualization. We will understand how data transforms into intelligence and what are the tools which can assist us in that process. Using the tools to manage and visualize data we will be able to see how they can play a key role in the process of informed decision-making.
We have covered many topics related to how to use the internet in a better way for data extraction. In the last chapter we also discussed about how to manage and visualize data for analysis purpose. Here we will be discussing about one of the most relevant topics related to the virtual world i.e., online security. We will learn what are the common threats that haunt us in the virtual space, how to identify them, and most importantly what steps can be taken to minimize the risk of becoming a victim of such attacks.
The information that we have learned to collect till now is in many forms. When it comes to basic analysis, these can be divided into entities and the relationship between the entities. Social network analysis (SNA) is basically about understanding data through network graphs in the form of entities and relationships and analyzing it to make generate a conclusion out of it. In this chapter we will cover some basic concepts of SNA to introduce the readers to this interesting subject so that they can get familiar with it and further explore it depending upon personal requirements.
It need not to be proved that automated tools are very helpful in performing various tasks. We have already discussed about a lot of such tools in previous chapters. Here we will be taking a step further to learn the basics of Python language to create our own custom tools for custom requirements. We will start with the learning about the essential components and then move on to create our own scripts, as committed in a previous chapter we will also use it to create transforms for our beloved Maltego.
print “Hello World”
python helloworld.py
chmod 755 helloworld.py
./helloworld.py
#!/usr/bin/python
>>>samplenum=10
>>>samplestr=”This is a string”
>>>samplestr2=’This is another string’
>>>sample_str=”There are total %d number of floors in the %s building”%(4,’xyz’)
>>>sample_str
>>>samplelist=[123, “str”, ‘xyz’, 321, 21.22]
>>>samplelist
[123, “str”, ‘xyz’, 321, 21.22]
>>>samplelist[1]
‘str’
>>>sampledict={‘test1’:’123’,’test2’:’234’,’test3’:’345’}
>>>sampledict[‘test1’]
‘123’
>>>sampledict[‘test4’]=’456’
>>>sampledict[‘test3’]=’333’
>>>sampledict
{‘test1’: ’123’, ’test2’: ’234’, ’test3’: ’333’, ’test4’: ’456’}
>>>dir(sampledict)
>>>help(sampledict)
>>>a=12
>>>b=2
>>>a∗b
24
>>>a=”test”
>>>b=”next”
>>>a+b
‘test next’
>>>lt1=[‘1’,’2’,’3’]
>>>lt2=[‘4’,’5’,’6’]
>>>lt1+lt2
[‘1’, ‘2’, ‘3’, ‘4’, ‘5’, ‘6’]
>>>a=1
>>>b=2
>>>a+b
3
>>>a=”test”
>>>b=”string”
>>>a+b
‘teststring’
>>>a.upper()
‘TEST’
>>>c=”This is a string”
>>>c.find(‘ring’)
12
>>>c.find(‘xyz’)
-1
>>>sample_list=[‘qw’,’er’,’ty’,123]
>>>sample_list.append(456)
>>>sample_list
[‘qw’, ’er’, ’ty’, 123, 456]
if condition:
then_this_statement
elif condition:
then_this_statement
else:
this_condition
#!/usr/bin/python
a=10
b=12
c=15
if (a==b):
print “a=b”
elif (b==c):
print “b=c”
elif (c==a):
print “c=a”
else:
print “none”
if condition:
then_this_statement
if nested_condition:
then_this_nested_statement
else nested-else_condition:
then_this_nested-else_statement
while this_condition_statement_is-true:
run_this_statement
#!/usr/bin/python
a=10
c=15
while (a<c):
print a
a=a+1
10
11
12
13
14
#!/usr/bin/python
sample_tup=(‘23’,’test’,12,’w2’)
for items in sample_tup:
print items
123
test
12
w2
#!/usr/bin/python
str=“String”
for items in str:
print items
S
t
r
i
n
g
#!/usr/bin/python
y=“Module String”
#!/usr/bin/python
import x
print x.y
Module String
#!/usr/bin/python
def simplefunc(atr_arg):
print “Print me first”
print atr_arg
return
str=“Sample String”
simplefunc(str)
Print me first
Sample String
#!/usr/bin/python
class sample_class:
def __init__(self, classarg):
self.cla=classarg
def firstfunc(self):
print “First Function”
return self.cla+“ Return”
def secfunc(self):
print “Second Function”
return self.cla+“ Return”
classobj=sample_class(“Argument”)
print classobj.firstfunc()
print classobj.secfunc()
First Function
Argument Return
Second Function
Argument Return
#!/usr/bin/python
class sample_class:
def __init__(self, classarg):
self.cla=classarg
def firstfunc(self):
print “First Function”
return self.cla+“ Return”
def secfunc(self):
print “Second Function”
return self.cla+“ Return”
classobj=sample_class(“Argument”)
#!/usr/bin/python
from mod import ∗
print classobj.firstfunc()
First Function
Argument
>>>sample_file=open(‘text.txt’,“w”)
>>>sample_file(“test data”)
>>>sample_file.close()
>>>sample_file=open(‘text.txt’,“r”)
>>>sample_file.read()
‘test data’
>>>sample_file.close()
#!/usr/bin/python
import sys
a=sys.argv[1]
print a
print a∗4
a=int(a)
print a
print a∗4
2
2222
2
8
#!/usr/bin/python
import sys
a=raw_input(“Enter something: ”)
print a∗4
#!/usr/bin/python
import sys
import urllib2
from bs4 import BeautifulSoup
url=“http://www.zoomeye.org/search?q=”
term=sys.argv[1]
comurl=url+term
response=urllib2.urlopen(comurl)
soup = BeautifulSoup(response)
for item in soup.findAll(“a”,{‘class’:’ip’}):
print item.string
#! /usr/bin/python
# A sample burp extension in python (needs jython) which extracts hostname from the request (Target Tab).
from burp import IBurpExtender
from burp import IMenuItemHandler
import re
import urllib2
class BurpExtender(IBurpExtender):
def registerExtenderCallbacks(self, callbacks):
self.mCallBacks = callbacks
self.mCallBacks.registerMenuItem(“Sample Extension”, hostnamefunc())
class hostnamefunc(IMenuItemHandler):
def menuItemClicked(self, menuItemCaption, messageInfo):
print “--- Hostname Extract ---”
if messageInfo:
request1=HttpRequest(messageInfo[0].getRequest())
req=request1.request
host=req[1]
print host
print “DONE”
class HttpRequest:
def __init__(self, request):
self.request=request.tostring().splitlines()
#!/usr/bin/python
import sys
import urllib2
mt = MaltegoTransform()
mt.parseArguments(sys.argv)
email=mt.getValue()
hibp=“https://haveibeenpwned.com/api/breachedaccount/”
getrequrl=hibp+email
try:
response = urllib2.urlopen(getrequrl)
for rep in response:
mt.addEntity(“maltego.Phrase”,“Pwned at ” + rep)
except:
print “”
mt.returnoutput()
#!/usr/bin/python
from MaltegoTransform import ∗
import sys
import urllib2
mt = MaltegoTransform()
mt.parseArguments(sys.argv)
email=mt.getValue()
hibp=“https://haveibeenpwned.com/api/breachedaccount/”
getrequrl=hibp+email
try:
response = urllib2.urlopen(getrequrl)
for rep in response:
mt.addEntity(“maltego.Phrase”,“Pwned at ” + rep)
except:
print “”
mt.returnoutput()
./emailhibp.py foo@bar.com
python ./emailhibp.py foo@bar.com
/usr/bin/python (for Linux)
/root/Desktop/transforms/emailhibp.py (for Linux)
#!/usr/bin/python
from MaltegoTransform import ∗
import sys
import urllib2
import re
mt = MaltegoTransform()
mt.parseArguments(sys.argv)
url=mt.getValue()
mt = MaltegoTransform()
opencnam=“http://www.my-ip-neighbors.com/?domain=”
getrequrl=opencnam+url
header={‘User-Agent’:’Mozilla’}
req=urllib2.Request(getrequrl,None,header)
response=urllib2.urlopen(req)
domains=re.findall(“((?:[0-9]∗[a-z][a-z\\.\\d\\-]+)\\.(?:[0-9]∗[a-z][a-z\\-]+))(?![\\w\\.])”,response.read())
for domain in domains:
mt.addEntity(“maltego.Domain”, domain)
mt.returnoutput()
In this chapter we will combine the knowledge we have gained in previous chapters to learn how to utilize it for practical usage. We will talk about different scenarios and examples and through those try to generate an understanding of which tool/technique would be useful in what situation and how to hop from one to another. Later in the chapter we will also learn how to create Maltego machines, which can prove very helpful when we need to perform various transforms on one entity or in sequence.
site:example.com inurl:vulnpar
ping <domain name>
site:domainname
site:domainname keyword filetype:ppt
machine(“sudhanshuchauhan.domaintoHIBP”,
displayName:“domaintoHIBP”,
author:“Sudhanshu”,
description: “Domain name to HaveIBeenPwned”) {
start {
paths{
path{
run(“paterva.v2.DomainToEmailAddress_AtDomain_SE”)
run(“sudhanshuchauhan.emailhibp”)
}
path{
run(“paterva.v2.DomainToEmailAddress_SE”)
run(“sudhanshuchauhan.emailhibp”)
}
path{
run(“paterva.v2.DomainToEmailAddress_Whois”)
run(“sudhanshuchauhan.emailhibp”)
}
path{
run(“paterva.v2.DomainToEmailAddress_PGP”)
run(“sudhanshuchauhan.emailhibp”)
}
}
}
}
We have dealt with various topics related to generating actionable intelligence and also learned about different scenarios and examples associated with it. In this last installment we will discuss about some topics of interest associated to computing and internet culture in general. Anyone who has practiced the knowledge shared in previous chapters must have encountered these topics somewhere, here we strive to create a basic understanding and explore the opportunities related to them.
Note: Page numbers followed by “f” and “b” indicate figures and boxes respectively.
Chapter 1. Foundation: Understanding the Basics
Chapter 2. Open Source Intelligence and Advanced Social Media Search
Advanced search techniques for some specific social media
Chapter 3. Understanding Browsers and Beyond
Some of the well-known custom browsers
Chapter 4. Search the Web—Beyond Convention
Chapter 5. Advanced Web Searching
Chapter 6. OSINT Tools and Techniques
Why do we need to be anonymous
Chapter 9. Deepweb: Exploring the Darkest Corners of the Internet
Chapter 10. Data Management and Visualization
Data management and analysis tools
Update operating system and other applications
Precautions against social engineering
Chapter 12. Basics of Social Networks Analysis
Chapter 13. Quick and Dirty Python
Chapter 14. Case Studies and Examples
In this chapter we will be setting up the basics clear and understand the working of the internet. After that we will talk about the World Wide Web. Then we will move ahead and learn about some basic technologies of the computer world, which will help us later to get a clear picture of what we are dealing with. Later in this chapter we will be describing about the basic environment setup, so that the examples and exercises discussed in the book could be easily replicated by the readers.
After understanding the basics of the internet in the first chapter, in this chapter we will be learning about the open source intelligence. We will see it from an average user's perspective and will talk about the basic ways in which we access it. Then we will move forward and understand the evolution of the web as we know it today, i.e., Web 2.0 and how it influences us. Afterward we will discuss about Social Media Intelligence. Then we will be dealing with social networks and talk about advanced social media search. In the end we will briefly talk about the web of the future, also termed as Web 3.0
In the last chapter we discussed about open source intelligence and learned about advanced search mechanisms of various social networking sites, now we will be moving on and we will be specifically dealing with browsers in this chapter. First we will talk about some basic information about browsers and their brief history. After that we will move on to the architecture of browsers. Then we will discuss about some of the common features of browsers such as autocomplete, bookmarks, proxy, etc. Further we will learn about possibility and need for custom browsers. Once we are into custom browsers, we will talk about some of the available custom browsers and their functionalities. Later in this chapter we will discuss the term addons and will also learn about some of the interesting addons. Toward the end of this chapter, bookmarks will be discussed and in the end we will briefly talk about the threat that browsers pose toward the users.
We learned about browsers and their advanced usage in last chapter, now we will be moving on to the search engines, but not the conventional ones. In this chapter we will discuss about various different kinds of search engines which are domain specific and allow us to find information which regular search engines don’t. We will discuss about their working, understand their usage, and see how they are different. There will be various categories under which we will discuss the best search engines that belong there. These platforms that we will discuss are not a replacement to the conventional ones but simply complement them.
In last few chapters we learned about searching various social media platforms, unconventional search engines, and better usage of our browsers. In this chapter we will learn how to use conventional search engines to perform better search using various different advanced search operators and techniques. We will check out advanced features of some of the most popular search engines and see how we use them to their full potential so that we can achieve precise results.
Till now we have learned a lot related to searching the web in various different ways just by using the browser. In this chapter we will learn about some applications and web-based services, which automate the process of data extraction. We will learn about different ways to extract data simply by clicking on a graphical interface to executing commands in a command-line interface (CLI). The tools' interface and their usage will be demonstrated in detail so that the users can get a strong hold of and can easily explore them further.
Lately we have learned a lot about the conventional and unconventional ways of extracting information from the internet. In this chapter we will move a step further and understand about metadata, the hidden treasure. It is mostly ignored due to the unawareness or negligence, but sometimes it can play a game changing role during cyber exercises. We will discuss about the basic definition of the term, where it is usually found, how to extract it, and also some defensive measures.
Previously we have covered a lot about finding information related to people, organizations, IT infrastructure etc. But what about our own identity? We use internet on a daily basis for professional as well as personal use, and during this usage we reveal our identity in many ways. In this chapter we will learn how to stay anonymous online using different tools and techniques. Though promising 100% anonymity is very difficult, still using the mentioned methods will help to achieve it up to a certain level.
After learning about being anonymous on the web, in this chapter we will be moving on to explore some of the deepest and darkest corners of the internet. We will learn about the places which are hidden from a casual internet user. Methods to access as well as creating this secret space in the internet will be discussed. We will find out some of the most common and frequently visited parts of this area and learn what can be expected there.
Till now we have learned to extract data from various sources and also about anonymity. In this chapter we will discuss about data management and visualization. We will understand how data transforms into intelligence and what are the tools which can assist us in that process. Using the tools to manage and visualize data we will be able to see how they can play a key role in the process of informed decision-making.
We have covered many topics related to how to use the internet in a better way for data extraction. In the last chapter we also discussed about how to manage and visualize data for analysis purpose. Here we will be discussing about one of the most relevant topics related to the virtual world i.e., online security. We will learn what are the common threats that haunt us in the virtual space, how to identify them, and most importantly what steps can be taken to minimize the risk of becoming a victim of such attacks.
The information that we have learned to collect till now is in many forms. When it comes to basic analysis, these can be divided into entities and the relationship between the entities. Social network analysis (SNA) is basically about understanding data through network graphs in the form of entities and relationships and analyzing it to make generate a conclusion out of it. In this chapter we will cover some basic concepts of SNA to introduce the readers to this interesting subject so that they can get familiar with it and further explore it depending upon personal requirements.
It need not to be proved that automated tools are very helpful in performing various tasks. We have already discussed about a lot of such tools in previous chapters. Here we will be taking a step further to learn the basics of Python language to create our own custom tools for custom requirements. We will start with the learning about the essential components and then move on to create our own scripts, as committed in a previous chapter we will also use it to create transforms for our beloved Maltego.
print “Hello World”
python helloworld.py
chmod 755 helloworld.py
./helloworld.py
#!/usr/bin/python
>>>samplenum=10
>>>samplestr=”This is a string”
>>>samplestr2=’This is another string’
>>>sample_str=”There are total %d number of floors in the %s building”%(4,’xyz’)
>>>sample_str
>>>samplelist=[123, “str”, ‘xyz’, 321, 21.22]
>>>samplelist
[123, “str”, ‘xyz’, 321, 21.22]
>>>samplelist[1]
‘str’
>>>sampledict={‘test1’:’123’,’test2’:’234’,’test3’:’345’}
>>>sampledict[‘test1’]
‘123’
>>>sampledict[‘test4’]=’456’
>>>sampledict[‘test3’]=’333’
>>>sampledict
{‘test1’: ’123’, ’test2’: ’234’, ’test3’: ’333’, ’test4’: ’456’}
>>>dir(sampledict)
>>>help(sampledict)
>>>a=12
>>>b=2
>>>a∗b
24
>>>a=”test”
>>>b=”next”
>>>a+b
‘test next’
>>>lt1=[‘1’,’2’,’3’]
>>>lt2=[‘4’,’5’,’6’]
>>>lt1+lt2
[‘1’, ‘2’, ‘3’, ‘4’, ‘5’, ‘6’]
>>>a=1
>>>b=2
>>>a+b
3
>>>a=”test”
>>>b=”string”
>>>a+b
‘teststring’
>>>a.upper()
‘TEST’
>>>c=”This is a string”
>>>c.find(‘ring’)
12
>>>c.find(‘xyz’)
-1
>>>sample_list=[‘qw’,’er’,’ty’,123]
>>>sample_list.append(456)
>>>sample_list
[‘qw’, ’er’, ’ty’, 123, 456]
if condition:
then_this_statement
elif condition:
then_this_statement
else:
this_condition
#!/usr/bin/python
a=10
b=12
c=15
if (a==b):
print “a=b”
elif (b==c):
print “b=c”
elif (c==a):
print “c=a”
else:
print “none”
if condition:
then_this_statement
if nested_condition:
then_this_nested_statement
else nested-else_condition:
then_this_nested-else_statement
while this_condition_statement_is-true:
run_this_statement
#!/usr/bin/python
a=10
c=15
while (a<c):
print a
a=a+1
10
11
12
13
14
#!/usr/bin/python
sample_tup=(‘23’,’test’,12,’w2’)
for items in sample_tup:
print items
123
test
12
w2
#!/usr/bin/python
str=“String”
for items in str:
print items
S
t
r
i
n
g
#!/usr/bin/python
y=“Module String”
#!/usr/bin/python
import x
print x.y
Module String
#!/usr/bin/python
def simplefunc(atr_arg):
print “Print me first”
print atr_arg
return
str=“Sample String”
simplefunc(str)
Print me first
Sample String
#!/usr/bin/python
class sample_class:
def __init__(self, classarg):
self.cla=classarg
def firstfunc(self):
print “First Function”
return self.cla+“ Return”
def secfunc(self):
print “Second Function”
return self.cla+“ Return”
classobj=sample_class(“Argument”)
print classobj.firstfunc()
print classobj.secfunc()
First Function
Argument Return
Second Function
Argument Return
#!/usr/bin/python
class sample_class:
def __init__(self, classarg):
self.cla=classarg
def firstfunc(self):
print “First Function”
return self.cla+“ Return”
def secfunc(self):
print “Second Function”
return self.cla+“ Return”
classobj=sample_class(“Argument”)
#!/usr/bin/python
from mod import ∗
print classobj.firstfunc()
First Function
Argument
>>>sample_file=open(‘text.txt’,“w”)
>>>sample_file(“test data”)
>>>sample_file.close()
>>>sample_file=open(‘text.txt’,“r”)
>>>sample_file.read()
‘test data’
>>>sample_file.close()
#!/usr/bin/python
import sys
a=sys.argv[1]
print a
print a∗4
a=int(a)
print a
print a∗4
2
2222
2
8
#!/usr/bin/python
import sys
a=raw_input(“Enter something: ”)
print a∗4
#!/usr/bin/python
import sys
import urllib2
from bs4 import BeautifulSoup
url=“http://www.zoomeye.org/search?q=”
term=sys.argv[1]
comurl=url+term
response=urllib2.urlopen(comurl)
soup = BeautifulSoup(response)
for item in soup.findAll(“a”,{‘class’:’ip’}):
print item.string
#! /usr/bin/python
# A sample burp extension in python (needs jython) which extracts hostname from the request (Target Tab).
from burp import IBurpExtender
from burp import IMenuItemHandler
import re
import urllib2
class BurpExtender(IBurpExtender):
def registerExtenderCallbacks(self, callbacks):
self.mCallBacks = callbacks
self.mCallBacks.registerMenuItem(“Sample Extension”, hostnamefunc())
class hostnamefunc(IMenuItemHandler):
def menuItemClicked(self, menuItemCaption, messageInfo):
print “--- Hostname Extract ---”
if messageInfo:
request1=HttpRequest(messageInfo[0].getRequest())
req=request1.request
host=req[1]
print host
print “DONE”
class HttpRequest:
def __init__(self, request):
self.request=request.tostring().splitlines()
#!/usr/bin/python
import sys
import urllib2
mt = MaltegoTransform()
mt.parseArguments(sys.argv)
email=mt.getValue()
hibp=“https://haveibeenpwned.com/api/breachedaccount/”
getrequrl=hibp+email
try:
response = urllib2.urlopen(getrequrl)
for rep in response:
mt.addEntity(“maltego.Phrase”,“Pwned at ” + rep)
except:
print “”
mt.returnoutput()
#!/usr/bin/python
from MaltegoTransform import ∗
import sys
import urllib2
mt = MaltegoTransform()
mt.parseArguments(sys.argv)
email=mt.getValue()
hibp=“https://haveibeenpwned.com/api/breachedaccount/”
getrequrl=hibp+email
try:
response = urllib2.urlopen(getrequrl)
for rep in response:
mt.addEntity(“maltego.Phrase”,“Pwned at ” + rep)
except:
print “”
mt.returnoutput()
./emailhibp.py foo@bar.com
python ./emailhibp.py foo@bar.com
/usr/bin/python (for Linux)
/root/Desktop/transforms/emailhibp.py (for Linux)
#!/usr/bin/python
from MaltegoTransform import ∗
import sys
import urllib2
import re
mt = MaltegoTransform()
mt.parseArguments(sys.argv)
url=mt.getValue()
mt = MaltegoTransform()
opencnam=“http://www.my-ip-neighbors.com/?domain=”
getrequrl=opencnam+url
header={‘User-Agent’:’Mozilla’}
req=urllib2.Request(getrequrl,None,header)
response=urllib2.urlopen(req)
domains=re.findall(“((?:[0-9]∗[a-z][a-z\\.\\d\\-]+)\\.(?:[0-9]∗[a-z][a-z\\-]+))(?![\\w\\.])”,response.read())
for domain in domains:
mt.addEntity(“maltego.Domain”, domain)
mt.returnoutput()
In this chapter we will combine the knowledge we have gained in previous chapters to learn how to utilize it for practical usage. We will talk about different scenarios and examples and through those try to generate an understanding of which tool/technique would be useful in what situation and how to hop from one to another. Later in the chapter we will also learn how to create Maltego machines, which can prove very helpful when we need to perform various transforms on one entity or in sequence.
site:example.com inurl:vulnpar
ping <domain name>
site:domainname
site:domainname keyword filetype:ppt
machine(“sudhanshuchauhan.domaintoHIBP”,
displayName:“domaintoHIBP”,
author:“Sudhanshu”,
description: “Domain name to HaveIBeenPwned”) {
start {
paths{
path{
run(“paterva.v2.DomainToEmailAddress_AtDomain_SE”)
run(“sudhanshuchauhan.emailhibp”)
}
path{
run(“paterva.v2.DomainToEmailAddress_SE”)
run(“sudhanshuchauhan.emailhibp”)
}
path{
run(“paterva.v2.DomainToEmailAddress_Whois”)
run(“sudhanshuchauhan.emailhibp”)
}
path{
run(“paterva.v2.DomainToEmailAddress_PGP”)
run(“sudhanshuchauhan.emailhibp”)
}
}
}
}
We have dealt with various topics related to generating actionable intelligence and also learned about different scenarios and examples associated with it. In this last installment we will discuss about some topics of interest associated to computing and internet culture in general. Anyone who has practiced the knowledge shared in previous chapters must have encountered these topics somewhere, here we strive to create a basic understanding and explore the opportunities related to them.
Note: Page numbers followed by “f” and “b” indicate figures and boxes respectively.
