Network Evidence Acquisition and Command Line Tools for Threat Response

Network Evidence
Acquisition
Why Data Acquisition?
• Hard drive for the infected hosts may not be available
• Data acquired ahead of time helps investigate the situation
• You cannot always anticipate when and where an
attack would take place
• Data acquired can help revisit the data later to identify
previous attacks and/or penetrations
• Data acquired helps reconstruct the attackers’ actions
• Helpful to understand the intent
• Helpful to analyze the damage caused
CYBR 5940 - Threat
Intelligence and Incident
Response
The Command Line
Shell
• The shell is a program that takes commands from
the keyboard and gives them to the OS to perform
• It is an environment in which we can run our
commands, programs, and shell scripts
[http://linuxcommand.org, https://www.tutorialspoint.com]
Bash
• In Linux, you get to choose which shell program to
use
• Ubuntu uses “The Bourne Again Shell (BASH)”
• Bash is a command-line only interface
• It contains a handful of built-in commands
• It can launch other programs
• It also controls the programs launched by it
• This might seem simple, but Bash is a very powerful
tool
Environment Variables
• Every instance of shell and every process has its
own “environment”
• Environment variables can be set by user
• To set an environment variable:
• VARIABLE=VALUE
Environment Variables
• You can use environment variables as parameters:
• VARIABLE=VALUE
• echo $VARIABLE
cat
• A command-line tool for reading files sequentially
• cat FILENAME
grep
• Helps search plain-text for lines that match a
regular expression
• grep SEARCH_QUERY filename
Pipes
• Helps send output of one command as input to
another command:
• cat filename | grep TERM
cut
• Helps extract sections from each line of input
• Can be used with a delimiter (-d)
• cat filename | cut –d ‘DELIMITER’ –f N
• N is the word to be selected (column)
• Can be used to print N number of tokens separated
by a delimiter
• cat filename | cut –d ‘DELIMITER’ –f 1-N
head and tail
• Head - display the beginning of a text file or piped
data
• Tail - display the end of a text file or piped data
• Default is 10 lines
• cat filename | head –N
• cat filename | tail -N
tr
• Can be used to replace a letter with another
• cat filename | tr ‘a’ ‘b’
• Very helpful to replace special characters (e.g. end
of line \n) and vice versa
• cat filename | tr ‘\n’ ‘ ’
awk
• Can be used to select a specific word
• cat filename | awk ‘{print $N}’
sort
• Can be used to sort a list of lines
• cat filename | sort
• -n numeric sort
• -r reverse sort
• -k sort by a column
uniq
• Can be used to remove repeated lines
• cat filename | sort | uniq
• cat filename | sort | uniq -c
Redirection
• Helps take the output of a program and save it to a
file:
• ls > /tmp/test
• You can append to an existing file by:
• ls >> /tmp/test
Redirection (cont’d.)
• The third class of redirection is using a file as input:
• grep “root” < /etc/passwd
• To create an empty file:
• touch FILENAME
Multiple commands
• To run multiple programs consecutively:
• command1; command2
• e.g., ls; cat /etc/passwd
• You can also use double ampersand && instead of a
semi colon
• e.g., ls && cat /etc/passwd
Backticks
• You can take the output of one program and make it a
parameter of another program
• Any text enclosed within backticks () is treated as a command to be executed e.g., command • You can also use $ sign and enclose the command within brackets e.g., $(command) • e.g., testing=dateecho "The date and time are: $testing" vs. echo "The date and time are:date`"
CYBR 5940 - Threat
Intelligence and Incident
Response
tcpdump/libpcap
tcpdump
• Packet capture tool
• Command-line based
• Also, network packet analyzer
• Based on libpcap library
• Distributed under the BSD license
• tcpdump is free software
• Reads and writes to .pcap, .pcapng files
.pcap, .pcapng files
• An API for capturing network traffic
• Stands for Packet CAPture
• .cap and .dmp extensions also used
• Defined by:
• libpcap for UNIX-based systems
• WinPcap for Windows-based systems
• Stores packets with their:
• TCP/IP stack layer info
• Packet headers
• Encrypted/unencrypted packets
• Encrypted/unencrypted payload
• etc.
pcap file format
• Global Header contains:
• Magic number: 0xd4c3b2a1
• Time zone indicator: By default, 0 for UTC
• Time stamps: Seconds and microseconds since EPOCH
• Time stamp accuracy: Always 0
• Snapshot length: Max. packet size (for full-packet, 0)
• etc.
https://www.elvidence.com.au
pcap file format (cont’d.)
• Packet Header contains:
• Time stamps: Seconds and microseconds since EPOCH
• Length of packet
• etc.
• Packet Data contains:
• Packet itself (TCP/IP headers and payload)
https://www.elvidence.com.au
.pcap vs. .pcapng
• .pcapng is a successor file type to .pcap
• Benefits of .pcapng:
• Can store captures from multiple interfaces in the same
file
• Better timestamp resolution
• Expanded metadata capability (statistics, etc.)
• .pcapng not universally supported
• Recommended to convert to .pcap before processing
• editcap –F pcap input.pcapng output.pcap
CYBR 5940 - Threat
Intelligence and Incident
Response
Data Reduction using
tcpdump
BPF primitives
• Stands for Berkeley Packet Filters (BPF)
• Provide a powerful tool for intrusion detection
analysis
• Helps reduce large packet captures to a reduced set
• Helps filter packets based on a specific type of traffic
• BPF expressions consist of one or more primitives
• Primitives usually consist of an id (name or
number) preceded by one or more qualifiers
https://www.ibm.com/support/knowledgecenter/SS42VS_7.4/com.ibm.qradar.doc/c_forensics_bpf.html
https://www.ntop.org/guides/nprobe/bpf_expressions.html
BPF primitives
• There are three different kinds of qualifiers:
• type
• Says what kind of thing the id name or number refers to
• Possible types are host, net and port
• e.g., host IP, net 192.128.1.0/24, port 20
• If there is no type qualifier, host is assumed
https://www.ntop.org/guides/nprobe/bpf_expressions.html
BPF primitives
• There are three different kinds of qualifiers:
(cont’d.)
• direction
• Specifies a particular transfer direction to and/or from id
• Possible directions are ‘src’, ‘dst’, ‘src or dst’ and ‘src
and dst’
• e.g., src host IP, dst net 192.128.1.0/24, src or dst port 25
• If there is no direction qualifier, src or dst is assumed
https://www.ntop.org/guides/nprobe/bpf_expressions.html
BPF primitives
• There are three different kinds of qualifiers:
(cont’d.)
• protocol
• Restricts the match to a particular protocol
• Possible protos are ether, ip, arp, rarp, tcp and udp
• e.g., ether, tcp
• If there is no proto qualifier, all protocols consistent with
the type are assumed
https://www.ntop.org/guides/nprobe/bpf_expressions.html
BPF primitives
• Examples:
• Filter by IP protocol
• ‘ip’
• Filter by MAC address
• ‘ether FF:FF:FF:FF:FF:FF’
• Filter by IP address
• ‘host 192.168.1.2’
• Filter by CIDR
• ‘net 10.24.123.0/24’
BPF primitives
• Examples: (cont’d.)
• Filter by port number
• ‘port 46’
• The directionality of the packet can be specified
using “src” and “dst”
• ‘src host’, ‘src port’, ‘src net’, ‘ether src’
• NOTE: For ether, unlike the others, directionality comes after
and not before, don’t know why ☺
BPF primitives
• Primitives may be combined using:
• and, or, not, ()
• Parenthesis are very helpful
• Examples
• ‘tcp and port 80’
• ‘not src host 192.168.1.2’
• ‘tcp and (port 80 or port 8080)’
In-class Exercise 4-1
• What would be the BPF syntax for catching TCP
packets sent to or from ports 80, 443, or 8080, but
not those sent to the IP address 192.168.5.2?
tcpdump Options
• -r Read from pcap file
• -w Write to pcap file
• -i Specify interface to capture packets from
• -n Prevent DNS resolution for all IPs (Important!)
• -C Rotate pcap after file size reached
• -G Rotate pcap after number of seconds reached
(timestamp format needs to be specified in
output filename, otherwise, overwrites)
tcpdump Options
• -W Limit number of rotated pcap files
• -F Load BPF from file
• -s Specify the number of bytes for each packet to
be captured (0 to capture everything)
• -c Specify the number of packets to capture
tcpdump Examples
• Capture and display traffic from a live network
interface
• sudo tcpdump -n -i ens33 -s 0
• Filter traffic from an input file to output file for a
specific host
• sudo tcpdump -n -r input.pcap –w
output.pcap ‘host 192.168.1.34’
tcpdump Examples (cont’d.)
• Capture 7 days of UDP traffic with one day of
content per file
• sudo tcpdump -n -i ens33 -w output.pcap
-%F.%T.pcap -G 86400 -W 7 ‘udp’
• Capture rotating 100MB files of data to and from
host 192.168.1.2
• sudo tcpdump -n -i ens33 -w output.pcap
-C 100 ‘host 192.168.1.2’
In-class Exercise 4-2
• http://cs.ucmo.edu/~aksoy/teaching/cybr5940/SkypeIRC.cap
• Find the top 10 IP addresses from which the
packets originated the most
• Find the top 10 IP addresses to which hosts
connected to the most
• Note: You can use the command line tools
discussed earlier to solve these questions
End of Lecture 7
CYBR 5940 - Threat
Intelligence and Incident
Response
Web Proxy Servers
Web Proxy Servers
• Proxy servers are used to allow hosts within a
network to connect to the Internet
• Why use Proxy Servers?
• Traditionally used for caching web pages
• Also used for load balancing
• To direct the traffic from large number of requests to small
number of servers
• Also used for controlling the traffic
• Blocking access to malicious websites
• Block unwanted traffic
• e.g., Ad Blocker
Web Proxy Servers – Forensics
relevance
• Proxies provide insightful information on user and
system activities in a network
• Proxies can also be used for security (to block
undesirable content)
• Proxies can help detect which users/systems
connect to malicious sites
• Since proxies keep a cache of pages and resources
accessed/downloaded by users, they can be used
for investigation purposes
Proxy Solutions
• Squid
• Symantec ProxySG
• Forcepoint
• Apache Traffic Server
• Zscaler
• WinGate
• Privoxy
• NGINX
Squid Proxy Server
• One of the most used proxy servers
• Free, released under GPLv2
• Runs on BSD, Linux, Unix, Windows
• By default, runs on TCP/3128
Squid Proxy Server
• Configuration file: /etc/squid/squid.conf
• Gives important information about the location of
files/folders that may be used as evidence in IR
• e.g., Log files, cache data
• Log file(s): /var/log/squid/*
• Log files are helpful in investigating requests/responses
originated within the network
• Cache data: /var/spool/squid/
• Caches stored can be helpful in investigating the
responses such as files downloaded, etc.
ACL (Access Control List)
• An ACL entry consists of an allow or deny action
followed by several ACL elements
• Using ACLs, system administrators can restrict:
• To which IPs the hosts can connect
• To which websites the hosts can connect
• Which browser software can be used based on the
identification information in User-Agent
• When/what time certain actions to be
allowed/disallowed
• etc.
Squid Log File Format

The client request time stamp (EPOCH)
The response time for the request
Client machine’s IP address
Whether the response was retrieved from the
cache or from the web / Response code
• (TCP_HIT – object found in cache, TCP_MISS – object
not found in cache)
https://www.websense.com/content/ support/library/web/v773/wcg_help/squid.aspx
Squid Log File Format (cont’d.)
Reply size
Request method, GET, POST, etc.
URL requested by the client (by default, query
strings not included)
Username (hyphen means no authentication was
required)
https://www.websense.com/content/ support/library/web/v773/wcg_help/squid.aspx
Squid Log File Format (cont’d.)
The proxy hierarchy route
• e.g., The name of the server that fulfilled the request
(like authoritative DNS server)
Response content type
https://www.websense.com/content/ support/library/web/v773/wcg_help/squid.aspx
What is EPOCH?
• UNIX represents time in numbers
• It does so by counting the number of seconds since
a special date in history
• Special as in a preset date
• The date is Jan 1st, 1970
• UNIX uses 32-bits to represent time
• The first bit is used for sign
• 0 (positive) 1 (negative)
• Remaining 31 bits are the actual data
• 2
31

1 = 2,147,483,647
What is EPOCH? (cont’d.)
• Why is there a sign? Shouldn’t time always be
positive?
• Since EPOCH starts from Jan 1st, 1970, how would you
represent the date of moon-landing (July 20th, 1969)?
• The sign helps go backwards in time before EPOCH date
• Since there are 31 bits to store number of seconds,
the largest number of seconds you can represent is
2,147,483,647
• If we convert this into a specific date, Jan 19th
,
2038, 03:14:08
What is EPOCH? (cont’d.)
• If we increment this value by 1, it will overflow to
the most significant bit which gives 1 followed by
31 0’s
• This is the earliest day that can be represented in
EPOCH which is Dec 13th, 1901 (Friday the 13th )
• As a solution, we are switching to systems using
larger numbers to represent time: 64-bits
What is EPOCH? (cont’d.)
• Why did we only double it and not used more bits?
Will that be enough number of bits?
• Do you have a guess as to when we would run out
of 64-bits?
• Remember that the numbers grow exponentially
• We will run out on Dec 4th at 15:30:08 of the year
292,277,026,596!
What is EPOCH? (cont’d.)
• Commands:
• Current number of seconds since EPOCH date
• date +%s
• Convert seconds to human-readable date
• date -d @1614206437
• Convert a given date to EPOCH time
• date -d ”Feb 24 2021" +%s
• To show the date in UTC
• date -d ”Feb 24 2021" +%s -u
Squid Cache File Format
Data
URL where file
was retrieved
HTTP headers
Header-Data
separator
(\r\n\r\n,
0d 0a 0d 0a)
Searching Cache Files
• grep command can be used to locate a cache file
• grep -rail www.google.com /var/spool/squid/*
• Parameters:
• -r Recursive search
• -a Treat all files as ASCII
• -i Case-insensitive search
• -l List matching filenames, not lines