Another solution to the Network Forensics Puzzle #3
Par malphx le samedi, février 6 2010, 11:13 - Network Forensics - Lien permanent
Well, submissions for this contest are closed since 4 days now. So I think it's time for me to publish my solution...
For this contest, We've had to deal with Apple TV traffic.
Tools Used:
_ TShark 1.2.2
_ macfinder.rb (custom ruby script giving IP/MAC bindings from a pcap file.) (require packetfu)
_ httpdumper (custom ruby script that can display and dump HTTP conversations) (require packetfu and terminal-table)
_ plist.rb (custom ruby script that can display informations extracted from Apple Property-List 1.0 XML documents)
Recommandations for using these tools:
macfinder and httpdumper rely on the wonderful ruby lib: packetfu.
Significant performances improvements have been done by its author in version 0.3.1. (8x faster)
So, You SHOULD use the last version of packetfu, eg: at least, packetfu 0.3.1
.
Detailled Answers
As usual, we have to verify the evidence file integrity:franck@ODIN:~/Analysis/Sources/Puzzle_3$ md5sum evidence03.pcapOk, we're good to go !
f8a01fbe84ef960d7cbd793e0c52a6c9 evidence03.pcap
First,I used to look at the protocol hierarchy stats given by tshark to take a first look at a pcap file:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ tshark -r evidence03.pcap -qz io,phs
===================================================================
Protocol Hierarchy Statistics
Filter: frame
frame frames:1778 bytes:1508750
eth frames:1778 bytes:1508750
ip frames:1778 bytes:1508750
udp frames:28 bytes:6102
dns frames:28 bytes:6102
tcp frames:1750 bytes:1502648
http frames:167 bytes:93189
image-gif frames:33 bytes:21202
xml frames:18 bytes:20852
tcp.segments frames:65 bytes:46469
http frames:65 bytes:46469
xml frames:17 bytes:11732
image-jfif frames:48 bytes:34737
===================================================================
Hum, I bet we'll have to work with HTTP and some XML documents/data !
Let's continue...
We know that Ann has recently aquired an AppleTV and has configured it with a static IP address: 192.168.1.10
For some obvious reason, we have to know the mac (or hardware) address of Ann's new HDTV box.
Tshark could help for this task:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ tshark -r evidence03.pcap -R "ip.src==192.168.1.10" -Tfields -e "eth.src" |uniq
00:25:00:fe:07:c4
I've also coded a small ruby script for this task: macfinder.rb. Here's the help screen:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./macfinder.rb
macfinder version 0.1
Copyright (C) 2009 Franck GUENICHOT
macfinder comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome
to redistribute it under certain conditions.
(GPL v3)
Usage: macfinder [options]
-i, --ipDisplay Mac address for the given IP address only (4-digit decimal dot notation form)
-v, --version Display version information
-h, --help Display this screen
Without any switch, macfinder.rb displays all the source IP/MAC address found in the pcap file:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./macfinder.rb evidence03.pcapFrom the listing above we can easily find Ann's AppleTV mac address.
Listing all Mac Address found !
IP: 8.18.65.10 | Mac: 00:23:69:ad:57:7b
IP: 8.18.65.32 | Mac: 00:23:69:ad:57:7b
IP: 8.18.65.88 | Mac: 00:23:69:ad:57:7b
IP: 8.18.65.89 | Mac: 00:23:69:ad:57:7b
IP: 8.18.65.67 | Mac: 00:23:69:ad:57:7b
IP: 8.18.65.58 | Mac: 00:23:69:ad:57:7b
IP: 8.18.65.82 | Mac: 00:23:69:ad:57:7b
IP: 8.18.65.27 | Mac: 00:23:69:ad:57:7b
IP: 192.168.1.10 | Mac: 00:25:00:fe:07:c4
IP: 4.2.2.1 | Mac: 00:23:69:ad:57:7b
IP: 66.235.132.121 | Mac: 00:23:69:ad:57:7b
But to be less verbose, and because we know the IP address, we can use the -i switch to display only the interresting MAC.
Here's the help screen:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./macfinder.rb -i 192.168.1.10 evidence03.pcap
Mac: 00:25:00:fe:07:c4
And Voila !
Going Deeper Part I : HTTP
Now, we have to go deeper in the pcap file to analyse Ann's networking activity and particularly
her AppleTV network conversations.
Tshark let us know that we'll have to deal with HTTP (and maybe XML documents, later), so I wrote a specialized tools
to facilitate the investigation: httpdumper
httpdumper basically displays informations about HTTP conversations. The HTTP protocol is a Request/Response protocol meaning
that a client makes a request to a server with HTTP request messages and the server answers with HTTP response messages.
httpdumper handles this mechanism and displays these conversations in an easy to understand manner.
Some terminology:
An HTTP conversation, for httpdumper, is the set of all REQUEST/RESPONSE HTTP messages involving the same 2 hosts and tcp ports.
An HTTP flow is an unidirectionnal flow of http data (eg: client to server (request) or server to client (response)
Here's the help screen:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -h
httpdumper version 0.1
Copyright (C) 2010 Franck GUENICHOT
httpdumper comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome
to redistribute it under certain conditions.
(GPL v3)
Usage: httpdumper [options] -r
-r, --readRead the given pcap file [REQUIRED]
-c, --conversation # List only flows for conversation #
-f, --flow # List only flow #
--with-headers For Display ONLY
-d, --dump Dump the selected conversation or flow
-p, --portDefine custom HTTP port
-s, --stats type,[val1],[val2] Displays statistics
Valid options:
Request stats: request,[requester_ip],[requested_host]
URI list: uri,[requester_ip],[target_hostname]
-v, --version Display version information
-h, --help Display this screen
httpdumper is only a passive (lightweight) analysis tool, it needs a file in entry, so -r options is required to launch this tool.
The default output (without any options) displays all the HTTP conversations found in the given pcap file.
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.385 sec.
Found 20 HTTP conversation(s)
+--------------------+--------------------------------------------+-----------------+---------+----------+-------------------+
| Conversation Index | Hosts | HTTP Flow count | Request | Response | Cumulative length |
+--------------------+--------------------------------------------+-----------------+---------+----------+-------------------+
| 0 | 192.168.1.10:49163 < - > 8.18.65.67:80 | 2 | 1 | 1 | 16551 |
| 1 | 192.168.1.10:49164 < - > 66.235.132.121:80 | 2 | 1 | 1 | 43 |
| 2 | 192.168.1.10:49165 < - > 8.18.65.32:80 | 8 | 4 | 4 | 22453 |
| 3 | 192.168.1.10:49166 < - > 66.235.132.121:80 | 8 | 4 | 4 | 172 |
| 4 | 192.168.1.10:49167 < - > 8.18.65.58:80 | 20 | 10 | 10 | 161118 |
| 5 | 192.168.1.10:49168 < - > 8.18.65.67:80 | 4 | 2 | 2 | 3157 |
| 6 | 192.168.1.10:49169 < - > 66.235.132.121:80 | 2 | 1 | 1 | 43 |
| 7 | 192.168.1.10:49170 < - > 8.18.65.82:80 | 44 | 22 | 22 | 675124 |
| 8 | 192.168.1.10:49171 < - > 8.18.65.27:80 | 6 | 3 | 3 | 13582 |
| 9 | 192.168.1.10:49172 < - > 66.235.132.121:80 | 6 | 3 | 3 | 129 |
| 10 | 192.168.1.10:49173 < - > 8.18.65.27:80 | 8 | 4 | 4 | 12744 |
| 11 | 192.168.1.10:49174 < - > 66.235.132.121:80 | 2 | 1 | 1 | 43 |
| 12 | 192.168.1.10:49175 < - > 66.235.132.121:80 | 6 | 3 | 3 | 129 |
| 13 | 192.168.1.10:49176 < - > 8.18.65.67:80 | 4 | 2 | 2 | 3493 |
| 14 | 192.168.1.10:49177 < - > 8.18.65.10:80 | 32 | 16 | 16 | 362826 |
| 15 | 192.168.1.10:49178 < - > 66.235.132.121:80 | 2 | 1 | 1 | 43 |
| 16 | 192.168.1.10:49179 < - > 8.18.65.88:80 | 20 | 10 | 10 | 5576 |
| 17 | 192.168.1.10:49180 < - > 66.235.132.121:80 | 20 | 10 | 10 | 430 |
| 18 | 192.168.1.10:49181 < - > 8.18.65.89:80 | 18 | 9 | 9 | 4861 |
| 19 | 192.168.1.10:49182 < - > 66.235.132.121:80 | 18 | 9 | 9 | 387 |
+--------------------+--------------------------------------------+-----------------+---------+----------+-------------------+
The table above show all the HTTP conversations found. (this kind of table is best viewed on large display)
The flow count indicates the number of flows in each conversations
Request and Response column, each displays the number of HTTP Request or HTTP response in each conversation
Cumulative length: the length (in Bytes) of the HTTP Payloads (or HTTP message body) exchanged in each conversation.
Note: this length takes only HTTP Response payloads into account. By now HTTP Request message body, if any, is not displayed (and not "dumpable")
Quickly, we learn interresting infos:
_ 20 HTTP conversations, all involving the same client (Ann's AppleTV)
_ 7 of them are composed of 18+ flows
_ Conversation #7 has the greater cumulative length (so, the largest HTTP payload)
But all these informations aren't enough: to continue our investigation we need to go deeper.
Let's try to answers Question #2: What User-Agent string did Ann’s AppleTV use in HTTP requests?
According to RFC2616 :
"The User-Agent request-header field contains information about the user agent originating the request. This is for statistical purposes,
the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations."
The User-Agent is just a string contained in the HTTP Request Message Header indicating which Software/Version is used to send the request.
So to get this information, we have to display the http header of Ann's AppleTV HTTP request.
httpdumper could help us for this task with the sub-options --with-header.
Let's take conversation 0 with headers displayed:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c0 --with-headers
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.471 sec.
Listing flows for conversation 0 with full http headers
----------------------------------------------------------------------
Flow Index: 0 192.168.1.10:49163 -> 8.18.65.67:80 REQUEST /WebObjects/MZStore.woa/wa/viewGrouping?id=39 0
-----------
HTTP HEADER
-----------
Accept: */*
Accept-Language: en
Accept-Encoding: gzip, deflate
Cookie: s_vi=[CS]v1|259C176A85010C29-6000010D80115D7F[CE]
User-Agent: AppleTV/2.4 <------------------------------------------------------------ Look Here
If-Modified-Since: Fri, 25 Dec 2009 04:42:31 GMT
X-Apple-Store-Front: 143441-1,3
Connection: keep-alive
Host: ax.itunes.apple.com
----------------------------------------------------------------------
Flow Index: 1 8.18.65.67:80 -> 192.168.1.10:49163 RESPONSE text/xml 16551
-----------
HTTP HEADER
-----------
Last-Modified: Sun, 27 Dec 2009 14:24:32 GMT
x-apple-lok-response-date: Sun Dec 27 06:27:25 PST 2009
Content-Encoding: gzip
x-apple-lok-current-storefront: 143441-1,3
x-apple-application-site: CUP
Content-Type: text/xml
x-apple-lok-expire-date: Sun Dec 27 07:04:32 PST 2009
x-apple-lok-stor: memcached
x-apple-max-age: 3600
x-apple-woa-inbound-url: /WebObjects/MZStore.woa/wa/viewGrouping?id=39
x-apple-application-instance: 6119
x-apple-lok-path: v0_1:MZStore/viewGrouping&id=39-143441-1,3,pc-3-Ak
x-apple-aka-ttl: Generated Sun Dec 27 06:27:25 PST 2009, Expires Sun Dec 27 07:27:25 PST 2009, TTL 3600s
x-apple-lok-ttl: Generated Sun Dec 27 06:24:32 PST 2009, Expires Sun Dec 27 07:04:32 PST 2009, TTL 2400s
x-webobjects-loadaverage: 0
Content-Length: 16551
Expires: Mon, 28 Dec 2009 04:08:02 GMT
Cache-Control: max-age=0, no-cache
Pragma: no-cache
Date: Mon, 28 Dec 2009 04:08:02 GMT
Connection: keep-alive
Vary: Accept-Encoding
Vary: X-Apple-Store-Front
X-Apple-Partner: origin.0
httpdumper "-c num" option is used to select only Conversation 0 flows, --with-headers tells httpdumper to display entirely the http header.
Now, easily, we can gather what we are searching for by looking at Flow 0 header's field: User-Agent : AppleTV/2.4
It indicates that the Software installed on Ann's AppleTV is version 2.4
To answer #3, we have to understand the protocol used to make movies search and so on...
Lets have another view of the HTTP traffic: Using -s options (or --stats) we could display the request statistics for a given client:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap --stats request,192.168.1.10
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.402 sec.
HTTP REQUEST STATISTICS
-----------------------
+----------------------------+-------------------+---------------+
| Requested Host | Requested Host IP | Request Count |
+----------------------------+-------------------+---------------+
| ax.search.itunes.apple.com | 8.18.65.32 | 30 |
| a1.phobos.apple.com | 8.18.65.58 | 48 |
| metrics.apple.com | 66.235.132.121 | 33 |
| ax.itunes.apple.com | 8.18.65.67 | 5 |
+----------------------------+-------------------+---------------+
Interresting, with the hostname of the servers in the table above, we could make some assumptions on their respective roles.
I bet that ax.search.itunes.apple.com has something to do with searching something...
Maybe metrics.apple.com is a kind of statistics gathering host. And for the two others we will look at them later.
Let's display all the URI requested by 192.168.1.10 to ax.search.itunes.apple.com :
(USE: --stats (or -s) uri,192.168.1.10,ax.search.itunes.apple.com
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap --stats uri,192.168.1.10,ax.search.itunes.apple.com
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.362 sec.
----------------------------------
Listing URI requested by 192.168.1.10
----------------------------------
----------------------------------
Requested to ax.search.itunes.apple.com
----------------------------------
[conv: 2] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=h
[conv: 2] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ha
[conv: 2] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hac
[conv: 2] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hack
[conv: 8] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=s
[conv: 8] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=sn
[conv: 8] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=sne
[conv: 10] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=sneb
[conv: 10] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=snea
[conv: 10] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=sneak
[conv: 10] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=i
[conv: 16] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ik
[conv: 16] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ikn
[conv: 16] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ikno
[conv: 16] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknow
[conv: 16] [flow: 8] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowy
[conv: 16] [flow: 10] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyo
[conv: 16] [flow: 12] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyou
[conv: 16] [flow: 14] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyour
[conv: 16] [flow: 16] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyoure
[conv: 16] [flow: 18] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourew
[conv: 18] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewa
[conv: 18] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewat
[conv: 18] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatc
[conv: 18] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatch
[conv: 18] [flow: 8] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatchi
[conv: 18] [flow: 10] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatchin
[conv: 18] [flow: 12] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatching
[conv: 18] [flow: 14] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatchingm
[conv: 18] [flow: 16] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=iknowyourewatchingme
My thoughts were right. The listing above displays all the research made by Ann on her AppleTV.
So the first four search term were:
- h
- ha
- hac
- hack
All these requests are within conversation 2 in httpdumper (look at the conv value in bracket on the left of the lines)
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 2
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.439 sec.
FLOWS TABLE
+------------+-------------------------------------+-------------------+------------------------------------------------------------------+---------------------+
| Flow Index | Hosts | HTTP message type | HTTP Request or Content type | HTTP Content Length |
+------------+-------------------------------------+-------------------+------------------------------------------------------------------+---------------------+
| 0 | 192.168.1.10:49165 -> 8.18.65.32:80 | REQUEST | /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=h | 0 |
| 1 | 8.18.65.32:80 -> 192.168.1.10:49165 | RESPONSE | text/xml; | 10087 |
| 2 | 192.168.1.10:49165 -> 8.18.65.32:80 | REQUEST | /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ha | 0 |
| 3 | 8.18.65.32:80 -> 192.168.1.10:49165 | RESPONSE | text/xml; | 10020 |
| 4 | 192.168.1.10:49165 -> 8.18.65.32:80 | REQUEST | /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hac | 0 |
| 5 | 8.18.65.32:80 -> 192.168.1.10:49165 | RESPONSE | text/xml; | 1173 |
| 6 | 192.168.1.10:49165 -> 8.18.65.32:80 | REQUEST | /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hack | 0 |
| 7 | 8.18.65.32:80 -> 192.168.1.10:49165 | RESPONSE | text/xml; | 1173 |
+------------+-------------------------------------+-------------------+------------------------------------------------------------------+---------------------+
What do we learn with this view ?
Each time, Ann's type a letter in maybe a kind of search engine, an HTTP request is sent to ax.search.itunes.apple.com.
Then, the server answers with an XML document encapsulated in an HTTP response message. We will look at this kind of document later.
[note : httpdumper can handle multi TCP segments HTTP response and reassemble all segments in one HTTP response if needed.]
It is interresting to note that more accurate is the research, more reduced is the size of the XML document. (= less matching results)
It seems that "hac" and "hack" lead to the same results.
Ok, let's continue !
We want to know now what was the title of the first movie Ann clicked on. If Ann has clicked on a link, her AppleTV may have made an HTTP request.
So, again lets display all the uri requested by 192.168.1.10, but this time without any hostname filter:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap --stats uri,192.168.1.10
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.434 sec.
----------------------------------
Listing URI requested by 192.168.1.10
----------------------------------
----------------------------------
Requested to ax.search.itunes.apple.com
----------------------------------
[conv: 2] [flow: 0] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=h
[conv: 2] [flow: 2] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=ha
[conv: 2] [flow: 4] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hac
[conv: 2] [flow: 6] /WebObjects/MZSearch.woa/wa/incrementalSearch?media=movie&q=hack
...
----------------------------------
Requested to a1.phobos.apple.com
----------------------------------
[conv: 4] [flow: 0] /us/r1000/038/Video/57/e5/af/mzi.hnmcsmdp.170x170-75.jpg
[conv: 4] [flow: 2] /us/r1000/051/Features/a8/de/6e/dj.nofulnci.170x170-75.jpg
[conv: 4] [flow: 4] /us/r1000/009/Video/f0/1e/ec/mzl.hhpbkslu.170x170-75.jpg
[conv: 4] [flow: 6] /us/r1000/037/Features/71/b1/8c/dj.orlnvciu.170x170-75.jpg
[conv: 4] [flow: 8] /us/r1000/032/Features/8e/dc/ca/dj.dzbaqgpw.170x170-75.jpg
[conv: 4] [flow: 10] /us/r1000/015/Video/88/d3/62/mzi.xtsujktt.170x170-75.jpg
[conv: 4] [flow: 12] /us/r1000/032/Music/f1/33/e0/mzi.kvyqgmsa.170x170-75.jpg
[conv: 4] [flow: 14] /us/r1000/032/Music/5c/86/a2/mzi.dutwwfyg.170x170-75.jpg
[conv: 4] [flow: 16] /us/r1000/032/Video/f0/48/dd/mzi.pizbdeal.170x170-75.jpg
[conv: 4] [flow: 18] /us/r1000/026/Music/68/ec/04/mzi.aydemkgw.170x170-75.jpg
[conv: 7] [flow: 0] /us/r1000/032/Video/f0/48/dd/mzi.pizbdeal.enc.jpg?downloadKey2=1265245618_f3a714a27ea9388f7c07104353e1d763
[conv: 7] [flow: 2] /us/r1000/038/Music/2e/10/15/mzi.qdnwlnpu.170x170-75.jpg
[conv: 7] [flow: 4] /us/r1000/026/Music/36/fd/a8/mzi.xvqemsit.170x170-75.jpg
[conv: 7] [flow: 6] /us/r1000/005/Video/87/fa/5e/mzi.fwmnbval.170x170-75.jpg
[conv: 7] [flow: 8] /us/r1000/021/Video/2e/7e/94/mzi.uamugbjf.170x170-75.jpg
[conv: 7] [flow: 10] /us/r1000/048/Video/bb/8d/f4/mzi.psezsery.170x170-75.jpg
...
----------------------------------
Requested to metrics.apple.com
----------------------------------
[conv: 1] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pageName=US-Movies-Movies-33&pccr=true&h5=appleitmsnatv%2Cappleitmsustv&ch=Movies%20main&g=http%3A%2F%2Fax.itunes.apple.com%2FWebObjects%2FMZStore.woa%2Fwa%2FviewGrouping%3Fid%3D39
[conv: 3] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dh&pageName=Movies-Search%20Hints-US&v2=h&h5=appleitmsnatv%2Cappleitmsustv&c2=h
[conv: 3] [flow: 2] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dha&pageName=Movies-Search%20Hints-US&v2=ha&h5=appleitmsnatv%2Cappleitmsustv&c2=ha
[conv: 3] [flow: 4] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dhac&pageName=Movies-Search%20Hints-US&v2=hac&h5=appleitmsnatv%2Cappleitmsustv&c2=hac
[conv: 3] [flow: 6] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dhack&pageName=Movies-Search%20Hints-US&v2=hack&h5=appleitmsnatv%2Cappleitmsustv&c2=hack
[conv: 6] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pageName=Movie%20Page-US-Hackers-Iain%20Softley-333441649&pccr=true&h5=appleitmsnatv%2Cappleitmsustv&ch=Movie%20Page&g=http%3A%2F%2Fax.itunes.apple.com%2FWebObjects%2FMZStore.woa%2Fwa%2FviewMovie%3Fid%3D333441649%26s%3D143441
[conv: 9] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Ds&pageName=Movies-Search%20Hints-US&v2=s&h5=appleitmsnatv%2Cappleitmsustv&c2=s
[conv: 9] [flow: 2] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsn&pageName=Movies-Search%20Hints-US&v2=sn&h5=appleitmsnatv%2Cappleitmsustv&c2=sn
[conv: 9] [flow: 4] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsne&pageName=Movies-Search%20Hints-US&v2=sne&h5=appleitmsnatv%2Cappleitmsustv&c2=sne
[conv: 11] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsneb&pageName=Movies-Search%20Hints-US&v2=sneb&h5=appleitmsnatv%2Cappleitmsustv&c2=sneb
[conv: 12] [flow: 0] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsnea&pageName=Movies-Search%20Hints-US&v2=snea&h5=appleitmsnatv%2Cappleitmsustv&c2=snea
[conv: 12] [flow: 2] /b/ss/applesuperglobal/1/G.6--NS?pccr=true&ch=Movies-Search&g=http%3A%2F%2Fax.search.itunes.apple.com%2FWebObjects%2FMZSearch.woa%2Fwa%2FincrementalSearch%3Fmedia%3Dmovie%26q%3Dsneak&pageName=Movies-Search%20Hints-US&v2=sneak&h5=appleitmsnatv%2Cappleitmsustv&c2=sneak
....
----------------------------------
Requested to ax.itunes.apple.com
----------------------------------
[conv: 0] [flow: 0] /WebObjects/MZStore.woa/wa/viewGrouping?id=39
[conv: 5] [flow: 0] /WebObjects/MZStore.woa/wa/viewMovie?id=333441649&s=143441
[conv: 5] [flow: 2] /WebObjects/MZStore.woa/wa/relatedItemsShelf?ct-id=3&id=333441649&storeFrontId=143441&mt=6
[conv: 13] [flow: 0] /WebObjects/MZStore.woa/wa/viewMovie?id=283963264&s=143441
[conv: 13] [flow: 2] /WebObjects/MZStore.woa/wa/relatedItemsShelf?ct-id=3&id=283963264&storeFrontId=143441&mt=6
By analysing all the uri requested, we could note at least two very explicit ones, requested to ax.itunes.apple.com /WebObjects/MZStore.woa/wa/viewMovie?id=333441649&s=143441 (in conversation 5) and /WebObjects/MZStore.woa/wa/viewMovie?id=283963264&s=143441 (in conversation 13)
Without any knowledge of the inner working, the meaning of the "viewMovie" command isn't hard to guess.
So let's assume that the first movie selected by Ann was requested in conversation #5 flow #0.
This lead us to think that Ann surely received this link in a response preceding conversation #5 and then clicked on it.
We also know that the keyword "hack" was searched in conversation #2 and some XML documents have been received after.
let's try to dump one of these documents, conversation 2 flow #7 seem to be a good candidate because it's the response received
after the full "hack" term search.
httpdumper gives you -d options to dump an HTTP response directly in a file (by now the filename is automatically generated)
You could use -d with a flow or an entire conversation, in the last case all the HTTP response will be dumped.
httpdumper can handle compressed content (Content-Encoding) and will decompress Gzipped content (only gzip or deflate by now)
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 2 -f 7 -d
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.529 sec.
Dumping data to disk: 8.18.65.32_80-192.168.1.10_49165-743066.xml
Inflating gzipped content
We have now a file named 8.18.65.32_80-192.168.1.10_49165-743066.xml.
... CUT ...
First information: the DOCTYPE line define the type of XML content described by the file: plist or PropertyList.
With some research on the Internet, we could easily found informations of this kind of file.
PropertyLists are used by Apple/itunes to describe media content, it is basically a collection of structures in which informations about the media are stored.
In our case, these files describe the informations to be displayed on the TV screen, like the movies descriptions, actors, etc... matching the research.
We have to go deeper, now, in the XML document itself to find more clues.
Going Deeper Part II: the XML Document (Or plist parsing)
Well, in fact, it is fairly easy to find the answers of the remaining questions.
when you have dumped the good XML document to disk with httpdumper.
You could simply use your favorite text editor and search through the XML the interresting informations about the movie.
But, to facilitate this work, I wrote a minimalist tool called: plist.rb.
plist.rb take an xml file in entry (a plist), reads it and parses it.
Then it will display all the information contained in the xml plist.
You could use grep, for example, to view only selected lines or attributes.
Now, it seems to be a good idea to search in our dumped xml file a reference to the url with the viewMovie command.
Let's do it
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.32_80-192.168.1.10_49165-743066.xml |grep "viewMovie?id=333441649&s=143441" -A 5 -B 5
link-metadata:
title: Hackers
title2: Released 1998
url-page-type: list
url: http://ax.itunes.apple.com/WebObjects/MZStore.woa/wa/viewMovie?id=333441649&s=143441
type: link
box-height: 170
url: http://a1.phobos.apple.com/us/r1000/026/Music/68/ec/04/mzi.aydemkgw.170x170-75.jpg
From the output above, we could view that our url is associated with a movie. And the title of this movie is : Hackers
Now, we want the full URL to the movie trailer, and we know that this information is stored in a key named : preview-url
Again with plist and grep:franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.32_80-192.168.1.10_49165-743066.xml |grep preview-url
franck@ODIN:~/Analysis/Sources/Puzzle_3$
Hum... it seems that this url is not in this plist file, so maybe we will find it in the xml file received by Ann after she has clicked on the movie link. So in conversation #5
Below are the step, that i took to find the answer:
Listing the conv#5 flows:franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 5
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.274 sec.
FLOWS TABLE
+------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+
| Flow Index | Hosts | HTTP message type | HTTP Request or Content type | HTTP Content Length |
+------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+
| 0 | 192.168.1.10:49168 -> 8.18.65.67:80 | REQUEST | /WebObjects/MZStore.woa/wa/viewMovie?id=333441649&s=143441 | 0 |
| 1 | 8.18.65.67:80 -> 192.168.1.10:49168 | RESPONSE | text/xml | 2278 |
| 2 | 192.168.1.10:49168 -> 8.18.65.67:80 | REQUEST | /WebObjects/MZStore.woa/wa/relatedItemsShelf?ct-id=3&id=333441649&storeFrontId=143441&mt=6 | 0 |
| 3 | 8.18.65.67:80 -> 192.168.1.10:49168 | RESPONSE | text/xml | 879 |
+------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+
Dumping flow 1, because it's the response to Ann's request in flow 0
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 5 -f 1 -d
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.275 sec.
Dumping data to disk: 8.18.65.67_80-192.168.1.10_49168-758883.xml
Inflating gzipped content
Now, requesting 'preview-url' in this new xml file:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49168-758883.xml |grep preview-url
preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v
preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v
preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v
preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v
preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v
preview-url: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v
Finally, a simple step to be sure:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49168-758883.xml |grep title
title: Hackers
title2: 22 Ratings
unmodified-title: Hackers
item-title: Jonny Lee Miller
item-title: Angelina Jolie
item-title: Jesse Bradford
item-title: Matthew Lillard
item-title: Laurence Mason
section-title: Actors
item-title: Iain Softley
section-title: Director
item-title: Michael Peyser
item-title: Ralph Winter
section-title: Producers
So the preview-url was: http://a227.v.phobos.apple.com/us/r1000/008/Video/62/bd/1b/mzm.plqacyqb..640x278.h264lc.d2.p.m4v
As this is already a long explanation, I will just write the steps and screen ouputs for the last answers.
Title of the second movie:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 13
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.267 sec.
FLOWS TABLE
+------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+
| Flow Index | Hosts | HTTP message type | HTTP Request or Content type | HTTP Content Length |
+------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+
| 0 | 192.168.1.10:49176 -> 8.18.65.67:80 | REQUEST | /WebObjects/MZStore.woa/wa/viewMovie?id=283963264&s=143441 | 0 |
| 1 | 8.18.65.67:80 -> 192.168.1.10:49176 | RESPONSE | text/xml | 2586 |
| 2 | 192.168.1.10:49176 -> 8.18.65.67:80 | REQUEST | /WebObjects/MZStore.woa/wa/relatedItemsShelf?ct-id=3&id=283963264&storeFrontId=143441&mt=6 | 0 |
| 3 | 8.18.65.67:80 -> 192.168.1.10:49176 | RESPONSE | text/xml | 907 |
+------------+-------------------------------------+-------------------+--------------------------------------------------------------------------------------------+---------------------+
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./httpdumper -r evidence03.pcap -c 13 -f 1 -d
Reading file evidence03.pcap
Parsing packets...
1778 packets read in 4.208 sec.
Dumping data to disk: 8.18.65.67_80-192.168.1.10_49176-879127.xml
Inflating gzipped content
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49176-879127.xml |grep title
title: Sneakers
title2: 43 Ratings
unmodified-title: Sneakers
item-title: Robert Redford
item-title: Dan Aykroyd
item-title: Ben Kingsley
item-title: Mary McDonnell
item-title: River Phoenix
item-title: Sidney Poitier
section-title: Actors
item-title: Phil Alden Robinson
section-title: Director
item-title: Lawrence Lasker
item-title: Lindsley Parsons Jr.
item-title: Walter F. Parkes
section-title: Producers
item-title: Phil Alden Robinson
item-title: Walter F. Parkes
item-title: Lawrence Lasker
section-title: Screenwriters
The title was : Sneakers
What was the price to buy it:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49176-879127.xml |grep price-display
price-display: $9.99
rent-price-display: $2.99
price-display: $9.99
rent-price-display: $2.99
So the price was: $9.99
We have multiple matches here, anyway they are the same. But we could display more lines with grep to be sure:
franck@ODIN:~/Analysis/Sources/Puzzle_3$ ./plist.rb 8.18.65.67_80-192.168.1.10_49176-879127.xml |grep price-display -A 5 -B 5
store-offers:
STDQ:
price: 9.99
duration: 7518393
size: 1236583962
price-display: $9.99
preview-url: http://a1738.v.phobos.apple.com/us/r1000/011/Video/7f/9d/ce/mzm.gbctwnmq..640x352.h264lc.D2.p.m4v
buy-params: productType=V&salableAdamId=283963264&pricingParameters=STDQ&price=9990
action-display-name: Buy
preview-duration: 164080.0
screen-format: widescreen
--
action-display-name: Rent
preview-duration: 164080.0
screen-format: widescreen
rent-params: productType=V&salableAdamId=283963264&pricingParameters=SDVOD&price=2990&rental=1
rental-duration: 43200
rent-price-display: $2.99
type: movie
English
--
flavors:
4:640x480LC-128:
price: 9.99
duration: 7518393
size: 1236583962
price-display: $9.99
preview-url: http://a1738.v.phobos.apple.com/us/r1000/011/Video/7f/9d/ce/mzm.gbctwnmq..640x352.h264lc.D2.p.m4v
buy-params: productType=V&salableAdamId=283963264&pricingParameters=STDQ&price=9990
action-display-name: Buy
preview-duration: 164080.0
screen-format: widescreen
--
action-display-name: Rent
preview-duration: 164080.0
screen-format: widescreen
rent-params: productType=V&salableAdamId=283963264&pricingParameters=SDVOD&price=2990&rental=1
rental-duration: 43200
rent-price-display: $2.99
the keyword action-display-name confirms that the price to BUY it was $9.99.
For the last questions, i've already answered it previously in this loooooooong document.
Last Words:
That was nice !
Jonathan, Sherri, please give us more challenge like this one !
Commentaires