Monday, June 28, 2010

Forensic contest #6 Answer

Ok, it is now 6/28/10 around the world, so here is my writeup on the latest forensics contest from forensicscontest.com.

---

Answer 1: http://10.10.10.10:8080/index.php
Answer 2: vEI
Answer 3a: index.phpmfKSxSANkeTeNrah.gif
Answer 3b: DF3E567D6F16D040326C7A0EA29A4F41
Answer 4: 1.3
Answer 5: 87.6
Answer 6a: Windows executable
Answer 6b: B062CB8344CD3E296D8868FBEF289C7C
Answer 7a: Every third packet
Answer 7b: Every packet
Answer 7c: Every 10-15 seconds
Answer 8: 123.7
Answer 9: B062CB8344CD3E296D8868FBEF289C7C
Answer 10: 198.4

---

Tools Used:
NetworkMiner 0.92 - http://networkminer.sourceforge.net/
SplitCap 1.3 - http://sourceforge.net/projects/splitcap/
Satori 0.71 - http://myweb.cableone.net/xnih/download/satori.zip
mz-filecarver 0.1 - http://myweb.cableone.net/xnih/download/filecarve-cmd.zip
contest6 pack - http://myweb.cableone.net/xnih/download/contest6.zip
frhed - http://frhed.sourceforge.net/
wireshark - http://www.wireshark.org/

Note:
The file forensics part could be done, just as easy, with linux, using tcpflow and foremost, but I wanted to to introduce some new tools and challenge myself with using something a little different and sticking to doing it all in Windows.

---

Timeline:
Date of packet capture: 2010-04-28

17:39.59.311284 - packet capture begins, client visits and grabs index.php via HTTP Get, this php file has
17:39.59.773396 - client requests index.phpmfKSxSANkeTeNrah.gif
17:40:00.577135 - Initial syn packet to port 4444 is sent that will setup a connection that stays active until 17:41:26, in which time it downloads a 748K file (meterpreter reverse tcp connection) among other things.
17:40:35.258314 - first attempted connection (out of 119 that failed) to port 4445. These packets have a unique fingerprint in how often (# of packets) sent they change their IPID, SeqNum and Source Port.
17:42:02.985483 - Initial syn packet to port 4445 is sent that will setup a connection that stays active until 17:43.17, in which time it also downloads a 748K file (another meterpreter reverse tcp session, most likely to keep a session open even after IE was closed) among other things.
17:43:17.753022 - last packet in capture

---

Process:

I always like to start by trying to determine the OS's involved in the process. This can help understand what else may be going on in the process. So the first thing we'll do is run the capture file through Satori:

We get 2 systems. According to the Overall system info we have SMB, TCP and Web User Agent identifications made:
10.10.10.70 - Windows XP SP2+ workstation (SMB, TCP, and Web fingerprints) named SaucyDev in the workgroup/domain 'workgroup' (SMB fingerprint), running what appears to be IE 6.0 (MSIE 6.0; Windows NT 5.1: SV1) - where SV1 indicates SP2 or greater installed.

10.10.10.10 - is some type of Linux system, running Apache, very generic TCP fingerprint. Over the course of the converstation we have TCP ports 4444 and 4445 open (TCP fingerprint).

Notes:
http://10.10.10.10:8080/index.php was the infected file, we see this in the Referer under Web. (question #1)

----

The packet capture started at: 17:39.59.311284 on 2010-04-28, this will be used to calculate how long since the packet capture started so we know when things happen. While in some cases I like to know how long it took to get from point A to B, I'm normally more interested in when it actually happened, so that I can try to coorelate that with system logs across multiple systems.

----

To find out when the TCP session to port 4444 was opened we need to look at the SYN packets sent. Unfortunetly, just because a SYN was sent doesn't mean the connection was actually made, there were numerous attempts made that were rejected!

It is easier to verify what happened by looking at output from the perl script (contest6.pl) that tracks 3 way handshakes (syn, syn/ack, ack):

contest6.pl -r evidence06.pcap -o conv

Attempted conversations: 119
[1] - Starts at packet number 1153
[2] - Starts at packet number 1155
[3] - Starts at packet number 1157
...
[117] - Starts at packet number 1650
[118] - Starts at packet number 1652
[119] - Starts at packet number 1654
Full 3 way handshake conversations: 2
[0] - Starts at packet number 13
[120] - Starts at packet number 1656
Total Packets: 2554

The first 3 way handshake started in packet #13

One thing I found interesting running this script is I see 119 attempted conversations that just ended in RST/ACK packets, but if I follow a lot of those conversations in Wireshark they are a different conversation #. This is due to the fact that it reuses the TCP Sequence number, Wireshark sees those as part of the same conversation. So while I see it as conversation 120, wireshark actually see's it as 42. Wireshark may be more accurate, but I think by looking at it that way it misses the fact that the remote system is making multiple attempts to connect in each TCP stream.

We can get all Syn packets using contest6.exe this way for readability:

contest6.exe -r evidence06.pcap -l S > syn-output.txt

The last 3 fields, in order are: IPID, TCP Sequence Number, and the TCP Acknowledgement Number.
13 17:40:00,577135 S 10.10.10.70:1036 -> 10.10.10.10:4444 53 72acc97a 00000000

So packet #13 was 1.265851 seconds after the start of the capture, on 1.3 seconds assuming you figure the beginning of the 3 way handshake vs the end of it. 1.3 seconds rounded off to the nearest 10th regardless. (question #4)

The other 3 way handshake took place in packet 1656, so back to syn-output.txt, we get:

1656 17:42:02,985483 S 10.10.10.70:1044 -> 10.10.10.10:4445 598 75fad66c 00000000

123.674199 seconds, or 123.7 rounded off to the nearest 10th. (question #9)

----

Now we need to look for Fin/Ack packets for connections closing, we can do that with:

contest6.exe -r evidence06.pcap -l FA > fa-output.txt

Which we see part of below:
1562 17:41:26,898379 FA 10.10.10.10:4444 -> 10.10.10.70:1036 38671 e3317217 72ae4105
1563 17:41:26,898438 FA 10.10.10.70:1036 -> 10.10.10.10:4444 551 72ae4105 e3317217

using our start time from above we get 87.587 seconds since the start, or 87.6 rounded to the nearest 10th. (answer #5)

We also see another closure here:
2552 17:43:17,751953 FA 10.10.10.70:1044 -> 10.10.10.10:4445 845 75fb02df 55a9adcc
2553 17:43:17,752630 FA 10.10.10.10:4445 -> 10.10.10.70:1044 24677 55a9adcc 75fb02e0

This ends up being 198.44 seconds from the start of the capture, or 198.4 rounded to the nearest 10th. (answer #10)

----

Going back to the syn-output.txt we can see that the system 10.10.10.70 had actually been attempting to do connections to port 4445 even before its connection to port 4444 was closed:

1533 17:41:22,305270 S 10.10.10.70:1041 -> 10.10.10.10:4445 537 00fa0ff6 00000000
1535 17:41:22,844924 S 10.10.10.70:1041 -> 10.10.10.10:4445 538 00fa0ff6 00000000
1537 17:41:23,282416 S 10.10.10.70:1041 -> 10.10.10.10:4445 539 00fa0ff6 00000000
1539 17:41:23,283067 S 10.10.10.70:1041 -> 10.10.10.10:4445 540 00fe281a 00000000
1541 17:41:23,829288 S 10.10.10.70:1041 -> 10.10.10.10:4445 541 00fe281a 00000000
1543 17:41:24,376142 S 10.10.10.70:1041 -> 10.10.10.10:4445 542 00fe281a 00000000
1545 17:41:24,376807 S 10.10.10.70:1041 -> 10.10.10.10:4445 543 0102b86a 00000000
1547 17:41:24,813664 S 10.10.10.70:1041 -> 10.10.10.10:4445 544 0102b86a 00000000
1549 17:41:25,360537 S 10.10.10.70:1041 -> 10.10.10.10:4445 545 0102b86a 00000000
1551 17:41:25,361178 S 10.10.10.70:1041 -> 10.10.10.10:4445 546 0106f45b 00000000
1553 17:41:25,798024 S 10.10.10.70:1041 -> 10.10.10.10:4445 547 0106f45b 00000000
1555 17:41:26,344909 S 10.10.10.70:1041 -> 10.10.10.10:4445 548 0106f45b 00000000
1557 17:41:26,345542 S 10.10.10.70:1041 -> 10.10.10.10:4445 549 010b3308 00000000
1559 17:41:26,782405 S 10.10.10.70:1041 -> 10.10.10.10:4445 550 010b3308 00000000
[connection closed in here]
1566 17:41:27,329294 S 10.10.10.70:1041 -> 10.10.10.10:4445 553 010b3308 00000000
1568 17:41:34,189450 S 10.10.10.70:1042 -> 10.10.10.10:4445 554 7c016356 00000000
1570 17:41:34,657404 S 10.10.10.70:1042 -> 10.10.10.10:4445 555 7c016356 00000000
1572 17:41:35,094902 S 10.10.10.70:1042 -> 10.10.10.10:4445 556 7c016356 00000000
1574 17:41:35,095570 S 10.10.10.70:1042 -> 10.10.10.10:4445 557 7c058b67 00000000
1576 17:41:35,641778 S 10.10.10.70:1042 -> 10.10.10.10:4445 558 7c058b67 00000000
1578 17:41:36,188636 S 10.10.10.70:1042 -> 10.10.10.10:4445 559 7c058b67 00000000
1580 17:41:36,189278 S 10.10.10.70:1042 -> 10.10.10.10:4445 560 7c0a5039 00000000
1582 17:41:36,735524 S 10.10.10.70:1042 -> 10.10.10.10:4445 561 7c0a5039 00000000
1584 17:41:37,282377 S 10.10.10.70:1042 -> 10.10.10.10:4445 562 7c0a5039 00000000
1586 17:41:37,283051 S 10.10.10.70:1042 -> 10.10.10.10:4445 563 7c0f0451 00000000
1588 17:41:37,829280 S 10.10.10.70:1042 -> 10.10.10.10:4445 564 7c0f0451 00000000
1590 17:41:38,376531 S 10.10.10.70:1042 -> 10.10.10.10:4445 565 7c0f0451 00000000
1592 17:41:38,376795 S 10.10.10.70:1042 -> 10.10.10.10:4445 566 7c13ac51 00000000
1594 17:41:38,813646 S 10.10.10.70:1042 -> 10.10.10.10:4445 567 7c13ac51 00000000
1596 17:41:39,360524 S 10.10.10.70:1042 -> 10.10.10.10:4445 568 7c13ac51 00000000
1598 17:41:46,149972 S 10.10.10.70:1043 -> 10.10.10.10:4445 569 2be336bc 00000000

This pattern repeats itself, both before and after port 4444 is closed, but as you can see every 15 packets it changes the source port it is sending from (question #7.3). Every packet it changes the IPID (question #7.2) and every 3 packets it changes the TCP Sequence Number (question #7.1).

This pattern repeats itself 2 more full times before the end of the capture from this point. It completes a total of 8 iterations over the packet capture.

"contest6.pl -r evidence06.pcap -o ipinfo" could also be used to determine the info above. Both programs have their useful depending on what you are trying to determine.

----

Now that we've looked at the simple stuff, lets feed the file to NetworkMiner and SplitCap. Erik Hjelmvik recently released verison 0.92 of NetworkMiner specifically for this contest, he added the ability for it to pick up better on conversations already in progress in the packet capture (no 3 way handshake), such as packets #1-12, which in the past would have been skipped. So thanks Erik! I'm actually still running the beta version of it, but I believe the full blown version has since been released of 0.92.

Using NetworkMiner we can extract the php file. Looking over that file and cleaning up code for readability we get this, plus a lot more, but for question #2 we see what it shoves in the data part of the array:

...
var c1 = "COMMENT";
var Array1 = new Array();
for (i = 0; i < 1300; i++)
{
Array1[i] = document.createElement(c1);
Array1[i].data = "vEI";
}

...

NetworkMiner also recovers a .gif file. After running through it we can go look at that file and get the MD5 off of the 1 by 1 pixel file: DF3E567D6F16D040326C7A0EA29A4F41 (question #3)

Unfortunetly NetworkMiner doesn't help us with the non-standard traffic on port 4444 or 4445, at least not in the current version, maybe a future release, we can hope at least.

So lets move over to another program by Erik Hjelmvik, SplitCap. It works nicely for this, though its one short coming in my opinion, like a lot of the scripts written that we've seen in the past, is that it doesn't take into account resent packets, out of order, etc like NetworkMiner does. Maybe that will be fixed in a future release, again we can hope.

Anyway, we'll run splitcap with:
"splitcap -r evidence06.pcap -s flow -y L7"

We get a directory created called evidence06, with ~21 files in there. This is the TCP or UDP "data" after the respective headers for each of the streams. I believe it is the same general information that we could get with tcpflow on linux.

The main ones of interest are:
evidence06.pcap.TCP_10-10-10-10_4444_10-10-10-70_1036.bin
evidence06.pcap.TCP_10-10-10-10_4445_10-10-10-70_1044.bin
evidence06.pcap.TCP_10-10-10-10_8080_10-10-10-70_1035.bin
evidence06.pcap.TCP_10-10-10-70_1035_10-10-10-10_8080.bin
evidence06.pcap.TCP_10-10-10-70_1036_10-10-10-10_4444.bin
evidence06.pcap.TCP_10-10-10-70_1044_10-10-10-10_4445.bin
evidence06.pcap.UDP_10-10-10-70_138_10-10-10-255_138.bin

The fun thing with using this is we can look at files like: evidence06.pcap.TCP_10-10-10-70_1035_10-10-10-10_8080.bin

And see exactly what was requested:
(question #1)
GET /index.php HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Host: 10.10.10.10:8080
Connection: Keep-Alive

(question #3)
GET /index.phpmfKSxSANkeTeNrah.gif HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, */*
Referer: http://10.10.10.10:8080/index.php
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Host: 10.10.10.10:8080
Connection: Keep-Alive

The 2 main ones we are interested in are the port 4444 and 4445 files that splitcap provided us. Opening these files up with a hex editor we see an initial 4 bytes 00 6a 0b 00, possibly a protocol header or a command to download a file, because immediately after this we see the MZ magic header info. (question #6.1) We'll come back to the initial 4 bytes here in a little bit.

mzcarver.exe was written specifically to look for MZ headers in tcp dump files. We could use foremost to carve this out, but I wanted to do this all on windows, without trying to get cygwin working on the system. mzcarver does have some limitations at this point. See the readme file that accompanies it for that.

We use mzcarver.exe like this:
mzcarver /r evidence06.pcap.TCP_10-10-10-10_4444_10-10-10-70_1036.bin /d
mzcarver /r evidence06.pcap.TCP_10-10-10-10_4445_10-10-10-70_1044.bin /d

The /d option dumps out to disk the pre and post data that surrounded the mz file. At this point since I only carve out PE files, it appends .pe on the extracted file (this file could be a dll, exe, etc).

After running mzcarever we get the following files:
evidence06.pcap.TCP_10-10-10-10_4445_10-10-10-70_1044.bin.post
evidence06.pcap.TCP_10-10-10-10_4445_10-10-10-70_1044.bin.pre
evidence06.pcap.TCP_10-10-10-10_4445_10-10-10-70_1044.bin.pe

evidence06.pcap.TCP_10-10-10-10_4445_10-10-10-70_1044.bin.post
evidence06.pcap.TCP_10-10-10-10_4445_10-10-10-70_1044.bin.pre
evidence06.pcap.TCP_10-10-10-10_4445_10-10-10-70_1044.bin.pe

Checking the files, the first thing we notice is that the .pe files are the same size (748,032). Checking their MD5 sums we get:
port 4444 one B062CB8344CD3E296D8868FBEF289C7C (question #6.2)
port 4445 one B062CB8344CD3E296D8868FBEF289C7C (question #9)

The same thing, so why did it download the same file twice, just on different ports? Or more to the point why did "they" fire up a 2nd reverse TCP meterpreter session? More on this still to come.

Both .pre files have the same data in them "00 6a 0b 00". With a small tweak to put it in the correct byte order we get: 000b 6a00 or in Decimal format 748,032, which is the size of the file that follows. Note: Erik Hjelmvik pointed this out to me in discussions we had on this contest. I had figured out there was the pre/post info already, but hadn't determined if it was a protocol command or what at that time.

Looking at the .post files, we see the first 14 bytes are the same, but after that they start to deviate:
"16 03 00 00 4a 02 00 00 46 03 00 4b d9 48"

Some quick searches in google seem to indicate the first 8-11 bytes there appear to be some type of SSL communication/handshake.

Scanning through the .pe file with a hex editor we can see a lot of Open SSL info. I believe it even tells the version of OpenSSL, 0.9.8k

So between the OpenSSL references in the .pe file and what appears to be some type of SSL handshake in the .post info I'd say they are doing some type of encrypted data traffic after the sending of the file is complete.

In an attempt to find out more about the .pe file I renamed it to .exe and tried executing it on a VM. It comes up as an invalid executable.

---

In an attempt to understand what was going on here I downloaded Metasploit 3.4 and ran the ms10_002_aurora exploit against a few test VMs. I found that my XP SP2 IE6 box (base install of SP2, no patches) did appeared to be vulnerable (image that). I ended up with this lovely screen:
[*] Sending stage (748032 bytes) to 192.168.4.9
[*] Meterpreter session 1 opened (192.168.4.10:4444 -> 192.168.4.9:1149) at 2010-06-23 21:33:55 -0700

As you can see here it sent 748032 bytes, again same size we saw above, so this capture at least appears to have been done with metasploit running on the linux attack machine.

Now the question of why the windows box was constantly trying to connect back to the Linux host on port 4445 and why they started up a 2nd meterpreter session once they did? Exploiting IE appears to work fine with the ms10_002_Aurora exploit, but if/when the user closes IE down, your connection drops. So how to overcome this.... As soon as you get your initial interactive session you upload a new program and launch it. This new program will try to reverse connect back to you as new process. Perhaps setting this up to run as a new service, or just a simple one time run. This way, even if IE gets closed down you get the new connection on port 4445 connected to you and go in over that connection.

I honestly don't know if that is what happened in this case, but, whatever was trying to contact back on port 4445 had to try multiple times and is quite chatty. Leaving a specific signature that would be fairly easy to detect with an IDS. Changing its source port every 15 packets, its seqnum every 3 and its ipid every 1 at least as long as it continues to try to connect to the remost system on port 4445 and fails. Typically you'd want to be listening ahead of time on that port, or perhaps have a more random and longer pause time between packets.

Note: In all fairness I wouldn't have even looked at this from the Metasploit side if Erik hadn't mentioned it in our ongoing conversations, he at least got me started down the Meterpreter road here. This brings up why I like to work on issues like this in a team setting, it gives you someone else to bounce ideas/issues off of!

---

So yes, Metasploit and the ms10_002_aurora exploit makes pretty easy work of compromising a XP box with IE6 still installed on it. This appears to be what happened overall in this case, but there may be other exploits that utilize some of these same things.

Monday, June 21, 2010

Full Disclosure vs Responsible Disclosure

There was an ongoing thread war last week (or the week before) on Full Disclosure vs Responsible disclosure when someone notified MS of a bug and then 4 days later released the info to the public. After being on vacation for the past week or so I now see there is a known exploit in the wild on this.

Over the years I've gone back and forth on the whole FD vs RD argument. Now that I support a few hundred systems I'm normally more on the RD side of things, but when is it the vendors responsibility to at least be forthcoming about information on the issue to people who report issues?

In the above case, I'm not sure 4 days is reasonable to expect MS to fix the issue, and I have no idea what, if anything, they responded to the person who informed them of it. But I got thinking of this again today when I logged into Twitter.

Back at the beginning of May it was reported that if you changed any of your settings in Twitter that your password would be sent in clear text. The original author of the post claimed they notified twitter of it. I know I also did, I figured if more than one person mentioned it it may get past the first line of Helpdesk personnel. Fast forward ~45 days, no response from twitter and the bug still exists.

I decided I'd poke around a bit more on their site, see if I could figure out a better way to contact them. After 10 mins of going in circles, I was back at the same form I'd tried before. They have a place that says "Check Existing Requests" and "View recently solved and closed tickets", but for the life of me, no way to open a new ticket!

Now I at least understand better what happens when we get upset clients, complaining about going round and round in circles and getting nowhere. We all put things in place to try to limit the number of actual calls that come in, hopefully allowing the user to find the answer themselves, but when it so frustrates the person reporting issues, I can see why some resort to FD from the get go.

I still like the idea of RD, but sometimes I have to admit, some things get fixed a lot quicker when an exploit is floating around out there. While this is great for getting things fixed, it still really sucks being the guy on the other end trying to rush a patch out! It also really sucks being the support guy that has to install that patch on 100's of systems!

Friday, June 4, 2010

Forensics contest #5 results

Well NetworkMiner is getting a huge amount of use in these forensic contests these days. By my count 6 of the 10 finalists used it this time around (still reading through all 10 of them, just did a quick term search, so some may have just mentioned it and not actually used it).

Reading through the winners entry, as noted by the contest owners was very well done, it provided a very nice walk through and is well worth the read.

He did his analysis on a windows box also (like I did), but the more I think about it, the more I think we should be looking at doing this on Linux. A lot of it is a comfort level, what tools you have available etc, but if you know you are working with something that is going to be attacking windows, doesn't it make sense to do your analysis on a system you know it can't infect? I went to great lengths to run mine in a sandbox, on a VM I was willing to scrub, and with no outside network, but the more I think about this the more I think doing analysis on the OS that the infection is going to go after is a bad idea.

With that said, I'm still working on contest #6 on a windows system currently because I'm writing a program that will specifically carve exe's out of a tcp data stream, but hey, I'm more comfortable on windows, give me a break!

On the other hand, you don't always know what payload you are going to find, and driveby malware is everywhere, so is any system actually safe these days!