Midnight Research Labs

python treemap module – PyTreeMap
aaron posted in tools on August 12th, 2010
comments:3

As part of a couple different projects I’m working (and *hoping* to release sometime soon), I’ve created/implemented a simple treemap module for python (Code here). There are a few python treemap modules already, but I couldn’t find a simple one with minimal prerequisites and that implements anything other than the “squarified layout”. Since I couldn’t find one when I was looking, I thought it might be worth releasing independently of the other work. Note that the module does not handle the actual graphing, it’s intentionally just the layout calculations (I do have pygame and jython/processing.org test implementations for the graphing, so if you’re interested in those email me).

There are many different layout algorithms for treemap graphs optimized for various features, and they have evolved over the years. This is a good page on the history of treemaps. The algorithm for this module is the “split” layout and was taken from this great paper on treemaps. Chapter 5 covers several different algorithms with their various features and implementation details.

Here’s a sample of how to use the module:

    from PyTreeMap import SimpleTreeMap
    # Arbitrary list of numbers
    items=[1,22,43,5,78,2,5,12,32,1,55,9,12,34,54,11,8,23,42]
    # Give the treemap its coordinates and title
    root = SimpleTreeMap(x=0, y=0, w=100, h=100, title="RootNode")
    # Add each of the items giving it a size or weight equivalent to its value
    for i in items:
        root.addItem(SimpleTreeMap(size=i))

    # Add a couple children to two different nodes
    root[1].addItem(SimpleTreeMap(size=25))
    root[1].addItem(SimpleTreeMap(size=40))
    root[2].addItem(SimpleTreeMap(size=25))
    root[2].addItem(SimpleTreeMap(size=40))

    print " [*] Setup [%s] top-level items to layout" % len(items)
    root.layout()

    # Iterate over treemap nodes and their children nodes
    for i in root:
        print " [*] Coordinates are x [%s] y [%s] w [%s] h [%s]" % i.getCoordinates()
        for j in i:
            print " [*] -- Child Coordinates are x [%s] y [%s] w [%s] h [%s]" % j.getCoordinates()

And this will output the following:

 [*] Setup [19] top-level items to layout
 [*] Laying out now...
 [*] Coordinates are x [0.0] y [0.0] w [1.53579926455] h [14.5017095894]
 [*] Coordinates are x [1.53579926455] y [0.0] w [33.78758382] h [14.5017095894]
 [*] -- Child Coordinates are x [1.53579926455] y [0.0] w [12.9952245462] h [14.5017095894]
 [*] -- Child Coordinates are x [14.5310238107] y [0.0] w [20.7923592739] h [14.5017095894]
 [*] Coordinates are x [0.0] y [14.5017095894] w [31.6438640133] h [30.2644374039]
 [*] -- Child Coordinates are x [0.0] y [14.5017095894] w [12.1707169282] h [30.2644374039]
 [*] -- Child Coordinates are x [12.1707169282] y [14.5017095894] w [19.4731470851] h [30.2644374039]
 [*] Coordinates are x [31.6438640133] y [14.5017095894] w [3.67951907131] h [30.2644374039]
 [*] Coordinates are x [35.3233830846] y [0.0] w [38.8059701493] h [44.7661469933]
 [*] Coordinates are x [74.1293532338] y [0.0] w [9.53129091385] h [4.67338897183]
... (snip)

The previous output shows the x,y,w,h coordinates for each block of the graph including the child nodes. Children can be added arbitrarily deep. You can find the source for the module here.

Just so you can see what a treemap looks like, this is a random screenshot from a small project I’m hoping to release soon (ignore the colors):

Let me know if you have any feedback/improvements/etc.

MRL Hacker Space is no more
aaron posted in Uncategorized on August 11th, 2010
comments:0

I’m sad to report that due to circumstances outside of our control, the hacker space that we started up over a year ago had to close its doors recently. We’ll continue to exist as a group and publish new things, but it won’t be based around a physical space. We have people affiliated with the group from all over now, so I anticipate things starting to look more like the Shmoo group where people can contribute as much or little as they prefer from wherever they are located. I’m looking forward to whatever the next phases bring us, .

(Photo from MRL sign at the space)

Defcon 2010 Review
aaron posted in cons on August 10th, 2010
comments:0

This year’s defcon was a lot of fun, but the overcrowding is a serious flaw which made it very difficult to get into many of the talks (especially the good ones). I spent a fair amount of time socializing with some of the people I only get to see at con-time, but I still managed to see a few pretty good talks. Here are notes on a few of the talks that I went to. Unfortunately though, I lost some of the notes that I took due to a phone issue.

• Pwnies! — This one really belongs in the black-hat wrap-up, but I neglected to cover it there. The pwnies were both fun and interesting as usual. They outlined some of the best and worst of the security industry over the last year. This was one of my favorites (and winner) from the award for most Epic Fail: Internet Explorer 8 was released with built in cross-site scripting filters which, for nearly a year after release, enabled cross-site scripting on otherwise secure sites. Ironic. Epic. Fail. Here is a list of the winners. Also entertaining, You got pwned — The Song.
•  
• DarunGrim3 — This is an open-source bin diff’er with taint analysis. Has an IDA plugin to help visualize changes. He also came up with a new method for automatically comparing patches to try to find the portions relevant to security by applying heuristic scoring for each section of changed code.
•  
• Moon-Bouncer — Interesting talk from a ham radio guy about non-standard communication lines, satellites and how to bounce signals off of the Moon (literally).
•  
• This was an interesting link from Fyodor’s talk on nmap scripting. They did some analysis on the top favicons on the internet and placed them in a zoomable map that scales each favicon’s size according to the number of sites/hosts it was on.
•  
• Ninja Badges and Party — The ninja guys did an amazing job with their badges and party this year. I wasn’t cool enough to score a badge, but @sweetums hooked me up with an invite to the party. The work they put into the badges is pretty impressive, and surpassed the defcon badge by far (but to be fair, the budget was a lot higher). For the party they rented out a newly remodeled hotel right next to the Alexis Park. It was outdoors next to the pool and palm trees which was reminiscent of the old Defcon days. I met a few new cool people here and also caught up with a few people I hadn’t seen in years. Some of the finishing touches on the party were the nintendo-core band MiniBosses and whole bunch of old-school video arcade games.
•  
• spraypal — This was a talk on a couple new tools that replay known attacks into IDS systems for testing. I didn’t actually see this talk, but the tool looks useful. This is about where I lost some of my notes due to phone issues, so my descriptions are more sparse, .
•  
• My Life as a Spyware Developer — This was a surprisingly entertaining look at one persons experiences doing development work for a spyware company.
•  
• Function hooking for OSX and Linux — Not a bad talk on function hooking. One entertaining portion was how he created an evil ruby build with “performance enhancements” to persuade people to run it. This worked by removing all garbage collection, so it actually was faster… at first, .
•  
• Kartograph — These guys had an interesting approach to memory analysis of video game processes through diff’ing and visualizing. They were able to locate specific parts of game (like maps or units) by snapshotting memory several times throughout game-play, and then graphing the results. By playing with graph alignment they could narrow it down and visually represent the game map areas of memory. Once they located these data structures they could manipulate them to do various things like reveal the entire map, or effectively give infinite life to their characters.
•  
• RazorBack — I didn’t see this talk, but this tool was released at defcon, and it sounds pretty interesting.
•  
• Hardware / USB talks — There were several talks on hardware hacking and custom USB dongles for generating keystrokes to compromise a system. One of presented devices had a wireless controller to be able to trigger the payloads remotely at an arbitrary time. Kind of a neat idea, but there was a lot of duplicate content (some of it was derivative work). One of the talks covered the arduino, and mentioned USB driver fuzzing, but didn’t really get into any of the interesting details.
•  

Thanks to everyone that I met or hung out with during and after the conference. I talked to a lot of great people, and I think the social and community aspect to Defcon is what makes it especially worth going to. I hear rumors that next year is going to take place at the Rio, so maybe space/crowding/lines won’t be an issue.

BlackHat 2010 Recap
aaron posted in Uncategorized on August 4th, 2010
comments:10

Here are some of the interesting things that I encountered this year at BlackHat. These are mostly talks that I went to, but there are a few things that I just happened to run across in the course of the conference. Overall it was a good conference and similar to last year. One improvement was that we were able to get our Defcon badges at BlackHat after waiting in a huge line instead of a really really huge line at Defcon.

• I had seen a talk and other information about BitBlaze before, but I mainly went to this talk to see security rockstar Charlie Miller. It ended up being a pretty interesting talk, and covered some of the ways that BitBlaze can help automate binary analysis. Among a lot of other things it has some neat features that allow you to do taint tracking and determine which registers are tainted from controlled input. There was also a white-paper released that has lots of details and examples.
•  
• I saw an interesting talk about a new routing protocol infrastructure attack tool called Loki. It’s written in python (yea), and covers many packet generation and attack modules for Layer 3 protocols, including BGP, LDP, OSPF, VRRP and quite a few others. It takes some previously released tools, adds some new functionality and wraps it in a nice GUI. It has some functional similarities to yersinia, but covers some protocols
  that it doesn’t. The live demos were pretty convincing.

•  
• javasnoop is an neat looking new tool for tampering and interacting with the internals of java applications, including function hooking/tracing, debugging and instruction overwriting, etc. He made a good point in his talk that Java is easy to decompile (jad), but if you need to interact with the software after that, re-building the software is often prohibitive.
•  
• rejava — This came up in the course of the above presentation, and it looks pretty neat as well. It’s another Java decompiler, but this one allows you to interact directly with the byte code, rather than just getting static code dumps.
•  
• psudp — I didn’t see this talk, but the tool sounds interesting. It is a tool for passive network-wide covert communication and covert file exfiltration. The basic gist, it seems, is that it encodes data into unused DNS fields. Source and slides are available.
•  
• Taviso Ormandy and Julien Tinnes talk on kernel exploits was pretty mind-blowing. They walked through several very technical kernel exploits that they’ve worked on in recent history. It’s amazing that these guys have such a firm grasp on kernels in multiple operating systems.
•  
• virt-ice — This was an interesting talk about a virtualization based malware analysis tool. I was slightly more interested before I found out that the tool wasn’t going to be released any time soon though.
•  
• libscizzle — Library for quickly detecting shellcode in a large binary stream.
•  

I was originally going to create just one BlackHat/Defcon post, but it took longer than expected, so I’ll be breaking it into two posts with the Defcon content tomorrow (maybe).