Skip to main content

Posts

Showing posts from 2010

Massive speed and memory improvements

Today marks the last day of working on speed / load / memory improvements for Maltego 3.0.3. The rest of the time is dedicated to bug fixes. We think we're at the edge of the 80/20 boundary on this. And it holds true - in roughly 4 weeks we've made MASSIVE improvements on the speed and load performance of Maltego. 3.0.3 is FASSST. The last performance release I tested was blindly fast and yet we haven't looked at things like caching results, link compression and license key caching. So there is room to optimize even more.

I am super excited about 3.0.3. Even though there are no new features, it's something we should have done a long time ago. You'll see the difference straight away....

Transform Tuesday++ : Facebook,SOA,SPF and Shodan integration!

Hey guys,

Transform Tuesday is here again! Sure it might be a Wednesday, but unfortunately the Fremont data center that two of our Linodes are hosted on went down yesterday (http://status.linode.com/2010/11/fremont-power-issues-rfo.html) !

None the less, there is a small possibility that it's Tuesday somewhere in the world, much like the fabled "it's 5 o'clock somewhere in the world".

This installment has a bunch of new transforms and I'm going to show off some that @achillean did integrating both Shodan and Exploit-db! Our transforms show integration with the Facebook graphAPI as well as some new DNS transforms (SPF and SOA).

Lets get to the good stuff!

Facebook GraphAPI:
I'd like to preface this by saying that we are not trying to break the terms of use of Facebook and we think that we completely abide by the principles listed on http://developers.facebook.com/policy/:
Create a great user experienceBuild social and engaging applicationsGive users choice and co…

New infra enum transforms - with sweet example

We're happy to release a couple of simple transforms via the TDS that assist with the foot printing / enumeration of infrastructure. These are:

NetblockToNetblocksEssentially this transform breaks large networks into smaller chunks of networks. This is useful when you have transforms (such as reverse DNS, portscans etc) that only works on class C networks...and you are stuck with a class B.

NetblockToIPsShows every IP within the netblock as a separate IP address entity. Useful when you need to run a transform on an IP address itself, and want to repeat the process over all the IP addresses in the network. An example of this will follow.

WebsitetoDNSNameNStoDNSNameMXtoDNSNameThese transform simply converts the NS,MX or website to a DNS name so that the enumerate numerically transform can work on it. In other words - see the next transform..

enumerateHostNamesNumericallyThis transform will test for the existence of DNS names that end with the same name, but another number. As example - …

Free BuiltWith.com Transforms!

Builtwith.com is a fantastic site for enumerating technologies used on a website, things such as JQuery, Google analytics and additional server information such as the type (Apache/IIS).

For example, if you had to perform a lookup for www.paterva.com/web5/ you will receive a results page (as seen on the left). This page includes that our website uses/is run on:
ApacheUbuntuMod_SSL
JQueryGoogle Analytics

But why Andrew, WHY?
So out the bag this may not seem that exciting, anyone could simply go and have a look at the source of a website and look for keywords relating to the technology, or even look for key directories (Wordpress' /wp-admin/ directory for example).

However imagine you were looking at a large number of websites, in this example gov.za space ( I placed a domain 'gov.za' onto my graph and then ran the "To Website DNS [using Search Engine]" Transform with my slider set to 255):



How would you correlate which technologies were being used with which websites? We…

Transform Tuesdays! Free Maltego Transforms!

Yes, it's not nearly as popular (yet) as Patch Tuesday, but at least it's an alliteration.

We have been working on a bunch of new TDS based transforms that we would like to share with the community in the hope that the community responds with more transforms of their own.

Today we will be releasing two sets of transforms:

BuiltWith.com integration via their fantastic API
Enumerate server side technologies of Websites and URLs. These include things like CMS (Joomla, Wordpress), Server information (Apache, IIS) and other technologies used (Jquery, Youtube, Silverlight, etc)

Various useful infrastructure transforms
Couple of transforms to help with infrastructure enumeration including Netblock to IP addresses and Netblock to Netblocks.

How do I use these Transforms within Maltego?
You will need two things to use any of the upcoming TDS transforms (and any we post in the future).

Maltego Entity Objects file (mtz) with any custom entities that are used for these transforms. NOTE: This is on…

Transform Distribution Server (TDS) is live!

We've been way busy getting the TDS ready - in the last couple of days we've created 4 videos of how the TDS works and what you can achieve with it. The last of the videos uploaded some minutes ago. You can catch them all [here].

The TDS gives everyone the ability to integrate their transforms with Maltego - without having to do the hard work yourself, but with the flexibility of having your own server. Combined with the ability to build and share custom entities we think it is a pretty powerful concept...

You can register on the TDS [here] and the documentation and libraries can be found on our website over [here].

Feel free to register and play around - and do gives us your feedback!

Labels on EWV lives!

Finally, after a bug in the graphing library was solved we have labels on Edge Weighted View! And I must say, its look pretty sexy. The sizing and spacing is also adjusted, so that entities are not so cramped.

InfoPorn

These are some of the things we are currently working on...
NER (Named Entity Recognition) Dynamic graph update in v3 New detailed view New edge weighted view We decided to roll it into a nice little video:




For the video we looked at the term "Chris Roper site:za" (he is a journalist here in ZA), then took that to websites (where his name occurs), then to URLs and finally let loose the NER stuff on every page. It's super cool to see how the entities starts gathering around common people, organizations and locations...well - I love it!

Now it's weekend time!
RT

Free Icons!

So within the entity creation wizard we have decided to start including a whole bunch of our own icons from our libraries. This is primarily so that when people start creating their own entities within Maltego there are some 'stock' icons to use that look great on the graphs!


Step 1 within the entity creation wizard


After clicking on browse you can now either use the stock icons or your own!

Freeze/Unfreeze

So the weekend is over and i figure its probably time i gave you a bit of an update before this becomes RT's Maltego Blog!

I often get frustrated at graphs moving around the whole time when the nodes are coming in and also the 'crunch time' required to rework the layout every time 1 node comes in when i have say another 100 still yet to arrive.

So bring in Freeze/Unfreeze! A nice little button near the view buttons where you can pause the graph layout, so it basically just holds your graph as is ( Note the button greyed out next to the freeze button ).




When new entities are ready to come in you can then the previously greyed out button becomes a green 'refresh' icon as you can see below.



So essentially now instead of watching the entire graph update i can wait until all the transforms are completed, and click the 'Update graph' button and have all my entities displayed out in one go:



Also, those new icons are looking awesome!

-AM

We won.

Seems the old version of Maltego is still kicking butt..this makes me happy.

Detail View - A powerful change

Yet another one of the huge changes made in 3 is the detail view, its now super useful in terms of being used during analysis. Previously selecting multiple entities merely showed you a list of these entities:



Now however it offers a much wider view, allowing you to sort by name, weight, incoming and outgoing links, as well being able to search/select these entities and have them selected on the graph, as seen below:



One of the other nice features is being able to go back to the graph by double clicking on one of the entities, this will then zoom that specific entity to the center of the graph.

The detail view is also extended for single entities and broken up into 3 different sections:
Relationships - showing your incoming and outgoing connections (also selectable, will change to that specific entity and center it on the graph)Notes - The ability to make notes on an entity (not yet implemented)Generator detail - the standard detail view showing snippets, transforms run to get to this ent…

New Icons

So i wasn't completely happy with the old icons, they smaller sizes seemed to be slightly blurry and we have discussed it and picked a bunch of new ones, which still might change again depending on how well they come out within the app, see some of the choices below :)

List view and so on...

Can't move. Played tennis last night (first time in a while)..and currently pulling the 17 Feb release. It's pushing 18MB. Wish we had more bandwidth. OK it's here - let's see....

Tested new client with some home grown transforms on a local TAS - discovery etc. works well. Lots of changes in List View (a new view we introduced in v3) - it's now almost working like it should.
The new icons still look uncool. Can do with it what we want, they still look crap. We might even go back to the old ones - or perhaps a mix of the two.

Context menus when running transforms now includes 'config transform', 'timing options' and 'help' buttons. I am hoping it's not overwhelming for users. We should still get them to do things.. The CPU problem turned out to be a memory problem. Increasing the memory available helps, but it simply delays the problem - thus...something is leaking. At least we can test further down the release.

Love to beta with this relea…