Tuesday, November 9, 2010

Free BuiltWith.com Transforms!

Builtwith.com is a fantastic site for enumerating technologies used on a website, things such as JQuery, Google analytics and additional server information such as the type (Apache/IIS).

For example, if you had to perform a lookup for www.paterva.com/web5/ you will receive a results page (as seen on the left). This page includes that our website uses/is run on:
  • Apache
  • Ubuntu
  • Mod_SSL
  • JQuery
  • Google Analytics


But why Andrew, WHY?
So out the bag this may not seem that exciting, anyone could simply go and have a look at the source of a website and look for keywords relating to the technology, or even look for key directories (Wordpress' /wp-admin/ directory for example).

However imagine you were looking at a large number of websites, in this example gov.za space ( I placed a domain 'gov.za' onto my graph and then ran the "To Website DNS [using Search Engine]" Transform with my slider set to 255):



How would you correlate which technologies were being used with which websites? Well simple, with Maltego of course (and a bit of code to integrate with the BuiltWith.com API)!

The next step would be simply to select all of the websites found and run the "ToServerTechnologiesWebsite" transform. This will then return the technologies used for each site as seen in this example with just 1 website:


You would then run this across all of your websites to view what kind of technologies were being returned!

Initially we noticed that there was a lot of excess data coming through, technologies such as Javascript, CSS and SEO_H1 which we have since discarded and are only looking at results in the following categories:
"cms","framework","server","Apache Module","Database","Hosting","Interent Communication Server","J2EE Server","Security","Server","Web Accelerator","Web Host","Web Master","Web Server","Web Server Plugin","Web Technology","analytics","javascript"

If there are others you would like to see from the API ( http://api.builtwith.com/ ) feel free to let us know.

So where to from here?
Well now that you have a list of all the technologies used you can see interesting correlation between the data, such as:
  • Are all the websites running the same technology or are there odd cases, and if so why? (Does some dev have a machine running thats accessible from the internet running a vulnerable server)
  • Is the infrastructure mostly Windows or *nix based? (Apache vs IIS)
  • Which CMS' are used, and where? Are they all the same or are there variants?
  • Which websites need their technologies upgraded!
  • What is the most common technology used between all the websites?
Show me the money!

In edge weighted view we can quickly identify the most common technologies used:


And by using the Entity List view I can easily identify the top used technologies by searching for 'BuiltWith' (its the type) and then sorting by incoming links:


Give me the Entity and the Seed!

Entity: http://ctas.paterva.com/TDSTransforms/BuiltWith/BuiltWithTechnology.mtz
SeedURL: https://cetas.paterva.com/TDS/runner/showseed/builtWith

You can follow our previous post on importing these here: http://maltego.blogspot.com/2010/11/transform-tuesdays-free-maltego.html

Last but not least
I'd just like to send out a thank you to Gary Brewer from BuiltWith.com for some changes we requested to do HTTPS as well as HTTP and for helping us out generally with our BuiltWith.com queries!

I'd also like to point out that these transforms run on not only a website but also an IP address as well as URL. Please also note that BuiltWith.com does not currently follow redirects so it will simply try connect to the website, IP address or URL and return the information based on that single page (no spidering).

We look forward to seeing the community respond with more transforms like this!

No comments:

Post a Comment