To the authors of Hamster.

I found your project listed on this blog (which is a great read for Linux users)!  I will not rewrite that author’s excellent post, if you’re curious what Hamster is, go read the blog!  This tool is great for anyone obsessed with data and time tracking.  Essentially, it will show you what you do during the day.  For those who are bad at time management (me), then this tool can help increase performance.  But in using it, I found that I wanted something more.  Specifically, I didn’t want to tell it what I was doing.  I mean, I switched from coding my wardriver to blogging five minutes ago and I still haven’t updated.  Even worse, what happens when I’m multitasking between work, programming, army, etc.  I would like this application to update what is happening as I change it, rather than take the five seconds to update my current task.  I know, I’m lazy.

So what do I think should happen?  Well, I’ve commented on the post that I found this program that I would like it to be automated.  Take a ‘picture’ every minute of active processes and then use that data to associate that program with your current activity!  For example, the browser is open (you can even catch the website name) and you have a setting that associates that program/page with an activity, Hamster then updates what you are doing.  You could even go so far as to use what window is in front, the next behind, the next behind, etc to rank the amount of work you are doing with that task.

With my Army project done, my wardriver being worked on at a steady pace, I think this would be a fun program.  Anyone else interested in this?  Let me know through your comments…

Garmin Wardriver – Prototype Released

This project went together smoother than I thought it would.  The source code and application are available on it’s SourceForge page.  I’m still working to create a range for the networks and then use circles as points.  I’m thinking if I factor in strength, I can come up with an approximate (no where near exact) range.  Unfortunately, without the right antenna, it’s not likely to come up with a direction but I can get the entry and exit and create a circle around it from that.  Unfortunately, to convert the minute data to feet will take some fancy math due to variations in minutes with the longitude.  More on that in a future post.

For those wanting to use the application, here are the steps to get it working.

  1. Ensure you have iwlist, gpsbabel and Google Earth installed.
    • sudo apt-get install wireless-tools gpsbabel googleearth-package
  2. Plug in your gps.
  3. Test with ‘sudo gpsbabel -T -i garmin -f /dev/ttyUSB0
  4. If that pulls data, run ‘sudo ./garminwardriver.py
  5. Open your ‘out.kml’ in Google Earth (Start Google Earth->File->Open)

And there you go, data that can be drawn on any wifi card with any GPS device (if you change the code for your device).  Try it out and let me know if it works or what errors you come up with.  This is just a prototype, so expect v1.01 to be out soon with updates.

Update:

I wrote this post and completely forgot to add the screen shots!

apprunning
Gathering Data
googleearth
Google Earth Data

Garmin Wardriver – Part 2

I’m very happy to say, this is a simple project, to get working in its most basic form.  Note, I said it’s most basic form so no hate mail on the fact that this is nothing more than a quick script!  There are plans to actually go deeper, make this open to more than my device, not use already existing programs, etc.  But for the basic concept and starter, I’ve made this program use gpsbabel to get the gps information and iwlist to get the network information.  I had started with pywifi but found this to be a problem because there was an overbound error.  This bug will be reported, it seems it has a problem with converting an integer to a float on a 64bit machine.

So with no futher ado, here is the script:

#!/usr/bin/env python
import os, re, time

# DEATH FUNCTION
def die(msg):
	print ' [*] ERROR: %s' % msg
	exit(1)

# ENSURE WE ARE ROOT
def checkroot():
	if os.getuid() != 0:
		die('Run this program as root.')
	return

# GET NETWORK INFO
def getessid(dev):
	nets = os.popen('iwlist %s scan' % dev).read()
	pat  = re.compile(r'ESSID:".*"')
	mess = re.findall(pat, nets)
	found= []
	for find in mess:
		found.append(find.split('"')[1].split('"')[0])
	return found

# GET GPS INFO
def getposition(dev):
	pos = os.popen('gpsbabel -i garmin,get_posn -f %s' % dev).read()
	pos = pos.splitlines()[1].split(' ')
	return [pos[1],pos[2]]

# RUN IT
def main():
	checkroot()
	found = getessid('eth1')
	pos   = getposition('/dev/ttyUSB0')
	for find in found:
		print 'ESSID: %s @ %s | %s' % (find,pos[0],pos[1])
	exit(0)

if __name__ == '__main__':
	main()

So what you see here is that I’m using os.popen() to call my programs to draw data.  Unfortunately, iwlist is painfully slow, to the point there’d be a five second delay on some slower machines between the call and having the data ready.  This is why I want to develop the application to not be dependent on other programs, but for the time being, this works as proof of concept in the data gathering stage.  It’s now just a matter of adjusting the code to run this constantly and load data to an overlay when entering and leaving an ESSID’s area.

Any comments for improvements?  Don’t forget, you can join this project any time on SourceForge!

Garmin Wardriver

For my birthday, my wife got me a GPS (Garmin Venture HC).  I have been disappointed in it’s integration with Linux, most particularly it’s ability to communicate with gpsd (usb input, not serial).  I had high hopes of doing some wardriving and kismet does not gather any of the data despite it’s configuration.  I can get gpsd to get some cryptic messages from the device but I found that this command: `gpsbabel -T -t garmin -f /dev/ttyUSB0` will generate the current coordinates.  So instead of getting frustrated, I’ve decided to open my first SourceForge project.  You can find it listed as Garmin Wardriver.

The concept seems simple enough, generate location through direct communication with the GPS device or through the gpsbabel command above and then gather a list of local ESSID’s with their strength.  With this data, updated each second, one could generate a location for entering an ESSID’s area, it’s strength as you move through and when out.  Output this to an overlay for a program such as Google Earth and that is a wardriver!  I can see this branching to more advanced uses but for the basic proof of concept, a command line program should suffice.  This will be written in Python and I’ve looked into utilizing the pygarmin and pywifi modules instead of generating information from preexisting programs.

DONE!!!

It is done!  The XFDL Loader is completed and it is functional.  It came down to the wire, with less than an hour left in submission time to complete but it did get done.  Here is a sample XFDL form and a sample CSV data file (RIGHT CLICK BOTH TO SAVE) to use.  The process to use the web application is (each step is a different page):

  1. Load the two sample files by file type.
  2. Set up data to merge to the form.
    • Use the drop down box on the right to select data by the header
    • Write data in the middle column input boxes to have default data for all forms.
  3. It’ll take awhile to generate the forms, be patient.  When completed, there will be a link to download a zipped archive of your forms.

And that’s it.  Be aware, because the project changed from PHP to Python, we had 10 days to generate the web application.  Therefore, there is no styling (it’s not pretty) and it might not work on all forms.  Also, it only selects text inputs.  Check boxes, more options, etc are yet to come, but this was needed to be functional by May 15th, and with only minutes to spare, it was done!

Approaching the final hours!

The project is due tomorrow!  I have a lot of work to do, but it finally appears possible to have this all done.  Thank you to my team member who contributed the code to correct the xml.dom.minidom’s output to be read correctly by PureEdge and Lotus Forms Viewer.  This will enable me to save, although I’m a bit sad that I spent so much time  this week working on that.  At this point, I need to complete the parsing scheme, develop a function to save the data to forms per the CSV file and then send back those files in a zipped archive.  All small steps, and assuming no road blocks, able to get done today.  A quick bit of documentation and packaging and this barely beta, functional web application will be ready for submission.  This may require an all-nighter and it’s down to the wire, but I think it will work.  Keep your eyes on the XFDL Loader as it will be updated through the day!

Frustration…

So I have now made a front page to upload a CSV file and an XFDL file for this XFDL web application. Nothing is more surprising than being a few short steps from having a working prototype to discover you’ve made a grave error. Here’s my mistake. I used python’s xml.dom.minidom module to parse and edit the file in it’s XML form. But when saving it back, it becomes obvious that the encoding causes some minor problems, but the major problem comes in the form for it’s saved state. Where the XFDL file will have a set of tabs, the minidom would correct these to read just which then is not able to be read by Lotus Forms Viewer or PureEdge! I’ll be frantically rewriting my parsing in other modules tonight to find one that is compatible in it’s write method!

PHP’s chunk_split() in python…

The other day I posted some code with PHP method for compressing XFDL files. I was unfamiliar with PHP’s chunk_split function, so I googled it and came up with this reference.  I then wrote the appropriate code in Python which is a quick script like this:

#!/usr/bin/env python

#string chunk_split ( string body [, int $chunklen = 76 [, string $end = "\r\n" ]] )
def chunk_split(body,chunklen=76,end="\r\n"):
	data = ""
	for i in range(0,len(body),chunklen):
		data += body[i:min(i+chunklen,len(body))] + end
	return data

You can see this code in action through this output:

zds@CF55 ~/Desktop
$ python
Python 2.5.2 (r252:60911, Dec  2 2008, 09:26:14)
[GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from chunk import *
>>> test = """Hello, how are you today?  This is a test of my python interpreta
tion of the PHP function, chunk_split().  It should take this one line of writi
ng and split it by 76 characters and add a DOS compatible line end (e.g. \r\n)
by default.  I've done my best to have code as close as possible to the PHP fun
ction.  This was done in an attempt to work out a method of compression XFDL fi
les.  I've found this function to work but still need to figure out a way to ru
n a compatible gzip method to PHP's gzencode from Python."""
>>> chunk_split(test)
"Hello, how are you today?  This is a test of my python interpretation of the\r\
n PHP function, chunk_split().  It should take this one line of writing and s\r\
nplit it by 76 characters and add a DOS compatible line end (e.g. \r\n) by defa\
r\nult.  I've done my best to have code as close as possible to the PHP functio\
r\nn.  This was done in an attempt to work out a method of compression XFDL fil\
r\nes.  I've found this function to work but still need to figure out a way to \
r\nrun a compatible gzip method to PHP's gzencode from Python.\r\n"
>>>

It appears to work!  This still doesn’t fix my compression problems with the XFDL files though.  It would appear that gzencode does the correct compression, now I just need to find a way to do the same compression in Python.  The GZipFile methods do not output the same as gzenecode() in PHP.

XFDL Viewer Update…

As I dig through miles of XML, I realize I have not posted lately. I’ve been a bit busy meeting the May 15th deadline for my project. But here’s a quick update. First, don’t forget to view the update viewer. It’s not pretty, but it works to the point I’ve set it. I’ve changed from PHP to using Python CGI (a language I’m more comfortable with) and I’ve done what took two weeks in two days. Too bad this wasn’t thought of before!! Anyhow, I figured I’d pass along this bit of code from Neil Funk that shows a better method for decompression of an XFDL in Python:

#!/usr/bin/python
from base64 import *
import zlib

# OPEN FILES
inp = open('example.xfdl','rb').read()
out = open('example.xml,'wb')

# TAKE OUT THE MIME DATA
magic = inp.splitlines()[0]
data   = inp.split(magic)

# THE TRICKY PART -- Look to the python manual for explanation.
wbits = 15+32
newData = zlib.decompress(b64decode(data),wbits)

out.write(newData)
out.close()

And drumroll please, for Chris Hutton contributed this PHP code for recompressing an XFDL from XML.  I’m working to translate this to Python, but for the time being, an XML with an XFDL extension works.

// Export XML, gzip, and base64 encode. Append XFDL header information and output to user.
// XML is the raw data to be saved.
$newxml = $xml->asXML();
// chunk_split is new to me!
$output = chunk_split(base64_encode(gzencode($newxml)));
// Add the header!
$output = "application/vnd.xfdl;content-encoding=\"base64-gzip\"\n". $output;

header("Content-disposition: attachment; filename=\"{$filename}\"");
header("Content-type: application/vnd.xfdl");

echo $output;

So thank you all for your contributions!  I’ll try to keep this blog more up-to-date.