Tag Archives: XML

Approaching the final hours!

The project is due tomorrow!  I have a lot of work to do, but it finally appears possible to have this all done.  Thank you to my team member who contributed the code to correct the xml.dom.minidom’s output to be read correctly by PureEdge and Lotus Forms Viewer.  This will enable me to save, although I’m a bit sad that I spent so much time  this week working on that.  At this point, I need to complete the parsing scheme, develop a function to save the data to forms per the CSV file and then send back those files in a zipped archive.  All small steps, and assuming no road blocks, able to get done today.  A quick bit of documentation and packaging and this barely beta, functional web application will be ready for submission.  This may require an all-nighter and it’s down to the wire, but I think it will work.  Keep your eyes on the XFDL Loader as it will be updated through the day!

XFDL Viewer Update…

As I dig through miles of XML, I realize I have not posted lately. I’ve been a bit busy meeting the May 15th deadline for my project. But here’s a quick update. First, don’t forget to view the update viewer. It’s not pretty, but it works to the point I’ve set it. I’ve changed from PHP to using Python CGI (a language I’m more comfortable with) and I’ve done what took two weeks in two days. Too bad this wasn’t thought of before!! Anyhow, I figured I’d pass along this bit of code from Neil Funk that shows a better method for decompression of an XFDL in Python:

#!/usr/bin/python
from base64 import *
import zlib

# OPEN FILES
inp = open('example.xfdl','rb').read()
out = open('example.xml,'wb')

# TAKE OUT THE MIME DATA
magic = inp.splitlines()[0]
data   = inp.split(magic)

# THE TRICKY PART -- Look to the python manual for explanation.
wbits = 15+32
newData = zlib.decompress(b64decode(data),wbits)

out.write(newData)
out.close()

And drumroll please, for Chris Hutton contributed this PHP code for recompressing an XFDL from XML.  I’m working to translate this to Python, but for the time being, an XML with an XFDL extension works.

// Export XML, gzip, and base64 encode. Append XFDL header information and output to user.
// XML is the raw data to be saved.
$newxml = $xml->asXML();
// chunk_split is new to me!
$output = chunk_split(base64_encode(gzencode($newxml)));
// Add the header!
$output = "application/vnd.xfdl;content-encoding=\"base64-gzip\"\n". $output;

header("Content-disposition: attachment; filename=\"{$filename}\"");
header("Content-type: application/vnd.xfdl");

echo $output;

So thank you all for your contributions!  I’ll try to keep this blog more up-to-date.

XFDL Viewer – Good and Bad.

I used to think of myself as a decent coder, but this XFDL project is a monster.  Particularly with the parsing schemes.  Let alone the May 15th suspense on the project!!  Oh well, if it doesn’t get done, I’m still working on it…  So I broke apart the XML and just to give a taste of the parsing for this, here’s my notes:

Mind you, I’m skipping the globals but here what needs to be parsed from there:

  • Title
  • Version
  • Fontinfo
  • Bgcolor
  • Print Settings
  • Bindings

This first block is the “field” form.  This is the main part to parse because this is where all values except checks and signatures will go.

<field sid="TO"> --> sid == div id
	<itemlocation>
		<ae>
			<ae>absolute</ae> -- style='position:absolute'
			<ae>y</ae>
			<ae>x</ae>
		</ae>
		<ae>
			<ae>extent</ae> -- dimensions
			<ae>l</ae>
			<ae>w</ae>
		</ae>
		<value>THIS IS WHAT NEEDS EDITTING!</value> - input, no border
		<broderwidth>0</broderwidth> --> style
		<fontinfo> --> style
			<ae>type</ae>
			<ae>size</ae>
			<ae>attribute</ae>
		</fontinfo>
		<justify>center</justify> --> style
		<scrollhoriz>wordwrap</scrollhoriz>
		<scrollvert>fixed</scrollvert>
		<next>TO</next> -> form taborder
		<previous>DATE_A</previous> -> form taborder
		<acclabel>lorum ipsum</acclabel>
		<format>
			<ae>string</ae> --> set form input type
			<ae>optional</ae> --> validation check
			<length>
				<ae>0</ae> vert
				<ae>18</ae> horiz
			</length>
		</format>
	</itemlocation>
</field>

Next part, check boxes.  Nothing too scary here.  Much the same as above.

<check sid=""> same
	<itemlocation></itemlocation> same
	<value>on|off</value>
	<fontinfo></fontinfo> same
	<next></next> same
	<previous></previous> same
	<acclabel></acclabel> same
</check>

Here we get complicated.  Nothing bad about the labels, except there form uses labels for definitions at the end and you’ll see it later where it is reused for a new purpose.  This could pose a good challenge!

<label sid=""> same
	<itemlocation></itemlocation> same
	<value>TEXT</value>
	<linespacing>1</linespacing> OPITONAL
	<fontinfo></fontinfo> same
	<fontcolor>black</fontcolor>
	<format></format> same
</label>

Lines, not too bad… will just be a simple parsing the xml to putting them on the page.

<line sid=""> same
	<itemlocation></itemlocation>same
</line>

Buttons!  These would be easy but they will associate with signatures.

<button sid=""> same
	<itemlocation></itemlocation> same
	<value compute="">FROM CERT</value>
	<type>signature|</type>
	<vfd_signmode>custom</vfd_signmode>
	<printvisible compute="">on|off</printvisible>
	<signformat>???WTF???</signformat>
	<signature>FOR BINDINGS</signature>
	<signer>FROM CERT</signer>
	<custom:onClick>function</custom:onClick>
	<signoptions>
		<ae>omit</ae>
		<ae>triggeritem</ae>
		<ae>coordinates</ae>
		<ae>ufv_settings</ae>
	</signoptions>
	<vfd_group>??</vfd_group>
	<format></format> same
	<previous></previous> same
	<next></next> same
	<image>FROM CERT?</image>
	<signatureimage>FROM CERT?</signatureimage>
	<signitemrefs>
		<ae>LOCK BINDS</ae> -- USE FOR VALIDATION
	</signitemrefs>
</button>

Not even touching signatures until I get the parsing scheme in place!

<signature>
	WILL DO LATER
</signature>

Data will be easy enough, just needs to be decoded and displayed.  Whew… may need to save the files to the server to display but I think HTML can handle an image from data.

<data sid=""> relates to signatures & images
	<filename></filename> -- image (optional)
	<mimedata encoding="base64-gzip"></mimedata>
</data>

Do I really need the toolbar if I’m not using it’s functions?

<toolbar sid="TOOLBAR">
	<bgcolor>
		<ae>gray60</ae>
	</bgcolor>
</toolbar>

Is IBM so dense they can’t find a new name for the toolbar tags instead of reusing label?

<label sid="TOP">
	<itemlocation>
		<ae>
			<ae>within</ae>
			<ae>TOOLBAR</ae>
		</ae>
		<ae>
			<ae>absolute</ae>
			<ae>0</ae>
			<ae>0</ae>
		</ae>
	</itemlocation>
	<image>PAGE1.ArmyLogoTop</image>
	<imagemode>clip</imagemode> -- style
	<active>off</active>
</label>

I won’t complain about this one!  Seems pretty simple, just like lines…

<spacer sid=""> same
	<itemlocation></itemlocation> same
</spacer>

So keep watching the blog… I’ll post more when I figure out how to handle all this data.  And this all needs to be cycled per page to output to the browser… yuck.

XFDL Viewer

The scope of the “Apps 4 the Army” competition that I am making my XFDL Viewer for is to use web apps.  So I did a quick change to PHP and read up and got a framework setup.  The idea is to be able to import the XFDL file and then parse the resulting XML to input values, either on a single form or by batch through an uploaded database.

The framework is complete and I’m pretty happy with the results as I’ve done much more application programming than web development, but PHP isn’t too bad.  After getting the frame work together, I started to work on parsing.  I mistakenly uploaded a XFDL file rather than the decompressed XML and I found something remarkable.  PHP’s SimpleXML apparently can decompress the XFDL file!! This takes out a very complicated step of my project!!!  I’m still working to figure out why the recompression does not produce the same output but that may not be a problem with the web interface.  Stay tuned for more updates.

XFDL in Linux :: Part 2

More for the ongoing series of producing an XFDL viewer in Linux.  In the previous tutorial, we decompressed an XFDL file, although I have had trouble recompressing the file.  It turns out that I need to do some experimentation and find the exact compression method used in gzip to be able to make the form readable.  That will be for the next update though.  I thought I would give a short preview of what’s next on this.

An XFDL file is an XML (xform) by IBM meant to run through their interpreter. IBM has some great documentation on this format.  PureEdge works much like a browser does to decompress the file by Mime-type and to then parse and read the file, including embedded binaries (for pictures, files, etc) and embedded coding (custom functions).  My interpreter will have a long ways to go so I’ll be happy to just be able to place my values in the correct fields.  I’m re-reading XML parsing within Python to make this an easy function, so be patient on that part.  But for those eager to see what I’m talking about, I’ve pasted a small section of XML from a decompressed XFDL.

      <field sid="NAME">
         <itemlocation>
            <ae>
               <ae>absolute</ae>
               <ae>9</ae>
               <ae>448</ae>
            </ae>
            <ae>
               <ae>extent</ae>
               <ae>461</ae>
               <ae>24</ae>
            </ae>
         </itemlocation>
         <value></value>
         <borderwidth>0</borderwidth>
         <scrollhoriz>wordwrap</scrollhoriz>
         <scrollvert>fixed</scrollvert>
         <fontinfo>
            <ae>Times New Roman</ae>
            <ae>10</ae>
            <ae>plain</ae>
         </fontinfo>
         <format>
            <ae>string</ae>
            <ae>optional</ae>
         </format>
         <previous>NAME</previous>
         <next>GRADE</next>
         <acclabel>d ay form 46 44-r, december 19 82.
ay p d. p e version 1.00.
edition of 1 august 19 77 is obsolete.
army reserve reenlistment data.
for use of this form, see ay r 1 40-1 11, the proponent agency is r c p ay c.
item 1. enter name using last name comma first name comma middle initial format.</acclabel>
      </field>
As you can see, there is a <value> tag for these nodes.  For my next post, I’ll write some python code to break this xml to an object that can print the label and insert a value into the xml.  There is a lot of work to interpret the embedded items, code and other tags, but this will be a start!

XFDL in Linux :: Part 1

Earlier, I wrote about using PureEdge Viewer, which is Windows software from IBM on Linux through Wine. This got me thinking, do we need to use Windows software. A quick look at the file and through Google and it’s easy to see that an *.xfdl file is a gzipped, base64 encoded xml file. So this is part one of what I hope to be a tutorial into designing software on Linux using python to open, read and write xfdl files in the same way as PureEdge Viewer. It gets annoying, to say the least, to need to open Windows in VirtualBox, or an actual install to read and edit *.xfdl files. The barriers I see at this preliminary point is to convert the xml to a readable image and then to make that editable where possible.

So, in this first part, we will do the very basic converting an *.xfdl file to an *.xml file. The code should be self explanatory but if there are questions, post them through comments and I’ll do my best at getting an answer to you.

#!/usr/bin/python
""" IMPORTS """
from base64 import *
import gzip, os, sys

""" DEATH!!! """
# Standard way to die...
def die(msg=None):
    if msg == None:
        msg = "Unknown error."
    print " [*] ERROR - %s" % msg
    sys.exit(1)

""" CHECK FOR FILE """
# No file name, then we have nothing to do!
if len(sys.argv) < 2:
    die("Did not specifiy a file name.")

""" GET FILE """
# In a more advanced version, this will check the magic value of the file as well to
# ensure it is an *.xfdl file.
filename = sys.argv[1]
print "Using %s" % filename

""" OPEN FILE AND SPLIT """
# Nothing tough, grab the magic number (1st line) and then store the rest as a variable.
f = open(filename,'r').read()
magic = f.splitlines()[0]
print "magic: %s" % magic
data = f.split(magic)[1]
print "Got Data."
del f

""" BASE64 DECODE """
# First we decode the base64.
f = open('temp.gz','wb')
f.write(b64decode(data))
f.close()
print "Base64 Decoded."

""" GUNZIP DATA """
# Yes, I know this writing to a file and then deleting it is ugly but I have not found
# a way to gunzip from a data stream.
f = gzip.open('temp.gz','rb')
gunzip_data = f.read()
f.close()
os.remove('temp.gz')
print "Gunzipped Data."

""" SAVE XML FILE """
# As this gets more advanced, it should be able to stay as a data stream for editing.
filename = filename.split(".")[0] + ".xml"
f = open(filename,'wb')
f.write(gunzip_data)
f.close()
print "Saved to temporary file '%s'" % filename

Nothing too involved here, simply open the file, strip out the first line and the decode the rest from base64 and gunzip that data to get the xml inside.  In the next tutorial, we’ll look at the structure of that xml, once I actually understand it or find decent documentation on it!

Gnome Applets.

Being an avid Ubuntu user, I wanted to tinker in building for the Gnome interface.  An applet was the perfect idea but a tutorial on the internet was hard to find.  I found a few like these that are in C, which I’m fond of C but for rapid development and tinkering, Python works much better.  Besides, that is my primary programming language and kindly, Ubuntu came with all the required modules to build in this way.  Most tutorials for Python were out of date.  One of the best I found in Python was this one and my personal favorite which shows a good use of classes.

Onto the tutorial!

The first thing you need to do is build a server file.  This will go to /usr/lib/bonobo/servers with root access.  The reason for this file is for Gnome to be able find your file and set it as an applet.   The lines to pay attention to are:

Line 4:          This is the full path to your source code file.
Lines 9-10:  Name and describe your applet.
Line 20-23: These are the options you can set for the applet browser.

<oaf_info>
    <oaf_server iid="OAFIID:SampleApplet_Factory"
        type="exe"
        location="/home/zbert/Desktop/testing/py/applet.py">
        <oaf_attribute name="repo_ids" type="stringv">
            <item value="IDL:Bonobo/GenericFactory:1.0"/>
            <item value="IDL:Bonobo/Unknown:1.0"/>
        </oaf_attribute>
        <oaf_attribute name="name" type="string" value="Sample Applet Factory"/>
        <oaf_attribute name="description" type="string" value="Sample Applet's Factory that launches the applet"/>
    </oaf_server>
    <oaf_server iid="OAFIID:SampleApplet"
        type="factory"
        location="OAFIID:SampleApplet_Factory">
        <oaf_attribute name="repo_ids" type="stringv">
            <item value="IDL:GNOME/Vertigo/PanelAppletShell:1.0"/>
            <item value="IDL:Bonobo/Control:1.0"/>
            <item value="IDL:Bonobo/Unknown:1.0"/>
        </oaf_attribute>
        <oaf_attribute name="name" type="string" value="Sample Applet"/>
        <oaf_attribute name="description" type="string" value="Sample applet's description."/>
        <oaf_attribute name="panel:category" type="string" value="Utility"/>
        <oaf_attribute name="panel:icon" type="string" value="gnome-laptop.png"/>
    </oaf_server>
</oaf_info>

Now with the server file in place, it’s time to build the applet. I used a picture to show my applet as a small 16×16 PNG smiley face.   As for the coding, I’ve commented what needs to be done here.

#!/usr/bin/env python
### NORMAL IMPORTS
import sys
import gtk
import pygtk
import gnomeapplet

pygtk.require('2.0')

### CREATE A MENU WITH XML
def create_menu(applet):
    xml = """
    <popup name="button3">
        <menuitem name="Item 1" verb="Networks" label="_Networks...."
            pixtype = "stock" pixname="gtk-properties"/>
        <menuitem name="Item 2" verb="Help" label="_Help"
            pixtype = "stock" pixname="gtk-help"/>
        <separator/>
        <menuitem name="Item 3" verb="About" label="_About..."
            pixtype = "stock" pixname="gnome-stock-about"/>
    </popup>
    """
    verbs = [('Networks',show_networks), ('Help',show_help), ('About',show_about)]
    applet.setup_menu(xml, verbs, None)

### WHAT HAPPENS WHEN MENU ITEM IS CLICKED
def show_about(*arguments):
    print(arguments)

### THE ASTERIK ALLOWS MULTIPLE ARGUMENTS TO BE PASSED
def show_networks(*arguments):
    print(arguments)

### ALL OF THESE SHOW IN THE DEBUG (-d) OPTION
def show_help(*arguments):
    print(arguments)

### WHERE IT ALL HAPPENS
def applet_factory(applet, iid):
    # CREATE THE XML MENU
    create_menu(applet)
    # CREATE AN IMAGE OBJECT
    im = gtk.Image()
    # SET THE FILE FOR THE IMAGE
    im.set_from_file("/home/zbert/Desktop/testing/py/face2.png")
    # ENABLE TRANSPARENCY
    applet.set_background_widget(applet)
    # CREATE AN EVENT FOR CLICKING THE IMAGE
    applet.connect('button-press-event',button_press)
    # ADD IT ALL TO THE APPLET
    applet.add(im)
    # SHOW IT!
    applet.show_all()
    # DEBUG EVENT
    print('Factory started')
    return True

### CREATE A DIALOG TO POP UP WHEN APPLET IS CLICKED
def test(*arguments):
    print(arguments)
    dia = gtk.Dialog("Message",None,gtk.DIALOG_MODAL)
    lbl = gtk.Label("this is a message")
    dia.vbox.pack_start(lbl)
    lbl.show()
    dia.run()
    dia.connect("destroy",dia.destroy)

### HANDLE THE IMAGE CLICK
def button_press(button, event):
    # LEFT BUTTON ACTIVATES THE CUSTOM MENU
    if event.button == 1:
        print "button 1"
        # CREATE A CUSTOM MENU
        m = gtk.Menu()
        i = gtk.MenuItem("Hello")
        i.show()
        # CONNECT THIS MENU ITEM TO THE DIALOG
        i.connect("activate",test,"Hello")
        m.append(i)
        # FINISH THE POPUP MENU
        m.popup(None, None, None, event.button, event.time, None)
    # RIGHT BUTTON ACTIVATES THE STANDARD MENU
    elif event.button == 2:
        # DEBUG
        print "button 2"

### STANDARD ENTRY
if __name__ == '__main__':
    # DEBUG
    print('Starting factory')
    # RUN THE APPLET AS A GTK WINDOW IN DEBUG (-d) MODE
    if len(sys.argv) > 1 and sys.argv[1] == '-d': #
        mainWindow = gtk.Window()
        mainWindow.set_title('Applet window')
        mainWindow.connect('destroy',gtk.main_quit)
        applet = gnomeapplet.Applet()
        applet_factory(applet, None)
        applet.reparent(mainWindow)
        mainWindow.show_all()
        gtk.main()
        sys.exit()
    # LET GNOME TAKE CARE OF IT AS AN APPLET
    else:
        gnomeapplet.bonobo_factory('OAFIID:SampleApplet_Factory',
            gnomeapplet.Applet.__gtype__,
            'Sample Applet', '0.1',
            applet_factory)

You can see in these pictures how the applet sits like any other in the corner.  A left click will bring up our custom coded menu with one selection, ‘Hello’ which when clicked brings up a GTK Dialog that is made in the button_press() event.  If you right click, you get the standard menu, plus our XML menu which, when in Debug (-d) mode, you will see the arguments sent by a click outputted in your terminal.  As I mentitoned, there are a handful of great resources for this that detail current Python modules with information on setting this up as I did or in a class for bigger projects.  I hope this tutorial has been helpful, and let me know what you think of these lessons with the comments interface!