JavaScript XMLHttpRequest Wrapper

Here is my version of the XMLHttpRequest Wrapper, Open Source, and available on Google Code. Enjoy.

Here is a simple example of a http request with the lib:

new fiji.xhr('get', 'echo.php?param=value', function(xhr) {
   if (this.readyState == 4 && this.status == 200) {
      alert(this.responseText);
   }
}).send();

The readystate callback handler executes in the scope of the XHR Instance. So you can use the this keyword to reference the XHR object instance. This behavior is the same as the specifications for the callback scope in the W3C XMLHttpRequest Specs. You can also receive a reference to the XHR library instance which is passed as the first parameter to the callback. In the case above it would be xhr. This allows you to attach further references or objects to the XHR library instance that would be persisted for the duration of the XHR call. For example, a request ID.

XSS (Cross Site Scripting) and stealing passwords

XSS (Cross Site Scripting) would be viewed by most web developers as the stealing of users session cookies by injecting JavaScript into a web page through URL. You do not associate it with stealing passwords, but worse then stealing session cookies, it can steal a users username and password directly from the browser.

Many users choose to have the browser remember their login credentials. So when ever they visit a login form, their username and password fields are pre-populated by the browser. Now if there is an XSS vulnerability on that login page, then a remote attacker can successfully retrieve the users username and password.

Hello World in XSS

You have a page that has an XSS vulnerability. Let say a website has a PHP page, mypage.php with the code:

<?php

// the variable is returned raw to the browser
echo $_GET['name'];

?>

Because the variable $_GET['name'] is not encoded into HTML entities, or stripped of HTML, it has an XSS vulnerability. Now all an attacker has to do is create a URL that a victim will click, that exploits the vulnerability.

mypage.php?name=%3Cscript%3Ealert(document.cookie);%3C/script%3E

This basically will make PHP write <script>alert(document.cookie);</script> onto the page, which displays a modal dialog with the value of the saved cookies for that domain.

How Does stealing passwords with XSS work?

The example above displays the cookies on the domain the webpage is on. Now imagine the same page has a login form, and the user chose to have their passwords remembered by the browser. Lets say the PHP page looks like this:

<?php

// the variable is returned raw to the browser
echo $_GET['name'];

?>

<form action="login.php">
<input type="text" name="username" />
<input type="password" name="password" />
<input type="submit" value="Login" />
</form>

Now an attacker just needs to craft a URL that retrieves the username and password. Here is an example that retrieves the password:

mypage.php?name=%3Cscript%3Ewindow.onload=function(){alert(document.forms[0].password);}%3C/script%3E

As you can see, it is just a normal XSS exploit, except it is applied to the username and password populated by the browser after the window.onload event.

Password stealing XSS vs Session Cookie stealing XSS

Well, they are both suck from a developers perspective. According to Wikipedia, 70% or so of websites are vulnerable to XSS attacks.

As a developer, I've always thought of XSS as an exploit on a users session, just as CSRF/XSRF (Cross Site Request Forgery), which requires an active session. Now, as you can see, XSS of the type described does NOT require an active session. The user does not have to be logged into the site. They could have logged out 10 years ago, but as long as the browser remembers their login credentials, the XSS exploit can steal those login credentials.

Due to its ability to be executed without having the user logged into a website, this exploit should be regarded worse then session based XSS.

Proof of Concept

Fill in the form below with dummy values and click the "Login" button.

Login Form

Username:
Password:

Now return to the same page, to simulate logging out. Now click the Exploit. This will simulate an XSS exploit on this page, and alert the saved password.

I've set up a proof of concept based on an actual XSS exploit here: http://xss-password.appjet.net/.

Preventing Stealing Passwords via XSS

The only way I can think of right now is to give your username and password fields unique names so that the browser does not remember their values. In PHP you can do this with the time() function. eg:

<input type="password" name="pass[<?php echo sha1(time().rand().'secret'); ?>]" />

The unique names prevents the browser from remembering the password field. This should work universally in all browsers.

Chaining Functions in JavaScript

A lil snippet I borrowed from the Google AJAX libraries API:

chain = function(args) {
    return function() {
     for(var i = 0; i < args.length; i++) {
      args[i]();
     }
    }
   };

You're probably familiar with the multiple window onload method written by Simon Willison which creates a closure of functions within functions in order to chain them.
Here is the same functionality using the chain function defined above.

function addLoad(fn) {
    window.onload = typeof(window.onload) != 'function' ? fn : chain([onload, fn]);
};

Chain can also be namespaced to the Function.prototype if thats how you like your JS.

Function.prototype.chain = function(args) {
    args.push(this);
    return function() {
     for(var i = 0; i < args.length; i++) {
      args[i]();
     }
    }
   };

So the multiple window onload function would be:

window.addLoad = function(fn) {
    window.onload = typeof(window.onload) != 'function' 
    ? fn : window.onload.chain([fn]);
   };

Hacking Google Loader and the AJAX Libraries API

Google recently released the AJAX Libraries API which allows you to load the popular JavaScript libraries from Google Servers. The benefits of this outlined in the description of the API.

The AJAX Libraries API is a content distribution network and loading architecture for the most popular open source JavaScript libraries. By using the google.load() method, your application has high speed, globally available access to a growing list of the most popular JavaScript open source libraries.

I was thinking of using it for a current project that would use JS heavily, however, since the project used a CMS (Joomla) the main concern for me was really how many times MooTools would be loaded. Joomla uses a PHP based plugin system (which registers observers of events triggered during Joomla code execution) and the loading of JavaScript by multiple plugins can be redundant as there is no central way of knowing which JavaScript library has already been loaded, nor is there a central repository for JavaScript libraries within Joomla.

MooTools is the preferred library for Joomla and in some cases it is loaded 2 or even 3 times redundantly. I did not want our extension to add to that mess. To solve the problem I would test for the existence of MooTools, if (typeof(MooTools) == 'undefined') and load it from Google only if it wasn't available. Now this would have worked well, however, I would have to add the JavaScript for AJAX Libraries API and it would only be loading 1 script, "MooTools", when I also had about 3-4 other custom libraries that I wanted loaded.

Now I thought, why don't I develop a JavaScript loader just like the Google AJAX Libraries API Loader. Should be just a simple function to append a Script element to the document head. So I started with:

function loadJS(src) {
    var script = document.createElement('script');
    script.src = src;
    script.type = 'text/javascript';
    timer = setInterval(closure(this, function(script) {
     if (document.getElementsByTagName('head')[0]) {
      clearTimeout(timer);
      document.getElementsByTagName('head')[0].appendChild(script);
     }
    }, [script]), 50);
   }
function closure(obj, fn, params) {
    return function() {
     fn.apply(obj, params);
    };
   }

The function loadJS would try to attach a script element to the document head, each 50 milliseconds until it succeeded.

This works but there is no way of knowing when the JavaScript file was fully loaded. Normally, the way to figure out if a JS file has finished loading from the remote server, is to have the JS file invoke a callback function on the Client JavaScript (aka: JavaScript Remoting). This however means you have to build a callback function into each JavaScript file, which is not what I wanted.

So to fix this problem I though I'd add another Interval with setInterval() to detect when the remote JS file had finished loading by testing a condition that exits when the file has completed. eg: for MooTools it would mean that the Object window.MooTools existed.

So I went about writing a JavaScript library for this, with a somewhat elaborate API, with JS libraries registering their "load condition test" and allowing their remote loading, about 1 wasted hour, (well not wasted if you learn something) only to realize that this wouldn't work for the purpose either. The reason is that it broke the window.onload functionality. Some remote files would load before the window.onload event (cached ones) and others after. This made the JavaScript already written to rely on window.onload fail.

Last Resort, how did Google Do it? I had noted earlier that if you load a JavaScript file with Google's API the file would always load before the window.onload method fired. Here is the simple test: (In the debug output, the google callback always fired first).

google.load("prototype", "1");
   google.load("jquery", "1");
   google.load("mootools", "1");
   google.setOnLoadCallback(function() {
    addLoad(function() {
     debug('google.setOnLoadCallback - window.onload');
    });
    debug('google.setOnLoadCallback')
   });
   addLoad(function() {
    debug('window.onload');
   });
   debug('end scripts');

I had to take a look at the source code for Google's AJAX Libraries API which is: http://www.google.com/jsapi to see how they achieved this.

It never occurred to me that you could force the browser to load your JavaScript before the window.onload event so I was a bit baffled. Browsing through their source code I came upon what I was looking for:

function q(a,b,c){if(c){var d;if(a=="script"){d=document.createElement("script");d.type="text/javascript";d.src=b}else if(a=="css"){d=document.createElement("link");d.type="text/css";d.href=b;d.rel="stylesheet"}var e=document.getElementsByTagName("head")[0];if(!e){e=document.body.parentNode.appendChild(document.createElement("head"))}e.appendChild(d)}else{if(a=="script"){document.write('<script src="'+b+'" type="text/javascript"><\/script>')}else if(a=="css"){document.write('<link href="'+b+'" type="text/css" rel="stylesheet"></link>'
)}}}

The code has been minified, so its a bit hard to read. Basically its the same as any javascript remoting code you'd find on the net, the but the part that jumps out is:

var e=document.getElementsByTagName("head")[0];
if(!e){e=document.body.parentNode.appendChild(document.createElement("head"))}
e.appendChild(d)

Notice how it will create a head Node and append it to the parentNode of the document body if the document head head does not exist yet.

Now that forces the browser to load the JavaScript right then, no matter what. Now following that method you can load remote JavaScript files dynamically and just used the regular old window.onload event or "domready" event and the files will be available.

Apparently this won't work on all browsers, since Google's code also has the alternative:

document.write('<script src="'+b+'" type="text/javascript"><\/script>')

with a bit of testing, you could discern which browsers worked with which and use that. I'd imagine that the latest browsers would accept the dom method and older ones would need the document.write

So my JavaScript file loading function became:

function loadJS(src) {
    var script = document.createElement('script');
    script.src = src;
    script.type = 'text/javascript';
    var head = document.getElementsByTagName('head')[0];
    if (!head) {
     head = document.body.parentNode.appendChild(document.createElement('head'));
    }
    head.appendChild(script);
    
   }

Anyways, I finally got my JavaScript library loader working just as I liked, thanks to the good work done by Google with the AJAX Libraries API.

JavaScript Cross Window Communication via Cookies

Using JavaScript we can communicate between browser windows given that we have a reference to each window. When a new window is created the the JavaScript method window.open() it returns a reference to the new window. The child window, also has a reference to the parent window that created it via the window.opener window object. These references allow the two windows to communicate with and manipulate each other.

There are times however, when we need to communicate with an open window for which there is no window object reference. A typical example of this is when a popup window is created, then the parent window is reloaded. When reloading the parent, all JavaScript objects are "trashed", along with the reference to the open popup window. Here is where cookies come into play - they are not trashed.

The problem with cookies is that it only saves strings, so you can't write a reference to a window object directly to a cookie, since serializing the reference is not possible. However, since both the child window and the parent window are able to read and write cookies, then they have a medium for which they can communicate. Even if the medium only allows strings.

To demonstrate how communicating between windows with cookies would work, lets assume we want to open a window, and then close it a few seconds later.

var win = window.open('child.html');
setTimeout(function() { win.close(); }, 5000);

The code will open a child window, and close it after 5 seconds using the reference to the child window and the method close(). However if we didn't have a reference for some reason, we would not be able to invoke the close method. So lets see how it could be done with cookies:

window.open('child.html');
setTimeout(function() { setCookie('child', 'close'); }, 5000);

Here we open a window but do not save a reference. Then after 5 seconds we write 'close', to the cookie named 'child' (using the pseudo setCookie() function). This does not do anything by itself, but if the child window was expecting the cookie, it could close itself when it read 'close'. Lets assume the following JS is in child.html.

// child.html
setInterval(function() { getCookie('child') == 'close' ? this.close() : ''; }, 500);

This would check the cookie every half a seconds and close the window if the cookie read 'close'.

Using this method we can send any commands to any open windows and have them execute it without having a reference to that window.

Synchronizing Date and Time in different Timezones with PHP, MySQL and JavaScript

How do you show the correct date and time (timestamp) to users in differnet time zones? In theory it should be simple:

• Save your your date and time with the correct timezone
• Retrieve the timezone of the user to whom you will display the date and time to
• Calculate the difference in hours between the saved date and time and the users date and time
• Add the difference in hours to the saved date and time and display

Save your your date and time with the correct timezone

The date and time with timezone is a timestamp. Though implementations differ, timestamps basically contain the same information (date/time and timezone).The timezone can be explicitly recorded in the timestamp (eg: 2005-10-30 T 10:45 UTC) or implicitly taken from the context in which the timestamp was generated or recorded (eg: unix timestamp is dependent on timezone of server generating the timestamp).

Something as simple as saving a timestamp in mysql with PHP can be not so simple due to the difference in the timestamp representation in the two languages.

The PHP timestamp is defined as the number of seconds since the Unix Epoch (January 1 1970 00:00:00 GMT) while the mysql timestamp is a representation of the present date and time in the format YYYY-MM-DD HH:MM:SS.

If you save the timestamp as a mysql timestamp field, then the timestamp is saved as UTC, however, when you access the timestamp it is converted to the timezone set in the mysql server, so basically you don't get to see the stored UTC version of the timestamp. If you save it as a PHP timestamp in a varchar or unsigned int field then it would be subject to the the PHP servers timezone. So in essence both the MySQL and PHP timestamp a dependent on the timezone of their respective servers.

Whichever format you save it in, just remember that both the PHP and MySQL timestamps reference the timezone on the server they are saved on, PHP during generation of the timestamp, and mysql during retrieval.

Retrieve the timezone of the user to whom you will display the date and time to

The easy way to do this is ask the user what their timezone is. You see this used in many registration forms on websites as well as many open source forums, CMSs, blog software etc. However, not every user will even bother giving you the correct timezone.

You can use JavaScript if it is available on the browser:

var tz = (new Date().getTimezoneOffset()/60)*(-1);

This depends on the users computer's clock, so if it is set wrong, then you will get a wrong result. Time is relative however, so if a user wants to be a few hours behind, let them be.

You can use the users IP address to assume their geographic location and thus their timezone. This can be done server side and thus is not dependent on the users browser having JavaScript. Using the IP to determine timezone is dependent on the accuracy of your IP geocoding service you use. Here is an example using the hostip.info API for geocoding and earthtools.org for lat/long conversion to timezone.

<?php
/**
* Retrieves the Timezone by the IP address
* @param String $IP (optional) IP address or remote client IP address is used
*/
function getTimeZoneByIP($IP = false) {

 // timezone
 $timezone = false;

 // users IP
 $IP = $IP ? $IP : $_SERVER['REMOTE_ADDR'];

 // retrieve geocoded data from http://hostip.info/ API in plain text
 if ($geodata = file('http://api.hostip.info/get_html.php?ip='.$IP.'&position=true')) {
  // create an associative array from the data
  $geoarr = array();
  foreach($geodata as $line) {
   list($name, $value) = explode(': ', $line);
   $geoarr[$name] = $value;
  }
  
  // retrieve lat and lon values
  $lat = trim($geoarr['Latitude']);
  $lon = trim($geoarr['Longitude']);
  
  if (strlen($lat) > 0 && strlen($lon) > 0) {
   // pass this lat and long to http://www.earthtools.org/ API to get Timezone Offset in xml
   $tz_xml = file_get_contents('http://www.earthtools.org/timezone-1.1/'.$lat.'/'.$lon);
   // lets parse out the timezone offset from the xml using regex
   if (preg_match("/<offset>([^<]+)<\/offset>/i", $tz_xml, $match)) {
    $timezone = $match[1];
   }
  }

 }
 return $timezone;
}
?>

You can also use a combination of the three in order to correlate the data and get a better guess of the timezone.

Calculate the difference in hours between the saved date and time and the users date and time

Now that we have the timestamp and the users timezone, we just need to adjust the timestamp to their timezone. First we need to calculate the difference between the timezone the timestamp is saved in, as the users timezone.

$user_tz_offset = $tz_user - $tz_timestamp; 

where $user_tz_offset is how far ahead or behind in hours the user timezone is from the timestamps timezone.

Add the difference in hours to the saved date and time and display

Now we have all we need to show the correct time to the user based on their timezone. Example in pseudo code:

$user_tz_offset = $tz_user - $tz_timestamp; 
$users_timestamp = $timestamp + $user_tz_offset;

Converting XML to JSON

Why would I want to convert XML to JSON. Mainly because JSON is a subset of JavaScript (JavaScript Object Notation) and XML isn't. It is much easier to manipulate JavaScript Objects, then it is to manipulate XML. This is because Objects are native to JavaScript, where as XML requires an API, the DOM, which is harder to use. DOM implementations in browsers are not consistent, while you will find Objects and their methods more or less the same across browsers.

Since, most of the content/data available on the web is in XML format and not JSON, converting XML to JSON is necessary.

The main problem is that there is no standard way of converting XML to JSON. So when converting, we have to develop our own rules, or base them on the most widely used conversion rules. Lets see how the big boys do it.

Rules Google GData Uses to convert XML to JSON

A GData service creates a JSON-format feed by converting the XML feed, using the following rules:

Basic

• The feed is represented as a JSON object; each nested element or attribute is represented as a name/value property of the object.
• Attributes are converted to String properties.
• Child elements are converted to Object properties.
• Elements that may appear more than once are converted to Array properties.
• Text values of tags are converted to $t properties.

Namespace

• If an element has a namespace alias, the alias and element are concatenated using "$". For example, ns:element becomes ns$element.

XML

• XML version and encoding attributes are converted to attribute version and encoding of the root element, respectively.

Google GData XML to JSON example

This is a hypothetical example, Google GData only deals with RSS and ATOM feeds.

<?xml version="1.0" encoding="UTF-8"?>
<example:user domain="example.com">
 <name>Joe</name>
 <status online="true">Away</status>
 <idle />
</example:user>

{
 "version": "1.0",
 "encoding": "UTF-8",
 "example$user" : {
  "domain" : "example.com",
   "name" : { "$t" : "Joe" },
   "status" : {
    "online" : "true",
    "$t" : "Away"
   },
   "idle" : null
  }
}

How Google converts XML to JSON is well documented. The main points being that XML node attributes become strings properties, the node data or text becomes $t properties and namespaces are concatenated with $.
http://code.google.com/apis/gdata/json.html#Background

Rules Yahoo Uses to convert XML to JSON

I could not find any documentation on the rules Yahoo uses to convert its XML to JSON in Yahoo Pipes, however, by looking the output of a pipe in RSS format and the corresponding JSON format you can get an idea of the rules used.

Basic

• The feed is represented as a JSON object; each nested element or attribute is represented as a name/value property of the object.
• Attributes are converted to String properties.
• Child elements are converted to Object properties.
• Elements that may appear more than once are converted to Array properties.
• Text values of tags are converted to string properties of the parent node, if the node has no attributes.
• Text values of tags are converted to content properties, if the node has attributes.

Namespace

• Unknown.

XML

• XML version and encoding attributes are removed/ignored - at least in the RSS sample I looked at.

The only problem I see with the rules Yahoo Pipes uses is that if an XML node has an attribute named "content", then it will conflict with the Text value of the node/element giving the programer an unexpected result.

Yahoo Pipes XML to JSON example

<?xml version="1.0" encoding="UTF-8"?>
<example:user domain="example.com">
 <name>Joe</name>
 <status online="true">Away</status>
 <idle />
</example:user>

{
 "example??user" : {
  "domain" : "example.com",
   "name" : "Joe",
   "status" : {
    "online" : "true",
    "content" : "Away",
   },
   "idle" : ??
  }
}

XML.com on rules to convert XML to JSON

The article on XML.com by Stefan Goessner gives a list of possible XML element structures and the corresponding JSON Objects.
http://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html

Pattern	XML	JSON	Access
1	<e/>	"e": null	o.e
2	<e>text</e>	"e": "text"	o.e
3	<e name="value" />	"e":{"@name": "value"}	o.e["@name"]
4	<e name="value">text</e>	"e": { "@name": "value", "#text": "text" }	o.e["@name"] o.e["#text"]
5	<e> <a>text</a> <b>text</b> </e>	"e": { "a": "text", "b": "text" }	o.e.a o.e.b
6	<e> <a>text</a> <a>text</a> </e>	"e": { "a": ["text", "text"] }	o.e.a[0] o.e.a[1]
7	<e> text <a>text</a> </e>	"e": { "#text": "text", "a": "text" }	o.e["#text"] o.e.a

If we translate this to the rules format given by Google it would look something like:

Basic

• The feed is represented as a JSON object; each nested element or attribute is represented as a name/value property of the object.
• Attributes are converted to @attribute properties. (attribute name preceeded by @)
• Child elements are converted to Object properties, if the node has attributes or child nodes.
• Elements that may appear more than once are converted to Array properties.
• Text values of tags are converted to string properties of the parent node, if the node has no attributes or child nodes.
• Text values of tags are converted to #text properties, if the node has attributes or child nodes.

Namespace

• If an element has a namespace alias, the alias and element are concatenated using ":". For example, ns:element becomes ns:element. (ie: namespaced elements are treated as any other element)

XML

• XML version and encoding attributes are not converted.

XML.com XML to JSON example

<?xml version="1.0" encoding="UTF-8"?>
<example:user domain="example.com">
 <name>Joe</name>
 <status online="true">Away</status>
 <idle />
</example:user>

{
 "example:user" : {
  "@attributes" : { "domain" : "example.com" },
   "name" : { "#text" : "Joe" },
   "status" : {
    "@attributes" : {"online" : "true"},
    "#text" : "Away"
   },
   "idle" : null
 }
}

Other rules being used to convert XML to JSON

Here is a blog on the topic of an XML to JSON standard. http://khanderaotech.blogspot.com/2007/03/mapping-between-xml-json-need-standard.html.
A good discussion on the differences between XML and JSON. http://blog.jclark.com/2007/04/xml-and-json.html

We need a standard way of converting XML to JSON

I'm tired of hearing the "XML vs JSON" debate. Why not just make them compatible. Now, that we see just how many different rules are being used, we can definitely see another reason why a standard would come in handy. But till then, I think I'll add to the confusion and come up with my own ruleset.

My rules of converting XML to JSON

My rules are simple and is based on the XML DOM. The DOM represents XML as DOM Objects and Methods. We will use the DOM objects only since JSON does not use methods. So each Element would be an Object, and each text node #text property and attributes an @attributes object with string properties of the attribute names. The only difference from the DOM Objects representation in JavaScript is the @ sign in front of the attributes Object name - this is to to avoid conflicts with elements named "attributes". The DOM goes around this by having public methods to select child nodes, and not public properties (the actual properties are private, and thus not available in an object notation).

Basic

• The feed is represented as a JSON object; each nested element or attribute is represented as a name/value property of the object.
• Attributes are converted to String properties of the @attributes property.
• Child elements are converted to Object properties.
• Elements that may appear more than once are converted to Array properties.
• Text values of tags are converted to $text properties.

Namespace

• Treat as any other element.

XML

• XML version and encoding attributes are not converted.

In order to convert XML to JSON with JavaScript, you first have to convert the XML to a DOM Document (to make things simpler). Any major browser willd do this either automatically in the case of the XML/XHTML Document you are viewing, or an XML document retrieved via XMLHttpRequest. But if all you have is an XML string, something like this will do:

function TextToXML(strXML) {
 var xmlDoc = null;
 try {
  xmlDoc = (document.all)?new ActiveXObject("Microsoft.XMLDOM"):new DOMParser();
  xmlDoc.async = false;
 } catch(e) {throw new Error("XML Parser could not be instantiated");}
 var out;
 try {
  if(document.all) {
   out = (xmlDoc.loadXML(strXML))?xmlDoc:false;
  } else {  
   out = xmlDoc.parseFromString(strXML, "text/xml");
  }
 } catch(e) { throw new Error("Error parsing XML string"); }
 return out;
} 

This will give you the XML represented as a DOM Document, which you can traverse using the DOM methods.

Now all you'll have to do to convert the DOM Document to JSON is traverse it, and for every Element, create an Object, for its attributes create an @attributes Object, and a #text attribute for text nodes and repeat the process for any child elements.

/**
 * Convert XML to JSON Object
 * @param {Object} XML DOM Document
 */
xml2Json = function(xml) {
 var obj = {};
 
 if (xml.nodeType == 1) { // element
  // do attributes
  if (xml.attributes.length > 0) {
   obj['@attributes'] = {};
   for (var j = 0; j < xml.attributes.length; j++) {
    obj['@attributes'][xml.attributes[j].nodeName] = xml.attributes[j].nodeValue;
   }
  }
  
 } else if (xml.nodeType == 3) { // text
  obj = xml.nodeValue;
 }
 
 // do children
 if (xml.hasChildNodes()) {
  for(var i = 0; i < xml.childNodes.length; i++) {
   if (typeof(obj[xml.childNodes[i].nodeName]) == 'undefined') {
    obj[xml.childNodes[i].nodeName] = xml2Json(xml.childNodes[i]);
   } else {
    if (typeof(obj[xml.childNodes[i].nodeName].length) == 'undefined') {
     var old = obj[xml.childNodes[i].nodeName];
     obj[xml.childNodes[i].nodeName] = [];
     obj[xml.childNodes[i].nodeName].push(old);
    }
    obj[xml.childNodes[i].nodeName].push(xml2Json(xml.childNodes[i]));
   }
   
  }
 }

 return obj;
};

Converting XML to Lean JSON?

We could make the JSON encoding of the XML lean by using just "@" for attributes and "#" for text in place of "@attributes" and "#text":

{
 "example:user" : {
  "@" : { "domain" : "example.com" },
   "name" : { "#" : "Joe" },
   "status" : {
    "@" : {"online" : "true"},
    "#" : "Away"
   },
   "idle" : null
 }
}

You may notice that "@" and "#" are valid as javascript property names, but not as XML attribute names. This allows us to encompass the DOM representation in object notation, since we are swapping DOM functions for Object properties that are not allowed as XML attributes and thus will not get any collisions. We could go further and use "!" for comments for example, and "%" for CDATA. I'm leaving these two out for simplicity.

What about converting JSON to XML?

If we follow the rules used to convert XML to JSON, it should be easy to convert JSON back to XML. We'd Just need to recurse through our JSON Object, and create the necessary XML objects using the DOM methods.

/**
 * JSON to XML
 * @param {Object} JSON
 */
json2Xml = function(json, node) {
 
 var root = false;
 if (!node) {
  node = document.createElement('root');
  root = true;
 }
 
 for (var x in json) {
  // ignore inherited properties
  if (json.hasOwnProperty(x)) {
  
   if (x == '#text') { // text
    node.appendChild(document.createTextNode(json[x]));
   } else  if (x == '@attributes') { // attributes
    for (var y in json[x]) {
     if (json[x].hasOwnProperty(y)) {
      node.setAttribute(y, json[x][y]);
     }
    }
   } else if (x == '#comment') { // comment
   // ignore
   
   } else { // elements
    if (json[x] instanceof Array) { // handle arrays
     for (var i = 0; i < json[x].length; i++) {
      node.appendChild(json2Xml(json[x][i], document.createElement(x)));
     }
    } else {
     node.appendChild(json2Xml(json[x], document.createElement(x)));
    }
   }
  }
 }
 
 if (root == true) {
  return this.textToXML(node.innerHTML);
 } else {
  return node;
 }
 
};

This really isn't a good example as I couldn't find out how to create Elements using the XML DOM with browser Javascript. Instead I had to create Elements using the document.createElement() and text nodes with document.createTextNode() and use the non-standard innerHTML property in the end. The main point demonstrated is how straight forward the conversion is.

What is the use of converting JSON to XML

If you are familiar with creating xHTML via the DOM methods, you'll know how verbose it can be. By using a simple data structure to represent XML, we can remove the repetitive code needed to create the xHTML. Here is a function that creates HTML Elements out of a JSON Object.

/**
 * JSON to HTML Elements
 * @param {String} Root Element TagName
 * @param {Object} JSON
 */
json2HTML = function(tag, json, node) {
 
 if (!node) {
  node = document.createElement(tag);
 }
 
 for (var x in json) {
  // ignore inherited properties
  if (json.hasOwnProperty(x)) {
  
   if (x == '#text') { // text
    node.appendChild(document.createTextNode(json[x]));
   } else  if (x == '@attributes') { // attributes
    for (var y in json[x]) {
     if (json[x].hasOwnProperty(y)) {
      node.setAttribute(y, json[x][y]);
     }
    }
   } else if (x == '#comment') { // comment
   // ignore
   
   } else { // elements
    if (json[x] instanceof Array) { // handle arrays
     for (var i = 0; i < json[x].length; i++) {
      node.appendChild(json2HTML(json[x][i], document.createElement(x)));
     }
    } else {
     node.appendChild(json2HTML(json[x], document.createElement(x)));
    }
   }
  }
 }
 
 return node;
 
};

Lets say you wanted a link <a title="Example" href="http://example.com/">example.com</a>. With the regular browser DOM methods you'd do:

var a = document.createElement('a');
a.setAttribute('href', 'http://example.com/');
a.setAttribute('title', 'Example');
a.appendChild(document.createTextNode('example.com');

This is procedural and thus not very pleasing to the eye (unstructured) as well as verbose. With JSON to XHTML you would just be dealing with the data in native JavaScript Object notation.

var a = json2HTML('a', {
 '@attributes': { href: 'http://example.com/', title: 'Example' },
 '#text': 'example.com'
});

That does look a lot better. This is because JSON seperates the data into a single Object, which can be manipulated as we see fit, in this case with json2HTML().

If you want nested elements:

var div = json2HTML('div', {
 a : {
  '@attributes': { href: 'http://example.com/', title: 'Example' },
  '#text': 'example.com'
 }
});

Which gives you

<div><a title="Example" href="http://example.com/">example.com</a></div>

The uses of converting JSON to XML are many. Another example, lets say you want to syndicate an RSS feed. Just create the JSON Object with the rules given for conversion between XML and JSON, run it through your json2Xml() function and you should have a quick and easy RSS feed. Normally you'd be using a server side language other than JavaScript to generate your RSS (however Server Side JavaScript is a good choice also) but since the rules are language independent, it doesn't make a difference which language is used, as long as it can support the DOM, and JSON.

Twittier - a live view of twitter

A preview of Twittier.com is available.

What is does is retrieve updates from the twitter public timeline, or specific topics using summize.com and presents it in a simple view.

On receiving a message it extracts "relevant" keywords and displays a "live cloud view" of keywords. This cloud view is updating in real time, and thus gives you an idea of the trending topics on twitter at any given time.

At the moment it doesn't work on IE6, though I haven't tested IE7. I'm developing this on Firefox3 beta but Firefox2.0 should display it fine also.

The mashup is fully client side, using the MooTools JavaScript library and summize.com for data.

Hacking Google Suggest (complete) with JavaScript Remoting

Google Suggest is an feature of the Google Search Engine User Interface and Server that creates suggestions (auto-completes) your search keywords.

This is performed using AJAX. Google will send the first few letters or word(s) you type into the Search Input back to the Google Server using the XMLHttpRequest Object (or a similar AJAX method)

When I type in the search keywords "javascript". I can watch the XMLHttpRequests created by Google Search using Firefox's Firebug extension. Google creates 5 XMLHttpRequests, each one a few letters more then the previous.

The Google Suggest XMLHttpRequests

• http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=ja - does not complete
• http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=java - completes and returns:

  window.google.ac.Suggest_apply(frameElement, "java", new Array(2, "java download", "21,600,000 results"
  
  , "java api", "9,000,000 results", "java runtime", "2,510,000 results", "java.com", "1,350,000 results"
  
  , "java update", "11,500,000 results", "javascript tutorial", "1,490,000 results", "java string", "3
  
  ,920,000 results", "java runtime environment", "894,000 results", "javanoid", "71,000 results", "java
  
   virtual machine", "2,050,000 results"), new Array(""));

• http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=javasc - completes and returns:

  window.google.ac.Suggest_apply(frameElement, "javasc", new Array(2, "javascript tutorial", "1,490,000
  
   results", "javascript substring", "120,000 results", "javascript redirect", "291,000 results", "javascript
  
   download", "20,300,000 results", "javascript replace", "283,000 results", "javascript settimeout", "198
  
  ,000 results", "javascript split", "251,000 results", "javascript indexof", "130,000 results", "javascript
  
   switch", "1,150,000 results", "javascript string replace", "153,000 results"), new Array(""));
  

• http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=javascri - does not complete
• http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=javascript - completes and returns:

  window.google.ac.Suggest_apply(frameElement, "javascript", new Array(2, "javascript tutorial", "1,490
  
  ,000 results", "javascript substring", "120,000 results", "javascript redirect", "291,000 results", "javascript
  
   download", "20,300,000 results", "javascript replace", "283,000 results", "javascript settimeout", "198
  
  ,000 results", "javascript split", "251,000 results", "javascript indexof", "130,000 results", "javascript
  
   switch", "1,150,000 results", "javascript string replace", "153,000 results"), new Array(""));
  

As you can see, each response is a JavaScript function call. Google Search makes the XMLHttpRequest call asynchronously as you type your search query, and aborts the last XMLHTTPRequest if you type more than 2 to 3 letters. Each XMLHttpRequest result is a call to the method: window.google.ac.Suggest_apply(). What what can do is create this method in our JavaScript, and wait for Google Suggest to call it with the suggested keywords and their "weight" as parameters.

Hacking Google Suggest for our own use

Now the fun part. What we do is make a HTTP request to the Google URL http://www.google.com/complete/search?hl=en&client=suggest&js=true&q={q}, where q is our keyword. Google will then return the Google Search Suggestions for that keyword.

Try it by clicking: http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=javascript

Now that we know how to get Google Suggest results from the Google server, we can implement it with JavaScript. Due to the XMLHttpRequest Same Domain Policy, we cannot use the XMLHttpRequest Object. However, the results of each Google Suggest query is javascript, so we can use JavaScript Remoting.

Here is an example:

Try our Google Suggest Hack yourself. View source to see how it works.

Native (W3C) XMLHttpRequest for IE6 and earlier browsers

With IE7 implementing XMLHttpRequest as a native JavaScript Object, the need to fork JavaScript for XMLHttpRequest is now limited to IE6 and earlier major browsers.

Here is my wrapper for IE6 and lower browsers so you can use the XMLHttpRequest syntax in those browsers also.

/**
* Emulate window.XMLHttpRequest in IE6-
*/
if (!window.XMLHttpRequest) {
 var ms_xhr_ver = false;
 window.XMLHttpRequest = function() {
  if (ms_xhr_ver) return new ActiveXObject(ms_xhr_ver);
  var xhr = false;
  var versions = [
  "Msxml2.XMLHTTP.7.0", 
  "Msxml2.XMLHTTP.6.0", 
  "Msxml2.XMLHTTP.5.0", 
  "Msxml2.XMLHTTP.4.0", 
  "MSXML2.XMLHTTP.3.0", 
  "MSXML2.XMLHTTP",
  "Microsoft.XMLHTTP"];
  var n = versions.length;
  for (var i = 0; i <  n; i++) {
   try {
    if (xhr = new ActiveXObject(versions[i])) {
     ms_xhr_ver = versions[i];
     break;
    }
   } catch (e) { /* try next */ }
  }
  return xhr;
 };
}

Here is some example usage:

/**
* Example usage of native XHR in IE6
*/
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() { 
 if (xhr.readyState == 4) {
  if (xhr.status == 200) {
   alert(xhr.responseText);
  }
 }
};
xhr.open('get', '/2008/02/native-w3c-xmlhttprequest-for-ie6-and.html');
xhr.send('');

Try It

You'll notice that you can use the same syntax in all major browsers, Firefox, IE6, IE7, Safari etc. There is not branching of code once you have emulated native XMLHttpRequest for IE6 and earlier browsers. This can be a good base to either make XMLHttpRequest requests directly, or to build your XMLHttpRequest library.

While I'm on the topic, I'd like to point out a major flaw in some popular XMLHttpRequest libraries. They wrap the ActiveX based XHR of IE6 and earlier before wrapping the native XMLHttpRequest. This will make IE7 use the older version of XMLHttpRequest.

Next I think I'll blog about cross-domain XMLHttpRequest.

Extending Native JavaScript Types breaks for in loop

I've read a few posts where JavaScript developers have been weighting different JavaScript libraries such as MooTools, Prototype, JQuery etc. based on their addition of methods to the native Types, such as Object, or Array.

The problem with extending native Types via prototyping is that the custom methods are iterated in the "for in loop".

And example is the best way to show this:

// extends the Native Object
Object.prototype.objMethod = function() {};
Object.prototype.objProperty = function() {};

// an instance of Object
var Obj = {
  myMethod: function() {},
  myProperty: ''
};

// iterate over the properties of Obj
for(var x in Obj) {
  alert(x + ' : ' + Obj[x]);
}

This will give you the result:

myMethod : function() {}
myProperty : 
objMethod : function() {}
objProperty  : 

Try It Notice that you not only iterated over the properties of Obj, but also the (user defined) properties Obj inherits from its constructor, Object.

Now, I've heard/seen a lot of developers complain about this. They have their nice "for in" loops that worked well before, and now they break. Before you start blogging about how bad it is to extend native Types, imagine what JavaScript would be like without it.

Javascript, a land dark and mysterious. Where constructors hide their evil prototype inherited methods from brave instances that must cross the __proto__ in order to see their true properties.

Ok, you get the picture...

A work around

The Object.hasOwnProperty() method differentiates between the properties of an Instance and those inherited from the Constructor.

Lets try our for in loop example again:

// extends the Native Object
Object.prototype.objMethod = function() {};
Object.prototype.objProperty = function() {};

// an instance of Object
var Obj = {
  myMethod: function() {},
  myProperty: ''
};

// iterate over the properties of Obj
for(var x in Obj) {
  if (Obj.hasOwnProperty(x))
    alert(x + ' : ' + Obj[x]);
}

Try It

Notice the new line I've added:

if (Obj.hasOwnProperty(x))

. Now I ask, would you rather be able to choose between inherited and non-inherited methods when iterating via a "for in loop", or not have the choice. I'd rather have that choice.

Use invalid CSS in a W3C standards compliant XHTML Document

This is a technique I've decided to offer in the Joomla Extensions I've developed for the Joomla community, in an effort to allow users of the extension to use invalid CSS on their pages, yet still have the page validate with the W3C CSS Validator.

Why use invalid CSS markup?

Now why would you want to use invalid CSS in the first place? The reason is that not all browsers support the W3C recommended CSS2 and CSS3 properties. (Ahem.. Microsoft.. Ahem...). Now with IE7 our CSS is diverging even more. So the use of invalid CSS is quite inevitable.

An Example Maybe?

Lets say you want to make an image transparent. Something that should be quite simple. So you use the W3C recommended CSS property which is opacity:

img {
   opacity:0.50;
}

Amazingly, this piece of CSS does not validate. Try copy and pasting it to the CSS validator.

Not only does it not validate, it doesn't work for a couple of browsers. So, in order to have image transparency across the largest number of browsers, you need:

img {  
   filter:alpha(opacity=50); 
   -moz-opacity:0.50;  
   opacity:0.50; 
   -khtml-opacity:0.50; 
}

Now this 4 property definitions will create exactly 4 errors in the CSS validator. Hahaha.

How do I use invalid CSS markup that validates?

Now the fun part. In our server logs, around 99% of actual users have JavaScript enabled on their browser. So why not use JavaScript to serve your CSS. If you're familiar with JavaScript Remoting, this is exactly the technique applied to a different problem. In JavaScript remoting, JavaScript files are loaded dynamically after the page has fully loaded. With this technique, CSS is loaded dynamically after the page has loaded.

window.onload = function() {
   if (document && document.getElementsByTagName) {
 var head = document.getElementsByTagName('head')[0];
 var link = document.createElement('link');
 link.type = 'text/css';
 link.rel = 'stylesheet';
 link.href = 'invalid_styles.css';
 head.appendChild(link);
   }
};

What we have is a piece of JavaScript that will load the CSS file, invalid_styles.css, after the HTML Document has loaded. So all you need to do is place your invalid CSS in invalid_styles.css. The W3C validator does not render JavaScript generated Elements on the page, thus the page will pass validation, while 99% of your users will enjoy your funky non-W3C CSS styles.

Fallback to valid CSS

What about the other 1%? They will only view your valid CSS since they do not have JavaScript enabled. So they have everything except your fancy styles. The actual layout of your page should be made with valid CSS.

This is not a new concept, it is used a lot for dynamic switching of CSS styles. However, the proposed application is for loading of non-W3C CSS content for extra formating in non-W3C compliant CSS browsers, while keeping all CSS layout styles in valid CSS for users without JavaScript.

Open Source xHTML, CSS and JavaScript Web Development Tools

Your web development is limited just as much by the tools you use, as by your expertize. Imagine trying to build a house with just your bare hands?

It is very possible to do web development with just notepad.exe, and an ftp client. But doing so seriously handicaps you. Here are a few tips to get you going.

Open Source Web Development Tools

• Firefox
• Firebug
• Aptana
• Notepad++
• WAMP Server

Firefox

If are using Internet Explorer as your browser, please switch to Firefox. Firefox has many extensions developed by Web Developers or Web Developers. The most indispensable one to date is Firebug.

Firebug

Firebug is all around the best client side web based development tool. It allows you to view and edit the source of your HTML, JavaScript and CSS directly on the web page, in real time. It has a JavaScript debugger, DOM viewer, HTTP Request/Response viewer and a lot of other nifty features.

You may also want to look at the other web development extensions provided for Firefox.

Aptana

Aptana is an open source IDE (code editor) built upon Eclipse. It supports Ruby on Rails, Adobe Air, IPhone among other development platforms. Up till now, I haven't found a better Open Source IDE for web developers.

Notepad++

Notepad++ is a drop in replacement for the default notepad.exe on Windows OS. It is also a syntax highlighter for many programming languages and has a built in file manager. This tool offers quick editing of your HTML, CSS, XML etc. if you don't want to waste time loading a full IDE.

WAMP Server

Apache, MySQL, PHP on Windows. If you're still doing your web development over FTP, its about time you worked on a local version of your web server. WAMP allows you to easily install an Apache Server, with PHP and MySQL pre-installed.