The Web ADF, may it REST in peace

There has been a lot of blowback against the ESRI ArcGIS Server WebADF in the last week. Dave’s complaint against an obvious short coming in the Web ADF was echoed by Doron Yaacoby and the steam really started to to vent in the comments of James’ post. Dave has since made a call to action, to paraphrase: “stop whining, or inventing your own secret personal version, and let’s get together and build something!” I could not agree more. In fact it is exactly what I’ve been turning over in my head for a while, but it didn’t seem like something that could be properly done unilaterally

The Basic Question

Why do most people want to use the Web ADF? They want to publish their content & let users interact with it. The Google/Yahoo/Microsoft map API’s & even more so, the OpenLayers & FeatureServer combination has shown us that it doesn’t require the bloated, overly chatty, buggy, difficult, and EXPENSIVE Web ADF solution. You could do more, so much more, easily with a good common-sense standards based approach that allows easy integration into your favorite Web Mapping display engine.

Where to Start

A well thought out RESTful interface with AGS would be a great start down this path. There are really only 2 resource types required to make web maps.

• Tiles (raster data or pre-rendered cartographic output)
• Vectors, of which points are somewhat of a special case

TileCache and FeatureServer are great examples of how to access and emit this data in an extensible, lightweight, standards based manner. While there a few things I would like to change about FeatureServer, I think this is a great model to start from.

True RESTful URL’s

There has been a lot of discussion in the last year about what constitutes a proper RESTful URL. The most vocal geobloggers on this topic have easily been Sean Gillies (the Grumpy Guardian of GeoREST) and Sebastian Good, though others have chimed in as well. Is it just a URL that will point to the same resource regardless of it’s structure (i.e.. Permalink & RPC/REST hybrids) or is it a URL with limited or no query string and each directory of the URL contains a standardized semantic meaning. I am strongly in favor of the second option and think that is what we should shoot for.

Vectors

Single Feature:
http://<server>/<service or map resource>/<layer>/<id>.<format>

Vertices within a Single Feature:
http://<server>/<service or map resource>/<layer>/<id>/<vertex index>.<format>
Feature Query (preferably using Open Search specs):
http://<server>/<service or map resource>/<layer>/q?<query string>

Often there are times when you only want the geometry or you only want the attributes so these should be separable
Single Feature, Geometry Only:
http://<server>/<service or map resource>/<layer>/<id>/geom.<format>
Single Feature, Attributes Only:
http://<server>/<service or map resource>/<layer>/<id>/attrib.<format>

Notice I didn’t put a format for the query return? That is because here is where I would diverge from FeatureServer. Instead of returning the actual features that match that query I would return resource links. This is because you have no idea of knowing wether or not such a request is truly a cacheable resource identifier or not before hand. However, you do know that the feature URL’s are and should be cached. In the REmapper.com maps, I cache the response to a feature URL on both the server & the client for as long as possible. Depending on how static or dynamic your data is, this may not be appropriate for you, but it works very well for us.

Tiles:
Tiles can come from many sources and may very well be remote and neither owned nor controlled by you. These also might be something that is created & cached on the fly. Additionally, you may not even want a server side cache to persist longer than a user’s session. Here is where something acting in the same way as TileCache could be very helpful. A tile request handler, configured via a config file, could intercept these requests and then reformat them and pass them on to the real resource and return & cache the resource in whatever manner you’ve instructed it to (disk, memory, Amazon S3, raster data catalogue, etc..).

The main drawback with TileCache is that it only accepts WMS-C, TMS, or WorldWind requests. The tile request handler should be flexible enough to take regular WMS requests and translate (normalize) them into WMS-C or TMS requests and cache a standardized tile, which could be returned the next time any arbitrary bounding box query came in that contained that tile. OpenLayers does that request translation on the client side but you could connect to many more client types if you did it on the server side. Additionally our tile request handler should understand Google, Yahoo, Virtual Earth, etc tile requests and translate those into a standardized cacheable request as well. (by the way TileCache IS flexible enough to handle all types of requests, but no one has written a provider for the types it doesn’t).

So for tiles the premise is accept just about any valid request, fetch or create & cache the requested tile, then return a reference to the resource or the resource itself.

The Plumbing

Following the lead of FeatureServer & TileCache yo have the basic processes:

For Vectors:

Request -> Translator -> Query Object -> Provider -> Common Feature Object -> Serializer -> Response

For Tiles:

Request -> Translator -> Storage Provider -> (Layer Type Provider, if not found by Storage Provider) -> Response

Using this method you can plug in any provider you want. You can interact with ArcGIS Server via SOAP API calls, directly connect to ArcObjects, ArcSDE, Spatial RDBMS, use the OGR library, whatever you want as long as you can translate the feature into some lingua franca that can then be serialized in to any available format. That format could be any of the wire formats (GeoJSON, KML, GML, GeoRSS, etc..) but it could also be a shapefile or other binary format. You just need to plug in the right provider for that “serialization”.

On the client side it would probably be helpful to provide some helper JavaScript functions that can help translate vector wire formats into map objects and vice versa.

So we have the basic blueprint, all the copper we need, and a workspace. Let’s get plumbing!

Comparing Mashup Platforms Using JSON & MySQL - Part 2 - Process JSON

Technologies Used: PHP 5, MySQL 4.1.2, JavaScript, JSON

Software Used: PHP Designer, Aptana, Firebug, Firefox

There are a wide variety of ways to handle JSON that is returned from a server. One of these methods is to provide a callback function pass it to the JSON emiting webservice via REST. When used with static or dynamic script tags, you are no longer restricted to Cross Site Scripting (XSS) limits requireing that the source be on the same server as your webpage. The XSS limit seems particularly ill-suited for mash-ups. The callback function method is what I have choosen for my examples. You will also see a static script tag for these examples. That is to simplify them and make them easier to read. The general preference is to use dynamic script tags rather than static ones.

Actual JSON string returned:

handleJSON({”Observations”:[{"Station":{"number":"1","lat":"29.5762","lon":"-98.7041","total_sp":"9","Species":[{"common":"Mourning Dove","number":"1","code":"MODO","scientific":"Zenaida macroura"},{"common":"Eastern Phoebe","number":"1","code":"EAPH","scientific":"Sayornis phoebe"},{"common":"Carolina Chickadee","number":"1","code":"CACH","scientific":"Poecile carolinensis"},{"common":"Black-crested Titmouse","number":"3","code":"BCTI","scientific":"Baeolophus atricristatus"},{"common":"Carolina Wren","number":"1","code":"CARW","scientific":"Thryothorus ludovicianus"},{"common":"Bewick's Wren","number":"2","code":"BEWR","scientific":"Thryomanes bewickii"},{"common":"Rufous-crowned Sparrow","number":"1","code":"RCSP","scientific":"Aimophila ruficeps"},{"common":"Northern Cardinal","number":"3","code":"NOCA","scientific":"Cardinalis cardinalis"},{"common":"Brown-headed Cowbird","number":"1","code":"BHCO","scientific":"Molothrus ater"}]}},{”Station”:{”number”:”2″,”lat”:”29.574″,”lon”:”-98.7036″,”total_sp”:”5″,”Species”:[{"common":"Black-crested Titmouse","number":"1","code":"BCTI","scientific":"Baeolophus atricristatus"},{"common":"Carolina Wren","number":"1","code":"CARW","scientific":"Thryothorus ludovicianus"},{"common":"Ruby-crowned Kinglet","number":"1","code":"RCKI","scientific":"Regulus calendula"},{"common":"Northern Cardinal","number":"1","code":"NOCA","scientific":"Cardinalis cardinalis"},{"common":"Brown-headed Cowbird","number":"1","code":"BHCO","scientific":"Molothrus ater"}]}}]});

It is not very human readable, but it is highly machine readable.

Below is an extremely simple example of handling JSON and doing something with it.

<html>
<head>
<meta http-equiv=“Content-Type” content=“text/html; charset=iso-8859-1″ />
<title>Simple JSON Handling</title>
<script type=”text/javascript” charset=”utf-8″>
function showCoords(jsd){
var lat = jsd.Observations[0].Station.lat;
var lon = jsd.Observations[0].Station.lon;
var msg = ‘Lat:’ + lat + ‘, Lon:’ + lon;
alert(msg);
}
</script>
</head>
<body>
<script src=http://www.plateauwildlife.com/bbc-mgmt/getstations.php?action=getdata&cid=2&year=2006&func=showCoords type=”text/javascript” charset=”utf-8″></script>
</body>
</html>

See It In Action

The above example is not particulalry usefull for anything other than demostration purposes. We want to actually DO something with our JSON to move us closer to creating the actual mashup. The number thing which I intially strugled with when using JSON & callback functions was that you MUST define the callback function BEFORE your dynamic or static script tag.

Google Maps, Google Earth, Yahoo Maps, and Virtual Earth all take HTML for the contents of the info window when you rollover or click on a point.

ArcWeb Explorer (AWX), however doesn’t take HTML as info window content. AWX does take styled text, videos, picture, audio, & swf for info window content. To this end, if you want to embed rich non-HTML content, AWX allows for some extremelly interesting content to be blended together and presented with great ease. The documentation for text styling is lacking, so creating simple content is actually more dificult in this platform than the others.

I’ve created a javascript file which we can reference in any of the HTML docs that actually embed the mashup.

This file contains

1. The main callback function
2. A function which builds an array of HTML tables containing the formated results from each station
3. A function which builds an array of jscript strings containg the weakly formated results from each station

See The HTML Builder In Action

    //A global variable to assign the parsed JSON to

var jsobj;

 

//Main Callback handler.

//simple assignment to a global variable allows me to reuse and pass

//around the object without any server trips

function handleJSON(reply){

    jsobj=reply;

}

 

    function buildHTML(Observations){

    var info_win = new Array();

    // Build a table element with Station number & total species observed

    for(var i=0;i<Observations.length;i++){

        str = ‘<table border=”1″><tbody><tr class=”station”>’;

        str += ‘<td colspan=”2″>Station ‘+ Observations[i].Station.number + ‘</td></tr>’;

        str += ‘<tr class=”sta_total”><td colspan=”2″>Total Species - ‘ + Observations[i].Station.total_sp + ‘</td></tr>’;

       str += ‘<tr class=”obs_header”><td>Species</td><td>Number</td></tr>’;

        var details = “”;

        var arr = new Array();

        // assign each Species array to a local variable to reduce typing & increase readibility

        //build an string of <tr> elements containing the details of species observed

        arr = Observations[i].Station.Species;

            for (var y=0;y<arr.length;y++){

            details+=‘<tr class=”obs_detail”><td><a href=”http://www.google.com/search?q=%22′;

            details += arr[y].scientific.replace(/\s/,“+”);

            details += ‘%22″>’ + arr[y].common + ‘ (’ + arr[y].code + ‘)</a></td>’;

            details += ‘<td>’ + arr[y].number + ‘</td></tr>’;

            }

        str +=    details;

        str += ‘</table></tr></tbody></table>’;

        //add table element to array of table element html strings

        info_win[i]=str;

    }

return info_win;

}

 

function buildAWXtxt(Observations){

        var info_win = new Array();

    // Build a formated text list for each Station number & total species observed

    for(var i=0;i<Observations.length;i++){

        str = ‘Station ‘ + Observations[i].Station.number + ‘\n’;

        str += ‘Total Species - ‘ + Observations[i].Station.total_sp + ‘\n’;

       str += ‘Species          Number’;

        var details = new Array();

        // assign each Species array to a local variable to reduce typing & increase readibility

        //build an array formatted text data elements containing the details of species observed

        //using this convention, we can assign a url property to each species line

        //through .data{elements[]} in the properties for each marker

        arr = Observations[i].Station.Species;

            for (var y=0;y<arr.length;y++){

            details[y] = arr[y].common + ‘ (’ + arr[y].code + ‘) - ‘ + arr[y].number + ‘\n’;

            }

        var obs_info = new Array([str,details]);

        //

        //add table element to array of table element html strings

        info_win[i]=obs_info;

    }

return info_win;

}

 

Previous Parts

1. Emit JSON

 

Next Parts

3. Arcweb Explorer Mashup

4. Yahoo Maps Mashup

5. Virtual Earth Mashup

6. Google Maps Mashup