Manifold Saves the Day

Twice in the last few days, one of our wildlife techs HAD to have some property boundary files loaded onto thier GPS unit. Unfortunately they had been created from a CAD drawing with “true curves”. The file had over 30k vertices. That’s not going to fit into any GPS unit we have.

I didn’t have my computer with ET Geowizards on it available, but I did have my laptop with Manifold. I quickly imported the shapefile into Manifold and used the Simplify tool to get it down to 1ft accuracy. That reduced it down to 385 vertices, which will fit in even the oldest GPS units we have.

So I got to thinking. Why would you want to pay for ET Geowizards for $199 when you can buy Manifold for $245. Manifold gives you a lot of functionality that is not present in the ArcView version of ArcGIS. Quite a bit more than you get with ArcView license and ET Geowizards. I also think it is ridiculous that if you spend $1,500 per single use license not to have the toolset that ET Geowizards gives you. They are merely using available ArcObjects functions to provide you with something that ESRI wants you to pony up $2,500 for.

UMN Mapserver + MS4W + MrSID = Happy Again

I saw on the Mapserver documentation of supported formats that MrSID and a number of other raster formats were also supported through the use of gdal as a 3rd party component. Well, I had read this to be about the same as Manifold saying that it supported MrSID through 3rd party software. Which was not very well, if at all.

You can preach to me about how uncool MrSID is until you are blue in the face, but the fact is that is the way most of our imagery data comes to us unless we want to pay more or wait for publicly available quarter quads in different formats. And these are mostly large full county mosaics, so decoding or gdal-translating them is a pretty inefficient process. So that’s what I’m stuck with unless I absolutely have to change.

On a lark, I decided to use one of the many sample Mapserver map files and edit it to point to one of the full county sid’s. Bam! Just like that, I now had a WMS image server. I was able to immediately hook it up to UDig and with a bit of experimentation, also hook it into ArcGIS Explorer. This is a ~250mb .sid file and it was being served out by my test server in the same intranet with equal or greater speed than AGX could read the file locally. UDig was even better, it actually displayed the raster faster from Mapserver’s WMS than ArcMap did reading it directly from the disk.

So, for the moment, I’m going to cool off about Manifold (I really didn’t want to have to do that data conversion right now anyway). And I’m going forward with Open Source. I’ll use Mapserver installed from the MS4W package to serve the imagery at least. The data will probably also be from Mapserver but I’ll be using SLD as the cartographic options for polygons and various fill types aren’t directly available in map files alone.

I like ka-Map or MapGuide OS as web-based clients. I’m most excited about using AGX and it’s .NET based custom tasks for a desktop based client.

My only gripe about Mapserver is that there doesn’t seem to be a good way to avoid hand coding some of the map file, unless you use MapScript. Since I installed PHP 5, MapLab doesn’t work very well, and you have to fill all that stuff by hand anyway.

If I have the time (read, not anytime soon), I’d like to make a simple multi-file select data uploader & styler that then writes the actual map file for you. And will automatically attempt to read the projection info and spatial extents of your data.

I probably won’t do much with this until the end of the week, but if I getting a working demo, I’ll defiantly post with pictures about it.

Manifold IMS Redux

In which I eat a small bite of crow, and then discover all this head banging is fairly irrelevant.

My previous post really took on a new life of it’s own when James Fee linked to it in his blog. I have recieved several emails from Manifold users & a number of comments pointing me it the right direction to get Manifold IMS working properly.

The gist of it is that I merely skimmed the Optimizing Performance document, as I thought it was just that (squezzing every thing you can from a service), and not an ESSENTIAL document on setting up Manifold map files for serving over the web. The main thrust of this document is that you need to minimize the size of the .map file in relation to your server’s RAM and that if you are connecting to a database of anykind, use the best driver for that connection.

With only 512kb of RAM in a AMD 1.8ghz machine running Windows XP as my home test server, a 1.4GB .map file is definatley NOT optimized in relation to the available RAM. So for the image server componentI exported my 256 quarter-quad ecw’s to a single even further compressed ecw. Then I created a new project and linked the single image, created a map and exported it as an ASP.NET webpage with support for WMS. I ended up with a 130kb .map file. I was able to view the imagery on the server and link to it in another application on the server. However, there are obviously some security permission issues that are preventing me from getting to the data on any remote machine. If I decide to pursue the Manifold IMS stuff further, I’ll post to the forums and get that figured out.

So the image server train wreck was fully my fault. Now as for the actual mapdata IMS project, that did not go as well as the image server. Even after linking the shapefiles and PGDB’s, the data was served very slowly and draw incompletely or not at all. I think the bottom line on that one is that I need to use a RDBM to store the spatial data in. The ESRI formats are just not efficient to work with on this scale in Manifold. I’d had already planned on using a RDBM for this stuff anyway, but I’d rather go with PostGIS, thus giving me spatial SQL accessable from a wide variety of clients.

Now, why didn’t I link this stuff to begin with. Well actually, I did, when setting up the desktop project. I figured that linking would be a much more efficient way to deal with data after reading some of the intial documentation and tutorials. However, every time I tried to pan, zoom or resize my Map with either the imagery and/or spatial data files linked, Manifold crashed. Now this was not on that crummy old test server but on an AMD Athlon 64 3400+ with 1GB RAM and a pretty good video card. ArcMap 8.3, UDig, QGIS, & the TatukGIS Viewer were all able to handle the same level of non-imagery data or much more without any severe problems.

So I’d like to apologize for fully flaming Manifold before at least trying to get a solution from Manifold first. However, I was just really tired of fighting with this software. There were the crashes I talked about above and I’m still pissed about the MrSID support. If you don’t really offer it, then say so. It would have been pretty easy to hyperlink the “with 3rd party decoder” statement next to it in the Supported Formats section over to the “Why we don’t really do MrSID” page of the online help manual. The proported support for this format, which I have a TON of data in and continue to recieve data in, is the main reason for purchasing Manifold in the first place. I’ve felt somewhat cheated ever sinceI realized that this was not really supported. So I had a lot of pent up frustration that just finnally boiled over when the promise of a “click-click-done” IMS solution didn’t work out as planned.

For why I think this all irrelevant now, see next post.

Manifold WMS + ArcGIS Explorer = Train Wreck

UPDATE:

After a bit more time and reading, I have come to a different conclusion.

Please see my next post “Manifold IMS Redux”.

I still have no intention of using Manifold IMS as an image server. However, it very well may get the nod for feature data. I’ll have a better idea next week as I pit it against some other options.

————————————————————————————–

Manifold’s IMS is not an unmitigated disaster.

So to be fair the fault is not with AGX at all. It is 100% with Manifold Me.

The reason we bought Manifold was to have a GIS in which to create nice maps of our large datasets in an easy way and serve them out to the web. While there were plenty of warnings that Manifold’s website was full of hyperbole & BS, I choose not to heed them.

According to the Manifold System 7 website:

Format’s supported includes MrSIDs.

• Well we now know that turned out to be completely false. It supports using them only if they are of a fairly small area as it uncompresses and rewrites files to an actually supported format. This is MUCH slower when using them in a map than taking the time to convert these files to a fully supported format such as ECW, jpg2k, etc. I chose to make them ECW’s to serve out through Manifold’s Image Server or WMS. This is a no go as well.

Manifold IMS delivers blistering hot performance

Only if you think grass growing & paint drying are lightining quick, would I agree with the statement above.

While the performance may be a bit better if you are using a RDBM backend, It is HORRIBLE when using shapefiles imported (not linked but actually imported) into Manifold & saved as a .MAP file.

I was trying to serve out 2 different Manifold projects and neither one worked. One contained 256 quarter quad ECW files (~1.4GB .map file) and was to act as an image server. Niether the WMS or Manifold Image Server protocals working in AGX, Manifold, or the default asp.net web page viewer. The next one was a subset of my large multicounty project using only Travis county. Two of the featuresets in this project were rather large (~300k features), but all the rest were small. The project also contained no imagery. It was ~500mb .map file. AGX hooked up to the WMS I created for this project, but then crashed waiting for the WMS to respond to requests. The default asp.net web page created by the export process took ~10 min to respond to each request and drew very incomplete features or often none at all.

I’m sure that Manifold excels in some respect somewhere, but I haven’t found it yet. I’ve been much more impressed with UDig & QGIS as alternate GIS platforms to ArcGIS. As for the server side of things, I’m going to give UMN Mapserver and Mapguide OpenSource a good try. If I can get either of them working, then I’ll little or no use for Manifold any more. I’m really sorry we wasted our money on it. I would have much rather spent our $1,200 on something else.