Mapserver versus ArcIMS
www.refractions.net 1
MUM/EOGEO 2005
A Comparison of
ArcIMS to MapServer
Brock Anderson
Refractions Research
Mapserver versus ArcIMS
www.refractions.net 2
MUM/EOGEO 2005
Refractions ResearchRefractions Research
• Victoria, BC,Canada
• Spatial SystemsConsulting
• Open SourceSoftware
• PostGISuDig / GeoTools
Mapserver versus ArcIMS
www.refractions.net 3
MUM/EOGEO 2005
The ClientThe Client
• BC’s Ministry of Sustainable ResourceManagement (Information ManagementBranch)
– Use mostly ESRI products in their mappinginfrastructure.
– Were frustrated with ArcIMS’s administrativeneediness.
– Were looking for WMS alternatives to ArcIMS.We suggested MapServer.
Mapserver versus ArcIMS
www.refractions.net 4
MUM/EOGEO 2005
Test PlanTest Plan
• We prepared a detailed test plan tocompare ArcIMS and MapServer as WMSs.
• The tests covered:– Ease of administration
– WMS 1.1.1 standard compliance
– Interoperability with other software
– Performance
* Ease of administration
- KEY TEST: time to add and reload services
* Interoperability with other software
- KEY TEST: supports ArcSDE 8.3 and 9.x?
* WMS Standard compliance
- KEY TEST: OGC WMS CITE
* Performance (with ArcSDE)
- KEY TEST: feature density
- KEY TEST: feature complexity
- KEY TEST: image output format
- KEY TEST: concurrency
- KEY TEST: reprojection
- KEY TEST: throughput "under regular operating conditions"
Mapserver versus ArcIMS
www.refractions.net 5
MUM/EOGEO 2005
Test EnvironmentTest Environment
Test Client Computer
JMeter 2.01
WMS
Protocol
Test Server
ArcIMS 4.0.1, WMS
Connector 1.0
MapServer 4.2.1
ArcSDE 8.3
Jmeter is open source software designed to load test functional behavior and
measure performance .
Mapserver versus ArcIMS
www.refractions.net 6
MUM/EOGEO 2005
Test PreparationsTest Preparations
• Created synthetic data to be used forcertain performance tests.
• Made a JMeter extension to simulate adiversity of GetMap requests.
• Created ArcIMS AXL files and MapServermap files.
•Synthetic data used for feature density and feature complexity test.
•JMeter extension used to generate WMS requests with random bboxes.
•Started with AXL files, and transformed them into .map files
Mapserver versus ArcIMS
www.refractions.net 7
MUM/EOGEO 2005
Creating .map files from .Creating .map files from .axlaxl files files
<!– This is part of an XSL transform that converts ArcIMS .axl files into MapServer .map files -->
<xsl:template match="ARCXML">
<xsl:text>MAP</xsl:text><xsl:text>
</xsl:text> <xsl:text>EXTENT </xsl:text><xsl:value-of select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@minx"/><xsl:text> </xsl:text><xsl:value-of select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@miny"/><xsl:text> </xsl:text><xsl:value-of
select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@maxx"/><xsl:text> </xsl:text><xsl:value-of select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@maxy"/><xsl:text>
</xsl:text> <xsl:text>UNITS </xsl:text><xsl:value-of
select="./CONFIG/MAP/PROPERTIES/MAPUNITS/@units"/><xsl:text>
</xsl:text> <xsl:text>WEB </xsl:text><xsl:text>
</xsl:text> <xsl:text>METADATA </xsl:text><xsl:text>
</xsl:text> <xsl:text>"wms_title" "[service_name_here]"</xsl:text><xsl:text>
</xsl:text>
<xsl:text>"wms_srs" "[EPSG:????]"</xsl:text><xsl:text>
</xsl:text> <xsl:text>END #end metadata </xsl:text><xsl:text>
</xsl:text> <xsl:text>END #end web</xsl:text><xsl:text>
</xsl:text>
<!-- process the layers --> <xsl:apply-templates select="./CONFIG/MAP/LAYER"/>
<xsl:text>END #end map</xsl:text><xsl:text>
</xsl:text> </xsl:template>
ArcIMS .axl service file
MapServer .map service file
•Our client had an “optimized” .axl file containing their provincial basemap
data.
•We used that .axl file to create a .map file from.
Mapserver versus ArcIMS
www.refractions.net 8
MUM/EOGEO 2005
Testing Began, andTesting Began, and……
• Early performance tests showed ArcIMSoutperformed MapServer
Number of Concurrent Requests vs. Response Time
702 7751125
25022703 2793
3732
8483
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
1 2 4 8
Number of Concurrent Requests
Ave
rag
e R
esp
on
se T
ime (
ms
)
ArcIMS
MapServer
•An initial test to convince ourselves that MapServer was comparable to
ArcIMS
•We requested the same map each time: 4 layers (including points, lines and
polygons)
•Each level of concurrency run for 10 minutes.
•Mention that 1) all graphs have at least 30 samples per point. 2) All tests
done against ArcSDE.
Mapserver versus ArcIMS
www.refractions.net 9
MUM/EOGEO 2005
MapServerMapServer’’s s BottlenecksBottlenecks
• Profiling revealed two mainbottlenecks:
– 1 to 2 seconds of ArcSDE
connection overhead per GetMap
request
– Additional overhead extracting
features from ArcSDE
Mapserver versus ArcIMS
www.refractions.net 10
MUM/EOGEO 2005
Minimizing the Connection OverheadMinimizing the Connection Overhead
• Persistent database connections wouldnearly eliminate ArcSDE connectionoverhead.
• As a CGI program, MapServer had nomeans to support persistentconnections.
• Added FastCGI support. Thanks Frank!
• Updated the ArcSDE module to utilizepersistent connections. Thanks Howard!
•Frank Warmerdam added FastCGI support to MapServer, and he created a
connection pooling API
•Howard Butler updated the ArcSDE module to utilize connection pooling
Mapserver versus ArcIMS
www.refractions.net 11
MUM/EOGEO 2005
Using Using MapServer MapServer with with FastCGIFastCGI
1. Compile MapServer with FastCGI support.
2. Configure your web server with a FastCGImodule.
3. Update your map files:
LAYER
PROCESSING “CLOSE_CONNECTION=DEFER”
#all other layer settings here...
END #LAYER
* Persistent connections are most useful for
data sources with large connection overhead,
such as ArcSDE.
Mapserver versus ArcIMS
www.refractions.net 12
MUM/EOGEO 2005
Other EnhancementsOther Enhancements
• Recall, connection overhead was not theonly slowdown.
• We also improved the MapServer codewhich pulled features from ArcSDE.
– This involved experimenting with ESRI’s ArcSDE
C API.
– We discovered which operations were costly,
and cut down on their use.
*Performance is best for the default version of ArcSDE layers.
Mapserver versus ArcIMS
www.refractions.net 13
MUM/EOGEO 2005
The Same Test AgainThe Same Test Again
• The earlier performance test was run again,this time with FastCGI and the otherperformance improvements.
Number of Concurrent Requests vs. Response Time
1067934
1114
2384
861 916
1104
1867
0
500
1000
1500
2000
2500
3000
1 2 4 8
Number of Concurrent Requests
Av
era
ge
Re
sp
on
se
Tim
e (
ms
)
MapServer
ArcIMS
Mapserver versus ArcIMS
www.refractions.net 14
MUM/EOGEO 2005
Performance Test Results Performance Test Results (Throughput)(Throughput)
• MapServer now outperforms ArcIMS underMSRM’s “regular operating conditions.”
Regular Traffic Over an Extended Period (Throughput)
0
0.5
1
1.5
2
2.5
3
3.5
4
1 2 3 4 5 6 7 8
Number of Concurrent Users
Ma
ps
Pe
r S
ec
on
d
ArcIMS
MapServer
•The higher the better on this graph only
•Difference between concurrency test and this test:
•This test uses random requests for 10 min.
•This test doesn’t hit the server with a sudden burst of requests. They
are ramped up.
Mapserver versus ArcIMS
www.refractions.net 15
MUM/EOGEO 2005
Performance Test ResultsPerformance Test Results (Image Format) (Image Format)
• MapServer is slightly faster to return GIFand PNG images. ArcIMS is faster for JPG.
Image Format vs. Response Time
0
500
1000
1500
2000
2500
3000
JPG GIF PNG
Image Format
Avera
ge R
esp
on
se T
ime (
ms)
ArcIMS
MapServer
Mapserver versus ArcIMS
www.refractions.net 16
MUM/EOGEO 2005
Performance Test ResultsPerformance Test Results ( (ReprojectionReprojection))
• MapServer reprojects faster than ArcIMS.
Reprojection vs. Response Time
0
1000
2000
3000
4000
5000
6000
7000
8000
BC Albers (NAD83) Lat-Long (NAD 83) UTM10 (NAD 83)
Output Projection
Avera
ge R
esp
on
se T
ime (
ms)
ArcIMS
MapServer
Mapserver versus ArcIMS
www.refractions.net 17
MUM/EOGEO 2005
Performance Test ResultsPerformance Test Results (Feature Complexity) (Feature Complexity)
• Feature complexity affects both serversalmost equally (from ArcSDE).
Feature Complexity vs. Response Time
0
5000
10000
15000
20000
25000
30000
4 8 16 32 64 128 256 512
Number of Points in Each of 10,000 Circles
Av
era
ge
Re
sp
on
se
Tim
e (
ms
)
MapServer (SDE)
ArcIMS (SDE)
MapServer (.shp)
Mapserver versus ArcIMS
www.refractions.net 18
MUM/EOGEO 2005
Performance Test ResultsPerformance Test Results (Feature Density) (Feature Density)
• MapServer extracts features from SDEslightly faster than ArcIMS does.
Feature Density vs. Response Time
0
10000
20000
30000
40000
50000
60000
1200
0
2400
0
3600
0
4800
0
6000
0
7200
0
8400
0
9600
0
1080
00
1200
00
1320
00
1440
00
Number of Features
Avera
ge R
esp
on
se T
ime (
ms)
MapServer (SDE)
ArcIMS (SDE)
MapServer (.shp)
Actually, this test demonstrates that MapServer (extracts features from SDE) +
(draws the map) faster
Mapserver versus ArcIMS
www.refractions.net 19
MUM/EOGEO 2005
WMS Standard ComplianceWMS Standard Compliance
• MapServer passed all 83 WMS CITE tests.
• ArcIMS (w/ WMS connector) passed 71 of83 WMS CITE tests.
– ArcIMS failed 12 tests because:
• Wrong MIME type for some responses.
• Wrong “exception code” in some exceptions.
•MIME type of responses is typically expected to be
“application/vnd.ogv.se_xml”
Mapserver versus ArcIMS
www.refractions.net 20
MUM/EOGEO 2005
Ease of AdministrationEase of Administration
• MapServer administration benefits
– No need to reload services (when service files
change)
– MapServer restarts faster (as fast as the web
server)
• ArcIMS administration benefits
– More granular control over log levels
Mapserver versus ArcIMS
www.refractions.net 21
MUM/EOGEO 2005
Interoperability and SupportInteroperability and Support
• Both WMSs support:
–ArcSDE 8.3 and ArcSDE 9.0
–Styled Layer Descriptor (SLD)
I don’t know how fully either server supports SLD, but our testing showed that
both support these basic features:
•Selecting and styling based on a attribute value
•Selecting and styling based on spatial location (within a bounding box)
Mapserver versus ArcIMS
www.refractions.net 22
MUM/EOGEO 2005
ConclusionsConclusions
MapServer is easier to administer thanArcIMS.
MapServer is more WMS standardcompliant than ArcIMS.
MapServer matches or surpasses ArcIMSin most performance tests.
* (MSRM adopted MapServer for theirCOINPacific web mapping application)
Mapserver versus ArcIMS
www.refractions.net 23
MUM/EOGEO 2005
Questions?Questions?
Contact me:
Brock Anderson
Refractions Research
www.refractions.net
(250) 383-3022