8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 1/23
Mapserver versus ArcIMS
www.refractions.net 1
MUM/EOGEO 2005
A Comparison of
ArcIMS to MapServer
Brock Anderson
Refractions Research
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 2/23
Mapserver versus ArcIMS
www.refractions.net 2
MUM/EOGEO 2005
Refractions ResearchRefractions Research
• Victoria, BC,Canada
• Spatial SystemsConsulting
• Open SourceSoftware
• PostGIS
uDig / GeoTools
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 3/23
Mapserver versus ArcIMS
www.refractions.net 3
MUM/EOGEO 2005
The ClientThe Client
• BC’s Ministry of Sustainable ResourceManagement (Information ManagementBranch)
– Use mostly ESRI products in their mappinginfrastructure.
– Were frustrated with ArcIMS’s administrativeneediness.
– Were looking for WMS alternatives to ArcIMS.We suggested MapServer.
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 4/23
Mapserver versus ArcIMS
www.refractions.net 4
MUM/EOGEO 2005
Test PlanTest Plan
• We prepared a detailed test plan to
compare ArcIMS and MapServer as WMSs.
• The tests covered:– Ease of administration
– WMS 1.1.1 standard compliance
– Interoperability with other software
– Performance
* Ease of administration
- KEY TEST: time to add and reload services
* Interoperability with other software
- KEY TEST: supports ArcSDE 8.3 and 9.x?
* WMS Standard compliance
- KEY TEST: OGC WMS CITE
* Performance (with ArcSDE)
- KEY TEST: feature density
- KEY TEST: feature complexity
- KEY TEST: image output format
- KEY TEST: concurrency
- KEY TEST: reprojection
- KEY TEST: throughput "under regular operating conditions"
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 5/23
Mapserver versus ArcIMS
www.refractions.net 5
MUM/EOGEO 2005
Test EnvironmentTest Environment
Test Client Computer
JMeter 2.01
WMS
Protocol
Test Server
ArcIMS 4.0.1, WMSConnector 1.0
MapServer 4.2.1
ArcSDE 8.3
Jmeter is open source software designed to load test functional behavior and
measure performance .
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 6/23
Mapserver versus ArcIMS
www.refractions.net 6
MUM/EOGEO 2005
Test PreparationsTest Preparations
• Created synthetic data to be used for
certain performance tests.
• Made a JMeter extension to simulate adiversity of GetMap requests.
• Created ArcIMS AXL files and MapServermap files.
•Synthetic data used for feature density and feature complexity test.
•JMeter extension used to generate WMS requests with random bboxes.
•Started with AXL files, and transformed them into .map files
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 7/23
Mapserver versus ArcIMS
www.refractions.net 7
MUM/EOGEO 2005
Creating .map files from .Creating .map files from .axlaxl filesfiles
<!– This is part of an XSL transform that converts ArcIMS .axlfiles into MapServer .map files -->
<xsl:template match="ARCXML">
<xsl:text>MAP</xsl:text><xsl:text>
</xsl:text><xsl:text>EXTENT </xsl:text><xsl:value-of
select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@minx"/><xsl:text> </xsl:text><xsl:value-ofselect="./CONFIG/MAP/PROPERTIES/ENVELOPE/@miny"/><xsl:text> </xsl:text><xsl:value-of
select="./CONFIG/MAP/PROPERTIES/ENVELOPE/@maxx"/><xsl:text> </xsl:text><xsl:value-ofselect="./CONFIG/MAP/PROPERTIES/ENVELOPE/@maxy"/><xsl:text>
</xsl:text>
<xsl:text>UNITS </xsl:text><xsl:value-of
select="./CONFIG/MAP/PROPERTIES/MAPUNITS/@units"/><xsl:text>
</xsl:text><xsl:text> WEB </xsl:text><xsl:text>
</xsl:text><xsl:text> METADATA </xsl:text><xsl:text>
</xsl:text><xsl:text>"wms_title" "[service_name_here]"</xsl:text><xsl:text>
</xsl:text>
<xsl:text>"wms_srs" "[EPSG:????]"</xsl:text><xsl:text>
</xsl:text><xsl:text>END #end metadata </xsl:text><xsl:text>
</xsl:text><xsl:text>END #end web</xsl:text><xsl:text>
</xsl:text>
<!-- process the layers --> <xsl:apply-templates select="./CONFIG/MAP/LAYER"/>
<xsl:text>END #end map</xsl:text><xsl:text>
</xsl:text>
</xsl:template>
ArcIMS .axl service file
MapServer .map service file
•Our client had an “optimized” .axl file containing their provincial basemap
data.
•We used that .axl file to create a .map file from.
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 8/23
Mapserver versus ArcIMS
www.refractions.net 8
MUM/EOGEO 2005
Testing Began, andTesting Began, and……
• Early performance tests showed ArcIMS
outperformed MapServerNumber of Concurrent Requests vs. Response Time
702 7751125
25022703 2793
3732
8483
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
1 2 4 8
Number of Concurrent Re uests
A v e r a g e R e s p o n s e T i m e ( m s )
ArcIMS
MapServer
•An initial test to convince ourselves that MapServer was comparable to
ArcIMS
•We requested the same map each time: 4 layers (including points, lines and
polygons)
•Each level of concurrency run for 10 minutes.
•Mention that 1) all graphs have at least 30 samples per point. 2) All tests
done against ArcSDE.
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 9/23
Mapserver versus ArcIMS
www.refractions.net 9
MUM/EOGEO 2005
MapServerMapServer’’ss BottlenecksBottlenecks
• Profiling revealed two main
bottlenecks:
– 1 to 2 seconds of ArcSDEconnection overhead per GetMaprequest
– Additional overhead extractingfeatures from ArcSDE
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 10/23
Mapserver versus ArcIMS
www.refractions.net 10
MUM/EOGEO 2005
Minimizing the Connection OverheadMinimizing the Connection Overhead
• Persistent database connections would
nearly eliminate ArcSDE connectionoverhead.
• As a CGI program, MapServer had nomeans to support persistentconnections.
• Added FastCGI support. Thanks Frank!
• Updated the ArcSDE module to utilizepersistent connections. Thanks Howard!
•Frank Warmerdam added FastCGI support to MapServer, and he created a
connection pooling API
•Howard Butler updated the ArcSDE module to utilize connection pooling
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 11/23
Mapserver versus ArcIMS
www.refractions.net 11
MUM/EOGEO 2005
UsingUsing MapServerMapServer withwith FastCGIFastCGI
1. Compile MapServer with FastCGI support.
2. Configure your web server with a FastCGImodule.
3. Update your map files:
LAYER
PROCESSING “CLOSE_CONNECTION=DEFER”
#all other layer settings here...
END #LAYER
* Persistent connections are most useful for
data sources with large connection overhead,such as ArcSDE.
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 12/23
Mapserver versus ArcIMS
www.refractions.net 12
MUM/EOGEO 2005
Other EnhancementsOther Enhancements
• Recall, connection overhead was not the
only slowdown.
• We also improved the MapServer codewhich pulled features from ArcSDE.
– This involved experimenting with ESRI’s ArcSDEC API.
– We discovered which operations were costly,and cut down on their use.
*Performance is best for the default version of ArcSDE layers.
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 13/23
Mapserver versus ArcIMS
www.refractions.net 13
MUM/EOGEO 2005
The Same Test AgainThe Same Test Again
• The earlier performance test was run again,
this time with FastCGI and the otherperformance improvements.
Number of Concurrent Requests vs. Response Time
1067934
1114
2384
861 916
1104
1867
0
500
1000
1500
2000
2500
3000
1 2 4 8
Number of Concurrent Re uests
A v e r a g e R e s p o n s e T i m e ( m s )
MapServer
ArcIMS
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 14/23
Mapserver versus ArcIMS
www.refractions.net 14
MUM/EOGEO 2005
Performance Test ResultsPerformance Test Results (Throughput)(Throughput)
• MapServer now outperforms ArcIMS under
MSRM’s “regular operating conditions.” Regular Traffic Over an Extended Period (Throughput)
0
0.5
1
1.5
2
2.5
3
3.5
4
1 2 3 4 5 6 7 8
Number of Concurrent Users
M a p s P e r S e c o n d
ArcIMS
MapServer
•The higher the better on this graph only
•Difference between concurrency test and this test:
•This test uses random requests for 10 min.
•This test doesn’t hit the server with a sudden burst of requests. They
are ramped up.
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 15/23
Mapserver versus ArcIMS
www.refractions.net 15
MUM/EOGEO 2005
Performance Test ResultsPerformance Test Results (Image Format)(Image Format)
• MapServer is slightly faster to return GIF
and PNG images. ArcIMS is faster for JPG.Image Format vs. Response Time
0
500
1000
1500
2000
2500
3000
JPG GIF PNG
Ima e Format
A v e r a g e R e s p o n s e T i m e ( m s )
ArcIMS
MapServer
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 16/23
Mapserver versus ArcIMS
www.refractions.net 16
MUM/EOGEO 2005
Performance Test ResultsPerformance Test Results ((ReprojectionReprojection))
• MapServer reprojects faster than ArcIMS.
Reprojection vs. Response Time
0
1000
2000
3000
4000
5000
6000
7000
8000
BC Albers (NAD83) La t-Long (NAD 83) UTM10 (NAD 83)
Out ut Pro ection
A v e r a g e R e s p o n s e T i m e ( m s )
ArcIMS
MapServer
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 17/23
Mapserver versus ArcIMS
www.refractions.net 17
MUM/EOGEO 2005
Performance Test ResultsPerformance Test Results (Feature Complexity)(Feature Complexity)
• Feature complexity affects both servers
almost equally (from ArcSDE).Feature Complexity vs. Response Time
0
5000
10000
15000
20000
25000
30000
4 8 16 32 64 128 256 512
Number of Points in Each of 10 000 Circles
A v e r a g e R e s p o n s e T i m e ( m s )
MapServer (SDE)
ArcIMS (SDE)
MapServer (.shp)
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 18/23
Mapserver versus ArcIMS
www.refractions.net 18
MUM/EOGEO 2005
Performance Test ResultsPerformance Test Results (Feature Density)(Feature Density)
• MapServer extracts features from SDE
slightly faster than ArcIMS does.Feature Density vs. Response Time
0
10000
20000
30000
40000
50000
60000
1 2 0 0
0
2 4 0 0
0
3 6 0 0
0
4 8 0 0
0
6 0 0 0
0
7 2 0 0
0
8 4 0 0
0
9 6 0 0
0
1 0 8 0
0 0
1 2 0 0
0 0
1 3 2 0
0 0
1 4 4 0 0 0
Number of Features
A v e r a g e R e s p o n s e T i m e ( m s )
MapServer (SDE)
ArcIMS (SDE)
MapServer (.shp)
Actually, this test demonstrates that MapServer (extracts features from SDE) +
(draws the map) faster
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 19/23
Mapserver versus ArcIMS
www.refractions.net 19
MUM/EOGEO 2005
WMS Standard ComplianceWMS Standard Compliance
• MapServer passed all 83 WMS CITE tests.
• ArcIMS (w/ WMS connector) passed 71 of 83 WMS CITE tests.
– ArcIMS failed 12 tests because:
• Wrong MIME type for some responses.
• Wrong “exception code” in some exceptions.
•MIME type of responses is typically expected to be
“application/vnd.ogv.se_xml”
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 20/23
Mapserver versus ArcIMS
www.refractions.net 20
MUM/EOGEO 2005
Ease of AdministrationEase of Administration
• MapServer administration benefits
– No need to reload services (when service fileschange)
– MapServer restarts faster (as fast as the web
server)
• ArcIMS administration benefits
– More granular control over log levels
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 21/23
Mapserver versus ArcIMS
www.refractions.net 21
MUM/EOGEO 2005
Interoperability and SupportInteroperability and Support
• Both WMSs support:
–ArcSDE 8.3 and ArcSDE 9.0
–Styled Layer Descriptor (SLD)
I don’t know how fully either server supports SLD, but our testing showed that
both support these basic features:
•Selecting and styling based on a attribute value
•Selecting and styling based on spatial location (within a bounding box)
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 22/23
Mapserver versus ArcIMS
www.refractions.net 22
MUM/EOGEO 2005
ConclusionsConclusions
MapServer is easier to administer than
ArcIMS.
MapServer is more WMS standardcompliant than ArcIMS.
MapServer matches or surpasses ArcIMSin most performance tests.
* (MSRM adopted MapServer for their
COINPacific web mapping application)
8/4/2019 Comparison of Mapserver vs ArcIMS
http://slidepdf.com/reader/full/comparison-of-mapserver-vs-arcims 23/23
Mapserver versus ArcIMS
f ti t 23
MUM/EOGEO 2005
Questions?Questions?
Contact me:
Brock Anderson
Refractions Research
www.refractions.net
(250) 383-3022