United Nations Statistics Division 1(31) 7_IT_Backlund_Paper Sten Bäcklund 10/07/2001 DESA/UNSD The role of IT in disseminating statistics. Focusing user needs and expectations Paper prepared for the “Seminar on user relations, marketing and dissemination of official statistics”, Vientiane 25-27 June Abstract In this paper we will make a non-technical approach to the present and future use of IT for disseminating statistics. We will focus on user needs and expectations and how this will decide on how dissemination strategies are formed and implemented. We will discuss recent IT practices for dissemination but also point at trends that will have an impact on how a statistical organization must adjust in the future. We will also concentrate on how the use of Internet techniques and methods can facilitate meeting public demands and what is needed to do so. Important notes : There are lots of hyperlinks referring to external web pages in this document. In order to use this facility you must be connected to the Internet. There are also internal hyperlinks referring to chapters in the current paper. Including links has intentionally been done to show how the use of advanced navigation may facilitate comprehension of disseminated information. You will need the latest Word versions to be able to follow the links. Normal reading and printing the document will not be affected. The printout format is set to US Letter. Changing the output format may of course result in unwanted text or diagram locations on the page. Do not confuse hyperlinks with emphasized text, which also is underlined. (On the screen hyperlinks are marked in blue).
The role of IT in disseminating statistics. Focusing user needs and expectations
Paper prepared for the “Seminar on user relations, marketing and dissemination of official
statistics”, Vientiane 25-27 June
Abstract In this paper we will make a non-technical approach to the present and future use of IT for disseminating statistics. We will focus on user needs and expectations and how this will decide on how dissemination strategies are formed and implemented. We will discuss recent IT practices for dissemination but also point at trends that will have an impact on how a statistical organization must adjust in the future. We will also concentrate on how the use of Internet techniques and methods can facilitate meeting public demands and what is needed to do so. Important notes : There are lots of hyperlinks referring to external web pages in this document. In order to use this facility you must be connected to the Internet. There are also internal hyperlinks referring to chapters in the current paper. Including links has intentionally been done to show how the use of advanced navigation may facilitate comprehension of disseminated information. You will need the latest Word versions to be able to follow the links. Normal reading and printing the document will not be affected. The printout format is set to US Letter. Changing the output format may of course result in unwanted text or diagram locations on the page. Do not confuse hyperlinks with emphasized text, which also is underlined. (On the screen hyperlinks are marked in blue).
Table of contents 1 Introduction.............................................................................................................. 3 2 Dissemination on the user’s terms – the user in focus ............................................. 5
2.1 Identifying users and stakeholders ................................................................... 5 2.2 Meeting the users’ needs and expectations ...................................................... 6
2.2.1 Provide quality data ................................................................................. 6 2.2.2 Present your information in a clear and easy-to-use way ........................ 7 2.2.3 Give your users a choice .......................................................................... 8 2.2.4 Recognize and adapt to new technologies on the user side ..................... 8
3.1.1 Conventional methods ........................................................................... 11 3.1.2 Using electronic media........................................................................... 11 3.1.3 Internet - a mix of both? ......................................................................... 11
3.2 Choosing the Internet as the main platform for disseminating statistics ....... 12 3.2.1 Deciding on an Internet policy............................................................... 12 3.2.2 Technology concerns ............................................................................. 14 3.2.3 The website and services offered ........................................................... 15 3.2.4 Analyzing information........................................................................... 18 3.2.5 Testing and monitoring .......................................................................... 20
3.3 Sharing information ....................................................................................... 20 3.3.1 Connecting offices and security. ............................................................ 20 3.3.2 What about XML? ................................................................................. 22
4 Additional remarks................................................................................................. 24 4.1 Legislation...................................................................................................... 24 4.2 Home pages of selected NSOs....................................................................... 24
References ...................................................................................................................... 26 Annex 1. Some indicators on how IT relates to quality................................................. 27 Annex 2. Conceptual firewalling ................................................................................... 29 Annex 3. The process of publishing .............................................................................. 30 Annex 4. Importance of web design features................................................................. 31
1 Introduction Identifying ways of marketing and disseminating information has always been important to statistical agencies. With the advent of the Internet it became possible to reach the public within a totally new framework and the technologically most advanced NSOs launched their first websites in the mid-nineties. Since then the Internet has grown beyond imagination and you can hardly attend any conference, seminar or workshop where not Internet solutions are discussed or promoted. In this context we will continue from the findings of two events held under the auspices of the UNSD and with a bearing on our further discussions on the combination of dissemination and IT. Referring to the conclusions from the UNSD Vienna seminar1 11-14 July 2000 directed to CIS countries we can identify some salient points:
q The importance of quality output in terms of relevance, accuracy and timeliness q Good media relations q Identification of important user groups q Pricing policy q Information sharing on good practices and examples
The seminar focused on reaching and servicing customers, addressing the needs of the private sector and on ways of approaching media and the public at large. The second ESCAP workshop on census data processing, storage and dissemination was held in Bangkok 27-30 March 20012. Of special interest here are item 4 of the agenda titled “Translation of data users’ needs into dissemination strategies” and item 5 of the same agenda “Innovative technologies for data dissemination technologies”. Papers were written and introduced by representatives from Australian Bureau of Statistics3,4 and Statistics New Zealand 5 while two country papers on census dissemination were presented (Viet Nam and Cambodia). From the vendor side Beyond 20/206, Space-Time Research7 and Statistics Sweden8 were invited to show their products.
1 Seminar on user relations, marketing and dissemination of official statistics. Summary paper. UNSD/UNFPA 2 The final report from the workshop is available at http://www.unescap.org/stat/pop -it/pop-wdt/wdt-rep.htm 3 S. Hardy. Maintaining relevance in an environment of change. Contributed paper. ABS. 4 S. Hardy. 2001 census dissemination: a world wide web transition. Contributed paper. ABS. 5 D. Archer. Responding to changes in users expectations. Contributed paper. Statistics New Zealand. 6 Go to http://www.beyond2020.com/ for more information 7 Space-Time Research, developers of SuperStar, visit http://www.str.com.au/index1.html 8 Information about PC AXIS can be found at http://www.scb.se/eng/databaser/ssd.asp (password on request)
The workshop recommendations were several and even if quite a few are relatively detailed the sole number shows how important the area is considered. The following list gives a snapshot of major topics on IT/dissemination and refers to the numbered paragraph in the final report:
q Use public domain software, mainstream solutions, off-the-shelf packages [76] q Start small, think big in Data Warehousing and On-Line Analytical Processing
(OLAP) [94] q Provide analytical flexibility, client customization [95] q Metadata management [97] q Low-cost GIS solutions [101] q User contacts/consultations [102] q Monitoring changes in user population [104] q Prototyping for user acceptance testing [105] q Attractive packaging, advertising, PR and other promotion [106] q Security concerns [107] q Link to other agencies in the region [108] q Adapt disseminating media to user/community profiles [109] q Internet is strategic [111] q House policy and guidelines [112] q Improve customers ability to self-service [118] q Monitor website traffic to identify key user groups [119] q Explore XML [125]
We will later on refer to the findings of the two meetings in their context9.
9 The reader is also referred to a contributed paper to the workshop (also available at the ESCAP website http://www.unescap.org/stat/pop -it/pop-wdt/pop-wdt.htm) that discusses what must be considered when designing and maintaining a website and how to deal with obstacles on the way. S. Bäcklund. Going on the net with your national statistics – what is there to consider?
2 Dissemination on the user’s terms – the user in focus 2.1 Identifying users and stakeholders Traditionally, main users of official statistics are considered to be different governmental bodies on central and local level, large businesses, regional or international agencies and research institutions. To some extent, this can be recognized as a result of how means have been allocated and channeled. But things are changing. That statistics are now being regarded as a public good, is partly an outcome from how statistics can be disseminated today as compared to yesterday since the presence of the Internet makes the difference. In this seminar we have seen how marketing strategies can be employed in order to identify current and potential customers and then reach them10. The role and influence from media and the public at large have also been discussed11. The government has been identified as a key user and stakeholder, while the importance of the private sector
has been stressed. Recent practices of dissemination in Asian countries have been presented12. Reading through the country reports to the ESCAP workshop five major user categories stand out by area of activity:
q All levels of government q International agencies q The private sector q Research institutions q The public
It is worthwhile noting that the media were never recognized as a user group. A different way of categorizing is suggested by Archer. He divides the users into:
1. General Data Users 2. Analysis Users
General data users are e.g. students, teachers, libraries and small businesses who have simple data requirements but from a great range of information. Needs are normally not known in advance. Analysis data users on the other hand are identified from their complex data requirements on detailed variable and regional breakdowns often based on many datasets. Such users are e.g. governmental departments, local authorities, researchers, VIP clients etc.
10 E. J. Spar. Customers: Introductory paper for the UN seminar on disseminating and marketing of statistical products in South East Asian Nations. 11 L. Ostergaard. Development and implementation of a media policy. 12 C. S. Marzan. Discussion paper on assessment of the ASIAN practices on dissemination.
Another grouping is suggested by Blanc et al13 in a recent paper to the Quality Conference in Stockholm in May where they classify users by type of demand according to table 1:
Table 1. Public demand vs. private demand in official statistics public/social demand individual demand
“citizen”, “society” “customer” long-term demand short-term contract expressed by the political representatives free bargaining (market) socio-political dialogue; government setting priorities contract between NSI and individual partner terms are not (explicitly) specified terms are explicitly specified
At the same conference Linacre14 makes distinction between
A. Members of the public, students B. Individual organizations who seek a tailored service, e.g. on a commercial basis,
or as members of the media etc. C. Sophisticated users, e.g. researchers, financial analysts, analysts in policy
departments D. Key users: Central banks, Governmental bodies, International agencies
(This is also is similar to the Archer classification). Main clustering characteristics are then the level of statistical capability, level of interest, ways of access and partnership willingness. But in whatever way we choose to describe and classify our beneficiaries, the objective is to meet their needs and expectations in the best possible manner, not only once but recurring. This is also the IT challenge – how we will set about to use IT in order to reach our goals.
2.2 Meeting the users’ needs and expectations
2.2.1 Provide quality data As has been emphasized earlier in the seminar, users expect quality information. If this cannot be provided the user will certainly stop asking for your data trying to find it elsewhere. A well functioning IT environment will without question make things easier. Quality is normally15 defined in terms of accuracy, relevance, timeliness, coherence and availability without specific order. Every now and then there will be conflicts between factors. It is not always easy at the same time to provide for accuracy and timeliness. Another example is coherence and relevance: it would be convenient if corresponding statistics from different surveys
13 M. Blanc, A. Desrosières, W. Radermacher, T. Körner. Quality and Users. Contributed paper to the final LEG report 14 S. Linacre. Understanding users and managing quality in a statistical agency. ONS, UK 15 The European Statistical System ESS lists Relevance, Accuracy, Timeliness, Accessibility, Comparability, Coherence, Completeness
equals or that aggregated monthly estimates and annual statistics coincide but this is rarely the matter due to variations in definitions and methodology. In Annex 1 some underlying factors related to IT are laid out, that often are at hand in less developed or emerging countries.
2.2.2 Present your information in a clear and easy-to-use way There are certain important aspects of disseminating information that are valid and non-depending on the media of choice, may it be hardcopy, CD or the Internet, in order to meet the user’s expectations. One of the tools for doing so is to provide for a good navigation system. If we use a statistics website as an example, this could be accomplished through standard navigation for the general public and through a set of alternate navigations that are available from a user’s perspective catering to various target audiences. Most advanced are enterprise portals that are database driven and often expensive to implement. Likewise a retrieval application should preferably accompany data disseminated on CD ROM. Statistics Sweden distributed its 1990 census tables on CD at the same time providing the first PC AXIS version as explorative tool. Often a GIS application for mapping statistics will be included. And even a paperbound report or a statistical yearbook should at least come with a table of contents, search index, table and diagram references. There are other matters relating to how information is packaged. Going back to a statistics website you might compare a first time user with a person visiting a store. You would certainly want him16 to look at what products you have to offer, walk through the shop, buy and in the end most important – come back a second time! Thus, try to locate what you want to trade as high in the navigation structure as possible and highlight news and press releases – keep your web site alive! Place related information close together or on the same navigation level and provide a site map. Users want links to related information and you should give the links but at a reasonable level. Sometimes you may find (scientific) websites overloaded with links to other sources of information and the risk is high that the user will wander away in cyberspace not returning to the site from where she started.
16 In order to facilitate writing, he and she, his, him and her will be alternately used instead of the semantic construction he/she, his/her to avoid making the text a gender issue.
2.2.3 Give your users a choice Taking all potential customers of statistical information into consideration we must remember that they constitute a heterogeneous group from e.g. governmental bodies to enterprises, researchers and the public. This means that any NSO must be prepared to offer optional ways of retrieving information. Preferred media Even if the web already is the major dissemination alternative for many agencies there are still users who will prefer hardcopies, CD ROM or even old-time floppies. Till date national libraries and archives often have had specific requirements on the delivered media e.g. microfiche, tapes or MO disks. Information exchange over computer networks could also call for special handling in accordance with agreed-on protocols. With no access to the WWW the user may still benefit from other Internet services e.g. SMPT for receiving information via electronic mail. In the end this is one of the factors that the NSO must keep in mind when designing, evaluating or changing its dissemination strategy. Print-on-demand Many users do not want to or need to have all the information that comes with a product. Some just want to receive a summary or a set of basic tables while others have more specific demands. Instead of having to buy or download e.g. a complete report or statistical yearbook the user may prefer to specify want he wants and then have it printed either by the agency and then mailed or downloaded and printed by the user himself. Statistics for further analysis Researchers or planners are often interested not only in macro data but also in the underlying micro information. Provided that measures for ensuring integrity are taken especially when dealing with sensitive data, these data could be disseminated through electronic networks e.g. the Internet. Additional security may then also be needed e.g. SSL encryption to prevent unauthorized access. The user should also be given the opportunity to specify preferred data formats17 at the time of delivery to facilitate further analysis.
2.2.4 Recognize and adapt to new technologies on the user side New technologies are emerging all the time. One of the latest trends in disseminating data is by Instant Messaging systems based on e.g. SMS18 and WAP19. Many statistical
17 This includes e.g. common database formats (dbf, mdb), worksheet formats (xls, wk*), formats of statistics software like SAS (sd2) or SPSS (sav) or ASCII 18 SMS (Short Message Service) is available on digital networks allowing messages of up to 160 characters to be sent and received via the network operator's message center to your mobile phone 19 WAP (Wireless Application Protocol) is a free, unlicensed protocol for wireless communications that makes it possible to create advanced telecommunications services and to access Internet pages from a mobile telephone. WAP is a de facto standard that is supported by a large number of suppliers. See also http://www.wapforum.org
organizations are already using these methods to disseminate business critical information to the private sector and the media e.g. price statistics or financial markets indicators. So now if the user community wants these services (which you will know from your focus groups or user contacts) you should be prepared to provide for them. And in the future, solutions based on advanced and high-speed connection (GPRS20, HSCSD21, Bluetooth22) will grow in importance.
20 GPRS (General Packet Radio Service), a packet-linked technology that enables high-speed (115 kilobit per second) wireless Internet and other data communications 21 HSCSD (High Speed Circuit Switched Data) is a circuit-linked technology for higher transmission speeds primarily in GMS systems 22 Find out about Bluetooth at http://www.bluetooth.com
3 Agency concerns 3.1 Outlining a dissemination strategy On the assumption that current and potential stakeholders and users are identified, in outlining a dissemination strategy for a NSO the following questions must be asked:
A. What type of information should be provided? Examples: data, metadata, administrative information, methodology, statistical activities, research results
B. What are the possible means and ways of dissemination? Examples: paper, hardcopy, diskettes and the like, CD, web pages, traditional mail, electronic mail, discussions, work shops, seminars, conferences, bi- or multilateral projects, networks
C. What will be the interventions and what are the assumptions? Examples: website creation, equipment and application costs, maintenance, co-ordination of activities, security and legislation
Chapter 2.2 on users’ needs and expectations forwards the importance of quality information and how data should be presented in a clear way and of the users choice. Based on the country reports on census activities to the ESCAP workshop the following simplified “translation” of users’ needs into possible dissemination strategies was made:
Figure 1. Outlining dissemination based on country reports on census activities
User groups are dimensioned the traditional way, user needs relate to census information, primary data are obtained through censuses and surveys and finally contents and media are linked together. Any dissemination strategy could then be considered as a realization of a subset of users, needs, contents and media, and with the time dimension added. If the NSO consider its web site as the primary target for disseminating activities, as is the case in many countries, users will naturally be categorized in accordance with what has been discussed earlier in chapter 2.1.
3.1.1 Conventional methods We will not elaborate on conventional methods for disseminating statistical information since this will be out of scope for this paper. It should be mentioned though that most NSOs have in the past installed printing offices to provide for printed output e.g. reports, press releases, yearbooks etc with often high costs for acquisition and maintenance of equipment. Today the needs for in-house publishing capacity are diminishing since there are dissemination alternatives now available that were not at hand in the past. Outsourcing printing when called for has also often proved to be cost effective. Other ways of disseminating information have been and still are workshops, seminars and conferences as well as through regional and interregional projects.
3.1.2 Using electronic media Electronic media have been a first-hand dissemination choice for a number of years already. We could say that this was becoming a true alternative with the introduction of personal computers and networking even if data had been transferred via magnetic tape between mainframe installations from the 1950s. Exchanging data on floppies (5 1/4 “, 3 ½ “) and in the most popular PC formats became common during the 1980s. In 1989 a process was developed with which a CD can be directly written by means of a laser beam. The way for the self-creation of CDs was opened. The CD is an ideal medium for distribution of information of any kind (text, images, sound, data and programs). Equipment for burning CD ROM is getting more and more inexpensive and this method is therefore within reach even for NSOs on a very low budget and for customers alike. Along with relatively low production costs, it provides high memory capacity with direct access and a long life span, and is still without competition, even when you only have low production numbers. (For this reason, a majority of all PCs today are already equipped with a CD-ROM drive). Thus CDs are frequently used for dissemination of statistics since not only (compressed) data can be stored but also retrieval applications e.g. for tabulation, drill-down analysis and GIS.
3.1.3 Internet - a mix of both? In the mid-nineties national statistical agencies started to create their first websites. In the beginning, and with the still undeveloped software available, the ambition was often limited to “be noticed” and this was accomplished through static HTML pages
organized in strict hierarchical navigations structures. Pages were often arbitrarily updated except from statistics that were published on a regular basis e.g. price indices or short-term business indicators. One common characteristic was that the information provided on the web site was primarily what had been decided by the producers – the website was not user driven in the first place. The reasons behind were often scarce staff resources and a “wait-and-see” view on the new technique. With the unprecedented growth of the Internet during the last half-decade it has become obvious that it will be the main channel for dissemination for any statistical agency. The Internet will encompass both the conventional methods and the new ones and will change the strategies of dissemination in a substantial way23.
3.2 Choosing the Internet as the main platform for disseminating statistics
3.2.1 Deciding on an Internet policy
3.2.1.1 What should be part of an Internet Policy? It must be remembered that establishing and maintaining a website as the main dissemination platform for a statistical agency will be a long-time undertaking. It is therefore important that an Internet Policy is outlined well in advance of implementing any kind of web infrastructure. First, we refer to chapter 3.1 where the dissemination strategy is discussed. The answers to questions A (on contents) and C (on interventions and assumptions) will form the cornerstone of the agency’s Internet policy. In a next step security issues should be addressed. You should not go for website hosting prior to a thorough security assessment since this will decide on e.g. if the hosting should be outsourced with an ISP or done in-house and what security layers should be implemented in order to secure your data. Following these steps a plan for implementing the website should be designed. It will now be decided what resources will be needed, how funds should be provided etc. Furthermore you must decide on how the website support should be organized and how staff training should be done. Finally production rules must be set wherein updating, upgrading and maintenance are covered.
3.2.1.2 A dissemination strategy – the case of Statistics Sweden in brief In the end of 1999 a board decision was taken, stating that from 2000-04-01
q All official statistics produced under the responsibility of Statistics Sweden should be stored at macro level in Sweden’s Statistical Databases. These statistics should then be made available to the public. One of the information channels for this purpose should be the Statistics Sweden website
23 It should be noted that electronic networks have been available before the WWW emerged
q All official statistics under the responsibility of Statistics Sweden and printed in Statistiska Meddelanden (a series of reports) should be available on the website. They should be standardized by the use of templates and print-on-demand should be offered (through download over the web).
q All other press releases and a set of most wanted statistics should also be published on the website by subject matter area/product in a standardized form
q Under condition of the above, official statistics may also be disseminated in publications and yearbooks
Statistics Sweden’s Internet Policy sets out to achieve these goals by returning a dynamic website and engaging staff at all organizational levels in keeping the site up-to-date. Strong measures have been taken in order to secure the in-house environment and the statistical databases. Extensive training schemes have been introduced and the agency applies mainstream technology and methods. See also chapter 3.2.3.4.
3.2.1.3 Contents What should be published on the website is first and foremost an issue for the top- level management of the NSO even if e.g. the subject matter departments will be the executing units. It must also reside with the leaders to decide on data integrity, security matters related to data transfer and what rules to be followed while authorizing the site.
3.2.1.4 Infrastructure The infrastructure for the website is to a great extent dependent on what your ambitions are. In a modest situation you might well be satisfied by outsourcing everything with an ISP only retaining the control over updating the website remotely. This is not uncommon for small agencies that cannot afford neither the equipment (server, firewall) or the backup taking and 24-hour availability. In any case before starting to establish a complete in-house environment for your website you should always make a cost-benefit analysis not excluding such important things like the needs of redundancy (e.g. backup servers) and staff competence.
3.2.1.5 Security Security is always an important concern. It covers to what extent you are prepare to retain data integrity e.g. by ways of implementing redundancy and firewalls. A risk analysis should always be done in order to identify steps needed in preventing unauthorized or unintentional data damage.
3.2.1.6 Training A staff development program should be established covering how to implement and use mainstream software for creating web pages and multi- layered web applications. Even with apt staff, you cannot lay the responsibility on them to learn things on their own. Commonly those involved are occupied with day-to-day work and you need to provide the time needed to attend courses, workshops or seminars.
3.2.1.7 Sustaining a quality website Without strict rules on how the website should be designed and maintained e.g. what software should be used or what standards should be applied there is a great risk that it
will deteriorate quickly. It is therefore important that there is a tight website management with restrictions on what is allowed and by whom. In most cases the webmaster or web coordinator will be solely responsible for what is published on the site and how it is done. To their assistance you might want to attach experts in different areas e.g. language, design and layout, multi-media and application developers. Steering groups or committees will also be needed to decide on contents and to guide the webmaster in her work.
3.2.2 Technology concerns
3.2.2.1 Hosting the site The Internet Policy will point out in which way the website should be hosted. As already mentioned the site may well be outsourced with an Internet Service Provider (ISP) if this meets the dissemination strategy needs, the level of security and possible cost restrictions. There are also a number of free web servers available on the net that provide limited disk space and some facilities to create dynamic web pages with extracts from databases. In any case you will need an ISP if you intend to host your website with own equipment in-house. There are of course a number of initial steps to be taken. You will e.g. have to implement the Internet Protocol (IP) in your LAN, since you will have to register an IP network and a DNS domain name in order to access the web. Virtual hosting can be an important add-on alternative when launching web driven multi- layered applications where e.g. electronic forms are used for collecting statistical data from respondents like administrative units, schools or businesses or where results/analyses from surveys, which are not part of the officially published statistics, are disseminated to designated users e.g. government bodies.
3.2.2.2 Established products As a general advise you should always choose mainstream products when you implement your website. There are no good reasons to go for cutting-edge technology or inexpensive but unstable solutions. For hardware this implies brand servers and workstations for web developing, testing and launching the site. You should choose well-known firewall and proxy techniques. You should also study how other NSOs have done especially in your region. The same principle holds when you are deciding on web enabling products. If you are already on the Windows platform the natural choice would today be Windows 2000 Advanced Server that contains the components needed. If you are on UNIX or LINUX platforms other products will normally be first in line. Likewise you must make sure that your database solutions will fit into your concept. You will also need software for web design and creating web pages. There are a number available, from simple HTML authorizing tools to Adobe Dreamweaver or Microsoft FrontPage. Still, the ambitions of your dissemination strategy as expressed in you Internet Policy should be able to guide your way.
3.2.2.3 Securing the environment In Annex 2 the principle behind securing the Statistics Sweden’s website is shown as an example. A firewall is implemented to protect the LAN from unauthorized access resulting in the formation of three security layers that are physically separated:
1. The unsafe network or what is here the same as the Internet 2. The services network located behind the (hardware) firewall 3. The safe network which is the corporate LAN
All incoming traffic from the unsafe network will have to pass through the firewall. On the services network, all Internet services offered are located e.g. HTTP, SMTP etc. There is no way for a visitor to reach the safe network. By the same token in-house staff cannot compromise resources on the services network since they are only accessed through dedicated applications that often are automated In reality the situation is more complex. So-called virtual private networking (p. 22) is implemented in order to allow staff to access the intranet from the unsafe network. All traffic to the official website must also go via the firewall.
3.2.2.4 Site testing Put your site under stress! Establish a schedule for testing your site and deal with latent risks before unwanted problems arise. There are third-party providers who can be contracted if you want to and can afford outsourcing these services.
3.2.2.5 Monitoring activities Keep track of the events occurring on your web site! Monitor activities and obtain data on hits from your ISP. Your own access log file can be very useful to you when identifying your clients and which pages they visit. Especially subject matter departments show a great interest in how their statistics are used, not only in the number of successful hits but also by whom and how often he returns. Try to find out if you are really providing what customers want or if in some cases it would be sufficient with e.g. automatic e-mail or other solutions. To analyze the log files you don’t need any fancy programs since log data comes as plain text and you can use your favorite statistical software to process them. But remember that log files occupy disk space. If you want to archive log data you should first compress it using Winzip or any other convenient software for this purpose.
3.2.3 The website and services offered
3.2.3.1 Web site design In designing your website you will have to consider who your intended users are and adapt to their needs. This will hopefully result in providing your visitors a nice portal to your organization. Virtual websites as described earlier may be one of possible solutions for meeting special demands. Statistics Sweden as an example provides virtual hosting of websites to meet specific target groups.
Figure 3 is borrowed from the Yale Style Manual24. Here users’ needs are classified in terms of “complexity” and “linearity” and then mapped to navigation structures. According to this you will have to design your structure(-s) in a way that best corresponds to the characteristics of your user groups, giving everyone what she wants being the ultimate goal. It is of course important that you also try to standardize your web pages to conform to layout decisions. There is a learning situation with all websites. The user will appreciate if your navigation structure remains unchanged and that he can expect related items to be found in similar locations throughout the site and over time.
Figure 3. Mapping users by “complexity” and “linearity” to navigation structures
Annex 4 gives the results from a recent survey on visitors’ ratings of welcome page features by Knowledge Systems and Research Inc. and published in the June issue of Internet World. Strikingly, ease of use and navigation is considered most important by 80% of the respondents.
3.2.3.2 Static or dynamic HTML Dynamic web pages are often server-script based using CGI, ASP, JSP25 or other methods. The problem is that e.g. ASP-generated HTML26 code contrary to static HTML does not cache. This means that whenever a client returns to a previous ASP page it will be re-created instead of returned from the client’s cache or the proxy. This will take time and if too many concurrent users are addressing your site, it may occasionally break down.
24 P. Lynch, S. Horton. Web Style Guide: Basic design principles for creating web sites. Chapters are also available on http://info.med.yale.edu/caim/manual/ 25 CGI – Common Gateway Interface, ASP – Active Server Pages, JSP – Java Server Pages 26 HTML – HyperText Markup Language
The right balance between static and dynamic pages should be established. It is also important to strip generated HTML code from all unnecessary formatting in order to speed up the traffic.
3.2.3.3 Downloads A vital part of your services should provide optional downloads of text information or pre-processed data e.g. tables. The best way is to use the PDF 27 format for text, which will retain all formatting within the document. Other popular choices are Word documents and Excel worksheets. If you use in-housed developed software for data retrieval or tabulation you should always make sure that output formats could be read or converted. Also remember to give file sizes explicitly and next to download buttons so the user can estimate download time and if it is worthwhile to go through this procedure.
3.2.3.4 The process of publishing Annex 3 is cut from a Statistics Sweden internal unpublished paper and shows how the agency organized its publishing based on the decision taken in late 1999. To begin with all statistics produced are stored into databases. Thereafter documents are created in various formats using different tools (e.g. dynamic database retrieval), converted to suitable formats depending on usage, disseminated and archived.
3.2.3.5 Instant messaging Instant messaging (IM) is becoming an essential form of communication, for both social and professional exchange and according to recent statistics for over 140 million Internet users. In chapter 2.2.4 the notion was introduced and some implications described. With IM you can
q target information to specific users, or to groups of users - based on their preferences
q promote your products q alert users when updates of specific web pages are available
In societies with an established infrastructure for electronic communication, IM may well be considered a strategic issue for any NSO. It should be kept in mind that IM services may be outsourced thus eliminating the needs for an in-house IM application development capacity.
3.2.3.6 The importance of metadata Establishing an inter-mediate layer of metadata is an essential pre-condition for dynamic retrieval of macro (or micro) data over the web. Users or visitors to a statistics website will search for information by choosing from lists of indicators and on information about those indicators. They will request tailor-made tables and graphs based on metadata.
Metadata will also make it easier to shield the underlying (primary) data from damage since they will only be accessible through applications. Another feature of metadata is that they will be needed when creating data warehouses. As an example PC-AXIS, which is jointly developed by the Nordic countries and used by many statistical agencies for disseminating statistics, is metadata driven.
3.2.4 Analyzing information
3.2.4.1 Establishing a Data Warehouse for OLAP The concept of Data Warehousing has been around for quite some time. There is a well-known and often quoted statement by R. D. Rogers28: “We’re drowning in information and starving for knowledge”, that would justify DW and at the same time puts it in a nutshell. DW is founded on relational database theory and is normally delivered as a “black-box” containing tools for creation, administration/maintenance and output. High quality metadata is one of the pre-requisites. Unfortunately it has been and is still expensive to set up a DW for exploratory analysis and data mining and this may be the reason that only the most well situated NSOs have been able to do this. A DW is by definition structured data optimized for storage while the corresponding term Data Mart refers to data that are optimized for output. Often they are treated as one concept and we will only use DW in that sense. Hence the main purpose of a DW is to
q Join or merge data in advance from a large number of underlying sources instead of doing this at run-time
q Provide an environment for data mining, i.e. searching and identifying patterns that are not otherwise readily observed through easy-to-use statistical analysis and reporting techniques in a client/server or tiered enablement
q Facilitate responding to ad-hoc requests from end-users and customers preferably over the Internet
A web enabled DW for OLAP could be described visually in accordance with figure 5. Sometimes you would also include the underlying sources in the DW.
28 R. D. Rogers (Yale Librarian). NY Times 25 February 1985
Figure 5. Implementing a web enabled Data Warehouse
Data
Data
DataDATA WAREHOUSE
Relational Data
Admintools
DATA Metadata
Web browser
Internet
OLAPtools
In the ESCAP workshop a paper was presented by Lim29, Statistics Singapore, that related to census data. In the paper he differs between a conventional DW and a statistical DW. In particular, a statistical DW will be used by professionals, since the key purpose is exploratory data analysis and it should thus provide maximum flexibility. Interesting trends in data warehousing is that the latest versions of RDBMS systems e.g. Oracle, Sybase or MS SQL Server have built- in facilities for web driven OLAP (On-Line Analytical Processing) applications, where users can utilize the Internet to retrieve and download key information on-the-fly from large databases in common formats for further analysis. Statistics software vendors SAS and SPSS have since long offered DW/OLAP solutions. There are also in-house benefits to consider:
q Common data definitions are encouraged q A single point of control q Metadata layers are established q Rapid OLAP application development could be done
Therefore at least an assessment of advantages and disadvantages of implementing a DW should be part of a statistical agency’s dissemination strategy.
3.2.4.2 GIS GIS software and applications are often key tools in communicating information. There are lots of vendors today targeting specific user groups e.g. interactive map creation/display over the web for monitoring traffic or for locating shops and stores in the cities.
29 E. Lim. Setting up a Data Warehouse – Salient points for consideration
Many NSOs have developed their own GIS software in order to meet their specific demands while others rely on off- the-shelf products like MapInfo30.
3.2.5 Testing and monitoring
3.2.5.1 Acceptance testing Whether you use “old-fashioned” media for dissemination or the Internet you still must make sure that you actually provide what the user wants and in a way that is suitable for him. It is often a good idea to test your product with a small group of intended users or stakeholders to determine what adjustments need to be done before launching in a larger scale. This may be easier said than done since end users only tend to materialize when there are complaints to be made. But if a good user/producer situation is established, acceptance testing should be encouraged and experiences from this activity will be of great future value.
3.2.5.2 Monitoring user community In chapter 3.2.2.5 the importance of logging the website was stressed. In this way you will also be able to some extent identify user segments by the way they move around your site. But the website should itself provide valuable information. It is very easy to design a user questionnaire for real-time response31. It may even at times be made mandatory to fill-out for the right to access and use a statistics product. You should always try to have an on-going communication with your key users since at the end of the day good user relations is what counts.
3.3 Sharing information
3.3.1 Connecting offices and security. An important part of your dissemination strategy will be how (internal) information are communicated to close partners. There are a number of ways in how branch offices can be connected to the head office, e.g. provincial statistical units to the NSO or how the NSO itself can connect to others e.g. governmental bodies or the central bank. We will here discuss the most common at present. Using Dial-Up-Networking (DUN) Remote access of different kind to branch offices have since long been used for transferring data. The requirements are modest. A stand-alone modem linked PC on one hand and a receiving/transmitting server (using a modem pool) on the other are sufficient. DUN will normally provide for RAS validation and gaining access to and browsing network resources through some login procedure. There is a security problem since most communication is performed over ordinary phone lines with none or limited authentication and encryption. (In Windows NT/Windows 2000 this is a facility that comes with the operating system).
30 Visit http://www.mapinfo.com/ for more information or go to http://www.spss.com/map/ to see how it is liaised with SPSS 31 Go to the Statistics New Zealand website to see how it works: http://www.stats.govt.nz/domino/external/web/feedback.nsf/feedback?OpenForm
Using Internet Services Transport protocols – HTTP and FTP The Hyper Text Transport Protocol or HTTP is used for exchanging data over what we in everyday terms call the Internet or WWW. It is via HTTP that web pages are requested and retrieved. The File Transport Protocol, FTP, is preferred if you are interested only in down- or uploading files since this normally faster and more reliable. A FTP server is typically installed at the same time as the web server. Enabling SSL32 or IPSec33, data can be encrypted prior to transmission. It is also possible to enforce authentication/authorization through different types of certificates. Electronic mail – SMPT Simple Mail Transfer Protocol is another Internet service that is provided for moving mail from one host to another. In order to connect offices by using SMTP you will need hardware on which mail services can be installed e.g. Microsoft Exchange Server. Mail clients e.g. workstations will also need software to administer in- and outgoing mail e.g. Microsoft Outlook.
Figure 6. Using SMPT for exchanging mail over the Internet
Internal Network
External Network(Internet)
ApplicationSMTP
MailServer vM̀ail
clients
Note: An application proxy should always be implemented to protect the internal network even if this does not show in this figure.
32 Secure Sockets Layer, protocol developed by Netscape for encrypting TCP/IP transmissions, used in e.g. Netscape Navigator and Internet Explorer 33 Internet Protocol Security, developed by the Internet Engineering Task Force (IETF) for encrypting messages transported over the Internet, often used in Virtual Private Networks (VPN)
Establishing secure WANs. Virtual Private networking A Wide Area Network or WAN is logically (and often technically) a concept for connecting different Local Area Networks (LANs). A WAN based on TCP/IP can be considered a corporate intranet and Internet techniques and methods can be applied. Virtual Private Networking (VPN) technology allows an agency to connect to branch offices or to other agencies or organizations over a public IP network like the Internet, while maintaining secure communications. To the user VPN is a “point-to-point” connection and how it works behind the scene is to most users irrelevant. The main advantage is that you only have to connect to local ISPs to establish VPN thus reducing the need of e.g. modem pools and remote dialing. VPN should therefore be considered as the major alternative for data exchange over long distance
3.3.2 What about XML? The interest in XML or eXtensible Markup Language is ever growing. Since it presents a standardized way for storing and delivering highly structured information over the web the implications on data interchange are apparent. XML’s structured syntax lets you describe virtually any type of information—from a simple recipe to a complex business database—and sort, filter, find, and manipulate that information in flexible ways. It separates data and metadata and is excellent for archiving information during long periods of time without loosing the possibility to recover the data at any specific point. The latest versions of databases include XML as an option, the latest generation of browsers provide XML parsing, and statistical software giants like SAS and SPSS supports it. As an example, among SAS programmers, the number one reason to use XML is importing XML-formatted information into SAS datasets. With systems receiving more and more information in XML format, SAS programmers will use this facility to access and analyze it. The Meta Group 34, one of the major analysts, predicts a continuous increase in the worldwide use of XML. It will become the “conventional syntax for application development within a year and the primary business-to-business Internet interaction within two years”.
4 Additional remarks 4.1 Legislation In a number of countries lack of adequate legislation may prevent or at least hamper web enabling of official statistics. This situation was recognized in the UNSD Vienna seminar but no recommendations could be formulated to address and possibly solve the problem.
4.2 Home pages of selected NSOs For this chapter the home pages of four statistics offices will be examined and we will locate things that are meant to serve many of the purposes that have been discussed. The sites have been chosen without any other objective than for demonstration. As a comparison the UNSD home page is also selected. The welcome page
Some statistics Table 2 contains some facts of the welcome pages:
q number of Page Down needed to see the whole page with different screen resolutions
q approx. no. of objects that can be selected/pushed q the size of the page in bytes
The plus (+) sign indicates that an extra PgDn is needed to read (a few) lines at the end of the page.
Table 2. Basic welcome page characteristics for selected NSOs
Agency No. of PgDn (800x600)
No. of PgDn (1024x800)
No. of objects that can be pushed (buttons, pictures, text etc)
Page size
UNSD 2+ 1+ 19 15725 Indonesia 1 0+ 22 13273 Singapore 2+ 2 48 25602 New Zealand 0+ 0 30 18766 Statistics Sweden 1 0 24 12079
What can be noted is that all pages are relatively small and therefore fast to download even via a slow modem; that the designs differ to a great extent in how much is stuffed into the page and how much you then have to scroll to see the whole page. There are
http://www.singstat.gov.sg Statistics Singapore http://www.stats.govt.nz Statistics New Zealand http://www.bps.go.id Statistics Indonesia http://www.scb.se/ Statistics Sweden http://www.un.org/Depts/unsd/ UNSD
studies36 that show that only 10 % of first time visitors ever scroll beyond the top of web pages. There are lots of articles and books written on web design that cover most of the topics that relate to user habits and expectations. Here we will just comment on a few that could be of interest for statistical agencies. Features and utilities Some of the most important features are a site map, a feed-back option and a search facility which allows searching the site by keywords and/or text strings or searching the layers of metadata. If you provide file downloads using the PDF format you should also give the user a chance to download the free Acrobat Reader if he wants to. What’s new? Most probably your visitor comes to get the most recent information on your statistics. This means that you should up-thrust news in your navigation structure and locate news links at the top of the welcome page. Contacts Your visitor will also need to know how to contact you. Therefore you would locate primary contact information on the welcome page e.g. an e-mail link to your webmaster and a link to a page with more comprehensive contact and feed back possibilities. It is also a good practice to give contact information on your product pages. Job opportunities Why not give your occasional visitor a chance to get a job within your agency? She may actually have gone to your site in order to look for exactly that kind of information. It will also save you money since you won’t have to rely on advertising or other channels – just provide your link where you might think that potential future employees would look. Courses, seminars Your welcome page gives a golden opportunity to promote your activities and your statistics to key users and stakeholders by inviting them to courses and seminars. Links to other websites Links to other websites are useful, especially if they identify your partners or points to other statistical sources within the country. Don’t put to many links on the page. The user’s temptation to follow a link may actually conflict with your more basic interests – to keep him as what is often called “a stayer”. Trivia? Why not lighten up your welcome page with something that encourages your occasional visitor to return? As an example Statistics Sweden (in the Swedish Edition) offers a database of all first or last names in the population by year and sex (lots of parents come) and a calculator that discounts prices by index (find out what you really paid for your old Toyota in current prices).
36 J. Nielsen (1996). Top Ten Mistakes in Web Design. Sun microsystems. http://www.sun.com/columns/alertbox/9605.html
References A complete set of references is given in the footnotes. Below you will find a list of authors and software vendors.
Authors D. Archer (2001). “Responding to changes in users expectations”. Paper to the ESCAP workshop. Statistics New Zealand. S. Bäcklund (2001). “Going on the net with your national statistics. What is there to consider?” Paper to the ESCAP workshop. UNSD. M. Blanc et al. (2001). ”Quality and users”. Paper to the Stockholm Seminar on Quality. INSEE. S. Hardy (2001). “Maintaining relevance in an environment of change”. Paper to the ESCAP workshop. ABS. S. Hardy (2001). “2001 census dissemination: a world wide web transition”. Paper to the ESCAP workshop. BS. E. Lim (2001). “Setting up a Data Warehouse – Salient points for consideration”. Slides. ESCAP workshop. Statistics Singapore. S. Linacre (2001). “Understanding users and managing quality in a statistical agency”. Paper to the Stockholm Seminar on Quality. ONS, UK. P. Lynch et al. (1999) ”Basic design principles for creating web sites”. Yale University. C. S. Marzan. “Discussion paper on assessment of the Asian practices on dissemination”. NSCB Philippines. J. Nielsen (1996). “Top ten mistakes in web design”. Sun microsystems. L. Østergaard (2001). “Development and implementation of a media policy”. Contributed paper. Danmarks statistik. E. J. Spar (2001). “Introductory paper for the UN seminar on disseminating and marketing of statistical products in South East Asian Nations”. US Bureau of Census.
Seminar and workshops Seminar on user relations, marketing and dissemination of official statistics, Vienna 11-14 July 2000. Summary paper. UNSD Workshop on census data processing, storage and dissemination, Bangkok 27-30 March, 200. Final report.
Annex 1. Some indicators on how IT relates to quality
QUALITY FACTORS
INDICATORS
Accuracy/ Completeness Reliability Relates to IT mainly in the way of how data are collected, edited and stored
• Variables included in survey questionnaires are left unattended and only major indicators are computed. This results in loss of information that could be used for e.g. within-record comparisons or later macro editing or analysis.
• Short planning perspective regarding designing or changing surveys does not give enough time for testing e.g. the questionnaire through pilot studies.
• Since statistical information often is collected from administrative aggregate sources compiled by e.g. line ministries, NSOs have limited ways of controlling the quality of the underlying primary data which in turn will affect final reporting. This fact makes it almost impossible to do in-depth statistical analysis of data. The same might be true for in-house reporting systems where they rely on aggregate reporting from underlying bodies e.g. provinces and districts.
• Unintentional modification/destruction of data may occur when there is a lack of good data management e.g. different versions of the same data are used for editing without coordination.
Timeliness Relates to IT in how data are captured, edited and stored into ready-to-go statistical databases and in what way these data are later processed and disseminated
• Staff shortage in certain sectors means a heavy workload on a few employees, which in turn will affect timely reporting.
• Financial constraints limit the staff number in different departments, units, divisions or sections. Other budget considerations hamper computer equipment acquisitions and training in IT areas.
• Statistical data are normally stored in computer data files residing on different media instead of central databases.
• Output databases are not created since no deadline for data editing is given.
• Lack of application developers and suitable software makes it difficult to process and disseminate data in a timely manner.
Relevance/ Contents A good IT environment in itself cannot provide for relevant statistics but it can make life easier for statisticians.
• NSOs are not using channels of exchanging information to the extent that could be done e.g. the Internet for sharing experiences and best practices
Annex 1. Some indicators on how IT relates to quality (cont.)
QUALITY FACTORS
INDICATORS
Availability/ Interpretability Accessibility Relates to a great extent to existing IT infrastructure and how it has been implemented
• Output databases are not available, instead input databases are used for tabulation and analysis
• Dissemination plans cannot be met due to lack of competence • Ad-hoc self service is not available i.e. doing your own analysis
on data
Coherence/ Comparability Consistency Relates to definitions, to what data are collected, how it is stored and if time series can be created
• Data from different sources are often used to describe and analyze a statistical problem. Basic definitions e.g. object, population, variables, statistics and time references should then be the same in all sources. If not the outgoing statistical quality will suffer and in worst cases make future analysis impossible.
• Time is an important factor. When definitions change over time depending on end users’ conflict it will affect consistency
• Databases containing similar information are not comparable from the use of different models or RDBMS systems
