+ All Categories
Home > Documents > Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target...

Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target...

Date post: 07-Sep-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
33
Results of the CAMS User Satisfaction Survey 2019 January 2020 Issued by: Data Support Team, ECMWF Date: 30/01/2020 REF.: CAMS User Satisfaction Survey 2019 report final.docx
Transcript
Page 1: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Results of the CAMS User Satisfaction Survey 2019 January 2020

Issued by: Data Support Team, ECMWF

Date: 30/01/2020

REF.: CAMS User Satisfaction Survey 2019 report final.docx

Page 2: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019

Contents Executive Summary ............................................................................................ 1

About the survey ................................................................................................ 2

Target audience ............................................................................................... 2

The questionnaire ............................................................................................ 3

Q1: Where are you based? .................................................................................. 5

Results ........................................................................................................... 5

Recommendations ........................................................................................... 5

Q2: Which sector does your organisation belong to? ............................................... 6

Results ........................................................................................................... 6

Recommendations ........................................................................................... 7

Q3: How do you use the Copernicus Atmosphere Monitoring Service (CAMS) (e.g. your user story)? ....................................................................................................... 7

Results ........................................................................................................... 7

Q4: Overall, how satisfied are you with CAMS? ....................................................... 8

Results ........................................................................................................... 8

Recommendations ........................................................................................... 8

Q5: How useful are CAMS products and services for you, and how satisfied are you with them? ........................................................................................................ 9

Results ........................................................................................................... 9

Recommendations ......................................................................................... 14

Q6: How useful is, and how satisfied are you with non-data services? ..................... 15

Results ......................................................................................................... 15

Recommendations ......................................................................................... 17

Q7: If you downloaded CAMS global data using the WebAPI, how satisfied are you with the WebAPI? .................................................................................................... 18

Recommendations ......................................................................................... 18

Q8: If you downloaded CAMS regional data using the WebAPI, how satisfied are you with the WebAPI? ............................................................................................. 19

Recommendations ......................................................................................... 19

Q9: What changes in CAMS products and services would have the most beneficial impact on your work?........................................................................................ 20

Results ......................................................................................................... 20

Page 3: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019

Q10: How do you prefer to find out about changes to CAMS (e.g. new dataset, system maintenance, etc)? ........................................................................................... 21

Q11: Do you have any other suggestions how we can improve CAMS for you? ......... 21

Results ......................................................................................................... 21

Q12: Please provide your email address if you agree to be contacted for: follow-up on your use case; testing the Atmosphere Data Store (new data access service) as a test user; further discussion regarding your user requirements ..................................... 22

Comparison with results from previous CAMS User Surveys ................................... 23

Results ......................................................................................................... 23

Annex: Summary tables and open-ended comments ............................................. 25

Page 4: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 1

Executive Summary

Users of the Copernicus Atmosphere Monitoring Service (CAMS) are satisfied overall with the service provided by CAMS and rate the services with an overall satisfaction rating of 4.3 out of 5; this is a slightly lower rate than see in the previous year’s survey (4.4).

62% of CAMS users who responded are based in the European Union. About 45% of the users work in academia/research, while 25% are in the business sector.

User uptake and satisfaction varies by individual CAMS products and services: the most popular services are information on global atmospheric composition and global reanalysis.

Users also remain satisfied with supporting services, in particular with the available data access mechanisms and with the product documentation. However, many users remain unaware of remote services (e.g. OGC services), the CAMS newsletter and the mailing list.

Overall the survey participation is lower than in 2018, possibly due their being no major CAMS developments prior to the survey to attract interest, as well as the time of year when the survey was run. In general, the results are similar to the previous year’s results.

The 2018 survey identified four areas for improvement, which all remain valid in 2019: (a) the footprint of CAMS in Eastern Europe (b) the attractiveness of CAMS to public sector users and in particular the utility and awareness of policy support products, (c) data access tailored to users’ specifications, and (d) more detailed scientific documentation. In addition, the 2019 survey identified the CAMS catalogue and data download speeds as aspects of CAMS which could be enhanced.

Page 5: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 2

About the survey

Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand how satisfied CAMS users are with services provided, to collect feedback, analyse them and help improve CAMS services.

The survey targeted all users who, in the 12 months preceding the survey, accessed CAMS data products or who interacted with Copernicus User Support, and for whom email addresses were available. This demographic comprised 3044 distinct users (with some users registered for multiple services), compared with 2816 in 2018:

• 1884 users of ECMWF-hosted CAMS/MACC data services (Global NRT, Global NRT archive, GFAS, CAMS Reanalysis, MACC Reanalysis, CAMS/MACC GHG inversion, CAMS climate forcing)

• 941 users of CAMS solar data services (CAMS_72) • 422 users of CAMS User Support • 177 users of the CAMS Regional service (CAMS_50) • 33 users of CAMS Anthropogenic Emissions data services (CAMS_81)

Some CAMS services allow anonymous access (e.g. CAMS_50, CAMS_71); users of these anonymous services are not included in the survey. Hence the actual number of CAMS users is expected to be higher than the targeted demographic.

The survey was deployed via an online questionnaire, in English, which was open between 1st July and 22nd July 2019. CAMS invited all 3044 identified users by email to participate in the survey.

In total 114 users responded (3.8% of addressees; for 2018 year there were 259 responses, corresponding to 9.2% of addressees).

For further information please contact [email protected].

Page 6: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 3

The questionnaire The following questionnaire was used:

Q1: Where are you based? (List of countries and territories)

Q2: Which sector does your organisation belong to? (Public authorities / University and Research / Commercial or private sector / International organisation, NGO / Other)

Q3: How do you use the Copernicus Atmosphere Monitoring Service (CAMS) (e.g. your user story)?

Q4: Overall, how satisfied are you with CAMS? (1 = not satisfied, 5 = very satisfied)

Q5: How useful are the following CAMS products and services for you, and how satisfied are you with them?

• Regional (European) air quality • Global atmospheric composition • Global reanalysis • Policy support products • Solar radiation • Greenhouse gas fluxes • Climate forcing • Anthropogenic emissions • Fire emissions

Q6: How useful is, and how satisfied are you with ...

• CAMS Website • Product catalogue • CAMS charts • Automated data download (WebAPI, FTP, ...) • Data download portal (apps.ecmwf.int) • Remote services (e.g. WMS, CSW) • Available data formats • Documentation • Knowledge base • Validation reports • Helpdesk / Support • Mailing list • Newsletter

Q7: If you downloaded CAMS global data using the WebAPI, how satisfied are you with the WebAPI?

• In terms of speed (1 = not satisfied, 4 = very satisfied) • In terms of availability (24/7) (1 = not satisfied, 4 = very satisfied)

Q8: If you downloaded CAMS regional data using the API, how satisfied are you with the API?

• In terms of speed (1 = not satisfied, 4 = very satisfied) • In terms of availability (24/7) (1 = not satisfied, 4 = very satisfied)

Page 7: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 4

Q9: What changes in CAMS products and services would have the most beneficial impact on your work?

Q10: How do you prefer to find out about changes to CAMS (e.g. new dataset, system maintenance, etc)?

Q11: Do you have any other suggestions how we can improve CAMS for you?

Q12: Please provide your email address if you agree to be contacted for:

• follow-up on your use case • testing the Atmosphere Data Store (new data access service) as a test user • further discussion regarding your user requirements

Page 8: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 5

Q1: Where are you based?

Results Responding CAMS users (Figure 1) are mostly based in the European Union (62% of all responses). Outside of Europe, most responses come from the USA (5%).

Overall and among the EU countries Germany is leading, followed by France (reversing the previous year’s top two positions), and by Spain and Italy (Figure 2).

Overall the regional distribution remains almost identical to the previous years’ survey results. More details of this comparison are provided later in this document.

As with the 2018 user survey, the user numbers do not show an increased user uptake in the eastern EU countries.

Recommendations The regional distribution shows that CAMS data and services are primarily used within in the EU, in line with its mandate.

Figure 1: Global responses by country

(where >= 2%)

Figure 2: EU responses by country

(where >= 2%)

Page 9: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 6

Respondent numbers correlate with the population of each country. However, the eastern EU countries are still underrepresented and the efforts to increase the user uptake in Eastern Europe should continue.

Q2: Which sector does your organisation belong to?

Results In terms of sectors, for all (global) users, the majority were work in academia and research, followed by commercial entities and public authorities (Figure 3).

Figure 3: Responses by sector, global (112 responses)

Compared to the previous year, the use of CAMS in the commercial sector increased (from 22% to 26%), and in the public sector it also increased (from 15% to 24%).

Responses from EU-based users show a similar distribution (Figure 4), and a similar change over the past year: the use of CAMS in the EU commercial sector increased (from 28% to 33%), and the public sector (16% to 23%), while there was a decrease in academia/research users who responded (49% to 41%).

Page 10: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 7

Figure 4: Responses by sector, EU only (70 responses)

Overall, CAMS remains well known and well used in academia and research in the EU, and again shows signs of growth in the commercial sector compared to last year’s survey.

These results should perhaps be treated with a degree of caution though, as that these results are based on a relatively small number of respondents.

Recommendations Further user uptake activities should continue to build on the public and commercial sectors.

Q3: How do you use the Copernicus Atmosphere Monitoring Service (CAMS) (e.g. your user story)?

Results To determine how CAMS users are making use of these data, users were asked to tell us their ‘user story’. Their responses are given in the Annex.

Example responses include: • “Copernicus has served me to be able to analyze the pollutant concentrations in my country,

state or municipality. With this, it allows me to make cartography and present it to an external and interested public.“

• “We use CAMS data to rate potential sites for algae production plants since the data are far more detailed and precise than other sources available to us.”

• “everyday regional AQ forecast policy service tools “

Page 11: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 8

Q4: Overall, how satisfied are you with CAMS?

Results Overall satisfaction with CAMS is high, both globally and from EU respondents. The weighted average rating is 4.3 out of 5.

The percentage of all users giving a 4 or 5 star rating to CAMS was 86% (figure 5) and 91% for EU users (Figure 6).

Recommendations Overall user satisfaction with CAMS remains high amongst respondents and does not indicate a need for radical change to the CAMS strategy and portfolio.

Figure 6: Overall satisfaction, EU respondents (50)

Figure 5: Overall satisfaction, all respondents (78)

Page 12: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 9

Q5: How useful are CAMS products and services for you, and how satisfied are you with them?

Results

Figure 7: Utility and user satisfaction with data products and services, all respondents (75 responses)

Page 13: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 10

The utility and satisfaction ratings remain largely unchanged compared to the previous years’ survey.

Utility is ranked highest for global reanalysis products and global atmospheric products. (Figure 7), for the global user base. User satisfaction is still raked highest for the Regional (European) analysis and Fire emissions services.

Awareness of Policy support products has decreased slightly since the 2018 survey, from 47% to 44%.

When examined by sector, commercial users find the CAMS services for global atmospheric composition, European air quality, and solar radiation most useful, and GHG fluxes and policy products least useful. For the GHG fluxes, this low rating may be because the data are not published. For the policy products, there is a significant lack of awareness amongst users.

These results are in line with those from the 2018 survey results. Most commercial users are ‘very satisfied’ or ‘extremely’ satisfied with all main services (Figure 8).

Page 14: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 11

Figure 8: Utility and user satisfaction with data products and services, Commercial and private sector

(41 responses)

Page 15: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 12

In academia and research, the global atmospheric composition and global reanalysis services are considered most useful. Users in this sector are generally very satisfied with the CAMS services. (Figure 9). Compared to the previous year, user awareness and satisfaction were seen to increase slightly.

In the public sector, global atmospheric composition data is considered most useful, followed by data on global reanalysis, European air quality and fire emissions. Satisfaction with all products is very high (Figure 10). As with 2018, policy support products stand out as being little known known by users and are even less known in the public sector than in other sectors.

Page 16: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 13

Figure 9: Utility and user satisfaction with data products and services, Academia and Research (29 responses)

Page 17: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 14

Figure 10: Utility and user satisfaction with data products and services, public sector (24 responses)

Recommendations CAMS users find data products and services generally useful and are increasingly satisfied with them, indicating that CAMS has had an impact and is fulfilling its

Page 18: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 15

mandate. The low level of user awareness of policy support products remains an issue, however and consideration should be given to promoting them more.

Q6: How useful is, and how satisfied are you with non-data services?

Results This question covers various supporting services, like data provision mechanisms, documentation, and user interaction.

Overall, users consider these services useful, with services related to data access ranked most useful. Notably the remote services (WMS, CSW) and CAMS communications (mailing lists and newsletter) are relatively unknown (Figure 11).

All non-data services receive high satisfaction ratings; although remote data services (OGC services), mailing lists and the newsletter lag somewhat behind.

Page 19: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 16

Figure 11: Utility and user satisfaction with non-data services, all respondents (75 responses)

Page 20: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 17

The most useful supporting services remain the CAMS website and the available data formats, (Figure 11).

Remote services (OGC services) and information services (mailing list, newsletter, knowledge base) are still not well known and considered not very relevant. These results are similar to the previous year’s findings; however the utility of the CAMS product catalogue and automated download service has increased noticeably.

Satisfaction with supporting services is high throughout (in most services fewer than 5% of users are not satisfied), only remote services (OGC services) and stand out as less satisfying.

Recommendations Urgent consideration should be given to improving the catalogue and documentation.

User comments included:

Regarding the website (including the product Catalogue and the charts) - our main problem is that it's hard to find the needed information. Regarding data formats - we had some issues with the GRIB2 format in CAMS Regional Air Quality - Archived data (older than 30 days). As it's not standard and standard tools can't open it. Regarding validation reports - information regarding pollen data accuracy will help. Regarding help desk and mailing list - we are happy with most answers and from the short time it takes to get answers. However, recently we got answers and emails with contradictory information, mainly regarding upcoming upgrades to the models. In addition, it will help if announcements regarding model upgrades will be clearer, as we had hard time to understand what are the expected changes and which product will be affected. (mainly the CAMS regional last update)..

the legends/units/explanations on the values (which unit for ozone, how is it measured, built, reanalysed, etc...) : they are hard to find and no very detailed

Navigation on the CAMS websites is not always easy and information partly hard to find. Download from ftp site only latest 3 days, sftp support not working properly.

Very hard to find relevant documentation for a specific service or API. Documentation is also fragmented and sometimes incomplete. Data format specifics may change unexpectedly.

The actual website would benefit from a small refreshment especially on the UX aspect as I believe it's not easy enough to visualize the available data. Also, the documentation is very poor, especially about the region products.

Page 21: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 18

Q7: If you downloaded CAMS global data using the WebAPI, how satisfied are you with the WebAPI?

The results show that some users are still dissatisfied, primarily with the speed of data delivery, but also indicate that the WebAPI service meets the expectations of a clear majority of users (Figure 12).

Figure 12: Satisfaction with the WebAPI for CAMS global data (49 responses)

Recommendations Only a small number of users expressed dissatisfaction with the CAMS global data API. Given that the data will eventually move to the Atmospheric Data Store (ADS), the numbers are still too small to justify mitigation beyond case-by-case assistance.

The survey also shows that users value speedy data access; this should be considered when migrating CAMS data to the ADS.

Page 22: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 19

Q8: If you downloaded CAMS regional data using the WebAPI, how satisfied are you with the WebAPI?

Satisfaction with the API for regional (European) data is high throughout and almost identical to satisfaction with the global data API (Figure 13). Compared with the 2018 survey results, a higher percentage of users were very satisfied with the download speeds available, although there was also a slight increase in the proportion of users who expressed dissatisfaction with speed and availability.

Figure 13: Satisfaction with the WebAPI for CAMS regional data (36 responses)

Recommendations The access mechanisms to regional data are well established and users are mostly satisfied. If anything, speedier access (perhaps by making some data available via the ADS) could potentially improve user satisfaction further.

Page 23: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 20

Q9: What changes in CAMS products and services would have the most beneficial impact on your work?

This question aims to identify user-driven improvements to CAMS. This is an open question. Responses mostly reiterate points already highlighted in other questions, hence no further analysis is undertaken and no additional recommendations are identified.

Results Responses from users to this question included:

• “Unlimited access to the entire data set from the CAMS Radiation Service (not just limited to 75 on the fly calculations per day). “

• “Reorganization of documentation in relation to specific services - Improvement of discoverability of different services and features”

• “faster downloading”

Page 24: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 21

Q10: How do you prefer to find out about changes to CAMS (e.g. new dataset, system maintenance, etc)?

The user responses indicated that the most popular communication channels for disseminating information to CAMS users were CAMS mailing lists and the CAMS newsletter (Figure 14), although user awareness of these channels could be increased.

Figure 14: The CAMS users preferred means of communication (70 responses)

Q11: Do you have any other suggestions how we can improve CAMS for you?

This question aims to identify any other user-driven improvements to CAMS. This is an open question. Responses mostly reiterate points already highlighted in other questions, hence no further analysis is undertaken and no additional recommendations are identified.

Results Responses from users to this question included:

• Any improvement in the pollen products will be helpful, such as validation reports, additional pollen species, extending pollen coverage * Improvement of the CAMS global spatial grid resolution will be helpful * Where the forecast temporal resolution is 3 hours, an improvement to hourly data will be helpful * Currently it takes roughly 8 hours for CAMS regional and 10 hours for CAMS global forecasts to be available in the FTP. Decrease of this time will be helpful. * Increase of the refresh rate of CAMS regional forecast to be twice a day will be helpful.

• Please improve the documentation, especially for the regional product. Otherwise, thanks a lot for the amazing work!

Page 25: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 22

Q12: Please provide your email address if you agree to be contacted for: follow-up on your use case; testing the Atmosphere Data Store (new data access service) as a test user; further discussion regarding your user requirements

This question aims to collect contact details for individuals willing to be engage further with CAMS, hence no further analysis is undertaken.

Page 26: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 23

Comparison with results from previous CAMS User Surveys

Results The graphics below (Figures 15-22) show the comparison of results from the CAMS User Survey’s undertaken in 2017, 2018 and 2019 for a number of selected areas.

The decrease in the total number of responses from CAMS users (both in absolute terms and as a percentage) is an area of concern, although it is notable there was a good response from EU users and that overall user satisfaction remains high.

In terms of sectors, a steady increase can be seen in the number of responses from the commercial sector, indicating an increasing engagement with CAMS from this community.

Figure 15: User satisfaction rating (%) Figure 16: Number of surveys sent out

Figure 17: Number of user responses (all) Figure 18: Number of user responses (all, %)

2017 2018 201980

82

84

86

88

90

Year

%

User Satisfaction Rating

2017 2018 20192500

3000

3500

4000

Year

Num

ber

Number Sent Out

Page 27: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 24

Figure 19: Number of user responses (EU, %) Figure 20: Number of user responses by country

(EU, %)

Figure 21: Number of user responses by sector

(all, %)

Figure 22: Number of user responses by sector

(all, %)

Page 28: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 25

Annex: Summary tables and open-ended comments

Q1. Where are you based?

Territories with <3 responses are excluded for anonymity

Answer Choices Percentage Responses

Germany 14% 15

France 12% 13

Spain 7% 8

United States of America 5% 6

United Kingdom 5% 5

Portugal 4% 4

Czech Republic 3% 3

Other 50% 59

Answered: 113, Skipped: 6

Q2. Which sector does your organisation belong to?

Answer Choices % of total Responses

Public authorities (European, National or local)

24% 27

University and Research 45% 50

Commercial or private sector 26% 29

International organisation, NGO 3% 3

Other (please specify) 3% 3

Answered: 112, Skipped: 7

Page 29: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 26

Q3. How do you use the Copernicus Atmosphere Monitoring Service (CAMS) (e.g. your user story)?

Answered: 101, Skipped: 18

Q4. Overall, how satisfied are you with CAMS? (The more stars the better)

Rating, 1 to 5, higher is better

1 2 3 4 5 Total Weighted Average

Responses

% of total

Responses

% of total

Responses

% of total

Responses

% of total

Responses

% of total

Responses

% of total

Responses

3% 2 3% 2 9% 7 31% 24 55 43 78 4.33

Answered: 78, Skipped: 41

Page 30: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 27

Q5. How useful are the following CAMS products and services for you, and how satisfied are you with them?

How useful

Am not aware of it

Irrelevant Somewhat useful Very useful Essential Total

% of total

Responses

% of total

Responses

% of total

Responses

% of total

Responses

% of total

Responses

Responses

European air quality

17% 10 12% 7 14% 8 28% 16 29% 17 58

Global atmospheric composition

9% 6 6% 4 8% 5 46% 30 31% 20 65

Global reanalysis

9% 6 5% 3 11% 7 41% 27 35% 23 66

Policy support products

44% 24 22% 12 13% 7 13% 7 9% 5 55

Solar radiation 19% 11 17% 10 19% 11 28% 16 17% 10 58

Greenhouse gas fluxes

21% 12 18% 10 14% 8 32% 18 16% 9 57

Climate forcing

25% 14 20% 11 14% 8 32% 18 9% 5 56

Anthropogenic emissions

27% 15 9% 5 11% 6 27% 15 27% 15 56

Fire emissions 22% 13 5% 3 29% 17 19% 11 24% 14 58

Answered: 75, Skipped: 44

How satisfied

Not satisfied at all

Somewhat satisfied

Very satisfied Extremely satisfied

Total

% of total

Responses

% of total

Responses

% of total

Responses

% of total

Responses

Responses

European air quality 3% 1 10% 4 62% 24 26% 10 39

Global atmospheric composition 0% 0 13% 6 73% 35 15% 7 48

Global reanalysis 2% 1 12% 6 67% 35 19% 10 52

Policy support products 5% 1 20% 4 55% 11 20% 4 20

Solar radiation 3% 1 23% 7 57% 17 17% 5 30

Greenhouse gas fluxes 0% 0 17% 4 71% 17 13 3 24

Climate forcing 0% 0 18% 4 73% 16 9% 2 22

Anthropogenic emissions 3% 1 17% 5 63% 19 17% 5 30

Fire emissions 6% 2 21% 7 48% 16 24% 8 33

Answered: 75, Skipped: 44

Page 31: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 28

Q6. How useful is, and how satisfied are you with ...

How useful

Am not aware of it

Irrelevant Somewhat useful Very useful Essential Total

% of total

Responses

% of total

Responses

% of total

Responses

% of total

Responses

% of total

Responses

Responses

CAMS website 1% 1 3% 2 17% 12 54% 38 24% 17 70

Product catalogue 8% 5 3% 2 21% 13 48% 30 21% 13 63

CAMS charts 16% 10 7% 4 30% 18 31% 19 16% 10 61

Automated data download (WebAPI, FTP, ...)

10% 6 2% 1 5% 3 25% 15 59% 36 61

Data download portal (apps.ecmwf.int)

15% 9 2% 1 12% 7 30% 18 42% 25 60

Remote services (e.g. WMS, CSW)

43% 23 6% 3 17% 9 28% 15 7% 4 54

Available data formats

6% 4 3% 2 9% 6 49% 32 32% 21 65

Documentation 10% 6 2% 1 28% 16 26% 15 34% 20 58

Knowledge base 28% 15 0% 0 15% 8 41% 22 17% 9 54

Validation reports 30% 16 2% 1 11% 6 43% 23 15% 8 54

Helpdesk / Support

32% 18 4% 2 11% 6 32% 18 21% 12 56

Mailing list 40% 21 10% 5 19% 10 21% 11 10% 5 52

Newsletter 37% 19 12% 6 23% 12 25% 13 4% 2 52

Answered: 75, Skipped: 44

How satisfied

Not satisfied at all Somewhat satisfied Very satisfied Extremely satisfied Total

% of total

Responses

% of total

Responses

% of total

Responses

% of total Responses

Responses

CAMS website 2% 1 29% 17 54% 32 15% 9 59

Product catalogue 2% 1 42% 22 44% 23 12% 6 52

CAMS charts 2% 1 40% 17 40% 17 17% 7 42

Automated data download (WebAPI, FTP, ...)

7% 3 24% 11 41% 19 28% 13 46

Data download portal (apps.ecmwf.int)

5% 2 33% 14 36% 15 26% 11 42

Remote services (e.g. WMS, CSW)

9% 2 23% 5 55% 12 14% 3 22

Available data formats

2% 1 14% 7 53% 26 31% 15 49

Documentation 7% 3 44% 20 36% 16 13% 6 45

Page 32: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

CAMS User Satisfaction Survey 2019 29

Knowledge base 3% 1 33% 11 48% 16 15% 5 33

Validation reports 3% 1 26% 8 55% 17 16% 5 31

Helpdesk / Support 6% 2 24% 8 48% 16 21% 7 33

Mailing list 12% 3 38% 10 35% 9 15% 4 26

Newsletter 15% 4 35% 9 38% 10 12% 3 26

Answered: 75, Skipped:44

Q7. If you downloaded CAMS global data using the WebAPI, how satisfied are you with the WebAPI?

How satisfied

Not satisfied Somewhat satisfied Mostly satisfied Very satisfied Total

% of total

Responses

% of total

Responses

% of total

Responses

% of total Responses

Responses

In terms of speed 12% 6 20% 10 35% 17 33% 16 49

In terms of availability (24/7)

10% 5 10% 5 38% 18 42% 20 48

Answered: 49, Skipped: 70

Q8. If you downloaded CAMS regional data using the WebAPI, how satisfied are you with the WebAPI?

How satisfied

Not satisfied Somewhat satisfied Mostly satisfied Very satisfied Total

% of total

Responses

% of total

Responses

% of total

Responses

% of total Responses

Responses

In terms of speed 8% 3 17% 6 25% 9 50% 18 36

In terms of availability (24/7)

6% 2 9% 3 40% 14 46% 16 35

Answered: 36, Skipped: 83

Q9. What changes in CAMS products and services would have the most beneficial impact on your work? Answered: 42, Skipped: 77

Q10. How do you prefer to find out about changes to CAMS (e.g. new dataset, system maintenance, etc)??

Answered: 70, Skipped:49

Q11. Do you have any other suggestions how we can improve CAMS for you?

Answered: 22, Skipped:97

Page 33: Results of the CAMS User Satisfaction Survey 2019 · 2020. 3. 12. · The questionnaire ... Target audience The purpose of the CAMS User Satisfaction Survey 2019 was to understand

Copernicus Atmosphere Monitoring Service

atmosphere.copernicus.eu copernicus.eu ecmwf.int

ECMWF Shinfield Park Reading RG2 9AX UK


Recommended