CEMAS Final MT Eval Report

CEMAS Mid-Term Evaluation Report 1

PRIVATE & CONFIDENTIAL

MID-TERM EVALUATION OF THE CENTRE OF EXCELLENCE IN MOBILE APPLICATIONS AND SERVICES (CEMAS)

The Innovation Partnership Ltd

Prepared for: Professor Khalid Al-Begain, CEMAS Programme Director, University of South Wales Prepared by: Mr Nigel Woodruff, Director, The Innovation Partnership Ltd Date: May 2014


Contents Executive Summary

1 INTRODUCTION

1.1 The Centre of Excellence in mobile Applications and Services (CEMAS)

1.2 CEMAS Performance Targets

1.2 CEMAS Activities

2 EVALUATION SCOPE AND METHODOLOGY 2.1 Evaluation Requirements

2.2 Evaluation Methodology

3 PROJECT CONTEXT

4 MARKET ANALYSIS

4.1 Welsh ICT Sector

4.2 European/UK App Market

5 CEMAS OUTPUT & PERFORMANCE AT THE MID-TERM POINT

6 INTERVIEW PROGRAMME ANALYSIS

6.1 Internal Stakeholders

6.2 External Stakeholders

6.3 Beneficiary Companies

7 SPECIFIC EVALUATION QUESTIONS

8 CONCLUSIONS AND ACTIONS

8.1 Conclusions

8.2 Actions


Executive Summary The Centre of Excellence in Mobile Applications and Services (CEMAS or The Project), assists start-up businesses and SMEs in the development and testing of apps, facilitating the definition and scoping of routes-to-market for beneficiaries. The Centre is based in the University of South Wales, and is part-funded by ERDF - Priority One, Theme Two - 2,995,996, with a further 3,437,679 of direct funding. The Project commenced in September 2010, with the initial phase due for completion in June 2015. The University of South Wales issued an Invitation To Tender (ITT) on 26 November 2013 (via the Sell2Wales website) to undertake a mid-term evaluation of CEMAS. The Innovation Partnership Ltd (TIP or The Consultants) was successful with its response to the competitive tender, proposing an evaluation made up of the key components including interview programmes with identified internal/external stakeholders and beneficiaries. This report represents the results of this evaluation exercise. Project Performance to Date The table below sets out the project targets placed on CEMAS as a condition of ERDF funding:

Table: CEMAS Performance Against Targets (as of May 2014) As of May 14 Target (June 15) % Achieved

Enterprises assisted 101 138 73 Collaborative R&D 41 60 68 New/Improved Products 25 30 83 Products, processes or services registered 6 10 60 Enterprises created 5 6 83 Gross jobs created 8 11 73

It is the Consultants opinion that CEMAS is showing significant progress towards the attainment of stated funding targets. Although certain delays were experienced in the early stages of the Project, relating to recruitment of qualified staff etc, CEMAS management has addressed these issues and the Project should attain or surpass targets by the end of this funding period. The Consultants have evaluated several similar ERDF programmes and it is a common aspect that target attainment comes to fruition in the second half of the funding period, once initial issues relating to programme launch have been addressed. The Consultants consider this project to have been researched, planned for, delivered, managed, and executed more effectively than any of the similar projects we have observed at the mid-term stage. Stakeholder Interview Programme Internal stakeholders within the University are keen to support CEMAS in its development. During the interview process, it was stated that there is a significant opportunity for the Project to reach sustainability, firstly by continuing to serve Welsh-based SMEs and, secondly, by tying CEMAS in more closely to key Welsh Government sectors. The opinion was put forward that there will be an ever-demanding and widening market for applications, and that for CEMAS to grow and prosper, it needs to be continually innovative in its offering.


University stakeholders also believed that there is a strong educational training and CPD element in play. At an educational level, staff within CEMAS and other internal stakeholders are convinced that they have the opportunity to support STEM subjects and encourage young people to consider a career in computing and programming. As Welsh Government is acutely aware, there is currently a shortage of 3,500 IT professionals within Wales. Internal stakeholders appreciated the value of building R&D competence and capacity within CEMAS. R&D capacity will lead to new revenue streams, potential spinouts, and sector specialisation. The management of CEMAS has been successful in drawing the right balance between the need for extensive research and the requirement to adhere to commercial timescales to provide the majority of beneficiaries with the best possible support. We believe this type of combination can be a model for any future business support projects. When asked to comment on the product/service approach and positioning of CEMAS, there was commonality in the responses of the internal stakeholders, which is indicative of a team that works together and has a common vision. The view on product/service position was that it was fit-for-purpose and that the beneficiary journey is readily and easily understood. External stakeholders were impressed with the professionalism of the academics and senior management at CEMAS, identifying that they operate in an extremely professional manner. It was stated that this opinion was backed up by feedback received by external stakeholders from the various beneficiary companies of CEMAS. The positive perception of CEMAS was enhanced by the fact that external stakeholders involved in other similar programmes were more than willing to discuss possible collaboration with CEMAS going forward, particularly in terms of Horizon 2020 and Structural Fund bids. Beneficiary Interview Programme Beneficiary companies interviewed for the purposes of this evaluation represented a range of sectors and geographic locations throughout Wales, with the majority being at the ideas only stage before engagement with CEMAS. This was reflected in the fact that getting product to market was the key motivation for engagement with CEMAS identified by beneficiaries.

It is pertinent to point out that the majority of companies supported represent micro-business whereby, funding eligibility notwithstanding, for greater economic impact for Wales, and an increase in GVA, CEMAS would be better disposed working with larger companies within the SME Convergence Region.

The Consultants would suggest that this independent evaluation has identified levels of customer satisfaction that are the highest that we have witnessed for a project of this nature. The vast majority of beneficiary interviewees believed the engagement to have added value to their business and, in addition to tangible benefits such as bringing their app to market, engagement had provided softer benefits, including a better understanding of customer needs and facilitating better relationships with the local supply chain. Evidence obtained from research methodologies deployed during this exercise would suggest that CEMAS is an exemplar project that has demonstrated excellence and best practice, both in its product and service offering, and its ability to take client companies on a journey that not only delivers, but also adds significant value on both a tactical and


strategic level. It is clear that the Project should be continued and expanded so that significantly more enterprises throughout Wales can benefit. As a result of the positive evidence obtained, a meeting was held between the Consultants and CEMAS senior management where it was agreed that a list of future actions for the Project would be the most beneficial output from this evaluation exercise. These actions have been outlined briefly below and in more detail in the full report: Action 1 - Continuation of CEMAS: To continue with the existing CEMAS programme, and not make any significant changes. The Project is on track to achieve its funding targets, with no requirement to de-commit to budgets/targets for ERDF/WEFO. Action 2 - Project Road Mapping: Within the scope of the current ERDF CEMAS project, we recommend that CEMAS try road mapping as an engagement and ERDF assist methodology. We believe this approach will prove to be useful in future CEMAS multi-disciplinary/joint partner programmes, focusing more on seamless and longer-term SME support. Action 3 - CEMAS Impact: To look at the existing cohort of beneficiary businesses and try to identify and align metrics that measure added value, and to then run an internal study over the remaining ERDF period, creating a body of evidence for future grant bids and integration with other suppliers/programmes. Action 4 - CEMAS Expansion (Phase II): Within the confines of the existing CEMAS programme, we agreed that management should undertake research to identify areas of additional activity leading to added value for future funded programmes. Action 5 - Competency & Skills Development: CEMAS management and senior staff generally accepted that, collectively, they would not benefit from a greater understanding of mobile enabling technologies, principally due to the universal dominance of key players resulting in common available technologies/platforms. Alternatively, more effective competency and skills will be gained by developing content capability. Action 6 - CEMAS Delivered Training and CPD: Action a future ESF bid, and possible other educational (STEM) training funding bids. Action 7 - Welsh Sector Engagement: CEMAS now needs to research, select, and focus on a number of the nine key WG sectors to which the Project can demonstrate added value. This action is necessary to identify likely future business needs, and to build relevant products and services. Action 8 - Horizon 2020: Horizon 2020 provides CEMAS with an excellent opportunity to develop, in a collaborative way, content and enterprise engagement both inside and outside of Wales, and outside of the UK. CEMAS first action will be to identify a specific Horizon 2020 call, including possible SME measures, and to seek funding via Welsh Governments SCoRE Cymru to write bids, with the target being to lead on at least one or two bids in 2014.


1 Introduction 1.1 The Centre of Excellence in Mobile Applications and Services (CEMAS) 'Apps' is an abbreviation of software applications and is frequently used to describe mobile applications that are designed to run on smart phone, tablet computers, and other mobiles devices. The apps market provides businesses in Wales with low cost barriers to entry to market their products, processes and services via e-commerce. The Centre of Excellence in Mobile Applications and Services (CEMAS or The Project), assists start-up businesses and SMEs in the development and testing of apps, and then helps to define and scope the route-to-market for beneficiaries. As stated in the revised Business Plan (May 2013), the Project will continue to be delivered in the following discrete stages: Stage 1: Disseminate, Promote and Recruit The first stage involves promoting the

potential use of apps within the business environment and the range of apps that could be developed using the dedicated project web site, open seminars and in house company visits. As there is no tangible benefit to the SME, this stage does not constitute state aid.

Stage 2: Collaborative Research, design and test new apps This stage involves working with SMEs on a one to one basis to develop work and test the app on the Universitys state-of-the-art 3G-IMS Core network facility. Where the research / technical assistance by CEMAS does not produce an app capable of being used, when the app is used in house or is available commercially but without IP protection, this assistance is recorded under the De Minimis state aid regulation.

Stage 3: Protect and Sell The third stage will entail working with the company to finalise

the specification, identifying specialised market opportunities for the new app, and protecting the resulting IPR. This stage of the project is administered under Article 33 of the General Block Exemption Regulation (GBER) and the SME will be required to contribute 50% towards the costs of this activity.

After a competitive procurement exercise, the University commissioned Geldards LLP to produce appropriate legal documentation governing Stage 3 of the Programme. This was commissioned in September 2013 to coincide with CEMASs first Stage 3 project. During this process, the University was advised that the most appropriate application of State Aid for Stage 3 was actually De Minimis and not GBER. Thus, Stage 3 is now governed as De Minimis aid and the SME is still required to make a contribution to the collaborative research project created in Stage 2. WEFOs response to this was that development advice given to undertakings during Stage 3 of the Project must have the appropriate State Aid cover. This will either be under GBER, the De Minimis Regulation or in accordance with the R&D&I Framework. It also states that the Project Sponsor must consider in each case what State Aid cover is most appropriate.


Thus following the legal advice received by Geldards LLP Stage 3 is now administered as De Minimis (and all stage 3 projects concluded have been administered as De Minimis) The Centre is based in the University of South Wales, and is part-funded by the European Regional Development Fund, ERDF - Priority One, Theme Two - 2,995,996, with a further 3,437,679 of direct funding. The Project commenced in September 2010, and is due for completion in June 2015.

1.2 CEMAS Performance Targets CEMAS has the following ERDF contract performance targets within the Convergence Region of Wales:

Table 1: CEMAS Performance Targets

Target Total

(Initial business plan September 2010)

Total (July 2013 additional grant award)

Enterprises Assisted 125 138 Collaborative R&D 50 60 New or Improved Products 20 30 Products processes or services registered 8 10 Enterprises Created 4 6 Gross Job Created 8 11

Source: ERDF Application

1.3 CEMAS Activities CEMAS supports eligible SMEs by undertaking assignments that transform business processes, products and services specifically framed by the following: To support SMEs to take-up and maximise product and business opportunities created by

e-business and mobile telecommunications worldwide. Target support on potential high growth companies in ICT industries with particular focus

on the creation of more innovative content, services, and applications. Support ICT applications that have an environmental benefit such as improving resource

efficiency and reducing the need to travel. Promote equality of opportunities for all SMEs, through the effective use of ICT. The University of South Wales issued an Invitation To Tender (ITT) on 26 November 2013 (via the Sell2Wales website) to undertake a mid-term evaluation of CEMAS. The Innovation Partnership Ltd (TIP or The Consultants) was successful with its response to the competitive tender and this report represents the result of this evaluation exercise.


2 Evaluation Scope and Methodology 2.1 Evaluation Requirements The successful evaluator was required to undertake a mid-term evaluation of the impact of the CEMAS project. This evaluation centres on the extent to which the Project has met its original aims, but also takes into account the changing environment and needs of target beneficiaries and other tangible and intangible benefits delivered. In essence, it will help CEMAS take stock, review progress and plan for the future. As set out in the ITT, the evaluation (as a minimum) will cover: 1 A review of the needs of SME technology against the changing ICT Environment is

Mobile ICT still an important requirement for SMEs in the future? 2 Integration with other business support programmes in the Convergence zone of Wales 3 Impact of CEMAS events on raising awareness of mobile ICT in Wales 4 The quality of the collaborative research undertaken to date in delivering a mobile

application for the supported SMEs 5 Appropriateness of the Project Indicators (Project Targets) 6 Progress on implementing the Cross Cutting Themes, Equal Opportunities and

Environmental Sustainability 7 Progress towards an Exit Strategy 8 Legacy Impacts, including the extent to which the project has contributed to a

structural and sustained impact on the targeted sector through its development and events based activity

9 CEMAS contribution to academic research in the area of mobile ICT 10 Is the Project achieving the objectives as set out in both the original and revised (May

2013) Business Plan with WEFO? 11 Has the Project met the business plan targets to date? 12 What parts of the Project are working well? 13 Where are the areas for improvement for the Project 14 On the basis of Project performance to date and existing /anticipated future demand

from SMEs based in the Convergence Region, is there scope for a successor (value added) project to be funded from the new Structural Fund Programme?

These specific evaluation questions have been answered in Section 7 of this report. The Consultants have also, within the overall scope and budget of the mid-term evaluation, reported on the following areas: Review of the Projects market strategy and approach Review of the Projects coverage of the Welsh Governments nine key sectors Indication of the added value beneficiaries have received as a result of assistance, ie

increased market penetration, increased range of products/services, increased sales/profitability etc

Areas into which CEMAS could expand A sense of the what kind of support has had the most profound economic impact on

business performance


2.2 Evaluation Methodology 2.2.1 Work Package Approach To successfully complete the evaluation, the Consultants proposed the eight packages of work identified below. Resources were identified and allocated to each reporting period. During discussions between the Project Management Team (PMT) and the Consultants, which took place at the inception meeting, it was decided that a personal interview programme with both internal/external stakeholders and beneficiary companies would prove the most appropriate method to obtain valuable in-depth information

Figure 1: CEMAS Mid-Term Evaluation Workflow Diagram

Activity Outputs

WP 1

Inception Meeting Inception Document / Workplan / Data Collections & Evaluation

WP 2 Data Review Data, Policy and Literature Review

WP 3 Internal Stakeholder

Interviews Questionnaire Design,

6 Interviews Set up, write and undertaking

WP 4 External Stakeholder

Interviews Questionnaire Design,

4 Interviews Set up, write and undertaking

WP 5 Beneficiary Survey Telephone survey with 16 willing

beneficiary participants

WP 6 Case Studies 5 follow-on case studies Set up, write and undertaking

WP 7 Impact Assessment

Assessment of current and projected

impact to date

WP 8 Analysis, Reporting and

Presentation Workshop, interim evaluation,

presentation and dissemination


3 Project Context Planning for the CEMAS took place during 2008 and 2009. In terms of economic context, in 2008 the economy and economic outlook were considerably healthier than has since transpired. No one could have predicted the crash in 2008, its longevity, and the still uncertain short-term future. Other ERDF-funded programmes have introduced changes and corrective measures at the mid-term to compensate for the economic downturn. Corrective measures used by other EU-supported programmes we have reviewed included greater marketing activity, re-profiling services/products and re-profiling outputs. We are aware that EU-funded programmes have been allowed to re-profile outputs based on economic conditions. At any project evaluation stage it is important to review and understand the context and the programmes positioning with key EU, UK and Welsh Government (WG) strategies. As a result of studying these key strategies, we have reported at a tactical level below, identifying the Projects fit-for-purpose, and at a strategic level as to how such a programme can continue to maximise investment and ensure long-term impact and sustainability The overarching WG policies, Economic Renewal: a New Direction (2010), and, Delivering a Digital Wales (2011) seek to create a thriving and competitive digital economy. Central to this is the support of a thriving ICT sector and up-skilling business in their adoption and use of ICT for which WG has specific targets going out until 2020. The Government also has an ICT strategy and a set of objectives and goals for the public sector in Wales, committing to new initiatives, the likes of the Welsh Cloud (Cwmwl Cymru). The ICT strategy equally calls for greater inclusion of the private sector in the delivery of the public-sector needs. The ICT sector strategy is critical for Wales, with the WG publicly backing its growth through: Supporting Welsh businesses Working closely with Academia Creating a world-class infrastructure Attracting world-class businesses The latest Government document for the ICT sector, produced by the Strategic Priorities Executive, presented to the Minister for Business, Enterprise and Technology identified the following key drivers: Anytime/anywhere/any device mobility Interoperability between IT products and services Data security (e-crime) Greater effectiveness and efficiency Creation of new services and routes to market The Strategy also reflects on the fact that the pace of change in the global ICT industry is remarkable. That said, the number of companies and number of people employed in this sector in Wales has declined since 2002 but still, by all means, is significant. It is also


recognised that there is significant and long-term investment in ICT infrastructure in Wales that has not been fully exploited and that the external perception of Wales and its ICT sector is neither negative nor positive. The ICT Strategy calls for Government to work with all partners and stakeholders to drive forward initiatives that will help to ensure that all businesses in Wales have access to a talent pool with the right ICT skills to meet current and future needs. It goes on to describe an environment where SMEs will receive increasing levels of support for the adoption of ICT and that all stakeholders, including educational establishments should be challenged to be more proactive in this sector. According to the WG, Wales has almost 40,000 people employed in the IT functions, with over 12,000 directly involved in the ICT industry. Reasons for the growth are plentiful but not least the return to the normal levels of investment, as witnessed pre 2008, advances in technologies and services, i.e. Cloud Computing, SaaS, Social Media etc, growth in business processes and increasing exploitation of big data, i.e. e-commerce, CRM, and an increasingly educated workforce. Government sees ICT as a key priority sector for Wales, as the application of ICT drives productivity and competitiveness across the whole economy. Wales has a mix of large multinational and small home-grown companies, but the WG realises that it is the SME Gazelles that will contribute significantly to employment and economic growth. The CEMAS project will help achieve the WG aspirations by contributing to a step change in the levels of advanced computing. The WG recognised this and via the ICT Sector Group and the Governments ICT Strategy sought to continually engage HEIs and FEIs. The Universities have significantly contributed to software business growth in Wales through programmes such as CEMAS, HPC, IT Wales, SAW and general support and KE initiatives, and the Welsh Universities produce 1,000 graduates each year in IT-related disciplines. The brief notes that computational science and engineering is increasingly acknowledged to be an equal partner with theory and experiment.


4 Market Background 4.1 Welsh ICT Sector CEMAS markets its services to eligible SMEs within the Convergence Region in Wales. CEMAS targets the ICT sector but also Welsh SMEs that can use apps as a vehicle to improving business competitiveness, route-to-market and reach. The ICT sector in Wales is one of the WGs nine key sectors, employing over 25,000 members of staff via 3,000 enterprises, comprising of over 95% SMEs. There are an estimated 11,000 IT professionals employed in the ICT sector with a further 25,000 IT professionals in other sectors.

The ICT sector is described by WG as being global and dynamic" with the sector positioned to design, develop, and commercially exploit cutting-edge technologies. ICT enterprises in Wales have enjoyed sustainable growth since the 2008 recession and by more than 10% pa since 2005. The sector continues to record the largest GVA figure of all the key sectors at 92,900. The ICT Sector Panel in Wales has commented that the sector could generate an additional 1.5bn GVA to the Welsh economy over the next five to seven years.

The UK app market currently leads in the EU with sales beyond 500m in 2012 after only four years. On 4 March 2013, the Wall Street Journal reported that the global app market is reaching some $25bn sales, with Google and Apple now offering over 700,000 apps in three stores.

4.2 European/UK App Market To identify and outline relevant market trends for the purposes of this evaluation, the Consultants utilised various sources of information, including: Sizing the EU App Economy, European Commission, February 2014 Internet Statistics Compendium, September 2012 4.2.1 EU Press Release Sizing the EU App Economy According to the European Commission, the EU's app sector has gone from zero to digital superhero in less than five years. By 2018 it could employ 4.8 million people and contribute 63 billion to the EU economy according to a report presented in Brussels today. The study, carried out by GIGAOM and NUI Galway for the European Commission, shows that Europe's app developers are up to the challenge of taking the global lead. Currently, EU and North American developers generate the same levels (42% each) of app revenues in crucial EU and US markets. Although the future is bright, developers have raised concerns about the skills gap, connectivity and fragmentation, which could put the app boom at risk.


Figure 2: EU App Market Revenues

Today the app economy employs 1 million developers, and 800,000 people in marketing & support posts. This could rise to 2.7 million developers + 2.1 million support staff by 2018. EU buyers and advertisers spent 6.1 billion on apps in 2013, 30% of total global app spending, growing to 18.7 billion in 2018. Consumer spending combined with advertising and contract work could lead to 63 billion annual revenue for the app sector within five years. Neelie Kroes, Vice-President of the European Commission, stated, "In the face of increasing youth unemployment, these figures give me new hope. The app sector is one area of the digital economy where Europe can really lead. But we have to address concerns about connectivity and fragmentation yet another reason to complete the telecom single market!" The study shows that: EU games app developers lead the field: 28 EU leading companies created 40% of the

top 100 grossing apps in the EU and US. Three of the top-five companies are Nordic games developers (1st King.com, 2nd, Supercell, 5th Rovio) with German, French, Spanish and UK app developers also finding success outside their native markets.

Growing market, growing jobs: In 2013, developers earned 11.5 billion making apps for consumer goods, banking, media, retail and other clients. They can expect to earn up to 46 billion through contracts of this nature in 2018. The app boom is creating jobs, for example contract developers Golden Gekko (London/Barcelona) plans to grow its staff 40-50% next year and London-based Grapple Mobile was a 3-person firm three years ago, employs 120 now, and intends to double next year.

Addressing the digital skills shortage: roughly 38% of independent and in-house developers said EU companies had difficulties competing with US salaries, 31% and 33% said that developer education was lagging, around 30% said start-up developers lacked


business expertise, and quarter of all surveyed said there were not enough developers. Worryingly only 9% of developers are female.

The Commission is tackling Europe's digital skills crisis in a number of ways. Firstly, by partnering with industry and other organisations in the Grand Coalition for Digital Jobs. In parallel, by working with schools to bring digital skills right into the connected classroom. Finally, by supporting grassroots initiatives such as EU Code week, organised by Neelie Kroes' young advisors. The first ever EU Code Week reached 10,000 people in 26 countries in November 2013. This year's Code week will take place on 11-17 October 2014

Technical bottlenecks and fragmentation were also a cause for concern. Roughly a quarter of developers wanted to see 4G develop more quickly in Europe. Around 35% were frustrated by the lack of interoperability between platforms like Android, iOS, Facebook. A majority of developers complained of their de facto total dependence on platforms developed by American giants, with subsequent revenue impact.

Neelie Kroes goes on to state that, "All apps and all mobile devices rely on broadband networks. Yet today, the framework for European telecoms is shattered and constrained. Meaning poor wireless connectivity, connections that can't easily cross borders, apps and services that are blocked or throttled by network operators, prohibitive surcharges to use your mobile abroad. In tomorrow's world, where even cars are connected, where we rely on mobile apps even for our healthcare resolving them will be absolutely critical. " Background The report, "Sizing the EU App Economy" was presented at a workshop in Brussels where Neelie Kroes, Rovio's Chief Marketing Officer Peter Vesterbacka and representatives of the digital community discussed the future of the EU app economy. Apps are self-contained programs or pieces of software that are mostly often designed for mobile devices or social platforms. For example, apps can advise you on the most efficient public transport route, help you bank online, track your runs or bike rides, match you with a potential partner, give health advice, games to entertain and relax. On the business side, many companies develop apps in house, or through contractors to support their main business, whether they are in financial services, consumer goods retail, media, etc. They use apps to add to their customer experience and utility, and to help market and sell their goods and services. According to estimates, 94.4 billion apps were downloaded globally during 2013. A digital single market of more than 500 million EU consumers has huge potential for Europe's digital business and app developers, but many are frustrated by inconsistencies which make it hard to sell their bright ideas across borders. The Commission is working to simplify EU rules on copyright and licensing to deliver greater access to online content. An ongoing consultation will feed into a white paper on copyright due before Summer 2014. Greater connectivity, including the high-speed 4G and broadband connections which fuel apps, are among the targets of the Commission's proposals to deliver a Connected Continent, currently being debated by the European Parliament.


Figure 3: EU App Jobs Market

Figure 4: EU App Market Revenues


Figure 5: Scale of the EU App Economy

4.2.2 Internet Statistics Compendium The Internet Statistics Compendium, published in September 2012, states that the size of the global market and forecasts for apps varies considerably. However, all forecasts predict continuing growth for apps in mobile devices over the coming years. The compendium of mobile statistics reported by mobiThink estimates that by the end of 2012, 1.2 billion people worldwide were using mobile apps. The Apple App Store and Google Play are the two largest stores with approximately 800,000 apps each. The number of downloads estimated for 2013 range from 56bn to 82bn, potentially rising to 200bn by 2017. According to industry analyst Gartner, app downloads are predicted to grow from 81bn in 2013 to 310bn by 2016, with paid downloads representing 10% of the market in 2013 and 7% by 2016. A report by Vision Mobile, estimates that apps developed in the EU28 generate 10.2bn of revenue which is 22% of the global value of apps developed. The US represents 42% and the Asian Pacific 18%. Revenue from the global market for app related products is expected to reach 51bn for 2013 and should treble within the next four years. Games make the most money from apps with 36% of all Apple App Store downloads globally generating over 71% of all revenue (MEF and Distimo, 30 Oct 2013).


Key facts include: 67% of smartphone owners have used a mobile app. [Source: Our Mobile Planet, May

2012] Smartphone owners have 25 apps on average downloaded to their device. [Source:

Our Mobile Planet, May 2012]

60% of UK smartphone owners have used an app. [Source: Our Mobile Planet, October 2011]

Figure 6: Average No. of Apps on a Smartphone

The average UK smartphone owner has 24 apps installed. [Source: Our Mobile Planet,

October 2011] 47% of adult smartphone users have ever downloaded an app, with one in five (20%)

doing so regularly. Regular apps downloaders are skewed male and age 25-34. [Source: Ofcom, August 2011]

54% of adult apps downloaders have paid for an app - with their mean average maximum spend on a single app being 3 3.99.

The top five paid-for apps are games (59%), music (30%), maps/navigation (28%),


travel/journey planning (21%) and books (14%). The top five most popular apps downloaded for free are games (73%), social networking (60%), music (37%), maps/navigation (35%) and news (35%).

Apps downloading is higher among teens than adults; around 63% of teen smartphone users have ever downloaded an app, with one in four (28%) doing so regularly. [Source: Ofcom, August 2011]

60% of teenage apps downloaders have paid for an app. The average maximum amount of spend among teens is 3.70 and the median is 3.00 - 3.99.

The top three most popular paid-for apps among teens are games (84%), music (57%), and social networking (22%). The same areas appear in the top three most popular apps downloaded for free: (games (64%), social networking (61%) and music (47%).

In the UK, Google Maps is the top connected mobile application with more than 6m unique visitors during April 2011. [Source: comScore, June 2011]

65% of the UKs connected mobile applications run via Apple iOS. [Source: comScore, June 2011]

Men are nearly 50% more likely to access the mobile Internet via an app than women. [Source: Orange, via The Guardian, October 2010]

46% of men who use mobile media use an app to do so compared to 35% of women. [Source: Orange, via The Guardian, October 2010]

Table 2: Total Connected App Users, UK

Around 70% of mobile internet users use a traditional browser, 55% use an app. [Source:

Orange, via The Guardian, October 2010] Most downloaded apps, UK [Source: digital strategy consulting, October 2010] 14% of mobile subscribers have downloaded an app in the last 30 days. [Source: Nielsen,

via AOP Digital Landscape Report, June 2010] 38% of online shoppers abandon websites or apps which take more than 10 seconds to

load. 45% then turn on to their competition. [Source: Rackspace, November 2011] 27% of UK smartphone owners are looking for a deal plan to use mobile app voucher

codes. [Source: Rackspace, November 2011] Over four out of 10 mobile users have used an app or browsed to search for

entertainment, apparel, consumer electronics and CPG/FMCG products during the purchase process. [Source: Microsoft, June 2011]


Figure 7: App usage in the Purchase Process

Figure 8: Types of App Downloaded

British smartphone users spend 581m a year via mobile apps. [Source: The BIO Agency,

via UTalkMarketing.com, March 2011] 78% of those who make purchases via smartphone apps cite convenience as the

primary reason for doing so. [Source: The BIO Agency, via UTalkMarketing.com, March 2011]


The top three product categories people buy via smartphone apps are groceries (75.69 a month), travel tickets (56.26 a month) and clothing (47.12 a month). [Source: The BIO Agency, via UTalkMarketing.com, March 2011]

How online drives offline sales Where UK consumers are finding the right offline products: [Source: Cisco, January 2012]

Retailer websites: 74% use now; 17% interested in using them in the future. Social networks: 30% use now; 15% interested in using them in the future. Mobile apps: 19% use now; 25% interested in using them in the future.

4m smartphone subscribers (25% overall) recalled having seen ads through a mobile browser or app in the quarter ending June 2011. [Source: comScore, September 2011]

Table 3: Advertising Reach

Apps Usage on Tablets

Figure 9: Types of App Downloaded


Games and 'just for fun' type apps are the most popular among tablet users. [Source:

Ofcom, July 2012] 33% of UK iPad owners have paid for 20-49 apps. [Source: Imano, June 2011] 4.2.3 General App information

Figure 10: Paid-For App Usage by Age Group

The App Market, Smartphones and Tablets

It is now not a surprise to state that smartphones and tablets are outstripping the sale of PCs. According to research by Deloitte (June 2013), smartphone owners reached 72% of all phones, 14% up on the previous ten months. Those aged 25 to 30 are most likely to own a smartphone (89%), followed closely with 85% of 16 to 24 year olds. By contrast, eMarketeer (March 2013) reported that in 2013 there are 30.9m smartphones in the UK (60.4% of all mobiles) and this is to rise to 43.4m by 2017 (80.9% of all mobiles). There are 13.4m tablet users in 2013 and these are predicted to rise to 23.2m by 2016. All figures show continuing growth in both smartphone and tablets as forecasters revise each new prediction upwards of the previous announcement.

According to a survey published by Flurry Analytics in October 2012, 56% of all smartphone app users are male while 51% of tablet app users are male. A third of all smartphone app uses are aged between 25 and 34. This age group is old enough to afford a smartphone but is also the generation that grew up with mobile phones. However, as the cost smartphones and tablets falls and their use are more readily deployed in schools and post 16 education, the age band of those who use apps the most is also likely to drop (Pew Internet and American Life Project, 30 November 2012).


Table 4: Worldwide App users by Age

Smart Phone and Tablet App Users Worldwide, by Age, Sept 2012

Smartphone app users Tablet app users

Gender

Male 56% 51%

Female 44% 49%

Age

13-17 18% 19%

18-24 21% 14%

25-34 33% 26%

35-54 21% 24%

55+ 7% 17%

Average Age 30 34

Source: Flurry Analytics as cited in company blog, 29 October 2012

There are increasing numbers of reports that challenge the stereo-typical impression that

men play more games than women, at least in the mobile and social mobile games segment of the market. That includes handheld devices particularly smartphones and tablets as opposed to PCs and laptops. Flurry reported that 53% of mobile and mobile gamers are women (February 2011), compared to just 40% of traditional gamers (PCs/Laptops). They particularly outnumber men in the over 25 age groups.

A Doritos study (July 2011) discovered that about the same numbers of women (49%) as men play games. More recently Mintel (September 2013) reported that more women than men played as casual players. While men go for the more action type games, women prefer puzzle games such as Candy Crush or Sudoku more often downloaded using apps on mobile devices. Men, they say, are more hard core gamers, deeply involved in the gaming culture, and regular gamers, enthusiasts, while there are more women who are casual gamers particularly on mobile devices.

Table 5: Frequency of Gaming Activity by Gender

Frequency of Gaming Activity by Gender Gender Level Percentage Users Male Hard core 13% Female Hard core 6% Male Regular 24% Female Regular 15% Male Casual 24% Female Casual 27%

Source: Mintel, September 2013

A number of publications have reported that more time is spent on smartphones and

tablets playing games and social networking than any other activity on the devices.


Figure 11: Category of App used by Platform

While Flurry report marginally more men than women using tablets, a more recent survey

by YouGov, in their Tablet Tracker report, in July 2013 has stated that 52% of all tablet owners are women compared to 43% last year. It suggests the change has been driven by smaller tablets being released onto the market, with women acquiring second-hand equipment from within the family.


5 CEMAS Output and Performance at the Mid-Term Point

Table 6: CEMAS Performance Against Targets (as of May 2014)

As of May 14 Target (June 15) % Achieved Enterprises assisted 101 138 73 Collaborative R&D 41 60 68 New/Improved Products 25 30 83 Products, processes or services registered 6 10 60 Enterprises created 5 6 83 Gross jobs created 8 11 73

It is the Consultants opinion that CEMAS is showing significant progress towards the attainment of stated funding targets. Although delays were experienced in the early stages of the Project, relating to recruitment of qualified staff etc, CEMAS management has properly addressed these issues and the Project should attain or even surpass targets by the end of this funding period. The Consultants have evaluated several similar ERDF programmes and it is a common aspect of the majority of programmes that target attainment comes to fruition in the second half of the funding period, once initial issues relating to launching such programmes have been addressed. It is the Consultants opinion that CEMAS compares favourably with other similar programmes at its mid-term stage and that the management team has built a strong programme that will perform well against all ERDF targets within this funding period. Figures 12 to 17 below show progress (actual delivery red line) against funding targets, as stated in the Business Plan (blue line), for each of the indicators utilised.

Figure 12: Enterprises Assisted

-2

18

38

58

78

98

118

138

Nov

-10

Feb-11

May-11

Aug-11

Nov

-11

Feb-12

May-12

Aug-12

Nov

-12

Feb-13

May-13

Aug-13

Nov

-13

Feb-14

May-14

Aug-14

Nov

-14

Feb-15

May-15

Business Plan

Actual Delivery


It can be seen in the majority of cases that, even accounting for the slight delays associated with Project launch, CEMAS is on or ahead of plan for each respective target. Indeed, only Collaborative R&D is slightly below target at the time of writing this report, but this is a target which CEMAS has been ahead of plan for much of the funding period and one which Project management are confident of attaining within the funding timeframe. It can be seen that progress against targets improves the later in the Project lifecycle you go. This is an aspect of ERDF-funded projects that is witnessed often by the Consultants and, as stated, we see no significant barriers to target achievement for CEMAS going forward.

Figure 13: Collaborative R&D

New or improved products (Figure 14) was a key area of engagement outlined by beneficiaries during the interview process and this is reflected in the strong progress made by CEMAS against performance targets.

Figure 14: New or Improved Products

0

10

20

30

40

50

60

70

Nov

-10

Feb-11

May-11

Aug-11

Nov

-11

Feb-12

May-12

Aug-12

Nov

-12

Feb-13

May-13

Aug-13

Nov

-13

Feb-14

May-14

Aug-14

Nov

-14

Feb-15

May-15

Business Plan

Actual Delivery

0

5

10

15

20

25

30

Nov

-10

Feb-11

May-11

Aug-11

Nov

-11

Feb-12

May-12

Aug-12

Nov

-12

Feb-13

May-13

Aug-13

Nov

-13

Feb-14

May-14

Aug-14

Nov

-14

Feb-15

May-15

Business Plan

Actual Delivery


Figure 15: Products, Processes or Services Registered

Products, processes and services registered took the longest of all performance targets to show any progress, reflecting planned targets set out in the Business Plan. Since November 13 progress has been good, The Project is ahead of plan and Project management are confident that this target will be met. Both enterprises and gross jobs created (Figures 16 and 17) tend to be sporadic in nature and this is something we see often in this type of evaluation. As the charts below identify, progress is good against both these targets and both CEMAS and the Consultants are confident that numbers will be achieved by the end of this funding period.

Figure 16: Enterprises Created

0 1 2 3 4 5 6 7 8 9

Nov

-10

Feb-11

May-11

Aug-11

Nov

-11

Feb-12

May-12

Aug-12

Nov

-12

Feb-13

May-13

Aug-13

Nov

-13

Feb-14

May-14

Aug-14

Nov

-14

Feb-15

May-15

Business Plan

Actual Delivery

0

1

2

3

4

5

6

7

Nov

-10

Feb-11

May-11

Aug-11

Nov

-11

Feb-12

May-12

Aug-12

Nov

-12

Feb-13

May-13

Aug-13

Nov

-13

Feb-14

May-14

Aug-14

Nov

-14

Feb-15

May-15

Business Plan

Actual Delivery


Figure 17: Gross Jobs Created

In conclusion, the Consultants see no significant barriers to CEMAS achievement of funding targets within the stated Project timeframe.

0

2

4

6

8

10

12

Nov

-10

Feb-11

May-11

Aug-11

Nov

-11

Feb-12

May-12

Aug-12

Nov

-12

Feb-13

May-13

Aug-13

Nov

-13

Feb-14

May-14

Aug-14

Nov

-14

Feb-15

May-15

Business Plan

Actual Delivery


6 Interview Programme Analysis

6.1 Internal Stakeholders 6.1.1 University Representatives As part of the mid-term evaluation, the Consultants interviewed two key internal stakeholders: 1. Professor Clive Mulholland, Deputy VC, Research and Student Experience, University of

South Wales; and 2. Luc Brown, KT Manager, Commercial Services, University of South Wales. Views on the CEMAS project: Professor Mulholland stated that whilst he did not have direct responsibility for CEMAS, he occasionally met with CEMAS director Professor Khalid Al-Begain to discuss CEMAS activities and performance. Professor Mulholland explained that through these meetings and other communications he could gauge that CEMAS was one of the better-run initiatives that has worthwhile and continuous engagements with SMEs, and which he believes has a developing and sustainable future. Luc Brown recalled the history of CEMAS, acknowledging that it took three years to get to contract. Luc claimed that this resulted in the Project director having the luxury to observe the market, the development of apps, and plan thoroughly for launch and the development of the Project. Luc went on to comment that despite early uncertainties associated with an industry in its infancy, CEMAS has developed into a well-run and well-managed project, and despite the depths of the recessions, apps was a growing market which CEMAS understood well. Luc felt that CEMAS had also grasped the marketing challenge in terms of engaging and working with SMEs, resulting in what Luc described as possibly the best programme he had seen under the ERDF, to date. Interestingly, Professor Mulholland observed that the CEMAS team had adapted admirably to a changing market, a market which was uncertain and had called for them to enhance their own experience and skill-set over the years. Professor Mulholland commented that there will be an ever-demanding and widening market for applications, and that for CEMAS to grow and prosper, it needs to be continually innovative in its product offering. Professor Mulholland felt that this was necessary on two fronts: firstly, so CEMAS can be in a position where it may conceive of displacing the market and particularly the private sector, and secondly so it can demonstrate that it will add significant worth and a longer-term, sustainable delivery. In regards to the latter, Prof Mulholland described this in terms of moving to the next platform, ie 5th generation, taking apps into specific sectors, ie engineering and healthcare, developing apps which have a price premium, and moving apps into mainstream and two-way communications. Finally, Professor Mulholland commented that a future Structural Fund, or possibly a Horizon 2020 bid, should work by linking apps with functional responsibilities. When discussing the future product/service direction with Luc Brown, it was clear that he had a more tactical day-to-day appreciation of CEMAS. From Lucs own perspective he concurred that there is great opportunity for the sustainability of CEMAS and believed that this was principally two-


fold. Firstly, to continue to serve Welsh-based SMEs. Secondly, Luc thought there would be great opportunity to tie in CEMAS more closely to the key WG sectors, with ICT being an obvious one. Luc recognised that CEMAS had run a number of high-profile events with the ICT/digital sector. In terms of other sectors, Luc was aware of discussions around health and the aging population and again cited a number of grant/support routes-to-market, namely Horizon 2020, commenting that this may be more appropriate as CEMAS has not had great success with FP7 and TSB calls in the past. In that regard, Luc now believes that, based on experience gained over the last two years, CEMAS is probably better equipped and has a stronger knowledge base, and is able to write more successful proposals. This is where commercial services can continue to help and possibly CEMAS could approach WEFO. In terms of health and the aging population, Professor Mulholland is willing to support developments in this area, believing there is other expertise within the University that CEMAS can draw upon and that this is a subject where networking with other universities/research centres inside and beyond Wales makes great sense. Conclusions Internal stakeholders within the University are keen to support CEMAS in its development, believing also that there is a strong educational training and CPD element. At an educational level, both staff within CEMAS and internal stakeholders are convinced that they have the opportunity to support STEM subjects and encourage young people to consider a career in computing and programming. As Welsh Government is acutely aware, there is a shortage of 3,500 IT professionals within Wales, which is only going to worsen as the economy grows. By using apps and games, CEMAS can contribute to the school curriculum, giving students and children good experience and an appetite to want to follow such careers. It was also recognised that, potentially through ERDF, there is an emerging opportunity to train communications and marketing professionals to the point that they can start to develop and deliver apps on behalf of their clients. It is clear that the University is keen to cascade knowledge down into the business community, again possibly through an ERDF project, so as to eliminate current market failure and move on to the next app/game challenges. University stakeholders also appreciated that games are increasingly being deployed in many areas but particularly in health and wellbeing, as witnessed within the current cohort of beneficiary businesses. Here, games are used to educate children in terms of healthy eating and to train health professionals to recognise disease earlier. Internal stakeholders appreciated the value of building R&D competence and capacity within CEMAS. R&D capacity will lead to new revenue streams, potential spinouts, and sector specialisation, according to Luc Brown. Luc stated that, This will start to challenge management and they will also face challenges such as delivering commercial income streams. As with many if not all ERDF/ESF projects, they still (after one funding round) require other forms of funding to carry them through to self-sustainability. For this to be achieved, CEMAS needs to think, not in terms of a three-year business strategy but, ideally, six.


In terms of future market direction and sectors, both Professor Clive Mulholland and Luc Brown felt that they did not have sufficient market information to comment, but both believe that management is now looking at market opportunities to profile new product/service delivery. 6.1.2 CEMAS STAFF During this exercise, we interviewed the following CEMAS staff: Dan Rees, Senior Programmer, joined August 2010 Nidal Albeiruti, Senior Programmer, Joined August 2011 We asked the members of CEMAS staff for their views on the original rationale for the Project, and to what degree it remains relevant today. Answers were based around the objective of de-risking investments in mobile applications for SMEs and assisting SMEs in creating applications that provided their business with key benefits. It was generally felt that the rationale is still current, however, the solutions are becoming more sophisticated as technology advances apps become more widespread and indeed SMEs become more demanding in terms of the features and functionality of their apps. We asked interviewees to comment on the product/service approach and positioning of CEMAS. Again, there was commonality in their responses, which is indicative of a team that works together and has a common vision. Their view on products/services position was that it was fit-for-purpose and the client journey readily and easily understood. A detailed and typical answer on Stages One, Two, and Three was given by Dan Rees and has been outlined below:

Stage One: A workshop seminar and one-to-one support to help develop the business case for your mobile application-based idea. Typically at this stage, the product specification is produced. If a valid app idea exists youll be required to sign a collaboration agreement with CEMAS in order to proceed to stage two. In terms of stage 1, Dan appreciates that they have targets and he believes they are achieving those targets. Indeed, we now probably turning away more projects than previously, but usually for the same reasons, i.e. it has been done before or it wont add value. Occasionally we help to fill in the original application document because we can articulate the job opportunity better than the beneficiaries. Stage Two: The idea from the previous stage is designed, developed, and tested accordance with the agreed specification. Companies may choose to exit the programme at this point, with the app, although associated foreground IP will be openly disseminated by the University. Dan stated that, This is where the bulk of my work is. For most apps we are probably working reasonably full-time, five days per week for 3 or 4 months. We often break the task down and more than one person could be working on that specific app. In terms of games, this is a recent addition that takes much longer usually because of the creativity around the visuals and images. Stage Two allows


you to interact with the client sufficiently enough to ensure that what is produced meets and hopefully surpasses the client's needs and recommendations. Stage Three: Commercialisation and marketing support for launched product. CEMAS can also provide assistance with the IPR protection and development of products at 50% of legal costs. Dan stated that this is usually when Louise Clements, and also Martin, assist on marketing. Dan can see a real need for Stage Three but in many instances CEMAS is working with companies that already have the capacity and understanding. It is evident that all stakeholders believe this stage gate approach to be effective when dealing with SMEs, it keeps the SMEs engaged, and they take away greater understanding of applications, more appreciation of their marketability, and experience which can have long-term benefits for the businesses. The evaluation sought to identify any patterns within industry applications and sectors. Unsurprisingly the common areas, ie B2B, ICT, professional services, and education were mentioned. Dan commented that recently there seems to be an emergence of interest within agricultural farming, husbandry, and animal health and it is clear that internal staff are not overly concerned with what industry they operate in. Internal stakeholders were keen to identify how the product/service offering could be developed, suggestions including offering NVQs, high level skills training, working closely with primes and their supplier base (BT mentioned specifically), and expanding the product offering to include pay-as-you-go type products, renting mobiles, hosting on cloud, and providing backend services. As far as members of staff are concerned, opportunities to develop apps and games would appear to be almost endless and clearly they appreciate that there are also opportunities to charge for products and services to allow CEMAS to move to a position of self-sustainability. Employees were extremely complementary about the management, management style, and the creative and supportive environment within CEMAS. Management would appear to encourage staff to develop and contribute and are fully supportive of personal development. We asked staff to think about what the Project could possibly do in the remainder of the funding period and beyond, with the following comments: We could move from apps and games into websites if there is market favour. We could do more to identify novel ways of doing things, creating IP, and possibly

patents. Work at developing niche expertise in key areas such as assistive living. Take on more research related projects, to build skills and competencies, and to add

clients longer-term benefits. To promote expertise in Wales, internationally.


Staff interviewed and their colleagues generally believed that CEMAS had made efforts to engage with and work alongside other ERDF and ESF projects. As one interviewee commented, Where we do seem to work well with others is in promotion and this has led to a number of CEMAS projects. Another staff member commented that, We could possibly do more in collaboration, i.e. we have tried to work with High Performance Computing (HPC) Wales on a number of occasions but have been relatively unsuccessful. Here, we were looking to share skill sets to produce added value. With HPC Wales and Software Alliance Wales (SAW) it should be easy to work together, considering the Universities are also delivery partners for other ERDF/ESF programmes. Whilst it makes sense to work with other programmes, it was also suggested that if CEMAS had funding to develop its own IP, this could provide benefits to the Welsh economy, wellbeing and society.

6.2 External Stakeholders The Consultants spoke with Caroline Thompson, Interreg Programme Manager, Inventorium, Bangor University, Dave Thomas, Managing Director, Bridge Innovation Centre, Pembrokeshire, and Richard Hooper, WEFO. Key points from Caroline Thompson: Note: This programme has now finished, however they worked with CEMAS in their later

months. Caroline informed us that the Interreg programme went extremely well. Caroline response to what she thought the rationale and operations of CEMAS were to

enable innovative companies looking to develop mobile apps. Caroline commented that their target market was not run of the mill businesses, citing four childrens nurseries that are working together.

Caroline has mainly been supporting CEMAS to engage with businesses in North Wales, who didnt have a relevant database.

Caroline was impressed by the professionalism of the academics and senior management at CEMAS, identifying that they operate in an extremely professional manner and make good use of event management software, and applications such as survey monkey.

As far as Caroline is aware, from their dealings with client companies, the feedback again is that they are extremely professional, and this is confirmed by the SMEs themselves. However, SMEs found it frustrating that it takes so long to get a project off the ground, given the available resources and the time invested in the project.

With regards to future collaborations, Caroline mentioned that they (Inventorium) had scoped out a Horizon 2020 bid and would certainly consider working with CEMAS. They work with an Irish company/organisation on their Interreg project, which was centred on UIT and apps and built up a good understanding. Caroline commented that if they are willing to work on Horizon 2020, they have discussed the opportunity of working on a Structural Funding bid, and maybe the opportunity of Bangor covering the northern region on behalf of CEMAS. Caroline would clearly be prepared to discuss and consider this.

Key points from David Thomas: David commented on the professionalism of the CEMAS team and staff, and their

complete understanding of the product, service, and market landscape. CEMAS is one


of, if not the, most professionally-run ERDF projects which interacts with the Bridge Innovation Centre.

David commented that CEMAS had a track record in delivering and supporting businesses, and clearly understood the needs of companies within the SME space.

David has worked on a number of events with CEMAS to basically promote CEMAS to his client base. He felt that these have gone down well and that he knows that a number of companies have gone on to benefit from CEMAS support. Those companies have been complementary on the quality and professionalism of received support.

David commented that they jointly ran a 'mobile apps in schools' competition with CEMAS, which went extremely well. This engaged with school children to increase understanding of the STEM subjects and received acclaim from the school headmaster, which was publicised.

We talk to David about the sustainability of CEMAS and it was his view that apps are still seen as gimmicky and mainly for amusement and leisure purposes. However, David could understand how apps would evolve to become serious productivity tools for business. David commented that he recognised in a region such as Pembrokeshire that business is sparse and those businesses need to engage and communicate in informative ways and apps can sometimes act as an extension of a website and other electronic materials and communications. As far as David is concerned, any support that he can give to get the message to local business he would endeavour to do so and he welcomes a long-term working relationship with CEMAS

David stated that he wants the relationship to continue where possible and to possibly be placed on a more formal footing so that their particular region can continue to benefit.

Key points from: Richard Hooper We attempted to meet with Richard Hooper, Project Development Officer for CEMAS at WGs European Funding office but we were unsuccessful. Richard did, however, respond to our questions by completing the mid-term evaluation questionnaire. Extracts from the questionnaire are as follows: Richard commented that, The objective of this project is to create the first ever Centre

of Excellence in mobile applications and services within Wales. The purpose is to assist SMEs in accessing and receiving academic/industry-based expert advice to create, develop and test innovative new app services and products pre-launch.

We asked Richard to comment on the recent economic conditions and he replied by saying that there had been an increase in demand for the Projects services, as evidenced by recent business cases provided to WEFO and approved to support additional European grant. This was very much seen by Richard as alleviating the bottleneck that had been generated in part from the greater marketing activity of CEMAS. Richards response clearly indicated that there was a specific need for CEMAS, and Richard clearly appreciated all three stages of CEMAS engagement

We asked Richard to comment on the management challenges within the Projects sector and Richard responded by saying: A positive challenge has been the unforeseen scale of growth in mobile devices and applications since the Project was launched. The Project has overcome this by requesting and receiving additional grant to help address the demand for its services. The fact that grant was provided was a positive recognition of the work undertaken by CEMAS.

Richard commented further by stating that, The Project is on track to achieve/exceed its approved targets.


In terms of the overall position of the Project at the last claim, Richard commented that it is performing adequately. (Note: improved performance since this claim has been witnessed). Richard also said that there is a current under-spend of 16% over the entire Project costs, which will be discussed at future review meetings. Depending on the rationale, the Project may have to undergo a further re-profile. The forecast figures indicate that the Project is on course to achieve its outcomes. Richard concluded by saying that he did not have any concerns at present in relation to the Project, the management, and its position.

In terms of general observations, Richard mentioned that there is scope within the existing operational programmes for projects to collaborate with organisations of other member states. The Project has already attended a World Mobile Congress Conference in Barcelona to network and provide publicity for Welsh SMEs.

6.3 Beneficiary Companies 16 beneficiary company interviews were completed for the purposes of this exercise, namely:

Table 7: Beneficiary Interviews Completed

Name Organisation Position

Nicola Knight Taste Wales Proprietor Mark Davies iMD Systems Development Manager Marc Griffiths StwdioBox Owner Sam Wright Bumpy Box Director Louise Austin Dyfed Archaeological Trust Head of Heritage Richard Wilson Dezrez Services Ltd Technical Director Aran Jones Say Something Cyr CEO David Messruther C-Toolz Director Ian Williams Create Two Eight Ltd Managing Director Dr Nerys Jones Agri Advisors Solicitors Principal Sebastion McBride Blue Steel Images Ltd Director Gareth Cavanagh Irontown Interactive Director Owain Schiavone Golwg Cyf Director Emily Rees Emily Rees Sports Massage Owner Adam Armytage AT Entertainment Ltd Director Robin Lyons Calon Digital Managing Director 6.3.1 Beneficiary Profile The geographic split of companies interviewed during this exercise can be seen in Table 8 below:

Table 8: Geographic Split of Interview Sample

Region No. of Interviewees Carmarthenshire 4 Rhondda 4 Ceredigion 3 Swansea 2 Merthyr Tydfil 2 Caerphilly 1 Total 16


The most common sector by far for interviewees was Creative industries, which would fit with general demand for app development. CEMAS does see a good spread of sectors, from ICT to food and farming and does cover many of the key WG sectors.

Table 9: Industries Represented within Interview Sample

Industry No. of Interviewees Creative Industries 5 Energy & Environment 2 ICT 2 Financial & Professional 1 Food & Farming 1 Other 5 Total 16

Table 10: Length of Time Trading

Industry No. of Interviewees Less than 1 year 1 Between 1 and 3 years 9 Between 3 and 5 years 0 Between 6 and 10 years 3 More than 10 years 3 Total 16

The majority of companies interviewed as part of this evaluation have been trading for less than three years and are either an independent sole trader or limited enterprise. As would be expected, the majority of the sample has one key employee, with only three having more than 11 employees.

Table 11: Type of Organisation

Industry No. of Interviewees Independent Sole Trader 5 Partnership 0 Limited Enterprise 8 Charity 1 Division of Large Enterprise 1 Other 1 Total 16

Table 12: No. of Employees

Industry No. of Interviewees 1 9 2 0 3 to 5 3 6 to 10 1 11 to 50 2 51 + 1 Total 16


The highest numbers of respondents were pre-revenue, with a further seven having a turnover of 99k or less. Sample respondents did, however, represent a good spread, with three interviewees stating a turnover of 500k plus.

Table 13: Turnover Band

Total () No. of Interviewees Pre-revenue 5 Up to 49k 3 50k to 99k 4 100k to 199k 0 200k to 499k 1 500k + 3 Total 16 6.3.2 Support Provided It can be seen from the evidence obtained during interviews that the majority of beneficiaries (ten or 63% of the sample) were at the Ideas Only stage of development when their engagement with CEMAS began, followed by Basic Structure & Outputs with three responses (19%) and Market Research, Fully Working Prototype, and Other (Calon Digital) with one response each. When questioned on progress made to date, it can be seen that six of the interviewees (38%) now have an App that has been live for 12 months plus, five (31%) are in the Preparing to Launch stage, followed by two (12%) who are currently in the Launch process. One interviewee (6%) has an App which has been live for less than six months, and one again (6%) between six and twelve months. Finally, one interviewee (Calon Digital) did not progress past the initial stages of engagement. Comments included: Having got to the stage that we have through CEMAS help, we are now speaking with

Head of Nutrition at Welsh Government, Save the Children, and other organisations about marketing, communications and growing the venture. (Nicola Wright Taste Wales)

When asked about their motivation for engaging with CEMAS in the first instance the following responses were provided (note: interviewees were allowed to tick more than one box):

Table 14: Motivations for engaging with CEMAS

Motivation No. of Responses Percentage of Sample Getting product to market 12 75 Desire to grow enterprise 6 38 Desire to enter new markets 6 38 Improve the way the enterprise is marketed 4 25 Encouraged by CEMAS 4 25 Access to the latest thinking/technological change 3 19 Keeping up with competitors 1 6 Pressure from customers to improve 1 6 Pressure from supply chain to improve 1 6


Example comments included: It was the technical side of producing an App that we required. We understood

animation greatly and needed to put these things together. We identified that an app would give a quick route-to-market and we wanted to work with people that really understood how to create an app. We have gathered a lot of understanding from this venture, which gives us the opportunity to develop our own apps in the future. We appreciate that we cannot be supported again under an ERDF-funded project but we have considered, and are willing to pay for, future support to make our apps more attractive and to benefit from app development generally. (Sam Wright, Bumpy Box)

Dyfed Archaeological Trust also stated that one motivation was to create political connections.

When asked how they first heard about the CEMAS service Word of Mouth was the most common answer, given by five of the interviewees (31%), with three interviewees (19%) stating that they first heard of CEMAS at an event (presentation at Aberystwyth University mentioned twice). Networking was identified by two interviewees (12%) as how they were first alerted to the CEMAS service, with the CEMAS Website, Telephone Call from CEMAS/Advisor and Advertising getting one response each. In addition to those channels stated, one interviewee heard of CEMAS through the South Wales Chamber and one saw a press release from the WG (possibly advertising?). The Consultants then asked the interviewees what level of engagement they had experienced with CEMAS (Stage 1, 2 or 3), with the majority (15 companies 94% of the sample) reaching Stage 2. Five beneficiaries (31%) had reached Stage 1, with only one of the interviewees (6%) reaching Stage 3. Interestingly, three companies stated that they would like to reach Stage 3, with costs generally being mentioned in the same sentence: The Trust would be interested in Stage 3, however they have limited funds. To date they

have invested in IT and staff time, and between 5k - 10k into the project, which they generally feel is money well spent. (Louise Austin, Dyfed Archaeological Trust)

They would like to progress to Stage 3, they appreciate that there will be costs that they will need to accrue but they feel it will be of great benefit. (Sam Wright, Bumpy Box)

We would now consider (when knowing more about the content) moving to Stage 3 with other projects if this is now permissible? (Mark Davies, iMD Systems)

The Consultants thought it useful to assess beneficiaries satisfaction levels with various aspects of the CEMAS service offering and, to this end, asked the respondents to rate each aspect on a scale of 1 to 5 (I being very satisfied and 5 being very dissatisfied). Weighted scores for each aspect have been provided in Table 15 below:

Table 15: Satisfaction with CEMAS Service

Aspect of Service Weighted Score

The CEMAS proposal 1.20 The proposals applicability to the beneficiarys needs 1.26 CEMAS understanding of the beneficiarys needs 1.31 Competence and Professionalism of CEMAS staff 1.31 Perception of the quality of facilities and equipment available 1.31 Follow-on support received 1.40 The final result 1.50 Communications/discussions throughout the project 1.63 Note: due to the scoring system used, the lower the weighted score, the better the result.


When recording the above scores, one beneficiary, Calon Digital, outlined an unsuccessful engagement with CEMAS and therefore provided very low scores for the areas of competence and professionalism, final result and communications, reducing the overall weightings for these aspects. It should be noted that Calon was the only beneficiary out of 16 to outline an unsuccessful engagement and CEMAS management were happy for TIP to include the company in the interview process. Having discussed this issue with CEMAS management, it is the Consultants opinion that Calon was an early engagement and that lessons have been learnt in terms of project management and managing expectations. We do not see any further issues in terms of Project success and target attainment, indeed we would suggest that this evaluation has identified levels of customer (beneficiary) satisfaction that are amongst the highest we have witnessed for a project of this nature. When asked what the engagement had delivered to their enterprise, the beneficiaries provided the following examples: It (engagement) firstly allowed us to get to proof of concept stage, not necessarily with

the app that we had faith in, but to understand how to present a product to the sports community and to gauge how beneficial it will be. We learnt a lot of lessons and it put us in excellent stead for the future. We are now thinking of developing apps for other sports and looking towards developing more feature-rich products, which would be chargeable! (Mark Davies, iMD Systems)

The engagement allowed us to get into the childrens digital market and it strengthened our underlying knowledge of the app market by giving us greater insight. (Owain Schiavone, Golwg Cyf)

Whilst we have not yet launched, we are sure it will deliver opportunities to take a product to market, define a new route-to-market and a way of self-publishing animation. At a second level it is probably giving us the confidence to work with other organisations content to bring the animation and create apps and potentially move forward, to create interactive games for the educational community. (Sam Wright, Bumpy Box)

It has allowed us to be more competitive with rival mobile applications companies. (Richard Wilson, Dezrez Services)

CEMAS has helped to increase the radio stations ratings. The App has been live for one week and it has already had 25 downloads and considering that the App has not yet been marketed, this is a good response. (Marc Griffiths, StwdioBox)

Ian received an iPhone equivalent version of his original android App. Ian also had his product branded by the University of South Wales (Ian being a self-proclaimed developer that lacked strength in the graphic design function), which he saw as a real bonus. CEMAS has provided him with a logo, a brand image, and brand guidelines, which will open his company to a new market. The only thing that is required for Ian (after Apple approve the App) is a marketing strategy, which he admits is not one of his strong points. CEMAS has also offered to help with this. (Ian Williams, Create Two Eight Ltd)

Due to the engagement with CEMAS, Davids company has gained creditability and professionalism by having its own app that is used by other electrical engineers worldwide. As the app is being used worldwide, this means his brand image is also being spread on an international level. (David Messruther, C-Toolz)

The engagement basically got the product to market, which I couldnt just do under my own resources. I believe that the product has now gained significant added value,


and its a product that shows people that I care about the health of the nation. (Nicola Knight, Taste Wales)

Added Value The real added value was it showed the Company a route-to-market previously

unconsidered due to lack of understanding. Whilst this route to market has not provided fantastic sales results to date, we have learnt a lot for future engagement. (Mark Davies, iMD Systems)

The real added value is they brought with them a skill-set that has allowed us a route-to-market. Any other route-to-market for our product would be expensive and take considerably more time. (Sam Wright, Bumpy Box)

Added extra resource, enhanced our skillset and improved the volume of work. (Richard Wilson, Dezrez Services)

It should be noted here that, when the interviewees were questioned about being unsatisfied with any areas of engagement, only two responses were obtained, namely: The only thing that concerned Ian was the lack of creative reiterations (x3) allowed in

the creation stages. This caused some tension and heads collided as Ian was worried what they would do to his product, and wondered if they had actually listened. However, when the final product was delivered Ian was more than happy. (Ian Williams, Create Two Eight Ltd)

We were unsatisfied against a number of metrics, performance, understanding, quality and competence. This has significantly delayed the project to the point that two of our graduates have moved on to other projects. From this project there is very little that we can rescue when we wish to take the project forward, which we will do under our own devices. (Robin Lyons, Calon Digital)

In relation to follow-on support, a minority of interviewees stated that they would have liked follow-on assistance for a specific activity. These included iMD Systems who would have liked help with app debugging, and Blue Steel Images who mentioned development of the app for the Android platform. C-Toolz and Agri Advisors commented that further commercial support post-launch might be helpful. The remainder of the sample were unsure as to what follow-on support they required or indeed were eligible for, particularly in relation to further funding etc. 6.3.3 Enterprise Performance & Outcomes The Consultants then quizzed the beneficiary companies on outcomes that had already been achieved as a result of engagement with CEMAS. Increase in Sales was the most common answer with eight respondents (50% of sample), followed by Entered New Markets which was identified as an outcome by seven interviewees (44%). Produced New Products followed with four responses (25%), with Increase in Profits and None Achieved to Date both receiving three responses (19%). When asked to provide a financial estimate for sales/profits seen, it was clear that the majority of interviewees were still in the early stages of commercialisation, with many seeing the launch of this 1st generation app as a stepping stone to future product development as opposed to a sales generation exercise. Those who could provide some figures included: Agri Advisors (30 downloads at 9.99) Blue Steel (200 downloads at 1.49)


iMD Systems (circa 1k of sales to date) Dezrez Services stated that the app has potentially and indirectly added value to t he

Company of 1-5% Emily Rees (not wishing to divulge too much information as two patents pending) stated

that the app is a key component of a product that is forecast to see sales of circa 1.000 units at 300 per unit.

In terms of jobs created/safeguarded, many of the interviewees found this difficult to answer as a result of being sole traders or limited companies with one employee, with eight (50%) commenting that this is not applicable to them. Those who could answer stated that: Dezrez Services confirmed that two jobs had been safeguarded and that new jobs had

been created as an indirect result of the engagement, with the potential to create further jobs going forward

Golwg Cyf confirmed that two jobs had been safeguarded Bumpy Box confirmed that one job (animator) had been created as a result of CEMAS

engagement Emily Rees stated that there is the potential for three jobs to be created (1x project

manager, 1 x admin, and 1x marketing) It was suggested by the majority of beneficiaries that the access to expertise and funding was key to the development of their specific app/project, with several stating that this process would not have taken place, or would have taken significantly longer, without CEMAS engagement. 6.3.4 Future Impacts It should be noted here that as the majority of the beneficiaries interviewed were sole traders or micro-businesses, they found it difficult to identify future impact as a result of CEMAS engagement due to limited forecasting and strategic planning. It can be said that only two interviewees did not expect to see any further commercial benefit going forward, with six beneficiaries (Dezrez Services, Emily Rees, Bumpy Box, Create Two Eight, Say Something Cyr and C-Toolz) expecting to see increased sales going forward. StwdioBox, Bumpy Box and Create Two Eight also identified increased profits as a future impact they expect to see. Aside from Emily Rees, whose forecast is outlined in 6.3.3 above, none of the companies identified above were able to provide an estimate of sales they expect to obtain. In terms of job creation/safeguarding, Dezrez Services, Bumpy Box, Create Two Eight, Golwg Cyf and Emily Rees expected to see activity in these areas as a result of CEMAS engagement, with Emily Rees stating that planned recruitment is forecast to happen within two years of product launch. Wider Benefits In trying to identify wider benefits, the beneficiaries were asked to rate the following innovation enhancements on a sliding scale. Weighted scores have been provided in Table 16 below:


Table 16: Innovation Enhancements Experienced by Beneficiaries

Innovation Enhancement Weighted Score Improved knowledge of apps 1.1 New or improved knowledge, processes or approaches 1.7 Effective marketing and communications 2.3 Improved higher-level skills or management skills within the firm 2.8 Improved operational efficiency 3.2 More effective e-commerce 3.3 Note: due to the scoring system used, the lower the weighted score, the better the result.

The interviewees were then asked about potential customer benefits that may occur as a result of the beneficiarys participation in the workshops:

Table 17: Potential Customer Benefits

Customer Benefits Weighted Score Better understanding of customer requirements 1.3 Helped beneficiary retain customers/enhance loyalty 1.5 Established links with/developed local supply chain 1.5 Converted visitors to customers/delivered new customers 2.0 Developed, maintained and/or grown market position 2.0 Generated a higher quantity and quality of enquiries/leads 3.3 Note: due to the scoring system used, the lower the weighted score, the better the result.

Finally, the beneficiaries were asked to what extent they believed their enterprise has the skills and knowledge to make future technological improvements as a result of their engagement with CEMAS. Only one interviewee (Calon Digital) did not believe that their knowledge had been enhanced, with five beneficiaries (31%) believing their skills had been improved to a great extent, six (38%) answering to some extent and three (19%) answering to a small extent. Other comments: The whole process was positive. There has been a coming together of technical

expertise and aesthetic desig

Date post:	13-Oct-2015
Category:	Documents
Upload:	mikecaluan
View:	201 times
Download:	0 times

CEMAS Final MT Eval Report

Documents