Prozorro Project

UKMD-2014-11-01: Ukraine - Policy Advice for eProcurement Reforms in Public Sector

Initial Audit

of Existing eProcurement Solutions

Operating in the Public Sector in Ukraine

(Prozorro Project)

Draft Report after Revision

Draft date:22 May 2015

Review date: 27 May 2015

Reviewed draft: 05 June 2015

Final report due: 10 June 2015

Draft Revised Report: Initial Audit of Existing eProcurement Solutions Operating in the Public Sector in Ukraine (Prozorro Project)

1 UKMD-2014-11-01: Ukraine - Policy Advice for eProcurement Reforms in Public Sector

TABLE OF CONTENT

List of tables ............................................................................................................................... 2

List of figures ............................................................................................................................. 2

List of abbreviations ................................................................................................................... 3

List of appendices ....................................................................................................................... 4

EXECUTIVE SUMMARY ........................................................................................................ 5

1. Prozorro project concept ................................................................................................... 12

2. System' s compliance with EU directive guidelines ......................................................... 31

3. Review of compliance with standards recommended by the EU eProcurement Golden

Book of Best Practice ............................................................................................................... 36

4. Asessment of software development maturity .................................................................. 42

5. Compliance with other MBD requirements: ..................................................................... 50

6. Prozorro end-user review .................................................................................................. 52

7. SWOT analysis of prozorro .............................................................................................. 52

8. Recommendations ............................................................................................................. 57



List of tables

Table 1 Prozorro Project Milestones 12

Table 2 Functional scope 18

Table 3 Checklist - the EU Golden Book of eProcurement Practice 36

Table 4 Business requirements 42

Table 5 Technical requirements 42

Table 6 Data storage needs 43

Table 7 Interface / communication requirements 43

Table 8 Transaction requirements 43

Table 9 Network capacity requirements 44

Table 10 Software development requirement 45

Table 11 Software development requirements 46

Table 12 Testing requirements 47

Table 13 Application management requirements 47

Table 14 Service requirements 48

Table 15 Training requirements 49

Table 16 Compliance with other MDB Requirements 50

Table 17 Compliance with other MDB Requirements 31

List of figures

Figure 1 External structure of the Project 15

Figure 2 Internal Structure of the Project 17

Figure 3 Major elements of the Platform 24

Figure 4 Technology stack 27



List of abbreviations

CAs Contracting Entities in Public Sector

CPV Common procurement vocabulary

DB Database

EOs Economic Operators/Suppliers

MEDT Ministry of Economic Development and Trade of Ukraine

NCR National Council of Reforms of Ukraine

PROZORRO

eProcurement project organised and operated as a public-private partnership by

Transparency International (Ukraine), based on the Memorandum launched by

Openprocurement.org

SLA Service Level Agreement

TED Tenders Electronic Daily

TI Transparency International (Ukraine)

TLS Transport Layer Security



List of appendices

APPENDIX 1 Content: Contracts between TI, Quinta Group and example of a

contract with commercial platform operators

Data acquisition

date: 10 May 2015

Source:

Mr Andriy Kucherenko

Position / role in the project structure:

Prozorro Project Coordinator

APPENDIX 2 Content: Completed audit questionnaires

Data acquisition

date: 10 May 2015

Source:


Mr Myroslav Opyr



Chief Technology Officer, Quinta

APPENDIX 3 Content Transparency International – interview questionnaire

Data acquisition

date: 19 May 2015

Source:

Mr Viktor Nestulia


Senior Analyst, Transparency International (Ukraine)

APPENDIX 4 Content Ministry of Defence – interview questionnaire

Data acquisition

date: 19 May 2015

Source:

Mr Artur Pereverzev

Mr Yuriy Husiev


eProcurement Project Manager, Ministry of Defence of

Ukraine, (contracting entity, ProZorro end-user)

Deputy Minister of Defence

APPENDIX 5 Content Record of purchasing procedures completed on Prozorro

during piloting (until May 18, 2015)

Data acquisition

date: 20 May 2015

Source:




APPENDIX 6 Content Prozorro Risk Mapping Report

Data acquisition

date: 19 May 2015

Source:




APPENDIX 7 Content Statements of Work signed between Transparency

International Ukraine and Quintagroup for developing

Prozorro dedicated software

Data acquisition

date: 19 May 2015

Source:

Mr Myroslav Opyr


Chief Technology Officer, Quintagroup

APPENDIX 8 Content Prozorro Release Management Standard

Data acquisition

date: 10 May 2015

Source:






EXECUTIVE SUMMARY

1. A review of existing eProcurement solutions operating in the public sector in Ukraine

was undertaken by the European Bank for Reconstruction and Development (EBRD)

under the technical cooperation project: UKMD-2014-11-01: Ukraine - Policy Advice

for eProcurement Reforms in Public Sector. The project launched in August 2014 by

the EBRD Legal Transition Programme and the Ministry of Economic Development

and Trade of Ukraine is dedicated to supporting the government of Ukraine in

designing, developing and implementing electronic procurement in public

procurement sector in Ukraine. The project is accompanying the technical cooperation

initiated between the EBRD and the Ministry of Economic Development and Trade of

Ukraine in 2011 and promoting Ukraine’s accession to Agreement on Government

Procurement (GPA) of the World Trade Organisation.

2. Following initial research regarding eProcurement projects operating in public sector

in Ukraine the review focused on Prozorro, the project launched by Transparency

International (Ukraine) in 2014. While still in the piloting stage, Prozorro has been

identified as a single eProcurement initiative in Ukraine that was (a) developed for

general, and not individual, use, (b) handling electronic procurement procedures in

practice and (c) at the time of the review practically employed in practice by more

than one contracting entity in the public sector in Ukraine.

3. The audit of Prozorro project was initiated on 30 March 2014, completed by 18 May

2015 and comprised a review missions to Ukraine, analysis of project and software

documentation, online assessment of the production version of the software and

interviews with key project stakeholders that include the project sponsor,

Transparency International (Ukraine), the project management team, including Project

Coordinator Mr Andrij Kuchrenko, project IT and development staff from

Quintagroup and Softserve and a selected key end-user, the Ministry of Defence of

Ukraine.

4. The objective of the review was to analyse the Prozorro pilot project launched in

February 2015 in terms of its organisational / institutional structure, functional

capacity for roll-out / development of the piloted IT system and its compliance with

international best practices for electronic procurement in public sector. The review



was divided between business, legal and technology sections and focused particularly

on compliance with standards for eProcurement promoted by the WTO GPA, the 2014

EU public procurement directives and standards for electronic tendering and electronic

reverse auctions promoted by multinational development banks. In particular, the

review referred to requirements for eProcurement included in the 2012 text of the

WTO GPA, the EU directive 2014/24/UE, the Golden Book of eProcurement Best

Practice published by the EU Commission in 2013 and the MDB ‘Handbook for e-

Government procurement’ last revised in 2014. To facilitate the speed of review a

methodology and checklist questionnaires were used; these were originally developed

for the EBRD to implement Task 1 and Task 2 of technical cooperation project

“Bulgaria: Policy Advice and Implementation Support for eProcurement Reforms”

funded by the Government of Bulgaria from EU Structural Fund.

5. This report presents the outcome of the completed audit questionnaires, as well as

initial findings from the audit of the Prozorro project, and is intended to provide input

to the discussion on the future of Ukrainian public procurement reform, eProcurement

reform in particular. The draft report review is scheduled to take place on 27 May

2015 at the session organised by the Ministry of Economic Development and Trade of

Ukraine and national and international reform stakeholders. Final report will be

published online by 15 June 2015 on the project website, available at

http://ukraine.ppl.ebrd.com.

6. The review findings and detailed recommendations are presented in section 9 and 10

of the report. In summary, the review exercise was established in the following

manner:

a. The Prozorro pilot project, dedicated to micro value purchasing (below the

threshold of the Ukrainian public procurement law application) is operative

and successful. An interview with the Ministry of Defence, the owner of the

majority of procurement procedures completed so far during the pilot, and the

Prozorro help desk record of assistance provided to contracting entities and

suppliers in pending procedures confirm that pilot of Prozorro system is

working effectively.

b. Pilot participants (contracting entities and suppliers) are in general agreement

that technical and functional features of the new eProcurement tool and the

business outcome of the purchasing processes completed via Prozorro is more

than satisfactory. However, while suitable for pilot purposes, the existing

http://ukraine.ppl.ebrd.com/



Prozorro system is to a very large extent based on pro bono work of IT staff.

Thus, it has limited institutional capacity and in order to be subject to roll-out

at the regional level (Phase 2 of the pilot project scheduled for autumn 2015) is

has to be strengthened in terms of technical and human resources. Firstly, a

new administration and maintenance scheme has to be funded and put in place

based on a Service Level Agreement with commercial platform operators. If

further private or donor funding is not available for this purpose (there is no

public funding available for project development), Prozorro should develop

Phase 2 of the pilot in the regions on self-funding terms and charge all end-

users with flat administrative fee. Secondly, additional regular security

procedures suitable for large amount of transactions should be employed.

Thirdly, to move from pilot stage dedicated to micro values to low value

procurement (below the GPA/ EU thresholds), full roll-out of the solution for

the long term strategy for the Prozorro concept development will have to be

formulated. With this in place, the public sector in Ukraine will have both a

short-term solution in place and a vision of the future shape of eProcurement

for low value contracts well established.

c. The Prozorro system built in an original institutional scheme of public-private

partnership under umbrella of an active anticorruption NGO, Transparency

International (Ukraine), is clearly a success. It demonstrated that progress in

public sector can be achieved in a challenging environment and in a relatively

short time, if government is ready to embrace business best practice and

innovative management structures, based on mutual trust between non-

governmental project sponsor and commercial stakeholders of the project. It

also demonstrated that private sector can dedicate resources when there is a

shared belief that project business case is a win-win situation for public sector

contracting entities, commercial platform operators who participate in the pilot

and suppliers active in the local market. This first step should not be

underestimated, because this is a step along the path towards making the

government more open and dedicated to one of the most treasured European

and democratic values, transparency in spending public funds. To build upon

this success and to assure the future of the Prozorro project the business

owners of the Prozorro system have to make a responsible decision as to

whether they are determined to develop Prozorro in the existing non-



governmental formula and make it commercially viable by self-funding or

whether they are prepared to transfer the project to the government to be

further developed based on public funding. A detailed development plan and

rational budget should be put together, and a funding formula should be agreed

to enable completion of Phase 2 of the pilot.

d. Presently, due to the lack of sufficient legal basis, electronic procurement

cannot be used in Ukraine for procurement covered by current public

procurement law. However, if electronic procurement is enabled by the

revision of the public procurement legislation, the procurement procedure

operated at the moment by Prozorro could be used only for so called ‘national

procurement’, below the GPA/EU thresholds, where international trade

requirements and standards do not apply. However, it needs to be mentioned

that improvements to hardware will be needed to support higher transaction

workloads, and changes to the software architecture are necessary to support

horizontal scaling.

e. In order to extend Prozorro operations to high value procurement (above the

GPA/EU thresholds) the Prozorro system would need to be substantially

developed and provide additional functionalities, including online contracts,

purchasing from catalogues and online payments. In addition, participating

commercial platform operators would need to implement uniform registration

and authentication procedures for local and international bidders, increase

transaction data security and develop and implement online workflows for

several tendering procedures suitable for different types of procurement. This

is a serious investment, and to make it commercially viable in the multi-

platform model it would need to be assured by (a) legal predictability and

stability, including a complaint mechanism appropriate for electronically

conducted procedures, (b) standardised business process for all contract types,

(c) creation of a viable certification institution to authorise commercial

operators to offer eProcurement services for high value procurement. In short,

Ukraine should follow the reform recipe of the Portuguese 2009–2012 reform

project. From this point of view, the Prozorro’s current status is just the

beginning of a long journey to match international best practice.



Scope and objective of the review

The review of existing electronic procurement solutions has been undertaken having in mind

that the ultimate goal of current Ukrainian government is to modernise public procurement

regulatory framework and establish electronic procurement procedures as a key means to

achieve successful implementation and create best practice for all public agencies and

institutions in Ukraine. Following initial research regarding eProcurement projects operating

in public sector in Ukraine the review focused on Prozorro, the project launched by

Transparency International (Ukraine) in 2014. Prozorro, while still in the piloting stage, has

been identified as a single eProcurement initiative in Ukraine that was (a) developed for

general, and not individual, use, (b) handling electronic procurement procedures in practice

and (c) at the time of the review practically employed by more than one contracting entity in

the public sector in Ukraine.

The objective of this review is to analyse existing electronic procurement solutions, the

Prozorro project in particular, and benchmark its business concept, technology solutions and

electronic procurement processes against international best practice, and in particular against

the European Union best practice for electronic procurement. To assess the project regulatory,

business and technological capacity several aspects of the Prozorro project had to be analysed.

First of all, an inventory of current organisational and implementation status has been made,

as implemented by the Prozorro project during the pilot initiated on February, 4, 2015.

Secondly, technical architecture and functional specifications of Prozorro (the central

application and cooperating commercial electronic procurement platforms as in the piloting

stage) has been benchmarked against international best practice. Thirdly, the procurement

procedures supported by Prozorro have been surveyed.

In particular, the following specific questions were asked:

- Is current technical / organisational set-up of the Prozorro project sufficient for piloting at

the central level and would it be possible to initiate the pilot at the regional level, in

municipalities and utilities sector?

- Does the existing IT solution have the capacity to successfully handle larger scale / greater

volume operations?

- What are the weak points / limitations of the existing system?

- What will be the cost of making the system fully compliant with international best

practice, and in particular the WTO GPA and the EU public procurement policies?



Audit methodology

To answer the questions stated above the following activities were undertaken: (i) review

missions to Ukraine, (ii) analysis of the project and software documentation, (iii) online

assessment of the production version of the software and (iv) interviews with key project

stakeholders: the project sponsor, Transparency International (Ukraine), the project

management team, including Project Coordinator Mr Andrij Kuchrenko, project IT and

development staff from Quintagroup and Softserve and a selected key end-user, the Ministry

of Defence of Ukraine. A full list of documents analysed is presented in the report appendices.

In particular, the following analysis was completed:

- examination of project documentation, including technical documentation of the project,

such as original strategic concept, blueprint and test documentation of the software and

contracting documentation of the project: Transparency International Ukraine contracts

with software developers, testing and help desk contractors, project administration

structure, a sample of contract with commercial eProcurement platform operators who

cooperate with Transparency International in the Prozorro project;

- review of publically available information about the Prozorro project, accessible via

http://prozorro.org and http://openprocurement.org;

- assessment of the functionalities supported by the software at the current pilot stage;

- survey of the software productive and testing instance;

- interview with key stakeholders of the Prozorro project listed in the table below:

Name and Surname Position / role in the Prozorro project:

Mr Andriy Kucherenko Prozorro Project Coordinator

Mr Myroslav Opyr Chief Technology Officer , Quintagroup

Mr Viktor Nestulia Senior analyst, Transparency International Ukraine

Mr Artur Pereverzev eProcurement Project Manager at MoD

Mr Yuriy Husiev Deputy Minister of Defence

The Prozorro project has been benchmarked against the following international best practices:

http://prozorro.org/

http://openprocurement.org/



- requirements for eProcurement included in the 2012 text of the WTO GPA,

- EU Directive 2014/24/UE,

- eProcurement Golden Book of Good Practice Final Report, published by the EU

Commission on 11 March 2013,

- the MDB ‘Handbook for e-Government procurement’, last revised in 2014.

Legal/regulatory review

Since the Prozorro project in the current pilot stage operates at a micro level, it covers the

procurement area that is not regulated by Ukrainian public procurement law. Thus, it was

impossible to assess compliance of the electronic procedures supported by the Prozorro

project in the pilot with the applicable Ukrainian public procurement legislation. For that

reason legal/regulatory review was limited to compliance of the piloted procedures with

general principles of public procurement as promoted by the WTO GPA, which include

transparency, competition, non-discrimination, accountability of public officials for

procurement decisions and ‘value for money’ concept.

Also, because the Prozorro project is organised and implemented outside Ukrainian

administration for public procurement, analysis of Ukrainian public institutions organisational

readiness to implement the electronic procurement procedures was also beyond the scope of

the review .

Since the Prozorro project is presently in a pilot phase, in many cases current ‘status quo’ of

the project organisation / software / procedures differs from its target condition, as declared in

the project strategy documents. In these cases the review was limited to analysing the

Prozorro pilot on an “as-is” basis; however the report will address whatever risks the review

has identified for the prospective development of the Prozorro project.

Standard checklists and questionnaires

Courtesy of Public Procurement Agency of Bulgaria, to facilitate the speed of audit exercise

the review utilised the methodology, the checklist and the questionnaires originally developed

for the EBRD to implement Task 1 and Task 2 of the technical cooperation project “Bulgaria:

Policy Advice and Implementation Support for eProcurement Reforms” completed under the

Project Implementation Support Service Agreement signed by the EBRD in 2013 and funded

by the Government of Bulgaria from the EU Structural Fund.



1. PROZORRO PROJECT CONCEPT

The Prozorro project was established in May 2014 on a pro bono basis and initiated by group

of anticorruption social activists interested in developing electronic procurement platform for

all Ukrainian public agencies with a goal to provide a reverse auction platform and reduce

corruption in public procurement processes in Ukraine.

Due to limited resources of the project and lack of relevant legislation the scope of Prozorro

project pilot is limited to micro procurement, i.e. low value purchases beneath public

procurement law threshold in Ukraine. With Prozorro project concept based on cooperation

with commercial platform operators, a small-scale central technical solution was built on a

proof-of-concept basis as a minimal viable technical solution and funded by Transparency

International Ukraine. Transparency International Ukraine is the Prozorro project sponsor and

a legal owner and administrator of the Prozorro dedicated software presently piloted in

Ukraine.

The table below presents the project milestones already achieved, as well as those planned in

the future.

Table 1 The Prozorro project milestones

Milestones Status Date

Strategic Concept based on cooperation with existing

commercial platform operators

Approved July 2014

Blueprint of the Prozorro system Approved October 2014

Prozorro dedicated software development Completed December 2015

Commercial Platforms Acceptance Testing of

Prozorro dedicated software

Completed January 2015

Prozorro Project live in pilot mode 4 February 2015

End of Phase 1 of the pilot (Kiev based central

contracting entities)

Planned 1 September 2015

End of Phase 2 of the pilot (Contracting entities in

regions and municipalities)

Planned 1 December 2015

Start of roll-out for low value public procurement Planned 1 January 2016

Start of roll-out for public procurement above the

GPA/EU threshold

Planned January 2017



1.1. Project organisation

The Prozorro Project is managed as follows:

1. Project Sponsor: Transparency International Ukraine. Transparency International

Ukraine (TI) is the legal owner and administrator of the central unit of Prozorro

project and a watchdog organisation that ensures public trust in the project based on

cooperation between Ukrainian anticorruption activists and commercial platform

operators. All legal issues, contracts and structure of the Prozorro project are

controlled by Transparency International. A dedicated project manager in charge of

the Prozorro project, Mr Victor Nestulya, is employed by the TI.

2. Business/Private Sector Partners: Committee of Platform Operators. The Committee is

formed by representatives of commercial eProcurement platforms operators who

subscribed to the general principles of the Prozorro project as proposed by

anticorruption activists and the Transparency International. The Committee

established a common standard for functional requirements and agreed to fund a

central single-sign-on unit of the Prozorro project enabling different existing

eProcurement platforms to communicate with each other, exchange procurement

transaction information and offer a streamlined eProcurement service to public sector

entities in Ukraine.

3. Public Sector Promotor: National Council of Reforms (NCR). NCR is a collective

body consisting of all top governmental officials (Prime Minister, Ministers, Leaders

of the political parties in the Parliament etc), which supervises the progress of reforms

in Ukraine and addresses cooperation issues between different governmental

structures. The President of Ukraine is a chairman of the NCR. The Prozorro project

pilot has been included in the list of NCR-promoted public initiatives, and a dedicated

project manager, Mrs Kristina Gutsalova, has been appointed in charge of the

ProZorro pilot project and is responsible for addressing administrative issues arising

during the pilot implementation.

4. Intended Project Owner: Ministry of Economic Development and Trade (MEDT).

Department of Public Procurement in the MEDT is the national public procurement

authority in Ukraine. Presently, MEDT participates in the pilot providing policy and

operational monitoring of micro procurement procedures included in the pilot,

however it is anticipated that upon completion of Phase 2 of the pilot the MEDT will



accept financial and managerial responsibility for full roll-out of the Prozorro project

in public sector in Ukraine. The Head of Department of Public Procurement in the

MEDT, Mr Oleksandr Starodubtsev, is responsible for cooperation with Prozorro.



Figure 1 Prozorro Project Organisation



1.2. The PROZORRO Project in Pilot

The pilot of the Prozorro project was commenced on 4 February 2015. The scope of project

deployment in the pilot is as follows:

- by 18 May 2015 a total of 64 public sector entities was registered to use the Prozorro

for their micro procurement;

- 32 purchasing processes have been completed on the platform by 18 May 2015;

- there are 324 purchasing procedures active in the system and are expected to be

completed within 30 days;

- the total volume of the processed transactions is 1,6 Million UAH;

- the volume of open transactions is 190 Million UAH.

Currently, three types of purchases are processed through Prozorro

- micro procurement that is not regulated by the Law on State Purchases (the Law);

typically these are purchases with an estimated value below 100 000 UAH;

- procurement of public sector entities which are not covered by the Law, based on

special exclusion or exemption;

- procurement using a simplified purchasing procedure, allowed by the Law to be used

by the Ministry of Defence.

All procurement tendered within the Prozorro project pilot is of micro value, which is not

only below the GPA / EU threshold but also below national thresholds or outside the

application of the Law. The procurement procedure supported by the pilot is reverse

electronic auction. The Prozorro project pilot does not support other procurement methods for

high value contracts, as implementation of above the threshold procurement procedures

requires legislative changes in Ukrainian law, as well as development / architectural changes

within the Prozorro project central unit and in the commercial platforms involved in the

Prozorro project.

The Prozorro project pilot is managed as follows:

- Coordination of development of the Prozorro project central unit and cooperation with

commercial platform operators who are connected to the Prozorro central unit is



performed by Prozorro Project Coordinator, Mr Andriy Kucherenko. Project

Coordinator takes care of the Prozorro project on a pro bono basis and is not employed

by Transparency International or other parties legally responsible for the project

(Platform Operators).

- Software development is done by Quinta Group on the basis of orders sent by

Transparency International. Orders are prepared by Andriy Kucherenko upon

consultation with Frontends and MEDT and updating the Blueprint.

- The testing of new Frontends before granting them access to productive instance of

Prozorro Central DB is being done by SoftServe, the biggest Ukrainian software

company. This is performed free of charge, and there is no contract between SoftServe

and the TI. The testing process is supervised by Andriy Kucherenko, the Prozorro

Project Coordinator .

- Maintenance, administration and development of the Prozorro central unit is provided

for Quinta Group, a Ukrainian IT company, on the basis of a contract (SLA) with TI.

Figure 2 Management of the Prozorro Project during Pilot



In brief, the volume of transactions processed within the Prozorro project is quite typical

for an early pilot project. Upon completion of Phase 1 of the pilot an end-user satisfaction

survey should be undertaken to ensure that all operational issues are identified and

addressed before entering the regional phase of the pilot.

So far the pilot project is very popular with the contracting entities, which is a strong

indicator of their commitment to reform and modernise public procurement. The outlooks

for the future success rate of the regional pilot phase, measured by the number of public

institutions accessing the programme, are therefore firm. The internal structure of the pilot

will however have to be solidified before embarking on the second stage of the pilot.

Firstly, while commercial platform operators will take care of traffic on their end of the

process, a dedicated team of software developers responsible for maintenance and

improvements in the Prozorro central unit will be necessary to handle large number of

public institutions participating in the pilot in the regions of Ukraine. High availability of

the central unit and short timeframes for problem resolution will not be attainable with

only pro bono IT staff.

1.3. Functionalities of the Prozorro Project system

Existing functionalities

Table below presents the current functionalities of the Prozorro roject system benchmarked

against a standardised set of electronic procurement platform functionalities as prescribed by

the MDB standards.

Table 2 Functional scope of the Prozorro system in the pilot

Scope Descriptio

n Status Comments

1.3.1 eRegistration economic

operator

registration

in the

system

compliant The Economic Operators (the EO) can register in

the system. This registration is performed through

the Front-end Platform of their choice (currently

there are three such platforms operating, several

others are scheduled to go operational). The

registration is free of charge to all EOs. However,

the Frontend Platforms are free to charge the EOs

for other added services they provide. The Prozorro

project does not register such services or the



charges associated with them formally. As those

added services are voluntary, the system is deemed

fully compliant with the EU guidelines in this

respect. However the emphasis should be put on

the fact that the SLA (Service Level Agreement)

between the Prozorro operator and Frontend

operator should legally bind the latter to provide

the eRegistration service free of charge to its

customers.

1.3.2 eAuthentification can the

economic

operators

log

electronica

lly into the

system

partially

compliant

Users can log into the system using their logins and

passwords. No electronic signature/certificate is

necessary to access the system. The system is not

compliant with European eSignature architecture

due to relevant Ukrainian legislation not being

compatible with that of the EU. The system is

compliant with the EU guidelines on this point.

There is however a security risk within the system

architecture associated with eAuthentication. End-

users access the eAuction module using a one-time

access link generated for them by the Prozorro

platform. Users accessing the eAuction module

benefit thereby of a single-sign-on process. They

are authenticated by means of username and

password by the Frontends and do not have to log

again to the Prozorro platforms. Those one-time

links are secure on its own but what makes them a

security risk is the way of delivering them to the

users. Since they are transmitted via Prozorro-

Frontend interface, as they can be easily spoofed

by Frontend administrators. They can also be a

subject to some security bug in Frontend software.

It is strongly recommended to change the way of

authenticating users in eAuction module. There are

several technical ways to strengthen this weak

point: delivering the one-time password through a

different channel, two-step authentication, signed-

cookies to verify end-users identity etc. One of

them should be implemented by the project.

1.3.3 eNotification Electronic

notificatio

partially

compliant

It is upon Frontend operators to provide

eNotification services. Not all of them provide such



n by email

to

economic

operators

regarding

new

procureme

nt notices,

changes to

the

procureme

nt notices

in which

they are

interested,

new

clarificatio

ns being

published

etc

services. They are also deemed by Frontend

operators as “added services” for which the

Economic Operators have to subscribe and which

may require additional charges. The SLA between

Prozorro and Frontend platform operators should

enforce providing eNotification services to its

users.

1.3.4 eNotices not

applicable

Integration with EU eNotices system via eSender,

working online and offline

- not compliant. Since Ukraine is not an EU

member it is not bound by the obligation to publish

any information within TED.

1.3.5 eAccess access to

tender

documents

and

specificati

ons

compliant The EO can access all tender documents and

specifications via the Frontend platform of their

choice. Due to integration of all Frontend

platforms with central database this access is not

restricted to the Frontend platform of the buyer.

The access is duly restricted in respect to who is

accessing the documents (only the authors can

change or delete them). The access is also properly

restricted in time (e.g. no changes to the

documents are allowed after the submission

deadline).

1.3.6 eSubmission/eTen

dering

downloadi

ng tender

compliant The EOs can download all tender documents and

upload their offers using regular internet browser,



documents

and

uploading

bids

without the need for special software. No specific

software is needed to open relevant documents

(apart from commercially available office

software). One of the Frontends requires its users

to use special software to access the documents but

the software can be downloaded from the website

which makes it compliant with the EU guidelines.

1.3.7 eEvaluation electronic

evaluation

of bid

compliant The Contracting Authorities can view the results of

the bidding / eAuction process within the system.

The current system is however subject to several

limitations which will make it non-compliant when

its use will be extended to above the threshold

purchases:

First, at present the only variable biddable within

the system is the price. This makes the system unfit

for purchase of e.g. services for which multi-

variable-bidding is required.

Second, the system does not support multi-lot

auctions yet. One eAuction has to contain one lot

for which the bidders offer their prices. eAuctions

with several lots, per lot bid evaluation and

awarding arenot supported by the current version

of the system.

Implementation of multi-variable-bidding and

multi-lot auctions may be difficult in current

technical architecture of the system. Currently,

during the eAuctioning phase, each bidder is

allowed the time slot to change his previous bid.

Those time slots are sequentially rotated among the

bidders. This means that no real-time bidding

engine in which all participants can change their

bids at any time according to their will is

implemented in the system. Straightforward

expansion of this sequential bidding model to

multi-variable and multi-lot auctions will be

impractical. Since the number of variables and

number of items multiply, this would mean that the

time slot duration for each bidder would have to be

increasedd. Placing a bid in multi-variable and

multi-lot eAuction would take a long time and



since such bid submissions are sequential and

exclusive this would result in excessive waiting

times for other bidders (who would have to wait in

queue to place their bid). This would make

eAuctions unusable for processes with several

bidders, line items and variables. Such eAuctions

would simply have to last many hours which is

hardly practical.

1.3.8 eAwarding notificatio

n of award

of contract

compliant After the eAuction event the Contracting Authority

qualifies the winner checking whether all formal /

technical / organisational and other requirements of

the tender are met. Information about awarding the

contract is publically available to all end-users via

the Frontends. The system is not compliant with

the EU guidelines in respect of prequalification of

bidders. No process is implemented that would

allow the Contracting Authorities to qualify the

bidders for the tender based on formal

requirements pre eAuction event. Also, the current

version of the system does not distinguish between

the formal and commercial parts of the offer, thus

implementation of this change will not be

straightforward. This is a gap in functionality

which will have to be addressed for purchases

above the threshold.

1.3.9 eCatalogues Common

use item

specificati

ons and

agreed

prices for

call-offs

under

framework

agreements

Not

applicable

No such functionality exists in the current version

of the system.

1.3.10 eStatistics Search

tools,

statistical

reports,

partially

compliant

The provision of statistical reports, interactive

dashboards and search tools to end-users is left to

the Frontend operators. The SLA between Prozorro

and frontend operators has to enforce such



interactive

dashboards

and audit

reports

functionalities for end-users. This is not the case

now and will have to be addressed in the future.

However, as of today all Frontend operators

provide the search tools for their end-users. The

Prozorro platform provides administration reports

and audit tools as a back-end system.

1.3.11 ePlanning Preparing /

publishing

procureme

nt plans

Not

applicable


of the system

1.3.12 eOrdering Based on

framework

agreements

Not

applicable


of the system

1.3.13 eInvoicing Not

applicable


of the system

1.3.14 ePayment Not

applicable


of the system

1.4. Future functionalities

It is envisaged that upon the completion of the pilot its results will be analysed by the NCR

and MEDT, and a new legislation on electronic public procurement will be prepared by the

MEDT and adopted by the Parliament. After adopting the new legislation the scope of the

Prozorro project will be extended to cover all procurement of goods and services below the

GPA/EU thresholds (below 133 000 EUR). Presently, there are no business or technical plans

for the Prozorro project development to cover procurement above the GPA/EU thresholds.

Existing technical architecture of the Prozorro Project System

The Prozorro Project system is composed of two main elements:

- Prozorro Central Unit, comprising Central Database, Reverse Auction Module and

Reporting Module, that is owned, developed and administered by Transparency

International

- Front-end Platforms, privately-owned and commercially operated eProcurement

platforms that deliver the end-user experience and services. Those front-end platforms



are interfaced with the Prozorro Central Unit by the use of API Communication

Module.

Figure 3 Major elements of the Platform



Functionalities provided by the Prozorro Project Central Unit:

- Central database of Contracting Authorities

- Central database of all purchasing processes conducted by all interfaced platforms

- Central database of all documents attached to the purchasing processes

- Central database of bids tendered to the bid invitations

- Central database of all documents attached to the bids

- Central Reverse eAuctioning module – allowing for dynamic tendering to the bid

invitations

- Central database of information about contracts signed as a result of eAuctions

Functionalities provided by the Front-ends:

- Contracting Authorities registration

- Publication of notices about future purchases by Contracting Authorities

- Bid invitation publication by Contracting Authorities

- Search for notices and bid invitations for Economic Operators

- Registration of Economic Operators

- View of the details of bid invitations and notices for end-users

- Downloading documents attached to the notices and bid invitations

- Notification about changes to the bid invitations

- Possibility to ask questions about bid invitations for Economic Operators

- Possibility to view and respond to the questions submitted by EO for Contracting

Authorities

- Possibility to view responses and clarifications submitted by Contracting Authorities

for EO

- Submission of bids for open bid invitations for Economic Operators

- Upload of documents required by bid invitations – for Economic Operators

- Access to eAuctioning module – the reverse auctioning module is placed on Prozorro

platform – but end-users access this module by the use of access link presented to

them by the frontend

- Publication of results of bid invitations and eAuctions

- Publication of information about contracts signed as a result of eProcurement



End-users do not access the Prozorro Central Database directly. They always use Frontend

platform of their choice to access the services they want (search for notices, submission of

bids etc.).

Frontend platforms are interfaced by the use of open source OpenProcurement API. The

interface transmits information about:

- Bid invitations – Frontends send those to the Central Database which forwards them to

all other frontends for publication thereby allowing the Economic Operators to use the

frontend of their choice regardless of the frontend the Contracting Authority is using.

Bid invitations are propagated to all frontends with associated documents

(attachments) and requirements.

Bid invitation changes – changes to bid invitations, changes to bid invitations dates,

documents, questions, clarifications and claims are all propagated from the frontend of

the end-user to the central database and further to other frontends by the API. This

way all changes to the bid invitation are stored in a central database and published on

all frontends.

- Bid submission – the Economic Operators can use the platform of their choice to

submit a bid and upload requisite documents. This bid and those documents are

transferred by the API to the Central Database and further pushed to the other

frontends.

- eAuction access link – Central Database pushes the unique access links for the e-

Auctions to all the platforms from which the bids originated. Those access links are

user-specific thereby allowing for easy user identification within the central eAuction

module. It’s upon the frontend to notify the bidder about the eAuction and provide him

with the access link. Central eAuctioning module (Prozorro) does not authenticate

bidders in any other way apart from assuming that since they are in possession of

unique access link they must be representing the party which submitted the bid.

- Awarding – the Frontend which is used by the Contracting Authority sends

information about the party that was awarded a contract as a result of eAuction

process to the central database. This piece of information is forwarded to other

frontends for publication.



Technology stack of Prozorro Project systems

The central database component has been tailor-made and does not use any commercial

software. The application follows current standards for the architecture of such systems. It is a

three-tiered application with separated persistence layer, business logic tier and presentation

layer. All technologies chosen are fully developed and are used to achieve objectives adequate

for their design.

The technological stack involves:

- CouchDB – open source non-relational database engine, non SQL one, available on

Apache Licence 2.0

- Python – high level interpreted programming language for implementation of business

logic.

The source code of business logic has been made an open source and is available at

https://github.com/openprocurement

- OpenProcurementAPI – open source web services based notation JSON for

implementing interfaces with frontend platforms

- HTML5 to create end-user presentation layer

- Document attachments (binary files, such as pdf, xls, etc.) are stored in the 3S-

compatible file server.

Figure 4 Technology stack

https://github.com/openprocurement



The current scale of system’s usage is adequate for the pilot stage of the implementation. It is

however insufficient to dissolve the doubts about the systems’ stability, performance and

security under high-volume loads. Performance tests have been run on the system and its

limits are not yet close. Further investments in software development and hardware are also

same necessary to make it fully scalable.

Processes within the system

The following business processes are carried out within the system:

- Preparation of procurement, including the following sub processes:

o registration of users (buyers and suppliers),

o definition of purchasing process by the buyer (description, requirements, dates)

o clarification phase, during which questions can be asked by the suppliers and

clarifications / answers can be submitted by the buyers

- Submission of bids by the suppliers: during this phase no changes to the bid invitation

are allowed and bids cannot be viewed

- Bids opening and preparation of post-bid ranking of suppliers

- Auction: dynamic bidding event for the bidders

- Evaluation: during this phase the winner’s offer is evaluated for meeting the bid

invitation legal / organisational requirements.

- Awarding: the bidder that has won the eAuction and has successfully undegone the

qualification procedure is awarded the contract, which is notified within the system

Processes not in scope of the pilot system:

- Appeal: the full system support for appeal process is at the top of the list of the future

system’s development priorities

- Administration Platforms (outside Pilot system): less crucial since end-users do not

access the Prozorro central unit itself.

- end-user Survey:planned for the future

- Data Analysis module (outside the Pilot system)

- Data archiving module (outside the Pilot System)



- Support for high-value purchases: currently not planned

Limitations of the current system

The Prozorro project systems contain several functional limitations. Those limitations are the

result of limited resources of the development team and are of different scale in respect of

future expansion of the system.

- One-variable bidding: price is the only biddable variable supported

- One-lot bidding: only tenders with one line-item are allowed within the system

- No uniqueness of bidders: bids are received by the central database with information

about bidders submitting them; no measures to assure uniqueness of bidders are taken

from the central database side, thus one supplier (if he is using different Frontends at

the same time) is able to submit several different bids within one tender, with the legal

consequences of such possible behaviou not addressed;

- Access link to the eAuctions: since the central database does not contain the database

of end-users it is not able to authenticate them correctly; instead, aone-time access link

is generated by the central database for each bid in each tender, and this link is

forwarded to the Frontend where the bid is originated by the means of API interface.

The obligation to authenticate users and provide them with access to respective

eAuctions is thus transferred to the Frontends. Legal consequences of this

implementation will have to be analysed. First, there is a chance that through a

security breach unauthorised persons may obtain the access links, which would allow

them to take part in eAuctions and act on behalf of legitimate bidders. Second, if these

access links are leaked there is a possibility to access the eAuction module avoiding

the Frontends, i.e. without any authorisation at all. Third, since no direct authorisation

of bidding parties is performed by the central database, obviousproblems will arise if

one of the parties taking part in the eAuction refuses to confirm the bids placed during

the Auction. Several technical measures can be taken to resolve this issue. Decision as

to which one of them is preferred is beyond the scope of this work.

- No real time auctions: the system does not utilise a real-time bidding engine. Instead,

the eAuction event is comprised of a fixed set of three rounds during which the

bidders place their bids sequentially. This process resolves several technical issues,

such asrequisite performance of the system, necessary network bandwidth,



synchronisation of the clocks for bidders etc. It however limits the chances to run

multi-variable and multi-item auctions within the system without its major overhaul.

This scheme also raises some questions as to its economic efficiency, since in future

buyers may insist that a regular real-time bidding engine is in place; with virtually

limitless number or bidding rounds this would result in higher savings than limiting

the number of bidding rounds to three. The current architecture of eAuction module is

deemed satisfactory for the purpose of transacting low-level purchases; at the same

time the system’s owners are encouraged to review this architecture before

implementing higher-value processes.



2. SYSTEM' S COMPLIANCE WITH EU DIRECTIVE GUIDELINES

Table 3 Checklist of Compliance with the 2014 EU Directive Standards on eProcurement

Standard Description of requirements Status

2.1 Interoperability The eProcurement system as a

whole, and any particular

platform must be interoperable

with ICT products in general use

and generally available; such as

hardware and software, web

browsers, email clients, etc

The Prozorro platform and the Frontend

platforms cooperating with it satisfy the

criterion of interoperability. No

proprietary software is needed to use

the system. The standard interface of

the system is a web browser.

Purchasing platforms provide a variety

of user interfaces (in different

languages, for different target

audiences) allowing to satisfy different

needs. Procuring entities are

recommended to use widely accepted

document formats (Office Suites, PDF,

images) for contract documentation.

The Central database API is

implemented with widely accepted

Internet standards: HTTP, JSON,

REST.

2.2 Traceability All actions in an eProcurement

procedure must be real-time

recorded, the actors are to be

identifiable, and the raw data is

to be preserved to ensure that it

can be demonstrated and verified

that the integrity of documents is

maintained, fair procedures are

followed and that infringements

or attempts to infringe are clearly

detectable.

The Current pilot version of the Central

Database has a record log of all actions

that procuring platforms perform on

behalf of their users (Contracting

Entities and Economic Operators). Once

a change to the tender is made the

system generates a time-stamped

change log. This change log is not yet

exposed, but can be audited if

necessary. It is planned to expose this

change log as Open Data. No document

uploaded to the system can be changed,



only a new version of it can be added.

Historical versions of documents are

preserved. It is planned to publish

central database operation logs via a

website. It is also planned to sign

documents uploaded into Central

Database on the level of purchasing

platform to ensure that documents were

not changed by the Central Database

operator. System does not limit the use

of Electronic signatures, and high

profile tenders can have their

documents signed by procuring entity

and suppliers. eSignature requirement is

not enforced at this stage.

Although the system seems compliant

with the requirement one crucial

problem remains. The system traces all

actions performed in it but one must

remember that it does not contain a

database of all end-users. All actions are

performed in the system by the

Frontend platforms on behalf of their

end-users. There is an implicit

understanding that:

- the action performed by

Frontend was performed by an

authorised user

- the data posted by the user was

not manipulated by the

Frontend

This understanding is not currently

supported by any technical or

organisational means. To satisfy

system’s conformity with the European

traceability standard a change is needed:



either a major development to enable

real end-user action tracing or an SLA

agreement between Prozorro and

Frontend must enforce the latter to

create such audit logs.

2.3 Security All European public IT systems

have to be secure. It means they

should have features in place that

ensure confidentiality and enable

virus scanning and encryption.

The network traffic between Central

Database and purchasing platforms is

encrypted with TLS. The traffic

between Auction module and the

suppliers is also encrypted with TLS.

The proposals of the Economic

Operators are secured with Central

Database access controls and invisible

to anybody except their owner. They are

not yet encrypted in Central Database.

Virus scanning is a responsibility of the

Frontend; another level of scanning can

be added at the Central Database level

in future.

The system satisfies the European

criteria of IT security, but again a

problem arises with its proxy

architecture. Not only the Prozorro

platform has to be secure, the security

measures have to be ensured by all the

Frontends. This has to be enforced by

the new SLA between them and

Prozorro. It is worth emphasising that a



security breach against either of the

Frontends will make the whole

landscape of systems vulnerable to

attacks.

2.4 Proportionality This well-established principle

of European law that seeks to put

a limit on administrative action

by national authorities equally

applies in the context of

establishing an eProcurement

system. For example, platforms

must be secure but not

impossible to audit, advanced e-

signatures may be used but not

universally, specialised tools and

devices can be preferred but to

restrict access and competition,

etc.

The transparency of the procurement

process provides a lot of credibility

reducing the need of electronic

signatures. In addition, any document

provided via electronic means can be

requested in paper form during

qualification process if concerns arise.

Contract can be signed in paper form or

electronically, and the system does not

limit the parties in this regard.

The system is deemed fully compliant

with the criterion of proportionality. It

does not create any undue restrictions or

does not enforce the parties using it to

any unnecessary administrative work.

The system does not require the use of

eSignatures which is proportional to the

scope of its usage.

2.5 Transparency This is also a well-established

and fundamental principle of EU

procurement and one of even

great importance as it underpins

the principle of equal treatment.

In an eProcurement context,

transparency involves having

information that is easy to read

and a process that is easy to use.

For instance, the Directive

requires that information on

In all the foregoing respects the system

is considered to be fully compliant with

European standard of transparency. Not

only the system is transparent but it also

encourages and promotes this universal

best practises among its users – both

Economic Operators and Contracting

Authorities. It is believed that this

promotional effect of the system

shouldn’t be undervalued. Even if the

system is not fully compliant with many



specifications for the electronic

submission of tenders and

requests to participate, including

encryption and time-stamping,

must be available to interested

parties, and not just those that

have registered (unless this can

be done anonymously).

Regarding functionality, the

Golden Book of eProcurement

gives some useful pointers on

how to make the platform easy to

use and therefore transparent.

Economic operators should be

able to search contract notices

using a set of search criteria be

able to evaluate whether tender

specifications are relevant for

them based on information

available in contract notices.

Changes to tender specifications

should be notified to those

expressing an interest. Economic

operators should be able to

create tenders using a core set of

structured data and unstructured

documents. Tenderers must

receive a proof of delivery upon

successful submission of their

tender, yet be able to resubmit

their tenders up until the

submission deadline.

European standards, it is so in many

cases because the authors of the system

decided to sacrifice these to the idea of

transparency. For example, the only

purchasing procedure supported by the

system is eAuction, because this is the

most transparent of procedures. The

only format of eAuctions available is

“the price is the only criterion”, for the

same reason, and this is the same for

other matters. It is believed that this

policy of forcing the most transparent

procedures into the use by Ukrainian

public institutions is the first step

towards transforming Ukrainian

governmental purchases into European

compatible processes.



3. REVIEW OF COMPLIANCE WITH STANDARDS RECOMMENDED BY THE

EU EPROCUREMENT GOLDEN BOOK OF BEST PRACTICE

The table below presents the analysis of the Prozorro project systems compliance with

European best practice as recommended by the EU eProcurement Golden Book of Best

Practice published in 2013.

Table 4 Compliance with the standards of EU eProcurement Golden Book Practices

Practice Status Comments

3.1 Platforms

automatically

transmit all their

notices to a single

point of access for

publication

compliant All Frontends send notices created through them to the

central database which forwards them to all other

Frontends where they are made public. This creates a

situation where several points of access are present,

with all of them containing information on all public

purchases. The Economic Operators can choose

whichever Frontend they deem most appropriate for

their needs gaining their single point of access to all

public notices.

3.2 Economic

operators and

contracting

authorities benefit

from affordable

training plans

compliant Professional call centre company has been employed to

answer EOs and CAs questions, electronic tutorials

and films are available for them on Prozorro’s and

Frontends’ websites

3.3 Platforms have

communication

plans in place to

promote the use

of eProcurement

compliant This is actually a forte of current system. Since

Frontends are commercial private enterprises earning

their money by selling added services to the EOs and

CAs using eProcurement platform, the promotion of

eProcurement usage is a crucial part of their business

plans. Since Frontends used by public procurements

are the same Frontends that are used by commercial

procurement there is a clear convergence of interests

between private and public sector.



3.4 Economic

operators can

access and

retrieve contract

notices and tender

specifications as

anonymous users

compliant Yes, through the Frontend of their choice

3.5 Economic

operators can

register on the

platform without

having to provide

country-specific

information

notcompliant The EOs have to indicate the country of their origin

during registration; this change is a simple technical

adjustment

3.6 Economic

operators

complete their

registration on a

platform by

clicking on an

activation link

sent by email

compliant After completing registration form an e-mail is sent to

the EO containing activation link

3.7 Platforms support

English in

addition to the

official

language(s) of the

member state(s)

where they

operate

partially

compliant

Currently only one of the Frontends supports English,

while the central eAuction module supports English.

Compliance with this practise should be achieved by

creating English versions of websites on Frontends by

revising the Service Level Agreements (SLAs)

between Prozorro and the Frontends.

3.8 Economic

operators can use

a username and a

password to log in

to a platform

compliant No eSignatures are needed to log into the Frontends,

all of them require user and password authentication

3.9 Economic compliant They can search for contract notices based on



operators can

search contract

notices using a set

of search criteria

description and CPV

3.10 Economic

operators can

evaluate whether

tender

specifications are

relevant for them

based on

information

available in

contract notices

compliant They can read the description of the contract notices

and download all documentation. It is ultimately up to

the CA to clearly specify who will be eligible for the

contract.

3.11 Economic

operators are

notified of any

changes to tender

specifications

not

compliant

Not all Frontends send such notices to their users,

sometimes they deem such notifications as added

service for which the users have to subscribe and pay.

This functionality should be implemented in Frontends

by means of the revised SLA or developed directly

within the central database.

3.12 Platforms support

automatic

transmission of all

types of notices to

TED

not

applicable

Ukraine is not currently an EU member state, and is

under no such obligation. Currently the system only

covers below the threshold purchases.

3.13 Economic

operators and

contracting

authorities can

search CPV

categories based

on their code or

their description

compliant A search for CPV exists

3.14 Contracting

authorities can re-

use information

compliant The feature is provided by Frontends; it should

however be implemented by the Frontends by means

of SLA.



contained in their

profile or in

previous notices

to create contract

notices, tender

specifications and

award notices

3.15 Economic

operators can

choose to

manually or

electronically sign

a submission

report containing

the hash value of

each submitted

document

not

compliant

Development plans exist containing such feature

3.16 Economic

operators receive

a proof of

delivery upon

successful

submission of

their tender

compliant A pop-up window appears to confirm the successful

submission or rejection of a bid

3.17 Economic

operators can

resubmit their

tenders up until

the submission

deadline

compliant This is a standard feature of the system

3.18 Platforms keep

tenders encrypted

until the opening

session

not

compliant

Implementation of this feature would require a

coordinated effort by both the central database and the

Frontends

3.19 Contracting

authorities can

partially

compliant

Currently price is the only criterion within the system.

Evaluation according to this parameter is performed



evaluate part of

their tenders

automatically

based on

predefined criteria

automatically. No functionality exists that allows to

perform multi-criteria purchases and evaluation. It is

planned to include quality as the second criterion

within the system.

3.20 Platforms use

European e-

Signature

validation

services to

validate e-

Signatures during

e-Submission

not

compliant /

not

applicable

eSignatures are not used within the system. Even if

they were, Ukrainian legislation in that respect is not

compliant with European framework. The certificates

format is incompatible.

3.21 Platforms clearly

indicate all costs

related to use of

the platform

compliant Access and use of the system is free for buyers. Access

to the system, searching, viewing and downloading of

notices and documentation is free for the suppliers.

Submission of the bid is payable: free for purchases of

planned value below 35 000 UAH and 175 UAH for

placing a bid in purchases of planned value over 35

000 UAH.

3.22 Economic

operators can

create tenders

using a core set of

structured data

and unstructured

documents

compliant The EOs create tenders by filling the forms and

uploading documents.

3.23 Economic

operators have the

freedom to choose

the platform of

their preference

without being

locked in by the

choice of the

compliant The EOs can choose whichever platform regardless of

the platform used by the CA; the data between those

platforms will be transferred through the central

database



contracting

authority

3.24 Platforms use

standard

specifications to

structure their

data and to

promote

interoperability

compliant The open source OpenProcurementAPI has been

developed and is implemented by all platforms and the

central database. It can also be adopted freely by any

new platform created.



4. ASESSMENT OF SOFTWARE DEVELOPMENT MATURITY

This step has been performed according to step 3 of the methodology developed by the MDB

Working Group on e-Government Procurement and was originally published in the MDB

eProcurement Toolkit in 2004.

This step focuses on key technical questions which have to be answered satisfactorily for the

eProcurement implementation to succeed. The main goal to achieve at this point is to assess

the maturity of processes associated with software development, administration and

maintenance.

4.1 Business requirements

Table 5 Business requirements

Description of requirements Status

4.1.1 Is it clear who the business owner of the system is? Yes, these are a number of

stakeholders listed in the section

on organisational structure of the

project.

4.1.2 Is it clear who dictates business priorities for the system

development?

Yes, currently it is Council of

Platforms (with mandatory

approval of major changes by

MEDT). After transfer of the

ownership to MEDT, this

Ministry will define the priorities

itself.

4.2 Technical requirements

Table 6 Technical requirements


4.2.1 Are the plans for future functionalities available and

formalised?

Yes



4.2.2 Is future architecture of the system planned and

documented?

Yes

4.3 Data storage needs

Table 7 Data storage needs


4.3.1 Have future data storage needs been evaluated? Will

they require architectural changes?

Yes, future data storage needs

have been evaluated and changes

to the architecture are included in

the functionality development

plan

4.3.2 Does the system have an established and documented

back-up strategy?

Yes

4.4 Interface/communication requirements in terms of new interoperability standards

Table 8 Interface/ communication requirements


4.4.1 Is it known what interfaces will be needed infuture? Yes

4.4.2 Is it known what interfaces will have to be maintained

in future?

Yes

4.4.3 Will these interfaces require any architectural changes? Yes, this is included in the

functionality development plan

4.5 Transaction workloads

Table 9 Transaction requirements


4.5.1. Have future transaction workloads been evaluated? Yes



4.5.2 What are they (in terms of concurrent users / transactions

/ number of documents)?

Ukrainian procurement system is

executing 2mln tender

procedures yearly. Current

workload is 0.2% of the total

projected load on the system.

4.5.3 Will meeting these transaction workloads require any

architectural changes?

System can scale with faster

hardware up to certain limits (up

to 10-20% of total projected

load). Before the limits are

reached the system architecture

should be altered to allow

horizontal scaling. This change

in architecture was planned but

not implemented in the pilot to

reduce implementation time and

cost.

4.6 Network capacity

Table 10 Network capacity requirements


4.6.1 Have the future network requirements been calculated? Yes

4.6.2 Will they involve any changes to the system’s

architecture?

Since Amazon S3 serves as the

document storage backend, its

scalable architecture provides

necessary means for meeting the

network bandwidth requirements.

The API network limits are

expected to scale horizontally with

the planned architectural change

outlined in 4.5 above.



4.7 Communications speed, reliability, security

Table 11 Software development requirement

Description of

requirements

Status

4.7.1 Have the physical security

measures been undertaken

with respect to the system?

Will they change in the

future?

Currently the system is hosted via Amazon Web Services

which makes compliance in this point impossible: the

system is hosted in the data centres of Amazon, and

physical security is ensured by this company. We assume

this architecture is sufficient for the current state of the

project and proportional to the value and importance of

processes registered through it. This however will change

after the pilot phase of the project. Since the volume and

confidentiality of data stored within the system will rise

significantly it will be of crucial importance to host the

system according to the best security practices i.e. having

professional physical security measures in place and SLA

agreement signed with hardware platform provider.

4.7.2 Is the network security

architecture in place

(firewalls / ddos protection /

load balancing / antivirus)?

Will it change in future?

Multiple firewalls are engaged. The Amazon AWS provides

Firewall services and Linux OS level firewall are engaged.

DDoS protection is limited to the services provided by

Amazon AWS. Traffic filtering services can be introduced

with system growth and the rise of DDoS attacks risks.

Load Balancing is expected to be introduced in planned

architectural change outlined in 4.5 above.

4.7.3 Has the system gone through

a formal security audit? Are

such audits regular / part of

the security procedures?

Not yet, planned before passing ownership to MEDT.

4.7.4 Does the system have

security incidents procedures

attached? Is the log of such

incidents available?

No security incidents have been registered before 18 May

2015. The formal procedures and logs will have to be

created and implemented before the end of pilot phase.

4.7.5 Does the system have the Yes



disaster recovery plan?

4.7.6 Have any formal software

audits been performed on the

system? Are they part of the

security procedure/regular?

Not yet, planned before passing ownership to MEDT

4.8 Software development methodology

Table 12 Software development requirements

Description of

requirements

Status

4.8.1 Is a formal software

development methodology

used? What is it? Is it

formalised?

Yes, all planned changes to the systems are initiated as

discussion in a dedicated forum (Google group), where all

platforms and key stakeholders are represented. If a change

is confirmed by majority of the participants following a

discussion, the update is added to the Release Plan, and

responsible person updates the Blueprint.

After this, development may be started. Upon completion of

the development, the party responsible for testing updates

automatic testing scripts and executes testing. Upon

completion, all platforms are invited to test the update in the

Sandbox. During this testing they confirm readiness of the

update and compatibility of their platforms.

After completion of the UAT, update is transferred to the

productive instance.

There is an official Release Management Standard, approved

by all stakeholders.



4.9 Testing methodology

Table 13 Testing requirements

Description of

requirements

Status

4.9.1 Is a formal testing

methodology used? What is

it? Is it formalised?

There are two levels of testing:

Automatic, based on Selenium technology. Used for testing

of Central DB only.

Manual, performed by party responsible for tests, based on

testing scripts, prepared by coordinators of the project and

approved by Platforms

Continuous integration testing is planned to be implemented

as a first priority change.

4.10 Application management strategy

Table 14 Application management requirements


4.10.1 Does the system have a well-documented

strategy to build/implement new fixes and

releases?

Regular fixes / small updates are regulated by

the official Release Management Standard.

Major updates should be formalised as sub-

projects which are regulated separately.

4.10.2 Does the system have a system landscape

containing separate productive, testing and

development environments?

Yes, the system comprises three physical

environments:

1. Development instance

2. Sandbox (testing environment)

3. Productive

4.10.3 Is a procedure available describing how /

when the changes are to be transferred

between the environments?

Yes, it is regulated by the official Release

Management Standard

4.10.4 Are the parties responsible for system

development/administration/management

Yes, as described in section on organisational

structure of the project



clearly specified?

4.11 Service management/user support

Table 15 Service requirements


4.11.1 Are the parties responsible for user

support clearly specified?

Yes

4.11.2 Is there an SLA in place regarding

the user-support level?

High-level

4.11.3 Are the errors in the system

quickly and efficiently removed?

Is it supported by an SLA?

Removal of errors is not a simple task in current

landscape since it requires a coordinated effort from

many parties, namely Prozorro developers and

Frontend developers. Since no real SLA exist between

Prozorro and Frontends, no real error removal

standards can be promised to the end users. The

analysis of errors registered by

https://github.com/openprocurement/frontend/issues

shows that the process of tracking issue resolution is by

no means tight. The sheer number of open issues (157

as of 19 May) displays the procedural problem here.

Some of the issues registered are not closed for a long

time although on technical level they have been

resolved.

https://github.com/openprocurement/frontend/issues



4.12 Training strategy

Table 16 Training requirements


4.12.1 Are the parties responsible for user training / training

materials preparation clearly specified?

Yes

4.12.2 What technologies/techniques are used for end-user

training?

Mostly webinars. Room

trainings (as requested by

biggest clients – government

organisations) are used rarely.

4.12.3 Were there any training quality assessments performed

(After training questionnaires etc.)?

No

To summarise the software development and maintenance maturity:

- System’s software development methodology is quite mature and adequate for the

current task.

- System’s hardware and software architecture will need to be upgraded before large-

scale implementation.

- The system will need to go under security audit/certification before the transfer to the

governmental institutions and full-scale launch.

- The system’s maintenance strategy needs a major overhaul; a precise SLA between

Frontend’s and Prozorro and between the system’s owner and administrator/error

removing party is necessary.



5. COMPLIANCE WITH OTHER MDB REQUIREMENTS:

Table 17 Compliance with other MDB Requirements


5.1 Is training available for buyers and suppliers? Yes

5.2 Is information on all procurement

opportunities advertised on a single internet

site?

The same information is advertised on

all platforms, which are connected to the

Central DB

5.3 No special hardware or software is required by

suppliers to use the system other than a web

browser and access to the internet.

Yes

5.4 Buyers and suppliers can register for business

online

Yes

5.5 Buyers and suppliers registries are linked to

the system (Is the system linked with national

register of economic operators?)

No, this is one of the gaps of the current

system. It does not ensure the uniqueness

of the EOs.

5.6 The system has a search engine to assist users

to find information

Yes

5.7 Procurement legislation, policies and

guidelines and information on how to use the

system can be accessed online

Yes

5.8 There is open access to all bidding and other

process documents

Yes

5.9 Access to the system for registered buyers and

suppliers is free or at a very affordable cost

Yes, Access and use of the system is free

for buyers.

Access to the system, searching, viewing

and downloading notices and

documentation is free for the suppliers.

Submission of the bid is payable: free for

purchases of planned value below 35 000

UAH and 175 UAH for placing a bid in

purchases of planned value over 35 000

UAH.



5.10 Electronic download of bidding documents is

available

Yes

5.11 Electronic upload of supplier proposal

documents is available

Yes

5.12 The system provides for security and privacy

of information. (Is system ISO 27001

compliant / has it gone through security audit?)

Central Database is storing no private

information. All information it stores is

classified as Open Data. To ensure

competitive auction some information is

initially access protected but is exposed

as Open Data as soon as the auction

ends. System has not gone through

security audit yet. System is being

prepared to undergo thorough security

audit recognised in Ukraine (developing

KSZI). ISO 27001 audit is being

considered by system’s developers.

5.13 Information on contract awards can be

accessed by the public free

Yes

5.14 Common interoperability and procurement

standards are applied to all systems (Can

buyers easily switch between different

purchasing platforms?)

Buyers can have one of their tenders

announced via one purchasing platform

and the other one via a different

platform. At the moment it is impossible

to start the purchasing procedure via one

platform and finish via another, but such

feature has been considered and can be

implemented. The interoperation

standard (Central Database API) that all

purchasing platforms are using is

documented. The data structures are

based on Open Contracting Data

Standard 1.0.



6. PROZORRO END-USER REVIEW

Appendix No 4 represents the results of the review for the Ministry of Defence, one of the

Prozorro System end-users.

Appendix No 5 represents the results of the review of two selected purchasing procedures

with the key stakeholder in the project Andriy Kucherenko.

7. SWOT ANALYSIS OF PROZORRO

This section presents the strengths, weaknesses, opportunities and risks of the Prozorro

Project identified by the review in technology and business areas.

Strengths

Business concept strengths:

The business model of Prozorro Project reduces government role in organising

access to electronic tools for public procurement and due to NGO’s start-up

funding eliminates initial pressure on the national budget.

Due to cooperation with several commercial platforms, Prozorro Project is the

biggest e-commerce platform in Ukraine in terms of number of registered

suppliers. A common pool of suppliers registered on all platforms connected to

Prozorro’s Central Unit makes the platform the largest one in Ukraine in terms number

of suppliers, which helps increase competition in tenders.

Any project-related risks may be shared among the Prozorro Project

participants. The system service architecture adopted by Prozorro is based on

outsourcing most transactional risks to external parties – commercial platform

operators, who are better prepared to handle business risk associated with handling

electronic transactions.

The Prozorro Project benefits from marketing efforts of commercial platforms to

promote electronic procurement among suppliers and contracting entities. The

utilisation of commercial eProcurement platforms under the umbrella of the Prozorro

Project creates a powerful convergence effect enabling a joint market effort by both

suppliers and contracting entities.

There are no platform switch costs for suppliers or contracting entities. Due to the

Prozorro Project’s Central Unit acting as an integrator among the commercial

eProcurement platforms, both contracting entities and suppliers can change the

eProcurement platform at any time/for each individual procedure at virtually no cost.



This encourages competition among commercial eProcurement platform operators

participating the Prozorro Project and should result in a higher quality of their

services.

Society trust in the operation of the Prozorro Project is increased due to

involvement of Transparency International and high level of transparency of

information on electronic bidding imposed by Prozorro.

Prozorro Project cooperates with commercial platforms operating both for

public and commercial purchases. By using these commercial platforms the

Government of Ukraine displays its dedication to adopt best business practice in

public procurement, which encourages trust in the operation of the system.

Technology strengths:

Open-source architecture decreases development costs of the Prozorro Project.

Since no proprietary technology is used, potentially every single software

development company can deliver development services. This increases

competitiveness and decreases the costs of the development of new functionalities of

the Central Unit.

Open-source architecture decreases maintenance costs. Using no proprietary

technology allows all competitors to offer support and maintenance services for the

Prozorro Project, increasing competitiveness. The switching cost associated with

change of the maintenance services supplier are also minimised.

Due to its multi-platform architecture, Prozorro Project allows sharing the

development costs among the participants. The investment costs necessary for

developing and maintaining the systems are divided between the operator of the

Central Unit and the commercial eProcurement platforms operators.

Technological simplicity of the current solution limits the probability of technical

glitches. The Central Unit is using a simplified reverse auction module. This module

operates in the technically simplest scenario possible: with one bid at time for lowest

price selection, with no multi-item auctions and no real time bidding engine. This is

suitable for purchasing of standardised goods and from technology point of view

simplifies the programming code, the testing of the system, makes the software more

error-resistant and should lower maintenance costs of the entire Central Unit.

Interoperability of the Central Unit decreases future integration costs. In future

the Central Unit will have to be integrated with Ukrainian eGovernment services and

other commercial eProcurement platforms, interested in cooperation with Prozorro and

offering more complex procedures and services. Development of such integrations

will be easier due to existence of a single point of integration. This should help lower

the cost of such developments.



Existence of an interconnected Central Unit allows for an easy data access.

Prozorro Project architecture involves a Central Unit which gathers all transactional

data from eCommerce platforms, which should facilitate satisfying future reporting

needs.

Weaknesses

Business concept weaknesses:

The Project’s existing business plan covers only micro purchases. There is no

business model developed for low and high value procurement in order to enable

calculating the long-term economics of the Prozorro Project including both the Central

Unit and the commercial eProcurement platforms.

No self-funding for public services is allowed by Ukrainian legislation. The

economic basis for eProcurement platforms and Central Unit functioning has not been

established by the authorities. Prozorro operates without state budget funding or a

framework agreement allowing charging fees from buyers and suppliers.

Start-up funding is based on NGO support and is insufficient. The NGO funding

has been used up for development of the first version of the software and initial stage

of the pilot. There is no funding to continue pilot in the regions which may delay

implementation.

Access to human resources is insufficient. Apart from the project coordinator from

Transparency International, all project staff are volunteers. This results in a limited

development and roll-out potential of the entire project.

Technology weaknesses:

System development requires complex management. Software development for

Prozorro requires synchronisation of activities performed by the Central Unit

developers and the eProcurement platforms’ developers. This extra effort requires

additional time and monetary resources.

Software error removal process requires complex management. This process

requires cooperation between parties responsible for maintenance of the Central Unit

and eProcurement platforms, which creates additional organisational burden.

A system’s multi-platform architecture is more exposed to security breaches than

a single platform solution. Security problems affecting any of the eProcurement

platforms may impact on the entire Prozorro solution. Therefore additional efforts to

assure security of the project are needed.



Users’ authorisation is based on mutual trust between eProcurement platforms

and the Central Unit. The Central Unit’s e-auctioning module does not authenticate

the users; it trusts that they were correctly authenticated and authorised by the

eProcurement platforms. Any errors in authentication and authorisation policy

implementation made by eProcurement platforms directly affect the Central Unit.

Prozorro currently supports only one procurement method, namely simplified

reverse auction. Such auctions have limited business application in terms of their

utilisation for sourcing non-standardised goods, works and services.

The system has not undergone security audit yet.

Opportunities

Business concept opportunities:

There is a potential for savings. Since purchasing prices were very high in Ukraine

due to low level of transparency, competitiveness and possible corruption, contracting

entities using Prozorro enjoy a very high potential of savings during their first

transactions, which should encourage more contracting entities to use the Prozorro

Project electronic procedures for their purchasing.

Savings were demonstrated already in the initial pilot phase. The high volume of

savings recorded in the initial stage of the pilot may create a favourable climate for

funding future development from state budget or obtaining a consent to self-funding -

charging fees from the suppliers and contracting entities.

Prozorro Project creates business opportunities for suppliers who previously did

not participated in public procurement. The user-friendly system, transparency of

the purchasing process and low cost of participation may convince more suppliers to

take part in electronic public procurement procedures enhancing the competitiveness

of the system.

The Prozorro Project is perceived transparent. Utilisation of commercial platforms

to conduct public procurement processes promotes transparency. This can lead to the

growth of trust in the system, which can translate into a more widespread usage of the

electronic procedures for public procurement.

Central government demonstrated commitment to implementing electronic

procurement in public sector. High level of commitment from the top of the

Government creates an opportunity to synchronise the project introduction with

legislative amendments timeline.

Exchange of best practices takes place in local market conditions. Cooperation

between the public and private parties involved in Prozorro Project allows sharing best

business practices from both sectors, helping increase the efficiency of public

procurement processes in Ukraine.



Help desk and maintenance expenses of Prozorro Project operations are shared.

In the multi-platform solution all participants are interested in the success of the

project, which can be directed towards developing a quality control mechanism.

Technology opportunities:

Open-source technology is highly flexible. The open-source technology used by the

project allows for easy adoption of new business requirements. This may allow

reducing the software delivery times.

Mutual testing is available. The architecture comprising several interconnected

platforms means that several parties are involved in software testing process. This can

lead to a high quality error-free programming code.

Flexible incorporation of other open-source applications is possible. The open-

source architecture allows incorporating other open-source software into the system at

reduced cost. This may be used to reduce the Prozorro Project development costs.

Exchange of technology know-how is enhanced. Within the multi-platform

integrated architecture the technological know-how is exchanged, which can have

mutually beneficial effects leading to continuous software quality improvement.

Risks

Business concept risks:

Current management and organisational structure is insufficient. Prozorro project

is staffed by volunteers whose participation in the initiative may be terminated without

notice, which endangers the business continuity of the project.

Participants are highly interdependent. Underperformance by one participant in

terms of service quality can potentially undermine end-users’ trust in the whole

solution.

Start-up funding is insufficient. The project may be terminated before reaching the

break-even point due to lack of funding, however if the central government allows

project self-funding, the project may become self-sufficient.

Contractual risk is present. Contract terms between Transparency International and

eProcurement platforms operators are very general. The specifications of

eProcurement platforms’ obligations and responsibilities are not fixed and no penalties

for defaults are specified. This exposes Transparency International to contractual risk

in case claims brought by contracting entities or suppliers.



Possible delays with adopting the new Law may postpone extension to higher-

value purchases. The Prozorro Project cannot develop and offer new procurement

procedures for low and high value procurement without regulatory changes in place.

Legal environment is unstable. Future legal changes may render the project non-

compliant with regulatory framework. Prozorro’s operation may also become

uneconomical due to regulatory changes restricting or banning collection of payments

from public procurement procedures participants.

Resistance to the system is possible. High levels of corruption in the current public

procurement system may lead to a strong resistance among both buyers and suppliers

against implementing the new system. This resistance may involve different measures

impeding the system’s roll-out.

Technology risks:

Exposure to technical errors is increased and requires strict quality management.

In multi-platform architecture a technical error in one of the processes (eProcurement

platforms or the Central Unit) may have a potential of destabilising the entire system.

The project participants are interdependent (e.g., during software error

elimination process). A delay in bugs fixing caused by one eProcurement platform

operator can potentially render the deployment of the software impossible for all

participants.

Growth may cause challenges. The increase in transaction volume may expose its

technical bottlenecks which may require major architectural changes to the system.

Security breaches may be difficult to handle. Since the information used in the

project is distributed and synchronised among different IT systems the origins of a

potential security breach may be difficult to locate.

Software reverse engineering may create additional obstacles. Open-source

architecture is exposed to software reverse engineering that may encourage potential

hacker attacks.

8. RECOMMENDATIONS

The recommendations developed after performing this review have to be divided between two

major groups: suggestions regarding the current phase of the project and proposals for the

direction of future system development. The former relate to the stage when only below-the-

threshold purchases are transacted through the system; the latter refers to the next projected



phase of the system development when it should support all available purchasing procedures,

both for low value and high value procedures.

Current phase recommendations

The current phase recommendations can be combined into two major areas: organisation and

technology, neither of them having precedence over the other. They are both vital, and their

ultimate goal is virtually the same, that is, to ensure the system’s security.

Organisational recommendations:

New framework agreements with Frontends. This point is absolutely necessary. Since

almost all end-user functionality is provided by the Frontends these framework

agreements need to ensure the system’s conformity with the EU standards. These

agreements have to require thatthe Frontend platforms provide such requisite services

as:

- eNotification,

- creation of audit logs,

- English version of their websites

- other functionalities required by the EU guidelines

Those agreements will have to include penalties for the Frontends not complying with

the guidelines. A business walkaround also has to be developed for cases where the

CA has started a purchasing procedure using the Frontend and an error occurs.

The Service Level Agreement with the platform hardware provider will have to be

signed:

- requiring a guaranteed availability of the solution from the platform provider,

- requiring performing security audits of the solution,

- establishing penalties on such party for system unavailability

While the current hosting by the Amazon Web Services is a cost-efficient

solution, the goal of the proposed changes is to delegate much responsibility

for the system to other parties. The business owner of the solution should not

be involved with legal and organisational consequences of hardware

unavailability or security breaches;

The organisational framework to remove errors will have to be strengthened. When

an error occurs in the system its resolution and remedy, in most cases, requires a

coordinated work of many parties, such as Frontends’ administrators, Frontends’

developers, Prozorro administrators and Prozorro developers. This is a complicated



matter, and a clear set of organisational procedures should be developed, specifying

each parties’ roles and obligations. The ultimate goal of reorganising this process is to

ensure that all errors are resolved in a timely fashion. It is also important that changes

to these procedures make them more measurable. The business owner should know

exactly when an error will be corrected; a business walkaround should also be

available on such occasions.

Technical recommendations:

These suggestions concern the major functionality gaps in the current solution. It is believed

that solving them is essential for the efficient business functioning of the system.

eAuction link: the current system does not authorise the users in a proper manner

during an aAuction event. This may raise issues related to evidence if, e.g., one of the

bidders denies performing some actions in the system.

uniqueness of bidders: the system does not ensure uniqueness of bidders. This gap

may create legal ambiguities if the bidder creates two offers within one purchasing

process.

bid encryption: this should be implemented by both the Frontends and the central

database. This will ensure that before the bid opening date nobody can see the offers.

prequalification of the bidders: this should be based on the formal criteria. The split of

the bid between its formal and commercial parts will have to be implemented, which

will allow the CAs to disqualify some competitors before the eAuction event

hash values of submitted docs: these should be available in the system to ensure that

no document manipulation has taken place within the system

Elaboration on the implementation of these technical changes and assessment of the amount

of work needed to make these changes happen in beyond the scope of this paper.

Recommendations for future development

It is a common understanding that the future functionality of the system should embrace

higher value purchases and other procurement procedures beyond eAuctions.

It was however not possible to assess whether this future business development of the system

should be done within the current technical landscape. There are several reasons for this

question remaining unanswered:



the scale of changes will be very broad: at the moment the system supports only one-

variable eAuctions, which is a small subset of the future functionalities

o all purchasing procedures will have to be implemented

o more e-services will have to be added, such as eOrdering, eCatalogs, eInvoices

etc.

the scale of implementation will change dramatically: the authors admit that the

current system can support up to 10-20% of the future projected workload without

changing the system’s architecture

o changes to hardware are needed to support bigger workloads

o changes to the software architecture are needed to support horizontal scaling

It is therefore recommended to perform a full-scale analysis before deciding on the way of

implementing future functionalities. This analysis should assess the entire spectrum of

scenarios ranging from extending the current solution, through developing the new system,

and include the case of purchasing a completely new system.

Date post:	13-Feb-2017
Category:	Documents
Upload:	dinhbao
View:	221 times
Download:	2 times

Prozorro Project

Documents