ARCHEE WORKSHOP 1

ArchEE WorkshopDAY 1: 17 JUNE, 2015

WASHINGTON, DC

17-18 June 2015

ARCHEE WORKSHOP 2

Welcome!

17-18 June 2015

ARCHEE WORKSHOP 3

Who is ArchEE?

17-18 June 2015

ARCHEE WORKSHOP 4

Who is in the room? What is your . . .

Name?Affiliation?

What would ArchEE look like to engage you totally?

What is one burning question?

17-18 June 2015

ARCHEE WORKSHOP 5

Why a Workshop? We will create a foundation for collective action to design, develop, implement, and sustain ArchEE.

17-18 June 2015

ARCHEE WORKSHOP 6

What questions will we consider?

Who are we and what are we about?

What is ArchEE and what will its success look like?

What is ArchEE’s Theory of Change?

Who will use ArchEE and how will they use it?

How will we will support ArchEE development and implementation?

How will we sustain ArchEE into the future?

What needs to be done and by whom and when?

What is next?

17-18 June 2015

ARCHEE WORKSHOP 7

WednesdayWhat? So what?

17-18 June 2015

8:00 – 8:30 Arrive and set up8:30 – 10:45 Get started10:45 – 11:00 Break11:00 – 12:30 Build foundation12:30 – 1:30 Lunch1:30 – 3:00 Define success criteria3:00 – 4:30

With breakHow will it work?

4:30 –5:00 Close the day5:00 – 5:30 Prepare for tomorrow7:00 – ??? Take me out to the ballgame!

ARCHEE WORKSHOP 8

ThursdayNow what?8:00 – 8:30 Arrive and set up8:30 – 9:30 Gather and settle9:30 – 10:45 Define opportunities arising from use cases10:45 – 11:00 Break

11:00 – 12:30 Plan for managing ArchEE in the short- and long-term

12:30 – 2:00 Lunch2:00 – 3:45 With break

Plan for action

3:45 – 4:30 Integrate plans and commit4:30 – 5:00 Close the meeting5:00 – 5:30 Clean up and exit7:00 – ??? Dinner at Matt’s house

17-18 June 2015

ARCHEE WORKSHOP 9

What will we produce? Technical and project requirements

Conceptual design

Draft Theory of Change

Project structure and building blocks for a project plan

Commitment from partners

17-18 June 2015

ARCHEE WORKSHOP 10

Patterns of Hope and Excitement

What do you hope for and what excites you about:Your own work with ArchEE?This workshop?ArchEE’s long-term impact?

17-18 June 2015

ARCHEE WORKSHOP 11

What structures the systems work?

Human Systems DynamicsComplex Adaptive SystemsAdaptive ActionPattern LogicRules of InquiryThree kinds of questions

17-18 June 2015

Complex Adaptive System (CAS)Self-Organizing System

Those system-wide patterns, in turn, influence the behaviors of the agents

Agents interact

System-wide patterns emerge

Adaptive Action

What? So what?

Now what?

ARCHEE WORKSHOP 14

Pattern Logic Method of reasoning based on conditions for self-organizing:

Similarities (C), differences (D), and connections (E)Patterns

Works under conditions of:Open boundariesMultiple interdependent variablesNonlinear causality

Supports best fit action under conditions of uncertainty

17-18 June 2015

ARCHEE WORKSHOP 15

Simple Rules of Inquiry Turn judgment into curiosity

Turn conflict into shared exploration

Turn defensiveness into self-reflection

Turn assumptions into questions

17-18 June 2015

ARCHEE WORKSHOP 16

Kinds of Questions Answers are:

Known by someone, we just have to find the right expertUnknown by anyone, but we have a method to find outUnknowable here and now, we watch patterns emerge over time

What is a question you have about ArchEE?

17-18 June 2015

ARCHEE WORKSHOP 17

What Is ArchEE? . . . an open-access repository of evaluations and other evaluative knowledge that can be analyzed and synthesized to support evidence-based practice and improved transfer and use of evaluative knowledge for environmental management.

In general . . . In general . . . , exceptOn one hand . . ., on the other . . . I am surprised that . . . I wonder . . .

17-18 June 2015

MaterialsAluminum

Energy

ManufactureFactory

ProductCanned Soda

UseThirsty

ReturnCollection

DisassemblySeparate

DisposeLandfill

MaterialsLifecycle

Cradle to Cradle Management

A Model to Emulate

AccessUser Interface

EvaluateCollect Data

Analyze

ReportMethods Findings

UseEvidence-Based Archive

Cabinet Shelf

Website

Cradle to GraveEvaluation

Local Learning

One Evaluation

ArchEELifecycleArchEE

Lifecycle

Cradle to Cradle Evaluation

Global Learning

Some products skip use and go directly into the database.

ResourcesUser Interface

ArchEE and associated repositories are now added to the pool of resources available to evaluators, managers, policy makers, researchers who may choose to access ArchEE to evaluate . . .

Materials

Evaluate Collect Data

AnalyzeSynthesizeM

anuf

actu

re

This is the process of using resources to create something new and useful to be seen as products and services

UseEvidence-Based

Just as the can is for use, the products and services that come out of the evaluation process are for use in: decisions, understanding, communication, learning, building evidence, design . . .

Use

CollectDatabase

Return

Just as the can is returned to a collections facility, evaluation products are returned to a repository/database. This can happen in many ways, but once it is returned, it is available for analysis etc . . .

The information is analyzed, disaggregated, disassembled, tagged, sometimes synthesized. After it is in the database and on the platform, that evaluative, evaluation knowledge is available and accessible as additional resources . . .

AnalysisTag

Disaggregate

Disassem

bly

Products & ServicesMethods Findings

Prod

uct

A report, paper, findings, questions, recommendations, methods, knowledge, information, websites, tools, evaluators, expertise, experience, topics and so on are then, hopefully for use . . .

The user can customize analysis to be fit for purpose.

ARCHEE WORKSHOP 21

What is a Theory of Change?

A theory of change (TOC) is a tool for developing solutions to complex social problems. A basic TOC explains how a group of early and intermediate accomplishments sets the stage for producing long-range results. A more complete TOC articulates the assumptions about the process through which change will occur and specifies the ways in which all of the required early and intermediate outcomes related to achieving the desired long-term change will be brought about and documented as they occur.

Adapted from Anderson, A. (2005). The community builder's approach to theory of change: A practical guide to theory and development. New York: The Aspen Institute Roundtable on Community Change.

17-18 June 2015

ArchEELifecycleArchEE

Lifecycle

1

5

4 3

2

6

ResourcesUser Interface

Evaluate Collect Data

AnalyzeSynthesize

Products & ServicesMethods Findings

UseEvidence-Based

CollectDatabase

AnalysisTag

Disaggregate

What happens here?

So what makes it successful?

Now what are we sure about and what questions do we have?

ARCHEE DESIGN NOTES 23

What Requirements?ArchEE will succeed if we . . . 1.0 Attract and retain the right user base

2.0 Include the right stuff

3.0 Use state-of-the-art methods

4.0 Keep it simple

5.0 Make it a learning system

6.0 Sustain it

7.0 And . . .

28APR15

ARCHEE DESIGN NOTES 24

What Requirements?Setting the Foundation

Dennis Bours & Anna Viggh

Kara Chrohn

Gabriela Fitz

Marc Hockings

Andrew Pullin

David Widawsky

28APR15

ARCHEE DESIGN NOTES 25

What Requirements?ArchEE will succeed if we . . . 1.0 Attract and retain the right user base

2.0 Include the right stuff

3.0 Use state-of-the-art methods

4.0 Keep it simple

5.0 Make it a learning system

6.0 Sustain it

7.0 And . . .

28APR15

What is the most exciting and challenging?

So what is missing, unnecessary, wrong headed?

Now what change will you make?Now what questions do you have?

ARCHEE DESIGN NOTES 26

1.0 Attract and retain the right user base Participatory engagement

Make it as bottom-up as possible.Transparent process.Develop broad community investment; use network; seed small experiments throughout to work on different aspects.Funding source for seed grants.Distribute small, defined tasks that build engagement and investment. Seed the community with opportunities to beta test, usability test, etc. – maybe give small seed grants for stakeholders to become involved, perhaps do their own “use case” studies, etc. Provide small ways people and organizations can contribute and engage.

Effective communicationsPlan marketing, visibility – where will it be highlighted?Target specific audiences.Who is your audience? Who do you want to engage? If all inclusive is that the best strategy?Identify what value you will add for what audiences. What are users doing now? Google? How would ArchEE help? The energy industry, public health and other sectors already have repositories of studies, so what do you add, or how do you connect or relate to other systems?

Manage power and politicsDepend on network structures and processes to reduce hierarchyBe careful of hierarchies within user organizations – people at different levels of stakeholder organizations will have different perspectives on use, etc.Who are the stakeholders? Buy-in? Investment?

28APR15

ARCHEE DESIGN NOTES 27

2.0 Include the right stuff Collect and retain the “best”

Establish clear inclusion criteria, e.g. is it environmental evaluation or public health?Minimum quality requirements.Make use of existing sources like Better Evaluation.Standards, boundaries, and transparent decisions about what’Establish clear criteria for inclusion; what crosses over the border from environmental evaluation to other sectors such as health?Single decision-maker with clear criteria, guidance.Keep it fresh with new materials

Recycle, reuseConnect to other clearinghouse sites.Link to other repositories rather than duplicating their work.Use groups like the Environmental Education Association of Oregon to get feedback.

Follow the rulesCopyrightOwnershipPrivacyClearance and review

28APR15

ARCHEE DESIGN NOTES 28

3.0 Use state-of-the-art methods Prototype:

Build a RAPID prototype showing end-to-end functionality. Find the audience. Find the money. Make it cheap. Make users work.Build simple prototype for testing and adding to.Make an end-to-end prototype.

Learn from others:ViVo-digital CV – health field.HUB Zero – open source.Ask developers of other repositories for advice.First step: survey users to find out what they need most, first.

Take one step at a time:Start small, follow staged, phased-in development process.Do beta testing.Start small and expand.Start small and focused, then add components.

28APR15

ARCHEE DESIGN NOTES 29

Language:Avoid jargon, acronyms, exclusionary languageExplain all jargon, terminology clearly.

Ease of use:Usability testingExpert reviewPilot test each phase.Usability; if the interface of this repository can’t be used intuitively people will not like using it.Easy entry; Easy navigation, easy searchQuick response time

Common standards.Wiki-styleADA compliant

Effective information architectureUse the best of library scienceSimple, transparent categoriesThere must be some kind of taxonomy for querying tables in advance of artificial intelligence.Establish a template and/or protocol for use cases and recruit lots of folks to contribute these.

4.0 Keep it simple

28APR15

ARCHEE DESIGN NOTES 30

5.0 Make it a learning system Use a login system so you know who’s using it; perhaps a username with no password?

Do “use cases” in the beginning to understand what users will actually do and how they will do it and what they need.

User feedback.A safe place (anonymous) for posting lessons learned and failures and problems and challenges.Make ArchEE an open source siteArchEE creates competition in the education market.

28APR15

ARCHEE DESIGN NOTES 31

6.0 Sustain It Technical environment

Develop a place and a model for long term maintenance.Arrange sustained funding for long-term maintenance and curation. Software becomes vulnerable and requires patching;

FundingFind out if a subscription model would work – is there willingness to pay? Or a donation model like Wikipedia?Establish a funding mechanism.Fundraising for administration, development, maintenance of the server, tech support, management of content, etc.

Collective decision makingThe community needs cultivation and outreach Collaborative ownership to ensure diverse voices in the development and in the gatekeeper role. Don’t start something that requires consensus that dictates whether the project continues.

Accountability:You need a lead person or organization to make it happen; there should be a clear leader.An organization that hosts responsibly.A clear organizational structure – who’s responsible for what – paid positions or rotating responsibilities among groups or something similar.Establish a home institution that can organize funding and maintenance/technical support.These tasks must be somebody’s job who is responsible and accountable.Complete annual assessment and adaptation

28APR15

ARCHEE DESIGN NOTES 32

7.0 What is missing?

28APR15

ARCHEE WORKSHOP 33

Who will use ArchEE?Users Uses

Evaluators • •

• •

Funders • •

• •

Managers • •

• •

Planners • •

• •

Policy Makers • •

• •

Researchers • •

• •

Teachers • •

• •

17-18 June 2015

ARCHEE WORKSHOP 34

So what? Complete Use Cases0.0 Submitting to ArchEE1.0 Evidence-Based Decisions2.0 Strategic Investment3.0 Policy Design4.0 Evaluation Design5.0 Collaboration

6.0 Recruiting7.0 Quality8.0 Evaluation9.0 Resource Allocation10.0 Research11.0 Capacity Development

17-18 June 2015

ARCHEE WORKSHOP 35

Close Day 1 What have we found?

What do you want more of, less of tomorrow?

What changes do we need to make to our questions?

What is our agenda for tomorrow?

What else for the good of the group?

17-18 June 2015

ARCHEE WORKSHOP 36

ArchEE WorkshopDAY 2: 18 JUNE, 2015

WASHINGTON, DC

17-18 June 2015

ARCHEE WORKSHOP 37

Get Started Overnight thoughts

Questions from yesterday—more? Different?

Revisit purpose and deliverables

Rules for Inquiry

And . . .

17-18 June 2015

ARCHEE WORKSHOP 38

So what? Complete Use Cases0.0 Submitting to ArchEE1.0 Evidence-Based Decisions2.0 Strategic Investment3.0 Policy Design4.0 Evaluation Design5.0 Collaboration

6.0 Recruiting7.0 Quality8.0 Evaluation9.0 Resource Allocation10.0 Research11.0 Capacity Development

17-18 June 2015

ARCHEE WORKSHOP 3917-18 June 2015

ARCHEE WORKSHOP 40

Now what about ArchEE Management?

What functions? What resources?Accumulating documentsEvaluating, coding and taggingStoring and maintainingAccessingCollecting and processing feedback about specific documents and processesResourcing and resource management

17-18 June 2015

ARCHEE WORKSHOP 41

Now what helps ArchEE learn?

What functions need to be in place to ensure on-going adaptation and growth?

17-18 June 2015

ARCHEE WORKSHOP 42

Now what will we do?Planning the Work

What tasks need to be done?

What are interdependencies?

What are resources?

What are task clusters?

17-18 June 2015

ARCHEE WORKSHOP 43

For each cluster, define . . .

Purpose

Deliverables

Assumptions

Tasks

Schedules and milestones

Resources

Responsibilities

Interdependencies with other clusters

17-18 June 2015

ARCHEE WORKSHOP 44

What questions will we consider?

Who are we and what are we about?

What is ArchEE and what will its success look like?

What is ArchEE’s Theory of Change?

Who will use ArchEE and how will they use it?

How will we will support ArchEE development and implementation?

How will we sustain ArchEE into the future?

What needs to be done and by whom and when?

What is next?

17-18 June 2015

ARCHEE WORKSHOP 45

What will we produce? Technical and project requirements

Conceptual design

Draft Theory of Change

Project structure and building blocks for a project plan

Commitment from partners

17-18 June 2015

ARCHEE WORKSHOP 46

Now what is next? What do we know for sure about ArchEE and its future?

So what do we wonder about? So what are the remaining questions?

Now what will we commit to do to move forward?

Now what are our next steps?

17-18 June 2015