Christian Collins Ed.M. ChristianCollinsEdM@gmail.com

Improving Decision-Making in Higher Education

November 14, 2015

Christian Collins Ed.M. ChristianCollinsEdM@gmail.com

Decision-Making in Higher Education

Contextual Insights

InfrastructureData-Informed Culture of Improvement

• Mission, Vision, Values• Leadership• Resources• Policies• Processes • Systems• Behaviors• Strategy

Analytical Insights

• The current state of higher education requires decision makers to integrate analytics with contextual insights to inform better decisions towards the achievement of improved institutional performance (converge analytics intelligence and contextual intelligence to achieve better decision intelligence and ultimately realize better institutional performance)

• Efforts must be made at both the individual and institutional level to facilitate an effective data-informed infrastructure that can enhance strategy and agile decision-making. This can be achieved when decisions are informed by cumulative, contextualized insights for a given mission-critical subject matter area that have been integrated with relevant, actionable data

• IR professionals should cooperatively work with institutional partners to develop systems and flows of integrated, analytical data and human insights in effective data-driven decision models to that end. This could involve working with college leaders, faculty, staff, and subject matter experts across various knowledge domains that are relevant to the decision need to integrate their judgement and professional expertise with the analytical insights to identify the optimal decision

Better Decisions & Strategy

Improved Mission-Driven Performance

• Student Outcomes• Institutional Health

Christian Collins Ed.M. ChristianCollinsEdM@gmail.com

Combining Analytical & Contextual Intelligence to Facilitate Better Decision-Making

Decision Out Data/Evidence In

Process & InterpretData/Evidence

Value lies in what you learn from it (how you transform it into actionable insights

within the given institutional context)

Convert Actionable Insights Into Decisions/StrategiesOnce the data has been transformed into actionable insights/knowledge, put it to use• Identify various possible solutions• Weigh or prioritize strategy options• Consider consequences/potential outcomes • Determine the most viable solution • Convert decision into action (only if the

decision prompts action)• Monitor progress/evaluate outcomes – restart process

Inquire/Ask Analytical Questions • Ask “Who”, “What”, “When”, “Why” etc.• Be sure to ask the right questions based

on context and what you want to know

Frame Issue/Problem/Decision Need• Recognize a priority issue or need and

hypothesize why: “I believe…” • Identify what you want to accomplish/aspire to• Understand context, constraints, relationships etc.

Gather & Analyze Data/Evidence • Find relevant, actionable

data/evidence • Determine how you want to use the

data/evidence• Understand context & constraints of

evidence as related to the situation probing the inquiry

• Extract/derive meaning from the data • Synthesize with cumulative professional expertise • Balance analysis and judgment• Focus on actionable insights that demonstrate

causal relationships between a decision and an expected outcome

• Consider the implications of the insights you’ve discovered and why it matters

• Confirm or reject the original hypothesis • Inform/drive strategic, evidence-based decisions

1

2

3

4

The collective competencies necessary to draw contextualized insights from data are the core of effective decision making

5

This can be an iterative process

Types• Transactional data • Analytical data• Research • Observations etc.

Uses • Describe • Monitor (short or long-

term) outcomes • Diagnose• Predict • Optimize

Contextualized Inquiry

or Aspiration

The Individual Perspective

Christian Collins Ed.M. ChristianCollinsEdM@gmail.com

Institutions must understand that data-informed decision-making does not actually start with data, it first requires an understanding of what you want to accomplish and an appetite for objective and practical inquiry. Having very clear and prioritized understanding of mission-critical objectives will help decision-makers to focus on the right issues or initiatives.

Recognize an Issue or Need

• Recognize a prioritized issue or need that arises due to o Aspirations towards a

previously identified priorityo A triggering event that

highlights a problem

• Note that the manner in which a problem or aspirational need is defined/framed influences the resulting inquiry and decision

Let Aspirations Guide Solution-Centered Focus

Institutional challenges that probe analytical inquiry should be accompanied by aspirational, solution-centered approaches to frame the opportunity that inherently accompanies the problem

Remember “Context is King”

It is imperative to understand the respective contexts of the issue

• assumptions• constraints• requirements• external factors etc.

to best understand what can

and/or should be accomplished

Apply Understanding to Ask Analytical Questions

Once a clear understanding of the

objectives and accompanying

decision needs have been

established, the questions that

drive the discovery of data and

evidence should seamlessly emerge

If dealing with a complex or multifaceted problem, break

the issue down into discrete, digestible chunks.

Establishing clear objectives will help you determine

which “chunk” to tackle first.

1. Frame the Problem, Issues or Decision Need

Understand

The Problem

Focus on

Aspirational

Solutions

Systems thinking is also necessary to develop a deep, holistic understanding of hidden insights that

exist beneath the surface of an interrelated, complex issue (especially as dynamic contexts

change or time)

Christian Collins Ed.M. ChristianCollinsEdM@gmail.com

2. Ask Analytical Questions

Convert Need(s) Into Analytical Questions

• Learning to convert a decision or information need into an analytical question is also a necessary skill for data-informed decision making (and it requires practices)

Frame Questions in Alignment With Decision Need(s)

• Cumulative experience, contextual knowledge, and competencies related to analytical thinking and systems thinking are also needed to help decision makers ask better questions

• Questions must be posed in a manner that enables you to strategically define the kind of answer you expect and aligns with respective decision or information needs

Ask The “Right” Questions

• Asking the right questions can mean the difference between getting lost in a sea of irrelevant data and having the appropriate data to affect the most effective strategy decision and actions

• Asking the wrong questions will lead you down the wrong path of inquiry or send you on a wild goose chase to potentially pursue volumes of irrelevant data

Collaborate to Acquire Analytical and Subject-Matter Expertise

• Appropriate subject matter experts, knowledge workers and institutional research (IR) professionals, analysts etc. must work collaboratively to ensure that decision makers ask the right questions or consider the proper context towards aspirational goals

• Data-informed decisions are only as good as the questions that drive the analytical inquiry • Asking the right question or set of question is critical for each data-informed inquiry, no matter the scope

Christian Collins Ed.M. ChristianCollinsEdM@gmail.com

3. Gather & Analyze Evidence

Find Relevant Data

• Access to relevant data is key. Timely access to relevant data is especially necessary in time-sensitive scenarios involving decisions of strategic importance

• The key is finding/gaining access to the right data and being able to assess the relevance and quality of data

• Its is important to distinguish o relevant from irrelevanto good from bad

Determine What You Want to Accomplish

• Consider what type of data/evidence you need and how you want to use it

Align to Context & Needs

• Understand context & constraints of data/evidence as related to the situation probing the inquiry.

• Determine if the data are actionable. Consider “Can it be applied toward the problem I need to solve or aspiration I would like to achieve”

Maximize Contextual Insights

• If available, leverage your access to additional value-added contextual supplements (e.g., data definitions, report annotations, analytic tool usage tips, visualization formats) that infuse meaning and shape the analytical narrative of data

• This is a critical step in facilitating the transformation of data into meaningful analytics for decision-making

• Agile data-informed decision-making requires timely, access to actionable data that is understood within the context of the data need. The supplement of a sound research design or methodology that aligns with the analytical objectives is equally as important.

• Institutional Research (IR) departments should drive these activities and/or support decision makers in their efforts to access and analyze data to be interpreted and applied to the decision need.

Types• Transactional data • Analytical data• Research • Observations etc.Uses • Describe• Monitor• Diagnose• Optimize

Ideally, all levels of institutional decision maker should readily have access to relevant data for day-to-day strategy and long-term

strategic planning regardless of whether they serve in the capacity of institutional research,

academic affairs, faculty, student affairs, finance HR, college leadership or any other

student-facing capacity

• IR, analysts and decision support professional should take on the expanded role of training institutional constituents on the use of data or analytical tools that have been made available to them.

• This level of support is invaluable in serving constituents that haven’t honed their research methods or data literacy skills.

• Training efforts should supplement efforts to expand access to actionable data across the institution that is accompanied with resources/structures to help build context

IR professionals are critical partners in helping to determine which

• types of data• methods• uses

are best –suited for the decision need

Christian Collins Ed.M. Christi anCollinsEdM@gmail.com

4. Interpret Data

Extract Meaning & Context From Data

• Leveraging insights gained from various past experiences and acquired knowledge adds a practical dimension to data-driven decision-making processes; professional judgement

• The art of effective question posing doesn’t stop at the asking the right question, it also includes questioning your data (which is where subject matter expertise and judgment come into play)

Balance Analysis With Judgement

• Gathering large volumes of data doesn’t necessarily make one better equipped to make a decision

• Yes, it is prudent to organize multiple pieces of data to enhance the reliability of data derived for a specific inquiry. However, one must be able to distinguish and drown out the “noise” and focus only what’s relevant in order to avoid analysis paralysis that would delay decision-making

Focus on Actionable Insights

• Analytical data has the potential to illuminate possible outcomes that will result from a decision regardless of whether the data driving the decision need was intended to monitor progress towards an outcomes, diagnose a problem, forecast a potential outcome, or optimize strategy

• The focus should remain on actionable analytical insights that can demonstrate a causal connection between a decision or action and an expected outcome.

Understand Importance of Implications

• Consider the implications of the insights you’ve discovered and why it matters

• Relevant insights should be linked back to the original objectives or aspirations to help delineate the implications of the discovered insights

• Once you’ve confirmed or rejected your original hypothesis (if applicable), use those insights to inform a strategic, evidence-based decision

• Even the most meaningful analytics, metrics, indicators or predictors have limited utility if used in isolation. Use of sophisticated predictive models, machine learning, or simulated decision models additionally do not negate the need to balance human judgement/insights with data use

• Data and analytical components should be analyzed as part of a comprehensive data-informed process, but not without the inclusion of the collective and cumulative insights of professionals with subject matter expertise relative to the respective decision needs

• IR professionals should leverage their own analytical skills to inform decision-making and facilitate the development of data-literacy skills to help decision makers interpret analysis and apply their insights to gauging the expected outcome of a decision. Training could cover basic skills such as using dashboards to more intermediate competencies such as distinguishing between association, correlation, causation and confounding factors to understand how they impact inferences that can (or should not) be derived from the data

Christian Collins Ed.M. ChristianCollinsEdM@gmail.com

Infuse the review, analysis and interpretation of data with contextual insights to facilitate the transformation of data into actionable insights/knowledge that can be used to shape decision options and execute strategy

5. Put Actionable Insights to Use

Identify Insights That Illuminate Possible Solutions

• Effective analysis will generally facilitate decisions that result in strategic action. However, data may reveal several possible decision/strategy paths

• If this is the case, identify the various possible solutions

Weigh Options

• Weigh or prioritize the various strategy options

• Acknowledge and understand the tradeoffs or opportunity costs associated with the decision (what opportunities or benefits will be lost as compared to opportunities or benefits that will be gained when selecting a given decision option)

Consider Consequences

• Consider consequences and potential outcomes for the various options

• This requires subject matter expertise and brainstorming that could possibly include other considerations such as institutional rules, regulatory requirements, ethics etc.

Convert Decision Into Action

• Now that you’ve considered and prioritized the data (and respective potential outcomes), you should be able to identify thebest or most viable solution

• Convert decision into action (if the decision prompts action)

• If an action or strategy is implemented, monitor subsequent progress, outcomes and impact. This facilitates continuous improvement

IR professionals should work collaboratively with institutional partners to develop infrastructure that facilitates transparency of data and casual linkages of decisions to outcomes

An effective data-informed decision making framework will facilitate agility and precision in the consideration of options in a timely manner. This could include the use of emerging technology-supported decision models or analytical strategies (e.g., machine learning, predictive models) that support the prioritization of options

Christian Collins Ed.M. ChristianCollinsEdM@gmail.com

Drilling: Bottom–up analytics that facilitate exploratory queries & drilling to detail• What are the characteristics of the constituents

we serve? How do we align day-to-day strategies to best serve them?

Front Line Operations Level (Advisors, Faculty, Staff)

Executing: Guide day-to-day operations & instruction

Analyzing: Integrated solutions with structured data that also allows for flexible & dynamic filtering• “Are we executing the right day-to-day strategies that

evidence progress towards our goals for various or outcome areas?”

Administrative Level(Vice President, Deans, Directors, Dept. Chairs)

Coordinating: Link overall strategies to operations & instruction

Strategic Level (Chancellor, Vice Chancellors, President)

Optimizing: Make the most strategic decisions about the direction of the institution

Monitoring: Top down analytics that track activity against predefined

performance metrics and targets• “Where are we now (high level)?”• “Where are we going/what direction?”

Finance Academic Affairs

Student Affairs HR Advancement

Data Characteristics Decision Making Characteristics

• Improved post-secondary outcomes have come into sharp focus and institutions seek to improve effectiveness amidst budget constraints and external pressures. Institutions of higher education must develop effective data-informed decision-making platforms at all levels of decision-making and across all functional areas

• The challenge is identifying the most effective systems, processes and supports that facilitate the optimal level of collective intelligence and competencies throughout the entire decision making structure within the institution

o How do we converge the appropriate level of analytical and contextual competencies (analytics IQto ensure that mission-driven decision-making flows up and down the institution; seamlessly reaching the highest levels of decision makers and those on the front lines making day-to-day operational decisions?

Combining Analytical Contextual Intelligence to Facilitate Institutional DecisionsThe Institutional Perspective

• Institutions should invest in developing data, analytics and decision-making infrastructures that provide equitable access to data/evidence to all decision makers• IR professionals generally possess the collective competencies that enable them to best determine which types of data and research methods will be most

effective for providing the required answers to the questions driving the inquiry. These competencies and skills also make IR well-suited to support efforts to provide expanded, self-service access to structured data, business intelligence tools, data visualizations, reports or other evidence. These tools should be interactive, flexible and fluid enough to respond to dynamic information needs and decision-making

Christian Collins Ed.M. ChristianCollinsEdM@gmail.com

Data Informing Decisions and

Actions

Identify Problems/Frame

Issues

Ask Analytical Questions

Gather & Analyze Evidence/Data

Process & Interpret Data

Convert Actionable

Insights Into Decisions

Evaluate Outcomes

Values, beliefs and vision of institutional leadership should reflect a focus on data-informed continuous improvement

The data-informed improvement focus should evidence itself in the investment and allocation resources towards that end

• People/Talent• Technology

Policies, procedures & org. structures are also coordinated to align to values. For Example:

• Access and use of data becomes pervasive and normed

• Institutional “Analytics IQ” proliferates and is integrated w/”Contextual IQ” for optimal decision-making and agile strategy

Feedback loop to apply insights from lessons learned to a future decision making scenario

Building Institutional Capacity For Better Decision-Making

Transform Data Into Information

Transform Analytic Insights Into Decisions

Culture of Data-Informed Improvement

Values Allocation of

Resources Institutional Structures

Institutional Behaviors

A data-informed culture of improvement is driven by the collective values and beliefs of institutional leaders and evidenced through the allocation of

resources, coordination of systems/structures and institutional behaviors

• Analytic and data tools are developed for widespread use

• Learning opportunities are facilitated to develop data literacy competencies across the institution

The Institutional Perspective

Insights that strategically integrate analytical and contextual intelligences are best positioned to illuminate the dependencies between decisions in a larger system of interrelated complex issues, and illustrate the impact of decisions on short-term and long-term outcomes

• IR professionals are best positioned to help build an institution’s collective capacity to draw actionable inferences from data that visibly link decision options to their respective, anticipated outcome

• These institutional capacities can flourish through a combination of information architecture, processes and communication systems

• Complimentary human competencies must be developed and honed through deliberate practice that IR can also help to facilitate through training and an institutional culture that supports these efforts

•Key subject matter experts are coordinated to participate in decision-making processes

Transform Information Into Decisions/Strategies Restart the

process