Can AI Do It? Should AI Do It? A Framework for Making AI Decisions

A human hand interacting with a laptop displaying colorful graphs and data visualizations, alongside a robotic hand also reaching towards the screen.

AI can do a lot of work to support our MarComm teams. However, just because AI can do a specific task, doesn’t automatically mean that’s the best idea. AI should be used intentionally in ways that align with institutional guiding principles, operational strategies and while mitigating risk.

Over the last several months, I’ve looked at how institutions should be thinking about AI use on campuses. However, for AI to be successful, universities have to make decisions about AI use. They must decide where AI use is appropriate and where it isn’t.

Today’s Partner

Logo for Enrollify by Element451, featuring stylized text in orange with unique design elements.

This content is proudly sponsored by Enrollify.

Three Tenets for AI Decisions

Most AI decisions can be assessed from three core ideas. When looking at an AI decision, these three core ideas can provide guidance to help teams determine if the AI use is appropriate for the institution and worth the risk. Using these three ideas as a framework is a smart place to start when thinking about AI. It isn’t perfect, but it does create a rubric (more on that later) that can help in evaluation.

Permission

It is important to decide what decisions AI is allowed to make. If using AI, it should provide value, either in terms of productivity or efficiency. That means AI use cases should have a strong connection to either the strategic plan or advancing key institutional priorities like fundraising. It’s also really important to take a step back and decide: Is this a situation we are comfortable (regardless of how it aligns strategically) with delegating to AI, or does it need to remain a human-led decision? There are some things that I think should always be human decisions.

Visibility

This is concerned with how much insight we have into the decision itself. It’s important to understand the logic that AI is using to make a decision before just letting it do so. It’s key to be able to provide guardrails about how to make certain decisions. Then, we need to review outputs, monitor the patterns and ensure alignment. If we can do that, then it’s really about effectiveness. Does this free up time for our teams to focus on challenging situations, or is it just automation because it can be done? That answer may help decide if the risk is worth it.

Intervention

When using AI, things will go wrong. This tenet is about mitigating as much of the risk as possible before that happens. How can we ensure human approval when needed? If something appears to be off, it will be important to quickly have a mechanism to pause and correct. The big question to think about here is whether the speed AI provides could create irreversible harm if something doesn’t go as intended. The answer to that will probably help institutions make decisions.

Two Decision Approaches

Using the tenets above, I put together two possible decision frameworks. Either can be used for a specific AI situation, an AI software evaluation, or to make decisions about a broader approach. The first is a decision tree that asks yes-or-no questions about how to proceed, and the second is a rubric that can help suss out the specific level of risk in a particular situation. I developed these using the tenets above, along with operational strategies and guiding principles I reviewed from dozens of universities across the country this summer.

A decision tree outlining a process for reviewing AI systems based on sensitivity, public impact, and potential harm, with specific review requirements at each step.
Table depicting a risk rubric with areas such as Data Sensitivity, Human Impact, Decision Authority, Public Visibility, Accuracy Requirement, Legal/Compliance Risk, Brand/Reputation Risk, and Bias/Fairness Risk, categorized into Low, Medium, and High Risk with corresponding descriptions and scores.

I also turned this into an online app (thanks AI-based Lovable) that allows you to quickly answer a few questions and receive either the decision tree or the risk assessment in a PDF that can be downloaded. This is a great tool to take an idea to an IT team or your leadership to illustrate the strategic approach you’re taking when thinking about responsible AI integration.

Header titled 'AI Project Approval Assessment' with a brief description about evaluating AI use cases. Two pathways offered: 'Quick Triage' with a 4-question decision tree, and 'Full Assessment' featuring an 8-area risk matrix.

Keep At It

The reality is that the way we make AI decisions and think about responsible AI use continues to change. As new tools come online and AI becomes more sophisticated, it will be important to continue evaluating how to best use AI. A strong understanding of risks, guiding principles and operational strategies can help when thinking through new use cases. Then, when making a decision, it is key to remember the three tenets that can guide decisions and use a framework to make the best decision for the institution. AI will keep getting more capable. Our job is to get better at deciding when to let it act.

Leave a Reply

Discover more from And Carrie On

Subscribe now to keep reading and get access to the full archive.

Continue reading