Hudson Valley Future Summit 2026 · Group 4

Advancing AI Knowledge Work

Systems thinking for people,
agents and everything they touch.

George Jensen  |  Qualitative Systems Research & Operational Ontology
Group 4 Session · November 6, 2026 · 10:15 a.m.
QR code for geo.rungeo.run
Hudson Valley Future Summit 2026 · Group 4

George Jensen

How what one person knows becomes
what an organization knows.

George Jensen  |  Qualitative Systems Research & Operational Ontology
Group 4 Session · November 6, 2026
QR code for geo.rungeo.run

What I've been watching

Open-source models

Reveal the hidden reasoning for sovereign AI.

Agent harnesses

The tools and steps around a model matter as much as the model.

System 1 models

Quick yes-or-no choices, made fast and at a fraction of the cost.

Always-on agents

They keep working on their own and check in when it matters.

Why now · higher education, and every employer

The people who know how the work runs are getting ready to leave.

Higher ed · faculty37%

of tenure-track faculty are 55 or older, vs. 23% of all U.S. workers. At Harvard’s Arts and Sciences, tenured faculty 65+ doubled: 18% → 37% (2003–2023).

CUPA-HR (2020); FAS annual report via Harvard Magazine (Nov 2023)
Higher ed · all employees70

average retirement age for faculty and staff with 20+ years of service. And 1 in 4 employees is likely to look for a new job this year.

Fidelity, ~110,000 faculty and staff, via InvestmentNews (Sept 2025); CUPA-HR (2025)

Every retirement is a tacit knowledge transfer problem.

“Age debt”: Joshua Kim, “The Future of Work Is Grey, Age Debt and Universities,” Inside Higher Ed (Aug 5, 2026)

What I mean by tacit

What a person knows and why,
but has not put into words.

The workflows we perform without describing.
The reasoning we hold without stating.

The knowledge isn’t missing. It’s unshared.

Iceberg model1.3 What I mean by tacit

Most of what people know sits below the waterline

WATERLINE: WHAT GETS WRITTEN DOWN Roles, policies, the org chart Workarounds Handoffs · who covers for whom Judgment calls · exceptions Guiding principles · mental models EXPLICIT written down TACIT KNOWLEDGE known, done, never written down

What makes
knowledge
durable

What makes
knowledge
durable

CAPTURED

What makes
knowledge
durable

CAPTURED REVIEWED

What makes
knowledge
durable

CAPTURED REVIEWED OWNED

What makes
knowledge
durable

CAPTURED REVIEWED OWNED FINDABLE

Also the shape
of a team
that learns

CAPTURED REVIEWED OWNED FINDABLE TRUSTED
Stock and flow4 What a knowledge vault is

Accurate knowledge is a level, not an achievement

PEOPLE'S HEADS Listening + review the inflow The vault accurate, shared knowledge THE STOCK Staff turnover Documents go stale Vendor terms change THE LEAKS

It stays up only while the inflow keeps pace with the leaks.

From the Field · Harvard Business Review

How AI Agents Orchestrate Work Across Silos

Kris Johnson Ferreira and Jordan Tong · September–October 2026

QR code for Read the articleRead the article

“Every service runs on things people know but rarely write down…
Agents only know what’s explicit.”

Sarah Gibbons, “Making the Tacit Explicit”

Act 3 · All of us

The bottleneck moved from analyzing to agreeing.

Monday · the machine

Cards in minutes

1x1 interviews with three staff. Each transcript becomes cards, workflows and classifications in minutes.

Tuesday onward · the people

Agreement in days

1x1 review of every card with the person who does the work, day after day, until each workflow was validated.

“Sometimes the kind of qualitative insights I try to find are the ones that defy categorization.”

George Jensen

Act 3 · All of us

The knowledge vault is the Cinderella of automation.

At the ball

Automation

Agents that do things. The big story about AI, and the one leadership loves.

Sweeping the floor

The knowledge vault

Years of wikis nobody updates, drives where documents go to die, knowledge captured at the exit interview.

Every automation runs on knowledge. When it's wrong, automation does the wrong thing faster.

Leadership loves agents that do things.
But when access is locked down,
the agent can't do much.
The knowledge system still can.

Don't underestimate smart knowledge systems.

How I work · four stages

Knowledge first. Automation last.

Stage 1 · start here

Lay the foundation

1x1 listening sessions turn into workflow cards, reviewed and signed by the person who does the work. One canvas of every workflow.

→
Stage 2

Analyze and prepare

Find the patterns, gaps and handoffs, and structure the cards so a system can take them in and staff can relearn from them.

→
Stage 3

Deploy

A knowledge twin of one role, or a full organizational knowledge vault that staff can ask.

→
Stage 4

Look for automation

Only a small share of workflows are good candidates. Now you can tell which ones.

The real foundation is the knowledge vault, and a culture that keeps tacit workflows from walking out the door.

Data security · privacy · policy

Set the boundaries before the first interview.

1 · Scope

Know-how only

In: steps, exceptions, approvals, systems.
Out: donor records, staff personal data.

2 · Precision

Answers that do the job

Specific and sourced, traced to cards the person signed.

3 · Pipeline

Interview → ontology → review

Nothing is used until the person signs off.

4 · Policy and models

Follow the AI policy

Check institution and state policy. Use models that never train on your data, or self-host.

A map of how a role moves money is valuable. Treat it like the keys to the building.

A deeper dive · a ten-person nonprofit

Inside one AI knowledge system

Documents
→
Chunks of 500 words
→
Embeddings
→
A vector index (Pinecone)
→
A question in staff chat
→
The 5 closest chunks
→
An answer, with links to sources

Plus: a weekly usage report, and an audit that flags documents with no owner or an old review date.

The evaluation lifecycleAfter Husain & Shankar

Analyze, measure, improve, and then look again

Analyze Read real outputs. Name the failures. Measure Count how often each one happens. Improve Fix the biggest one. Check it worked. New data, new users, new failures: back to reading

The Analyze–Measure–Improve lifecycle, adapted from Hamel Husain & Shreya Shankar, Evals for AI Engineers (O’Reilly, 2026), ch. 1. Redrawn.

Thinking, fast and slow

A chat window is built to let you ask fast

and get back deeper thoughts.

System 1 · fast

Automatic, effortless

Recognizing a face. Reading a stop sign. Typing a quick question.

System 2 · slow

Deliberate, effortful

Weighing evidence. Checking the exceptions. Showing your work.

You type · System 1“How do we handle a gift that comes in after the fiscal year closes?”
→
It answers · System 2The steps, the exception, who approves it, and where that came from.

The answer can only be as deep as what’s inside. Someone has to do the slow thinking first.

System 1 and System 2: Daniel Kahneman, Thinking, Fast and Slow (2011). Used here as an analogy, not a model of the mind.

Two kinds of tooling

System 1 flags. System 2 decides.

System 1 · fast, continuousSystem 2 · slow, deliberate
What it isAI reading the flow of work: notes, chat, requestsListening sessions, the board, staff review
What it noticesRepeats, handoffs, the same question twiceWhy the pattern exists: judgment, habit or workaround
CostConstant and cheapScheduled and human
How it failsConfident and wrong; can feel like surveillanceSlow; depends on trust

When System 1 isn’t sure, it hands the question to a person.

Boundary mapthe thesis / close

A fast question only gets back the depth that's inside

THE BOUNDARY what somebody wrote down The system I build vault · agents · review Workflows Judgment Handoffs Workarounds listening

Knowledge ops · human-centered AI knowledge systems

A knowledge system isn’t launched. It’s operated.

Captured

Listen on a rhythm

Listening sessions every month, not just at the exit interview.

Reviewed

The person signs off

Nothing becomes the team’s until the person who does the work says it’s right, and that it can be shared.

Owned

Every card has an owner

A name and a review date. When the date passes, it gets checked, not trusted.

Findable

Staff can ask it

Plain-language questions, answers with sources, and “I don’t know” when it doesn’t.

Checked

Read real answers

Every week, someone reads what the AI told people and notes what was wrong or missing.

Transitions

Hires and retirements trigger it

A new hire learns from the cards. Someone leaving gets a session, months ahead.

Human-centered means people are the source, the reviewers and the deciders. The AI does the filing.

Where to start

Someone nearing retirement, or already back on contract? Start with stage one.

I’ll sit with them and map every workflow they carry onto one canvas:

  • workflow cards from 1x1 listening sessions
  • reviewed and signed by the person who does the work
  • ready for whoever comes next, and for the knowledge vault you build later

George Jensen · geo@researchOps.ai · geo.run

Thank you

Advancing AI Knowledge Work

Slides and survey results: geo.run

George Jensen  |  ResearchOps.ai
geo@researchOps.ai · researchops.ai
QR code for geo.rungeo.run

Appendix

Appendix

Backup slides for questions. More detail is in the knowledge vault at v4.geo.run.

Appendix A

Opening and why now

Why now · a wave, not an event

Leaders turn over faster than the knowledge they lead.

4 · 3 · 2

median years in the job: presidents, chief HR officers, provosts

CUPA-HR, Turnover in the Higher Education Workforce (2025–26)
26% or 12%?

New York’s state workforce eligible to retire within five years: the Governor’s briefing said 26%, Civil Service counted about 12% (16,237 people)

Empire Center (2024)

Every retirement is a knowledge transfer.

“Age debt”: Joshua Kim, “The Future of Work Is Grey, Age Debt and Universities,” Inside Higher Ed (Aug 5, 2026)

Why now · what leaves with them

The job description stays.
The job leaves.

Written down · stays

Roles and rules

The job description, the org chart, the policy manual, the system’s user guide.

Never written down · leaves

How it actually gets done

Why the year-end close runs in that order. Which donor needs a call, not an email. The workaround that keeps two systems in step. Who to call when the report looks wrong.

A machine can retrieve only what somebody wrote down. So the question is what gets written down before they go.

Before we start · the chasm I’m not crossing today

What this hour won’t cover

Which AI assistant to buy
Coding agents and “vibe coding”
Model benchmarks and leaderboards
Vendor demos and tool tours
Image, video and voice generation
AI policy, law and the EU AI Act
Training or fine-tuning models
Prompt tricks
AI in the classroom and academic integrity
Forecasts about jobs

One thing: how what people know gets into a knowledge vault, and how it's responses are expressed with grounded insights.

All of us
Us
Me

The map for today

How does what one person knows become what an organization knows?

Four stories, one at each crossing. Each one hands you a tool.

Appendix B

The vault and the pivot

What is a knowledge vault?

Plain files an AI can read, with an index that tells it where to look.

Why 2026: AI agents learned to read and write files. At Meta, a personal AI second brain spread to 63,000 people without anyone ordering it.

Why now: every AI conversation starts cold. The vault is what it knows before you type.

The last century’s problem: knowledge management failed because people had to organize it. Now the machine does the organizing.

It solved finding. It didn’t solve writing it down.

What I do now

From researching customers to auditing what staff know

Onboarding: what a new person needs, when they need it

Collaboration: what one teammate knows that the others need

AI: from my tools, to our team’s tools, to the organization’s

Most AI stays personal. The knowledge never makes the trip.

Appendix C

The paradox and Cinderella

AI made me faster on my own.
The speed stays with me.

The paradox: my vault grows. The team's doesn't.

Reinforcing loop (causal loop diagram)7 The paradox

AI made me faster on my own. The speed stays with me.

− + + − Reliance on personal AI Asking teammates Shared team knowledge Usefulness of team AI R THE PARADOX

Two minus signs make a reinforcing loop: it quietly runs in the wrong direction. Build one arrow per click.

Adding a balancing loop7 The paradox, and the fix

Listening is the loop you add to break it

− + + − − + + delay Reliance on personal AI Asking teammates Shared team knowledge Usefulness of team AI Gap: what one person knows Listening + review R THE PARADOX B THE FIX

The same R loop, plus a balancing loop through shared knowledge. Note the delay mark: review takes days.

Systems archetype: shifting the burden8 The Cinderella of automation

The quick fix starves the real one

+ − + − delay − + Work isn't flowing well Buy automation Do the knowledge work Capacity for knowledge work B QUICK FIX B REAL FIX R SIDE EFFECT

Automation relieves the pressure fast. The knowledge work is slow (delay), so it gets skipped, and the ability to do it erodes. Senge, The Fifth Discipline (1990).

Appendix D

The RAG deep dive

Behavior-over-time graph9 The RAG service: what happened

Questions per week, after launch

60 0 WEEKS SINCE LAUNCH what we expected 6 54 5 0 0 0 5 0 0 launch testing

Systems thinkers draw the shape of change before explaining it. The gap between the two lines is the story.

A deeper dive · what happened

Weekly questions: 6 · 54 · 5 · 0 · 0 · 0 · 5 · 0 · 0

The 54 was us, testing it at launch

96 documents: 20 named an owner; the audit flagged 76 as stale

It could answer who does what. It couldn’t answer how does this actually get done.

Systems archetype: limits to growth9 The RAG service

The engine was fine. It hit a limit.

+ + + + + − Good answers Questions asked Use and trust Questions about workflows Gaps hit: never written down R GROWTH ENGINE B THE LIMIT

Pushing adoption harder doesn't help. Remove the limit: get the workflows written down.

A deeper dive · what I changed

The retrieval was fine.
What it could retrieve wasn’t.

From a vector index to plain files: a knowledge vault an agent reads directly

From documents to listening: 1x1 sessions, cards, review, 61 frames signed by the people who do the work

The Whole Board

61 frames in 9 categories

Donor & Financial11
Program & Media11
Technical Companions11
Sponsorship7
Research & Evidence7
Governance5
Strategy & Roadmap5
Photo Audit3
System Map index1

Some knowledge gaps discovered

  • The photo judgment step lived in one person's head
  • Four people each carry a process only partly written down
  • Protecting a child's story: practiced, never written
  • Records describe the systems, not the work in each country
  • No rule for which incoming signal goes to whom
  • A spreadsheet decides what teams think is finished
  • Photo rights are granted by email, recorded nowhere
  • Only 20 of 96 documents name an owner
  • “ACH” means three different things
  • A workflow stopped when one person left
  • A fix was made and never documented

Appendix E

Evals

How would you know it’s right?

Evals: the discipline of checking what an AI system actually does

Built on Hamel Husain & Shreya Shankar, Evals for AI Engineers (O’Reilly, 2026)

Three Gulfs, and a fourthAfter Husain & Shankar

Where an AI system goes wrong before anyone notices

You The data The AI system People who do the work Comprehension Do you know your data? Specification Did you say what you meant? Generalization Does it hold on new inputs? The unwritten Was it ever written down?

Three Gulfs adapted from Hamel Husain & Shreya Shankar, Evals for AI Engineers (O’Reilly, 2026), after Shankar et al. (2025) and Norman (1988). Redrawn. The fourth gulf is my addition, not theirs.

Error analysis · the first step

The first step in evals
is a research method.

Step 1 · open coding

Read real outputs

Read about 100 real traces. Write what feels wrong in plain words. “Put yourself in the perspective of the user.”

Step 2 · axial coding

Group what you found

Cluster the notes into failure types, and count them. Stop when new failure types stop appearing.

Quoted and adapted from Hamel Husain & Shreya Shankar, Evals for AI Engineers (O’Reilly, 2026), ch. 3. Open and axial coding come from qualitative research.

“If the retriever is not surfacing the relevant documents, no amount of prompt engineering or model tuning on the generation side will fix the output.”

Hamel Husain & Shreya Shankar, Evals for AI Engineers (O’Reilly, 2026), ch. 7

Evaluating a knowledge system · four pitfalls

Four checks before you trust the answers

The pitfallThe question for your team
Judging only the final answerDid it find the right documents? Was the answer right? Check them separately.
Testing on made-up questionsHave you tried 20–30 questions real staff actually asked?
Ignoring how documents are cut upDoes every document get in, in full, including PDFs?
Watching usage instead of qualityWhen nobody asks, is it because nobody needs it, or because it can’t answer?

The first three pitfalls are paraphrased from Hamel Husain & Shreya Shankar, Evals for AI Engineers (O’Reilly, 2026), ch. 7. The fourth row is mine, from the nonprofit.

Who decides what a good answer is?

Before they retire,
let them grade the answers.

From the book

One trusted expert

When one person’s expertise covers the work, a single “benevolent dictator” can be the judge of what’s right.

On a campus

The person who is leaving

Ask the system their questions. They grade its answers before they go. “Trust the AI” becomes “they checked it.”

“Benevolent dictator”: Hamel Husain & Shreya Shankar, Evals for AI Engineers (O’Reilly, 2026), ch. 4.

Appendix F

System 1 and System 2

Ask in System 1. Get back System 2 depth. Only if someone did the slow thinking first.

After Daniel Kahneman, Thinking, Fast and Slow (2011)

Appendix G

Quadrant Map

Quadrant Map for AI Maturity

The access a client gives me decides what I can change

Systems
People
Written down
Full access

IT infrastructure

Tools, data, integrations, access rights, vendor terms

Governance

Roles, policies, the org chart, who may decide what

Never written down

Workflows as practiced

How work really moves between systems, and the workarounds

Cultural norms

Unwritten rules, who people really ask, what's safe to question

Current client: full access to their systems. I cloned them, ran agents on the copy, and reviewed every document myself. Six months later: a searchable document management system for tech leads.

Quadrant Map for AI Maturity

Governance: the written rules about who decides

Systems
People
Written down

IT infrastructure

Tools, data, integrations, access rights, vendor terms

Needs leadership

Governance

Roles, policies, the org chart, who may decide what

Never written down

Workflows as practiced

How work really moves between systems, and the workarounds

Cultural norms

Unwritten rules, who people really ask, what's safe to question

Governance: the written rules about people: roles, policies and who may decide what. I can read it on day one, but I can only change it when leadership invites me in.

Quadrant Map for AI Maturity

Workflows as practiced: how the work really moves

Systems
People
Written down

IT infrastructure

Tools, data, integrations, access rights, vendor terms

Governance

Roles, policies, the org chart, who may decide what

Never written down
Needs staff time

Workflows as practiced

How work really moves between systems, and the workarounds

Cultural norms

Unwritten rules, who people really ask, what's safe to question

Workflows as practiced: how work really moves between systems, including the workarounds. No system records it, so I only find it by sitting with the people who do the work.

Quadrant Map for AI Maturity

Cultural norms: the rules nobody wrote down

Systems
People
Written down

IT infrastructure

Tools, data, integrations, access rights, vendor terms

Governance

Roles, policies, the org chart, who may decide what

Never written down

Workflows as practiced

How work really moves between systems, and the workarounds

Needs trust

Cultural norms

Unwritten rules, who people really ask, what's safe to question

Cultural norms: the unwritten rules: who people really ask, and what's safe to question. It takes the most trust to reach, and it decides whether changes in the other three quadrants stick.

Tool 6 · where to intervene

Leverage points

WeakestSwap in a newer model: a parameter
StrongerChange how knowledge crosses: who gets which information
StrongestChange what people believe

Donella Meadows, "Leverage Points: Places to Intervene in a System" (1999)

Appendix H

Close

Three things to try Monday

  1. Draw the boundary of one AI workflow: what can it read, and what does it guess at?
  2. Name one person within five years of retirement, and book an hour to listen to how they actually do the work.
  3. Read twenty real answers your AI gave last week, and write down what’s wrong in plain words.

Thank you

← All Hudson Valley presentations at geo.run