Why users want AI in some places and refuse it in others.
UC Berkeley CalCentral · Four research methods · Presented to CalCentral engineering team
Students said AI was 96% positive — until we asked about it inside the system they actually use. Then four out of ten said no.
Context
CalCentral is UC Berkeley's student portal. It serves tens of thousands of students for everything from class registration to financial aid to tax forms. The team running it was considering adding AI-powered search, and asked us to research how students would respond.
The framing of the request was telling. The team assumed the question was "should we add AI?" We thought the question was actually "what kind of AI, in what context, with what controls?"
The question we asked
The obvious version: "do students want AI in CalCentral?" But that misses the context-dependence that makes the answer interesting.
Could we add AI to CalCentral in a way that students actually wanted, rather than the way AI is usually added — as a chatbot bolted onto an existing UI?
What we did
- Five semi-structured qualitative interviews to understand current pain points with CalCentral and general AI attitudes.
- Reaction card exercises across four search engines (AOL, Bing Copilot, Google Search, Google Maps) — n=10, using a standardized 64-word reaction card set.
- Cognitive walkthrough of our prototype with internal users.
- Usability testing of the final prototype with 10 students across undergraduate, graduate, and PhD populations.
The finding that mattered
When we asked students to rate the four search engines in the abstract, Bing Copilot (fully AI) scored 96.7% positive. When we asked them the same question in the context of CalCentral, only one out of four said yes.
This was the moment the project pivoted. Students weren't anti-AI. They were against AI in this context — a system that holds their tuition records, their financial aid, their academic standing. The trust calculation shifted entirely when the stakes were personal and institutional.
Our recommendation became: don't add fully autonomous AI. Build a hybrid search — traditional results plus AI-generated summaries and suggested queries, with the AI components opt-in and always optional. Students wanted the affordances of AI; they didn't want to be forced into a chatbot relationship with their school's official platform.
What changed
We presented findings to the CalCentral engineering team at their annual developer conference. The hybrid recommendation aligned with the constraints they were already working under — they couldn't reasonably ship a fully autonomous AI into a system that serves financial aid records — and gave them research-grounded justification for the conservative approach.
What I'd do next
The "context-collapses-AI-acceptance" finding is reusable. I'd want to study it across other institutional contexts — healthcare portals, banking, HR systems — to see whether the pattern holds. My hypothesis: AI acceptance is inversely proportional to the consequences of being wrong, and institutional contexts amplify those consequences in ways consumer contexts don't.