Introduction

Not way back, certainly one of my engineering administrators got here to me with a request: His workforce appeared overloaded, and he needed to rent one other engineer. I made a decision to check a analysis assistant I had been constructing—an AI agent related to our inside methods by way of MCP—by asking it to investigate the workforce’s workload and write a hiring case.

What got here again was thorough. Headcount, service possession, dash velocity, ticket backlog, and capability allocation, all of it neatly summarized. However studying by the doc, I felt the identical frustration I’d felt with each AI-generated organizational report that’s come throughout my desk. It advised me what was occurring with out serving to me perceive why, or what I ought to really do. It was organized across the knowledge somewhat than across the determination. Briefly, it was the type of response that’s simple to agree with and troublesome to behave on.

Then I added yet one more factor to the configuration: the O’Reilly Professional MCP server. I reran the identical evaluation and requested a barely totally different query: “How would the specialists on O’Reilly evaluate this request?”

As an alternative of main with headcount and ticket counts, the output now opened with a discovering: “The operational overhead downside is structural, not a staffing deficiency.” Citing the Google SRE framework’s idea of operational toil, it famous that the workforce was working at roughly 67% toil, properly above the edge at which the SRE literature recommends structural intervention, and made particular, concrete suggestions: run a toil audit, set specific discount targets, and assign operational runbook possession. This wasn’t a advice for whether or not to rent or not. It was a grounded, traceable argument for doing one thing else as a substitute.

That distinction—between a knowledge abstract and an expert-grounded advice—is what this paper is about.

What follows is a case examine of how we constructed an organizational intelligence system at O’Reilly, utilizing our personal platform as a core element. The method I describe is grounded in engineering as a result of that’s the place I work, however it generalizes to any perform the place essential information is scattered throughout a number of methods and essential selections require synthesizing all of it. The recipe has 4 steps: map your data hierarchy; join these methods to an LLM by way of MCP and write a ability file that defines the way it ought to purpose; add the O’Reilly Professional MCP as an knowledgeable evaluate layer that grounds the evaluation in established frameworks; and construct a light-weight system for human-in-the-loop evaluate. I’ll clarify every step intimately and make the case for why the third step is the one which modifications every part.

Why organizational intelligence is getting more durable

To know the issue this method solves, it helps to look briefly at how engineering has modified over the previous three a long time. These forces have performed out first and quickest in engineering, however as AI instruments proliferate past the engineering workforce, the underlying dynamic of extra output, extra selections, and extra scattered data is spreading to each a part of the group.

Within the waterfall period of the Nineteen Nineties, software program organizations ran on central plans. All the pieces was specified up entrance, and leaders maintained visibility exactly as a result of all data flowed by a single coordinating doc. The plans had been brittle and infrequently fictional by the point they had been executed, however not less than everybody knew what was alleged to be occurring.

Agile changed central plans with small, autonomous groups working briefly sprints, and this solved the reliability downside whereas making a visibility downside. Vital selections started occurring domestically and rapidly—the best groups making the best calls—however the data wanted to see throughout all of these selections splintered into dozens of separate instruments. Product technique lived in a single system, mission execution in one other, code in a 3rd, and repair possession in a fourth. Extra issues bought shipped, however the big-picture view bought more durable to keep up.

The agentic period has intensified this dynamic dramatically. Particular person engineers as we speak can ship in a day what used to take a full dash workforce. The output is extraordinary, however the visibility is sort of gone.

Any effort that spans a number of groups, similar to a platform migration, a shared infrastructure change, or a reorganization, now requires monumental coordination overhead just because the knowledge decision-makers want to know the complete image is distributed throughout too many locations. And this isn’t an issue distinctive to engineering. It exists in any perform that runs on knowledge unfold throughout a number of methods.

Confronted with this visibility downside, I needed to construct one thing I may ask big-picture questions and get synthesized solutions again rapidly. Issues like:

• What’s the standing of this cross-team migration effort, and which groups are behind?
• A workforce appears overloaded. Do they really want one other engineer, or is one thing else occurring?
• What are the trade-offs of adopting this new infrastructure know-how?
• Assist me produce a scope assertion from this product temporary.

Constructing one thing that would reply these properly took two foundational steps, and getting it to supply suggestions primarily based on my particular enterprise context took two extra. Whereas my particular instruments are from engineering, the construction applies equally to a gross sales workforce synthesizing CRM knowledge and market analysis, or a finance workforce working throughout an ERP, a planning instrument, and exterior benchmarks.

Step 1: Map your data hierarchy

Each group has a set of methods the place essential information lives, and people methods kind a pure hierarchy that spans from strategic intent on the prime to operational element on the backside. Earlier than you may construct a helpful analysis assistant, it’s good to make that hierarchy specific, as a result of it’s the map of how selections get made, which sources carry probably the most authority, and the way totally different sorts of questions needs to be approached.

At O’Reilly, our engineering hierarchy appears like this:

Layer	System	Function
Roadmap	Productboard	Strategic targets, initiatives, and have prioritization
Execution	Jira	Epics, tales, sprints, and contributor monitoring
Implementation	GitHub	Supply code, PR historical past, and occasion instrumentation
Service catalog	Cortex	Service possession, dependencies, on-call, and Slack channels
Observability	Datadog	System efficiency, errors, and incidents

Your group can have a special set of instruments. A gross sales group would possibly place Salesforce on the prime, adopted by a income intelligence platform, advertising automation, and market analysis. A authorized workforce would possibly begin with a contract administration system, adopted by a regulatory tracker, inside coverage documentation, and a analysis database. The particular methods matter lower than the act of mapping them: understanding which layer solutions which type of query, and which sources take priority once they battle.

Step 2: Join your methods by way of MCP and write a ability that describes how one can purpose

This step has two components that should work collectively. First, it’s good to join your methods to your AI instruments by way of MCP. Then you need to write a ability file that tells the mannequin what to do with that entry. At O’Reilly, we name this whole grounding layer Professional Intelligence.

Configuring MCP is easy. Most main instruments now provide MCP connectors, and connecting them is often a matter of routine JSON configuration. For methods with out MCP connectors, a bash-capable agent with curl and jq can typically attain a REST API immediately. MCP simply makes it cleaner and extra dependable.

However MCP connections alone aren’t sufficient, and that is the half most implementations get unsuitable. MCP offers the agent entry to your knowledge, however it doesn’t inform the agent how one can use it successfully. With out specific steerage, the agent retrieves data and organizes it the best way the underlying methods manage it, which produces a knowledge dump, not an evaluation.

The ability file—a CLAUDE.md or SKILLS.md doc that gives particular reasoning directions—transforms retrieval into evaluation. Mine defines the reasoning hierarchy (which methods to seek the advice of for which forms of questions, and how one can weigh them), the output format (this isn’t a coding agent—it produces experiences and proposals, not code), epistemic requirements (present your work, identify gaps, floor assumptions for human verification), and tone. On that final level, I borrowed one of the crucial helpful directions from Ted Lasso: “be curious, not judgmental.” Including it meaningfully improved the standard of the output.

The ability is a codified model of how a talented analyst would method these questions. It encodes your group’s reasoning course of and makes it repeatable.

Step 3: Add the knowledgeable layer

With the analysis assistant related to our inside methods, I had one thing genuinely helpful: quick, synthesized solutions to questions that beforehand would have taken days to analysis. However I stored working into the identical downside: The experiences felt generic, and folks didn’t belief them. This problem factors to a basic limitation of AI-generated organizational evaluation that goes past any explicit implementation.

The generic evaluation downside

Normal-purpose AI assistants have a tendency to supply a recognizable type of organizational evaluation: technically affordable, balanced, cautious, and finally not very helpful. This isn’t primarily a failure of data—each main LLM has absorbed an unlimited quantity of administration and organizational considering. It’s a failure of grounding. When an AI assistant has no particular framework anchoring its response, it tends to supply suggestions broad sufficient to use to virtually any state of affairs: contemplate the trade-offs, weigh your choices, and guarantee alignment throughout stakeholders. These responses are laborious to disagree with and simply as laborious to behave on.

When a report says, “The workforce seems overloaded. Take into account including headcount,” it’s not unsuitable. However that advice may apply to virtually any workforce in virtually any firm! It received’t make a director change their thoughts, and it’s not one a management workforce can debate, refine, and act on.

What occurred once I added the knowledgeable layer

Calling on the O’Reilly Professional MCP didn’t present the mannequin with new information—a lot of the data was technically accessible already. Nevertheless, with out the Professional MCP and related expertise, the mannequin couldn’t use that data for something however the broadest analyses. Incorporating the Professional MCP and related expertise modified the character of the analyses by grounding them in particular frameworks, citing named authors and thresholds, and organizing their conclusions round established our bodies of practitioner information somewhat than basic ideas.

To make this concrete, right here’s the type of output the analysis assistant produced earlier than including the Professional MCP:

The workforce seems overloaded. The backlog is giant and the migration mission is consuming important dash capability. Take into account including headcount or lowering scope.

And right here’s what it produced after:

Based on Google’s SRE steerage, sustained operational toil above roughly 50% signifies structural inefficiency somewhat than a staffing scarcity. This workforce’s telemetry suggests roughly 67% operational toil. Hiring one other engineer would probably improve whole toil except operational possession is first diminished. Advisable actions: run a structured toil audit, set an specific toil-reduction goal under 50%, and assign runbook possession for recurring operational duties.

The second report cites a framework by identify, references the precise threshold that framework establishes, applies it to the workforce’s precise knowledge, reaches a special conclusion than the apparent one, and makes actionable suggestions. It’s the type of evaluation that modifications a dialog as a result of the director can see the place the conclusions got here from, interact with the reasoning, push again on the framework in the event that they disagree, or settle for it with confidence that it was reasoned somewhat than pattern-matched.

After I shared this model with my engineering director, their response was quick: That is defensible.

Frameworks aren’t information

Probably the most underappreciated side of O’Reilly’s content material library is that the worth isn’t primarily informational. A lot of the information in an O’Reilly guide can be found on the web, and LLMs have already learn a lot of the web.

The deeper worth of O’Reilly’s catalog is that it’s organized round coherent frameworks—full psychological fashions constructed by practitioners who spent years or a long time growing them. Google SRE. Staff topologies. Speed up. Area-driven design. The Supervisor’s Path. Wardley mapping. Designing Information-Intensive Functions. These are structured methods of serious about particular lessons of issues, developed with sufficient rigor that they’ll really information selections.

Frameworks are distinct from information in a essential means: They let you know not simply what’s true however what’s related, what to measure, what threshold issues, and what to do while you exceed it. A mannequin with entry to the SRE framework as an organized physique of practitioner information is extra more likely to floor it explicitly, apply it to the precise query at hand, and use it to anchor its suggestions, producing output that human reviewers can really interrogate.

This factors to the organizing precept behind the method described on this paper:

Organizational knowledge offers native proof about what is occurring in your particular context. Professional frameworks present collected practitioner information about how to consider issues of that sort. Good organizational judgment requires each.

The Professional MCP is the bridge between your particular enterprise context and practitioner insights. It connects the AI’s entry to your inside methods with a curated physique of experience related to the selections your group must make.

Why use MCP somewhat than importing your individual paperwork

The pure objection at this level is “Couldn’t I get the identical impact by dumping related PDFs into Claude, or utilizing Claude Tasks, or NotebookLM?”

The quick reply is just not fairly, and the explanations are sensible as a lot as they’re technical.

Importing paperwork offers you retrieval from these particular paperwork. The O’Reilly Professional MCP differs in a number of operationally important methods. First, the corpus is editorially curated round coherent practitioner frameworks. In contrast to a set of PDFs, which tends to mirror no matter you occurred to search out, the Professional MCP provides a sustained curatorial perspective: The authors are vetted, the content material has been by editorial evaluate, and it’s organized round established our bodies of data somewhat than assembled advert hoc. This can be a way more expansive type of proof base. Second, the corpus is maintained and up to date by O’Reilly. New titles are added, new editions substitute previous ones, and the content material stays present with none administration in your half. Third, the Professional MCP is configured as soon as and works constantly throughout your whole group and toolchain somewhat than being tied to a single consumer’s Claude Venture or a doc add that expires. Lastly, accessing content material by a correct API respects the suitable utilization phrases in a means that importing copyrighted texts doesn’t.

And when paired with a well-written ability, the agent could be directed to look explicitly for competing frameworks, floor instances the place the literature disagrees, and identify gaps within the accessible proof, offering a significant verify towards the frequent tendency of AI instruments to quietly favor no matter framework first appears to suit. That’s one thing you may encourage with any retrieval setup, however it works extra reliably when the underlying corpus is organized round coherent our bodies of thought somewhat than a heterogeneous assortment of paperwork.

What we’re not claiming

I wish to be clear concerning the limits of what Professional MCP does as we speak. O’Reilly doesn’t declare that Professional MCP robotically selects the one appropriate framework for each state of affairs, or that including it to your configuration produces consultant-quality evaluation with out considerate prompting and human evaluate.

The outcomes described on this paper had been the result of all 4 components—the interior organizational knowledge, the rigorously designed ability structure, the Professional MCP, and human evaluate—together working collectively.

The Professional MCP is a vital differentiator, however it’s not a magic layer you may add to an in any other case generic setup and count on to breed these outcomes. The system works as a result of every aspect does one thing the others can’t. The ability defines the reasoning course of, the interior MCP connections present the organizational proof, the Professional MCP offers the knowledgeable frameworks, and human evaluate provides the judgment and context that no AI system can generate by itself.

What the Professional MCP reliably contributes to that system is entry to a curated physique of practitioner information: technical and managerial frameworks which can be editorially organized round coherent our bodies of thought and troublesome to reconstruct from scattered net content material or assembled doc collections. Your organizational knowledge nonetheless tells you what’s occurring, whereas the O’Reilly Professional MCP helps interpret what it means. That’s a significant and concrete enchancment over an ungrounded AI assistant, and it’s one thing you may put in manufacturing and construct on as we speak.

A notice on hallucinations

No AI system eliminates the danger of hallucination. The Professional MCP doesn’t make the mannequin infallible.

What it does is change the burden of proof. When each advice is grounded in a named framework, a named writer, and a traceable quotation, a human reviewer can verify the reasoning somewhat than merely accepting or rejecting a conclusion. The query shifts from “Is that this proper?” (unanswerable in isolation) to “Does this framework really say this, does it apply right here, and do I agree with the conclusion?” That’s a query people can interact with productively, which is strictly what you need from a decision-support instrument.

Step 4: Human evaluate is nonnegotiable

Organizational methods hardly ever comprise the complete context behind a call. The assembly that modified every part occurred final Tuesday and hasn’t been written up but. A key particular person is quietly planning to depart. A strategic course shifted in a dialog that was by no means documented. AI can synthesize every part in your methods with exceptional constancy, however it could actually’t know what isn’t there, and organizational actuality modifications sooner than documentation does.

Extra essentially: AI can establish trade-offs, however it could actually’t determine which trade-offs matter. That judgment requires human information of context, priorities, and danger tolerance that may’t be absolutely encoded in any system. The purpose isn’t to take away people from the loop however to provide them better-structured enter to purpose from.

Prolong the knowledgeable layer by fixing collaboration

As I began sharing analyses extra broadly, I bumped into a brand new set of limitations within the collaboration layer. The analysis assistant produced paperwork. I shared them in Google Docs, and folks added feedback, however when the AI up to date a doc primarily based on reviewer suggestions, I needed to paste in a brand new model, which worn out the prevailing feedback. Paperwork proliferated with out clear relationships between them, and the AI had no visibility into the discussions within the feedback, which was the place an important context and pushback lived.

To unravel the collaboration downside, I labored with certainly one of our engineering administrators to construct what we name Superanswers, a system that makes use of GitHub because the supply of fact for AI-generated analysis paperwork and their related discussions.

The structure is easy: Paperwork are saved as Markdown recordsdata in a GitHub repository, a GitHub Pages website renders them with a clear interface that helps inline commenting, and all dialogue occurs in GitHub Discussions, that means each remark, query, and revision is versioned and traceable. As a result of the paperwork and their discussions dwell in GitHub, Claude Code has full entry to each. It will probably learn the doc content material plus the complete dialog that’s developed round it.

This permits a qualitatively totally different type of AI participation. As an alternative of producing a doc and stepping again, we are able to now ask:

What’s the consensus round this mission primarily based on the dialogue thus far? What questions stay unresolved? Incorporate the reviewer feedback and produce an up to date model.

The AI turns into a participant in an ongoing dialog somewhat than a one-shot report generator, which meaningfully shifts how organizational information will get constructed and refined.

What groups are utilizing Superanswers for

As Superanswers has unfold throughout our engineering group, the vary of questions individuals carry to it has been broader than I anticipated:

Theme	Typical questions
Structure and infrastructure	Ought to we make this transformation? What is going to it price? What would possibly break?
Operational effectiveness	The place is our toil coming from? What ought to we automate, simplify, or retire?
Staff well being and capability	The place is the workforce’s time going? What’s limiting execution?
Group and technique	How ought to we manage, prioritize, and make investments?
Engineering measurement	How do we all know if we’re wholesome and bettering?
AI and organizational studying	How can we construct higher methods for reasoning and decision-making?

None of those questions is about writing code. They’re about understanding a corporation, making selections, and coordinating work, and most of them would map naturally onto the issues of leaders in different features. The identical questions come up in any group navigating fast change with data scattered throughout too many locations.

How one can use the recipe

The AI dialog thus far has been dominated by a specific set of questions. However there are extra fascinating questions we needs to be asking.

We’ve spent lots of time asking…	What else could be attainable?
How can we make individuals extra productive?	How can we make organizations simpler?
How can we produce sooner?	How can we make sooner selections?
How can we generate output?	How can we generate understanding?
How can we speed up execution?	How can we enhance outcomes?
How can we collect knowledge?	How can we construct institutional information?
How can we automate duties?	How can we enhance organizational studying?

The challenges outlined on this chart aren’t distinctive to engineering. They exist wherever essential data is scattered throughout a number of methods and essential selections require synthesizing all of it.

Particular person productiveness issues, however organizations don’t succeed by having contributors go sooner in arbitrary instructions. They achieve this by making good selections about the place to take a position, allocating assets properly, surfacing issues earlier than they compound, and constructing institutional information that persists over time.

The recipe I’ve described might help organizations make these selections and construct that information.

The recipe for constructing an organizational intelligence system:

1. Map your data hierarchy. Establish the methods the place essential information lives in your group, from strategic intent all the way down to operational element. That is an organizational activity, not a technical one, and doing it properly requires understanding how selections really get made.

2. Join these methods by way of MCP and write a ability that describes how one can purpose. The MCP connections give the AI entry to your knowledge; the ability file tells it how one can suppose with that knowledge. With out the ability, you get retrieval. With it, you get evaluation.

3. Add the O’Reilly Professional MCP as an knowledgeable evaluate layer. Organizational knowledge offers native proof about what is occurring in your particular context; knowledgeable frameworks present collected practitioner information about how one can purpose about issues of that sort. This step bridges the 2. The O’Reilly library spans engineering, administration, knowledge science, safety, finance, product, and extra, organized not as a set of information however as coherent frameworks developed by practitioners who spent careers constructing them. The result’s evaluation grounded in named frameworks with traceable citations, one thing human reviewers can interact with and query, somewhat than generic recommendation they’ll solely settle for or reject.
4. Construct a light-weight system for human-in-the-loop consensus. AI-generated evaluation is a place to begin, not an finish level. You want a mechanism for individuals to evaluate, problem, and refine what the AI surfaces, one the place these discussions change into a part of the context the AI can study from in subsequent iterations.

The most important sensible lesson I took from this work is reframing what AI is definitely for in an organizational context. The distinction between a helpful AI analysis assistant and a generic one isn’t primarily about which mannequin you employ or how a lot knowledge you feed it. It’s about whether or not the reasoning combines native organizational proof with established knowledgeable frameworks. Your knowledge tells you what occurred. Professional frameworks assist interpret what it means. That mixture, with human judgment utilized on the finish, is what makes the distinction between a report that will get learn (possibly) and filed away and a advice that modifications a call.