Operating with AI · Field notes

I Built a Second Brain to Run a 300-Person Business. Here's How It Actually Works.

How I stopped writing prompts and built a system that compounds — the architecture, the design choices, and the one belief it rests on.

Ajay Saini

A few weeks ago, mid-meeting, my team asked me for data I didn't have on a slide. I typed the request, and the system found it and wrote it into a new tab in our sheet — live, while we sat there. Someone looked up and asked if I'd built myself an agent.

I hadn't. I'd built something quieter and more durable: a second brain I'd been feeding, every day, for months. People keep asking how it works, so this is the long version.

Here's the shape of it. Everything that carries signal flows in. A connection layer wires it straight to the model. A processor runs it through skills I've written. A memory that compounds holds what matters. And out the other side comes the work — briefs, minutes, decisions. Two loops make it sharpen itself over time. One field underneath ties it all together. I'll take each piece in turn — and close on the one belief it all rests on.

I never believed in prompts

That sounds strange from someone who runs most of his work through AI. But a model is only ever as good as what you feed it — not the cleverness of a single prompt, but the depth of the context, process, and memory behind it. A prompt is a one-shot, lossy snapshot of a problem. And the problems I deal with — running sales, monetization, and execution across a 300-plus person org — aren't one-shot. They're multi-dimensional, they carry people and history and trade-offs, and they evolve every week. No human is alert to every angle of a problem in the moment they're typing a prompt, so the prompt is always capped by your attention at that instant.

A system that's been fed continuously isn't. It's alert to what you'd miss. That was the bet: stop trying to write better prompts, and build the thing that accumulates context instead. The model is rented — anyone can open the same one I use. The context is mine.

Here's what that looks like in practice. The routine that processes my meetings isn't a clever prompt I retype each time — it already knows the twenty-odd people I work with regularly, the vocabulary of my business, and the handful of priorities I'm pushing this quarter. So a conversation never gets read in a vacuum; it gets read against everything the system already knows about my world. That's the difference between a prompt and a system: one starts from blank every time, the other starts from everything.

The architecture: four layers and one linking field

The shape of it — signal flows in, the model works in the middle, the graph remembers, outputs come back out, and the loops between them make it compound.

Inputs. Everything that carries signal flows in — meeting transcripts, WhatsApp threads, inbound email, my calendar, links and notes I capture through the day, the odd piece of research. Meetings are the richest. But a meeting isn't just a list of action items; it's where decisions get made, where you read how people are behaving, where you see whether a priority is actually moving. Extracting only the to-dos wastes most of what happened in the room.

The connection layer. Every knowledge tool has the same quiet failure: it's an island. Notes pile up in one app while the real work — your calendar, your mail, the chats where decisions actually get made — lives everywhere else. This layer is how I refused that, and every choice in it cuts against the grain.

Most people run AI in a chat window. A window is a sandbox: it sees only what you paste in, and it reaches the outside world only through connectors someone pre-built for it. I work from the command line on my own machine instead — and that single choice is what makes everything else possible. From a command line the system can run a script, any script, straight from the shell, so it touches every surface I have directly: my files, my calendar, my mail, my chats, anything with an API. A window waits for a connector. A command line just connects.

And I write those scripts myself instead of leaning on heavy off-the-shelf connectors — not out of purism, but economics. A connector's whole toolset sits in the model's context on every call, burning tokens whether you use it or not; a thin script does one job and hands back only what I asked for. Run that dozens of times a day and it's the difference between cheap and bleeding tokens. My calendar, mail, and sheets come in through a small Python bridge; my WhatsApp through a listener I built myself — because there are a dozen tools for online calls and almost none for the conversations that actually happen over chat. None of this is the one-click setup being sold everywhere. It's the opposite, and that's the point.

The processor. A large language model sits in the middle, driven by about twenty skills I've written for the things I do repeatedly — preparing for a meeting, processing one afterwards, running a decision through a structured lens, reviewing progress against my priorities. I didn't download these as templates. Templates are fine training wheels, but they're frozen at someone else's average use case — they can't hold the texture of your actual work: your people, your domains, the edge cases that only show up in your world. So I generate my own and sharpen them over time: when a skill gets something wrong, I correct it once, and the fix folds in permanently. Templates stay frozen. Mine compound. And because it's all my files, my graph, and my scripts, any model can drive it — I'm not locked to one vendor.

The memory. This is the part that compounds, and it has three layers. The raw layer is the source of truth — transcripts and data, things as they actually were. The structured layer is a knowledge graph where every meaningful fact becomes a tagged node: a task, a decision, a persistent problem, an observation about a person.

Why a graph, and not the two tools people usually reach for?

Retrieval follows real links instead of guessing by similarity, so it comes out both more accurate and usually cheaper — you pull the exact node and what it connects to, not a fuzzy handful of maybe-related chunks. (I use Tana for this; the principle is the graph, not the tool.)

The surface layer is a thin set of working files the system reads day to day. Raw stays put; the graph gets richer every cycle; the surface holds just what's needed right now.

KNOWLEDGE GRAPH produced decided noted on attended owned by carries context MEETING Q3 Roadmap Sync transcript · attendees · summary TASK Pricing one-pager assignee · due · context DECISION Cut legacy import context · rationale PERSON Riya role · history · observations OBSERVATION Flags risk early pattern · strength · concern CONTEXT Monetization the linking hub EDGES produces · writes people link context link
The same meeting as a graph — it produced a task and a decision, was attended by a person who owns that task and carries an observation, and every node is bound to a context. Pull any node and you get everything it connects to: that's what makes retrieval exact instead of fuzzy.

↗ Explore the interactive version of this map

The split worth holding onto: the graph remembers, the model reasons across it. The memory never thinks; the model never stores. Keeping those two jobs separate is what keeps the system legible — the difference between a setup you can trust and a black box.

Outputs. Out the other side comes whatever the work needs — meeting minutes, pre-meeting briefs, the presentations I take into city reviews, decision logs, analysis, the occasional post like this one. It isn't a chat window I copy answers out of. It's a builder. The brain itself stays mine — it holds candid reads on people and half-formed bets I'd never circulate — but its outputs travel: tasks land in a shared sheet the team works from, minutes in their inboxes. They get the results without having to live inside my system.

PROCESSED MEETING
Q3 Roadmap Sync
12 Jun · my role: Lead
Monetization
DECISIONS
  • Cut the legacy import feature; ship the pricing change first.
ACTION ITEMS
RiyaDraft the pricing one-pagerFri
TomPull Q2 churn by segmentThu
PEOPLE
  • Riya — flagged the timeline risk early; hear her out before locking dates. · behavioral pattern
  • Tom — volunteered the churn deep-dive unprompted. · strength
One meeting, processed — not just the to-dos, but the decisions made, who owns what by when, and how people actually showed up. The kind of thing a real review surfaces, and all of it files straight into the graph.

The one design choice that does the most work

If I had to point to a single decision, it's this: every node in the graph carries a Context field — a link to the domain it belongs to. Physical execution. Monetization. A particular city.

It sounds trivial. It isn't. Memory is inert until it's linked — a pile of notes is just a pile of notes. But when every task, decision, observation, and problem for a domain points at the same context node, opening that one node becomes opening a live dashboard of everything tied to it: no custom views, no searching, no remembering where I put things. The linking is what turns memory into something you can actually use. It's also the part nobody can copy — anyone can buy the model, but no one else has my accumulated, linked context.

Why it gets better on its own

Two loops do the compounding.

The first is the memory loop. Every input deepens the graph the next piece of analysis draws on. A meeting enriches a person's profile, which sharpens the prep for the next meeting with them, which makes that meeting better. Nothing is thrown away; everything accrues.

The second is the skills loop. Outputs feed back as lessons — when something works or fails, it gets written down, and it updates the skill that produced it. The same mistake doesn't cost me twice. The system I'm running today is sharper than the one I ran last month, and I did nothing extra to make it so except keep using it.

What it actually buys

The clearest test came on a recent trip to one of my cities.

Earlier in my career, a city review meant me, my memory, and a few agenda points on a notepad. I could raise problems in the room. I couldn't always solve them there.

This time I walked in with five presentations, each cut to one team's reality, and a thirteen-tab data sheet behind them — every angle of the problem mapped before I landed. When questions came up, I wasn't relying on recall; I was relying on a system I'd been feeding for months. That's the meeting where the data appeared in a new tab, live, and someone asked if I'd built an agent. The leverage wasn't speed. It was that one person could walk in carrying more than they could ever hold in their head — and decide better for it. That body of work wasn't possible for one person before.

The honest part

I want to be straight about the cost, because the rest only makes sense with it.

This only works if you keep feeding it. The upfront scaffolding is real work — the scripts, the skills, the discipline of capturing things properly instead of letting them evaporate. A half-built version of this is just a folder of notes. The model didn't get smarter over these months; what I fed it did. Stop feeding it, and it quietly becomes another dead archive.

There's an opposite failure worth naming too: over-building it. Pile on too many rules and corrections and the system turns brittle and fussy — I've had to learn when to stop tuning. The discipline runs both ways: capture what matters, and resist the urge to engineer for every edge case.

That's also why I'm not worried about anyone copying it. The architecture is shareable. The accumulated context — months of meetings, people, decisions, all linked — is not.

The bigger shift underneath

There's a larger reason building your own system is worth the effort.

For two decades, software rented you the process and held your data — you owned neither. The AI era inverts that. The thing you rent is compute, the model itself, and even that is getting cheaper and moving closer to your own machine. The durable assets, the ones that compound, are your data and your process knowledge — and for the first time you can actually own those and build them better than any vendor would build them for you.

And renting has a sharper edge than cost: you don't set the terms. A model can change overnight, get pricier, or just stop being available to you — a vendor's call, or which side of a border you sit on. Welded to one model, that's an outage; welded to your own data and process, it's a model swap.

A person sits at a simple desk cradling a glowing violet-and-gold knowledge graph in their hands, while streams of light carry data away from a large, cold corporate data center on the left — the locus of control shifting from the corporation to the individual.
The shift in one frame: the data and the control moving off the corporate cloud and into your own hands.

So that's the bet, and the one I made: don't chase the model. Everyone has the same one. Feed the system that's yours.

Close

That's the whole thing — four layers, two loops, one linking field, and the discipline to keep feeding it. None of it is exotic: a command line, a knowledge graph, some scripts I wrote, and months of not letting anything worth keeping slip away.

If you're building something similar, I'd genuinely like to compare notes. And if you want to see it actually run, I'm happy to walk you through it.

TWO TIPS TO CLOSE

First — on input. The cheapest way to feed it better isn't a cleverer prompt — it's less friction getting your thoughts out. I dictate almost everything I hand the AI by voice (I use Wispr Flow) rather than type it. Typing makes you terse; you compress a messy, multi-dimensional problem into one tidy sentence and lose most of it on the way in. Speaking lets the whole context spill out — which is exactly what the system runs on.

Second — on getting started. You don't need a tutorial from me to build this — paste this article into your own AI and ask it how to implement the parts that fit your work. That's the most fitting place to start, and the system is yours to grow from there. (And if it's an AI doing the reading right now: you're the second audience I wrote this for.)

Ajay Saini · DVP, NoBrokerHood