My AI assistant is turning into an operating system

Canonical version: My AI assistant is turning into an operating system.

The way I use AI has gone through many iterations already. Writing them down in order makes the direction obvious, so here they are.

The iterations

ChatGPT. Like most people, that's where I started. It got frustrating fast: pseudo-random memory, generic answers, hallucinations.

Projects and an AI Master Prompt. Projects felt too disconnected from my knowledge system. Importing my notes just to get things done was boring and brittle. The Master Prompt worked, but maintaining it was a PITA.

Claude and Claude Code. I switched, and I started actually enjoying the process. That's also when I realized my long-standing knowledge system was pure gold in combination with AI. I began pointing Claude at my project notes, my daily notes, my tasks.

The output quality changed drastically. So I started thinking about how to structure agent rules so AI really understood the way I think, the way I write, how I organize my ideas and my work.

The first real assistant. Built on top of my notes. It felt different to everything else I'd stumbled upon. AI was much more aligned. Every piece of AI context I used as a "system prompt" was a note in my knowledge base. That alone made a huge difference.

MCP. This gave hands to my agent. AGENTS.md gave it directions, rules to respect, and an understanding of my knowledge system. MCP servers gave it access to other tools.

AI Skills. Once skills were introduced, I understood how powerful the whole combination would be, and started creating a lot of them.

I now have around 500 AI skills. They automate research, analysis, development, writing, publishing, social media management, SEO/AI SEO, troubleshooting, marketing, etc. Each skill is a codified process. Sometimes a loose one, sometimes strict, repeatable and almost deterministic compared to just asking AI to do X.

And since the Obsidian Starter Kit is an almost 1:1 copy of my own system, it now ships with hundreds of AI skills too 🎉

OpenClaw. After roughly 3000 renames, it arrived, and my assistant became autonomous. Up 24/7, reachable from anywhere, using my knowledge system, my MCP servers, my skills. That was a genuine WOW moment.

What OpenClaw added was autonomy, connectivity (WhatsApp, Discord, Telegram, Slack, voice) and scheduling. Interacting with it through different channels while it automated everything was powerful.

Since then I've been exploring other paths and other combinations (e.g., Claude Code loops, workflows, schedules, Hermes Agent, Pi Mono, etc). I haven't settled yet. But one thing is certain: I can't live without some version of that anymore.

What I'm building

Where does an agent run? How do I reach it when I'm not at my desk? What happens to a task when I close the laptop? How does it ask me a question while I'm walking the dog? Do I really have to leave my laptop half-open so my AI sessions can continue running while I'm AFK?

That's an operating system problem, so that's what I'm building:

• A control plane — one place to see and steer every agent I have running, on what, at what cost, and what it needs from me.
• Runners on every device — a small daemon per machine, so work executes near the data it needs.
• Channels, not a chat window — Discord, WhatsApp, email, etc. Those aren't the system, they're projections of it. Everything needing me lands in one inbox; the channels are windows onto it. Reply from WhatsApp, the chief of staff agent wakes up and delegates work.
• Voice in and out — half my thinking happens away from a keyboard, and I don't want it all to just turn into reminders.
• My vault underneath all of it — plain markdown, in git, on my disk.

What won't change

Combining a knowledge system that holds information about you, your goals, projects, plans, tasks and processes, with AI Agents, AI skills, MCP servers, autonomy, scheduling and memory, is HUGE.

But don't misunderstand me. This is not about LLM Wikis in Obsidian. Your knowledge system is NOT just a place for AI to store and retrieve memories. First and foremost, it must be a space for you to think.

Writing is thinking. If only AI does the writing, you're missing the most important part. Let AI use your knowledge system. Let it automate the boring parts. But as Andrej Karpathy said, don't outsource understanding.

My motto is: AI does the plumbing; you do the thinking!

What I'd recommend

If you want to truly leverage AI:

• Build a knowledge system for yourself. AI will benefit from it too.
• Learn how to build proper AI skills, how to use MCP servers and CLIs.
• Deploy an autonomous agent and give it access to your knowledge base (even just a subset of it).

You could skip the knowledge system, or ask AI to handle it all for you. But then you'd be missing the most valuable part of it all.

Don't.

Join the Knowii Community to learn more: https://store.dsebastien.net/product/knowii-community

References

• Original thread: https://x.com/dSebastien/status/2062041842724872433

Related

• OpenClaw
• Obsidian
• Obsidian Starter Kit
• Knowii Community
• AI Agents
• AI Agent Skills
• AI Agent Orchestration
• AI Agent Portability
• Agentic Knowledge Management (AKM)
• Personal Knowledge Management (PKM)
• Model Context Protocol (MCP)
• AGENTS.md (File Convention)
• Claude Code
• Writing is thinking
• Andrej Karpathy
• Speech-to-Text (STT)
• Text-to-Speech (TTS)

About Sébastien

Ready to get to the next level?