Jarvis: a personal AI assistant for tasks, calendar, and people

I live across a few countries, and I kept dropping things: tasks, plans, and worst of all, people. So I built Jarvis, a personal assistant that handles the operational layer of life through plain-language messages. Add a task, check the week, log that I caught up with a friend, turn a flight confirmation into calendar events. Here’s the architecture, and the handful of decisions that separated “fun demo” from “I rely on this daily.”

The core idea: two speeds, one brain

The biggest design insight was that I needed two interfaces, not one:

• A reasoning interface (a full Claude conversation) for anything complex: multi-step planning, reasoning over my notes, judgement calls.
• A fast interface (a Telegram bot) for quick, repeatable actions: “add task X”, “what’s on tomorrow?”, “log that I spoke to Sam.”

The fast path doesn’t need a model to reason. It needs to call an API and get out of the way. So those two paths have different plumbing on purpose.

You
 ├─ Claude conversation   → reasoning, notes, complex tasks
 └─ Telegram bot          → fast actions + proactive nudges
        └─ workflow engine → calls Calendar / database APIs directly

Trying to make one interface do both jobs is the most common mistake. Speed and reasoning pull in opposite directions, so give each its own lane.

The pieces

• Telegram bot: mobile-native, always in my pocket, instant.
• A workflow engine (I self-host n8n): the automation layer that receives messages and calls APIs. Think of it as the wiring between the chat and your data.
• An AI agent inside the workflow: a Claude model with a set of tools (get tasks, add task, create event, log contact, and so on) and short-term memory. No rigid command parsing. You write naturally, it picks the right tool.
• A database for state: I use Notion (two databases, a to-do list and a lightweight relationship CRM). Plain, queryable, and editable by hand when needed.

Why an agent beats a command parser

My first version used keyword routing: “if the message starts with add task”. It was brittle, and every new phrasing broke it. Replacing it with a proper AI agent that has tools changed everything. Now any phrasing works, in any casing, including messy multi-step requests like “move my dentist thing to next week and remind me to call the insurer.” The agent figures out which tools to call and in what order.

The lesson: don’t parse intent yourself when a model can. Give it well-described tools and let it route.

The feature that earns its keep: relationship nudges

The to-do and calendar features are table stakes. The one I’d never give up is the relationship CRM. Each important contact has a target cadence (say, “every 3 months”). A scheduled job each morning checks who’s overdue and nudges me on Telegram. “Log that I spoke to X today” resets the clock.

It sounds cold to systematise friendship. In practice it’s the opposite: it means distance and busyness stop quietly eroding relationships I care about. The system doesn’t replace warmth. It just makes sure I never lose someone to inertia.

Three trigger types

Jarvis listens three ways:

1. Telegram message: the main path.
2. A scheduled job: a morning brief (today’s tasks, calendar, overdue contacts, a quote) and an evening check-in.
3. An email label: I tag any email with a label and the agent processes it. Forward a flight or hotel confirmation and it parses the details and creates the calendar events automatically.

That third one is deceptively useful. Booking confirmations are structured enough that an agent turns them into clean calendar entries with zero effort.

Giving it memory

Out of the box, the agent forgets everything between sessions. I added two layers:

• Short-term memory: the last several messages, so a conversation has continuity.
• Long-term memory: a small distilled profile (my preferences, recurring context) stored in a database. A weekly job distils recent interactions into the long-term store and prunes old noise.

Memory is what makes it feel like an assistant rather than a stateless function. But keep it small and curated. Dumping everything in degrades answers. Distil ruthlessly.

The unglamorous lessons (where the real time went)

Most of the work wasn’t the AI. It was reliability:

• Persistence bites. My first deploy mounted the data volume to the wrong path, and a restart silently wiped everything. Check that your state actually survives a reboot before you trust it.
• Monitor it. I run an uptime check and an infra alarm. When the server hiccuped during maintenance, I knew in minutes instead of discovering it via a missing morning brief.
• Empty results are a trap. Several silent failures came from a query returning an empty list and downstream steps quietly skipping. Handle the empty case explicitly.
• Soft-delete, don’t delete. “Delete task” sets status to Cancelled rather than removing the row. You keep an audit trail and avoid irreversible mistakes.
• Keep secrets out of code. Every API call uses stored credentials, never keys pasted into a node. Non-negotiable, and it’s what makes a setup safe to share.

Was it worth it?

Completely, but be honest about the trade. This is a system, and systems need occasional maintenance. The payoff is compounding: I stopped dropping tasks, my calendar reflects reality, and I genuinely keep up with people I’d otherwise lose touch with. The assistant is good. The forcing function of having one is better.

---

Part of a series on building with AI around a full-time job. A clean, anonymised starter version of this stack will be linked here once it’s ready to share. Previous: building a second brain.