# A field report on separating command transmission, software-known state, and physical verification across calendar, voice, and household-device workflows.
The goal for Jarvis is simple to describe: speak at home, have the system read the calendar, operate a device, and send a notification when needed.
That description makes a home agent sound like a contest in conversational intelligence.
Building one taught us the opposite lesson.
The difficult part is reporting exactly what was verified after a command leaves the conversation.
This article was updated on July 24, 2026 after checking configuration, the day's job status, and responses from running APIs.
Connection details, credentials, and personal information are intentionally omitted.
What Jarvis connects
Jarvis is not a particular smart-speaker product here.
It is the name of a home agent built around Hermes Agent that connects conversation, schedules, audio, and household devices.
Its job has four parts.
Interpret whether a request means research, notification, or physical action.
Retrieve Google Calendar events and turn them into a spoken briefing.
Route actions to separate paths for speakers, television, air conditioning, and garden equipment.
Report failure or missing verification instead of hiding it.
Prototype control boards and wiring; the display has been masked for privacy
Photo: prototype control hardware with the display masked for privacy. The exposed boards and wiring have not been evaluated for enclosure safety, strain relief, or unattended permanent installation. This image alone does not verify any connection target or physical device action.
The daily 8:00 JST job retrieves the day's Google Calendar events and registers the spoken briefing through POST /alarms.
Its payload requests execution one second later and includes the spoken text and output: announcement; the script only exits successfully when the response includes an alarm ID.
home-worker sends each due item to playback and deletes it only after that playback path returns success.
On July 24, 2026, the scheduler recorded the job as completed successfully at 08:02 JST.
The morning item no longer appeared in the alarm list afterward, so calendar retrieval, API registration, and API-level consumption were verified.
Physical speaker playback was not independently verified.
“A morning briefing is configured” and “the speaker played it this morning” are different facts.
That distinction is the foundation for every automation in this series.
Completion has four levels
Calling every device action a success hides important uncertainty.
Jarvis therefore treats completion as a ladder.
Level
What it establishes
What remains unknown
Request accepted
The conversation or button input was understood
Whether it reached the execution path
Command sent
The API or infrared sender responded
Whether the device received it
Known state changed
Software updated its stored state
Whether the room or appliance physically changed
Physical result observed
A sensor or independent observation detected the change
Whether it will remain reliable over time
This model makes an honest response possible: “The command was sent; the physical result is not yet verified.”
It is less polished than a confident completion message, but much safer in a real home.
Keep voice paths separate
One voice path combines speech recognition and text-to-speech.
Another handles a direct real-time speech-to-speech session.
Running both for the same conversation can produce duplicate replies or let the wrong audio engine interrupt.
The normal voice path and real-time path therefore remain separate.
An alarm or calendar briefing is also not complete merely because text reached Discord.
If the requirement is audible playback, speaker playback is part of the job.
What a false intercom alert taught us
There was one report of an intercom notification appearing even though the bell had not rung.
The cause remains unconfirmed.
The dangerous response would be to choose a plausible cause and implement a fix as though that cause were established fact.
Sensor bounce, network retries, and stale state are all possible, but the existing record does not select one.
The next occurrence needs a better trace: detection time, input value, decision reason, notification ID, and retry count joined by one correlation ID.
Until false positives are understood, the message should say “signal detected,” not “visitor confirmed.”
Build the stop path before expanding control
Jarvis can reach functions with very different risk profiles: volume changes, infrared air-conditioner commands, a watering pump, and a bath switch.
They should not share one permission level or one definition of success.
Change volume in small steps.
Send complete air-conditioner state signals and rate-limit them.
Give a pump a maximum runtime and a manual stop.
Do not retry a bath toggle while its state is unknown.
When a sensor or execution path is uncertain, default to no action.
For home automation, stopping conditions matter more than the ability to start.
Automation that cannot be stopped is not convenience; it is a new failure mode.
An honest snapshot of the system
As of July 24, 2026, we confirmed the daily 8:00 JST Google Calendar job and that day's run, spoken-briefing registration through POST /alarms, and consumption of the due item after execution.
Both garden controllers responded to their own /status endpoints, and the air conditioner's software-known state was cool_18.
The live API exposes POST /bath/switch/toggle, but it has no bath state endpoint.
Physical speaker playback, infrared reception and room-temperature change, physical bath-switch contact and bath-fill start or completion, actual watering, and long-term operation remain unverified.
Closing that gap is not mainly an AI-model problem.
Each device needs independent observation, commands and outcomes need a shared trace, and Jarvis must preserve the word “unverified” when that is the truth.
A fluent agent that says “done” is less trustworthy than one that can say, “The command was sent, but I cannot yet confirm the physical result.”
