From a footstep to a kind word, in five stages.
Happy Steps splits the work in two. Signal processing stays inside the home, on a small device. Only a short summary goes to Gemini, which reasons, decides and speaks.
信号処理は家の中で。Geminiには短いまとめだけを送ります。
The dashed line is the home boundary. Raw vibration never crosses it.
Sense the floor
床の振動を感じるA geophone lies on the floor. When someone walks, each heel strike makes the floor move by about a millionth of a metre per second. The geophone turns that motion into a voltage, which we sample at 2–8 kHz.
Why the floor?
A floor carries footsteps for several metres, so one sensor can cover a room. It sees nothing and hears no speech. It only feels force on the floor.
Understand each step, at the edge
家の中で一歩ずつ読み取るA Raspberry Pi isolates each footstep from background vibration, turns it into a time–frequency picture with a continuous wavelet transform, and runs DeepStep, a 22 MB model, to answer two questions: who is walking, and how.
What it measures
- Identity: the resident, or someone new (open-set)
- Pace: steps per minute
- Steadiness: how even the step timing is
- Balance of rhythm: left–right timing symmetry
Summarise to a tiny event
数字だけのまとめにするWhen a walk ends, the device writes one small JSON event. Every home, floor and sensor uses the same format, so one home and one thousand homes look the same to the cloud.
{
"schema": "footstep-event/v1",
"home_id": "H001",
"time": "2026-11-01T10:00:42+09:00",
"duration_s": 8.3,
"person": "resident_01",
"id_similarity": 0.97,
"steps": 17,
"cadence_spm": 114,
"step_interval_cv": 0.041,
"temporal_symmetry": 0.97,
"event_type": "walk",
"raw_signal_included": false
}Reason with Gemini
Geminiが「いつも」と比べるGemini receives today’s events and the resident’s 14-day baseline. A system instruction keeps it kind and careful: never diagnose, treat unknown people as visitors, suggest simple human actions. Function calling is set to ANY, so Gemini must choose one action.
{
"systemInstruction": { … },
"contents": [{ "parts": [{ "text": "today + baseline JSON" }] }],
"tools": [{ "functionDeclarations": [
{ "name": "send_family_report", … },
{ "name": "raise_alert", … }
]}],
"toolConfig": {
"functionCallingConfig": { "mode": "ANY" }
}
}Speak and deliver
家族へ届け、本人に語りかけるThe function call goes to the family through any channel: a messaging app, email, or a care-home dashboard. The message for the resident is spoken at home with Gemini text-to-speech, in a warm Japanese voice.
Two calls, any app
send_family_report for usual days and small changes. raise_alert when someone should check soon. Integrators only need to handle these two calls.
Designed to grow from one home to a city.
One open format
Any sensor that can write footstep-event/v1 can join: geophones, accelerometers, smart flooring.
Cheap at the edge
A sensor and a Raspberry Pi per room. No GPU, no streaming, very little bandwidth.
Light in the cloud
One Gemini call per home per day, plus rare alerts. Cost grows with homes, not with hours of signal.
See a real footstep become a kind word.
The live demo runs real floor recordings through the full pipeline and calls Gemini in your browser.