AI

Local AI camera sensors with VisionState

Turn a camera frame into a Home Assistant sensor (garage door open or closed, car in the driveway, a meter reading) with models that run on your own CPU. No cloud, no GPU.

Some hardware links on this page are affiliate links. As an Amazon Associate I earn from qualifying purchases. We may earn a commission if you buy through them, at no extra cost to you. See the affiliate disclosure.

A garage door tilt sensor is one more battery to change. A camera already looks at the door. VisionState is an open-source Home Assistant app (add-on) that turns what a camera sees into ordinary sensors. It runs small AI models on the Home Assistant host’s own CPU. Nothing leaves the box, and you don’t need a GPU or a subscription.

What it can make

  • States. Garage door open, closed or halfway. A gate shut. A light on in the shed. You click a few example frames of each state and it learns them in about a second.
  • Objects. A car in the driveway, a person at the door, a cat on the lawn. It knows 80 common objects without any training.
  • Readings. The digits on a water, gas or power meter, which can go into the Energy dashboard.

Each sensor shows up as a normal device through MQTT, so you use it in automations like any other sensor.

What you need

  • Home Assistant OS or Supervised, 2025.10 or newer. It is an app, so a plain container install won’t run it.
  • The Mosquitto broker app and the MQTT integration. VisionState publishes its sensors through MQTT discovery.
  • A camera. Any camera.* entity in Home Assistant (Reolink, an ESP32-CAM on ESPHome, Frigate and others), or an RTSP or HTTP snapshot URL.
  • CPU. It runs on amd64 and aarch64, including a Raspberry Pi 4 or 5. A state sensor that checks now and then is light. Many object sensors that check every few seconds are not.

Install

  1. Take a full backup first. Settings → System → Backups.
  2. Install and start Mosquitto broker, then add the MQTT integration if Home Assistant doesn’t find it on its own.
  3. Go to Settings → Apps → App store → ⋮ → Repositories and add https://github.com/oleost/VisionState.
  4. Install VisionState, start it, and open it from the sidebar.

Your first sensor: garage door open or closed

  1. New sensor, then pick the camera.
  2. Draw a region around the door only. Leave out the driveway, the sky and anything else that changes. A tight region is the biggest accuracy gain you can make.
  3. Pick State and name the states: open, closed, and halfway if you want it.
  4. Give it examples. Open and close the door a few times and label frames for each state. Include night frames and frames with the light on. A door that only learned daylight will guess at night.
  5. The wizard tests the sensor on a fresh frame right away.

You now have a sensor such as sensor.garage_door with the state and a confidence value, plus a button to check now and a switch to pause it.

Check it against something you trust

Before an automation depends on it, run it for a week next to a sensor you already trust, like the opener’s own state or a tilt sensor. Make a history graph with both. Every time they disagree, label that frame in VisionState. Unsure frames also land in its review queue. Clear it now and then, because the sensor gets better each time.

Use it

Notify when the door has been open for 10 minutes:

triggers:
  - trigger: state
    entity_id: sensor.garage_door
    to: open
    for: "00:10:00"
actions:
  - action: notify.notify
    data:
      message: Garage door has been open for 10 minutes.

To save CPU, don’t let every sensor check constantly. VisionState can check when something happens instead: on motion, on a door contact, or when the image changes. It can also turn on a light for a camera that is dark at night.

Reading a meter with an ESP32-CAM

Mount an ESP32-CAM above the meter, add it to Home Assistant with ESPHome, and draw a box around the digits. VisionState reads the rolling wheels of a mechanical meter or the digits of an LCD, and checks every value before it publishes it (a counter never goes down or jumps). Light the meter evenly. A shadow across the digits is the most common cause of bad readings.

Privacy

  • Models run on the Home Assistant host. There is no telemetry.
  • The VisionState UI is only reachable through Home Assistant (Ingress), so it needs a Home Assistant login.
  • Face recognition is off until you turn it on for a camera. If you use it, the faces stay in the app’s own storage. Think about who walks past that camera before you turn it on.

Don’t share frames from your own cameras, or say which camera covers which door, when you ask for help online. A frame of a garage tells a stranger more than you think.