Skip to content

Commit

Permalink
Merge pull request andrewjfreyer#176 from andrewjfreyer/beta
Browse files Browse the repository at this point in the history 
0.2.187
  • Loading branch information
@andrewjfreyer
andrewjfreyer committed Apr 30, 2019
2 parents dbd5457 + ee76fee commit b95e856
Show file tree
Hide file tree
Showing 7 changed files with 455 additions and 224 deletions.
300 changes: 165 additions & 135 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,9 @@

![version](https://img.shields.io/badge/version-0.2-green.svg?maxAge=2592000) ![mosquitto](https://img.shields.io/badge/mosquitto-1.5+-blue.svg?maxAge=2592000)

____
[**Frequently Asked Questions**](https://github.com/andrewjfreyer/monitor/blob/master/support/README.md)

### *Table of Contents*

* [**Highlights**](#highlights)

* [**Oversimplified Analogy of the Bluetooth Presence Problem**](#oversimplified-analogy-of-the-Bluetooth-presence-problem)

* [**Oversimplified Technical Description**](#oversimplified-technical-description)

* [**How `monitor` Works**](#how-monitor-works)

* [**Example with Home Assistant**](#example-with-home-assistant)

* [**Installing on a Raspberry Pi Zero W**](#installation-instructions-for-raspberry-pi-zero-w)

____
<details><summary><b>Background & Technical Details</b></summary>

# *Highlights*

Expand Down Expand Up @@ -164,122 +150,14 @@ Since iBeacons include a UUID and a mac address, two presence messages are repor
In some cases, certain manufacturers try to get sneaky and cause their beacons to advertise as "anonymous" (or "random") devices, despite that their addresses do not change at all. By default, `monitor` ignores anonymous devices, so to force `monitor` to recognize these devices, we add the "random" address to a file called `known_static_beacons`. After restarting, `monitor` will know that these addresses should be treated like a normal beacon.
___

# Example with Home Assistant

Personally, I have four **raspberry pi zero w**s throughout the house and garage. My family spends most of our time on the first floor, so our main `monitor` node or sensor is on the first floor. Our other 'nodes' on the second and third floor and garage are set up for triggered use only - these will scan for ***ARRIVAL*** and ***DEPART*** only in response to mqtt messages, with option ```-tad```. The first floor node is set up to send mqtt arrive/depart scan instructions to these nodes by including the `-tr` flag ("report" to other nodes when an arrival or depart scan is triggered).

The first floor constantly monitors for beacons (`-b`) advertisements and anonymous advertisements, which may be sent by our phones listed in the `known_static_addresses` file. In response to a new anonymous advertisement, `monitor` will initiate an ***ARRIVAL*** scan for whichever of our phones is not present. If one of those devices is seen, an mqtt message is sent to Home Assistant reporting that the scanned phone is "home" with a confidence of 100%. In addition, an mqtt message is sent to the second and third floor and garage to trigger a scan on those floors as well.

When we leave the house, we use either the front door or the garage door to trigger an mqtt trigger of ```monitor/scan/depart``` after a ten second delay to trigger a departure scan of our devices that were previously known to be present. The ten second delay gives us a chance to get out of Bluetooth range before a "departure" scan is triggered. Different houses/apartments will probably need different delays.

[Home Assistant](https://www.home-assistant.io) receives mqtt messages and stores the values as input to a number of [mqtt sensors](https://www.home-assistant.io/components/sensor.mqtt/). Output from these sensors is combined to give an accurate numerical occupancy confidence.

For example (note that 00:00:00:00:00:00 is an example address - this should be your phone's private, static, Bluetooth address):

```
- platform: mqtt
state_topic: 'monitor/first floor/00:00:00:00:00:00'
value_template: '{{ value_json.confidence }}'
unit_of_measurement: '%'
name: 'First Floor'
- platform: mqtt
state_topic: 'monitor/second floor/00:00:00:00:00:00'
value_template: '{{ value_json.confidence }}'
unit_of_measurement: '%'
name: 'Second Floor'
- platform: mqtt
state_topic: 'monitor/third floor/00:00:00:00:00:00'
value_template: '{{ value_json.confidence }}'
unit_of_measurement: '%'
name: 'Third Floor'
- platform: mqtt
state_topic: 'monitor/garage/00:00:00:00:00:00'
value_template: '{{ value_json.confidence }}'
unit_of_measurement: '%'
name: 'Garage'
```

These sensors can be combined using a [min_max](https://www.home-assistant.io/components/sensor.min_max/):

```
- platform: min_max
name: "Home Occupancy Confidence"
type: max
round_digits: 0
entity_ids:
- sensor.third_floor
- sensor.second_floor
- sensor.first_floor
- sensor.garage
```

Thereafter, I use the entity **sensor.home_occupancy_confidence** in automations to control the state of an **input_boolean** that represents a very high confidence of a user being home or not.

As an example:

```
- alias: Occupancy On
hide_entity: true
trigger:
- platform: numeric_state
entity_id: sensor.home_occupancy_confidence
above: 10
action:
- service: input_boolean.turn_on
data:
entity_id: input_boolean.occupancy
- alias: Occupancy Off
hide_entity: true
trigger:
- platform: numeric_state
entity_id: sensor.home_occupancy_confidence
below: 10
action:
- service: input_boolean.turn_off
data:
entity_id: input_boolean.occupancy
```
</details>

If you prefer to use the `device_tracker` platform in Home Assistant, a unique solution is to use the undocumented `device_tracker.see` service:
<details><summary><b>Installation Instructions</b></summary>

As an example:
<br>

```
- alias: Andrew Occupancy On
hide_entity: true
trigger:
- platform: numeric_state
entity_id: sensor.andrew_occupancy_confidence
above: 10
action:
- service: device_tracker.see
data:
dev_id: andrew
location_name: home
source_type: bluetooth
<details><summary><i>Set Up Raspberry Pi From Scratch</i></summary>

- alias: Andrew Occupancy Off
hide_entity: true
trigger:
- platform: numeric_state
entity_id: sensor.andrew_occupancy_confidence
below: 10
action:
- service: device_tracker.see
data:
dev_id: andrew
location_name: not_home
source_type: bluetooth
```

For more information, see [here](https://community.home-assistant.io/t/device-tracker-from-script/97295/7) and [here](https://github.com/andrewjfreyer/monitor/issues/138).

___

# Installation Instructions for Raspberry Pi Zero W

Expand Down Expand Up @@ -360,10 +238,15 @@ sudo wget http://repo.mosquitto.org/debian/mosquitto-stretch.list
#update caches and install
apt-cache search mosquitto
sudo apt-get update
sudo apt-get install libmosquitto-dev mosquitto mosquitto-clients
sudo apt-get install -f libmosquitto-dev mosquitto mosquitto-clients libmosquitto1
```
</details>

<details><summary><i>Monitor Setup</i></summary>

8. Clone `monitor` git:
## Setup `monitor`

1. Clone `monitor` git:
```bash
#install git
cd ~
Expand All @@ -380,7 +263,7 @@ git checkout beta

```

9. Initial run:
2. Initial run:

Configuration files will be created with default preferences. Any executables that are not installed will be reported. All can be installed via `apt-get install ...`

Expand All @@ -389,26 +272,167 @@ sudo bash monitor.sh
```


10. Edit **mqtt_preferences** file:
3. Edit **mqtt_preferences** file:

```bash
sudo nano mqtt_preferences
```

11. Edit **known_static_addresses** (phones, laptops, some smart watches):
4. Edit **known_static_addresses** (phones, laptops, some smart watches):

```bash
sudo nano known_static_addresses
```

12. Read helpfile:
Alternatively, send an mqtt message to `monitor/setup/ADD STATIC DEVICE` with a message including a mac address and an alias separated by a space:


**topic:** `monitor/setup/ADD STATIC DEVICE`
**message:** 00:11:22:33:44:55 alias


Use, `monitor/setup/DELETE STATIC DEVICE` with a message containing a mac address to remove a device from all `monitor` nodes.

5. Read helpfile:

```bash
sudo bash monitor.sh -h
```

Now the basic setup is complete. Your broker should be receiving messages and the `monitor` service will restart each time the Raspberry Pi boots. As currently configured, you should run `sudo bash monitor.sh` a few times from your command line to get a sense of how the script works.

</details>

___

</details>


<details><summary><b>Home Assistant Example</b></summary>

# Example with Home Assistant

Personally, I have four **raspberry pi zero w**s throughout the house and garage. My family spends most of our time on the first floor, so our main `monitor` node or sensor is on the first floor. Our other 'nodes' on the second and third floor and garage are set up for triggered use only - these will scan for ***ARRIVAL*** and ***DEPART*** only in response to mqtt messages, with option ```-tad```. The first floor node is set up to send mqtt arrive/depart scan instructions to these nodes by including the `-tr` flag ("report" to other nodes when an arrival or depart scan is triggered).

The first floor constantly monitors for beacons (`-b`) advertisements and anonymous advertisements, which may be sent by our phones listed in the `known_static_addresses` file. In response to a new anonymous advertisement, `monitor` will initiate an ***ARRIVAL*** scan for whichever of our phones is not present. If one of those devices is seen, an mqtt message is sent to Home Assistant reporting that the scanned phone is "home" with a confidence of 100%. In addition, an mqtt message is sent to the second and third floor and garage to trigger a scan on those floors as well. As a result of this configuration, when we leave the house, we use either the front door or the garage door to trigger an mqtt trigger of ```monitor/scan/depart``` after a ten second delay to trigger a departure scan of our devices that were previously known to be present. The ten second delay gives us a chance to get out of Bluetooth range before a "departure" scan is triggered. Different houses/apartments will probably need different delays.

More specifically, each of these `monitor` nodes uses the same name for each device so that states can be tracked easily by Home Assistant. For example, on each node, my `known_static_addresses` file looks like this (note that 00:00:00:00:00:00 is an example address - this should be your phone's private, static, Bluetooth address):

```bash
00:00:00:00:00:00 alias #comment that is ignored
```

The address I want to track is separated by a space from the *alias* that I want to use to refer to this device in Home Assistant. If you prefer to use the address instead of an alias, set the value `PREF_ALIAS_MODE=false` in your `behavior_preferences` file.

In this manner, [Home Assistant](https://www.home-assistant.io) receives mqtt messages and stores the values as input to a number of [mqtt sensors](https://www.home-assistant.io/components/sensor.mqtt/). Output from these sensors is combined to give an accurate numerical occupancy confidence:

```
- platform: mqtt
state_topic: 'monitor/first floor/alias'
value_template: '{{ value_json.confidence }}'
unit_of_measurement: '%'
name: 'First Floor'
- platform: mqtt
state_topic: 'monitor/second floor/alias'
value_template: '{{ value_json.confidence }}'
unit_of_measurement: '%'
name: 'Second Floor'
- platform: mqtt
state_topic: 'monitor/third floor/alias'
value_template: '{{ value_json.confidence }}'
unit_of_measurement: '%'
name: 'Third Floor'
- platform: mqtt
state_topic: 'monitor/garage/alias'
value_template: '{{ value_json.confidence }}'
unit_of_measurement: '%'
name: 'Garage'
```

These sensors can be combined using a [min_max](https://www.home-assistant.io/components/sensor.min_max/):

```
- platform: min_max
name: "Home Occupancy Confidence"
type: max
round_digits: 0
entity_ids:
- sensor.third_floor
- sensor.second_floor
- sensor.first_floor
- sensor.garage
```

Thereafter, I use the entity **sensor.home_occupancy_confidence** in automations to control the state of an **input_boolean** that represents a very high confidence of a user being home or not.

As an example:

```
- alias: Occupancy On
hide_entity: true
trigger:
- platform: numeric_state
entity_id: sensor.home_occupancy_confidence
above: 10
action:
- service: input_boolean.turn_on
data:
entity_id: input_boolean.occupancy
- alias: Occupancy Off
hide_entity: true
trigger:
- platform: numeric_state
entity_id: sensor.home_occupancy_confidence
below: 10
action:
- service: input_boolean.turn_off
data:
entity_id: input_boolean.occupancy
```

If you prefer to use the `device_tracker` platform in Home Assistant, a unique solution is to use the undocumented `device_tracker.see` service:

As an example:

```
- alias: Andrew Occupancy On
hide_entity: true
trigger:
- platform: numeric_state
entity_id: sensor.andrew_occupancy_confidence
above: 10
action:
- service: device_tracker.see
data:
dev_id: andrew
location_name: home
source_type: bluetooth
- alias: Andrew Occupancy Off
hide_entity: true
trigger:
- platform: numeric_state
entity_id: sensor.andrew_occupancy_confidence
below: 10
action:
- service: device_tracker.see
data:
dev_id: andrew
location_name: not_home
source_type: bluetooth
```

For more information, see [here](https://community.home-assistant.io/t/device-tracker-from-script/97295/7) and [here](https://github.com/andrewjfreyer/monitor/issues/138).

</detail>

<details><summary>Detailed Info & Fine Tuning</summary>


## Fine Tuning

Expand Down Expand Up @@ -465,6 +489,7 @@ When `monitor` is first run, default preferences are created in the `behavior_pr
| PREF_PASS_FILTER_MANUFACTURER_ARRIVE | .* | See above. |
| PREF_FAIL_FILTER_ADV_FLAGS_ARRIVE | NONE | See above. |
| PREF_FAIL_FILTER_MANUFACTURER_ARRIVE | NONE | See above. |
| PREF_ALIAS_MODE | true | Disable or enable alias mode; if disabled, MQTT messages are sent using a device's mac address. |

3. **Advanced configuration options:**

Expand All @@ -486,11 +511,14 @@ PREF_DEVICE_TRACKER_REPORT|false|If true, this value will cause `monitor` to rep
PREF_DEVICE_TRACKER_HOME_STRING|home|If `PREF_DEVICE_TRACKER_REPORT` is true, this is the string that is reported to the device_tracker when the device is home.
PREF_DEVICE_TRACKER_AWAY_STRING|not_home|If `PREF_DEVICE_TRACKER_REPORT` is true, this is the string that is reported to the device_tracker when the device is not home.
PREF_DEVICE_TRACKER_TOPIC_BRANCH|device_tracker|If `PREF_DEVICE_TRACKER_REPORT` is true, this is last path element of the mqtt topic path that will be used to publish the device tracker message.
PREF_ADVERTISEMENT_OBSERVED_INTERVAL_STEP|This is the minimum interval used to estimate advertisement intervals reported in the MQTT message. Default is 15 seconds.
PREF_DEPART_SCAN_INTERVAL|If using periodic scanning mode, this is the minimum interval at which depart scans are triggered automatically.
PREF_ARRIVE_SCAN_INTERVAL|If using periodic scanning mode, this is the minimum interval at which arrive scans are triggered automatically.


## RSSI Tracking

This script can also track RSSI changes throughout the day. This can be useful for very rudimentary room-level tracking. Only devices in `known_static_addresses` that have been paired to a `monitor` node can have their RSSI tracked. Here's how to pair:
This script can also track RSSI changes throughout the day. This can be used for very rudimentary room- or floor-level tracking. Only devices in `known_static_addresses` that have been paired to a `monitor` node can have their RSSI tracked. Here's how to pair:

1. Stop `monitor` service:

Expand Down Expand Up @@ -540,4 +568,6 @@ sensor:
precision: 1
```
</details>
Anything else? Post a [question.](https://github.com/andrewjfreyer/monitor/issues/new)
Loading

0 comments on commit b95e856

Please sign in to comment.