Yes, it's "out of the door" at last. After some final "stress" testing and the creation / testing of 32 (yes, thirty-two) example programs, v2.0 finally goes public.
Intervening health problems added 8 months to the date(!) but at last I can now relax somewhat- with a well-earned beer.
Get Esparto v2.0 here at github
Showing posts with label Chez Toi. Show all posts
Showing posts with label Chez Toi. Show all posts
Monday, 27 August 2018
Wednesday, 1 November 2017
The "truc" Home Automation "ecosystem": The NODE-RED controller
Did I already mention that I love NODE-RED ? I don't think I did, so: I love NODE-RED. If you haven't come across it before, google it now and then get it, install and run it and enjoy it.
I use it to centralise control of my many IOT devices and provide a "global" view of the system with a nice UI. It also provides my "update server" so that all devices automatically upgrade themselves when I change the firmware. In future it may tweet me when odd things happen and send facebook posts showing weekly summaries so I can see how my home is doing while on holiday...on second thoughts, I'm not sure I'm quite that sad yet!
One of its main benefits is that by putting all the "wiring" between the devices in one place on a raspberry Pi, the code in each device can be a lot smaller. It also allows me to "link" devices so that - for example - if one device in a zone goes on, they all do. I can also send MQTT control messages at the press of a button such as "tell all Sonoff s20's to upgrade now" or "tell all floor lamps to turn on".
I'm going to go into a lot more detail about how it does those things at a later date, once Esparto v2.0 has been released, but for now for the sake of completeness, here's a few snapshots:
1) The main "estate view"
3) The internal "routing" flow (one of five NODE-RED "flow"s)
I use it to centralise control of my many IOT devices and provide a "global" view of the system with a nice UI. It also provides my "update server" so that all devices automatically upgrade themselves when I change the firmware. In future it may tweet me when odd things happen and send facebook posts showing weekly summaries so I can see how my home is doing while on holiday...on second thoughts, I'm not sure I'm quite that sad yet!
One of its main benefits is that by putting all the "wiring" between the devices in one place on a raspberry Pi, the code in each device can be a lot smaller. It also allows me to "link" devices so that - for example - if one device in a zone goes on, they all do. I can also send MQTT control messages at the press of a button such as "tell all Sonoff s20's to upgrade now" or "tell all floor lamps to turn on".
I'm going to go into a lot more detail about how it does those things at a later date, once Esparto v2.0 has been released, but for now for the sake of completeness, here's a few snapshots:
1) The main "estate view"
2) The "Device" view
Esparto v2.0 - sneak preview: Outside
"Esparto" is a type of coarse grass. That's neither here nor there really but it's a nice name for this Arduino ESP8266 library. Originally it stood for ESP All-purpose Run-Time Object, but it could just as well be "ESP All Ready To Operate" because that's what it is.
It is a C++ class object which does almost everything you need to do to turn your Wemos D1, NodeMCU, ESP-01S or Sonoff device from a pretty brick into an MQTT-controlled IOT device. Some folk would call it "firmware" and indeed you can upload a binary if you choose, but the real power comes when you extend it yourself with your own code.
As it stands, it will connect to your WiFi and MQTT controller and automatically reconnect after they fail - without rebooting. It will give you a visual (near*) real-time view of all the GPIO pins, with the correct labels on each depending on the device its deployed on. It will accept OTA updates and runs a web server that allows simple reconfiguration.
Using MQTT you can switch it on of off, reboot it (though you shouldn't need to) or factory-reset it. You can even dynamically change its deviceID and SSID+Password. You can read or write any IO pin and/or reconfigure them. What that last one means exactly will become clear later. Through all of this the basic on/off functionality will continue to work, even if your MQTT controller or complete local network are down.You don't need someone else's cloud and a flaky app on your iPhone. Anything that runs a web browser inside your network can control it. So even without your own additional code, it already does a lot.
Once you start adding your own code, the world of Home Automation and IOT is your oyster. In the example that follows, the device is running mu own Chez Toi ioT "truc" firmware, which provides a common interface for a number of sensors (PIR, light, sound, temperature) adds RF433MHz "dumb" switch control and stores the location and function of the device so that the NODE-RED controller can present a smooth UI for the overall management of the whole system.
Basic Esparto:
It is a C++ class object which does almost everything you need to do to turn your Wemos D1, NodeMCU, ESP-01S or Sonoff device from a pretty brick into an MQTT-controlled IOT device. Some folk would call it "firmware" and indeed you can upload a binary if you choose, but the real power comes when you extend it yourself with your own code.
As it stands, it will connect to your WiFi and MQTT controller and automatically reconnect after they fail - without rebooting. It will give you a visual (near*) real-time view of all the GPIO pins, with the correct labels on each depending on the device its deployed on. It will accept OTA updates and runs a web server that allows simple reconfiguration.
Using MQTT you can switch it on of off, reboot it (though you shouldn't need to) or factory-reset it. You can even dynamically change its deviceID and SSID+Password. You can read or write any IO pin and/or reconfigure them. What that last one means exactly will become clear later. Through all of this the basic on/off functionality will continue to work, even if your MQTT controller or complete local network are down.You don't need someone else's cloud and a flaky app on your iPhone. Anything that runs a web browser inside your network can control it. So even without your own additional code, it already does a lot.
Once you start adding your own code, the world of Home Automation and IOT is your oyster. In the example that follows, the device is running mu own Chez Toi ioT "truc" firmware, which provides a common interface for a number of sensors (PIR, light, sound, temperature) adds RF433MHz "dumb" switch control and stores the location and function of the device so that the NODE-RED controller can present a smooth UI for the overall management of the whole system.
Basic Esparto:
From the top: Device name, SSID+password change. No reboot, it just reconnects to the new settings. You will need to browse to another "devicename.local" page of course if you change the name or another IP page if the new SSID has a different range.
We'll come back to the "settings" cog icon in a moment, but the yellow circular arrow reboots the device and the factory icon resets it to as close to its original state as possible.
The next panel shows the current state of all the GPIOs, each labelled appropriately for the type of hardware its running on. The top row is the "raw" state of the pin, i.e. its instantaneous value, irrespective of what is connected to it. If it is configured as an output (e.g. the LED or RELAY in a Sonoff device) clicking the top row pin will toggle the state and switch the GPIO on or off. The bottom row will make more sense in the next article when we look "inside" at the code, but for now, just know that these are what Esparto calls the "cooked" state.
Esparto is based on the SmartPins library, which is itself based on the H4 timer / scheduler library. (See the side panel links to these for full documentation - all the functionality will be available in Esparto V2.0, plus a lot more of course. Or see http://github.com/philbowles/smartpins and http://github.com/philbowles/h4) . I'll also cover in the next post why those are both "good things" and will save you months of pain. SmartPins defines number of useful input types such as "debounced", "latching", "encoder" and several more. It does all the work to manage the pin, keep those lights updated and call your code back when the pin changes. So a "raw" button which is bouncy will flicker on and off as it bounces, but once SmartPins has done all the hard work and debounced the pin it puts the corresponding bottom line "cooked" pin on and calls your code.
The next panel shows some vaguely-useful static information and the final panel shows dynamic information - including MQTT connection status - which is useful when you start adding your own code.
Back to the "setting" gear wheel. Out-of-the-box it will give you a "404" error. That is because it links to a "config" page that you add for the extra functionality that your code provides. Esparto has been specifically designed to make this easy for you, and we'll see just how much so in the next article. For the moment let's look at what Chez Toi ioT "truc" firmware adds:
Chez Toi needs to know where each device lives, to "wire together" important sequences, e.g. "Going to bed" turns on the bedroom light and the landing light and maybe later, the heater if the temperature is below 14degrees. Much of that added "intelligence" is in the raspberry Pi NODE-RED controller in order to keep the code "footprint" in each device as small as possible.
By knowing the location and function, I can add MQTT commands to turn on all devices in a zone, or all floor lamps in the whole system. "Restart" determines if the device is on , off or whatever it was before in the (rare) case of reboot or power outage. A safety nightlight for example would be (as before) but generally "off" is safest.
The numbers (all in milliseconds) are the configuration values for the sensors. PIR(t) is the PIR timeout and PIR(h) is its "hysteresis", meaning the "dead time" between two movement events to prevent it flicking on and off too rapidly. Sound is the timeout value for the re-triggering sound sensor, and light and temp are the frequency to poll those two sensors, since they change slowly.
This particular "truc" is also an RF master controller. When it switches on, those three 433MHz switches will also go on. Or off, of course. Clicking on any will toggle its current state to allow degree of "manual" control. No more searching for the handset - any browser-capable device will do, tablet, phone, PC etc.
Testbed 5 - The "business end"
The whole point about most IOT devices is that the switch on a mains-powered appliance. In the real world (and my "Chez Toi ioT" home automation system) this will be done by commercially available Sonoff devices. The main ones are the S20 sockets, but I'm also going to deploy a few "Basic" modules in places where I'm never likely to change appliances (dusty corners, overhead lights atop beams, for example) and the low-voltage (or "safe voltage" as ITEAD call it) SV model.
The SV will be powered from a 12v battery and kept "topped up" by a solar powered controller. These units will be set up on buildings away from the main house, driving PIR lights so I don't have to go running for a half-dead torch every time I hear wild boar ripping up my lawns. I'd also like to be able to switch them on remotely before going out as there is a lot of ground to cover in the dark in some parts of the "estate" before the sensors will pick me up.
So the "business end" of all my devices (apart from the wall switches) is a power relay and so my testbed has to have one. I have chosen the ubiquitous (on ebay at least) Omron G3MB-202P which is only rated at 2A, but I'm only going to be testing it with a single low-wattage lamp plugged in: all I need is to see it go on or off at the right time.
Now then, here's the thing with 3.3v devices: the solid-state G3MB needs 5v to switch cleanly and it also "draws" way more current than a GPIO pin can give or take for long periods, i.e. it needs about 20mA. So a "driver" is essential. A single 2N2222 is wired with a 10k base resistor into the GPIO. The "top end" of the G3MB relay is wired to 5v and through the LED to the collector of the 2222. The "bottom end" is wired - with the 2222 emitter to GND. It's about as simple as you can get, providing both the necessary level conversion and sufficient drive current.
Now we are almost ready to go...
The SV will be powered from a 12v battery and kept "topped up" by a solar powered controller. These units will be set up on buildings away from the main house, driving PIR lights so I don't have to go running for a half-dead torch every time I hear wild boar ripping up my lawns. I'd also like to be able to switch them on remotely before going out as there is a lot of ground to cover in the dark in some parts of the "estate" before the sensors will pick me up.
So the "business end" of all my devices (apart from the wall switches) is a power relay and so my testbed has to have one. I have chosen the ubiquitous (on ebay at least) Omron G3MB-202P which is only rated at 2A, but I'm only going to be testing it with a single low-wattage lamp plugged in: all I need is to see it go on or off at the right time.
Now then, here's the thing with 3.3v devices: the solid-state G3MB needs 5v to switch cleanly and it also "draws" way more current than a GPIO pin can give or take for long periods, i.e. it needs about 20mA. So a "driver" is essential. A single 2N2222 is wired with a 10k base resistor into the GPIO. The "top end" of the G3MB relay is wired to 5v and through the LED to the collector of the 2222. The "bottom end" is wired - with the 2222 emitter to GND. It's about as simple as you can get, providing both the necessary level conversion and sufficient drive current.
Now we are almost ready to go...
Testbed 4 - more sub-assemblies: temp and rf433
This one has the (very "bouncy") tact button that all my IOT devices have on GPIO0
It also has the TMP36 temperature sensor and the output module to transmit RF codes to my "el cheapo" sockets: These are what I bought for the 300+ year old farmhouse before I had the (insane) idea to replace everything with a fully-fledged WiFi IOT Home automation system.
Being one who doesn't have a lot of cash to splash, I wondered if I could re-use them...Then I found the wonderful rcswitch library https://github.com/sui77/rc-switch and I found some ridiculously cheap RX/TX pairs on ebay (buy similar here) 5 sets for eu2.79 or 56c a pair. So the TX you see here set me back - what - 28c? I love cheap.
Setting it up was easy: The rcswitch lib has an example that uses the RX to read the codes of the handsets. I was short of Wemos D1s at the time, so I dragged a dusty old Arduino UNO out of a drawer and had the code running in about five minutes flat. I captured all the codes from my four handsets for on and off. I then knocked a little transmit library for the Wemos with a few "hard-coded" values and no-one was more surprised than me to hear the happy sound of clicking less than an hour or so after I started! Tweaking the code, removing the hard-coded values and setting up the automatic re-configuration then occupied me for the next few days, of course...
Now my system uses these as "slaves". Each "truc" or Wemos-controlled "big box of sensors" reads MQTT configuration data at startup to see which RF switches are linked to it. When it next switches on or off, it automatically sends the RF codes to make all its "slaves" match. In the real world, the main box would be wired in with the overhead light in each room, and the el-cheapos would be floor lamps, or corner lamps. When commanded either by touch or remote MQTT, all go on/off at once.
- quick press: switch on / off
- medium: reboot
- long: factory reset
It also has the TMP36 temperature sensor and the output module to transmit RF codes to my "el cheapo" sockets: These are what I bought for the 300+ year old farmhouse before I had the (insane) idea to replace everything with a fully-fledged WiFi IOT Home automation system.
Being one who doesn't have a lot of cash to splash, I wondered if I could re-use them...Then I found the wonderful rcswitch library https://github.com/sui77/rc-switch and I found some ridiculously cheap RX/TX pairs on ebay (buy similar here) 5 sets for eu2.79 or 56c a pair. So the TX you see here set me back - what - 28c? I love cheap.
Setting it up was easy: The rcswitch lib has an example that uses the RX to read the codes of the handsets. I was short of Wemos D1s at the time, so I dragged a dusty old Arduino UNO out of a drawer and had the code running in about five minutes flat. I captured all the codes from my four handsets for on and off. I then knocked a little transmit library for the Wemos with a few "hard-coded" values and no-one was more surprised than me to hear the happy sound of clicking less than an hour or so after I started! Tweaking the code, removing the hard-coded values and setting up the automatic re-configuration then occupied me for the next few days, of course...
Now my system uses these as "slaves". Each "truc" or Wemos-controlled "big box of sensors" reads MQTT configuration data at startup to see which RF switches are linked to it. When it next switches on or off, it automatically sends the RF codes to make all its "slaves" match. In the real world, the main box would be wired in with the overhead light in each room, and the el-cheapos would be floor lamps, or corner lamps. When commanded either by touch or remote MQTT, all go on/off at once.
Monday, 30 October 2017
"Chez Toi" IOT - the need and the naming
In France where I live, "toi" means "you" and "Chez toi" means "your home". Its handy that "toi" is IOT backwards and so I jumped for the obvious name of my still-under-construction home automation system to be "Chez Toi ioT" or sometimes when I'm lazy - which is quite often - "ToiioT" for short.
Also, a "thing" in French when used to mean doodad / widget / gadget / thingamajig is a "truc"hence all the "things" in my Internet of things are called truc1, truc42, truc666 etc. Just so's you know.
The plan was to use the smallest devices possible for the various components, hence I use ESP-01S (the extra "S" is important, you'll learn why later) for wall switches, ITEAD Sonoff devices for the sockets and a few hard-to-get-at places and "homebrew" Wemos D1s for my "big box of sensors" devices which monitor daylight, movement, sound and control the el-cheap RF433 sockets I owned before I started building the system.
The overall aim is to make life easier in a house + barns that was built when candles were the only light source and provide controllable and informative outdoor devices (webcam, PIR lights) for when visitors arrive or wild boar rampage through my garden.
The main design goal was robustness. The Internet here is flaky beyond belief. I can only get a half-decent speed via a satellite dish. I don't want the lights / power to become inoperative every time there is a storm and the Internet goes down (which it invariably does). Secondly I want practicability / ease of maintenance, so all devices have to be upgradeable remotely, via "OTA" (over-the-air): Once I have a solar-powered Sonoff SV-controlled PIR light device 35feet up on the roof of "the big barn", I don't want to have to climb a ladder to reboot it!
Putting all that together, I realised I might need a lot of different bits of code for all the different devices / functions, so I took a decision early on that all the code must run on all the devices, so I could mix-n-match during the early design and experimentation stages.
After a steep learning curve - and several changes of heart - the system fell slowly into place. It started with a monolithic piece of code called "truc". It developed into a series of external libraries which break the functionality down into manageable sizes chunks and allow others to benefit from the learning curve by encapsulating all the difficult bits. (See the right hand links to github where these are published for anybody to use)
"Truc" became "Esparto" which I will talk about in a later post but is essentially all the general-purpose bits of the system that are needed by anybody doing this. All my own specific bits are a "layer" on top of that, still called "truc", but much, much smaller and lighter.
Also, a "thing" in French when used to mean doodad / widget / gadget / thingamajig is a "truc"hence all the "things" in my Internet of things are called truc1, truc42, truc666 etc. Just so's you know.
The plan was to use the smallest devices possible for the various components, hence I use ESP-01S (the extra "S" is important, you'll learn why later) for wall switches, ITEAD Sonoff devices for the sockets and a few hard-to-get-at places and "homebrew" Wemos D1s for my "big box of sensors" devices which monitor daylight, movement, sound and control the el-cheap RF433 sockets I owned before I started building the system.
The overall aim is to make life easier in a house + barns that was built when candles were the only light source and provide controllable and informative outdoor devices (webcam, PIR lights) for when visitors arrive or wild boar rampage through my garden.
The main design goal was robustness. The Internet here is flaky beyond belief. I can only get a half-decent speed via a satellite dish. I don't want the lights / power to become inoperative every time there is a storm and the Internet goes down (which it invariably does). Secondly I want practicability / ease of maintenance, so all devices have to be upgradeable remotely, via "OTA" (over-the-air): Once I have a solar-powered Sonoff SV-controlled PIR light device 35feet up on the roof of "the big barn", I don't want to have to climb a ladder to reboot it!
Putting all that together, I realised I might need a lot of different bits of code for all the different devices / functions, so I took a decision early on that all the code must run on all the devices, so I could mix-n-match during the early design and experimentation stages.
After a steep learning curve - and several changes of heart - the system fell slowly into place. It started with a monolithic piece of code called "truc". It developed into a series of external libraries which break the functionality down into manageable sizes chunks and allow others to benefit from the learning curve by encapsulating all the difficult bits. (See the right hand links to github where these are published for anybody to use)
"Truc" became "Esparto" which I will talk about in a later post but is essentially all the general-purpose bits of the system that are needed by anybody doing this. All my own specific bits are a "layer" on top of that, still called "truc", but much, much smaller and lighter.
Subscribe to:
Posts (Atom)