Even-driven programming - why you need it on ESP8266

Remember when you were a kid and your dad sat you down for that difficult "birds and bees" talk? Well, there's something we need to get out of the way now:

Programming the ESP8266 properly is not easy. In fact it's pretty tricky.

Sure, if all you want to do is flash an LED a la "blinky" sketch - that's ridiculously easy: pre-teenage kids can do it. But building an IOT infrastructure, writing firmware that will run on ESP-01, Sonoff, Wemos and NodeMCU that will never crash or reboot and will seamlessly reconnect after any network problem and never stop the attached hardware working...i.e. something actually useful, well that's harder. And it's virtually impossible unless you adopt the "event-driven programming" style.

Sure that are a lot of simple examples out there that don't do this - they look just like the code you are already used to writing - but the key is in the word "simple". They are examples to introduce you to a new concept: they are deliberately stripped back to the bare bones help you learn. They cannot hope to also teach you in a few lines of demo code the best way to use the new idea in the real world.

Rocket Science 101 is probably taught using a lot of fireworks, but no-one goes to the moon on one...

So why do you need to start adapting to the event-driven style?

Whatever device you are reading this on will be doing at least a dozen other things too: receiving an incoming SMS, updating your GPS location and playing your favourite tune. We live in an age where we are so used to such things, that we don't even notice it any more, we expect it.

The reason that your device is able to do all of those things apparently at the same time, is that for the last 40 or 50 years, hordes of programmers have learned the techniques of "multitasking" and built systems such as Windows, Linux, IOS, Android which are operating systems (OSs) that allow many things to happen as if at once, so that you don't have to understand muiltitasking when you write programs on those devices. But the ESP8266 doesn't have an operating system* so if you want to do more than one thing at a time, you do need to understand multitasking.

"But" - I hear you say - "I only want to do one thing at a time with my sketch! That's what I've always done with Arduino / AVR / STM32 etc and I didn't have to learn this "multitasking" thing!"

And I reply "Maybe so, but they didn't have built-in WiFi, did they? The ESP gives you no choice: it comes with WiFi built-in, and that's why you bought it. Having that WiFi changes everything - it's not "free" and it has important consequences. The first is that it doesn't operate by magic, it needs CPU time just like your sketch does.

The WiFi code in the ESP8266 needs to run all the time to keep the connection alive, as well as being ready to send and recieve data when your code needs it. So the ESP is partitioned into two sets of code: the WiFi code and your sketch. It is designed to run both at the same time. The main point here is that - unlike many other systems you may be used to - yours is not the only code running. Failing to adapt your coding style to these consequences - in anything but those simple examples - often leads to crashes, exceptions, "random" failures, "watchdog timer" resets and many other forms or programming pain. Event-driven programming is the easiest way to avoid that pain.

Instead of your code saying "do this, do that, then do the other" as you are used to doing, it now needs to say "tell me when X happens, tell me when Y happens and I'll sit here quietly - doing nothing - till you actually do". Buzzword time: The "old" way is called "synchronous" the "new" way is called "asynchronous" as well as "event-driven". Synchronous = you control when things happen, Asynchronous=you don't  - someone else does. In our case, "someone else" is the 200k+ of ESP firmware that manages the WiFi (amongst other things)

Your code has to co-operate and be ready to do what you need when the firmware says its OK. If your code runs in a tight loop or waits for a long time for an external resource (hardware, remote web page etc) and "blocks" the other code from running, bad things happen. This is why sometimes you will also hear about "blocking" (synchronous) and "non-blocking" (asynchronous) code.

To get the best out the ESP8266, you need to write "non-blocking" code.

The most common way of doing that is with "callbacks". A callback is a function that you write that gets called by some other piece of code, when that other code knows it is a good time to do so. You tell the "other" code what you are interested in, and it "calls you back" when the interesting thing happens. Till then you just twiddle your thumbs in the main loop. Telling the other code the name of your function is known as "registering a callback". Luckily for us, many of the libraries that come with the Arduino ESP8266 add-ons are designed this way.

Before we get into more details, let's get some other buzzwords out of the way. Those two "partitions" (your sketch and the WiFi firmware) are sometimes called "threads". Often when a system has only two threads - as in the ESP - they are known as the "background thread" (WiFi) and the "foreground thread" (Your sketch). After "setup" completes, most of your code starts from the "loop" function. So now its easy to understand why your code is sometimes said to "run on the main loop thread". Once the two threads start trying to "talk to each other" is when the fun begins...

The new way of thinking that comes along with the event-driven style is that events can happen at any time and often in a different order from what you might expect. Your code has to be ready. It needs to cope with suddenly being called "out of the blue" and it has to what it needs to do quickly, so that it doesn't block other (more important) code from running.

So now you know why, the next article will start looking into how, but before it does you need to realise that this isn't the end of the story...There are numerous (often complex) other consequences of two pieces of code both wanting to access a single resource (e.g. the CPU, a shared "flag") at the same time and there will be a lot more new buzzwords flying around when we meet them and learn how to deal with them.