The ESP8266 is an incredibly cheap and powerful WiFi chip which is geared towards "internet of things" applications, it comes from an obscure Chinese company, but they've released its SDK as open source. With its high availability on eBay, it was hard to resist, so I bought one to play with. At the moment, recently inspired by
cnlohr's work, I've got it serving webpages and detecting when "dumb" wireless PIRs and door sensors around my house are triggered.
A long story (skip it if you're here only to see stuff working):
For a long time after purchase, it sat unused like all useful electronics junk off the 'Bay. The thing was frustrating: I didn't have a 3.3v regulator, and I didn't have a USB to serial adapter to talk to the thing. Nevertheless, an Arduino board has both those things, so I powered it off the Arduino's 3v3 pin, and connected it to the tx and rx pins, using a voltage divider to make sure the ESP8266 didn't get fried. The aim was to get the Arduino to talk to it and send "AT commands" to it - the default firmware on the ESP8266 makes it act as a modem which can be controlled over serial/UART. This was a long shot. The Arduino's 3.3v rail is far undersized for an ESP8266, and three devices, one of them at a different voltage, on a single serial line, is asking for undefined behavior.
On testing, it didn't work; the ESP8266 never joined my WiFi network as it was supposed to. The Arduino was printing AT commands fine though (I could only sniff the Arduino's output because that is where the FTDI's rx is wired to). The setup not working wasn't surprising because it was a terrible way of doing things and there were too many unknowns.
Trying to make do without having to order then wait weeks for the proper parts off eBay, I forgot about talking to the Arduino, and tried to communicate directly with the ESP8266 through a serial monitor on the computer. To do this, the ESP8266's rx needs to be wired to the Arduino rx pin and tx to tx. The idea was to have the ESP8266 talk to the FTDI adapter and have the Arduino's ATmega328 remain silent. From memory, I think this worked just well enough to flash the chip, but commands and such often failed and caused resets because the Arduino's 3.3v regulator couldn't supply enough current. That sort of explains the first setup not working either.
This is the point where the ESP8266 went into storage for a long time.
Renewed interest:
The ESP8266 has become much more powerful and flexible with the release of its SDK, so I gave it another shot. I setup the SDK on Ubuntu in a virtual machine, I would highly recommend this; it works very well with USB pass through, and Linux is a joy to develop on (in my opinion). Also, I made a programming/breakout board for the module, otherwise it's a bit of a PITA as it can't be plugged directly into a breadboard.
The breakout board simply passes through all the pins to a header, but CH_PD (chip power down) and RST (reset) are pulled high through 4.7k resistors; this is a pre-requisite for normal operation (I'll go into the ESP8266's strange boot modes a bit later on).
The jumper on the breakout board, when inserted, pulls GPIO 0 to ground. This boots the ESP8266 into flash programming mode when the chip is reset by pulling RST low or by powering the chip off and on. To change the jumper, the ESP8266 has to be removed and then reinserted, this became a pain, so I started temporarily sticking a wire where the jumper would go and then briefly tapping a lead connected to RST to GND. The TX LED on the ESP8266 module would flash very briefly when RST was grounded, then stop completely - this behaviour indicates (not with certainty) that the chip is booted in flashing mode and is waiting for a program to be sent over serial to it. GPIO 0 can be un-grounded at this point.
If you were to be listening to the serial port at this point, you'd see a bunch of garbage, unless your baud rate was set to 74880. This is the baud rate which the ESP8266's bootloader prints a bunch of information at on boot. One important piece of information is boot mode. Boot mode (1,6) or (1,7) indicates the module is in flashing mode. Boot mode (3,6) or (3,7) indicates it will load and run the program stored in flash. Listening to the ESP8266's serial output is the most reliable way to verify if it has booted into UART flashing mode properly, rather than trying to judge the board's TX LED.
It is important to leave GPIO 2 floating or pull it high (it has an internal pull up resistor anyway) whenever you reset or boot the ESP8266, otherwise it goes haywire, possibly trying to boot into SD card mode - but of course there is no SD card. This advice stands whether you are trying to boot into flashing mode or run-from-flash mode. GPIO 0 and 2 act as perfectly normal IO ports after booting, but the fact that they are so important at reset and when power is applied make them hard to use in any permanent application if you want to attach sensors. For example, with this project, I had to detach the 433MHz radio receiver from GPIO 2 every time the ESP8266's bootloader had control (on resets or power applied) then reconnect the receiver once booted.
This is the radio receiver I had lying around and used. It comes with a matching 433MHz transmitter, but I didn't need it. The receiver has two data pins which are connected together; I'm not sure of the point of doing this. Anyway, the module drives the data pin high when it detects any carrier being present in the vicinity of 433MHz, and low when there isn't.
An antenna has to be soldered onto the board, or the reception range is merely a few centimetres. For 433MHz, a quarter wave monopole should be about 17 cm long. Use solid core wire for an antenna.
The 433MHz receiver module is regenerative one, rather than being a more expensive heterodyning one. From what I can tell, it has two stages of RF tuning and amplification: the first being being wide band and the second being more selective. Then the signal is fed into an op-amp, which I guess provides the massive gain which makes the output seem digital: high when a carrier is present, and low when there isn't.
This is an example of a 433MHz door/window sensor which the receiver can listen to. The door sensor simply chirps an ID repeatedly for a few hundred milliseconds when the magnet is taken away. The short transmission occurring only on triggering is pretty much a requirement for saving battery and not hogging the shared frequency (there is no collision avoidance or re-transmission). The downside is that the base station has to keep track of the state of all sensors and not miss any transmissions. Also, the battery in a sensor can die without the user knowing. This system would also be easily jammed or otherwise hacked, so these alarms are just toys, but that tends to be the nature of wireless - wireless on the cheap, anyway. These observations came from listening in with an RTL-SDR dongle, which I'll go into more detail on later.
The sensor's PCB is mostly un-populated. The 433MHz oscillator on the top right and the sheer lack of components (especially RF ones) hinted that data modulation wouldn't be much more sophisticated than switching a 433MHz carrier on and off like a Morse code transmitter. The name for this is on-off keying (OOK), which is a subset of amplitude-shift keying (ASK). The datasheet for the HS1527 (the IC on the sensor board), would've confirmed this guessing, had I had it at the time.
So here's an annotated screenshot from SDR Sharp, tuned in to the 433MHz area. I had just triggered a door sensor and a PIR sensor, which can be seen on the waterfall chart. Just quickly, the whole RTL-SDR project is amazing - it's made radio affordable to get into and incredibly flexible. Definitely check it out if you're interested even slightly in radio or what $15 of technology can do.
Signal 1. I figured was unrelated, or at least irrelevant, because it occurred every ~20 seconds and was comparatively weak. The weak signal suggested it came from farther away than the other two signals. Additionally, if it came from a door or PIR sensor, then I would have seen more than one instance of this signal, as there are about 8 sensors around my house. I wondered if it was the base station, but what use is it having the base station transmit when none of the remote sensors have receivers in them? Anyway, I was intending to bypass the existing base station.
Signal 2. and 3. appeared when I triggered a PIR and then a door sensor by walking along a corridor, stopping, opening a door, then walking back. So the burst which appeared twice must surely be from the PIR, having detected movement twice, leaving the other signal to be the door sensor.
Additionally, it was apparent that all the transmissions came from transmitters which must have
cost the lesser part of a few dollars in RF-related components. It was something between the massive bandwidth the signals occupied, the ever-so-slight frequency drifting and how far off the labelled oscillator frequency of 433.92MHz they were. But the spread in frequency just made it easier to tell sensors apart from a glance in SDR Sharp, so it's not all bad.
The next step from here was to record the signals as audio and take a closer look in Audacity. I set SDR Sharp to AM demodulation and set a decent squelch, so noise wouldn't appear between transmissions in recordings. I made audio recordings on the lowest sampling quality setting. On-off keying is a subset of amplitude modulation (but with only two signal levels), so this was most appropriate.
Above is one of the signals, shown in Audacity, zoomed in on a single packet from a much larger transmission. A typical transmission is made up of the same packet repeated over and over again. This is done so that a receiver's AGC and noise suppression have time to respond (if present) and also to provide some noise immunity. Much like the output of the hardware receiver the ESP8266 was connected to, we see a high when the transmitter's oscillator has power and a low when it doesn't. In short, the waveform shows the state of the output pin of the HS1527 encoder inside a sensor over a period of time.
The datasheet for the HS1527 indicates that in its on-off encoding scheme, a short pulse is a 0 and a long pulse is a 1, additionally, the unique identifier each chip spits out is 20 bits long. Below is the output of the 433MHz receiver module; it shows a different sensor from the one in the Audacity screenshot transmitting, hence some of the differences:
This output on the oscilloscope was promising because it looked exactly like what the RTL-SDR snifffed, showing that the 433MHz receiver was receiving the sensors well. The software decoder I wrote for the ESP8266 works by interrupting on a rising edge, then on a falling edge, and so forth, to measure the duration of pulses and gaps between them. Measuring the gaps, in addition to the pulses, turned out to be useful for detecting the end of a packet or a whole transmission and for rejecting non-conforming signals.
The packet on the oscilloscope screen, despite its similarities, is a different length to the one recorded in SDR Sharp. This is because, in the 10ms radio silence between packets, the RTL-SDR's AGC re-adjusted, raising the noise floor, which triggered the squelch (seen in the recording where the slight noise disappears completely). This would then have had to all be undone when transmission resumed, hence why the first two bits are missing in the Audacity screenshot. Set a fixed gain and turn squelch off if you want to record in SDR Sharp. I didn't even notice at the time.
But back to analysing the packet captured on the oscilloscope, which is actually complete... The datasheet for the HS1527 says that: a short pulse is a zero; a long pulse is a one; a short zero pulse with a slightly longer gap afterwards is a pre-amble bit. This works out well: the pulse at the start of the packet is a pre-amble; then there are 20 bits, which is the sensor ID; then there are the last 4 bits, which supposedly indicate the state of four digital inputs on the chip.
When I wrote the ESP8266 OOK decoder, I didn't know what was inside the sensors, let alone have the datasheet for the HS1527 chip. I mistakenly treated the whole packet, including pre-amble and digital input state, as the sensor ID. Luckily the pre-amble is constant and the digital inputs never change for these simple sensors (they aren't connected to anything), so I got away with it.
Above is a packet from signal 1. received through the 433MHz receiver module. Even though it wasn't relevant to this project, I had a look at it anyway. At first glance it looks like it might use the same on-off keying scheme as the sensors in my alarm system, but it is actually encoded using Manchester encoding from what I can tell. If you try interpreting this signal using the same coding scheme as the previous signal, you might notice that the length of time each apparent bit takes up is varying, i.e. pulses aren't padded predictably with gaps. This is odd and tends to indicate that the signal had been misinterpreted because data streams usually have a constant data rate.
These two captures show the typical noise which appeared on the 433MHz module's output when there was no transmission present. Though this noise is easily rejected in software because it doesn't conform to what an expected OOK signal looks like, all the edges unfortunately create a lot of interrupts for the ESP8266 to deal with. It did not cause problems, from what I could tell, but if it did, the number of edges could be reduced with a small capacitor between the data line and ground. The capacitor would need to be connected to the output via a resistor to increase charge/discharge times, and then the ESP8266's GPIO would have to be connected to the junction of the resistor and capacitor.
More noise - in more detail.
Above: the ESP8266 all connected up. The board at the top is a Zigbee CC2530 development board. I discovered it lying around and detached the Zigbee module to reduce power usage. It was perfect for the ESP8266 because it has a CP2102 USB-to-UART/serial converter on it, a beefy 3.3V rail and a 5V rail fed from USB power or the 3xAA battery pack on the bottom of it. Importantly, all the buttons, LEDs, UART pins and power rails come out to header pins so they were easy to connect over to my ESP8266 programming board. Also, it's hard to see in the photo, but I added a decoupling capacitor between the ESP8266's VCC and GND, anecdotally it seemed to improve it's accuracy when flashing and stability in general. I did this because a few things made me suspicious that my programs were getting corrupted when I flashed them.
The grey wire and green wire left dangling grounded RST. A reset button on the programming board would have been much better. Also, another annoyance was having to disconnect the 433MHz receiver every time the ESP8266 was reset or turned on (I've mentioned this quite a few times now), hence the connection via an orange jumper and blue jumper, which made for easier disconnection.
The circuit above didn't work well though: the 433MHz receiver module's reception range was atrociously short because I was powering it off 3.3v so that I wouldn't have to level-shift its output for the ESP8266 and then have two power rails if I made a dedicated board for this project in future. The 433MHz receiver is designed to be powered off 5V, but I tested it briefly with 3.3V on the oscilloscope and it seemed fine. Granted, it was being tested with a nearby sensor. Later, I switched to powering it off 5V and used a voltage divider on the output; this improved reception dramatically, as expected.
Results/final product:
On startup, or when there are no triggered sensors:
When sensors have been triggered:
The source code for this project is
here and could be useful for finding examples of how to use the ESP8266's interrupts and create a simple HTTP web server on it. The ESP8266's SDK is slightly thin on coherent documentation, but looking through other people's code and the SDK's header files is enough information to figure most things out.
Other than that, be mindful of clearing dynamic memory as soon as possible, and be careful where your pointers point before passing them to functions or otherwise using them. If you try writing or reading from a memory address which is obviously invalid e.g. outside dynamic memory, the ESP8266 tends to catch it and print some debugging information then reset itself, like this:
Fatal exception (29):
epc1=0x400127f0, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000064, depc=0x00000000
Exception 29 indicates that the program tried to write to a prohibited memory address. This is similar to exception 28, which results from trying to read a prohibited address. In this particular exception, the program instruction at 0x400127f0 has tried to write to 0x00000064... epc1 and excvaddr are the two most important details.
The rest of my run time debugging consisted of os_printf statements and commenting out lines.
An important consideration when working with the ESP8266 is that it has a whole bunch of internal registers which aren't changed when you flash a program to it. For example, WiFi credentials, station mode and whether to automatically reconnect to WiFi. These are modified at run time by making calls in your program to functions such as wifi_station_set_auto_connect(1);. If you are not explicit in setting or disabling settings such as this in your program, expect undefined and inconsistent behavior. It also appears that sometimes these persistent internal registers get corrupted or misconfigured, which apparently results in the alarming but not-always-fatal error MEMCHECK FAIL!!! being printed. I found that I stopped getting this message after upgrading the version of the SDK I was using.
It felt like there were endless traps and general weirdness with the ESP8266, but it is possible to do useful stuff using it and its SDK.
