218 lines
8.7 KiB
Markdown
218 lines
8.7 KiB
Markdown
# Introduction
|
|
|
|
Welcome to the NodeMCU self-test suite. Here you will find our growing effort
|
|
to ensure that our software behaves as we think it should and that we do not
|
|
regress against earlier versions.
|
|
|
|
Our tests are written using [NTest](./NTest/NTest.md), a lightweight yet
|
|
featureful framework for specifying unit tests.
|
|
|
|
# Building and Running Test Software on NodeMCU Devices
|
|
|
|
Naturally, to test NodeMCU on its intended hardware, you will need one or more
|
|
NodeMCU-capable boards. At present, the test environment is specified using
|
|
two ESP8266 Devices Under Test (DUTs), but we envision expanding this to mixed
|
|
ESP8266/ESP32 environments as well.
|
|
|
|
Test programs live beside this file. While many test programs run on the
|
|
NodeMCU DUTs, but there is reason to want to orchestrate DUTs and the
|
|
environment using the host. Files matching the glob `NTest_*.lua` are intended
|
|
for on-DUT execution.
|
|
|
|
## Manual Test Invocation
|
|
|
|
At the moment, the testing regime and host-based orchestration is still in
|
|
development, and so things are a little more manual than perhaps desired. The
|
|
`NTest`-based test programs all assume that they can `require "NTest"`, and so
|
|
the easiest route to success is to
|
|
|
|
* build an LFS image containing
|
|
|
|
* [package.loader support for LFS](../lua_examples/lfs/_init.lua)
|
|
|
|
* [NTest itself](./NTest/NTest.lua)
|
|
|
|
* Any additional Lua support modules required (e.g., [mcp23017
|
|
support](../lua_modules/mcp23017/mcp23017.lua) )
|
|
|
|
* build a firmware with the appropriate C modules
|
|
|
|
* program the board with your firmware and LFS images
|
|
|
|
* ensure that `package.loader` is patched appropriately on startup
|
|
|
|
* transfer the `NTest_foo` program you wish to run to the device SPIFFS
|
|
(or have included it in the LFS).
|
|
|
|
* at the interpreter prompt, say `dofile("NTest_foo.lua")` (or
|
|
`node.LFS.get("NTest_foo")()`) to run the `foo` test program.
|
|
|
|
## Experimental Host Orchestration
|
|
|
|
Enthusiastic testers are encouraged to try using our very new, very
|
|
experimental host test runner, [tap-driver.expect](./tap-driver.expect). To
|
|
use this program, in addition to the above, the LFS environment should contain
|
|
[NTestTapOut](./tests/utils/NTestTapOut.lua), an output adapter for `NTest`,
|
|
making it speak a slight variant of the [Test Anything
|
|
Protocol](https://testanything.org/). This structured output is scanned for
|
|
by the script on the host.
|
|
|
|
You'll need `expect` and TCL and some TCL libraries available; on Debian, that
|
|
amounts to
|
|
|
|
apt install tcl tcllib tclx8.4 expect
|
|
|
|
This program should be invoked from beside this file with something like
|
|
|
|
TCLLIBPATH=./expectnmcu ./tap-driver.expect -serial /dev/ttyUSB3 -lfs ./lfs.img NTest_file.lua
|
|
|
|
This will...
|
|
|
|
* transfer and install the specified LFS module (and reboot the device to load LFS)
|
|
|
|
* transfer the test program
|
|
|
|
* run the test program with `NTest` shimmed to use the `NTestTapOut` output
|
|
handler
|
|
|
|
* summarize the results
|
|
|
|
* return 0 if and only if all tests have passed
|
|
|
|
This tool is quite flexible and takes a number of other options and flags
|
|
controlling aspects of its behavior:
|
|
|
|
* Additional files, Lua or otherwise, may be transferred by specifing them
|
|
before the test to run (e.g., `./tap-driver.expect a.lua b.lua
|
|
NTest_foo.lua`); dually, a `-noxfer` flag will suppress transferring even the
|
|
last file. All transferred files are moved byte-for-byte to the DUT's
|
|
SPIFFS with names, but not directory components, preserved.
|
|
|
|
* The `-lfs LFS.img` option need not be specified and, if not given, any
|
|
existing `LFS` image will remain on the device for use by the test.
|
|
|
|
* A `-nontestshim` flag will skip attempting to shim the given test program
|
|
with `NTestTapOut`; the test program is expected to provide its own TAP
|
|
output. The `-tpfx` argument can be used to override the leading `TAP: `
|
|
sigil used by the `NTestTapOut` output handler.
|
|
|
|
* A `-runfunc` option indicates that the last argument is not a file to
|
|
transfer but rather a function to be run. It will be invoked at the REPL
|
|
with a single argument, the shimmed `NTest` constructor, unless `-nontestshim`
|
|
is given, in which case the argument will be `nil`.
|
|
|
|
* A `-notests` option suppresses running tests (making the tool merely another
|
|
option for loading files to the device).
|
|
|
|
Transfers will be significantly faster if
|
|
[pipeutils](../lua_examples/pipeutils.lua) is available to `require` on the
|
|
DUT, but a fallback strategy exists if not. We suggest either including
|
|
`pipeutils` in LFS images, in SPIFFS, or as the first file to be transferred.
|
|
|
|
# NodeMCU Testing Environment
|
|
|
|
Herein we define the environment our testing framework expects to see
|
|
when it runs. It is composed of two ESP8266 devices, each capable of
|
|
holding an entire NodeMCU firmware, LFS image, and SPIFFS file system,
|
|
as well as additional peripheral hardware. It is designed to fit
|
|
comfortably on a breadboard and so should be easily replicated and
|
|
integrated into any firmware validation testing.
|
|
|
|
The test harness runs from a dedicated host computer, which is expected
|
|
to have reset- and programming-capable UART links to both ESP8266
|
|
devices, as found on almost all ESP8266 boards with USB to UART
|
|
adapters, but the host does not necessarily need to use USB to connect,
|
|
so long as TXD, RXD, DTR, and RTS are wired across.
|
|
|
|
A particular implementation of this can be found at [Test Harness](HardwareTestHarness.md).
|
|
|
|
## Peripherals
|
|
|
|
### I2C Bus
|
|
|
|
There is an I2C bus hanging off DUT 0. Attached hardware is used both as
|
|
tests of modules directly and also to facilitate testing other modules
|
|
(e.g., gpio).
|
|
|
|
#### MCP23017: I/O Expander
|
|
|
|
At address 0x20. An 16-bit tristate GPIO expander, this chip is used to
|
|
test I2C, GPIO, and ADC functionality. This chip's interconnections are
|
|
as follows:
|
|
|
|
MPC23017 | Purpose
|
|
---------|--------------------------------------------------------------
|
|
/RESET |DUT0 reset. This resets the chip whenever the host computer resets DUT 0 over its serial link (using DTR/RTS).
|
|
B 0 |4K7 resistor to DUT 0 ADC.
|
|
B 1 |2K2 resistor to DUT 0 ADC.
|
|
B 5 |DUT1 GPIO16/WAKE via 4K7 resitor
|
|
B 6 |DUT0 GPIO13 via 4K7 resistor and DUT1 GPIO15 via 4K7 resistor
|
|
B 7 |DUT0 GPIO15 via 4K7 resistor and DUT1 GPIO13 via 4K7 resistor
|
|
|
|
Notes:
|
|
|
|
- DUT 0's ADC pin is connected via a 2K2 reistor to this chip's port
|
|
B, pin 1 and via a 4K7 resistor to port B, pin 0. This gives us the
|
|
ability to produce approximately 0 (both pins low), 1.1 (pin 0 high,
|
|
pin 1 low), 2.2 (pin 1 high, pin 0 low), and 3.3V (both pins high)
|
|
on the ADC pin.
|
|
- Port B pins 6 and 7 sit on the UART cross-wiring between DUT 0 and
|
|
DUT 1. The 23017 will be tristated for inter-DUT UART tests, but
|
|
these
|
|
- Port B pins 2, 3, and 4, as well as all of port A, remain available
|
|
for expansion.
|
|
- The interrupt pins are not yet routed, but could be. We reserve DUT
|
|
0 GPIO 2 for this purpose with the understanding that the 23017's
|
|
interrupt functionality will be disabled (INTA, INTB set to
|
|
open-drain, GPINTEN set to 0) when not explicitly under test.
|
|
|
|
ESP8266 Device 0 Connections
|
|
----------------------------
|
|
|
|
ESP | Usage
|
|
----------|----------------------------------------------------------
|
|
GPIO 0 |Used to enter programming mode; otherwise unused in test environment.
|
|
GPIO 1 |Primary UART transmit; reserved for host communication
|
|
GPIO 2 |[reserved for 1-Wire] [+ reserved for 23017 INT[AB] connections]
|
|
GPIO 3 |Primary UART recieve; reserved for host communication
|
|
GPIO 4 |I2C SDA
|
|
GPIO 5 |I2C SCL
|
|
GPIO 6 |[Reserved for on-chip flash]
|
|
GPIO 7 |[Reserved for on-chip flash]
|
|
GPIO 8 |[Reserved for on-chip flash]
|
|
GPIO 9 |[Reserved for on-chip flash]
|
|
GPIO 10 |[Reserved for on-chip flash]
|
|
GPIO 11 |[Reserved for on-chip flash]
|
|
GPIO 12 |
|
|
GPIO 13 |Secondary UART RX; DUT 1 GPIO 15, I/O expander B 6
|
|
GPIO 14 |
|
|
GPIO 15 |Secondary UART TX; DUT 1 GPIO 13, I/O expander B 7
|
|
GPIO 16 |
|
|
ADC 0 |Resistor divider with I/O expander
|
|
|
|
ESP8266 Device 1 Connections
|
|
----------------------------
|
|
|
|
ESP | Usage
|
|
----------|----------------------------------------------------------
|
|
GPIO 0 |Used to enter programming mode; otherwise unused in test environment.
|
|
GPIO 1 |Primary UART transmit; reserved for host communication
|
|
GPIO 2 |[Reserved for WS2812]
|
|
GPIO 3 |Primary UART recieve; reserved for host communication
|
|
GPIO 4 |
|
|
GPIO 5 |
|
|
GPIO 6 |[Reserved for on-chip flash]
|
|
GPIO 7 |[Reserved for on-chip flash]
|
|
GPIO 8 |[Reserved for on-chip flash]
|
|
GPIO 9 |[Reserved for on-chip flash]
|
|
GPIO 10 |[Reserved for on-chip flash]
|
|
GPIO 11 |[Reserved for on-chip flash]
|
|
GPIO 12 |HSPI MISO
|
|
GPIO 13 |Secondary UART RX; DUT 0 GPIO 15, I/O exp B 7 via 4K7 Also used as HSPI MOSI for SPI tests
|
|
GPIO 14 |HSPI CLK
|
|
GPIO 15 |Secondary UART TX; DUT 0 GPIO 13, I/O exp B 6 via 4K7 Also used as HSPI /CS for SPI tests
|
|
GPIO 16 |I/O expander B 5 via 4K7 resistor, for deep-sleep tests
|
|
ADC 0 |
|
|
|
|
|