By the time we get informed that there's an LFS reload we've already
opened all the user libraries as well, so it's not safe to simply redo
that with a new lua_State. Currently that results in a hang, and a
manual reset is required to progress. Doing the automatic restart is
the correct approach here, even though this adds another reboot to the
LFS reload process.
Also fixed up a couple of minor issues when embedded-LFS is used.
* Proof-of-concept multi-type console support via stdio
* Address crashes on linput's use of printf.
On an empty line input, a C3 with UART console would panic while attempting
to output the new Lua prompt. The backtrace shows a xQueueSemaphoreTake
with uxItemSize==0 as the panic cause, deep inside the uart driver, invoked
via vfs_uart and vfs_console layers, from printf.
Similarly, the printf for outputting a backspace/erase sequence would also
trigger a panic.
This workaround (of not mixing fflush() with printf) is likely merely hiding
a deeper issue, but it appears to be consistent. Plus, printf with no args
and a user-supplied format string is a no-no and should be fixed anyway.
* Work around IDF inconsistency with stdout buffering.
* Increase console task stack size.
Seems on Xtensa it ended up not being enough.
* Switch to single-byte console reads.
* Stop cheating and feed Lua from the right context.
* Work around IDF buffering stdout even when told not to, on ACM consoles.
* Initial build support for esp32c6.
Plus fixup of module selection for a variety of targets.
* Update github actions to node 20 versions.
* Update github build to deal with Lua 5.3 being default.
* Address fatal compiler warning.
Newer IDF toolchain is stricter, and we'd apparently failed to build test
the Lua-5.1 path for some time.
* Initial build support for esp32h2.
* Upgrade IDF to v5.1.3
* Fix left-over incorrect type in uzlib.
* Avoid null pointer crashes when debugging startup.
* Workaround for using wifi module on S2 with USB-CDC console.
---------
Co-authored-by: Jade Mattsson <github@frozenlogic.org>
Plenty of dependency adjustments, printf format specificier updates,
FreeRTOS type and macro name modernisation, not to mention API changes.
Still plenty of legacy/deprecated drivers in use which will need updating.
The following features have been removed due to no longer being available
from the IDF:
- ADC hall effect sensor reading
- Configuration of SD SPI host via sdmmc module (now must be done first
via the spimaster module)
- FAT partition selection on external SD cards; only the first FAT
partition is supported by the IDF now
On the other hand, the eth module now supports the following new chipsets:
- KSZ8001
- KSZ8021
- KSZ8031
- KSZ8051
- KSZ8061
- KSZ8091
- Possibly additional models in the LAN87xx series (the IDF docs aren't
clear on precisely which models are handled)
Further, the sdmmc module is now available on the ESP32-S3 as well.
Also removed old, very unsafe node.osoutput(). We're now integrating cleanly
with the IDF/newlib way of redirecting stdout.
Added necessary depends in Kconfig to ensure VFS support is enabled, as
otherwise you'd only get a mysterious crash when attempting to enable
output redirection.
The IDF-provided VFS resolves several issues:
- The IDF components having a different view of the (virtual) file system
compared to the Lua environment.
- RTOS task/thread safety. Our legacy VFS was only ever safe to use
from the LVM thread, which limited its usability. Upgrading it
would have effectively required a reimplementation of the IDF VFS,
which would have been a bigger task with larger on-going maintenance
issues.
- We're no longer needing to maintain our own SPIFFS component.
- We're no longer needing to maintain our own FATFS component.
- The legacy of the 8266's lack of standard C interface to the file system
is no longer holding us back, meaning that we can use the standard
Lua `io` module rather than the cobbled-together swiss army knife
also known as the file module.
Of course, the downside is that we'll either have to declare a backwards
breakage in regard to the file module, or provide a Lua shim for the old
functions, where applicable.
Also included is some necessary integer type fixups in unrelated code,
which apparently had depended on some non-standard types in either the
SPIFFS or FATFS headers.
A memory leak issue in the sdmmc module was also found and fixed while
said module got switched over to the Espressif VFS.
Module documentation has been updated to match the new reality (and I
discovered in some places it wasn't even matching the old reality).
Search-and-replace considered harmful. I completely missed the need to
explicitly declare "fast" tag functions (__xyz) in the mask field to
LROT_BEGIN()/LROT_END() when I brought over the 5.1+5.3 support.
Without those flags set properly, the LVM doesn't even bother going
looking for those methods, which in this case led to garbage collection
not calling the __gc functions, among other horrible things.
Mea culpa.
The IDF provides all we need these days, and the old driver was just
needlessly conflicting with the IDF settings and setup.
This also simplifies our uart input path as we no longer need to
duplicate the raw byte handling for when "run_input" is false.
Changes have been kept to a minimum, but a serious chunk of work was
needed to move from 8266isms to IDFisms.
Some things got refactored into components/lua/common, in particular
the LFS location awareness.
As part of this work I also evicted our partition table manipulation
code, as with the current IDF it kept breaking checksums and rendering
things unbootable, which is the opposite of helpful (which was the
original intent behind it).
The uart module got relocated from base_nodemcu to the modules component
properly, after I worked out how to force its inclusion using Kconfig alone.
The uzlib and parts of Lua had to be switched over to use the
C standard int types, as their custom typedefs conflicted with
RISC-V toolchain provided typedefs.
UART console driver updated to do less direct register meddling
and use the IDF uart driver interface for setup. Still using our
own ISR rather than the default driver ISR. Down the line we
might want to investigate whether the IDF ISR would be a better
fit.
Lua C modules have been split into common and ESP32/ESP32-S
specific ones. In the future there might also be ESP32-C3
specific modules, which would go into components/modules-esp32c3
at that point.
Our old automatic fixup of flash size has been discarded as it
interferes with the checksumming done by the ROM loader and
results in unbootable systems. The IDF has already taken on
this work via the ESPTOOL_FLASHSIZE_DETECT option, which handles
this situation properly.
* uart 1/2
* call -> pcall in uart_on_* functions
* fix docs
* fixed console driver when using custom console uart
* fixed line_inverse and error callback
* fixed a crash when uart.start() called more than one time
node.dsleep() no longer takes options.
node.output() not yet supported (needs syscall registration/chaining support)
Dynamic CPU frequency changing not currently supported in the IDF.
Various chip IDs not currently available/obtainable.
Boot reason completely revamped in ESP32, will need new code.
RTOS driver evicted as it did not play nice with stdio etc.
Implemented a minimal driver to fully support Lua console on UART0. Output
on UART0 done via stdout (provided by the IDF). Input and setup handled
via driver_console/console.c. In addition to the direct input function
console_getc(), the driver also registers in the syscall tables to enable
regular stdio input functions to work (yay!). The Lua VM is still using the
direct interface since it's less overhead, but does also work when going
through stdin/fd 0.
Auto-bauding on the console is not yet functional; revisit when the UART docs
are available.
Module registration/linking/enabling moved over to be Kconfig based. See
updates to base_nodemcu/include/module.h and base_nodemcu/Kconfig for
details.
The sdk-overrides directory/approach is no longer used. The IDF is simply
too different to the old RTOS SDK - we need to adapt our code directly instead.
Everything in app/ is now unused, and will need to be gradually migrated
into components/ though it is probably better to migrate straight from the
latest dev branch.
Jade Mattsson
Jade Mattsson
Yiheng Cao
Tom Sutcliffe
h2zero
Johny Mattsson
Marcel Stör
Javier Peletier
devsaurus
zelll