The IDF-provided VFS resolves several issues:
- The IDF components having a different view of the (virtual) file system
compared to the Lua environment.
- RTOS task/thread safety. Our legacy VFS was only ever safe to use
from the LVM thread, which limited its usability. Upgrading it
would have effectively required a reimplementation of the IDF VFS,
which would have been a bigger task with larger on-going maintenance
issues.
- We're no longer needing to maintain our own SPIFFS component.
- We're no longer needing to maintain our own FATFS component.
- The legacy of the 8266's lack of standard C interface to the file system
is no longer holding us back, meaning that we can use the standard
Lua `io` module rather than the cobbled-together swiss army knife
also known as the file module.
Of course, the downside is that we'll either have to declare a backwards
breakage in regard to the file module, or provide a Lua shim for the old
functions, where applicable.
Also included is some necessary integer type fixups in unrelated code,
which apparently had depended on some non-standard types in either the
SPIFFS or FATFS headers.
A memory leak issue in the sdmmc module was also found and fixed while
said module got switched over to the Espressif VFS.
Module documentation has been updated to match the new reality (and I
discovered in some places it wasn't even matching the old reality).
Search-and-replace considered harmful. I completely missed the need to
explicitly declare "fast" tag functions (__xyz) in the mask field to
LROT_BEGIN()/LROT_END() when I brought over the 5.1+5.3 support.
Without those flags set properly, the LVM doesn't even bother going
looking for those methods, which in this case led to garbage collection
not calling the __gc functions, among other horrible things.
Mea culpa.
They don't need the target in there as the byte code is agnostic, but
it's marginally cleaner than letting the different builds overwrite the
artifacts.
The IDF provides all we need these days, and the old driver was just
needlessly conflicting with the IDF settings and setup.
This also simplifies our uart input path as we no longer need to
duplicate the raw byte handling for when "run_input" is false.
Changes have been kept to a minimum, but a serious chunk of work was
needed to move from 8266isms to IDFisms.
Some things got refactored into components/lua/common, in particular
the LFS location awareness.
As part of this work I also evicted our partition table manipulation
code, as with the current IDF it kept breaking checksums and rendering
things unbootable, which is the opposite of helpful (which was the
original intent behind it).
The uart module got relocated from base_nodemcu to the modules component
properly, after I worked out how to force its inclusion using Kconfig alone.
The uzlib and parts of Lua had to be switched over to use the
C standard int types, as their custom typedefs conflicted with
RISC-V toolchain provided typedefs.
UART console driver updated to do less direct register meddling
and use the IDF uart driver interface for setup. Still using our
own ISR rather than the default driver ISR. Down the line we
might want to investigate whether the IDF ISR would be a better
fit.
Lua C modules have been split into common and ESP32/ESP32-S
specific ones. In the future there might also be ESP32-C3
specific modules, which would go into components/modules-esp32c3
at that point.
Our old automatic fixup of flash size has been discarded as it
interferes with the checksumming done by the ROM loader and
results in unbootable systems. The IDF has already taken on
this work via the ESPTOOL_FLASHSIZE_DETECT option, which handles
this situation properly.
Philip Gladstone
Johny Mattsson
Marcel Stör
Johny Mattsson