Reworked linker magic to work with IDF 3.3 (#2805)

With the IDF asserting full control over the linker scripts and insisting on
the application description being the first entry in the .flash.rodata
section, or previous method of doing link-time arrays stopped working.
Why? Because the build patched in a SHA256 digest straight into our arrays.

With the limited language of the gcc linker scripts I could find no other
way of getting it in cleanly.

The IDF "linker fragments" support can not be made to work for our needs:
  - no support for setting alignment before including objects
  - no support for declaring symbols
  - no support for adding our terminating zeros
  - insists on grouping objects by lib rather than by declared grouping,
    which means we could at most have a single link-time-array using
    the IDF mechanism
  - also does not like underscores in section names, but that's just an
    annoyance

So, the least bad option that I could come up with was to use a project-wide
makefile snippet to add a target in-between the IDF's generation of the
esp32.project.ld file, and the linking of our NodeMCU.elf. In this target
we read in the esp32.project.ld linker script, check whether we have our
arrays in there, and if not rewrites the linker script.

Oh, and the esp32.project.ld file only came into existence on the IDF 3.3
branch, so I had to change up the IDF to the latest release/3.3 as well.
I would've preferred a stable tag, but the v3.3-beta3 had a really nasty
regression for us (can't add partition entry), so that was a no-go.

components/base_nodemcu/Makefile.projbuild

@@ -0,0 +1,8 @@
+BASE_NODEMCU_DIR:=$(dir $(lastword $(MAKEFILE_LIST)))
+BASE_NODEMCU_BUILD_DIR:=$(BUILD_DIR_BASE)/base_nodemcu
+
+$(BUILD_DIR_BASE)/$(PROJECT_NAME).elf: $(BASE_NODEMCU_BUILD_DIR)/ld_patched
+
+$(BASE_NODEMCU_BUILD_DIR)/ld_patched: $(BUILD_DIR_BASE)/esp32/esp32.project.ld
+	"$(BASE_NODEMCU_DIR)/add_rodata_ld.sh" "$<" "$(BASE_NODEMCU_DIR)/ld/nodemcu_rodata.ld"
+	touch $@

components/base_nodemcu/add_rodata_ld.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+# syntax: add_rodata_ld.sh /path/to/esp32.project.ld /path/to/snippet_file
+set -eu
+ldfile="$1"
+partial="$2"
+out="${ldfile}.new"
+IFS='
+'
+msg="/* NodeMCU patched */"
+if [[ $(head -n 1 "$ldfile") =~ "$msg" ]]
+then
+  echo "NodeMCU rodata already patched into $(basename $ldfile)"
+  exit 0
+else
+  echo "Patching in NodeMCU rodata into $(basename $ldfile)"
+  echo "$msg" > "$out"
+fi
+
+cat "$ldfile" | while read line
+do
+  if [[ $line =~ "drom0_0_seg" ]]
+  then
+    cat "$partial" >> "$out"
+  fi
+  echo $line >> "$out"
+done
+mv "$out" "$ldfile"

components/base_nodemcu/component.mk

@@ -1,4 +0,0 @@
-COMPONENT_ADD_INCLUDEDIRS:=include
-# Note: It appears this component must come lexicographically before esp32
-# in order to get the -T arguments in the right order.
-COMPONENT_ADD_LDFLAGS:=-L $(COMPONENT_PATH)/ld -T nodemcu_core.ld -lbase_nodemcu

components/base_nodemcu/ld/nodemcu_rodata.ld

@@ -1,6 +1,7 @@
-SECTIONS {
+
-  .flash.rodata : ALIGN(4)
+    /* ----- Begin NodeMCU link-time arrays ------- */
-  {
+
+    . = ALIGN(4);
     /* Link-time arrays containing the defs for the included modules */
     lua_libs = ABSOLUTE(.);
     KEEP(*(.lua_libs))
@@ -11,6 +12,6 @@ SECTIONS {
     esp_event_cb_table = ABSOLUTE(.);
     KEEP(*(.esp_event_cb_table))
     LONG(0) LONG(0) /* Null-terminate the array */
-  }
+
-}
+    /* ----- End NodeMCU link-time arrays ------- */
-INSERT BEFORE .flash.text
+

sdk/esp32-esp-idf

@@ -1 +1 @@
-Subproject commit 4aa1058e8a8f7f3fb17d9ac1158227ad161f2996
+Subproject commit 890a341db429db03a328035b59ca101a5952d096