* espconn: remove unused espconn code, take 1
This is the easiest part of https://github.com/nodemcu/nodemcu-firmware/issues/3004 .
It removes a bunch of functions that were never called in our tree.
* espconn: De-orbit espconn_gethostbyname
Further work on https://github.com/nodemcu/nodemcu-firmware/issues/3004
While here, remove `mqtt`'s charming DNS-retry logic (which is neither
shared with nor duplicated in other modules) and update its :connect()
return value behavior and documentation.
* espconn: remove scary global pktinfo
A write-only global! How about that.
* net: remove deprecated methods
All the TLS stuff moved over there a long time ago, and
net_createUDPSocket should just do what it says on the tin.
* espconn_secure: remove ESPCONN_SERVER support
We can barely function as a TLS client; being a TLS server seems like a
real stretch. This code was never called from Lua anyway.
* espconn_secure: more code removal
* espconn_secure: simplify ssl options structure
There is nothing "ssl_packet" about this structure. Get rid of the
terrifying "pbuffer" pointer.
Squash two structure types together and eliminate an unused field.
* espconn_secure: refactor mbedtls_msg_info_load
Split out espconn_mbedtls_parse, which we can use as part of our effort
towards addressing https://github.com/nodemcu/nodemcu-firmware/issues/3032
* espconn_secure: introduce TLS cert/key callbacks
The new feature part of https://github.com/nodemcu/nodemcu-firmware/issues/3032
Subsequent work will remove the old mechanism.
* tls: add deprecation warnings
* luacheck: net.ifinfo is a thing now
* tls: remove use of espconn->reverse
* mqtt: stop using espconn->reverse
Instead, just place the espconn structure itself at the top of the user
data. This enlarges the structure somewhat but removes one more layer
of dynamic heap usage and NULL checks.
While here, simplify the code a bit.
* mqtt: remove redundant pointer to connect_info
Everywhere we have the mqtt_state_t we also have the lmqtt_userdata.
* mqtt: doc fixes
* mqtt: note bug
* tls: allow :on(...,nil) to unregister a callback
* Remove stale putative MD2 support
This hasn't worked in a while, presumably since one of our upstream
merges. Don't bother making it work, since MD2 is generally considered
insecure.
* Land mbedtls 2.16.3-77-gf02988e57
* TLS: remove some dead code from espconn_mbedtls
There was some... frankly kind of scary buffer and data shuffling if
ESP8266_PLATFORM was defined. Since we don't, in fact, define that
preprocessor symbol, just drop the code lest anyone (possibly future-me)
be scared.
* TLS: espconn_mbedtls: run through astyle
No functional changes
* TLS: espconn_mbedtls: put the file_params on the stack
There's no need to malloc a structure that's used only locally.
* TLS: Further minor tidying of mbedtls glue
What an absolute shitshow this is. mbedtls should absolutely not
be mentioned inside sys/socket.h and app/mbedtls/app/lwIPSocket.c is not
so much glue as it as a complete copy of a random subset of lwIP; it
should go, but we aren't there yet.
Get rid of the mysterious "mbedlts_record" struct, which housed merely a
length of bytes sent solely for gating the "record sent" callback.
Remove spurious __attribute__((weak)) from symbols not otherwise
defined and rename them to emphasize that they are not actually part of
mbedtls proper.
* TLS: Rampage esp mbedtls glue and delete unused code
This at least makes the shitshow smaller
* TLS: lwip: fix some memp definitions
I presume these also need the new arguments
* TLS: Remove more non-NodeMCU code from our mbedtls
* TLS: drop support for 1.1
Depending on who you ask it's either EOL already or EOL soon, so
we may as well get rid of it now.
The PR removed the bulk of non-newlib headers from the NodeMCU source base.
app/libc has now been cut down to the bare minimum overrides to shadow the
corresponding functions in the SDK's libc. The old c_xyz.h headerfiles have been
nuked in favour of the standard <xyz.h> headers, with a few exceptions over in
sdk-overrides. Again, shipping a libc.a without headers is a terrible thing to do. We're
still living on a prayer that libc was configured the same was as a default-configured
xtensa gcc toolchain assumes it is. That part I cannot do anything about, unfortunately,
but it's no worse than it has been before.
This enables our source files to compile successfully using the standard header files,
and use the typical malloc()/calloc()/realloc()/free(), the strwhatever()s and
memwhatever()s. These end up, through macro and linker magic, mapped to the
appropriate SDK or ROM functions.
* Update TLS protocol support
TLS1.0 is past PCI's EOL; BEAST is no more
Enable elliptic curve key exchanges
Do not enable the smallest ECs for security
Do not enable the largest ECs for computational time
Do not enable 25519 (sad) because it doesn't go across the wire
Drop non-PFS key exchanges
Drop ARC4, Blowfish, DES, genprime, XTEA code
Drop renegotiation support completely
It takes so much heap that it's not likely to work out well
Tidy handling of SSL_BUFFER_SIZE
Update docs
Drop mention of startcom, since they are no more, for letsencrypt
* Update mbedtls to 2.7.7
Preserve our vsnprintf and platform hacks
* Introduce TLS maximum fragment size knob
Reduce buffer size to 4Ki by default and advertize that. That's the
largest we can advertize with the TLS MFL extension, so there's no
point in making them larger. The truly adventurous can re-raise
SSL_BUFFER_SIZE and undefine the SSL_MAX_FRAGMENT_LENGTH_CODE and get
back to the earlier behavior.
* Default to mbedTLS debug with DEVELOP_VERSION
Initial commit for
https://github.com/nodemcu/nodemcu-firmware/issues/2225 .
Replay patches from Espressif's repository at
https://github.com/espressif/ESP8266_NONOS_SDK between tags v2.1.0 and
v2.2.0:
0001-sync-from-ccca00f2.patch
Superseded by existing changes, but lines reordered in app/driver/key.c
to minimize divergences.
0002-sync-from-3f38ad5a.patch
Upstream files only
0003-Update-links.patch
Not meaningful to NodeMCU
0004-sync-from-01990ad0.patch
0005-sync-from-cdf6877d.patch
Upstream files only
0006-sync-from-f29e744c.patch
Upstream files only, user_interface.h override non-conflicting
0009-feat-lwip-Move-lwip-source-code-to-third_party-folde.patch
Merged change to lwip/app/espconn_udp.c; rest is just moves or
appears to not apply.
0010-feat-mbedtls-Add-mbedtls-source-code-in-third_party-.patch
Does not apply; we use our own mbedtls
0011-added-C-support.patch
Merged to Makefile
0012-feat-mbedtls-Rebuild-libmbedtls.patch
Already applied
0013-fix-at-Fix-some-bugs-of-AT.patch
Upstream files only
0014-feat-err_t-Redefine-err_t-to-s32_t.patch
Merged to app/include/arch/cc.h and ./app/include/lwip/app/espconn.h;
the rest is upstream files.
0015-fix-wpa-Fix-wpa-wpa2-ptk-gtk-reinstallation-vulnerab.patch
0016-fix-wifi-Remove-group-key-entry-before-connecting-to.patch
0017-feat-lib-Remove-time-function-in-libmain.patch
Upstream files only
0018-feat-espconn-Modification-for-espconn.patch
Merged to app/include/lwip/app/espconn.h,
app/include/lwip/app/espconn_tcp.h, app/lwip/app/espconn.c,
app/lwip/app/espconn_tcp.c
0019-feat-at-Use-new-espconn_recv-to-fix-tcp-server-issue.patch
0020-feat-examples-Update-mqtt-demo-and-auto-bin-generate.patch
Upstream files only
0021-wifi-Add-scan-threshold-and-dwell-time.patch
0022-feat-wifi-Add-country-code-API.patch
0023-feat-wifi-Record-more-information-of-scanned-ap.patch
Upstream files only, user_interface.h override non-conflicting
0024-fix-example-Fix-IoT_Demo-user-sector-error.patch
Upstream files only
0025-fix-lwip-Fix-sequence-number-error-of-RST-ACK.patch
Merged app/lwip/core/tcp_in.c
0026-fix-mbedtls-Fix-memory-leak.patch
Merged app/mbedtls/app/lwIPSocket.c
0027-fix-mbedtls-Fix-call-send-callback-function-failed.patch
Merged app/mbedtls/app/espconn_mbedtls.c
0028-feat-Add-USE_OPTIMIZE_PRINTF-in-third_party-Makefile.patch
Merged app/Makefile
0029-fix-api-Fix-ets_delay_us-declaration.patch
Upstream files only, osapi.h override non-conflicting
0030-fix-wifi-Remove-max_tx_power-in-wifi_country_t-in-li.patch
0031-fix-wifi-Fix-softAP-wrong-behavior-after-call-system.patch
0032-fix-wifi-bugfix-of-scan-fail-after-connected-if-max-.patch
0033-feat-at-Enable-scan-time-scan-type-and-add-scan-resu.patch
0034-feat-at-Add-command-AT-CWCOUNTRY.patch
0035-fix-at-Fix-that-AT-CIPSTART-causes-busy-if-the-serve.patch
Upstream files only
0036-feat-mbedtls-Speed-up-mbedtls-handshake-process.patch
Merged app/mbedtls/app/espconn_mbedtls.c
0037-fix-api-Fix-os_calloc-declaration.patch
Merged app/include/lwip/mem.h; sdk-overrides/include/mem.h
non-conflicting.
0038-fix-mbedtls-Fix-disconnect-callback-function-never-b.patch
Merged app/mbedtls/app/espconn_mbedtls.c; minor revision to logic in
6576af959b. Whitespace fixes.
0039-feat-at-Add-country-code-start-channel-in-AT-CWCOUNT.patch
0040-fix-net80211-Fix-Null-pointer-in-ieee80211_rfid_locp.patch
Upstream files only
0041-feat-wifi-Add-new-esp_init_data_default-v08-bin.patch
Upstream files only, but impacts Makefile
0042-fix-mbedtls-Fix-load-cert-fail-when-the-private-key-.patch
Merged app/mbedtls/app/espconn_mbedtls.c
0043-fix-wifi-The-start-channel-can-be-any-valid-channel.patch
0044-fix-wifi-Fix-scan-do-not-start-after-connect.patch
0045-feat-wifi-Add-keep-connection-for-station-to-keep-co.patch
0046-feat-at-Update-AT-version-to-1.6.0.0.patch
0047-fix-at-Fix-GSLP-too-long-time.patch
0048-fix-at-Fix-the-message-is-incorrect-when-creating-UD.patch
0049-feat-at-Add-AT-CIPSERVERMAXCONN.patch
Upstream files only
0050-feat-system-Add-softap-distributes-station-ip-event.patch
Upstream files only, user_interface.h override non-conflicting
0051-feat-example-Use-libmbedtls.a-instead-of-libssl.a-in.patch
Upstream files only
0052-feat-mesh-Remove-mesh-support.patch
Upstream files only, but go ahead and remove comment from
ld/nodemcu.ld.
0053-fix-example-Fix-forget-to-add-integer-parameter-when.patch
Upstream files only
0054-fix-mbedtls-Fix-reconnect_callback-is-not-triggered-.patch
Merged app/mbedtls/app/espconn_mbedtls.c
0055-feat-at-Add-AT-SYSMSG-to-enable-some-report-informat.patch
0056-fix-at-Fix-the-incorrect-link-id-when-client-connect.patch
0057-fix-at-Fix-the-bug-that-it-should-be-error-when-the-.patch
0058-fix-smartconfig-Fix-the-smartconfig-scan-time-issue.patch
0059-fix-lwip-Fix-the-bug-of-lwip-output.patch
Upstream files only
0060-fix-lwip-Fix-the-length-of-TCP-data-in-one-packet-is.patch
0061-fix-lwip-Fix-send-TCP-data-with-two-or-more-pbuf.patch
Merged app/lwip/core/tcp_out.c
0062-fix-wifi-Fix-assert-happen-when-smartconfig-start-th.patch
Upstream files only
0063-fix-mbedtls-Fix-memory-leak-when-ESP8266-as-SSL-TLS-.patch
Merged app/mbedtls/app/espconn_mbedtls.c
0064-fix-mbedtls-Fix-already-freed-and-exception-bug-when.patch
Merged app/mbedtls/app/lwIPSocket.c
0065-fix-at-Fix-bug-that-there-is-no-result-when-sending-.patch
0066-feat-example-Add-AT-bin-version.patch
0067-feat-version-Update-version-to-2.2.0-and-add-version.patch
0068-feat-bin-Update-AT-bin-for-SDK-2.2.0.patch
Upstream files only
Apply local changes to build:
app/include/lwip/app/espconn.h pulls changes (and license decl) from
upstream SDK. Makefile is altered to use this file ahead of the
SDK's.
Remove lwip's sntp support, since it was never really wired in anyway.
See https://github.com/nodemcu/nodemcu-firmware/issues/2042 for more
information. Patch Makefile to strip time.o, the consumer of lwip's
sntp functionality, from libmain.a, resulting in much
easier-to-understand error messages.
This has consequences for mbedtls. The simplest thing to do, which is,
impressively, not a change in behavior, is to completely disable TLS
certificate time validation; a later patch can optionally couple this to
RTCTIME support.
Similarly, it happens that the sqlite3 import was calling time(), but
this was not going to work out well for it. Just stub it out to always
return unix timestamp 0, as would have happened anyway.
Changes unprocessed:
0007-sync-from-080c37e1.patch
0008-feat-lib-Compile-some-libraries-with-ffunction-secti.patch
These two make changes to the linker script; perhaps
they are worth porting over, but I have not done so
here.
This is build-tested (ADC, BIT, COLOR_UTILS, CRON, CRYPTO, DHT, ENCODER,
FILE, GPIO, HTTP, I2C, MQTT, NET, NODE, OW, PCM, PERF, PWM, RTCFIFO,
RTCMEM, RTCTIME, SNTP, SPI, SQLITE3, STRUCT, TLS, TMR, UART, WIFI,
WS2812, WS2812_EFFECTS) and boots, but only limited run-time testing has
been performed. Testing done does, however, include having made a few
TLS connections through the HTTP module, so things are not hopelessly
broken, at the very least.
* mbedtls 2.7.0 (mbedtls-2.7.0-0-g32605dc8)
Wholesale import, with a few changes from earlier preserved through.
Ideally we would soon get to the point of having no divergences from
upstream.
* tls: add function to adjust mbedTLS debug level
* mbedTLS update
* mbedtls: vsnprintf macroification
* Further update mbedTLS to 2.6.1
* mbedtls: make debugging work again
* Silence SSL messages on normal teardown
* Drop DTLS support from mbedtls
* Update LWIP from SDK
* mbedTLS integration
* Fix argument type in dbg_printf (#1624)
* Migrate to espressif’s download center (#1604)
* Fixed BBS links to firmware
* Adjust net module docs with mbedTLS info
* Remove unrelevant axTLS notice
Nathaniel Wesley Filardo
Terry Ellison
Johny Mattsson
Johny Mattsson
Nathaniel Wesley Filardo
sergio
Yury Popov