fix md links

This commit is contained in:
Arnim Läuger 2016-01-16 13:42:05 +01:00
parent e48be07d8b
commit e5f351e59b
1 changed files with 28 additions and 28 deletions

View File

@ -6,7 +6,7 @@
## What is this FAQ for? ## What is this FAQ for?
This FAQ does not aim to help you to learn to program or even how to program in Lua. There are plenty of resources on the Internet for this, some of which are listed in [[#Where to start|Where to start]]. What this FAQ does is to answer some of the common questions that a competent Lua developer would ask in learning how to develop Lua applications for the ESP8266 based boards running the [[http://NodeMCU.com/index_en.html|NodeMCU]] firmware. This FAQ does not aim to help you to learn to program or even how to program in Lua. There are plenty of resources on the Internet for this, some of which are listed in [Where to start](#where-to-start). What this FAQ does is to answer some of the common questions that a competent Lua developer would ask in learning how to develop Lua applications for the ESP8266 based boards running the [NodeMcu](http://NodeMCU.com/index_en.html) firmware.
## Lua Language ## Lua Language
@ -14,28 +14,28 @@ This FAQ does not aim to help you to learn to program or even how to program in
The NodeMCU firmware implements Lua 5.1 over the Espressif SDK for its ESP8266 SoC and the IoT modules based on this. The NodeMCU firmware implements Lua 5.1 over the Espressif SDK for its ESP8266 SoC and the IoT modules based on this.
* The official lua.org **[[http://www.lua.org/manual/5.1/manual.html|Lua Language specification]]** gives a terse but complete language specification. * The official lua.org **[Lua Language specification](http://www.lua.org/manual/5.1/manual.html)** gives a terse but complete language specification.
* Its [[http://www.lua.org/faq.html|FAQ]] provides information on Lua availability and licensing issues. * Its [FAQ](http://www.lua.org/faq.html) provides information on Lua availability and licensing issues.
* The **[[http://www.luafaq.org/|unofficial Lua FAQ]]** provides a lot of useful Q and A content, and is extremely useful for those learning Lua as a second language. * The **[unofficial Lua FAQ](http://www.luafaq.org/)** provides a lot of useful Q and A content, and is extremely useful for those learning Lua as a second language.
* The [[http://lua-users.org/wiki/|Lua User's Wiki]] gives useful example source and relevant discussion. In particular, its [[http://lua-users.org/wiki/Learning|Lua Learning Lua]] section is a good place to start learning Lua. * The [Lua User's Wiki](http://lua-users.org/wiki/) gives useful example source and relevant discussion. In particular, its [Lua Learning Lua](http://lua-users.org/wiki/Learning) section is a good place to start learning Lua.
* The best book to learn Lua is *Programming in Lua* by Roberto Ierusalimschy, one of the creators of Lua. It's first edition is available free [[http://www.lua.org/pil/contents.html|online]] . The second edition was aimed at Lua 5.1, but is out of print. The third edition is still in print and available in paperback. It contains a lot more material and clearly identifies Lua 5.1 vs Lua 5.2 differences. **This third edition is widely available for purchase and probably the best value for money**. References of the format [PiL **n.m**] refer to section **n.m** in this edition. * The best book to learn Lua is *Programming in Lua* by Roberto Ierusalimschy, one of the creators of Lua. It's first edition is available free [online](http://www.lua.org/pil/contents.html) . The second edition was aimed at Lua 5.1, but is out of print. The third edition is still in print and available in paperback. It contains a lot more material and clearly identifies Lua 5.1 vs Lua 5.2 differences. **This third edition is widely available for purchase and probably the best value for money**. References of the format [PiL **n.m**] refer to section **n.m** in this edition.
* The Espressif ESP8266 architecture is closed source, but the Espressif SDK itself is continually being updated so the best way to get the documentation for this is to [[https://www.google.co.uk/search?q=Espressif+IoT+SDK+Programming+Guide|google Espressif IoT SDK Programming Guide]] or to look at the Espressif [[http://bbs.espressif.com/viewforum.php?f=5|downloads forum]] . * The Espressif ESP8266 architecture is closed source, but the Espressif SDK itself is continually being updated so the best way to get the documentation for this is to [google Espressif IoT SDK Programming Guide](https://www.google.co.uk/search?q=Espressif+IoT+SDK+Programming+Guide) or to look at the Espressif [downloads forum](http://bbs.espressif.com/viewforum.php?f=5) .
* The **[[http://www.NodeMCU.com/docs/|NodeMCU documentation]]** is available online. However, please remember that the development team are based in China, and English is a second language, so the documentation needs expanding and be could improved with technical proofing. * The **[NodeMCU documentation](http://www.NodeMCU.com/docs/)** is available online. However, please remember that the development team are based in China, and English is a second language, so the documentation needs expanding and be could improved with technical proofing.
* As with all Open Source projects the source for the NodeMCU firmware is openly available on the [[https://github.com/NodeMCU/NodeMCU-firmware|GitHub NodeMCU-firmware]] repository. * As with all Open Source projects the source for the NodeMCU firmware is openly available on the [GitHub NodeMCU-firmware](https://github.com/NodeMCU/NodeMCU-firmware) repository.
### How is NodeMCU Lua different to standard Lua? ### How is NodeMCU Lua different to standard Lua?
Whilst the Lua standard distribution includes a host stand-alone Lua interpreter, Lua itself is primarily an *extension language* that makes no assumptions about a "main" program: Lua works embedded in a host application to provide a powerful, light-weight scripting language for use within the application. This host application can then invoke functions to execute a piece of Lua code, can write and read Lua variables, and can register C functions to be called by Lua code. Through the use of C functions, Lua can be augmented to cope with a wide range of different domains, thus creating customized programming languages sharing a syntactical framework. Whilst the Lua standard distribution includes a host stand-alone Lua interpreter, Lua itself is primarily an *extension language* that makes no assumptions about a "main" program: Lua works embedded in a host application to provide a powerful, light-weight scripting language for use within the application. This host application can then invoke functions to execute a piece of Lua code, can write and read Lua variables, and can register C functions to be called by Lua code. Through the use of C functions, Lua can be augmented to cope with a wide range of different domains, thus creating customized programming languages sharing a syntactical framework.
The ESP8266 was designed and is fabricated in China by [[http://espressif.com/new-sdk-release/|Espressif Systems]]. Espressif have also developed and released a companion software development kit (SDK) to enable developers to build practical [[wp>Internet of Things|IoT]] applications for the ESP8266. The SDK is made freely available to developers in the form of binary libraries and SDK documentation. However this is in a *closed format*, with no developer access to the source files, so ESP8266 applications *must* rely solely on the SDK API (and the somewhat Spartan SDK API documentation). The ESP8266 was designed and is fabricated in China by [Espressif Systems](http://espressif.com/new-sdk-release/). Espressif have also developed and released a companion software development kit (SDK) to enable developers to build practical IoT applications for the ESP8266. The SDK is made freely available to developers in the form of binary libraries and SDK documentation. However this is in a *closed format*, with no developer access to the source files, so ESP8266 applications *must* rely solely on the SDK API (and the somewhat Spartan SDK API documentation).
The NodeMCU Lua firmware is an ESP8266 application and must therefore be layered over the ESP8266 SDK. However, the hooks and features of Lua enable it to be seamlessly integrated without loosing any of the standard Lua language features. The firmware has replaced some standard Lua modules that don't align well with the SDK structure with ESP8266-specific versions. For example, the standard `io` and `os` libraries don't work, but have been largely replaced by the NodeMCU `node` and `file` libraries. The `debug` and `math` libraries have also been omitted to reduce the runtime footprint. The NodeMCU Lua firmware is an ESP8266 application and must therefore be layered over the ESP8266 SDK. However, the hooks and features of Lua enable it to be seamlessly integrated without loosing any of the standard Lua language features. The firmware has replaced some standard Lua modules that don't align well with the SDK structure with ESP8266-specific versions. For example, the standard `io` and `os` libraries don't work, but have been largely replaced by the NodeMCU `node` and `file` libraries. The `debug` and `math` libraries have also been omitted to reduce the runtime footprint.
NodeMCU Lua is based on [[http://www.eluaproject.net/overview|eLua]], a fully featured implementation of Lua 5.1 that has been optimized for embedded system development and execution to provide a scripting framework that can be used to deliver useful applications within the limited RAM and Flash memory resources of embedded processors such as the ESP8266. One of the main changes introduced in the eLua fork is to use read-only tables and constants wherever practical for library modules. On a typical build this approach reduces the RAM footprint by some 20-25KB and this makes a Lua implementation for the ESP8266 feasible. This technique is called LTR and this is documented in detail in an eLua technical paper: [[http://www.eluaproject.net/doc/master/en_arch_ltr.html|Lua Tiny RAM]]. NodeMCU Lua is based on [eLua](http://www.eluaproject.net/overview), a fully featured implementation of Lua 5.1 that has been optimized for embedded system development and execution to provide a scripting framework that can be used to deliver useful applications within the limited RAM and Flash memory resources of embedded processors such as the ESP8266. One of the main changes introduced in the eLua fork is to use read-only tables and constants wherever practical for library modules. On a typical build this approach reduces the RAM footprint by some 20-25KB and this makes a Lua implementation for the ESP8266 feasible. This technique is called LTR and this is documented in detail in an eLua technical paper: [Lua Tiny RAM](http://www.eluaproject.net/doc/master/en_arch_ltr.html).
The mains impacts of the ESP8266 SDK and together with its hardware resource limitations are not in the Lua language implementation itself, but in how *application programmers must approach developing and structuring their applications*. As discussed in detail below, the SDK is non-preemptive and event driven. Tasks can be associated with given events by using the SDK API to registering callback functions to the corresponding events. Events are queued internally within the SDK, and it then calls the associated tasks one at a time, with each task returning control to the SDK on completion. *The SDK states that if any tasks run for more than 10 mSec, then services such as Wifi can fail.* The mains impacts of the ESP8266 SDK and together with its hardware resource limitations are not in the Lua language implementation itself, but in how *application programmers must approach developing and structuring their applications*. As discussed in detail below, the SDK is non-preemptive and event driven. Tasks can be associated with given events by using the SDK API to registering callback functions to the corresponding events. Events are queued internally within the SDK, and it then calls the associated tasks one at a time, with each task returning control to the SDK on completion. *The SDK states that if any tasks run for more than 10 mSec, then services such as Wifi can fail.*
The NodeMCU libraries act as C wrappers around registered Lua callback functions to enable these to be used as SDK tasks. ***You must therefore use an [[wp>Event-driven programming|Event-driven programming]] style in writing your ESP8266 Lua programs***. Most programmers are used to writing in a procedural style where there is a clear single flow of execution, and the program interfaces to operating system services by a set of synchronous API calls to do network I/O, etc. Whilst the logic of each individual task is procedural, this is not how you code up ESP8266 applications. The NodeMCU libraries act as C wrappers around registered Lua callback functions to enable these to be used as SDK tasks. ***You must therefore use an Event-driven programming style in writing your ESP8266 Lua programs***. Most programmers are used to writing in a procedural style where there is a clear single flow of execution, and the program interfaces to operating system services by a set of synchronous API calls to do network I/O, etc. Whilst the logic of each individual task is procedural, this is not how you code up ESP8266 applications.
## ESP8266 Specifics ## ESP8266 Specifics
@ -47,7 +47,7 @@ The NodeMCU libraries act as C wrappers around registered Lua callback functions
### How is coding for the ESP8266 different to standard Lua? ### How is coding for the ESP8266 different to standard Lua?
* The ESP8266 use onchip RAM and offchip Flash memory connected using a dedicated SPI interface. Both of these are *very* limited (when compared to systems than most application programmer use). The SDK and the Lua firmware already use the majority of this resource: the later build versions keep adding useful functionality, and unfortunately at an increased RAM and Flash cost, so depending on the build version and the number of modules installed the runtime can have as little as 17KB RAM and 40KB Flash available at an application level. This Flash memory is formatted an made available as a **SPI Flash File System (SPIFFS)** through the `file` library. * The ESP8266 use onchip RAM and offchip Flash memory connected using a dedicated SPI interface. Both of these are *very* limited (when compared to systems than most application programmer use). The SDK and the Lua firmware already use the majority of this resource: the later build versions keep adding useful functionality, and unfortunately at an increased RAM and Flash cost, so depending on the build version and the number of modules installed the runtime can have as little as 17KB RAM and 40KB Flash available at an application level. This Flash memory is formatted an made available as a **SPI Flash File System (SPIFFS)** through the `file` library.
* However, if you choose to use a custom build, for example one which uses integer arithmetic instead of floating point, and which omits libraries that aren't needed for your application, then this can help a lot doubling these available resources. (See Marcel Stör's excellent [[http://frightanic.com/NodeMCU-custom-build/|custom build tool]] that he discusses in [[http://www.esp8266.com/viewtopic.php?f=23&t=3001|this forum topic]]). Even so, those developers who are used to dealing in MB or GB of RAM and file systems can easily run out of these resources. Some of the techniques discussed below can go a long way to mitigate this issue. * However, if you choose to use a custom build, for example one which uses integer arithmetic instead of floating point, and which omits libraries that aren't needed for your application, then this can help a lot doubling these available resources. (See Marcel Stör's excellent [custom build tool](http://frightanic.com/NodeMCU-custom-build/) that he discusses in [this forum topic](http://www.esp8266.com/viewtopic.php?f=23&t=3001)). Even so, those developers who are used to dealing in MB or GB of RAM and file systems can easily run out of these resources. Some of the techniques discussed below can go a long way to mitigate this issue.
* Current versions of the ESP8266 run the SDK over the native hardware so there is no underlying operating system to capture errors and to provide graceful failure modes, so system or application errors can easily "PANIC" the system causing it to reboot. Error handling has been kept simple to save on the limited code space, and this exacerbates this tendency. Running out of a system resource such as RAM will invariably cause a messy failure and system reboot. * Current versions of the ESP8266 run the SDK over the native hardware so there is no underlying operating system to capture errors and to provide graceful failure modes, so system or application errors can easily "PANIC" the system causing it to reboot. Error handling has been kept simple to save on the limited code space, and this exacerbates this tendency. Running out of a system resource such as RAM will invariably cause a messy failure and system reboot.
* There is currently no `debug` library support. So you have to use 1980s-style "binary-chop" to locate errors and use print statement diagnostics though the systems UART interface. (This omission was largely because of the Flash memory footprint of this library, but there is no reason in principle why we couldn't make this library available in the near future as an custom build option). * There is currently no `debug` library support. So you have to use 1980s-style "binary-chop" to locate errors and use print statement diagnostics though the systems UART interface. (This omission was largely because of the Flash memory footprint of this library, but there is no reason in principle why we couldn't make this library available in the near future as an custom build option).
* The LTR implementation means that you can't easily extend standard libraries as you can in normal Lua, so for example an attempt to define `function table.pack()` will cause a runtime error because you can't write to the global `table`. (Yes, there are standard sand-boxing techniques to achieve the same effect by using metatable based inheritance, but if you try to use this type of approach within a real application, then you will find that you run out of RAM before you implement anything useful.) * The LTR implementation means that you can't easily extend standard libraries as you can in normal Lua, so for example an attempt to define `function table.pack()` will cause a runtime error because you can't write to the global `table`. (Yes, there are standard sand-boxing techniques to achieve the same effect by using metatable based inheritance, but if you try to use this type of approach within a real application, then you will find that you run out of RAM before you implement anything useful.)
@ -59,7 +59,7 @@ The NodeMCU libraries act as C wrappers around registered Lua callback functions
```lua ```lua
node.restart(); for i = 1, 20 do print("not quite yet -- ",i); end node.restart(); for i = 1, 20 do print("not quite yet -- ",i); end
``` ```
* You therefore *have* to implement ESP8266 Lua applications using an event driven approach. You have to understand which SDK API requests schedule asynchronous processing, and which define event actions through Lua callbacks. Yes, such an event-driven approach makes it difficult to develop procedurally structured applications, but it is well suited to developing the sorts of application that you will typically want to implement on an [[wp>Internet of Things|IoT]] device. * You therefore *have* to implement ESP8266 Lua applications using an event driven approach. You have to understand which SDK API requests schedule asynchronous processing, and which define event actions through Lua callbacks. Yes, such an event-driven approach makes it difficult to develop procedurally structured applications, but it is well suited to developing the sorts of application that you will typically want to implement on an IoT device.
### So how does the SDK event / tasking system work in Lua? ### So how does the SDK event / tasking system work in Lua?
@ -73,7 +73,7 @@ The NodeMCU libraries act as C wrappers around registered Lua callback functions
The NodeMCU firmware simply mirrors this structure at a Lua scripting level: The NodeMCU firmware simply mirrors this structure at a Lua scripting level:
* A startup module `init.lua` is invoked on boot. This function module can be used to do any initialisation required and to call the necessary timer alarms or libary calls to bind and callback routines to implement the tasks needed in response to any system events. * A startup module `init.lua` is invoked on boot. This function module can be used to do any initialisation required and to call the necessary timer alarms or libary calls to bind and callback routines to implement the tasks needed in response to any system events.
* The Lua libraries provide a set of functions for declaring application functions (written in Lua) as callbacks (which are stored in the [[#So how is the Lua Registry used and why is this important?|Lua registry]]) to associate application tasks with specific hardware and timer events. These are non-preemptive at an applications level. * The Lua libraries provide a set of functions for declaring application functions (written in Lua) as callbacks (which are stored in the [Lua registry](#so-how-is-the-lua-registry-used-and-why-is-this-important)) to associate application tasks with specific hardware and timer events. These are non-preemptive at an applications level.
* The Lua libraries work in consort with the SDK to queue pending events and invoke any registered Lua callback routines, which then run to completion uninterrupted. * The Lua libraries work in consort with the SDK to queue pending events and invoke any registered Lua callback routines, which then run to completion uninterrupted.
* Excessively long-running Lua functions can therefore cause other system functions and services to timeout, or allocate memory to buffer queued data, which can then trigger either the watchdog timer or memory exhaustion, both of which will ultimately cause the system to reboot. * Excessively long-running Lua functions can therefore cause other system functions and services to timeout, or allocate memory to buffer queued data, which can then trigger either the watchdog timer or memory exhaustion, both of which will ultimately cause the system to reboot.
* By default, the Lua runtime also 'listens' to UART 0, the serial port, in interactive mode and will execute any Lua commands input through this serial port. * By default, the Lua runtime also 'listens' to UART 0, the serial port, in interactive mode and will execute any Lua commands input through this serial port.
@ -110,7 +110,7 @@ This example defines five Lua functions:
| On Receive| Connection listener | c, l (socket, input) | Yes | | On Receive| Connection listener | c, l (socket, input) | Yes |
| On Disconnect | Connection listener | c (socket) | Yes | | On Disconnect | Connection listener | c (socket) | Yes |
`s`, `con_std` and `s_output` are global, and no [[#Why is it importance to understand how upvalues are implemented when programming for the ESP8266?|upvalues]] are used. There is no "correct" order to define these in, but we could reorder this code for clarity (though doing this adds a few extra globals) and define these functions separately one another. However, let us consider how this is executed: `s`, `con_std` and `s_output` are global, and no [upvalues](#why-is-it-importance-to-understand-how-upvalues-are-implemented-when-programming-for-the-esp8266) are used. There is no "correct" order to define these in, but we could reorder this code for clarity (though doing this adds a few extra globals) and define these functions separately one another. However, let us consider how this is executed:
* The outer module is compiled including the four internal functions. * The outer module is compiled including the four internal functions.
* `Main` is then assigning the created `net.createServer()` to the global `s`. The `connection listener` closure is created and bound to a temporary variable which is then passed to the `socket.listen()` as an argument. The routine then exits returning control to the firmware. * `Main` is then assigning the created `net.createServer()` to the global `s`. The `connection listener` closure is created and bound to a temporary variable which is then passed to the `socket.listen()` as an argument. The routine then exits returning control to the firmware.
@ -205,7 +205,7 @@ Most of us have fallen into the trap of creating an `init.lua` that has a bug in
### How do I minimise the footprint of an application? ### How do I minimise the footprint of an application?
* Perhaps the simplest aspect of reducing the footprint of an application is to get its scope correct. The ESP8266 is an IoT device and not a general purpose system. It is typically used to attach real-world monitors, controls, etc. to an intranet and is therefore designed to implement functions that have limited scope. We commonly come across developers who are trying to treat the ESP8266 as a general purpose device and can't understand why their application can't run. * Perhaps the simplest aspect of reducing the footprint of an application is to get its scope correct. The ESP8266 is an IoT device and not a general purpose system. It is typically used to attach real-world monitors, controls, etc. to an intranet and is therefore designed to implement functions that have limited scope. We commonly come across developers who are trying to treat the ESP8266 as a general purpose device and can't understand why their application can't run.
* The simplest and safest way to use IoT devices is to control them through a dedicated general purpose system on the same network. This could be a low cost system such as a [[https://www.raspberrypi.org/|RaspberryPi (RPi)]] server, running your custom code or an open source [[wp>home automation|home automation (HA)]] application. Such systems have orders of magnitude more capacity than the ESP8266, for example the RPi has 2GB RAM and its SD card can be up to 32GB in capacity, and it can support the full range of USB-attached disk drives and other devices. It also runs a fully featured Linux OS, and has a rich selection of applications pre configured for it. There are plenty of alternative systems available in this under $50 price range, as well as proprietary HA systems which can cost 10-50 times more. * The simplest and safest way to use IoT devices is to control them through a dedicated general purpose system on the same network. This could be a low cost system such as a [RaspberryPi (RPi)](https://www.raspberrypi.org/) server, running your custom code or an open source home automation (HA) application. Such systems have orders of magnitude more capacity than the ESP8266, for example the RPi has 2GB RAM and its SD card can be up to 32GB in capacity, and it can support the full range of USB-attached disk drives and other devices. It also runs a fully featured Linux OS, and has a rich selection of applications pre configured for it. There are plenty of alternative systems available in this under $50 price range, as well as proprietary HA systems which can cost 10-50 times more.
* Using a tiered approach where all user access to the ESP8266 is passed through a controlling server means that the end-user interface (or smartphone connector), together with all of the associated validation and security can be implemented on a system designed to have the capacity to do this. This means that you can limit the scope of your ESP8266 application to a limited set of functions being sent to or responding to requests from this system. * Using a tiered approach where all user access to the ESP8266 is passed through a controlling server means that the end-user interface (or smartphone connector), together with all of the associated validation and security can be implemented on a system designed to have the capacity to do this. This means that you can limit the scope of your ESP8266 application to a limited set of functions being sent to or responding to requests from this system.
* *If you are trying to implement a user-interface or HTTP webserver in your ESP8266 then you are really abusing its intended purpose. When it comes to scoping your ESP8266 applications, the adage **K**eep **I**t **S**imple **S**tupid truly applies.* * *If you are trying to implement a user-interface or HTTP webserver in your ESP8266 then you are really abusing its intended purpose. When it comes to scoping your ESP8266 applications, the adage **K**eep **I**t **S**imple **S**tupid truly applies.*
@ -213,10 +213,10 @@ Most of us have fallen into the trap of creating an `init.lua` that has a bug in
* It is possible to write Lua code in a very compact format which is very dense in terms of functionality per KB of source code. * It is possible to write Lua code in a very compact format which is very dense in terms of functionality per KB of source code.
* However if you do this then you will also find it extremely difficult to debug or maintain your application. * However if you do this then you will also find it extremely difficult to debug or maintain your application.
* A good compromise is to use a tool such as [[http://luaforge.net/projects/luasrcdiet/|LuaSrcDiet]], which you can use to compact production code for downloading to the ESP8266: * A good compromise is to use a tool such as [LuaSrcDiet](http://luaforge.net/projects/luasrcdiet/), which you can use to compact production code for downloading to the ESP8266:
* Keep a master repository of your code on your PC or a cloud-based versioning repository such as [[https://github.com/|GitHub]] * Keep a master repository of your code on your PC or a cloud-based versioning repository such as [GitHub](https://github.com/)
* Lay it out and comment it for ease of maintenance and debugging * Lay it out and comment it for ease of maintenance and debugging
* Use a package such as [[https://github.com/4refr0nt/ESPlorer|Esplorer]] to download modules that you are debugging and to test them. * Use a package such as [Esplorer](https://github.com/4refr0nt/ESPlorer) to download modules that you are debugging and to test them.
* Once the code is tested and stable, then compress it using LuaSrcDiet before downloading to the ESP8266. Doing this will reduce the code footprint on the SPIFFS by 2-3x. * Once the code is tested and stable, then compress it using LuaSrcDiet before downloading to the ESP8266. Doing this will reduce the code footprint on the SPIFFS by 2-3x.
* Consider using `node.compile()` to pre-compile any production code. This removes the debug information from the compiled code reducing its size by roughly 40%. (However this is still perhaps 1.5-2x larger than a LuaSrcDiet-compressed source format, so if SPIFFS is tight then you might consider leaving less frequently run modules in Lua format. If you do a compilation, then you should consider removing the Lua source copy from file system as there's little point in keeping both on the ESP8266. * Consider using `node.compile()` to pre-compile any production code. This removes the debug information from the compiled code reducing its size by roughly 40%. (However this is still perhaps 1.5-2x larger than a LuaSrcDiet-compressed source format, so if SPIFFS is tight then you might consider leaving less frequently run modules in Lua format. If you do a compilation, then you should consider removing the Lua source copy from file system as there's little point in keeping both on the ESP8266.
@ -225,8 +225,8 @@ Most of us have fallen into the trap of creating an `init.lua` that has a bug in
* The Lua Garbage collector is very aggressive at scanning and recovering dead resources. It uses an incremental mark-and-sweep strategy which means that any data which is not ultimately referenced back to the Globals table, the Lua registry or in-scope local variables in the current Lua code will be collected. * The Lua Garbage collector is very aggressive at scanning and recovering dead resources. It uses an incremental mark-and-sweep strategy which means that any data which is not ultimately referenced back to the Globals table, the Lua registry or in-scope local variables in the current Lua code will be collected.
* Setting any variable to `nil` dereferences the previous context of that variable. (Note that reference-based variables such as tables, strings and functions can have multiple variables referencing the same object, but once the last reference has been set to `nil`, the collector will recover the storage. * Setting any variable to `nil` dereferences the previous context of that variable. (Note that reference-based variables such as tables, strings and functions can have multiple variables referencing the same object, but once the last reference has been set to `nil`, the collector will recover the storage.
* Unlike other compile-on-load languages such as PHP, Lua compiled code is treated the same way as any other variable type when it comes to garbage collection and can be collected when fully dereferenced, so that the code-space can be reused. * Unlike other compile-on-load languages such as PHP, Lua compiled code is treated the same way as any other variable type when it comes to garbage collection and can be collected when fully dereferenced, so that the code-space can be reused.
* Lua execution is intrinsically divided into separate event tasks with each bound to a Lua callback. This, when coupled with the strong dispose on dereference feature, means that it is very easy to structure your application using an classic technique which dates back to the 1950s known as [[wp>Overlay (programming)|Overlays]]. * Lua execution is intrinsically divided into separate event tasks with each bound to a Lua callback. This, when coupled with the strong dispose on dereference feature, means that it is very easy to structure your application using an classic technique which dates back to the 1950s known as Overlays.
* Various approaches can be use to implement this. One is described by DP Whittaker in his [[http://www.esp8266.com/viewtopic.php?f=19&t=1940|Massive memory optimization: flash functions]] topic. Another is to use *volatile modules*. There are standard Lua templates for creating modules, but the `require()` library function creates a reference for the loaded module in the `package.loaded` table, and this reference prevents the module from being garbage collected. To make a module volatile, you should remove this reference to the loaded module by setting its corresponding entry in `package.loaded` to `nil`. You can't do this in the outermost level of the module (since the reference is only created once execution has returned from the module code), but you can do it in any module function, and typically an initialisation function for the module, as in the following example: * Various approaches can be use to implement this. One is described by DP Whittaker in his [Massive memory optimization: flash functions](http://www.esp8266.com/viewtopic.php?f=19&t=1940) topic. Another is to use *volatile modules*. There are standard Lua templates for creating modules, but the `require()` library function creates a reference for the loaded module in the `package.loaded` table, and this reference prevents the module from being garbage collected. To make a module volatile, you should remove this reference to the loaded module by setting its corresponding entry in `package.loaded` to `nil`. You can't do this in the outermost level of the module (since the reference is only created once execution has returned from the module code), but you can do it in any module function, and typically an initialisation function for the module, as in the following example:
```lua ```lua
local s=net.createServer(net.TCP) local s=net.createServer(net.TCP)
@ -284,10 +284,10 @@ Note that if you use `require("XXX")` to load your code then this will automatic
### How do I get a feel for how much memory my functions use? ### How do I get a feel for how much memory my functions use?
* You should get an overall understanding of the VM model if you want to make good use of the limited resources available to Lua applications. An essential reference here is [[http://luaforge.net/docman/83/98/ANoFrillsIntroToLua51VMInstructions.pdf|A No Frills Introduction to Lua 5.1 VM Instructions]] . This explain how the code generator works, how much memory overhead is involved with each table, function, string etc.. * You should get an overall understanding of the VM model if you want to make good use of the limited resources available to Lua applications. An essential reference here is [A No Frills Introduction to Lua 5.1 VM Instructions](http://luaforge.net/docman/83/98/ANoFrillsIntroToLua51VMInstructions.pdf) . This explain how the code generator works, how much memory overhead is involved with each table, function, string etc..
* You can't easily get a bytecode listing of your ESP8266 code; however there are two broad options for doing this: * You can't easily get a bytecode listing of your ESP8266 code; however there are two broad options for doing this:
* **Generate a bytecode listing on your development PC**. The Lua 5.1 code generator is basically the same on the PC and on the ESP8266, so whilst it isn't identical, using the standard Lua batch compiler `luac` against your source on your PC with the `-l -s` option will give you a good idea of what your code will generate. The main difference between these two variants is the size_t for ESP8266 is 4 bytes rather than the 8 bytes size_t found on modern 64bit development PCs; and the eLua variants generate different access references for ROM data types. If you want to see what the `string.dump()` version generates then drop the `-s` option to retain the debug information. * **Generate a bytecode listing on your development PC**. The Lua 5.1 code generator is basically the same on the PC and on the ESP8266, so whilst it isn't identical, using the standard Lua batch compiler `luac` against your source on your PC with the `-l -s` option will give you a good idea of what your code will generate. The main difference between these two variants is the size_t for ESP8266 is 4 bytes rather than the 8 bytes size_t found on modern 64bit development PCs; and the eLua variants generate different access references for ROM data types. If you want to see what the `string.dump()` version generates then drop the `-s` option to retain the debug information.
* **Upload your `.lc` files to the PC and disassemble then there**. There are a number of Lua code disassemblers which can list off the compiled code that you application modules will generate, `if` you have a script to upload files from your ESP8266 to your development PC. I use [[http://luaforge.net/projects/chunkspy/|ChunkSpy]] which can be downloaded [[http://files.luaforge.net/releases/chunkspy/chunkspy/ChunkSpy-0.9.8/ChunkSpy-0.9.8.zip|here]] , but you will need to apply the following patch so that ChunkSpy understands eLua data types: * **Upload your `.lc` files to the PC and disassemble then there**. There are a number of Lua code disassemblers which can list off the compiled code that you application modules will generate, `if` you have a script to upload files from your ESP8266 to your development PC. I use [ChunkSpy](http://luaforge.net/projects/chunkspy/) which can be downloaded [here](http://files.luaforge.net/releases/chunkspy/chunkspy/ChunkSpy-0.9.8/ChunkSpy-0.9.8.zip) , but you will need to apply the following patch so that ChunkSpy understands eLua data types:
```diff ```diff
--- a/ChunkSpy-0.9.8/5.1/ChunkSpy.lua 2015-05-04 12:39:01.267975498 +0100 --- a/ChunkSpy-0.9.8/5.1/ChunkSpy.lua 2015-05-04 12:39:01.267975498 +0100
@ -303,9 +303,9 @@ Note that if you use `require("XXX")` to load your code then this will automatic
perform = ChunkSpy_Interact perform = ChunkSpy_Interact
``` ```
* Your other great friend is to use `node.heap()` regularly through your code. * Your other great friend is to use `node.heap()` regularly through your code.
* Use these tools and play with coding approaches to see how many instructions each typical line of code takes in your coding style. The Lua Wiki gives some general [[wp>Program optimization|optimisation]] tips, but in general just remember that these focus on optimising for execution speed and you will be interested mainly in optimising for code and variable space as these are what consumes precious RAM. * Use these tools and play with coding approaches to see how many instructions each typical line of code takes in your coding style. The Lua Wiki gives some general optimisation tips, but in general just remember that these focus on optimising for execution speed and you will be interested mainly in optimising for code and variable space as these are what consumes precious RAM.
### What is the cost of using functions? ### ### What is the cost of using functions?
Consider the output of `dofile("test1a.lua")` on the following code compared to the equivalent where the function `pnh()` is removed and the extra `print(heap())` statement is placed inline: Consider the output of `dofile("test1a.lua")` on the following code compared to the equivalent where the function `pnh()` is removed and the extra `print(heap())` statement is placed inline:
@ -342,7 +342,7 @@ Of course you should still use functions to structure your code and encapsulate
### Why file writes fail all the time on DEVKIT V1.0? ### Why file writes fail all the time on DEVKIT V1.0?
* NodeMCU DEVKIT V1.0 uses ESP12-E-DIO(ESP-12-D) module. This module runs the Flash memory in [[#What's the different between DIO and QIO mode?|Dual IO SPI]] (DIO) mode. This firmware will not be correctly loaded if you uses old flashtool version, and the filesystem will not work if you used a pre 0.9.6 firmware version (<0.9.5) or old. The easiest way to resolve this problem s update all the firmware and flash tool to current version. * NodeMCU DEVKIT V1.0 uses ESP12-E-DIO(ESP-12-D) module. This module runs the Flash memory in [Dual IO SPI](#whats-the-different-between-dio-and-qio-mode) (DIO) mode. This firmware will not be correctly loaded if you uses old flashtool version, and the filesystem will not work if you used a pre 0.9.6 firmware version (<0.9.5) or old. The easiest way to resolve this problem s update all the firmware and flash tool to current version.
- Use the latest [esptool.py](https://github.com/themadinventor/esptool) with DIO support and command option to flash firmware, or - Use the latest [esptool.py](https://github.com/themadinventor/esptool) with DIO support and command option to flash firmware, or
- Use the latest [NodeMCU flasher](https://github.com/NodeMCU/NodeMCU-flasher) with default option. (You must select the `restore to default` option in advanced menu tab), or - Use the latest [NodeMCU flasher](https://github.com/NodeMCU/NodeMCU-flasher) with default option. (You must select the `restore to default` option in advanced menu tab), or
- Use the latest Espressif's flash tool -- see [this Espressif forum topic](http://bbs.espressif.com/viewtopic.php?f=5&t=433) (without auto download support). Use DIO mode and 32M flash size option, and flash latest firmware to 0x00000. Before flashing firmware, remember to hold FLASH button, and press RST button once. Note that the new NodeMCU our firmware download tool, when released, will be capable of flashing firmware automatically without any button presses. - Use the latest Espressif's flash tool -- see [this Espressif forum topic](http://bbs.espressif.com/viewtopic.php?f=5&t=433) (without auto download support). Use DIO mode and 32M flash size option, and flash latest firmware to 0x00000. Before flashing firmware, remember to hold FLASH button, and press RST button once. Note that the new NodeMCU our firmware download tool, when released, will be capable of flashing firmware automatically without any button presses.
@ -354,4 +354,4 @@ Of course you should still use functions to structure your code and encapsulate
<TODO> <TODO>
### How does DEVKIT use DTR and RTS enter download mode? ### How does DEVKIT use DTR and RTS enter download mode?
<TODO> <TODO>