diff --git a/docs/lua-modules/fifo.md b/docs/lua-modules/fifo.md new file mode 100644 index 00000000..9fb60542 --- /dev/null +++ b/docs/lua-modules/fifo.md @@ -0,0 +1,108 @@ +# FIFO Module +| Since | Origin / Contributor | Maintainer | Source | +| :----- | :-------------------- | :---------- | :------ | +| 2019-02-10 | [nwf](https://github.com/nwf) | [nwf](https://github.com/nwf) | [fifo.lua](../../lua_modules/fifo/fifo.lua) | + +This module provides flexible, generic FIFOs built around Lua tables and +callback functions. It is specifically engineered to work well with the +NodeMCU event-based and memory-constrained environment. + +## Constructor +```lua +fifo = (require "fifo").new() +``` + +## fifo.dequeue() +#### Syntax +`fifo:dequeue(k)` + +Fetch an element from the fifo and pass it to the function `k`, together with a +boolean indicating whether this is the last element in the fifo. If the fifo +is empty, `k` will not be called and the fifo will enter "immediate dequeue" +mode (see below). + +Assuming `k` is called, ordinarily, `k` will return `nil`, which will cause the +element given to `k` to be removed from the fifo and the queue to advance. If, +however, `k` returns a non-`nil` value, that value will replace the element at +the head of the fifo. This may be useful for generators, for example, which +stand in for several elements. + +When `k` returns `nil`, it may also return a boolean as its second result. If +that is `false`, processing ends and `fifo:dequeue` returns. If that is +`true`, the fifo will be advanced again (i.e. `fifo:dequeue(k)` will be *tail +called*). Elements for which `k` returns `nil, true` are called "phantom", as +they cause the fifo to act as though they were not there. Phantom elements are +useful for callback-like behavior as the fifo advances: when `k` sees a phantom +element, it knows that all prior entries in the fifo have been seen, but the +phantom does not necessarily know how to generate the next element of the fifo. + +#### Returns +`true` if the queue contained at least one non-phantom entry, `false` otherwise. + +## fifo.queue() +#### Syntax +`fifo:queue(a,k)` + +Enqueue the element `a` onto the fifo. If `k` is not `nil` and the fifo is in +"immediate dequeue" mode (whence it starts), immediately pass the first element +of the fifo (usually, but not necessarily, `a`) to `k`, as if `fifo:dequeue(k)` +had been called, and exit "immediate dequeue" mode. + +## FIFO Elements + +The elements stored in the FIFO are simply the integer indices of the fifo +table itself, with `1` being the head of the fifo. The depth of the queue for +a given `fifo` is just its table size, i.e. `#fifo`. Direct access to the +elements is strongly discouraged. The number of elements in the fifo is also +unlikely to be of interest; especially, decisions about the fifo's emptiness +should instead be rewritten to use the existing interface, if possible, or may +peek a bit at the immediate dequeueing state (see below). See the discussion +of corking, below, too. + +## Immediate Dequeueing + +The "immediate dequeue" behavior may seem counterintuitive, but it is very +useful for the case that `fifo:dequeue`'s `k` arranges for subsequent +invocations of `fifo:dequeue`, say by scheduling the next invocation of a timer +or by sending on a socket with an `on("sent")` callback wired to +`fifo:dequeue`. + +Because the fifo enters "immediate dequeue" mode only when `dequeue` has been +called and the fifo was empty at the time of the call, rather than when the +fifo *becomes* empty, `fifo:queue` will sometimes not invoke its `k` even if +the queued element `a` ends up at the front of the fifo. This, too, is quite +useful: it ensures that `k` will not be called in contexts where it would +overlap any ongoing processing of the most-recently dequeued, fifo-emptying +element. + +The immediate deququeing status of the fifo is visible as the `_go` member, +which may be read (even if said reads are politely discouraged, but on occasion +it is handy to know) but should never be written. + +## Corking + +The fifo has no special support for corking (that is, queueing several elements +which are guaranteed to not be dequeued until some later point, called +"uncorking"). As one often wants to cork only when the fifo is transitioning +out of immediate deququeing mode, the existing machinery is generally good +enough to provide an easy emulation thereof. While it is typical to pass the +same `k` to both `:queue` and `:dequeue`, there is nothing necessitating this +convention. And so one may, as in the `fifosock` module, use the `:queue` +`k` to record the transition out of immediate dequeueing mode for later, +when one wishes to uncork: + +```lua +local corked = false +fifo:queue(e1, function(e) corked = true ; return e end) + -- e1 is now in the fifo, and corked is true if the fifo has exited + -- immediate dequeue mode. e1 will be returned back to the fifo and + -- so will not be deququed by the function argument. + + -- We can now queue more elements to the fifo. These will certainly + -- queue behind e1. +fifo:queue(e2) + + -- If we should have initiated draining the fifo above, we can do so now, + -- instead, having built up a backlog as desired. +if corked then fifo:dequeue(k) end +``` diff --git a/docs/lua-modules/fifosock.md b/docs/lua-modules/fifosock.md new file mode 100644 index 00000000..38d69dbd --- /dev/null +++ b/docs/lua-modules/fifosock.md @@ -0,0 +1,70 @@ +# fifosock Module +| Since | Origin / Contributor | Maintainer | Source | +| :----- | :-------------------- | :---------- | :------ | +| 2019-02-10 | [TerryE](https://github.com/TerryE) | [nwf](https://github.com/nwf) | [fifosock.lua](../../lua_modules/fifo/fifosock.lua) | + +This module provides a convenient, efficient wrapper around the `net.socket` +`send` method. It ensures in-order transmission while striving to minimize +memory footprint and packet count by coalescing queued strings. It also serves +as a detailed, worked example of the `fifo` module. + +## Use +```lua +ssend = (require "fifosock")(sock) + +ssend("hello, ") ssend("world\n") +``` + +Once the `sock`et has been wrapped, one should use only the resulting `ssend` +function in lieu of `sock:send`, and one should not change the +`sock:on("sent")` callback. + +### Advanced Use + +In addition to passing strings representing part of the stream to be sent, it +is possible to pass the resulting `ssend` function *functions*. These +functions will be given no parameters, but should return two values: + +- A string to be sent on the socket, or `nil` if no output is desired +- A replacement function, or `nil` if the function is to be dequeued. Functions + may, of course, offer themselves as their own replacement to stay at the front + of the queue. + +This facility is useful for providing a replacement for the `sock:on("sent")` +callback channel. In the fragment below, "All sent" will be `print`ed only +when the entirety of "hello, world\n" has been successfully sent on the +`sock`et. + +```lua +ssend("hello, ") ssend("world\n") +ssend(function() print("All sent") end) -- implicitly returns nil, nil +``` + +This facility is also useful for *generators* of the stream, roughly akin to +`sendfile`-like primitives in larger systems. Here, for example, we can stream +SPIFFS data across the network without ever holding a large amount in RAM. + +```lua +local function sendfile(fn) + local offset = 0 + local function send() + local f = file.open(fn, "r") + if f and f:seek("set", offset) then + r = f:read() + f:close() + if r then + offset = offset + #r + return r, send + end + end + -- implicitly returns nil, nil and falls out of the stream + end + return send, function() return offset end +end + +local fn = "test" +ssend(("Sending file '%s'...\n"):format(fn)) +dosf, getsent = sendfile(fn) +ssend(dosf) +ssend(("Sent %d bytes from '%s'\n"):format(getsent(), fn)) +``` diff --git a/docs/lua-modules/httpserver.md b/docs/lua-modules/httpserver.md index fccf7b46..957bac04 100644 --- a/docs/lua-modules/httpserver.md +++ b/docs/lua-modules/httpserver.md @@ -32,7 +32,8 @@ Function to start HTTP server. #### Notes Callback function has 2 arguments: `req` (request) and `res` (response). The first object holds values: -- `conn`: `net.socket` sub module +- `conn`: `net.socket` sub module. **DO NOT** call `:on` or `:send` on this + object. - `method`: Request method that was used (e.g.`POST` or `GET`) - `url`: Requested URL - `onheader`: value to setup handler function for HTTP headers like `content-type`. Handler function has 3 parameters: diff --git a/lua_examples/telnet/telnet.lua b/lua_examples/telnet/telnet.lua index db5a1ff7..e3d3f05a 100644 --- a/lua_examples/telnet/telnet.lua +++ b/lua_examples/telnet/telnet.lua @@ -26,113 +26,28 @@ would exceed 256 bytes. Once over this threashold, the contents of the FIFO are concatenated into a 2nd level FIFO entry of upto 256 bytes, and the 1st level FIFO cleared down to any residue. -The sender dumps the 2nd level FIFO aggregating records up to 1024 bytes and once this -is empty dumps an aggrate of the 1st level. - ]] local node, table, tmr, wifi, uwrite, tostring = node, table, tmr, wifi, uart.write, tostring local function telnet_listener(socket) - local insert, remove, concat, heap, gc = - table.insert, table.remove, table.concat, node.heap, collectgarbage - local fifo1, fifo1l, fifo2, fifo2l = {}, 0, {}, 0 - local s -- s is a copy of the TCP socket if and only if sending is in progress - - local wdclr, cnt = tmr.wdclr, 0 - local function debug(fmt, ...) - if (...) then fmt = fmt:format(...) end - uwrite(0, "\r\nDBG: ",fmt,"\r\n" ) - cnt = cnt + 1 - if cnt % 10 then wdclr() end - end - - local function flushGarbage() - if heap() < 13440 then gc() end - end - - local function sendLine() - -- debug("entering sendLine") - if not s then return end - - if fifo2l + fifo1l == 0 then -- both FIFOs empty, so clear down s - s = nil - -- debug("Q cleared") - return - end - - flushGarbage() - - if #fifo2 < 4 then -- Flush FIFO1 into FIFO2 - insert(fifo2,concat(fifo1)) - -- debug("flushing %u bytes / %u recs of FIFO1 into FIFO2[%u]", fifo1l, #fifo1, #fifo2) - fifo2l, fifo1, fifo1l = fifo2l + fifo1l, {}, 0 - end - - -- send out first 4 FIFO2 recs (or all if #fifo2<5) - local rec = remove(fifo2,1) .. (remove(fifo2,1) or '') .. - (remove(fifo2,1) or '') .. (remove(fifo2,1) or '') - fifo2l = fifo2l - #rec - - flushGarbage() - s:send(rec) - -- debug( "sending %u bytes (%u buffers remain)\r\n%s ", #rec, #fifo2, rec) - end - local F1_SIZE = 256 - local function queueLine(str) - -- Note that this algo does work for strings longer than 256 but it is sub-optimal - -- as it does string splitting, but this isn't really an issue IMO, as in practice - -- fields of this size are very infrequent. - - -- debug("entering queueLine(l=%u)", #str) - - while #str > 0 do -- this is because str might be longer than the packet size! - local k, l = F1_SIZE - fifo1l, #str - local chunk - - -- Is it time to batch up and flush FIFO1 into a new FIFO2 entry? Note that it's - -- not worth splitting a string to squeeze the last ounce out of a buffer size. - - -- debug("#fifo1 = %u, k = %u, l = %u", #fifo1, k, l) - if #fifo1 >= 32 or (k < l and k < 16) then - insert(fifo2, concat(fifo1)) - -- debug("flushing %u bytes / %u recs of FIFO1 into FIFO2[%u]", fifo1l, #fifo1, #fifo2) - fifo2l, fifo1, fifo1l, k = fifo2l + fifo1l, {}, 0, F1_SIZE - end - - if l > k+16 then -- also tolerate a size overrun of 16 bytes to avoid a split - chunk, str = str:sub(1,k), str:sub(k+1) - else - chunk, str = str, '' - end - - -- debug("pushing %u bytes into FIFO1[l=%u], %u bytes remaining", #chunk, fifo1l, #str) - insert(fifo1, chunk) - fifo1l = fifo1l + #chunk - end - - if not s and socket then - s = socket - sendLine() - else - flushGarbage() - end - - end + local queueLine = (require "fifosock")(socket) local function receiveLine(s, line) - -- debug( "received: %s", line) node.input(line) end local function disconnect(s) - fifo1, fifo1l, fifo2, fifo2l, s = {}, 0, {}, 0, nil + socket:on("disconnection", nil) + socket:on("reconnection", nil) + socket:on("connection", nil) + socket:on("receive", nil) + socket:on("sent", nil) node.output(nil) end socket:on("receive", receiveLine) socket:on("disconnection", disconnect) - socket:on("sent", sendLine) node.output(queueLine, 0) print(("Welcome to NodeMCU world (%d mem free, %s)"):format(node.heap(), wifi.sta.getip())) end diff --git a/lua_modules/fifo/fifo.lua b/lua_modules/fifo/fifo.lua new file mode 100644 index 00000000..d21b1dab --- /dev/null +++ b/lua_modules/fifo/fifo.lua @@ -0,0 +1,45 @@ +-- A generic fifo module. See docs/lua-modules/fifo.md for use examples. + +local tr, ti = table.remove, table.insert + +-- Remove an element and pass it to k, together with a boolean indicating that +-- this is the last element in the queue; if that returns a value, leave that +-- pending at the top of the fifo. +-- +-- If k returns nil, the fifo will be advanced. Moreover, k may return a +-- second result, a boolean, indicating "phantasmic" nature of this element. +-- If this boolean is true, then the fifo will advance again, passing the next +-- value, if there is one, to k, or priming itself for immediate execution at +-- the next call to queue. +-- +-- If the queue is empty, do not invoke k but flag it to enable immediate +-- execution at the next call to queue. +-- +-- Returns 'true' if the queue contained at least one non-phantom entry, +-- 'false' otherwise. +local function dequeue(q,k) + if #q > 0 + then + local new, again = k(q[1], #q == 1) + if new == nil + then tr(q,1) + if again then return dequeue(q, k) end -- note tail call + else q[1] = new + end + return true + else q._go = true ; return false + end +end + +-- Queue a on queue q and dequeue with `k` if the fifo had previously emptied. +local function queue(q,a,k) + ti(q,a) + if k ~= nil and q._go then q._go = false; dequeue(q, k) end +end + +-- return a table containing just the FIFO constructor +return { + ['new'] = function() + return { ['_go'] = true ; ['queue'] = queue ; ['dequeue'] = dequeue } + end +} diff --git a/lua_modules/fifo/fifosock.lua b/lua_modules/fifo/fifosock.lua new file mode 100644 index 00000000..7a149a1c --- /dev/null +++ b/lua_modules/fifo/fifosock.lua @@ -0,0 +1,134 @@ +-- Wrap a two-staged fifo around a socket's send; see +-- docs/lua-modules/fifosock.lua for more documentation. +-- +-- See fifosocktest.lua for some examples of use or tricky cases. +-- +-- Our fifos can take functions; these can be useful for either lazy +-- generators or callbacks for parts of the stream having been sent. + +local BIGTHRESH = 256 -- how big is a "big" string? +local SPLITSLOP = 16 -- any slop in the big question? +local FSMALLLIM = 32 -- maximum number of small strings held +local COALIMIT = 3 + +local concat = table.concat +local insert = table.insert +local gc = collectgarbage + +local fifo = require "fifo" + +return function(sock) + -- the two fifos + local fsmall, lsmall, fbig = {}, 0, fifo.new() + + -- ssend last aggregation string and aggregate count + local ssla, sslan = nil, 0 + local ssend = function(s,islast) + local ns = nil + + -- Optimistically, try coalescing FIFO dequeues. But, don't try to + -- coalesce function outputs, since functions might be staging their + -- execution on the send event implied by being called. + + if type(s) == "function" then + if sslan ~= 0 then + sock:send(ssla) + ssla, sslan = nil, 0; gc() + return s, false -- stay as is and wait for :on("sent") + end + s, ns = s() + elseif type(s) == "string" and sslan < COALIMIT then + if sslan == 0 + then ssla, sslan = s, 1 + else ssla, sslan = ssla .. s, sslan + 1 + end + if islast then + -- this is shipping; if there's room, steal the small fifo, too + if sslan < COALIMIT then + sock:send(ssla .. concat(fsmall)) + fsmall, lsmall = {}, 0 + else + sock:send(ssla) + end + ssla, sslan = "", 0; gc() + return nil, false + else + return nil, true + end + end + + -- Either that was a function or we've hit our coalescing limit or + -- we didn't ship above. Ship now, if there's something to ship. + if s ~= nil then + if sslan == 0 then sock:send(s) else sock:send(ssla .. s) end + ssla, sslan = nil, 0; gc() + return ns or nil, false + elseif sslan ~= 0 then + assert (ns == nil) + sock:send(ssla) + ssla, sslan = nil, 0; gc() + return nil, false + else + assert (ns == nil) + return nil, true + end + end + + -- Move fsmall to fbig; might send if fbig empty + local function promote(f) + if #fsmall == 0 then return end + local str = concat(fsmall) + fsmall, lsmall = {}, 0 + fbig:queue(str, f or ssend) + end + + local function sendnext() + if not fbig:dequeue(ssend) then promote() end + end + + sock:on("sent", sendnext) + + return function(s) + -- don't sweat the petty things + if s == nil or s == "" then return end + + -- Function? Go ahead and queue this thing in the right place. + if type(s) == "function" then promote(); fbig:queue(s, ssend); return; end + + s = tostring(s) + + -- cork sending until the end in case we're the head of line + local corked = false + local function corker(t) corked = true; return t end + + -- small fifo would overfill? promote it + if lsmall + #s > BIGTHRESH or #fsmall >= FSMALLLIM then promote(corker) end + + -- big string? chunk and queue big components immediately + -- behind any promotion that just took place + while #s > BIGTHRESH + SPLITSLOP do + local pfx + pfx, s = s:sub(1,BIGTHRESH), s:sub(BIGTHRESH+1) + fbig:queue(pfx, corker) + end + + -- Big string? queue and maybe tx now + if #s > BIGTHRESH then fbig:queue(s, corker) + -- small and fifo in immediate dequeue mode + elseif fbig._go and lsmall == 0 then fbig:queue(s, corker) + -- small and queue already moving; let it linger in the small fifo + else insert(fsmall, s) ; lsmall = lsmall + #s + end + + -- if it happened that we corked the transmission above... + -- if we queued a good amount of data, go ahead and start transmitting; + -- otherwise, wait a tick and hopefully we will queue more in the interim + -- before transmitting. + if corked then + if #fbig <= COALIMIT + then tmr.create():alarm(1, tmr.ALARM_SINGLE, sendnext) + else sendnext() + end + end + end +end diff --git a/lua_modules/fifo/fifosocktest.lua b/lua_modules/fifo/fifosocktest.lua new file mode 100644 index 00000000..54aeb8d6 --- /dev/null +++ b/lua_modules/fifo/fifosocktest.lua @@ -0,0 +1,141 @@ +-- +-- Set verbose to 0 for quiet output (either the first assertion failure or +-- "All tests OK"), to 1 to see the events ("SEND", "SENT", "CHECK") without +-- the actual bytes, or to 2 to see the events with the bytes. +-- +local verbose = 0 + +local vprint = (verbose > 0) and print or function() end + +-- +-- Mock up enough of the nodemcu tmr structure, but pretend that nothing +-- happens between ticks. This won't exercise the optimistic corking logic, +-- but that's probably fine. +-- +tmr = {} +tmr.ALARM_SINGLE = 0 +function tmr.create() + local r = {} + function r:alarm(_i, _t, cb) vprint("TMR") cb() end + return r +end + +-- +-- Mock up enough of the nodemcu net.socket type; have it log all the sends +-- into this "outs" array so that we can later check against it. +-- +local outs = {} +local fakesock = { + cb = nil, + on = function(this, _, cb) this.cb = cb end, + send = function(this, s) vprint("SEND", (verbose > 1) and s) table.insert(outs, s) end, +} +local function sent() vprint("SENT") fakesock.cb() end + +-- And wrap a fifosock around this fake socket +local fsend = require "fifosock" (fakesock) + +-- Verify that the next unconsumed output is as indicated +local function fcheck(x) + vprint ("CHECK", (verbose > 1) and x) + assert (#outs > 0) + assert (x == outs[1]) + table.remove(outs, 1) +end + +-- Enqueue an empty function to prevent coalescing within the fifosock +local function nocoal() fsend(function() return nil end) end + +-- Send and check, for when the string should be sent exactly as is +local function fsendc(x) fsend(x) fcheck(x) end + +-- Check that there are no more outputs +local function fchecke() vprint("CHECKE") assert (#outs == 0) end + +-- +-- And now for the tests, which start easy and grow in complexity +-- + +fsendc("abracadabra none") +sent() ; fchecke() + +fsendc("abracadabra three") +fsend("short") +fsend("string") +fsend("build") +sent() ; fcheck("shortstringbuild") +sent() ; fchecke() + +-- Hit default FSMALLLIM while building up +fsendc("abracadabra lots small") +for i = 1, 32 do fsend("a") end +nocoal() +for i = 1, 4 do fsend("a") end +sent() ; fcheck(string.rep("a", 32)) +sent() ; fcheck(string.rep("a", 4)) +sent() ; fchecke() + +-- Hit string length while building up +fsendc("abracadabra overlong") +for i = 1, 10 do fsend(string.rep("a",32)) end +sent() ; fcheck(string.rep("a", 320)) +sent() ; fchecke() + +-- Hit neither before sending a big string +fsendc("abracadabra mid long") +for i = 1, 6 do fsend(string.rep("a",32)) end +fsend(string.rep("b", 256)) +nocoal() +for i = 1, 6 do fsend(string.rep("c",32)) end +sent() ; fcheck(string.rep("a", 192) .. string.rep("b", 256)) +sent() ; fcheck(string.rep("c", 192)) +sent() ; fchecke() + +-- send a huge string, verify that it coalesces +fsendc(string.rep("a",256) .. string.rep("b", 256) .. string.rep("c", 260)) +sent() ; fchecke() + +-- send a huge string, verify that it coalesces save for the short bit at the end +fsend(string.rep("a",256) .. string.rep("b", 256) .. string.rep("c", 256) .. string.rep("d",256)) +fsend("e") +fcheck(string.rep("a",256) .. string.rep("b", 256) .. string.rep("c", 256)) +sent() ; fcheck(string.rep("d",256) .. "e") +sent() ; fchecke() + +-- send enough that our 4x lookahead still leaves something in the queue +fsend(string.rep("a",512) .. string.rep("b", 512) .. string.rep("c", 512)) +fcheck(string.rep("a",512) .. string.rep("b", 512)) +sent() ; fcheck(string.rep("c",512)) +sent() ; fchecke() + +-- test a lazy generator +local ix = 0 +local function gen() vprint("GEN", ix); ix = ix + 1; return ("a" .. ix), ix < 3 and gen end +fsend(gen) +fsend("b") +fcheck("a1") +sent() ; fcheck("a2") +sent() ; fcheck("a3") +sent() ; fcheck("b") +sent() ; fchecke() + +-- test a completion-like callback that does send text +local ix = 0 +local function gen() vprint("GEN"); ix = 1; return "efgh", nil end +fsend("abcd"); fsend(gen); fsend("ijkl") +assert (ix == 0) + fcheck("abcd"); assert (ix == 0) +sent() ; fcheck("efgh"); assert (ix == 1); ix = 0 +sent() ; fcheck("ijkl"); assert (ix == 0) +sent() ; fchecke() + +-- and one that doesn't +local ix = 0 +local function gen() vprint("GEN"); ix = 1; return nil, nil end +fsend("abcd"); fsend(gen); fsend("ijkl") +assert (ix == 0) + fcheck("abcd"); assert (ix == 0) +sent() ; fcheck("ijkl"); assert (ix == 1); ix = 0 +sent() ; fchecke() ; assert (ix == 0) + +print("All tests OK") diff --git a/lua_modules/http/http-example.lua b/lua_modules/http/http-example.lua index 3080dcb6..ea03a46e 100644 --- a/lua_modules/http/http-example.lua +++ b/lua_modules/http/http-example.lua @@ -9,7 +9,7 @@ require("httpserver").createServer(80, function(req, res) print("+R", req.method, req.url, node.heap()) -- setup handler of headers, if any req.onheader = function(self, name, value) - -- print("+H", name, value) + print("+H", name, value) -- E.g. look for "content-type" header, -- setup body parser to particular format -- if name == "content-type" then @@ -23,13 +23,11 @@ require("httpserver").createServer(80, function(req, res) -- setup handler of body, if any req.ondata = function(self, chunk) print("+B", chunk and #chunk, node.heap()) - -- request ended? if not chunk then -- reply - --res:finish("") res:send(nil, 200) res:send_header("Connection", "close") - res:send("Hello, world!") + res:send("Hello, world!\n") res:finish() end end diff --git a/lua_modules/http/httpserver.lua b/lua_modules/http/httpserver.lua index 9f1e3f25..8455c688 100644 --- a/lua_modules/http/httpserver.lua +++ b/lua_modules/http/httpserver.lua @@ -12,27 +12,24 @@ do -- request methods ------------------------------------------------------------------------------ local make_req = function(conn, method, url) - local req = { + return { conn = conn, method = method, url = url, } - -- return setmetatable(req, { - -- }) - return req end ------------------------------------------------------------------------------ -- response methods ------------------------------------------------------------------------------ - local send = function(self, data, status) - local c = self.conn + local make_res = function(csend, cfini) + local send = function(self, data, status) -- TODO: req.send should take care of response headers! if self.send_header then - c:send("HTTP/1.1 ") - c:send(tostring(status or 200)) + csend("HTTP/1.1 ") + csend(tostring(status or 200)) -- TODO: real HTTP status code/name table - c:send(" OK\r\n") + csend(" OK\r\n") -- we use chunked transfer encoding, to not deal with Content-Length: -- response header self:send_header("Transfer-Encoding", "chunked") @@ -43,55 +40,51 @@ do if self.send_header then self.send_header = nil -- end response headers - c:send("\r\n") + csend("\r\n") end -- chunked transfer encoding - c:send(("%X\r\n"):format(#data)) - c:send(data) - c:send("\r\n") + csend(("%X\r\n"):format(#data)) + csend(data) + csend("\r\n") end - end - local send_header = function(self, name, value) - local c = self.conn + end + local send_header = function(self, name, value) -- NB: quite a naive implementation - c:send(name) - c:send(": ") - c:send(value) - c:send("\r\n") - end - -- finalize request, optionally sending data - local finish = function(self, data, status) - local c = self.conn - -- NB: req.send takes care of response headers + csend(name) + csend(": ") + csend(value) + csend("\r\n") + end + -- finalize request, optionally sending data + local finish = function(self, data, status) + -- NB: res.send takes care of response headers if data then self:send(data, status) end -- finalize chunked transfer encoding - c:send("0\r\n\r\n") + csend("0\r\n\r\n") -- close connection - c:close() - end - -- - local make_res = function(conn) - local res = { - conn = conn, - } - -- return setmetatable(res, { - -- send_header = send_header, - -- send = send, - -- finish = finish, - -- }) + cfini() + end + -- + local res = { } res.send_header = send_header res.send = send res.finish = finish return res end - ------------------------------------------------------------------------------ + ------------------------------------------------------------------------------ -- HTTP parser ------------------------------------------------------------------------------ local http_handler = function(handler) return function(conn) + local csend = (require "fifosock")(conn) + local cfini = function() + conn:on("receive", nil) + conn:on("disconnection", nil) + csend(function() conn:on("sent", nil) conn:close() end) + end local req, res local buf = "" local method, url @@ -108,7 +101,7 @@ do cnt_len = tonumber(v) end if k == "expect" and v == "100-continue" then - conn:send("HTTP/1.1 100 Continue\r\n") + csend("HTTP/1.1 100 Continue\r\n") end -- delegate to request object if req and req.onheader then @@ -118,8 +111,6 @@ do -- body data handler local body_len = 0 local ondata = function(conn, chunk) - -- NB: do not reset node in case of lengthy requests - tmr.wdclr() -- feed request data to request handler if not req or not req.ondata then return end req:ondata(chunk) @@ -153,8 +144,10 @@ do if method then -- make request and response objects req = make_req(conn, method, url) - res = make_res(conn) + res = make_res(csend, cfini) end + -- spawn request handler + handler(req, res) -- header line? elseif #line > 0 then -- parse header @@ -166,11 +159,6 @@ do end -- headers end else - -- spawn request handler - -- NB: do not reset in case of lengthy requests - tmr.wdclr() - handler(req, res) - tmr.wdclr() -- NB: we feed the rest of the buffer as starting chunk of body ondata(conn, buf) -- buffer no longer needed diff --git a/mkdocs.yml b/mkdocs.yml index 54b25627..4b2b1f2c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -41,6 +41,8 @@ pages: - 'bh1750': 'lua-modules/bh1750.md' - 'ds18b20': 'lua-modules/ds18b20.md' - 'ds3231': 'lua-modules/ds3231.md' + - 'fifo' : 'lua-modules/fifo.md' + - 'fifosock' : 'lua-modules/fifosock.md' - 'ftpserver': 'lua-modules/ftpserver.md' - 'hdc1000': 'lua-modules/hdc1000.md' - 'httpserver': 'lua-modules/httpserver.md'