mgr's weblog

Oh, I forgot to continue posting… I just published a quite comprehensive HTTP function supporting put, post, get, auth, HTTP and HTTPS, and more for uLisp at ulisp-esp-m5stack.

Activate #define enable_http and #define enable_http_keywords to get it; the keywords used by the http function are to be enabled separately as they might be used more general and not just by this function.

Note that you need to connect to the internet first. Usually with WIFI-CONNECT.

Here is the full documentation with example calls:

Function http
 
Syntax:
   http url &key verbose
                 (https t)
                 auth
                 (user default_username)
                 (password default_password)
                 accept
                 content-type
                 (method :get)
                 data
     => result-string

Examples:
   ;; HTTP GET:
   (http "http://192.168.179.41:2342" :https nil)
 
   ;; HTTP PUT:
   (http "http://192.168.179.41:2342"
         :https nil
         :accept "application/n-quads"
         :content-type "application/n-quads"
         :auth t :user "foo" :password "bar"
         :method :put
         :data (format nil "<http://example.com/button> <http://example.com/pressed> \"~a\" .~%"
                           (get-time)))

It can be tested with an minimal HTTP server simulation using bash and netcat:

while true; do echo -e "HTTP/1.1 200 OK\n\n $(date)" | nc -l -p 2342 -q 1; done

See also Again more features for uLisp on M5Stack (ESP32):
time via NTP, lispstring without escaping and more space, More features for uLisp on M5Stack (ESP32):
flash support, muting of the speaker and backlight control and uLisp on M5Stack (ESP32).

I just pushed three small things to ulisp-esp-m5stack: Get time via NTP, add optional escape parameter to function lispstring, increased WORKSPACESIZE and SYMBOLTABLESIZE for M5Stack.

Getting time via NTP

Function init-ntp
 
Syntax:
   init-ntp
     => nil

Function get-time
 
Syntax:
   get-time
     => timestamp

Add optional escape parameter to function lispstring

The C function lispstring takes a C char* string and return a uLisp string object. When parsing data in the n-triples format retrieved via HTTP I noticed that the data got modified already by lispstring which broke my parser implemented in uLisp.

As lispstring might be used in other contexts that expect this behavior, I just added the option to switch the un-escaping off.

Increased WORKSPACESIZE and SYMBOLTABLESIZEfor M5Stack

I increased WORKSPACESIZE to 9000 cells, which equals 72,000 bytes, and SYMBOLTABLESIZE to 2048 bytes. These sizes seem to work still safely even with bigger applications and a lot of consing.

Warning: You cannot load images created with different settings!

The SRAM of the M5Stack ESP32

The ESP-IDF and thus also the Arduino environment use only SRAM0 for instructions and SRAM1 and SRAM2 for data, which is fine for uLisp as it is an interpreter and therefore more RAM for data is perfect. SRAM0 will just hold the machine code of the uLisp implementation but no code written in the language uLisp.

Of the remaining 320 kB another 54 kB will be dedicated for Bluetooth if Bluetooth is enabled in ESP-IDF (which it is by default, #define CONFIG_BT_RESERVE_DRAM 0xdb5c) in the SRAM2 block. And if trace memory is enabled, another 32 kB of SRAM1 are reserved (by default it is disabled, #define CONFIG_TRACEMEM_RESERVE_DRAM 0x0).

So, by default with Bluetooth enabled and trace memory disabled, 266 kB are left. At the bottom of SRAM2 right after the 62 kB used for Bluetooth and ROM are the application's data and BSS segments. Sadly, at around the border between SRAM1 and SRAM2 there seem to be two small reserved regions again of a bit more the 1 kB each, limiting statically allocated memory.

Thus, the "shared data RAM" segment dram0_0_seg in the linker script memory layout is configured to have a default length of 0x2c200 - CONFIG_BT_RESERVE_DRAM. That is, 176.5 kB (= 180,736 bytes) without Bluetooth and 121.66 kB (= 124,580 bytes) with Bluetooth enabled.

But actually I have already written more than I have intended for this blog post and the rest of my notes, calculations and experiments will have to wait for a future article. For now, I just increased the size of the statically allocated uLisp workspace to make more use of the available memory of the ESP32 in the M5Stack.

See also More features for uLisp on M5Stack (ESP32):
flash support, muting of the speaker and backlight control and uLisp on M5Stack (ESP32).

References

I finished the IoT sensor device prototype and shipped it last Thursday. It just has a stub bootstrap system in the flash via uLisp's Lisp Library and downloads the actual application in a second boot phase via HTTPS. More on that later.

To make it happen I've added a bunch of things to ulisp-esp-m5stack: flash support, fixes for some quirks of the M5Stack, time via NTP, an HTTP function supporting methods PUT, POST, GET, Auth, HTTP and HTTP, temperature sensors via one wire, and more. I plan to publish all these features in the next days.

Today you get: flash support, muting of the builtin speaker and control of the LED backlight of the builtin display.

Flash support

To add this was simple as the interface to use SD cards is the same as for flash. Most of the code to support flash is shared between both. The flash memory will be formatted with SPIFFS (SPI flash file system). When I decode the binary partition table that the Arduino software creates when compiling uLisp with the command:

python gen_esp32part.py ulisp-esp.ino.partitions.bin ulisp-esp.ino.partitions.csv

# Espressif ESP32 Partition Table
# Name, Type, SubType, Offset, Size, Flags
nvs,data,nvs,0x9000,20K,
otadata,data,ota,0xe000,8K,
app0,app,ota_0,0x10000,1280K,
app1,app,ota_1,0x150000,1280K,
spiffs,data,spiffs,0x290000,1472K,

Muting of the builtin speaker

The function MUTE-SPEAKER does this little configuration task and the nuisance stops. There is also the define flag #define m5stack_boot_mute which mutes the speaker on startup.

Brightness of the backlight

See also uLisp on M5Stack (ESP32).

A good ten month ago I switched away from a full desktop environment being finally tired enough that user software gets more and ever more features and tries to anticipate more and more what I might want but in the end my own computer never actually does what I want and only that. PulseAudio being the most dreaded example of a piece of code that gets more, more and more magic and complexity and in the end it never does what you actually want while at the same time telling it to do so got completely impossible because of many layers of abstractions and magic. PulseAudio has a lot of "rules", "profiles", "device intended roles", "autodetecting", "automatic setup and routing" and "other housekeeping actions". Look at this article PulseAudio under the hood by Victor Gaydov (which is also the source of the terms I just quoted): It has 174 occurrences of words starting with "auto-": automatically – 106, automatic – 27, autoload – 16, autospawn – 14, autodetect – 4, autoexit – 2, automate – 2, auto timing – 1, auto switch – 1, and once "magically" when it is even too much for the author.

So, more control and less clutter instead. After years again I use just a good old window manager, individual programs, and got rid of PulseAudio.

I switched to StumpWM which is written in Common Lisp. It is easy to modify and try stuff. While it's running. I have it run Slime so that I can connect to it from Emacs and hack stuff that is missing. From time to time I got StumpWM hanging while hacking, so I added a signal handler for POSIX signal SIGHUP to force a hard stumpwm restart. (There is a new version of that signal handler without the CFFI dependency but that pull request is not merged yet.) When I did something stupid I switch to a console, fire a killall -HUP stumpwm to have it reset hard. Since then I haven't lost a X11 session even while changing quite a bit. [[rss-mark]]

Right away I wrote some minimalistic modules to control audio volume (stump-volume-control) and to play internet radio stations (stump-radio). They put the multimedia keys or other extra keys on most keyboards to use. Both modules have also been added to stumpwm-contrib.

Note that StumpWM is a tiling window manager and keyboard driven. So no windows with decorations, just frames. But if you really want a window, you can call FLOAT-THIS and the current frame turns into a window. Or you can make a floating group (aka virtual desktop) with GNEW-FLOAT and all windows in that group are actual windows.

Recently I got a new wide 34″ screen with a resolution of 3440x1440 pixels. I noticed that just splitting the screen space vertically into two frames makes the text appear either too far left or too much on the right, and also both are too big. There is VSPLIT-EQUALLY but with the screen equally divided into three columns, the main program in the middle is too narrow.

I want the whole thing more adapted to my taste. To do so I just wrote this little function and command:

(defun vsplit-three* (&key (ratio .3) (group (current-group)) (dir :column))
  (let* ((split-1 (- 1 ratio))
         (split-2 (* ratio (/ 1 split-1))))
    (only)
    (stumpwm::split-frame-in-dir group dir split-1)
    (stumpwm::split-frame-in-dir group dir split-2)
    ratio))
 
(defcommand (vsplit-three stumpwm::tile-group) (&optional (ratio ".3"))
    (:string)
  (vsplit-three* :ratio (read-from-string ratio)))
 
(define-key *root-map* (kbd "O") "vsplit-three")

When called with a default ratio of 3/10 it makes two small frames left and right that take 30 % of the total screen real estate and a middle frame that gets the remaining 40 %. This seems to be good for generic use. A value of 0.27 seems to be better for coding with a slightly bigger main window. And for the web browsing group, I rather have it 25 % – 50 % – 25 % right now.

Update 20240322: Fixed the display of stars in the source code snippet

Thank you Tim Macdonald, for pointing out that the stars were not displayed in the source code of vsplit-three* and vsplit-three above. I made extra sure that the source code is now shown correctly.

Glad you like the function and thanks for your mail!