mgr's weblog

Lisp Archives for February 2021

"Curl/Wget for uLisp"
Or: An HTTP(s) get/post/put function for uLisp

February 23, 2021, Lisp
Last edited on July 26, 2024

Oh, I forgot to continue posting… I just published a quite comprehensive HTTP function supporting put, post, get, auth, HTTP and HTTPS, and more for uLisp at ulisp-esp-m5stack.

Activate #define enable_http and #define enable_http_keywords to get it; the keywords used by the http function are to be enabled separately as they might be used more general and not just by this function.

Note that you need to connect to the internet first. Usually with WIFI-CONNECT.

Here is the full documentation with example calls:

Function http
 
Syntax:
   http url &key verbose
                 (https t)
                 auth
                 (user default_username)
                 (password default_password)
                 accept
                 content-type
                 (method :get)
                 data
     => result-string

Arguments and values:
   verbose---t, or nil (the default); affects also debug output of the argument decoding itself and should be put in first position in a call for full effect.

   https---t (the default), nil, or a certificate as string; uses default certificate in C string root_ca if true; url needs to fit: "http://..." for true and and "https://..." for false.

   auth---t, or nil (the default).

   user---a string, or nil (the default); uses default value in C string default_username if nil; only used if :auth t.

   password---a string, or nil (the default); uses default value in C string default_password if nil; only used if :auth t.

   accept---nil (the default), or a string.

   content-type---nil (the default), or a string.

   method---:get (the default), :put, or :post.

   data---nil (the default), or a string; only necessary in case of :method :put or :method :post; error for :method :get.

Examples:
   ;; HTTP GET:
   (http "http://192.168.179.41:2342" :https nil)
 
   ;; HTTP PUT:
   (http "http://192.168.179.41:2342"
         :https nil
         :accept "application/n-quads"
         :content-type "application/n-quads"
         :auth t :user "foo" :password "bar"
         :method :put
         :data (format nil "<http://example.com/button> <http://example.com/pressed> \"~a\" .~%"
                           (get-time)))

It can be tested with an minimal HTTP server simulation using bash and netcat:

while true; do echo -e "HTTP/1.1 200 OK\n\n $(date)" | nc -l -p 2342 -q 1; done
(To test with HTTPS in a similar fashion you can use openssl s_server, as explained, for example, in the article Create a simple HTTPS server with OPENSSL S_SERVER by Joris Visscher on July 22, 2015, but then you need to use certificates.)


See also Again more features for uLisp on M5Stack (ESP32):
time via NTP, lispstring without escaping and more space
, More features for uLisp on M5Stack (ESP32):
flash support, muting of the speaker and backlight control
and uLisp on M5Stack (ESP32).

Again more features for uLisp on M5Stack (ESP32):
time via NTP, lispstring without escaping and more space

February 16, 2021, Lisp
Last edited on July 26, 2024

I just pushed three small things to ulisp-esp-m5stack: Get time via NTP, add optional escape parameter to function lispstring, increased WORKSPACESIZE and SYMBOLTABLESIZE for M5Stack.

Getting time via NTP

Enable #define enable_ntptime to get time via NTP. New functions INIT-NTP and GET-TIME. Note that you need to connect to the internet first. Usually with WIFI-CONNECT.

Function init-ntp
 
Syntax:
   init-ntp
     => nil

Description:
   Initializes and configures NTP.

Function get-time
 
Syntax:
   get-time
     => timestamp

Arguments and values:
   timestamp---a string; containing a timestamp in the format of xsd:dateTime.

Description:
   Returns a timestamp in the format of xsd:dateTime.

Add optional escape parameter to function lispstring

I have changed the function lispstring to have an optional escape parameter to switch off the default behavior of handling the backslash escape character. The default behavior is not changed.

The C function lispstring takes a C char* string and return a uLisp string object. When parsing data in the n-triples format retrieved via HTTP I noticed that the data got modified already by lispstring which broke my parser implemented in uLisp.

As lispstring might be used in other contexts that expect this behavior, I just added the option to switch the un-escaping off.

Increased WORKSPACESIZE and SYMBOLTABLESIZEfor M5Stack

The M5Stack ESP32 has 320 kB of usable DRAM in total. Although with a lot of restrictions (see next section),

I increased WORKSPACESIZE to 9000 cells, which equals 72,000 bytes, and SYMBOLTABLESIZE to 2048 bytes. These sizes seem to work still safely even with bigger applications and a lot of consing.

Warning: You cannot load images created with different settings!

The SRAM of the M5Stack ESP32

In total the M5Stack ESP32 comes with 520 kB of SRAM. The catch is that the ESP32 is based on the Harvard architecture and 192 kB is in the SRAM0 block intended(!) for instructions (IRAM). There is another 128 kB block in block SRAM1 which can be used either for instructions or data (DRAM). The third block SRAM2 has got a size of 200 kB and is for data only. But 8 kB of SRAM2 is lost for ROM mappings.

The ESP-IDF and thus also the Arduino environment use only SRAM0 for instructions and SRAM1 and SRAM2 for data, which is fine for uLisp as it is an interpreter and therefore more RAM for data is perfect. SRAM0 will just hold the machine code of the uLisp implementation but no code written in the language uLisp.

Of the remaining 320 kB another 54 kB will be dedicated for Bluetooth if Bluetooth is enabled in ESP-IDF (which it is by default, #define CONFIG_BT_RESERVE_DRAM 0xdb5c) in the SRAM2 block. And if trace memory is enabled, another 32 kB of SRAM1 are reserved (by default it is disabled, #define CONFIG_TRACEMEM_RESERVE_DRAM 0x0).

So, by default with Bluetooth enabled and trace memory disabled, 266 kB are left. At the bottom of SRAM2 right after the 62 kB used for Bluetooth and ROM are the application's data and BSS segments. Sadly, at around the border between SRAM1 and SRAM2 there seem to be two small reserved regions again of a bit more the 1 kB each, limiting statically allocated memory.

Thus, the "shared data RAM" segment dram0_0_seg in the linker script memory layout is configured to have a default length of 0x2c200 - CONFIG_BT_RESERVE_DRAM. That is, 176.5 kB (= 180,736 bytes) without Bluetooth and 121.66 kB (= 124,580 bytes) with Bluetooth enabled.

But actually I have already written more than I have intended for this blog post and the rest of my notes, calculations and experiments will have to wait for a future article. For now, I just increased the size of the statically allocated uLisp workspace to make more use of the available memory of the ESP32 in the M5Stack.


See also More features for uLisp on M5Stack (ESP32):
flash support, muting of the speaker and backlight control
and uLisp on M5Stack (ESP32).

References

Espressif Systems, ESP32 Technical Reference Manual, Shanghai, 2020, section 2.3.2 Embedded Memory.

More features for uLisp on M5Stack (ESP32):
flash support, muting of the speaker and backlight control

February 15, 2021, Lisp
Last edited on February 23, 2021

I finished the IoT sensor device prototype and shipped it last Thursday. It just has a stub bootstrap system in the flash via uLisp's Lisp Library and downloads the actual application in a second boot phase via HTTPS. More on that later.

To make it happen I've added a bunch of things to ulisp-esp-m5stack: flash support, fixes for some quirks of the M5Stack, time via NTP, an HTTP function supporting methods PUT, POST, GET, Auth, HTTP and HTTP, temperature sensors via one wire, and more. I plan to publish all these features in the next days.

Today you get: flash support, muting of the builtin speaker and control of the LED backlight of the builtin display.

Flash support

My M5Stacks all have got 4 MB of flash memory attached via SPI. Newer version seem to come with 16 MB. The original ESP32 version of uLisp already uses flash to store workspace images with SAVE-IMAGE but it uses Arduino's EEPROM emulation that is limited to only use 4 kB (of the whole 4 MB). Now you can increase your workspace size and still store images without having to insert an SD card. If you want it, just activate the line #define flashsupport.

To add this was simple as the interface to use SD cards is the same as for flash. Most of the code to support flash is shared between both. The flash memory will be formatted with SPIFFS (SPI flash file system). When I decode the binary partition table that the Arduino software creates when compiling uLisp with the command:

python gen_esp32part.py ulisp-esp.ino.partitions.bin ulisp-esp.ino.partitions.csv
I get:

# Espressif ESP32 Partition Table
# Name, Type, SubType, Offset, Size, Flags
nvs,data,nvs,0x9000,20K,
otadata,data,ota,0xe000,8K,
app0,app,ota_0,0x10000,1280K,
app1,app,ota_1,0x150000,1280K,
spiffs,data,spiffs,0x290000,1472K,
So there is 1,472 kb free in the SPIFFS which is certainly enough for some workspace images. And maybe some other files. If you need more, you can still use an SD card.

Muting of the builtin speaker

The M5Stack / M5Core Basic comes with a little 1W speaker connected to analog PIN 25. If you don't configure the PIN and set it to zero, the speaker gives a little chirping sound when you send data to other devices and it clicks when you blank the display. This can be quite annoying.

The function MUTE-SPEAKER does this little configuration task and the nuisance stops. There is also the define flag #define m5stack_boot_mute which mutes the speaker on startup.

Brightness of the backlight

The builtin screen of the M5Stack / M5Core Basic has an LED backlight connected to pin 32 and the intensity can be controlled via PWM channel 7. You can change the birghtness with the new function SETUP-BACKLIGHT-PWM:

Syntax: setup-backlight-pwm [brightness] => nil

Brightness between 0-255: 0 is off, 255 brightest, 80 (the default) offers a good compromise: good readability for inside rooms. Set to small value to conserve power.

M5Display::begin already does such a setup with a default brightness value of 80.


See also uLisp on M5Stack (ESP32).

StumpWM: vsplit-three

February 7, 2021, Lisp
Last edited on July 26, 2024

A good ten month ago I switched away from a full desktop environment being finally tired enough that user software gets more and ever more features and tries to anticipate more and more what I might want but in the end my own computer never actually does what I want and only that. PulseAudio being the most dreaded example of a piece of code that gets more, more and more magic and complexity and in the end it never does what you actually want while at the same time telling it to do so got completely impossible because of many layers of abstractions and magic. PulseAudio has a lot of "rules", "profiles", "device intended roles", "autodetecting", "automatic setup and routing" and "other housekeeping actions". Look at this article PulseAudio under the hood by Victor Gaydov (which is also the source of the terms I just quoted): It has 174 occurrences of words starting with "auto-": automatically – 106, automatic – 27, autoload – 16, autospawn – 14, autodetect – 4, autoexit – 2, automate – 2, auto timing – 1, auto switch – 1, and once "magically" when it is even too much for the author.

So, more control and less clutter instead. After years again I use just a good old window manager, individual programs, and got rid of PulseAudio.

I switched to StumpWM which is written in Common Lisp. It is easy to modify and try stuff. While it's running. I have it run Slime so that I can connect to it from Emacs and hack stuff that is missing. From time to time I got StumpWM hanging while hacking, so I added a signal handler for POSIX signal SIGHUP to force a hard stumpwm restart. (There is a new version of that signal handler without the CFFI dependency but that pull request is not merged yet.) When I did something stupid I switch to a console, fire a killall -HUP stumpwm to have it reset hard. Since then I haven't lost a X11 session even while changing quite a bit. [[rss-mark]]

Right away I wrote some minimalistic modules to control audio volume (stump-volume-control) and to play internet radio stations (stump-radio). They put the multimedia keys or other extra keys on most keyboards to use. Both modules have also been added to stumpwm-contrib.

Note that StumpWM is a tiling window manager and keyboard driven. So no windows with decorations, just frames. But if you really want a window, you can call FLOAT-THIS and the current frame turns into a window. Or you can make a floating group (aka virtual desktop) with GNEW-FLOAT and all windows in that group are actual windows.

Recently I got a new wide 34″ screen with a resolution of 3440x1440 pixels. I noticed that just splitting the screen space vertically into two frames makes the text appear either too far left or too much on the right, and also both are too big. There is VSPLIT-EQUALLY but with the screen equally divided into three columns, the main program in the middle is too narrow.

I want the whole thing more adapted to my taste. To do so I just wrote this little function and command:

(defun vsplit-three* (&key (ratio .3) (group (current-group)) (dir :column))
  (let* ((split-1 (- 1 ratio))
         (split-2 (* ratio (/ 1 split-1))))
    (only)
    (stumpwm::split-frame-in-dir group dir split-1)
    (stumpwm::split-frame-in-dir group dir split-2)
    ratio))
 
(defcommand (vsplit-three stumpwm::tile-group) (&optional (ratio ".3"))
    (:string)
  (vsplit-three* :ratio (read-from-string ratio)))
 
(define-key *root-map* (kbd "O") "vsplit-three")

When called with a default ratio of 3/10 it makes two small frames left and right that take 30 % of the total screen real estate and a middle frame that gets the remaining 40 %. This seems to be good for generic use. A value of 0.27 seems to be better for coding with a slightly bigger main window. And for the web browsing group, I rather have it 25 % – 50 % – 25 % right now.

Update 20240322: Fixed the display of stars in the source code snippet

Thank you Tim Macdonald, for pointing out that the stars were not displayed in the source code of vsplit-three* and vsplit-three above. I made extra sure that the source code is now shown correctly.

Glad you like the function and thanks for your mail!

Select a Theme:

Basilique du Sacré-Cœur de Montmartre (Paris) Parc Floral de Paris Castillo de Santa Barbara (Alicante) About the photos

Entries: