December Adventure

December Adventure is here and I will be part of it (and hopefully finish the month) by working on Growl, my programming language.

For those not aware, Growl is a statically-typed concatenative language I'm working on. I've rewritten the core like five times at this point and hopefully this will be the last, so stay around to see how it goes!

Some of the main goals with this project are:

• Learning! For example this project taught me how to do type inference for stack languages, with row-polymorphism and all.
• Making something useful that I would use. This is a very lofty goal but I would like to see it done in the future.

I'm using December Adventure mostly as a vehicle to keep myself constant in the work I do and train myself to write about it more often, as I'm really bad at doing technical writing and keeping logs of… everything. ^^"

But also I want to take the opportunity of it being "write a bit of code every day" to also try to write less code. I do a lot of programming in my free time and it's probably about time that I slow down.

About Growl, one of the main points I'm working on is expressing types that would be infinite in Hindley-Milner type inference systems, like the type of dup call, which can't be inferred using HM.

Previously I solved this by adding a fix construct, which does a similar thing and does not require recursive types, but I feel like this is a very "band-aid" like solution, so I want to solve this the right way, if there's any.

Anyways. That's all for today, tomorrow I'll come back with code and stuff to write about it.

I've been working on making the current implementation of Growl amenable to the public before I upload it to my Git forge server. This means ironing out some important bugs — up until now, mutually recursive definitions weren't working! — but also some quality-of-life things, like proper parsing errors, which require some work to get with Menhir (OCaml's de-facto yacc-like.)

I've been also playing with Wasm_of_ocaml to make a playground in the browser for the language, and it's been refreshing how easy it is to make everything just work. As far as I can tell, it heavily uses some of the infrastructure in Js_of_ocaml alongside its own, but it's really cool how I can just export an object with methods and have it work in JS without much fuzzing around. I was going to use Js_ofocaml, but it generates some beefy files; the total bundle size for the playground code was around 2.5 MB, as opposed to 200 KB for Wasm_of_ocaml.

I was going to begin working on implementing rank-2 polymorphism inside the type system, which allows for more expressivity in the language (at the cost of having to annotate some polymorphic expressions), but I'm still doing the research and I wanted to get it up and running into Git first before making any bigger refactors and changes, so for today this will have to suffice.

♪ Listening to: Oneohtrix Point Never Newsom — Rodl Glide

I added forall types to Growl's type system. This is because expressing some higher-order combinators requires rank-2 polymorphism, which means that types where quotations themselves have forall quantification must be valid and expressible.

I use a De Brujin-style representation, where a type can be wrapped in many Forall nodes, and QVar nodes count up from zero being the innermost forall node upwards, so the type for a stack shuffling word like over is expressed as ∀(∀(∀('λ2 'λ1 'λ0 → 'λ1 'λ2 'λ1 'λ0)))

I also have been reading about how Cat uses a concept called "self types" to express limited equi-recursion in types, which allows for types like the one of dup call to be expressible by marking the quotation as being able to accept itself without making it a proper recursive type.

Tomorrow I will likely implement self types, and move the type inference engine to a bi-directional inference system to support for type annotations. This will be needed for more complex expressions that might need some help from the user for the type inference to be able to do its job…

On the personal note, I got prescribed Ritalin. Let's see how that goes.

♪ Listening to: Peace Forever Eternal — Reveille (In Loving Memory)

Today was a slower day, and last night, conversations about writing long-term notes happened in the #decadv IRC channel which left me thinking, so I worked on getting a comfy setup for notes and publishing them to the internet using Emacs, using Protesilaos' Denote for handling the notes within Emacs, and Emacs' ox-publish. I like how it turned out, it's simple and pretty frictionless with some of the commands I've written to do so.

You're reading this wiki right now! :)

Later in the night I decided to take on fixing one bug that had left me baffled in my Uxn emulator library after a week of not being able to track it down.

It left both me and neauoire puzzled in the #uxn IRC channel as to what could be going wrong, until d_m suggested:

<d_m> wolfdog: one annoying but simple test would be to create a file with
      something like `|100 #aa #bb #cc ___ #010e DEO #800f DEO BRK`
<d_m> and if you make it through all of those and all the outputs are identical
      i can suggest similar load/store tests

I was getting prepared to spend like 30 minutes just testing each opcode until I got to DUPk:

And then it was obvious. My code was doing the following:

DUP:
  a := peek from stack
  push a to stack

The peek was bypassing the keep mode machinery! It should've been":

DUP:
  a := pop from stack
  push a to stack
  push a to stack

After that simple fix, everything started working just fine. That was a fun nighttime activity, I guess.

The type system of Growl in its current state is getting quite complicated, and I'm not sure I can continue working on it since I'm not, like, a type theory wizard, I'm just a wolf.

Today I've been working on Rodl, a new Growl implementation, and most importantly, thinking about the prospects of lowering to Lambda-compose as an IR, and I feel like this is the best course to go, as it lifts the complexity from the type system to the lowering, which is more tenable and easier to manage.

Yesterday I didn't do anything in the computer, since I was celebrating my friend's birthday.

Today I worked on a song for my upcoming EP. It's been in the making for a month now and I've finally reached stages of "this is close to finished", so I'm happy!

I've been thinking about whether Growl truly benefits off being statically typed. It is, after all, meant to be a scripting language, and the implementation of a solid type system has proven to be the major bottleneck in getting past implementation hell (perhaps because of my lack of experience…)

I've been also thinking even more about how I want Growl to be, and what to take as inspiration from other languages. Notably, I've been thinking about Mirth's syntax sugar for words taking quotations, and it's made me think, why not do the same? Who knows…

On other notes, lately I've been revisiting my love for Tcl, thanks to Pup, an older project of mine implementing a tiny concatenative language in Jim Tcl. I might rewrite Pup to work with Tcl/Tk so I can use Tk in my programs, and possibly so that it compiles to Tcl just-in-time instead of interpreting the token tree.

Thinking about syntax again, and a comment someone made to me earlier during Growl's development comes to mind:

This language is very difficult to understand to me.

In the Fizzbuzz example I was reading nested quotations and was waiting for the moment they start making sense, and by the end of the each-integer word i was like "oh, there is the if word, I have to read these two quotations again in the context of a conditional structure."

My approach to this is block application. A word can be suffixed with a colon to make it a block-applicative word, and write one or more blocks after the word, like so:

if: {
  ;; true branch
} {
  ;; false branch
}

Note that blocks use curly braces and not square brackets. Using square brackets would pose a problem, for example: is the second block in if: [...] [...] part of the block application or by itself?

You can also put code in between the word and the first colon, which lends itself nicely to organize code:

when: condition? {
  ;; do thing
}

A call like word: {} de-sugars to [] word, and a call like word: ...code {} desugars to ...code [] word.

For a not so contrived example:

#import std

def: "fizzbuzz?" {
  dup if: 3 % 0 = { drop true } { 5 % 0 = }
}

each-integer: 100 {
  1 + dup
  if: fizzbuzz? {
    keep: { when: 3 % 0 = { "Fizz" print } }
    when: 5 % 0 = { "Buzz" print }
  } {
    itoa print
  }
  "\n" print
}

Yesterday I reinstalled NixOS on my laptop and discovered that my backup drive was failing after wiping my drive, so that was fun. Lost all progress on the EP I was working on so I released the two tracks I had to not call it a total loss: WFM - Territorial

Today I went out with some friends and later at night started working (again) on Petro, a younger sibling to Growl written in Ruby. I got the lexer and parser done, and sketched out the interpreter a bit before bed.

♪ Listening to: Joanna Newsom — Cassiopeia

Since it was one of the things I lost the source for in my backup disaster, I'm remaking my website. I'm promoting my wiki/digital garden to be the main website rhzm.org, and I'm no longer using Soupault for post-processing for it, as I discovered that org-publish can create a sitemap for a project (as seen in the index) which obviates the need for a post-processing step to create it.

Can I ramble a bit about how much I love this? Since Org doesn't impose any organization in the index generation (and delegates it to the user), I can use it to create hierarchical organization and keep the notes in a flat-file fashion, and even add entries for headings inside of a page like I'm doing for this one. And pages can have more than one index entry, so if a page belongs to more than one hierarchical category I can just make it happen! So cool.

Unfortunately one small pain point I couldn't solve was the long filenames for the export. I can use Org's #+EXPORT_FILE_NAME attribute, but then links export using the source file's name, which breaks all links I create using Denote's denote-link command (and Org's file: links, too.) I've put a disclaimer in the homepage that sometimes page links may change if they're not publicly in the sitemap.

Aside from that, I think I've nailed it with the process in building my website, it feels seamless now. Without a build process I can just hit C-u M-x org-publish-all and open the files directly on my browser.

♪ Listening to: Cities Aviv — POWER APPROACHES

This entry also involves last night's work, since I made yesterday's entry at around 6 PM, but I finished my Uxn emulator's console and system devices implementation, and added a rough file device implementation. The emulator can now run Drifblim to compile other ROMs! Previously it could only run Drifloon as it missed the file device and command line argument handling in the console device.

Today's work is starting the rewrite of Growl in C. I came to a stop trying to think about handling opaque values inside the OCaml interpreter, and I've come to the realization that while OCaml is pretty good at writing compilers, for interpreters it's a bit annoying.

♪ Listening to: Drew McDowall — Out of Strength comes Sweetness

Today was a rest day, so I haven't written any code.

♪ Listening to: ML Buch — High speed calm air tonight

Home sysadmin day! Been working on moving my media server to run NixOS instead of the Raspberry Pi default Linux distro, because if I'm already running NixOS on my laptop then why not go all in! This involves:

• Setting up Jellyfin, for the media server aspect
• Setting up slskd, for downloading my music in a very legal way, wink.
• (Potentially) setting up Copyparty to transfer music to the server, though with slskd I don't see much need for it anymore. NixOS also doesn't have a service definition for it unfortunately…

The Raspberry Pi is running with a tmpfs-as-root setup, as I like to minimize leftover state as much as possible. I do the same in my laptop, but with BTRFS snapshots, since I expect caches and /tmp to use more space on regular use, so I prefer to have them backed by the disk rather than by RAM.

I might write a page about the setup process soon.

♪ Listening to: Steve Reich — New York Counterpoint

Today I'm working on my Uxn emulator.

I reworked how devices are handled, so now there's a Device module signature that devices have to implement, exposing some information and DEI/DEO handlers, and a Compose functor to create a "super-handler" that handles two devices, so instead of having the device handling in the emulator code, it can instantiate the devices and then instantiate a "composed" device handler which handles all of them:

(* Instantiate devices *)
module System = Varvara.System.Make ()
module Console = Varvara.Console.Make ()
module File = Varvara.File.Make (...)

(* Instantiate the composition of all devices *)
module Devices = Uxn.Device.Compose (System) (Uxn.Device.Compose (Console) (File))

And then it can simply handle devices as follows, without ever knowing which devices its actually querying:

try Machine.dispatch ~trace m pc with
| effect Machine.DEI (`Byte, port), k -> (
    match Devices.dei m port with
    | Some v -> continue k v
    | None -> continue k (Bytes.get_uint8 dev port))
| effect Machine.DEI (`Short, port), k -> (
    match Devices.dei2 m port with
    | Some v -> continue k v
    | None -> continue k (Util.get_uint16_wrap dev port))
| effect Machine.DEO (port, value), k ->
    if Devices.can_handle port then Devices.deo m port value;
    continue k ()
(* ... *)

I also tried using a hash table of device modules instead as a micro-optimization, using OCaml's first-class modules extension, but it didn't seem to make a difference in speed, so I reverted back to the old composition method, which simply tests left to right until a module can handle the port. At some point I might try the hash table method again if I ever want to allow the devices to be used to be defined at runtime instead of compile-time.

This was a major pain point with my code since the device handling was making the core loop a bit hard to read because of the device handling, so I'm glad I got it reworked :).

♪ Listening to: Oli XL — HOODIE MUSIC

Visiting my mom for the holidays and recovering from the flight, so not a lot of code done today.

I ended up implementing the hash table device lookup method I mentioned yesterday on my Uxn emulator, and implemented the Datetime device.

♪ Listening to: Syzy — Caught up (in circles)

Working on Growl. Implemented Cheney's algorithm for a garbage collector and added implementations of prototype-based tables. Tomorrow the work will be documenting this code and wiring up the lexer to be able to read code.

♪ Listening to: Andy Stott — In Oath

Continuing to work on rewriting Growl in C. Borrowed some of the lexing code from an old project and wrote a recursive-descent parser. The parser also exercises the garbage collector a bit since the AST it returns is encoded using the runtime value datatype. The code is still poor in documentation but I will be working on that as I go.

Next step is building the interpreter, which I will be doing tomorrow.

How do you deal with development burnout hell? Working on Growl has proved to be an exercise of my ability to cope with burnout, as my motivation to work on this project seems to go in waves.

I think I will call it quits for now, especially as Christmas rolls around, so I can take a break from writing code and focus more on myself. I definitely appreciate the existence of December Adventure, as it has pushed me towards being more consistent with working and writing about said work.

A big thank you to Eli, for organizing this every year, to everyone in the #decadv IRC channel, and everyone who has read my musings on the Fediverse or other chat spaces.

Big hugs to everyone. <3

♪ Listening to: Ricardo Villalobos — What You Say Is More Than I Can Say (edit)