Blog on Topi Kettunen

When the Fire Goes Out

Tue, 24 Feb 2026 21:47:34 +0100

For years, various hobbies and past-times have functioned as an internal compass for me. They have structured my time and justified me this “waste of time”. Days have been bent around those. Decisions have been filtered through them. Relationships have been negotiated in their shadow. These sort of past-times haven’t been an addition to my life but a more or less the directing feature.

Losing an interest or seeing the fire go out in these sort of activities is not like getting bored of a TV show or an album/playlist. It could be seen as a disturbance in one’s identity. When an activity that once organized your “inner world” no longer requires the same level of attention. The problem isn’t so much about what to do with your free time but what to do with yourself.

This creates weird tension. You still have the memories of images of intensity, discipline, and the emotional investment, while the present can only offer neutrality. This sort of feeling can be very disorienting. Former source of urgency now produced no internal signal at all. Inevitable now feels optional.

Detachment to these can announce itself quietly. Skipped sessions start to feel natural instead of “wrong”. Preparation became mechanical. Anticipation thinned. Task that previously generated urgency feels only something inherited from your former self. This kind of contrast can be relatively stark. Where there had been obsession, there was now mild recognition.

I wouldn’t label this sort of shift in terms of laziness, because the discipline is still intact elsewhere. I wouldn’t also call it “burnout”, because rest did not restore this desire. The enjoyment for this activity is still functioning, but the reason for using it had dissolved. The problem isn’t exhaustion. It is irrelevance.

What disappears is not just an activity but a self-concept. The person who organized life around a given pursuit not longer exists. Skills are there but the narrative around it has dissolved. This can easily produce guilt and shame. Abandoning something once treated almost “sacred” feels like betrayal. Like the past effort demands future loyalty.

What about if similar structural shift happens with your profession? I’d argue it wouldn’t automatically dissolve it. Mainly since work is embedded in systems larger than personal intensity. For example, financial stability, contractual obligation, accumulated expertise, social positioning, and etc.. Unlike passion-driven pursuits (while profession can definitely be one), profession doesn’t require constant emotional ignition to remain functional.

Professional tasks continue to be executed with precision, but the internal narrative that once fused identity and activity weakens. The profession becomes less about expression and more about execution. Efficiency replaces obsession. The work still benefits from the discipline forged in the earlier years, yet it no longer serves as the primary site of meaning.

There is also a recalibration of risk and ambition. Earlier phases may have involved identity-level stakes attached to outcomes. Now, the profession is treated as a stable platform rather than a proving ground. This shift can be misread as stagnation, but it often reflects strategic adaptation. Energy is conserved for survival, maintenance, or redirection into other domains. The professional role becomes a container for skill rather than a mirror of the self.

Over time, the profession may assume a quieter, more utilitarian meaning. It continues not because it defines existence, but because it sustains it. This redefinition strips away these cheesy romantic narratives while preserving functional value. What remains is a disciplined engagement with reality: work performed with clarity, detached from the earlier compulsion to derive identity from it.

What I Read Recently? (November 2025)

Mon, 24 Nov 2025 21:52:53 +0100

I’ve almost completely forgotten about keeping my reading list up to date. Looking at the list here, it seems it was almost exactly year ago that I lastly updated it. I’ve kept on reading, although, probably not as much as I should, but still, haven’t just kept the habit of writing few words down about each of those. Well, let’s try reawakening that habit again with some recent readings that I read.

Mauno Saari: Juoksemisen salaisuudet

I’ve recently started to wake up my running practice again. Although, I’ve never been super into it, I always done at least a little bit of running as an auxiliary workout to other sports. Recently though, I’ve wanted to get better as a runner. I happened to be in Finland visiting my parents and happened to found this old book about Lasse Viren in my father’s library. So of course knowing who he was, thought that this is a great book to read since I’ve not started to take running more seriously.

The book worked because it exposed the machinery behind Viren’s results without mythmaking. The mix of training detail, physical strain, and the era’s competitive realities sharpened the respect I already had for him. Reading it clarified how much discipline and calculation sat beneath his calm public presence, and it made his achievements feel heavier and more deliberate.

Haruki Murakami: What I Talk About When I Talk About Running

While I’ve never been too big of fan of Murakami, I’ve still read a few of his works, and happen to know he is a big fan of running, so in the similar theme as the last one, I was curious about this one. Not necessarily because Murakami is some great running influence, nothing like that. I was just curious how he put the discipline of running into his words, how he describes it.

The book functioned as a stripped account of endurance as a mental architecture rather than a hobby. Murakami’s framing of running as a daily discipline, a stabilizing ritual, and a tool for shaping attention made the narrative useful rather than decorative. His refusal to romanticize the effort aligned with the core appeal: a direct view of how steady practice builds a mind capable of long work.

Juha Numminen: En päivääkään vaihtaisi pois

This was also a book that I happened to found from my father’s library. It’s a memoir/life-story about Tapio Rautavaara. The book captured Rautavaara as a complete figure, not just an athlete or entertainer. Numminen’s account of his training, competitions, and artistic pursuits conveyed the weight of discipline and personal choice behind each achievement. Reading it strengthened my admiration, showing how a life balanced between sport, music, and performance could remain authentic and fully claimed, day by day.

Stupidly Simple Plain Text Running Log

Fri, 07 Nov 2025 13:45:20 +0200

I’ve recently started revitalize my running hobby. Mainly due to the fact that I’ve gathered up some extra “unnecessary” kilos, so I’ve felt that I need to step up my aerobic exercises. Although, truth be told, I’ve never been too keen into that sort of exercises and have always preferred stuff like strength training etc. But I think now would be good time to start.

Of course, being the geek I am, I wanted to start tracking my practice in one form or another. Obviously, something like Strava is a very common solution for this and most likely if I end getting more “hooked” on this hobby, I’ll probably start using it more. But at the same time, I kinda would want to have something self-hosted for this, at least for sharing these to others.

I’ve always been a fan of “Plain Text X” type of solutions. Great example would be something like Plain Text Accounting, where something like Ledger is a commonly used software. So naturally, I wanted to have something similar for my running log.

For this purpose, I decided to go with a simple CSV file and gnuplot. Considering, one of the reasons why I wanted to do this was to be able to share these. So naturally, something like HTML table and JS chart library could’ve done the trick. But at the same time, I kinda wanted to have some non-JS solution.

CSV Format for Runs

I decided to go with very simple structure for logging my runs. Essentially, I only cared about the date of the run, the length of the run and also the ability to give few words of notes about the run. In CSV, it could look something like:

date,distance_km,notes
2025-11-01,5.2,Morning jog
2025-11-03,7.5,Evening run
2025-11-05,10.0,Long run with hills

Adding new runs would also be simple as you can just manually edit the file or run something like:

echo "2025-11-06,6.3,Easy recovery run" >> runs.csv

Also if someone cares about this, in this format, it’s also easily version-controllable e.g. in Git.

Plotting the Runs

To get the plotting started, I just started with something simple:

set datafile separator comma
set xdata time
set timefmt "%Y-%m-%d"
set format x "%m-%d"
set title "Running Distance Over Time"
set xlabel "Date"
set ylabel "Distance (km)"
set grid
set term pngcairo size 800,400
set output "runs.png"

plot "runs.csv" using 1:2 with linespoints title "Distance (km)" lw 2

and creating the plot itself:

$ gnuplot plot.gp

With the data defined above, it would result into something like this:

Of course, gnuplot offers a wide variety of different configurations for these plots, so you can go nuts with them. But at least for me, this is a good starting point.

Creating a Static Page for the Running Log

Lastly, I just wanted to generate some sort of HTML page from it, that I can just save to my server. I decided to just read through the CSV file and just create a HTML table out of that in a simple shell script format.

#!/usr/bin/env bash

# Generate chart
gnuplot plot.gp

# Start HTML
cat > index.html <<EOF
<!doctype html>
<meta charset="utf-8">
<title>Running Log</title>
<style>
  table { width: 100%; border-collapse: collapse; }
  th { text-align: left; }
  th, td { border: 1px solid; padding: 0.4em; }
  img { display:block; margin:auto; max-width:100%; }
</style>
<h1>Running Log</h1>
<figure>
  <img src="runs.png" alt="Running chart" style="">
</figure>
<table>
  <tr><th>Date</th><th>Distance (km)</th><th>Notes</th></tr>
EOF

# Convert CSV rows to HTML table rows (skip header)
tail -n +2 runs.csv | while IFS=, read -r date dist note; do
  echo "  <tr><td>$date</td><td>$dist</td><td>$note</td></tr>" >> index.html
done

# End HTML
cat >> index.html <<EOF
</table>
EOF

And yes, that is a valid HTML, since you can omit optional tags in HTML5.

If you happen to want to show the table so that it shows the most recent entry first, which is most likely the case, you can just include tac in the while loop:

# Convert CSV rows to HTML table rows (skip header)
tail -n +2 runs.csv | tac | while IFS=, read -r date dist note; do
  echo "  <tr><td>$date</td><td>$dist</td><td>$note</td></tr>" >> index.html
done

And there it is! Intentionally (or stupidly) simple. Only depedency being gnuplot. All data in simple plain text format, published in HTML. Of course, there are plenty of improvements that could be made like weekly/monthly summaries (this could be easily done with awk) or just creating different plots from the given data.

Using Hugo

If you happen to be using Hugo, you can read CSV directly and create a new shortcode for generating similar table as above. You can read more about that in here

I’ve also started to gather up my own runs in here in case you happen to be interested in how much I struggle with this new hobby.

Sila Dev Log: Waffling on Next Steps but Getting Back on the Saddle

Sat, 04 Oct 2025 00:48:15 +0200

Some time ago, I was actively working on my own compiler project whenever I had spare time. Unfortunately, life got in the way, and the project sat on the back-burner for a while. A big reason was my new solo music project, which took up much of my energy.

Alongside recording, I’ve also been diving into Opusmodus for composition and analysis, a wonderful showcase of what Common Lisp can enable in creative domains.

Another factor behind the pause was that I had coded myself into a corner due to some early architectural decisions. I even tried refactoring, but although the codebase was still relatively small, it quickly became frustrating. The dynamic nature of Common Lisp, while incredibly powerful, made larger-scale refactoring feel trickier than I’d like. To be fair, that’s more a reflection of my skills than a flaw in Lisp itself. I still deeply value its flexibility, macro system, and directness.

Recently, I found myself needing to work quite a bit with C++ and LLVM in other contexts, which naturally made me wonder if my compiler project should make the jump as well. LLVM, in particular, seemed appealing: it already provides a robust backend infrastructure, excellent tooling, and cross-platform portability, which would spare me from reimplementing a lot of low-level machinery. The timing also aligned with my recent switch to an Apple Silicon machine. Since most of the existing code was tailored for x86-64, I was facing the task of implementing a new backend for aarch64 anyway. Using LLVM started to feel like not only a practical choice, but also an opportunity to learn its internals more deeply.

I briefly toyed with the idea of sidestepping the C++ API by generating LLVM IR directly from Lisp, which would have allowed me to keep most of my workflow in Lisp while still taking advantage of LLVM. However, I figured that diving into the C++ API would refresh my C++ knowledge and give me a more complete view of how LLVM is actually designed to be used. At the same time, I also considered jumping ship entirely and using a statically typed language like Haskell, which might have made large-scale refactoring and correctness guarantees easier in the long run.

Haskell, with its type system, has always impressed me, and projects like Coalton make that style of typing accessible within Common Lisp. Still, as much as I admire Haskell, it simply doesn’t spark the same joy as Lisp.

So despite the temptations of C++, LLVM, or even Haskell, I’ve decided to continue with good ol’ Common Lisp. It’s the language that excites me the most, and ultimately, joy in the process is what keeps projects like this alive.

RIP Ozzy Osbourne 1948-2025

Wed, 23 Jul 2025 15:47:57 +0200

Yet another massive loss in music. Black Sabbath was probably the first band I ever heard. Definitely the first band that I really got into. This was due to the fact my father introduced me into it when I was a literal baby. Hugely important band for me. Tragic loss.

Thank you for everything.

RIP Douglas McCarthy

Wed, 11 Jun 2025 00:55:38 +0200

This week has been horrible when it has come to death of musicians. We just lost Sly Stone and Brian Wilson. Now, one of the most influental artists for me, Douglas McCarthy from Nitzer Ebb. What a horrible year and week.

Nitzer Ebb along with Front 242 were probably the two most influental bands for me which made me eventually to delve more deeply into the realms of EBM and industrial. Naturally, after those came bands like Ministry and Nine Inch Nails that also have always been super important bands for me. Nitzer Ebb were for me the gateway drug into this sort of sonic world.

Thanks for everything!

Art Should Comfort the Disturbed and Disturb the Comfortable

Tue, 26 Nov 2024 00:11:47 +0100

I recently stumbled upon a beautiful quote from Cesar A. Cruz that I just wanted to ponder.

“Art should comfort the disturbed and disturb the comfortable”

Cruz’s quote is both provocative and elusive, a tidy aphorism masking a deeper tension. It implies a dual responsibility for art: to console those in pain while challenging those at ease. But who decides which group we belong to? What comforts one person might deeply unsettle another, and vice versa.

The beauty of Cruz’s idea is that it resists easy answers. Comfort isn’t inherently weak; it can heal. Discomfort isn’t automatically righteous; it can harm. Art lives in this paradox, in the liminal space where consolation and confrontation coexist. Cruz’s words don’t prescribe solutions, they challenge us to sit with the ambiguity, to feel both the sting and the salve of what art can do. That’s its power: to disturb us into wakefulness, but also to cradle us when we need it most.

What I Read Between May and October 2024

Wed, 13 Nov 2024 00:45:03 +0100

Haven’t updated my “What I Read Between” -series in a while mainly due just being busy in my own life but also due to the fact that I’ve reading and rereading veeery big novels. But finally having been managed to finish those, let’s gather some thoughts.

Fyodor Dostoyevsky: Crime and Punishment (reread)

I’ve always been a huge fan of Russian literature, or at least a big fan of few Russian authors. Mainly due to this reason I wanted to embark on this “rabbit hole” during the summer months. Naturally, starting with Dostoyevsky.

I’ve read Crime and Punishment a couple of times during my lifetime. First, as a mandatory homework from school and later few times just out of pure enjoyment and I can say that it’s possibly one of my favorite books. To summarize, book is about Raskolnikov’s internal and external battles and the repercussions of the crimes he has done.

I love how the “crime” aspect of this book happens only in the 100 or so pages, and the rest (600 or so) are devoted to the “punishment”, which in this case means battles with psychological consequences of the crimes, insanity and isolation, ending in the moment of divine grace and beginning of Raskolnikov’s redemption.

Great book, cannot recommend it enough.

Fyodor Dostoyevsky: Notes from Underground (reread)

Along side Crime and Punishment, Notes from Underground is also a book that I’ve read multiple times from Dostoyevsky. Probably I stumbled upon this book when I was a “edgy” teenager, so I thought I could see some similarities between me and the protagonist.

Book focuses mainly on diving deep into the mind of man living in solitude. So a majority of the book is about questioning various everyday concepts, such as logic, reason, free will, and showing how suffering can add depth to our everyday life.

Again, very good book that encourages reader to not just read but also to think and reflect.

Fyodor Dostoyevsky: The Brothers Karamazov

The Brother Karamazov was one of the classics of Dostoyevsky that again for one reason or another I hadn’t read so I finally was able to finish it and what a book it was! Book itself focused where heavily on rich and absorbing ideas such as spirituality and logic and their contrast. So much so, that the plot of the book itself (while being great on its own) feels secondary to the exploration of these themes.

While the high level topics might be the main thing behind the book, I especially loved the way how Dostoyevsky embedded these themes in to the main characters of the book, the Karamazov family.

Book was pretty big with lots of different events and stories (even stories within a story) so couple of paragraphs doesn’t do it justice. Just this book alone deserves a separate post. I would even argue that just reading the book once doesn’t do it justice. That being said, I enjoyed the book tremendously, and I will be reading it again in the future.

David Foster Wallace: The Infinite Jest

The Infinite Jest has always been an interesting book for me. For many reasons. First, I’ve always thought David Foster Wallace to be an interesting character on its own right. But at the same time, for the the literary prowess that he had. I had read couple of works from him previously but Infinite Jest always felt like a somewhat of a herculean task to finish. Firstly due to its length, but also for how it was written.

I remember reading some interview from DFW, where he mentioned how he “wanted to write a best-seller, that was meant for literature students/professors”, which can definitely be seen in the book. The writing style in the book is very hard and difficult. Especially to someone from whom English is not their native language.

But overall I think the book was great. Especially the time in the current world made the book feel sort of “prophetic” in many ways. Despite the book itself is set in this dystopian world. But looking it with perspective of today’s world, it’s not that dystopian.

Similarly to The Brother Karamzov, couple of paragraphs doesn’t do the book justice, since it features hundreds of different characters, many of whom are just coping with modern technological world filled with loneliness and disconnection.

Again, a great book once you get into it, which definitely deserves an another read.

Appreciation of Low Bandwidth Websites

Sun, 21 Jul 2024 20:48:56 +0200

As you might’ve seen in my last post, I’ve been without an internet for good some time already. Still (like I mentioned in the last post) I’ve really enjoyed my time without it. I feel that I can switch off much better after working hours, and I still have energy to partake in various other extracurricular activities that I hold dear to my heart, like hobby programming projects, music, reading, sports and much more. So really there isn’t much to complain.

Currently, my new neighbourhood does have a pretty good signal, at least generally speaking. My apartment just happens to be in relatively awkward spot, which means that it’s surrounded by other buildings. I happen to have a relatively big balcony in my apartment as well and even in there I have basically one bar of connectivity going on. Naturally, inside I don’t have any signal or very little. Fortunately, I’ve been able to have just enough of connectivity so in case some emergency or something along those lines, I’m able to do something with my phone. But, when it comes to “surfing the web”, that’s not much that I can currently do.

That being said, I can do just some browsing in handful of site that don’t clog the whole bandwidth for silly updates and requests and mainly just focus on sharing good content. So I wanted to write a small appreciation post for people making websites that work with practically no internet at all.

TODO: Maybe gather some great examples of lite websites to somewhere?

Of course, when writing about this sort of topic, it also brings little bit of sadness to me to see how many websites is basically unusable without a relatively good internet. Fortunately, I live in a place where even at its worst, I can access relatively “fast” internet and of course most of the places (outside my apartment) is very well connected. I don’t need to go far from my house to get basically full 5G connectivity with varying speeds (whatever your contract at the moment happens to be). In places like Finland, 1 Gbit fiber connectivity is getting more and more normal even in somewhat rural areas. Germany is not on that level yet (and by many standards, very far from it) but it’s still good nonetheless. Something like this cannot be said about the vast majority of the countries around the world.

I kind of know the reason why something like this has happened that the significant portion of popular websites are basically unaccessible by many people. I think it’s largely due to this aggressive over-engineering software engineers tend to do in their line of work. Also, I’m not innocent in that front as well, since I’ve done my fair share of over-engineering during the years working in the industry. Sometimes since I wanted, sometimes it was needed and sometimes since I was told to.

But it begs an interesting question, why something like this so normal in our industry? Sure, there are cases where added complexity to your infrastructure, codebase, etc. has actually brought some benefits. But the amount of times that has happened is greatly outweighed by the amount of times when it, well, just has brought added complexity without any benefits. But hey! At least, the engineers feel accomplished when they are able to deliver that is “complex”.

From the UX point of view, this is also quite interesting topic, since I believe that no one in this world wants to have their app/product/software/whatever to be slow for the users. But often, this added complexity automatically hinders the performance of whatever you happen to we writing. You start doing more stuff than you would need to, this can be inform of unnecessary web requests, unnecessary computations and so on. I guess there isn’t any argument about the fact that something like this hinders the whole UX. Which begs the question? Why more people don’t care about the performance more in their products? You might have an application that might have a traditionally speaking “bad UX”, meaning maybe lots of menus, you need to press a lot of stuff to access whatever you might be after, etc. But, if everything happens instantly, I’d dare to argue it’s still pretty good UX, at least for many people. If then on the other end of the spectrum, you would have a great UX, but everything is slow and has to load lots of stuff, how good of an UX it really is at that point?

But yeah, I digress, seems that lack of internet just makes me want to write about stuff. Any case, TLDR, write light websites.

German Unefficiency and Living Without Internet

Sat, 06 Jul 2024 18:06:29 +0200

As some of you might know, I’ve been living in Berlin, Germany for a few years now. Wonderful city! Just what I was looking for couple years back when I moved here. City definitely has its flaws (like does every city in the world) but I still enjoy being here. City is definitely massive. Each district having their own life, own people and own quirks. Spectrum is wide from somewhat poshy areas of Mitte and Prenzlauer Berg to more hip and trendy areas like Neukolln and Kreuzberg to more suburban areas around the Ringbahn. Of course, since I’m mainly into various hip and trendy activities most of my time has been spent lingering around and roaming the streets of eastern Berlin, mainly Neukolln and Kreuzberg since those were the districts I used to reside. Due to its sheer size, there’s no way, I’ve been able to experience all the city has to offer. Despite this, I feel that the city has offered me tremendously.

While I’ve really enjoyed living in Berlin, one thing I can’t praise at all is how things work in this country. So this is not necessarily Berlin’s fault, although, I feel its also partly due to that. But more or less everything seems to be difficult here. At least in a bureaucratic sense. Difficult in a sense that nothing really works efficiently or effortlessly, or that’s how I at least feel coming from Finland to this country. I’m not saying Finland is perfect in how country should operate, far from it. I’m just saying that if you want to move from place to place, making this sort of address change to your government shouldn’t be too difficult. In Finland for example, if you want to move, there’s one URL that you go to, fill in your new details, and send the information. Pretty easy. In Germany, if you want to Anmeldung, you need to print your papers necessary, fill the forms, book time in your nearest Burgeramt, go there physically, just so someone can put some stamp in your papers. Sounds pretty unefficient, right? That’s because it is.

I also just recently read from the news how Germany was excited to have digitalised their medical patient information so that more stuff would happen digitally instead of manually. Making it easier to share medical history from doctor to doctor. Sounds amazing! Of course, something like this has been implemented in many different countries for 20 or so years, but hey, Germany is in no rush!

This sort of a ranty-detour gets us to the whole topic of today’s post about living without internet. So I just wanted to write some words down on how I feel after some undisclosed amount of weeks that I have been without internet at home.

This all started mainly due to the fact that I just happened to move into a new place and naturally I wanted to move my internet contract into the new address. Of course, naive and blue-eyed me thought that it would be pretty straight-forward. I just change the address, configure the router possibly in the address in case something needs to be done and that’s about it, right? Well of course not. Turns out, for some reason, there needs to be a internet technician come to your place and turn on the connection for you. Okay, sure. Interesting that it couldn’t be done remotely, but hey let’s go with it. But of course to make it not so easy, the technician isn’t able to find my home, which is interesting again since they know the address. Also, of course they could’ve just called me but that’s too much to ask I guess. So there’s an another try, but with the same result. Oh, and did I mention that there was two weeks between these attempts so it not like that they tried it again in a short period of time.

Okay, I start ranting again. Back to the topic. I wanted to jot down some thoughts that I’ve encountered while living without an internet. Turns out, I have really enjoyed my time! I’ve written in my blog about my “struggles” with for example social media and how it used to grab my attention almost wholly. But generally speaking, it feels that everything I tend to online, tends to be more on the excessive end of the spectrum. Call it excessive use of YouTube or other various places filled with endless amounts of content. I just feel that I tend to sink vast amount of hours in these sort of places, without getting much in return. While I’m not saying they are all bad, they certainly have good bits and pieces here and there, but the vast majority just feels brain-rot.

With these undisclosed amount of weeks that I’ve now been without internet, naturally, I’ve been away from all them. Even the cellural signal is quite weak in my new apartment so I haven’t been able to use them via it even that much. Fortunately, I’ve been able to keep in touch with family and friends with the little signal I’ve had at home and in case I’ve really had to use internet for something, I’ve had just to leave my home to find Wi-Fi, which also has been quite good for me, since I’ve a habit of going “monk- or hermit-mode” quite often.

I also recently changed jobs so fortunately I haven’t been in on any on-call rotation so I haven’t really had the need to work from home. Also, the new apartment has been quite close to the new office, so it hasn’t been too big of a hassle to go there daily. Even that hasn’t been too bad. Why it’s to my surprise is the fact that before this I really never went to the office. This employer or the previous ones. I wouldn’t say the office that we have currently is something extra-ordinary that I would really want to spent time in there, it’s still nice, don’t get me wrong. I’ve never really just seen myself as this social butterfly who can effortlessly manage themselves mingling around in an office environment. So, I’ve always just enjoyed working from home more.

The lack of internet at home has really shown itself in my life in the form of vastly increased productivity on the stuff that I value. I’m not saying that some sort of ultimate productivity is something that everyone should strive towards, which seems to often be the case in various toxic “hustling culture” ideas, but I’m more so approaching the topic from the point of view of how much time you really can use in the stuff that brings something positive and beneficial to your life. In my case, it has shown itself in the form of increased productivity for example in exercise, making music, writing, reading and also programming passion projects. I.e. more or less most of the stuff outside personal relationships and health that I find to be important in my life.

So now I’m just spitballing with the idea of really not just getting any internet to my home. Of course, if the current job situation would allow something like that. Considering all the good effects I’ve seen in my life AND considering the fact that it would be just one less thing to worry my head in German bureaucracy, I don’t really see negative side in this.

Dump Your Currently Set Faces in Emacs

Sun, 26 May 2024 15:45:54 +0200

Every once in a while I like to play around with Emacs themes. Of course, mainly due to yak-shaving and procrastination reasons and nothing actually valid. But it’s fun nonetheless, although occasionally very tedious. Tedious in a sense that Emacs tend to come with a lot of different faces, especially if you happen to use more than a few third-party packages.

Defining a custom theme in Emacs

Emacs comes with couple relatively straight forward way to set custom themes in the function custom-theme-set where you define the name of the theme and then the faces you want to modify:

(deftheme fancy-theme)

(custom-set-faces
 'fancy-theme
 '(default ((t (:foreground "black" :background "white"))))

 ;; [... and so on]
 )

You’re also able to control on what sort of displays your custom theme should support. For example, if you want to use theme in only terminals that support minimum number of 16 different colors, you’re able to define that like:

'(default ((((class color) (min-colors 16)) (:foreground "black" :background "white"))))

Of course that can be slightly verbose, so often you might see custom themes to be set like:

(deftheme fancy-theme)

(let ((class '((class color) (min-colors 16))))
  (custom-set-faces
   'fancy-theme
   `(default ((,class (:foreground "black" :background "white"))))

   ;; [... and so on]
   ))

But it still tedious since there is a lot of faces and there can be lot of variance between the faces so often if you want to write your own full fledged theme, it’s better to explicitly set all the faces that you plan to use. Mainly since, if the face is not set in the theme it’ll use that lastly set face for that face in question, so if you session has used multiple different faces, it can lead into some funky color combinations.

Currently in my Emacs session, which is relatively light package-wise, amount of currently set faces equals to 869.

Simple tool written for aspiring theme developers

This is why, I decided to write a simple Emacs Lisp helper function to dump all the currently set faces to a buffer where you can copy them to your own theme at will. I also made it so you can simply just dump the faces from a give package if you only care, e.g. about Magit’s faces.

(require 'cl-lib)

(defconst theme-dump--buffer-name "*theme-dump*")
(defconst theme-dump--default-format-string "%s %s ")
(defconst theme-dump--string-format-string "%s \"%s\" ")

(defun theme-dump-current-faces (pkg)
  (interactive (list (read-string "Dump faces from package (leave empty for all packages): ")))
  (let ((faces '()))
    (dolist (f (theme-dump--filter-faces-by-package pkg))
      (let ((attr-str (mapconcat (lambda (a) (theme-dump--format-attr-str f a))
                                 (face-all-attributes f)
                                 "")))
        (push (if (not (string-equal attr-str ""))
                  (format "`(%s ((t (%s))))\n" f (string-trim-right attr-str))
                (format "`(%s ((t nil)))\n" f))
              faces)))
    (let ((buf (get-buffer-create theme-dump--buffer-name)))
      (with-current-buffer buf
        (erase-buffer)
        (insert (mapconcat 'identity faces))
        (sort-lines nil (point-min) (point-max))
        (local-set-key (kbd "q") 'kill-buffer-and-window))
      (split-window-right)
      (other-window 1)
      (switch-to-buffer buf)
      (setq buffer-read-only t))))

(defun theme-dump--filter-faces-by-package (pkg)
  (cl-flet ((pkg-face-p (face)
              (cl-search pkg (symbol-name face))))
    (if (not (string-equal pkg ""))
        (cl-remove-if-not #'pkg-face-p (face-list))
      (face-list))))

(defun theme-dump--format-attr-str (face attr)
  (unless (equal (face-attribute face (car attr))
                 'unspecified)
    (cl-typecase (face-attribute face (car attr))
      (string (format theme-dump--string-format-string
                      (symbol-name (car attr))
                      (face-attribute face (car attr))))
      (t (format theme-dump--default-format-string
                 (symbol-name (car attr))
                 (face-attribute face (car attr)))))))

Essentially, this just gets all the faces with face-list and either returns all of the or filters them by package name if you want. Function prints the faces into a new buffer in a format that is already suitable for new custom themes:

`(magit-section-highlight ((t (:extend t :background "grey95"))))
`(magit-section-heading ((t (:weight bold :extend t))))
`(magit-section-secondary-heading ((t (:weight bold :extend t))))
`(magit-section-heading-selection ((t (:extend t :foreground "salmon4"))))
`(magit-section-child-count ((t nil)))

;; and so on...

The printed faces can then easily be copied to custom-set-faces for your possible new theme. Faces are also backticked by default, in case you want to change faces to use variables instead of hardcoded names.

Code is in public domain and also available in SourceHut.

Dereferencing Null Pointer

Fri, 24 May 2024 17:07:38 +0200

Bret Hart saving the day in code review.

RIP Steve Albini

Tue, 07 May 2024 13:15:20 +0200

This one hurts. Steve Albini has his hands in so many of my favourite albums.

Sunsetting Mastodon

Sat, 04 May 2024 21:06:06 +0200

Some months ago I decided to join back to social media in the form of registering to Mastodon. I thought that maybe this could be the platform to fill the void (which I’ve later realised to be non-existent) of social medias in my life. But now I’ve decided to leave it as well. Not necessarily due to same reasons why I left social medias like Facebook, Instagram or Twitter. But still, partly due to same reasons.

One of the main reasons why I left mainstream social media platforms was their draconian behaviour behind the scenes. Addictive design principles behind their applications, algorithms promoting content that you didn’t ask for, data mining and profiling, and so on. Fortunately, in Mastodon, these sort of issues weren’t present, which was one of the reasons why it felt so welcoming. It had this weird old internet type of aura in it. Like being part of this weird digital clique.

Also, just from the pure technological point of view, the federated elements of Mastodon (or ActivityPub in general) seemed quite interesting. Finally, a platform competing with the centralized behemoths where users were able to retain control over the data and interaction without being riddled with profit-driven algorithms. If I would want to be part of some social media, it still would be Mastodon, which might sound slightly odd considering the title of this post.

So why did I then decide to leave it? One of the main reasons why I didn’t enjoy using e.g. other mainstream social medias, was the fact how they started affect my own behaviour. The constant urge to check what’s going on. Picking up your phone, opening the app, or just going to the site itself out of habit without necessarily even having a reason for it. It is addicting. It is meant to be addicting. Although, in Mastodon’s case, probably not maliciously.

So while I thought investing time and energy into building connections and contributing to various discussions in Mastodon would be beneficial, and no doubt, they can be. For me, it just started feel like sacrificing my mental well-being for digital engagement, again.

Mastodon is by no means a bad platform. It just isn’t suitable for me who wants to navigate digital landscape mindfully and intentionally without being ensnared into various addictive grasps once more.

What I Read in April 2024

Tue, 30 Apr 2024 17:07:39 +0200

Haven’t written one of these in couple of months since I’ve mainly spent my time reading various papers - mainly computer science related - and not so much literature, but I finally got back to the saddle so here’s the latest reviews.

William Strunk Jr.: The Elements of Style (reread)

English is not my native language (even though I mainly write in it for this site) so grammatical and other style rulings when it comes to writing is something that I always struggle. While I would consider these rules to be relatively easy, I tend to always forgot them since they’re still so different from the rules of my native language, Finnish.

The Elements of Style is a short book is comprised of these sort of elementary rules for writing English. Due to the length of this book, I have read this many times and every once in a while, I tend to come back to it. It was quite a useful read for me since I have had a few month break from writing altogether so it was beneficial for waking the old habit.

Kimmo Svinhufvud: Kokonaisvaltainen kirjoittaminen (reread)

On similar theme as above, this focuses mainly on writing style in Finnish.

Stephen King: On Writing: A Memoir of the Craft

This book has been on my reading list for some time just out of sheer curiosity. I’ve always enjoyed Stephen King’s writing and I know how proficient he is at writing. Especially considering how much he publishes yearly. On Writing: A Memoir of the Craft is a wonderful book that gives a glimpse to the life of writer and how one becomes one.

Writing has always interested me, and especially lives of remarkable writers. What sort of life events might’ve affected their style and stories. So it’s wonderful to hear such life stories written by the writer themselves.

Andrew Appel: Modern Compiler Implementation in ML (reread)

If you’ve followed my blog, I have spent some of my free time hacking my own programming language and compiler. Appel’s book on implementing compiler has been a standard that every once in a while I tend to go back to refresh some memories. Wonderful book.

George Saunders: Lincoln in the Bardo

I’ve been a big fan of George Saunders’ works for long time, especially his short stories. Lincoln in the Bardo was his first novel released in 2017 and I have had it on my reading list for a long time and I finally got into reading it. Thankfully I did so since it was such a pleasurable read.

Novel tells the story of aftermath of passing of Lincoln’s side and it was written in quite weird style. The style the novel was written is might definitely not so easily approachable for every reader and it started like that for me as well. Once you get into it, it start to open up. Definitely well written (despite the unusual style) and worth a read.

C++ and the Complexity of Return Values

Tue, 02 Apr 2024 12:38:17 +0200

In the domain of C++ optimization, we encounter two key players: Return Value Optimization (RVO) and Named Return Value Optimization (NRVO). These techniques excel in simplifying function returns by avoiding unnecessary object copies or moves, akin to a direct transfer from temporary to destination.

Looking at the example:

class Object {
public:
  Object() { std::cout << "Construct at " << this << "\n"; };

  ~Object() { std::cout << "Destruct at " << this << "\n"; };

  Object(const Object &) { std::cout << "Const Copy at " << this << "\n"; };

  Object(Object &&) { std::cout << "Move at " << this << "\n"; };

  Object &operator=(const Object &) {
    std::cout << "Const Copy Assignment at " << this << "\n";
    return *this;
  };

  Object &operator=(Object &&) {
    std::cout << "Move Assignment at " << this << "\n";
    return *this;
  };
};

Object TestRVO() { return Object{}; }

Object TestNRVO() {
  Object obj;
  return obj;
}

Object obj = TestRVO();
Object obj2 = TestNRVO();

Since C++17, RVO has been governed, and the moment when a constructor is invoked is termed “materialization”. C++17 specifies that materialization should be delayed as much as possible, typically until it binds to references or until a class’s member is accessed using the dot operator, or until an array is subscripted using square brackets or converted to a pointer. Alternatively, materialization occurs when the value is ultimately discarded, ensuring that at least one temporary is created. On the other hand, NRVO lacks regulation but is commonly implemented by proficient compilers.

RVO and NRVO can be disabled under certain circumstances, such as conditional returns:

(if (xx) return x; else return y;)

Returning a function parameter or global variable instead of a local variable, or returning a non-id-expression. This explains why using std::move(id) is discouraged when NRVO is available. In scenarios where RVO/NRVO is applicable, no move operations are triggered. Conversely, in their absence, move operations become inevitable. Another optimization step occurs where the compiler attempts to interpret an id-expression as an xvalue. Consequently, return x; is equivalent to return std::move(x);, obviating the need for copying.

For example:

struct S {
  S() = default;

  S(const S &) = delete;

  S(S &&) { std::cout << "here;"; };

  ~S() = default;
};

// NRVO happens, no move at all.
S foo() {
  S s;
  return s;
}

// BAD, NRVO can happen, but it's disabled, so move ctor is called.
S bar() {
  S s;
  return std::move(s);
}

// NRVO is disabled due to conditional return; however, the subsequent
// optimization ensures no unnecessary copy is made.
S foofoo(bool use) {
  S obj1, obj2;

  if (use)
    return obj1;

  return obj2;
}

// Similar to the previous function, but redundant std::move calls are added
// unnecessarily.
S barbar(bool use) {
  S obj1, obj2;

  if (use)
    return std::move(obj1);

  return std::move(obj2);
}

When NRVO and RVO optimizations fail, a copy operation ensues. In such cases, manual move operations are required if moving is preferred. Note that t.s isn’t considered an id-expression, hence both optimizations fail, necessitating explicit move calls like std::move(t.s) or std::move(t).s.

The subsequent optimization, officially known as implicit move, is a feature anticipated in C++23. Before its introduction, implicit moves are subject to stricter constraints compared to id-expressions.

Complexity of C++ never ceases to amaze me.

Getting Back to the Writing and Reading Saddle

Sun, 31 Mar 2024 20:59:00 +0200

So, again, I’ve been in this annoying procrastination rut when it comes to my writing (literature and coding) and reading. I’ve been quite active in other interests although, especially music, since I just recently had one gig after a long break of not gigging and it was super fun! But now, I’ve felt the urge to get back to books and programming, while still of course trying to keep musical activities relatively active.

I’ve started to read through couple new tech books focusing on compilers and garbage collection and also rereading some German literature classics, mainly for trying to learn the language. Also, I should get back to Sila development at the same time. Next steps for the Sila development is probably going to be to start working with code generation for aarch64. This is due to the fact that I recently got a new Macbook with ARM processor (M3 Max), so now I need to start worry about code generation for both x86-64 and aarch64. Which is probably going to be quite a lot of work but hey… it’s fun (I hope).

Hugo and Fixing RSS Errors

Fri, 02 Feb 2024 10:08:07 +0100

Somebody contacted me and mentioned that there is something off with my RSS feeds, thanks for that. I don’t know what had happened and when, but for some reason my normal RSS link. Didn’t contain the correct information. It seemed to only contain the summary of the post. I recently did some overhaul in my own Hugo theme and its layouts so most likely I had put something to wrong place while doing that causing this.

When I started looking at what Hugo generated, it seemed that the correct feed was actually located at the root of my site, so topikettunen.com/feed.xml but this isn’t right. I remember making it so at one point that I only generate the feed in the blog section since - personally speaking - I don’t really see the benefit of generating the feed elsewhere.

This seemed to be caused by error in my Hugo configs, which most likely I had accidentally changed at some point. I had to add the following to my config.toml:

[outputs]
  home = ["HTML"]
  section = ["HTML", "RSS"]
  taxonomy = ["HTML"]

This makes it so that Hugo generates only HTML when it comes to home and taxonomy page (so archive and tag for me). In section, like blog, it’ll generate HTML and RSS. So that was fixed.

Like I mentioned above, the RSS feed generated under section was still wrong since it only contained the summary. In my Hugo theme, I had already created a custom layouts/index.rss.xml in there with the change:

-<description>{{ .Summary | html }}</description>
+<description>{{ .Content | html }}</description>

So that it’ll generate the whole content in there instead of summary, which Hugo does by default. Unfortunately, this was one was in the wrong place for section RSS. It should be in layouts/section/section.rss.xml, and then it works fine!

Also having full content under description in the RSS XML seems quite odd so I fixed it to look like:

<description>{{ .Summary | html }}</description>
<content:encoded>{{ .Content | html }}</content:encoded>

To have both in their own correct fields.

I also noticed that my custom notice shortcode looked quite ugly in the RSS feed. The way I had done this shortcode was the having the following in layouts/shortcodes/notice.html:

<table class="notice">
  <tbody>
    <tr>
      <td>
        {{ .Inner }}
      </td>
    </tr>
  </tbody>
</table>

I know, this probably could be done with something else than table, but hey, I’m not a designer.

Which I then would call in my Markdown files like:

{{< notice >}}
<strong>Plug</strong>: <a href="https://git.sr.ht/~tok/sila" target="_blank">Follow the Sila development here.</a>
{{< /notice >}}

Also while writing this, I learned how to escape shortcodes in Hugo in case you want to use them in write them in your Markdown, but not actually use them. You need wrap the short code inside {{</* notice */>}}.

But unfortunately it would just generate that in the beginning of the RSS content without proper HTML tags etc. So instead of using the notice as a shortcode, I removed the shortcodes from Markdown and just added the following to my layouts/_default/single.html:

{{ if in .Params.tag "sila"}}
<table class="notice">
  <tbody>
    <tr>
      <td>
        <strong>Plug</strong>: <a href="https://git.sr.ht/~tok/sila" target="_blank">Follow the Sila development here.</a>
      </td>
    </tr>
  </tbody>
</table>
{{ end }}

This only works when blog has a tag sila in it. Of course, if I would like to use such a notice in some other posts, this wouldn’t work, but for now, it’s good enough for me. This way also Hugo doesn’t include the notice in the generated RSS, making the feed look little bit neater!

But hey, at least now the RSS feed should be working how it should!

Sila Dev Log: Initial Control Flow

Sat, 27 Jan 2024 13:44:45 +0100

Have had a small pause on programming and writing due to vacations etc., but fortunately, now I’m back to normal schedule, so I’ll try to get back to it as much as possible!

I recently wrote about implementing local variables to my compiler and next I would like to write about implementing the initial control flow to it. With the initial control flow, I mean stuff like loops and conditionals in a relatively simple fashion for now.

Starting with simple conditionals. In my language I’ve decided to go with the usual if and else just not to over-complicate things for me. Starting with the required assembly code. Considering code like:

if 1 == 0 { return 1; } return 0;

Nothing too complicated going on. Of course in that, it would never return 1, but let’s not focus on that for now. Assembly for something like this (with the function prologue and epilogue) would look something like this:

	; Prologue
	push %rbp
	mov %rsp, %rbp
	sub $0, %rsp

	mov $0, %rax	; Move the value 0 into the register %rax.
	push %rax	; Push the value of %rax onto the stack.
	mov $1, %rax	; Move the value 1 into the register %rax.
	pop %rdi	; Pop the topmost value from the stack and store it in the %rdi register.
	cmp %rdi, %rax	; Compare the values in %rdi and %rax.
	sete %al	; Set the least significant byte of the register %rax to 1 if the previous comparison was equal, otherwise, set it to 0.
	movzb %al, %rax	; Zero-extend %al (the least significant byte) to the entire %rax register.
	cmp $0, %rax	; Compare the value in %rax with 0.
	je .L.else.1	; Jump to .L.else.1 if the zero flag is set (if the previous comparison resulted in equality).
.L.else.1:		; Label for the else block.
	nop		; No operation (this instruction does nothing).
.L.end.1:		; Label for the end of the code block.
	mov $0, %rax	; Move the value 0 into %rax.

	; Epilogue
	mov %rbp, %rsp
	pop %rbp
	ret

This probably could be written more cleanly but for now I’m not too worried about code generation optimizations, so I let future Topi worry about those.

In the last part, where we focused on implementing our local variables, I already added support for doing tokenization for our keywords that are required for the control flow. So when it comes to lexer, everything should already be in order.

Parser on the other hand requires little bit of work. First, let’s implement structures for conditionals:

(defstruct (ast-node-cond
            (:include ast-node)
            (:copier nil))
  (expr (util:required 'expr) :type t :read-only t)
  (then (util:required 'then) :type t :read-only t)
  (else (util:required 'else) :type t :read-only t))

(defmethod next-node ((node ast-node-cond))
  (ast-node-cond-next node))

For now, conditionals will be parsed as a statement node, so we’ll need add cases for conditional keywords in our parse-statement-node:

(defun parse-statement-node (tok)
  (alexandria:switch ((lex:token-value tok) :test #'string=)
    #| [...] |#

    ("if"
     (parse-cond-statement-node (lex:token-next tok)))

    #| [...] |#))

parse-cond-statement-node on the other hand looks something like this:

(defun parse-cond-statement-node (tok)
  ;; TOK passed in should be the token after "if".
  (let (expr then else)
    (multiple-value-bind (expr-node rest)
        (parse-expression-node tok)
      (setf expr expr-node
            tok rest))

    (multiple-value-bind (then-node rest)
        (parse-statement-node tok)
      (setf then then-node
            tok rest))

    (when (string= (lex:token-value tok) "else")
      (multiple-value-bind (else-node rest)
          (parse-statement-node (lex:token-next tok))
        (setf else else-node
              tok rest)))

    (values (make-ast-node-cond :expr expr
                                :then then
                                :else else)
            tok)))

So we’ll parse the conditional out of

if <cond> { <then-node> } else { <else-node> }

and proceed to parse the bodies inside curly braces. Parsing of the bodies of the conditional proceed down in a recursive descent manner how we have implemented the parses earlier.

To be able to produce somewhat similar assembly as I wrote above, naturally, the code generation needs some work:

(defun generate-statement (node)
  (let ((insts (make-inst-array)))
    (cond
      #| [...] |#

      ((parser:ast-node-cond-p node)
       (incf *label-count*)

       (let ((count *label-count*))
         (do-vector-push-inst (generate-expression
                               (parser:ast-node-cond-expr node)) insts)
         (vector-push-extend "cmp $0, %rax" insts)
         (vector-push-extend (format nil "je .L.else.~d" count) insts)

         (do-vector-push-inst (generate-statement
                               (parser:ast-node-cond-then node)) insts)
         (vector-push-extend (format nil "jmp .L.end.~d" count) insts)

         (vector-push-extend (format nil ".L.else.~d:" count) insts)
         (if (parser:ast-node-cond-else node)
             (do-vector-push-inst (generate-statement
                                   (parser:ast-node-cond-else node)) insts)
             (vector-push-extend "nop" insts))

         (vector-push-extend (format nil ".L.end.~d:" count) insts)

         (unless (parser:next-node node)
           (vector-push-extend "nop" insts)))
       insts)

      #| [...] |#)))

Essentially what is happening here is that we compare the conditional in the given statement to 0 and if or not comparison is true, we’ll jump to a label .L.else.<label no> or we just skip it and proceed forward to instructions generated from the “then” node. Label numbering is naturally for the reason that pretty much always in real programs functions might have multitude of different conditional in varying levels of nesting. So we need to keep track of which conditional is in question at this point.

But when it comes to initial implementation of conditionals, that should be more or less it and we can proceed on to implementing loops.

Loops in many way works similarly to how conditionals were implemented above, since you often have conditionals e.g. in “for” loops. In assembly, how something like this would be implemented is basically the same as above, but in the end, we would just jump back to the beginning depending of the conditional of the loop itself.

So code something like

{ for ;; { return 3; } return 5; }

would equal to code like:

	.globl main
main:
	push %rbp
	mov %rsp, %rbp
	sub $0, %rsp
.L.begin.1:
	mov $3, %rax
	jmp .L.return
	jmp .L.begin.1
.L.end.1:
	mov $5, %rax
	jmp .L.return
.L.return:
	mov %rbp, %rsp
	pop %rbp
	ret

Or little bit more complex for loop

{ i:=0; j:=0;for i:=0; i<=10; i:=i+1 { j:= i+j; } return j; }

would equal to:

	.globl main
main:
	push %rbp
	mov %rsp, %rbp
	sub $16, %rsp
	lea -16(%rbp), %rax
	push %rax
	mov $0, %rax
	pop %rdi
	mov %rax, (%rdi)
	lea -8(%rbp), %rax
	push %rax
	mov $0, %rax
	pop %rdi
	mov %rax, (%rdi)
	lea -16(%rbp), %rax
	push %rax
	mov $0, %rax
	pop %rdi
	mov %rax, (%rdi)
.L.begin.1:
	mov $10, %rax
	push %rax
	lea -16(%rbp), %rax
	mov (%rax), %rax
	pop %rdi
	cmp %rdi, %rax
	setle %al
	movzb %al, %rax
	cmp $0, %rax
	je .L.end.1
	lea -8(%rbp), %rax
	push %rax
	lea -8(%rbp), %rax
	mov (%rax), %rax
	push %rax
	lea -16(%rbp), %rax
	mov (%rax), %rax
	pop %rdi
	add %rdi, %rax
	pop %rdi
	mov %rax, (%rdi)
	lea -16(%rbp), %rax
	push %rax
	mov $1, %rax
	push %rax
	lea -16(%rbp), %rax
	mov (%rax), %rax
	pop %rdi
	add %rdi, %rax
	pop %rdi
	mov %rax, (%rdi)
	jmp .L.begin.1
.L.end.1:
	lea -8(%rbp), %rax
	mov (%rax), %rax
	jmp .L.return
.L.return:
	mov %rbp, %rsp
	pop %rbp
	ret

Again, let’s start by adding structures for our loops:

(defstruct (ast-node-for
             (:include ast-node)
             (:copier nil))
  (init (util:required 'init) :type t :read-only t)
  (inc (util:required 'inc) :type t :read-only t)
  (cond (util:required 'cond) :type t :read-only t)
  (body (util:required 'body) :type t :read-only t))

(defmethod next-node ((node ast-node-for))
  (ast-node-for-next node))

(defstruct (ast-node-loop
             (:include ast-node)
             (:copier nil))
  (body (util:required 'body) :type t :read-only t))

(defmethod next-node ((node ast-node-loop))
  (ast-node-loop-next node))

(defstruct (ast-node-break
             (:include ast-node)
             (:copier nil))
  (depth (util:required 'depth) :type integer :read-only t))

(defmethod next-node ((node ast-node-break))
  (ast-node-break-next node))

I’ve included ast-node-loop in here, which in my language depicts while loop. Similarly to e.g. Rust.

Parsing loops will similarly to conditionals happen in parse-statement-node which in current state looks fully something like this:

(defvar *break-depth* 0
  "Depth counter for BREAK keyword to know from what level it should break out.")

(defun parse-statement-node (tok)
  (alexandria:switch ((lex:token-value tok) :test #'string=)
    ("return"
     (multiple-value-bind (node rest)
         (parse-expression-node (lex:token-next tok))
       (values (make-ast-node-return :expr node)
               (lex:token-next (skip-to-token ";" rest)))))

    ("if"
     (parse-cond-statement-node (lex:token-next tok)))

    ("for"
     (parse-for-statement-node (lex:token-next tok)))

    ("loop"
     (parse-loop-statement-node (lex:token-next tok)))

    ("break"
     (values (make-ast-node-break :depth *break-depth*)
             (lex:token-next (skip-to-token ";" tok))))

    ("{"
     (parse-compound-statement-node (lex:token-next tok)))

    (otherwise
     (parse-expression-statement-node tok))))

Notice the *break-depth* variable, that’ll work in similar fashion as our label counter in the code generation for conditionals.

Parsing functions for our loop keywords looks like this:

(defun parse-for-statement-node (tok)
  ;; TOK passed in should be the token after "for"
  (let (init cond inc body)
    (multiple-value-bind (init-node rest)
        (parse-expression-statement-node tok)
      (setf init init-node
            tok rest))

    (unless (string= (lex:token-value tok) ";")
      (multiple-value-bind (cond-node rest)
          (parse-expression-node tok)
        (setf cond cond-node
              tok rest)))
    (setf tok (skip-to-token ";" tok))

    ;; Entering "for" scope.
    (incf *break-depth*)

    (unless (string= (lex:token-value (lex:token-next tok)) "{")
      (multiple-value-bind (inc-node rest)
          (parse-expression-node (lex:token-next tok))
        (setf inc inc-node
              tok rest)))
    (setf tok (skip-to-token "{" tok))

    (multiple-value-bind (body-node rest)
        (parse-statement-node tok)
      (setf body body-node
            tok rest))

    ;; Left "for" scope.
    (decf *break-depth*)

    (values (make-ast-node-for :init init
                               :cond cond
                               :inc inc
                               :body body)
            tok)))

(defun parse-loop-statement-node (tok)
  ;; TOK passed in should be the opening brace of the block after the "loop"
  ;; keyword.
  ;; TODO(topi): Add proper error handling.
  (assert (string= (lex:token-value tok) "{"))

  ;; Entering "loop" scope.
  (incf *break-depth*)

  (multiple-value-bind (body rest)
      (parse-statement-node tok)

    ;; Left "loop" scope.
    (decf *break-depth*)

    (values (make-ast-node-loop :body body)
            rest)))

So as you can see, parsing loops work in a very similar fashion to how conditionals are parsed since the structure for the syntax is quite similar. Same also applies to code generation so our generate-statement currently fully looks like this:

(defun generate-statement (node)
  (let ((insts (make-inst-array)))
    (cond
      ((parser:ast-node-block-p node)
       (loop :for body := (parser:ast-node-block-body node)
               :then (setf body (parser:next-node body))
             :until (null body)
             :do (do-vector-push-inst (generate-statement body) insts))
       insts)

      ((parser:ast-node-return-p node)
       (do-vector-push-inst (generate-expression
                             (parser:ast-node-return-expr node)) insts)
       (vector-push-extend "jmp .L.return" insts)
       insts)

      ((parser:ast-node-break-p node)
       (vector-push-extend (format nil "jmp .L.end.~d"
                                   (parser:ast-node-break-depth node)) insts)
       insts)

      ((parser:ast-node-cond-p node)
       (incf *label-count*)

       (let ((count *label-count*))
         (do-vector-push-inst (generate-expression
                               (parser:ast-node-cond-expr node)) insts)
         (vector-push-extend "cmp $0, %rax" insts)
         (vector-push-extend (format nil "je .L.else.~d" count) insts)

         (do-vector-push-inst (generate-statement
                               (parser:ast-node-cond-then node)) insts)
         (vector-push-extend (format nil "jmp .L.end.~d" count) insts)

         (vector-push-extend (format nil ".L.else.~d:" count) insts)
         (if (parser:ast-node-cond-else node)
             (do-vector-push-inst (generate-statement
                                   (parser:ast-node-cond-else node)) insts)
             (vector-push-extend "nop" insts))

         (vector-push-extend (format nil ".L.end.~d:" count) insts)

         (unless (parser:next-node node)
           (vector-push-extend "nop" insts)))
       insts)

      ((parser:ast-node-for-p node)
       (incf *label-count*)

       (let ((count *label-count*))
         (do-vector-push-inst (generate-statement
                               (parser:ast-node-for-init node)) insts)
         (vector-push-extend (format nil ".L.begin.~d:" count) insts)

         (when (parser:ast-node-for-cond node)
           (do-vector-push-inst (generate-expression
                                 (parser:ast-node-for-cond node)) insts)
           (vector-push-extend "cmp $0, %rax" insts)
           (vector-push-extend (format nil "je .L.end.~d" count) insts))

         (do-vector-push-inst (generate-statement
                               (parser:ast-node-for-body node)) insts)

         (when (parser:ast-node-for-inc node)
           (do-vector-push-inst (generate-expression
                                 (parser:ast-node-for-inc node)) insts))

         (vector-push-extend (format nil "jmp .L.begin.~d" count) insts)
         (vector-push-extend (format nil ".L.end.~d:" count) insts)

         (unless (parser:next-node node)
           (vector-push-extend "nop" insts)))
       insts)

      ((parser:ast-node-loop-p node)
       (incf *label-count*)

       (let ((count *label-count*))
         (vector-push-extend (format nil ".L.begin.~d:" count) insts)

         (do-vector-push-inst (generate-statement
                               (parser:ast-node-loop-body node)) insts)

         (vector-push-extend (format nil "jmp .L.begin.~d" count) insts)
         (vector-push-extend (format nil ".L.end.~d:" count) insts)

         (unless (parser:next-node node)
           (vector-push-extend "nop" insts)))
       insts)

      ((parser:ast-node-expression-p node)
       (do-vector-push-inst (generate-expression
                             (parser:ast-node-expression-expr node)) insts)
       insts))))

Mainly the difference between loops and conditionals is just the label .L.begin.<label no> which we use in case we want to jump back to the beginning of the loop with jmp .L.begin.<label no>.

And that’s about it! I have already implemented tests for these and they should now be be passing!

Testing System sila/tests

;; testing 'sila/tests/compiler'
test-compilation-and-compare-rc
  Integer
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0; }" 0) to be true. (1013ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ ;;;;; return 1; }" 1) to be true. (1018ms)
  Arithmetics
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 5 + 40 - 20; }" 25) to be true. (1022ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 2 / (1 + 1) * 8; }" 8) to be true. (1015ms)
  Unary
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return - -10; }" 10) to be true. (1011ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return -10+20; }" 10) to be true. (1022ms)
  Comparisons
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0==1; }" 0) to be true. (1023ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1!=1; }" 0) to be true. (1067ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0==0; }" 1) to be true. (1059ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1!=0; }" 1) to be true. (1059ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0<1; }" 1) to be true. (1061ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1<1; }" 0) to be true. (1087ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1<=1; }" 1) to be true. (1042ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 2<=1; }" 0) to be true. (1055ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0<1; }" 1) to be true. (1021ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1<1; }" 0) to be true. (1014ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1>=1; }" 1) to be true. (1025ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1>=2; }" 0) to be true. (1014ms)
  Multiple statements
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1; 2; 3; }" 1) to be true. (1019ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ 1; return 2; 3; }" 2) to be true. (1023ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ 1; 2; return 3; }" 3) to be true. (1031ms)
  Variables
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ a:=8; return a; }" 8) to be true. (1043ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ a:=3; b:=5; return a+b; }" 8) to be true. (1024ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ foo:=3; bar:=5; return foo+bar; }" 8) to be true. (1032ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC
              "{ foo2:=3; bar2:=5; return foo2+bar2; }" 8) to be true. (1056ms)
  Block
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ 1; { 2; } return 3; }" 3) to be true. (1070ms)
  Conditionals
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ if 1 { return 1; } return 2; }" 1) to be true. (1085ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC
              "{ if 0 { return 1; } else { return 2; } }" 2) to be true. (1077ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC
              "{ if 1<0 { return 1; } else { return 2; } }" 2) to be true. (1024ms)
  For loop
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ for ;; { return 3; } return 5; }" 3) to be true. (1011ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC
              "{ i:=0; j:=0;for i:=0; i<=10; i:=i+1 {j := i+j;} return j; }" 55) to be true. (1019ms)
  Loop
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ loop { return 3; } return 5; }" 3) to be true. (1046ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC
              "{ i := 0; loop { i := 3; break; } return i; }" 3) to be true. (1030ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC
              "{ i := 0; loop { i := i + 1; if i == 10 { break; } } return i; }" 10) to be true. (1038ms)

;; testing 'sila/tests/codegen'
test-codegen-x86-64
  Integer
    ✓ { return 0; }
    ✓ { return 42; }
  Add and subtraction
    ✓ { return 5+20-4; }
    ✓ { return    5   +  20  -  4   ; }
  Division and multiplication
    ✓ { return 2 / (1 + 1) * 8; }
  Unary
    ✓ { return - -10; }
    ✓ { return -10+20; }
    ✓ { return - - -10; }
  Comparison
    ✓ { return 1==1; }
    ✓ { return 1>=1; }
    ✓ { return 1<=1; }
    ✓ { return 1<1; }
    ✓ { return 1>1; }
  Multiple statements
    ✓ { return 1;2;3; }
    ✓ { 1;return 2;3; }
    ✓ { 1;2;return 3; }
  Variables
    ✓ { a:=8;return a; }
    ✓ { foo:=5;bar:=8;return foo+bar; }
  Block
    ✓ { 1; { 2; } return 3; }
  Conditional
    ✓ { if 1 == 0 { return 1; } return 0; }
    ✓ { if 0 { return 1; } else { return 2; } }
  For loop
    ✓ { for ;; { return 3; } return 5; }
    ✓ { i:=0; j:=0;for i:=0; i<=10; i:=i+1 {j := i+j;} return j; }

✓ 1 test completed

Summary:
  All 1 test passed.

List of Now Playing

Wed, 29 Nov 2023 18:33:31 +0100

I just decided to gather all the songs I have included in my posts in the “Now playing” section to one neat page. This page can be found here.

Sila Dev Log: Implementing Local Variables

Thu, 16 Nov 2023 21:14:23 +0100

I’ve been little bit slacking on the writing aspects of things but also little bit on the side of adding new features to the language as well. So much of my time have been spent on various bikeshedding topics. Although important work nonetheless. I won’t be writing too much about those cleanups and refactorings but you can follow the development of Sila from the link above.

After the bikeshedding, or yak-shaving, I got to writing new features such as initial implementation of the control flow for the language (conditionals and loops). So I could open a little bit on the development of those.

Before I started implementing either conditionals or loops, I wanted to implement the ability to assign local variables, such as a := 0. Starting with tokenization, I needed to differentiate somehow between identifiers and reserved keywords. Starting with my tokenize, adding support for those was simply adding following to it:

(defun tokenize (src)
  "Generate tokens from the given source code."
  (let* ((head (make-token))
         (cur head)
         (src-pos 0))

    (macrolet ((gentoken (kind)
                 (let ((token-gen-fn (intern (format nil "GEN-~a-TOKEN" kind))))
                   `(multiple-value-bind (token pos)
                        (,token-gen-fn src src-pos)
                      (setf (token-next cur) token)
                      (setf cur (token-next cur))
                      (setf src-pos pos)))))

      (loop :while (< src-pos (length src))
            :do (cond
                  ;; [...]

                  ;; Ident or keyword
                  ((alpha-char-p (char src src-pos))
                   (gentoken ident-or-keyword))

                  (t
                   (error 'lexer-error
                          :lexer-input src
                          :error-msg "Invalid token."
                          :token-pos src-pos)))))
    ;; No more tokens.
    (setf (token-next cur) (make-token :kind :eof :position src-pos))
    (setf cur (token-next cur))

    (token-next head)))

Which basically checks that if the current character in the given source code starts with a letter, it should be tokenized as either identifier or keyword. I wrote a simple helper macro genmacro to cleanup some code, which when called with something like (genmacro ident-or-keyword) would call function gen-ident-or-keyword and inserts the generated token to the token list.

gen-ident-or-keyword itself looks like this:

(defvar *sila-keywords*
  #("return" "if" "else" "for" "loop" "break"))

(defun gen-ident-or-keyword-token (src src-pos)
  "Generate IDENT or KEYWORD token and return it and the SRC-POS to the next
token in SRC."
  (flet ((keyword-lookup (input pos)
           (let ((keyword-end (skip-to #'whitespacep input pos)))
             ;; Keyword not found
             (when (null keyword-end)
               (return-from keyword-lookup (values nil pos)))
             (let ((keyword (subseq input pos keyword-end)))
               (if (find keyword *sila-keywords* :test #'string=)
                   (values keyword
                           (skip-to #'(lambda (c) (not (whitespacep c)))
                                    input keyword-end))
                   (values nil pos))))))

    (multiple-value-bind (keyword next-token-pos)
        (keyword-lookup src src-pos)

      (let* ((punct-pos (skip-to #'punctuatorp src src-pos))
             (token-len (cond (keyword (length keyword))
                              (punct-pos (- punct-pos src-pos))
                              (t (- (length src) src-pos))))
             (token-val (if keyword
                            keyword
                            (trim-whitespace
                             (subseq src src-pos (+ src-pos token-len))))))

        (setf src-pos (cond (keyword next-token-pos)
                            (punct-pos punct-pos)
                            (t (length src))))

        (values (make-token :kind (if keyword :keyword :ident)
                            :value token-val
                            :length (length token-val)
                            :position src-pos)
                src-pos)))))

Which is pretty straight-forward function that tries to tokenize something like a := 0; into a following structure:

SILA/LEXER> (tokenize "a := 0;")
#S(TOKEN
   :KIND :IDENT
   :POSITION 2
   :LENGTH 1
   :VALUE "a"
   :NEXT #S(TOKEN
            :KIND :PUNCT
            :POSITION 4
            :LENGTH 2
            :VALUE ":="
            :NEXT #S(TOKEN
                     :KIND :NUM
                     :POSITION 6
                     :LENGTH 1
                     :VALUE "0"
                     :NEXT #S(TOKEN
                              :KIND :PUNCT
                              :POSITION 7
                              :LENGTH 1
                              :VALUE ";"
                              :NEXT #S(TOKEN
                                       :KIND :EOF
                                       :POSITION 7
                                       :LENGTH 0
                                       :VALUE ""
                                       :NEXT NIL)))))

One thing I immediately noticed with my current tokenization setup was that if I wrote an identifier that start with a number, it would have been tokenized similarly to how rest of the numbers would get tokenized. Which of course wouldn’t work with identifiers. So I had to add a simple error handling in the number generation side to check if the identifier can’t start with numbers:

(defun gen-number-token (src src-pos)
  "Generate token for NUMBER and return it and the SRC-POS to the next token in
SRC."
   #| [...] |#

    ;; Idents starting with a letter will be caught with
    ;; a different conditional so if this is hit, ident
    ;; starts with a number but contains letters, which
    ;; isn't acceptable.
    (unless (every #'digit-char-p token-val)
      (error 'lexer-error
             :lexer-input src
             :error-msg "Ident can't start with a number."
             :token-pos src-pos))

    #| [...] |#)

This setup also handles keywords as intended, which naturally will be needed for the implementation of the control flow:

SILA/LEXER> (print-tokens (tokenize "{ a := 0; return a; }"))
#S(TOKEN :KIND PUNCT	:POSITION 1	:LENGTH 1	:VALUE {)
#S(TOKEN :KIND IDENT	:POSITION 4	:LENGTH 1	:VALUE a)
#S(TOKEN :KIND PUNCT	:POSITION 6	:LENGTH 2	:VALUE :=)
#S(TOKEN :KIND NUM	:POSITION 8	:LENGTH 1	:VALUE 0)
#S(TOKEN :KIND PUNCT	:POSITION 9	:LENGTH 1	:VALUE ;)
#S(TOKEN :KIND KEYWORD	:POSITION 17	:LENGTH 6	:VALUE return)
#S(TOKEN :KIND IDENT	:POSITION 18	:LENGTH 1	:VALUE a)
#S(TOKEN :KIND PUNCT	:POSITION 19	:LENGTH 1	:VALUE ;)
#S(TOKEN :KIND PUNCT	:POSITION 21	:LENGTH 1	:VALUE })
#S(TOKEN :KIND EOF	:POSITION 21	:LENGTH 0	:VALUE )
; No value

Parsing side of things is little bit more involved on the other hand. I started the implementation with writing following structures:

(defstruct (ast-node-variable
            (:include ast-node)
            (:copier nil))
  (object (util:required 'object) :type object :read-only t))

(defstruct (object
            (:copier nil))
  (name (util:required 'name) :type string :read-only t)
  (offset 0 :type integer)
  (next nil :type t))

(defstruct (ast-node-assign
            (:include ast-node)
            (:copier nil))
  (var (util:required 'var) :type ast-node-variable :read-only t)
  (expr (util:required 'expr) :type t :read-only t))

Parsing the variables and assign statements then looks like this:

(defun parse-assign-node (tok)
  (let (node)
    (multiple-value-bind (eql-node rest)
        (parse-equality-node tok)
      (setf node eql-node
            tok rest))

    (when (string= (lex:token-value tok) ":=")
      (multiple-value-bind (expr-node rest)
          (parse-assign-node (lex:token-next tok))
        (setf node (make-ast-node-assign :var node
                                         :expr expr-node)
              tok rest)))

    (values node tok)))

;;; [...]

(defvar *local-variables* nil
  "Global variable for holding local variable objects.")

(defun parse-primary-node (tok)
  (cond
    #| [...] |#

    ((eq (lex:token-kind tok) :ident)
     (let* ((name (lex:token-value tok))
            (var (find-local-var name)))

       (when (null var)
         (setf var (make-object :name name :next *local-variables*))
         ;; New object should be in front of the list.
         (setf *local-variables* var))

       (values (make-ast-node-variable :object var)
               (lex:token-next tok))))

    #| [...] |#

    (t (error "Unexpected token value: ~a" tok))))

Since I’m using recursive descent parsing, parsing the variable node happens via the equality parsing function which trickles down all the way to primary node. Again relatively straight forward parsing here. One thing to worth noting is how I save all the local variables to separate variable called *local-variables*. This is mainly for the fact that when it comes to code generation I need to save the fixed offsets in the memory for each variable in use. So I need to know all the variables that I have parsed. There is probably bunch of optimizations that could be done, but for now, I’m not interested in those.

Setting those variable offsets happens after the whole program has been parsed:

(defstruct (func
            (:copier nil))
  (body (util:required 'body) :type t :read-only t)
  (locals (util:required locals) :type t :read-only t)
  (stack-size 0 :type integer))

;;; [...]

(defun parse-program (tok)
  (labels ((align-to (n align)
             "Round N to the nearest multiple of ALIGN."
             (* (ceiling n align) align))

           (set-lvar-offsets (program)
             (let ((offset 0))
               (loop :for obj := (func-locals program)
                       :then (setf obj (object-next obj))
                     :until (null obj)
                     :do (progn
                           (incf offset 8)
                           (setf (object-offset obj) (- offset))))
               (setf (func-stack-size program) (align-to offset 16)))
             (values)))

    (let* ((head (make-ast-node))
           (cur head))

      (loop :until (eq (lex:token-kind tok) :eof)
            :do (multiple-value-bind (node rest)
                    (parse-statement-node tok)
                  (setf (ast-node-next cur) node)
                  (setf cur (ast-node-next cur))
                  (setf tok rest)))

      (let ((program (make-func :body (ast-node-next head)
                                :locals *local-variables*)))
        (set-lvar-offsets program)
        program))))

And that’s about for tokenization and parsing. When parsing a program with local variables it should look something like this:

SILA/PARSER> (parse-program (lex:tokenize "{ a := 0; return a; }"))
#S(FUNC
   :BODY #S(AST-NODE-BLOCK
            :NEXT NIL
            :BODY #S(AST-NODE-EXPRESSION
                     :NEXT #S(AST-NODE-RETURN
                              :NEXT NIL
                              :EXPR #S(AST-NODE-VARIABLE
                                       :NEXT NIL
                                       :OBJECT #S(OBJECT
                                                  :NAME "a"
                                                  :OFFSET -8
                                                  :NEXT NIL)))
                     :EXPR #S(AST-NODE-ASSIGN
                              :NEXT NIL
                              :VAR #S(AST-NODE-VARIABLE
                                      :NEXT NIL
                                      :OBJECT #S(OBJECT
                                                 :NAME "a"
                                                 :OFFSET -8
                                                 :NEXT NIL))
                              :EXPR #S(AST-NODE-INTEGER-LITERAL
                                       :NEXT NIL
                                       :VALUE 0))))
   :LOCALS #S(OBJECT :NAME "a" :OFFSET -8 :NEXT NIL)
   :STACK-SIZE 16)

Finally, we can proceed to the code generation aspects!

To implement local variables with x86 assembly language, we need to write epilogue and prologue for our function (currently only main function) which prepares the stack for use within the function. These will look like this:

	; Prologue
	push %rbp
	mov %rsp, %rbp
	sub STACK_SIZE, %rsp

	; Epilogue
	mov %rbp, %rsp
	pop %rbp
	ret

So what’s happening here:

Prologue:
- push %rbp: Saves the value of the base pointer register (%rbp) onto the stack.
- mov %rsp, %rbp: Sets the value of the stack pointer (%rsp) as the new base pointer (%rbp).
- sub STACK_SIZE, %rsp: Allocates space on the stack for local variables by subtracting a amount required by our local variables (STACK_SIZE, calculated above) from the stack pointer (%rsp).
Epilogue:
- mov %rbp, %rsp: Restores the stack pointer to its original value by copying the value of the base pointer back to the stack pointer.
- pop %rbp: Restores the original value of the base pointer by popping it from the stack.
- ret: Returns control flow back to the calling function.

in x86, similar thing could be achieved with enter and leave instructions but they do little bit more than just pushing/popping and moving. So I might start using those at some point if the prologue and epilogue starts to get more complicated. For now, this raw assembly is good enough for me.

So, if we want to write generate x86-64 code for the code we have above ({ a := 0; return a; }), we have to write the following assembly:

	.globl main
main:
	; Prologue
	push %rbp
	mov %rsp, %rbp
	sub $16, %rsp

	; Saving 'a' to an address offset relative to the base pointer.
	lea -8(%rbp), %rax
	push %rax
	mov $0, %rax
	pop %rdi
	mov %rax, (%rdi)

	; Calculating the address offset of 'a' and moving it to rax for return.
	lea -8(%rbp), %rax
	mov (%rax), %rax

	; Epilogue
	mov %rbp, %rsp
	pop %rbp
	ret

Again, this code could be cleaned up and optimized quite a bit and doesn’t need to be so involved with simple code like this, but I’m leaving the code generation optimizations tasks for future me.

With the code above, we now have a great test for starting to implement the code generation itself. In that, with the AST tree that we generated above, generating code for it is also pretty trivial:

(defun generate-expression (node)
  "Recursively generate the x86-64 assembly code."
  (flet ((lea (node)
           "Load effective address"
           (format nil "lea ~d(%rbp), %rax"
                   (parser:object-offset
                    (parser:ast-node-variable-object node)))))
    (let ((insts (make-inst-array)))
      (cond
        #| [...] |#

        ((parser:ast-node-variable-p node)
         (vector-push-extend (lea node) insts)
         (vector-push-extend "mov (%rax), %rax" insts)
         insts)

        ((parser:ast-node-assign-p node)
         (vector-push-extend (lea (parser:ast-node-assign-var node)) insts)
         (vector-push-extend (asm-push) insts)
         (do-vector-push-inst (generate-expression
                               (parser:ast-node-assign-expr node)) insts)
         (vector-push-extend (asm-pop "rdi") insts)
         (vector-push-extend "mov %rax, (%rdi)" insts)
         insts)

        #| [...] |#))))

So the code above essentially is for the following assembly code:

	lea OFFSET(%rbp), %rax
	push %rax
	mov VALUE, %rax
	pop %rdi
	mov %rax, (%rdi)

In case we want to return the value we need to do the following:

(defun generate-statement (node)
  (let ((insts (make-inst-array)))
    (cond
      #| [...] |#

      ((parser:ast-node-return-p node)
       (do-vector-push-inst (generate-expression
                             (parser:ast-node-return-expr node)) insts)
       (vector-push-extend "jmp .L.return" insts)
       insts)

      #| [...] |#)))

Which is for generating assembly code for statement like return <expr>;. Here I decided the use jmp jmp .L.return, so in case I would write an return statement deeply nested in other statements, like if or for etc., I could just exit early from those and jump directly to the epilogue.

So printing the whole program might look something like this:

(defun emit-code (src &key (stream nil) (indent 2) (indent-tabs t))
  "Emit assembly code from given source code. Currently emits only x86-64 and
only Linux is tested."
  ;; Init environment
  (setf parser:*local-variables* nil
        *stack-depth* 0
        *label-count* 0)

  (let ((indent (if indent-tabs
                    #\Tab
                    (coerce (make-list indent
                                       :initial-element #\Space)
                            'string))))

    (let ((program (parser:parse-program (lex:tokenize src))))

      ;; TODO(topi): these instructions probably should be collected to some
      ;; structure so they can be divided in to sections more easily when the
      ;; programs become more complex.
      (format stream
              "~{~a~%~}"
              (alexandria:flatten
               (list
                ;; ASM Directive
                (format nil "~a.globl main" indent)

                ;; Main Label
                "main:"

                ;; Prologue
                (format nil "~apush %rbp" indent)
                (format nil "~amov %rsp, %rbp" indent)
                (format nil "~asub $~a, %rsp" indent
                        (parser:func-stack-size program))

                ;; ASM Routine
                (loop :for inst
                        :across (generate-statement (parser:func-body program))
                      :collect (if (string= (subseq inst 0 3) ".L.")
                                   ;; If instruction is label (.L. prefix),
                                   ;; don't indent it.
                                   inst
                                   (format nil "~a~a" indent inst)))

                ;; Return label
                ".L.return:"

                ;; Epilogue
                (format nil "~amov %rbp, %rsp" indent)
                (format nil "~apop %rbp" indent)

                ;; Return
                (format nil "~aret" indent)))))))

Parameters stream, indent and indent-tabs are not needed for the functionality of code generation itself, but they are just used here as helpers.

With that we can write a couple of test to see that programs actually work as intended:

(deftest test-compilation-and-compare-rc
  (testing "Integer"
    (ok (compile-program-and-compare-rc "{ return 0; }" 0))
    (ok (compile-program-and-compare-rc "{ ;;;;; return 1; }" 1)))

  (testing "Arithmetics"
    (ok (compile-program-and-compare-rc "{ return 5 + 40 - 20; }" 25))
    (ok (compile-program-and-compare-rc "{ return 2 / (1 + 1) * 8; }" 8)))

  (testing "Unary"
    (ok (compile-program-and-compare-rc "{ return - -10; }" 10))
    (ok (compile-program-and-compare-rc "{ return -10+20; }" 10)))

  (testing "Comparisons"
    (ok (compile-program-and-compare-rc "{ return 0==1; }" 0))
    (ok (compile-program-and-compare-rc "{ return 1!=1; }" 0))
    (ok (compile-program-and-compare-rc "{ return 0==0; }" 1))
    (ok (compile-program-and-compare-rc "{ return 1!=0; }" 1))
    (ok (compile-program-and-compare-rc "{ return 0<1; }" 1))
    (ok (compile-program-and-compare-rc "{ return 1<1; }" 0))
    (ok (compile-program-and-compare-rc "{ return 1<=1; }" 1))
    (ok (compile-program-and-compare-rc "{ return 2<=1; }" 0))
    (ok (compile-program-and-compare-rc "{ return 0<1; }" 1))
    (ok (compile-program-and-compare-rc "{ return 1<1; }" 0))
    (ok (compile-program-and-compare-rc "{ return 1>=1; }" 1))
    (ok (compile-program-and-compare-rc "{ return 1>=2; }" 0)))

  (testing "Multiple statements"
    (ok (compile-program-and-compare-rc "{ return 1; 2; 3; }" 1))
    (ok (compile-program-and-compare-rc "{ 1; return 2; 3; }" 2))
    (ok (compile-program-and-compare-rc "{ 1; 2; return 3; }" 3)))

  (testing "Variables"
    (ok (compile-program-and-compare-rc "{ a:=8; return a; }" 8))
    (ok (compile-program-and-compare-rc "{ a:=3; b:=5; return a+b; }" 8))
    (ok (compile-program-and-compare-rc "{ foo:=3; bar:=5; return foo+bar; }" 8))
    (ok (compile-program-and-compare-rc "{ foo2:=3; bar2:=5; return foo2+bar2; }" 8)))

  (testing "Block"
    (ok (compile-program-and-compare-rc "{ 1; { 2; } return 3; }" 3))))

Testing System sila/tests

;; testing 'sila/tests/compiler'
test-compilation-and-compare-rc
  Integer
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0; }" 0) to be true. (492ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ ;;;;; return 1; }" 1) to be true. (441ms)
  Arithmetics
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 5 + 40 - 20; }" 25) to be true. (423ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 2 / (1 + 1) * 8; }" 8) to be true. (431ms)
  Unary
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return - -10; }" 10) to be true. (419ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return -10+20; }" 10) to be true. (454ms)
  Comparisons
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0==1; }" 0) to be true. (476ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1!=1; }" 0) to be true. (483ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0==0; }" 1) to be true. (470ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1!=0; }" 1) to be true. (491ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0<1; }" 1) to be true. (496ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1<1; }" 0) to be true. (487ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1<=1; }" 1) to be true. (474ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 2<=1; }" 0) to be true. (498ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 0<1; }" 1) to be true. (503ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1<1; }" 0) to be true. (502ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1>=1; }" 1) to be true. (471ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1>=2; }" 0) to be true. (418ms)
  Multiple statements
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ return 1; 2; 3; }" 1) to be true. (433ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ 1; return 2; 3; }" 2) to be true. (426ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ 1; 2; return 3; }" 3) to be true. (441ms)
  Variables
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ a:=8; return a; }" 8) to be true. (432ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ a:=3; b:=5; return a+b; }" 8) to be true. (454ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ foo:=3; bar:=5; return foo+bar; }" 8) to be true. (467ms)
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC
              "{ foo2:=3; bar2:=5; return foo2+bar2; }" 8) to be true. (452ms)
  Block
    ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "{ 1; { 2; } return 3; }" 3) to be true. (427ms)

;; testing 'sila/tests/codegen'
test-codegen-x86-64
  Integer
    ✓ { return 0; }
    ✓ { return 42; }
  Add and subtraction
    ✓ { return 5+20-4; }
    ✓ { return    5   +  20  -  4   ; }
  Division and multiplication
    ✓ { return 2 / (1 + 1) * 8; }
  Unary
    ✓ { return - -10; }
    ✓ { return -10+20; }
    ✓ { return - - -10; }
  Comparison
    ✓ { return 1==1; }
    ✓ { return 1>=1; }
    ✓ { return 1<=1; }
    ✓ { return 1<1; }
    ✓ { return 1>1; }
  Multiple statements
    ✓ { return 1;2;3; }
    ✓ { 1;return 2;3; }
    ✓ { 1;2;return 3; }
  Variables
    ✓ { a:=8;return a; }
    ✓ { foo:=5;bar:=8;return foo+bar; }
  Block
    ✓ { 1; { 2; } return 3; }

✓ 1 test completed

Summary:
  All 1 test passed.

He-hey! Tests seemed to pass! Until next time!

Giving Mastodon a Go

Fri, 13 Oct 2023 21:20:20 +0200

NB: I’ve since decided to quit Mastodon

In the digital landscape, social media is an indispensable tool for communication, networking, and content sharing. Yet, the way these platforms have been implemented and designed isn’t praiseworthy. From content suppression algorithms to opaque data policies, it’s no secret that the leading tech giants have cultivated a restrictive environment.

I’ve been vocal about social media and how wrong it has always felt to me, leading me to delete most of my social media accounts apart from LinkedIn. However, is LinkedIn truly a social media platform or merely a glorified job board? I also wrote a post about social media from the perspective of digital minimalism some time ago.

For years, I was content without any social media accounts, and still am. Recently, I yearned for an online community to engage in meaningful discussions and banter, perhaps due to my move to a new country last year or simply because of an extended hiatus from social media.

I didn’t want to return to the draconian social media platforms I once used, like Facebook, Instagram, and Twitter. Even Reddit didn’t pique my interest. Nevertheless, I sought something new, which led me to consider Mastodon.

I had followed Mastodon’s development from afar due to my reluctance to join, not because Mastodon is malevolent, but because I felt I spent too much time on computers, and joining might exacerbate that.

However, after stumbling upon intriguing threads (is there a Mastodon-specific term for these?), I decided to join Mastodon and participate in discussions. Intrigued by its promise of a democratic and transparent social media experience, I took the plunge and joined the Mastodon community, finding it a breath of fresh air.

I’m thrilled to contribute to a platform that values user autonomy, fosters genuine human connections, and champions transparency and inclusivity. Embracing Mastodon signifies not just a personal choice but also a deliberate step toward fostering a more democratic and empowering social media environment for all.

In a world rife with censorship and control, Mastodon serves as a beacon of hope, embodying technological innovation that prioritizes user well-being and digital freedom above all else.

Sila Dev Log: Defining Macro for Parser Rules

Tue, 03 Oct 2023 14:35:19 +0200

So, like I mentioned in a previous post, my hands have been quite full with Baldur’s Gate 3, so I haven’t been able to program too much Sila. But thankfully, while I enjoyed the game through and through, it’s nice to be back to hacking.

I started to add more parser rules for Sila, simple ones still, but crucial nonetheless. These included stuff like parsing equality (==, !=), relational (>, <, <=, >=) and unary nodes (-1, +2). While writing these rules, I quickly realized that I’m repeating myself quite a bit. So as a Lisp hacker, naturally I decided to reach for macros in this case to make my own life just a little bit easier.

If we look at the structure on how I decided to parse equality and relational nodes, they looked something like this:

(defun parse-equality-node (tok)
  "equality-node ::== relational-node ( '==' relational-node
                                      | '!=' relational-node ) *"
  (multiple-value-bind (node rest)
      (parse-relational-node tok)
    (loop
      (cond ((string= (token-val rest) "==")
             (multiple-value-bind (node2 rest2)
                 (parse-relational-node (token-next rest))
               (setf node (make-ast-node :kind :equal :lhs node :rhs node2))
               (setf rest rest2)))
            ((string= (token-val rest) "!=")
             (multiple-value-bind (node2 rest2)
                 (parse-relational-node (token-next rest))
               (setf node (make-ast-node :kind :not-equal :lhs node :rhs node2))
               (setf rest rest2)))
            (t
             (return-from parse-equality-node
               (values node rest)))))))

(defun parse-relational-node (tok)
  "relational-node ::== add ( '<'  add
                            | '<=' add
                            | '>'  add
                            | '>=' add ) *"
  (multiple-value-bind (node rest)
      (parse-add-node tok)
    (loop
      (cond ((string= (token-val rest) "<")
             (multiple-value-bind (node2 rest2)
                 (parse-add-node (token-next rest))
               (setf node (make-ast-node :kind :lesser-than :lhs node :rhs node2))
               (setf rest rest2)))
            ((string= (token-val rest) "<=")
             (multiple-value-bind (node2 rest2)
                 (parse-add-node (token-next rest))
               (setf node (make-ast-node :kind :lesser-or-equal :lhs node :rhs node2))
               (setf rest rest2)))
            ((string= (token-val rest) ">")
             (multiple-value-bind (node2 rest2)
                 (parse-add-node (token-next rest))
               (setf node (make-ast-node :kind :greater-than :lhs node :rhs node2))
               (setf rest rest2)))
            ((string= (token-val rest) ">=")
             (multiple-value-bind (node2 rest2)
                 (parse-add-node (token-next rest))
               (setf node (make-ast-node :kind :greater-or-equal :lhs node :rhs node2))
               (setf rest rest2)))
            (t
             (return-from parse-relational-node
               (values node rest)))))))

So the structure between these are pretty much identical. First, I bind the values that I get from the next parser rule, e.g. parse-relational-node or parse-add-node, and I run infinite loop and check the next tokens and create nodes based on that.

Macro Definition

Function definition can be broken down to a following macro:

(defmacro define-parser (name &key descent-parser
                                   comparison-symbols
                                   bnf)
  "Macro for generating new parser rules."
  (let ((parser-name (intern (format nil "PARSE-~a-NODE" name)))
        (descent-parser-name (intern (format nil "PARSE-~a-NODE" descent-parser))))
    `(defun ,parser-name (tok)
       ,bnf
       (multiple-value-bind (node rest)
           (,descent-parser-name tok)
         (loop
           (cond
             ,@(loop :for symbol in comparison-symbols
                     :collect `((string= (token-val rest) ,(car symbol))
                                (multiple-value-bind (node2 rest2)
                                    (,descent-parser-name (token-next rest))
                                  (setf node (make-ast-node :kind ,(cdr symbol)
                                                            :lhs node
                                                            :rhs node2))
                                  (setf rest rest2))))
             (t
              (return-from ,parser-name
                (values node rest)))))))))

So what is happening here:

First I define new symbols to the package that I will use inside the macro. This is done with the intern function.
When defining macros in Lisp, you often see code that is inside a backquote (`), this signals that every expression inside that is not preceded by a comma is to be quoted. So above you can see some places where there is comma in front of some expressions, those will be evaluated when the macro is run.
- For example, if parser-name equals to parse-example-node then `(defun ,parser-name ()) would evaluate to (defun parse-example-node ()).
Last crucial piece in the macro is the way I build the conditional for the parsing itself.
- Essentially how I do this is that I build a list of backquoted expressions like:
```
((string= (token-val rest) "<")
  (multiple-value-bind (node2 rest2)
      (parse-add-node (token-next rest))
    (setf node (make-ast-node :kind :lesser-than :lhs node :rhs node2))
    (setf rest rest2)))
```
  Based on all the comparison symbols are given in to macro.
- The collected list is inside ,@ which basically means that evaluate the following expression (,) and splat the containing list (@). So if some-list equals to (1 2 3), then `(fn ,@some-list) would equal to (fn 1 2 3).

Now when the macro is defined, I can just define the parser rules in a following manner:

(define-parser equality
  :descent-parser relational
  :comparison-symbols (("==" . :equal)
                       ("!=" . :not-equal))
  :bnf "equality-node ::== relational-node ( '==' relational-node | '!=' relational-node ) *")

(define-parser relational
  :descent-parser add
  :comparison-symbols (("<" . :lesser-than)
                       ("<=" . :lesser-or-equal)
                       (">" . :greater-than)
                       (">=" . :greater-or-equal))
  :bnf "relational-node ::== add ( '<'  add | '<=' add | '>'  add | '>=' add ) *")

(define-parser add
  :descent-parser multiplicative
  :comparison-symbols (("+" . :add)
                       ("-" . :sub))
  :bnf "add-node ::== multiplicative-node ( '+' multiplicative-node | '-' multiplicative-node ) *")

(define-parser multiplicative
  :descent-parser unary
  :comparison-symbols (("*" . :mul)
                       ("/" . :div))
  :bnf "multiplicative-node ::== unary-node ( '*' unary-node | '/' unary-node ) *")

To see what those macros expand to you can just run macroexpand on them, for example:

(define-parser relational
  :descent-parser add
  :comparison-symbols (("<" . :lesser-than)
                       ("<=" . :lesser-or-equal)
                       (">" . :greater-than)
                       (">=" . :greater-or-equal))
  :bnf "relational-node ::== add ( '<'  add | '<=' add | '>'  add | '>=' add ) *")

Expands to:

(defun parse-relational-node (tok)
  "relational-node ::== add ( '<'  add | '<=' add | '>'  add | '>=' add ) *"
  (multiple-value-bind (node rest)
      (parse-add-node tok)
    (loop
     (cond
      ((string= (token-val rest) "<")
       (multiple-value-bind (node2 rest2)
           (parse-add-node (token-next rest))
         (setf node (make-ast-node :kind :lesser-than :lhs node :rhs node2))
         (setf rest rest2)))
      ((string= (token-val rest) "<=")
       (multiple-value-bind (node2 rest2)
           (parse-add-node (token-next rest))
         (setf node
                 (make-ast-node :kind :lesser-or-equal :lhs node :rhs node2))
         (setf rest rest2)))
      ((string= (token-val rest) ">")
       (multiple-value-bind (node2 rest2)
           (parse-add-node (token-next rest))
         (setf node (make-ast-node :kind :greater-than :lhs node :rhs node2))
         (setf rest rest2)))
      ((string= (token-val rest) ">=")
       (multiple-value-bind (node2 rest2)
           (parse-add-node (token-next rest))
         (setf node
                 (make-ast-node :kind :greater-or-equal :lhs node :rhs node2))
         (setf rest rest2)))
      (t (return-from parse-relational-node (values node rest)))))))

Cool, seems to be identical to the earlier definition that I had. So now when I need to add new parser rules, I can just utilize this macro to do them, saving me of writing unnecessary boilerplate. I probably am not able to use this macro for all the definitions. For example currently the topmost parser rule is defined in a following manner:

(defun parse-expression-node (tok)
  "expression-node ::== equality"
  (parse-equality-node tok))

So it doesn’t really make sense to use that macro for defining something like that. Similarly, unary and primary nodes are defined in a slightly different manner currently:

(defun parse-unary-node (tok)
  "unary-node ::== ( '+' | '-' ) unary | primary-node"
  (cond ((string= (token-val tok) "+")
         (parse-unary-node (token-next tok)))
        ((string= (token-val tok) "-")
         (multiple-value-bind (node rest)
             (parse-unary-node (token-next tok))
           (values (make-ast-node :kind :neg :lhs node)
                   rest)))
        (t
         (parse-primary-node tok))))

(defun parse-primary-node (tok)
  "primary-node ::== '(' expression-node ')' | number"
  (cond ((eq (token-kind tok) :num)
         (values (make-ast-node :kind :number :val (token-val tok))
                 (token-next tok)))
        ((string= (token-val tok) "(")
         (multiple-value-bind (node rest)
             (parse-expression-node (token-next tok))
           (values node (token-next (skip-to-token ")" rest)))))
        (t (error 'parser-error))))

Which I could make it so that the macro above would define these kind of parser rules if e.g. some special key is given in, but for now, I’m completely fine by defining these by hand.

What I Read in September 2023

Sun, 01 Oct 2023 14:16:55 +0200

During September I seemed to spend quite a bit of time by reading books about addiction due to personal reasons.

Judtih Grisel: Never Enough

Grisel’s book focused mainly on how different substances work and how they cause addictions offering wonderful knowledge about the drug use and the development of human brain. Great book!

Gabor Maté: In the Realm of Hungry Ghost

Maté’s book approached addiction from the point of view of trying to answer why people become addicts. One of the most common factors between hard addicts seems to be, according to Maté, some form of trauma that causes them to seek chemical satisfaction from various substances. Book also raises a great point that addiction is really a spectrum. Everyone of us lies in some place in this spectrum.

Dr. Tom O. Bryan: You Can Fix Your Brain

While fixing somebody’s brain is definitely a hard task, it is not impossible. Bryan raises a point in this book that there is no silver-bullet for fixing your brain, but it is possible with small wins in multiple small areas. He calls these four faces of brain health pyramid, which are structure, mindset, biochemistry and electromagnetism, with what you can design a protocol for yourself.

Herman Hesse: Siddhartha (reread, but this time in German)

I’ve been tremendously interested in Buddhism ever since I was a teenager and I would consider myself being a practicing Buddhist. I have decided to start taking this practice more and more serious to try to fix somethings in my life. Siddhartha Gautama’s story is obviously a crucial part of the whole thing and Hesse’s book is a great novel for painting a picture of this. I’ve read this book earlier but in Finnish and English. Since last year I moved to Germany, I’ve been practicing my German and, at the same time, I’ve never been a huge fan of translations in books since I always feel that something always gets lost during the translation. So to improve my German I decided to read this book in its original language, German! While I’m very familiar with the story, from reading this book earlier but also from stuff like Pali canon, it’s still one of my favorite books!

So... Baldur's Gate 3 Happened

Sun, 24 Sep 2023 21:31:38 +0200

Soo… Baldur’s Gate 3 was finally released after many years in open beta, and I have to be honest, it really made me forget pretty much any other project I had going on and just focus on playing it. Yesterday, I was finally able to finish the game after over 100 hours, and all I can say is that it truly was a great experience!

At one point in time, I played quite a bit of video games, but then I lost interest in playing them. Well, losing interest is maybe the wrong word, since I was still interested in what was happening in the gaming world, but I just followed it from afar without spending countless hours in those games.

Baldur’s Gate 3 really sparked this old fire again since it had many things going on in it. I loved the old Baldur’s Gates, and at the same time, many Dungeons and Dragons campaigns with friends hold a dear place in my heart, and naturally, Baldur’s Gate 3 combined those two in a truly remarkable fashion.

So I just wanted to write this small appreciation post to Larian Studios. Yes, the game had some minor bugs here and there, but nothing game-breaking (at least in my experience), but it was an absolutely amazing sequel to one of my favorite gaming series while respecting the legacy of D&D. I doubt that I will start consuming other video games endlessly like I used to, but nonetheless, Baldur’s Gate 3 was an amazing experience, and I’m glad I spent many, many hours in it.

Now, when the game is finished, I can finally return to hacking.

What I Read During the Summer (May-Aug) 2023

Tue, 05 Sep 2023 14:28:50 +0200

Summer months went by fast when you had lot on your hands. Didn’t feel like keeping the reading log up to date during these months, so I’ll just do one big post-summer update here.

Aldous Huxley: Brave New World (reread)

On May, I seemed to be rereading bunch of classics especially with the common theme of dystopian future. Why you might ask? That I don’t know, maybe something inside me just thinks that future depicted in these classics is inevitable. Brave New World is probably one of my favorite books ever written. Wonderful vision of advanced utopia, but at what cost? Masterful critique of the dehumanizing effects of a highly controlled and pleasure-driven society challenges readers to reflect on the consequences of sacrificing personal freedom for comfort and conformity.

George Orwell: 1984 (reread)

Continuing on the series of dystopia and nightmarish tales of possible future. Tale about the dangers of totalitarianism and government surveillance. Serves as a stark reminder of the importance of safeguarding individual freedoms, critical thinking, and truth itself.

William Golding: Lord of the Flies (reread)

More classics! Compelling exploration of human nature and the thin veneer of civilization that separates order from chaos.

J.D. Sallinger: The Catcher in the Rye

The Catcher in the Rye was, one of many, classics that I hadn’t read before and to be honest, wasn’t a huge fan of it. Maybe since I read this as a little bit older. Don’t know. Topics in this book was really interesting, considering teenager alienation, identity etc. While Holden’s struggles in his adolescent were definitely unique, I just felt more annoyed about him that anything else.

Jacques Ellul: The Technological Society

Jacques Ellul: Propaganda: The Formation of Men’s Attitudes

Ted Kaczynski: Industrial Society and Its Future

Ted Kaczynski: Anti-Tech Revolution: Why and How

Every once in a while my inner luddite wakes up and I start hating everything about technology. Reading Jacques Ellul was definitely part of this, but this time it was mainly the death of Unabomber, Ted Kaczynski, that brought me to him. While I don’t agree/support on what Ted Kaczynski did, he had some great points about how technology affects us. Ellul was many ways inspiration for him in these sort of anti-technology philosophies and it affection on humans. But Kaczynski and Ellul had one big difference between them, Ellul was a pacifist were as Kaczynski was a domestic terrorist.

Nonetheless, all of these books make some great points about technology so despite your opinion about it, these are definitely worth a read.

Cormac McCarthy: The Road

Cormac McCarthy: No Country for Old Men

During the summer unfortunately we had a couple of passings of great authors, one of which were Cormac McCarthy. I had never read his books before, but I was familiar with as a movie from (No Country for Old Men by Coen Brothers) and I had heard great things about his writing. So I grabbed a copy of The Road and No Country for Old Men, since those were quite highly recommended. And the recommendations were definitely true.

The Road offers a haunting depiction of post-apocalyptic America and the struggles of father and son in this world. Immersing in bleak landscape where hope and love endure against all odds. No Country for Old Men delves into the realms of crime, fate, and the inexorable consequences of one’s choices.

Both of these works showcase McCarthy’s masterful storytelling, unique narrative styles, and philosophical depth, making them essential reads for those interested in literature that explores the human condition in its most challenging and thought-provoking forms.

Richard P. Gabriel: Patterns of Software

Richard P. Gabriel is a pretty known name in the software world and especially in the Lisp world. One of the famous writings of Richard P. Gabriel is the concept of “Worse Is Better”.

Patterns of Software gives a great look into software design, programming and business around it. Definitely a must read for everyone working in this industry.

Sila Dev Log: Initial Recursive Descent Parsing

Thu, 24 Aug 2023 18:00:36 +0200

For the initial implementation of Sila’s parses, I decided to go with the recursive descent route due to its simplicity. So what it is? Recursive descent parsing is a top-down parsing technique employed to analyze the syntax of structured input, such as programming languages or mathematical expressions. It operates by breaking down the input into smaller, manageable components using a set of recursive functions that correspond to the non-terminal symbols in the grammar.

Beginning with a starting symbol, typically representing the highest-level construct, these functions are invoked recursively to recognize and process the various syntactic elements of the input. Each function handles a specific grammar rule, and by calling each other based on the grammar’s structure, the parser traverses the input while building a parse tree that reflects the hierarchical organization of the language’s syntax.

This approach allows for clear code organization and error reporting, though it can face challenges with certain types of grammars and may require augmentation with techniques like lookahead or memoization for optimal efficiency and effectiveness. But for early stages of the project, I don’t really mind that.

Parser Implementation

First I create some structures and helpers that I will be using in the parsing process.

(deftype ast-node-kind ()
  "Sila AST node kind."
  '(member
    :add
    :sub
    :mul
    :div
    :number))

(defstruct ast-node
  "Structure for Sila AST nodes."
  kind
  val
  lhs
  rhs)

(defun skip-to-token (val tok)
  "Search for token which is `val'. Returns `nil' when not found.

TODO(topi): Probably should do some proper error handling if `val' isn't
found."
  (cond ((eq (token-kind tok) :eof) nil)
        ((equal (token-val tok) val) tok)
        (t (search-token val (token-next tok)))))

For now, I’m interested in parsing something simple like 2 / (1 + 1) * 8, so simple multiplicative calculations and expressions inside parenthesis. So if I would write some metanotation like BNF for that, it would look something like this:

expr ::== mul ( '+' mul | '-' mul ) *
mul ::== primary ( '*' primary | '/' primary ) *
primary ::== '(' expr ')' | number

`expr`

This function is responsible for parsing arithmetic expressions. It follows the grammar rule: expr ::= mul ( '+' mul | '-' mul ) *. It starts by calling the mul function to parse the initial term. Then, it enters a loop to handle addition and subtraction operators. If it encounters a +, it parses the next term using mul and creates an AST node representing addition. If it encounters a -, it does the same for subtraction.

The loop continues until it encounters an operator that’s not + or -, at which point it returns the parsed expression node and the remaining token stream.

(defun expr (tok)
  "expr ::== mul ( '+' mul | '-' mul ) *"
  (multiple-value-bind (node rest)
      (mul tok)
    (loop
      (cond ((eq (token-val rest) #\+)
             (multiple-value-bind (node2 rest2)
                 (mul (token-next rest))
               (setf node (make-ast-node :kind :add :lhs node :rhs node2))
               (setf rest rest2)))
            ((eq (token-val rest) #\-)
             (multiple-value-bind (node2 rest2)
                 (mul (token-next rest))
               (setf node (make-ast-node :kind :sub :lhs node :rhs node2))
               (setf rest rest2)))
            (t
             (return-from expr
               (values node rest)))))))

`mul`

Parsing mul expressions works in a similar fashion:

(defun mul (tok)
  "mul ::== primary ( '*' primary | '/' primary ) *"
  (multiple-value-bind (node rest)
      (primary tok)
    (loop
      (cond ((eq (token-val rest) #\*)
             (multiple-value-bind (node2 rest2)
                 (primary (token-next rest))
               (setf node (make-ast-node :kind :mul :lhs node :rhs node2))
               (setf rest rest2)))
            ((eq (token-val rest) #\/)
             (multiple-value-bind (node2 rest2)
                 (primary (token-next rest))
               (setf node (make-ast-node :kind :div :lhs node :rhs node2))
               (setf rest rest2)))
            (t
             (return-from mul
               (values node rest)))))))

`primary`

This function parses primary expressions, which can be either numeric values or sub-expressions enclosed in parentheses. It follows the grammar rule: primary ::= '(' expr ')' | number. If the current token is a number, it creates an AST node representing a numeric value. If the token is an opening parenthesis, it recursively calls the expr function to parse the enclosed expression and skips to the closing parenthesis. It then returns the parsed node and the remaining token stream.

(defun primary (tok)
  "primary ::== '(' expr ')' | number"
  (cond ((eq (token-kind tok) :num)
         (values (make-ast-node :kind :number :val (token-val tok))
                 (token-next tok)))
        ((eq (token-val tok) #\()
         (multiple-value-bind (node rest)
             (expr (token-next tok))
           (values node (token-next (skip-to-token #\) rest)))))
        (t (error 'parser-error))))

Testing the Initial Parser

For now, I will be testing this parser only from the REPL. Firing up it, and running the command should print something like this:

* (expr (tokenize "2 / (1 + 1) * 8"))
#S(AST-NODE
   :KIND :MUL
   :VAL NIL
   :LHS #S(AST-NODE
           :KIND :DIV
           :VAL NIL
           :LHS #S(AST-NODE :KIND :NUMBER :VAL 2 :LHS NIL :RHS NIL)
           :RHS #S(AST-NODE
                   :KIND :ADD
                   :VAL NIL
                   :LHS #S(AST-NODE :KIND :NUMBER :VAL 1 :LHS NIL :RHS NIL)
                   :RHS #S(AST-NODE :KIND :NUMBER :VAL 1 :LHS NIL :RHS NIL)))
   :RHS #S(AST-NODE :KIND :NUMBER :VAL 8 :LHS NIL :RHS NIL))
#S(SILA/LEXER::TOKEN :KIND :EOF :POS 15 :LEN NIL :VAL NIL :NEXT NIL)
*

Here we can see how our parser parses the expressions inside it. Transforming it to ASCII would look something like this:

     Mul(*)
    /     \
 Div(/)    8
 /   \
2   Add(+)
     /  \
    1    1

Code Generation

Assembly code of something like 2 / (1 + 1) * 8 is still quite simple, but definitely more instruction will be needed compared to the simple addition and subtraction. Assembly code for it would equal to:

	.globl main
main:
	mov $8, %rax
	push %rax
	mov $1, %rax
	push %rax
	mov $1, %rax
	pop %rdi
	add %rdi, %rax
	push %rax
	mov $2, %rax
	pop %rdi
	cqo
	idiv %rdi, %rax
	pop %rdi
	imul %rdi, %rax
	ret

So what is happening here:

Initializing and Pushing Values:
- mov $8, %rax: This instruction moves the immediate value 8 into the %rax register.
- push %rax: The value in %rax (which is now 8) is pushed onto the stack.
Further Operations:
- mov $1, %rax: Another immediate value, 1, is moved into the %rax register.
- push %rax: The value in %rax (now 1) is pushed onto the stack.
- mov $1, %rax: Yet another immediate value, 1, is moved into %rax.
- pop %rdi: The top value from the stack (1) is popped into the %rdi register.
- add %rdi, %rax: The value in %rdi (previously 1) is added to the value in %rax (also 1), and the result is stored in %rax.
More Operations:
- push %rax: The value in %rax (now 2) is pushed onto the stack.
- mov $2, %rax: The immediate value 2 is moved into %rax.
- pop %rdi: The top value from the stack (2) is popped into %rdi.
Division and Multiplication:
- cqo: This instruction extends the sign of %rax into %rdx in preparation for the division operation.
- idiv %rdi, %rax: The value in %rax (now 2) is divided by the value in %rdi (previously 1). The quotient is stored in %rax and the remainder in %rdx.
- pop %rdi: The value 2 is popped from the stack and stored in %rdi.
- imul %rdi, %rax: The value in %rax (the quotient of the division) is multiplied by the value in %rdi (previously 2), and the result is stored in %rax.

Implementation

For the code generation aspects, we can start by creating some unit tests since we already know what would be the output of our function:

(deftest test-emit-asm-div-mul-parens
  (let ((emit-asm-tests '(("2 / (1 + 1) * 8" . "  .globl main
main:
  mov $8, %rax
  push %rax
  mov $1, %rax
  push %rax
  mov $1, %rax
  pop %rdi
  add %rdi, %rax
  push %rax
  mov $2, %rax
  pop %rdi
  cqo
  idiv %rdi, %rax
  pop %rdi
  imul %rdi, %rax
  ret
"))))
    (dolist (test emit-asm-tests)
      (ok (string-equal (sila::emit-asm (car test)) (cdr test))
          (format nil "Expect to be equal: ~%~a~%=>~a"
                  `(sila::emit-asm ,(car test))
                  (cdr test))))))

The implementation aspects also will be relatively straight forward.

(defun asm-inst (inst)
  (format nil "  ~a~%" inst))

(defun asm-directive (dir)
  (asm-inst dir))

(defun asm-label (label)
  (format nil "~a:~%" label))

(defun asm-push ()
  (asm-inst "push %rax"))

(defun asm-pop (reg)
  (asm-inst (format nil "pop %~a" reg)))

(defun generate-expr (node)
  (let ((asm ""))
    (flet ((asm-conc (inst)
             (setf asm (concatenate 'string asm inst))))
      (when (eq (ast-node-kind node) :number)
        (asm-conc (asm-inst (format nil "mov $~d, %rax"
                                    (ast-node-val node))))
        (return-from generate-expr asm))
      (asm-conc (generate-expr (ast-node-rhs node)))
      (asm-conc (asm-push))
      (asm-conc (generate-expr (ast-node-lhs node)))
      (asm-conc (asm-pop "rdi"))
      (alexandria:switch ((ast-node-kind node) :test #'eq)
        (:add (asm-conc (asm-inst "add %rdi, %rax")))
        (:sub (asm-conc (asm-inst "sub %rdi, %rax")))
        (:mul (asm-conc (asm-inst "imul %rdi, %rax")))
        (:div (asm-conc (asm-inst "cqo"))
              (asm-conc (asm-inst "idiv %rdi, %rax")))
        (t (error 'parser-error))))
    asm))

This function recursively generates assembly code from an abstract syntax tree node representing an arithmetic expression. It constructs assembly instructions based on the kind of node encountered. It concatenates these instructions into a string that represents the generated assembly code.

For a number node, it generates a move instruction (mov) to load the value into %rax.
For binary operations (addition, subtraction, multiplication, division), it generates assembly for the right-hand side expression, pushes %rax onto the stack, generates assembly for the left-hand side expression, pops the value into %rdi, and then performs the corresponding operation.

Then lastly we just define the emit function, like so:

(defun emit-asm (src)
  "Emit assembly code from given source code. Currently emits only x86-64 and
only Linux is tested."
  (let ((asm ""))
    (flet ((asm-conc (inst)
             (setf asm (concatenate 'string asm inst))))
      (multiple-value-bind (node rest)
          (expr (tokenize src))
        (unless (eq (token-kind rest) :eof)
          (error 'parser-error :error-msg "Extra tokens"))
        (asm-conc (asm-directive ".globl main"))
        (asm-conc (asm-label "main"))
        (asm-conc (generate-expr node))
        (asm-conc (asm-inst "ret"))))
    asm))

Again, we can test it directly in the REPL first:

* (emit-asm "2 / (1 + 1) * 8")
"  .globl main
main:
  mov $8, %rax
  push %rax
  mov $1, %rax
  push %rax
  mov $1, %rax
  pop %rdi
  add %rdi, %rax
  push %rax
  mov $2, %rax
  pop %rdi
  cqo
  idiv %rdi, %rax
  pop %rdi
  imul %rdi, %rax
  ret
"

Seems to work. Now to run the tests:

# make test
rove sila.asd

Testing System sila/tests

;; testing 'sila/tests'
test-compilation-and-compare-rc
  ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "0" 0) to be true. (2726ms)
  ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "5 + 40 - 20" 25) to be true. (2693ms)
  ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "2 / (1 + 1) * 8" 8) to be true. (2703ms)
test-emit-asm-integer
  ✓ Expect to be equal:
    (EMIT-ASM 0)
    =>  .globl main
    main:
      mov $0, %rax
      ret

  ✓ Expect to be equal:
    (EMIT-ASM 42)
    =>  .globl main
    main:
      mov $42, %rax
      ret

test-emit-asm-add-sub
  ✓ Expect to be equal:
    (EMIT-ASM 5+20-4)
    =>  .globl main
    main:
      mov $4, %rax
      push %rax
      mov $20, %rax
      push %rax
      mov $5, %rax
      pop %rdi
      add %rdi, %rax
      pop %rdi
      sub %rdi, %rax
      ret

  ✓ Expect to be equal:
    (EMIT-ASM     5   +  20  -  4   )
    =>  .globl main
    main:
      mov $4, %rax
      push %rax
      mov $20, %rax
      push %rax
      mov $5, %rax
      pop %rdi
      add %rdi, %rax
      pop %rdi
      sub %rdi, %rax
      ret

test-emit-asm-div-mul-parens
  ✓ Expect to be equal:
    (EMIT-ASM 2 / (1 + 1) * 8)
    =>  .globl main
    main:
      mov $8, %rax
      push %rax
      mov $1, %rax
      push %rax
      mov $1, %rax
      pop %rdi
      add %rdi, %rax
      push %rax
      mov $2, %rax
      pop %rdi
      cqo
      idiv %rdi, %rax
      pop %rdi
      imul %rdi, %rax
      ret


✓ 1 test completed

Summary:
  All 1 test passed.

Cool, seems to work!

Sila Dev Log: Tokenization and Compiling Basic Arithmetic

Sat, 19 Aug 2023 21:03:04 +0200

A while ago, I wrote about why would anyone start writing a new programming language. Consequently, I decided to gradually initiate the project. Simultaneously, I found the idea of documenting my progress to be intriguing. I opted to adopt a strategy of incremental development, wherein I would construct small, functional components and refine them over time.

This incremental methodology draws inspiration from the paper by Abdulaziz Ghuloum. Other tremendously useful resources include works such as tcc, A small C compiler written by Fabrice Bellard, lcc: Another small C compiler. The creators wrote a book about the internals of lcc, which I found a good resource to see how a compiler is implemented, and also chibicc by Rui Ueyama.

Before embarking on this endeavor, I want to underscore that I currently harbor modest ambitions for this project. Consequently, my approach might be somewhat sporadic, aligning with my personal inclination and availability. Primarily, I intend this project to serve educational and research purposes, surpassing other objectives.

Getting Started With Sila Compiler

I decided to call this project Sila, which comes from Pali and is often used in Buddhist texts meaning:

śīla (Pali: sīla; T. tshul khrims ཚུལ་ཁྲིམས་), literally, ‘acting appropriately’. Sila is translated as “skillful conduct,” “discipline,” “ethical conduct,” “moral conduct,” “virtue,” etc.

Sila is said to be a way of being that is conducive to positive and happy states of mind.

With the Buddhist teachings, sila is identified as:

one of the three trainings

one of the six paramitas in the Sanskrit tradition

one of the ten paramis in the Pali tradition

From https://encyclopediaofbuddhism.org/wiki/Śīla

To initiate the process, my initial step was to attempt the compilation of the most elementary language conceivable. Essentially, my goal was to input an integer into the compiler, which would then produce corresponding assembly code. This assembly code, when compiled, would result in an exit code identical to the provided integer. On a x86-64 Linux platform, the assembly program to achieve this might resemble the following:

	.globl main
main:
	mov $0, %rax
	ret

So what is happening here?

.globl main: This directive tells the assembler that the label main should be considered a global symbol. This means that the symbol main can be accessed from other parts of the program, possibly in other source files or modules.
main:: This is a label that marks the beginning of the main function or code block. Labels are used as targets for jumps and branches in assembly code.
mov $0, %rax: This instruction moves the immediate value 0 into the %rax register. In the x86-64 architecture, %rax is a general-purpose register that is often used to hold return values of functions.
ret: This instruction is used to return from a function. In this case, since we’re looking at the main function, it’s effectively ending the program. The value in %rax would typically be used as the exit status of the program.

Simple arithmetic, like “5 + 20 - 4”, is also easily represented x86 assembly:

	.globl main
main:
	mov $5, %rax
	add $20, %rax
	sub $4, %rax
	ret

mov $5, %rax: This instruction moves the immediate value 5 into the %rax register.
add $20, %rax: This instruction adds the immediate value 20 to the current value stored in the %rax register. After this operation, %rax would hold the value 25 (5 + 20).
sub $4, %rax: This instruction subtracts the immediate value 4 from the current value in the %rax register. After this operation, %rax would hold the value 21 (25 - 4).

Why Common Lisp?

The internet is already saturated with fervent support for Lisp, leaving me with a sense that there is little additional I can contribute on the matter. My decision to opt for Common Lisp stems from pragmatic considerations: it is a language in which I am effective, and, above all, it aligns with my personal enjoyment of programming.

Testing Infrastructure

Considering my primary focus on Linux and x86-64, I must undertake certain preparations to facilitate program testing. While I possess an x86-64 machine, it operates on macOS, rendering the same assembly directives ineffective as they are in Linux. Fortunately, this predicament is effortlessly resolved through the utilization of Docker.

My container requirements are rather minimal. Given my work with Common Lisp, the inclusion of a CL compiler is paramount. I’ve chosen SBCL for this purpose, primarily due to my familiarity with it. While other implementations might suffice, my present focus won’t extend to testing those alternatives. Alongside the CL compiler, an assembly compiler for the generated code is essential. In this context, GCC serves as an apt choice.

So the Dockerfile for development could look something like this:

FROM fukamachi/sbcl:2.3.7-debian

LABEL maintainer="Topi Kettunen <topi@topikettunen.com>"

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update -qq \
    && apt-get install -qq -y --no-install-recommends \
         build-essentials \
    && rm -rf /var/lib/apt/lists/*

# For unit testing
RUN sbcl --eval '(ql:quickload :rove)'

WORKDIR /root/.roswell/local-projects/sila

ENTRYPOINT ["/bin/bash"]

I decided to use Eitaro Fukamachi’s dockerfiles. These Dockerfiles are built around roswell, which I’m also using locally.

I also write some simple Makefile to just make my life easier:

LISP ?= ros -Q run

.PHONY: build
build:
	$(LISP) --load sila.asd \
		--eval '(ql:quickload :sila)' \
		--eval '(asdf:make :sila)' \
		--eval '(quit)'

.PHONY: test
test:
	$(LISP) --load sila.asd \
		--eval '(ql:quickload :sila)' \
		--eval '(asdf:test-system :sila)' \
		--eval '(quit)'

.PHONY: docker-build
docker-build:
	docker build -t sila .

.PHONY: docker-run
docker-run:
	docker run -it --rm -v ${PWD}:/root/.roswell/local-projects/sila sila

Initial Tests

Since I already know what I want to output from my compiler at this point, it’s easy to start development by writing some nifty little unit tests for it. For now, I decided to write tests for emitting the assembly code and than separately the compilation process.

(deftest test-emit-asm-integer
  (let ((emit-asm-tests '(("0" . "  .globl main
main:
  mov $0, %rax
  ret
")
                          ("42" . "  .globl main
main:
  mov $42, %rax
  ret
"))))
    (dolist (test emit-asm-tests)
      (ok (string-equal (sila::emit-asm (car test)) (cdr test))
          (format nil "Expect to be equal: ~%~A~%=>~A"
                       `(sila::emit-asm ,(car test))
                       (cdr test))))))

(deftest test-emit-asm-add-sub
  (let ((emit-asm-tests '(("5+20-4" . "  .globl main
main:
  mov $5, %rax
  add $20, %rax
  sub $4, %rax
  ret
")
                          ("    5   +  20  -  4   " . "  .globl main
main:
  mov $5, %rax
  add $20, %rax
  sub $4, %rax
  ret
"))))
    (dolist (test emit-asm-tests)
      (ok (string-equal (sila::emit-asm (car test)) (cdr test))
          (format nil "Expect to be equal: ~%~A~%=>~A"
                       `(sila::emit-asm ,(car test))
                       (cdr test))))))

These tests are relatively straight forward, I generate the output and just compare strings to each other to see that they both are the same. Compilation itself is little bit more involved. Most likely you would achieve easier test setup by just simple shell scripts, but since I want to write Common Lisp, I decided to write some small testing harness for it.

(defvar *bin-name* "sila")
(defvar *tmp-name* "tmp.s")
(defvar *tmp-bin-name* "tmp")

(defun sila-project-root (path)
  (namestring (asdf:system-relative-pathname :sila path)))

(defun compile-program-and-compare-rc (input expected-rc)
  (multiple-value-bind (out err rc)
      (uiop:run-program (list (sila-project-root *bin-name*)
                              input)
                        :output :string
                        :error-output :string)
    (declare (ignore err rc))
    (with-open-file (fstream (sila-project-root *tmp-name*)
                             :direction :output
                             :if-exists :supersede
                             :if-does-not-exist :create)
      (format fstream out))
    (uiop:run-program (list "gcc" "-static" "-o" *tmp-bin-name* *tmp-name*))
    (multiple-value-bind (out err got-rc)
        (uiop:run-program (sila-project-root *tmp-bin-name*)
                          ;; Since we're working with arbitrary return codes.
                          :ignore-error-status t)
      (declare (ignore out err))
      (= expected-rc got-rc))))

(deftest test-compilation-and-compare-rc
  (ok (compile-program-and-compare-rc "0" 0))
  (ok (compile-program-and-compare-rc "5 + 40 - 20" 25)))

What I’m doing here is that I use the freshly created executable that outputs assembly and pipe the output to a file called tmp.s. After that I run gcc for that generated assembly and compile it to a binary called tmp. Then I just run the program and check what return code it returns and compare those to what I expect them to be.

Obviously the tests will fail for now, but those give me a nice foundation on top of which is nice to start developing.

Tokenization

Before I do anything for my compiler, I need to do some lexical analysis and generate some tokens from given input. I start this by defining some structures and types for holding these tokens:

(deftype kind ()
  '(member
    :punct
    :num
    :eof))

(defstruct token
  kind
  pos
  len
  val
  next)

Since I’m currently only working with strings given as input, I will do the tokenization simply moving a pointer through and creating tokens based on the subsequences I encounter in the given input string. Before that, I need some helper functions:

(defun whitespacep (c)
  (member c '(#\Space #\Tab #\Return #\Newline)))

(defun punctuatorp (c)
  ;; TODO(topi): Add more punctuators when needed.
  (member c '(#\+ #\-)))

(defun skip-to-punctuator (input start)
  ;; TODO(topi): Improve this. Won't work when new tokens are added.
  (position-if-not #'digit-char-p input :start start))

I also define a condition for encountering invalid tokens, which I can throw in case something bad happens:

(define-condition invalid-token (error)
  ((token :initarg :token
          :initform nil
          :reader token)
   (token-position :initarg :token-position
                   :initform nil
                   :reader token-position))
  (:report (lambda (condition stream)
             (format stream "Invalid token: '~A' at position ~A.~&"
                     (token condition)
                     (token-position condition))))
  (:documentation "Condition for when we encounter invalid token."))

As you can see in the token structure above, there is a slot called next indicating the next token. So I’m working with linked lists here. I’ll start the tokenization by creating head node and also cur node which I will be populating while I’m iterating through the list. I also initialise a variable for pointing me where I’m currently at in the given input string.

The way I decided to do this initial tokenization is simply looping through the given input with loop. At the moment, I’m not interested in whitespace, so when I encounter whitespace character, I’ll just move the src-pos forward.

Currently, I’m interested in at what position I will encounter a punctuator, either + or -, in the given input string. To do that, I’ll use the skip-to function defined above, which returns the position from the input string where first given item is found at. When I know that, it’s easy the deduce the positions of each token in the given input.

(defun tokenize (src)
  (let* ((head (make-token))
         (cur head)
         (src-pos 0))
    (loop :while (< src-pos (length src))
          :do (let ((punct-pos (skip-to-punctuator src src-pos)))
                (cond (;; Whitespace
                       (whitespacep (char src src-pos))
                       (incf src-pos))
                      (;; Numeric literal
                       (digit-char-p (char src src-pos))
                       (setf (token-next cur)
                             (make-token :kind num
                                         :val (parse-integer src
                                                             :start src-pos
                                                             :junk-allowed t)
                                         :pos src-pos))
                       (setf (token-len (token-next cur))
                             (if punct-pos
                                 (- punct-pos src-pos)
                                 ;; No more punctuators.
                                 (- (length src) src-pos)))
                       (setf cur (token-next cur))
                       (if punct-pos
                           (setf src-pos punct-pos)
                           ;; No more punctuators, move `src-pos` to the end.
                           (setf src-pos (length src))))
                      (;; Punctuator: + | -
                       (punctuatorp (char src src-pos))
                       (setf (token-next cur)
                             (make-token :kind punct
                                         :val (char src src-pos)
                                         :pos src-pos
                                         ;; TODO(topi): Currently I only have
                                         ;; 1 char puncts, change when more is
                                         ;; needed.
                                         :len 1))
                       (setf cur (token-next cur))
                       (incf src-pos))
                      (t
                       (error 'invalid-token
                              :token (char src src-pos)
                              :token-position src-pos)))))
    ;; No more tokens.
    (setf (token-next cur) (make-token :kind eof
                                       :pos src-pos))
    (setf cur (token-next cur))
    (token-next head))

We can then test this out in the REPL, to see what it prints out:

SILA> (sila/lexer:tokenize "1 + 1 - 123")
#S(SILA/LEXER::TOKEN
   :KIND :NUM
   :POS 0
   :LEN 2
   :VAL 1
   :NEXT #S(SILA/LEXER::TOKEN
            :KIND :PUNCT
            :POS 2
            :LEN 1
            :VAL #\+
            :NEXT #S(SILA/LEXER::TOKEN
                     :KIND :NUM
                     :POS 4
                     :LEN 2
                     :VAL 1
                     :NEXT #S(SILA/LEXER::TOKEN
                              :KIND :PUNCT
                              :POS 6
                              :LEN 1
                              :VAL #\-
                              :NEXT #S(SILA/LEXER::TOKEN
                                       :KIND :NUM
                                       :POS 8
                                       :LEN 3
                                       :VAL 123
                                       :NEXT #S(SILA/LEXER::TOKEN
                                                :KIND :EOF
                                                :POS 11
                                                :LEN NIL
                                                :VAL NIL
                                                :NEXT NIL))))))

And it seems to work nicely giving us crucial information such as how long the tokens are in the input and at what position they’re found at, something that will be useful in the future.

Emitting Assembly

Since now the tokenization is working more or less as intended, we can move forward with the assembly generation. For now, the only thing that matters to me is that we can generate some simple assembly to standard output. The way I decided to this, is to just concatenate assembly instructions to one string and just print the result. I’m only working with one label main for now, so I’m not starting to do anything too fancy yet.

The way I’m emitting the assembly is simply by looping all the tokens that I got from tokenize until I encounter token which kind is :eof. Then based on the token, I’ll just concatenate assembly instructions as needed. I’ll also define simple entrypoint to my executable which only accepts one string as input that will generated to assembly.

(defun main ()
  "Sila programming language entrypoint."
  (multiple-value-bind (options free-args)
      (opts:get-opts)
    ;; TODO(topi): Add some useful options.
    (declare (ignore options))
    (unless (= (length free-args) 1)
      (format *error-output* "Invalid number of arguments, expected 1.~%")
      (opts:exit 1))
    (format *standard-output* (emit-asm (first free-args)))
    (opts:exit)))

(defun emit-asm (src)
  "Emit assembly code from given source code. Currently emits only x86-64 and only Linux
is tested."
  (let ((asm "")
        (tok (tokenize src)))
    (labels ((asm-conc (inst)
               (setf asm (concatenate 'string asm inst)))
             (asm-inst (inst)
               (format nil "  ~A~%" inst))
             (asm-directive (dir)
               (asm-inst dir))
             (asm-label (label)
               (format nil "~A:~%" label)))
      (asm-conc (asm-directive ".globl main"))
      (asm-conc (asm-label "main"))
      ;; For now, first token must be a number.
      (asm-conc (asm-inst (format nil "mov $~d, %rax" (token-val tok))))
      (setf tok (token-next tok))
      (loop :until (equal (token-kind tok) :eof)
            :do (cond ((eq (token-kind tok) :num)
                       (setf tok (token-next tok)))
                      ((char= (token-val tok) #\+)
                       (asm-conc (asm-inst
                                  (format nil "add $~d, %rax"
                                          (token-val (token-next tok)))))
                       (setf tok (token-next tok)))
                      ((char= (token-val tok) #\-)
                       (asm-conc (asm-inst
                                  (format nil "sub $~d, %rax"
                                          (token-val (token-next tok)))))
                       (setf tok (token-next tok)))))
      (asm-conc (asm-inst "ret"))
      asm)))

With these we should be able to run our testing suite to see what happens.

$ make test

[...]

Testing System sila/tests

;; testing 'sila/tests/main'
test-compilation-and-compare-rc
  ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "0" 0) to be true. (2763ms)
  ✓ Expect (COMPILE-PROGRAM-AND-COMPARE-RC "5 + 40 - 20" 25) to be true. (2509ms)

;; testing 'sila/tests/emit-asm-x86-64'
test-emit-asm-integer
  ✓ Expect to be equal:
    (EMIT-ASM 0)
    =>	.globl main
    main:
    	mov $0, %rax
    	ret

  ✓ Expect to be equal:
    (EMIT-ASM 42)
    =>	.globl main
    main:
    	mov $42, %rax
    	ret

test-emit-asm-add-sub
  ✓ Expect to be equal:
    (EMIT-ASM 5+20-4)
    =>	.globl main
    main:
    	mov $5, %rax
    	add $20, %rax
    	sub $4, %rax
    	ret

  ✓ Expect to be equal:
    (EMIT-ASM     5   +  20  -  4   )
    =>	.globl main
    main:
    	mov $5, %rax
    	add $20, %rax
    	sub $4, %rax
    	ret


✓ 1 test completed

Summary:
  All 1 test passed.

Cool seems to pass!

Next Steps

I’m continuing the development of my compiler at my own pace, and will write when something new happens. I think I will come back to some parts when I decide to some improvements or refactorings so that everything happens in small iterations. Bugs will be found and hopefully fixed. This might be a long project, but I don’t mind, it’s fun! If you’re interested, feel free to follow my blog to know when something new comes.

Why Would Anyone Build a New Programming Language?

Fri, 28 Jul 2023 20:03:07 +0200

In the vast and ever-evolving landscape of software development, programming languages play a pivotal role in how developers interact with computers and build applications. While a plethora of well-established programming languages exist, there has always been an intriguing curiosity and desire among some developers, including myself, to embark on the quest of creating our own languages.

Existing programming languages come with their own set of strengths and weaknesses. However, they may not always cater optimally to specific domains or problem sets. This has been a driving force for many language creators who seek full control over the language’s semantics. Building a new language offers the opportunity to tailor the language’s features and syntax, empowering developers and teams to solve certain classes of problems more efficiently compared to existing solutions.

However, it is essential to acknowledge that creating a new language is no easy task. Beyond the initial development, considerations for onboarding other developers and promoting adoption must be taken into account. Moreover, when considering building a language within a company setting, one must be mindful of potential monetary implications.

The decision to build a new programming language should not solely be driven by pragmatic concerns; it is also an intellectually stimulating exercise. The process involves delving into various aspects of language design, including syntax, semantics, and the implementation of the compiler or interpreter. Aspiring language creators gain a deeper understanding of computer science principles and enhance their problem-solving skills through this complex undertaking.

Throughout my journey in the world of computer science, I have dabbled in creating multiple toy compilers. Though they may not have been groundbreaking, they ignited my curiosity about compiler implementation. Despite having explored various compilers and interpreters of varying complexity, I felt a lingering sense of not fully comprehending the intricacies occurring under the hood.

Fortuitously, the summer provided a respite from my regular work commitments, granting me the perfect opportunity to embark on a more serious attempt at building a language from scratch. This time, I was determined to delve deep into the intricate details of lexing, parsing, and code generation. This ambitious undertaking offered an exciting challenge, and I was eager to explore uncharted territories in language design.

The journey of building a new programming language is undoubtedly exciting and rewarding, fueled by an array of motivations. Whether it is optimizing performance to meet specific demands, addressing domain-specific challenges, or simply exploring innovative language design paradigms, language creation is a conduit for personal growth and innovation.

As the development of my language progresses, I anticipate gaining a more comprehensive understanding of how programming languages shape our interactions with computers and applications. I hope that my creation, like many others in the world of programming languages, will contribute to the diversity of tools available to developers, enabling them to craft exceptional solutions tailored to their unique needs.

In conclusion, the decision to build a new programming language should be approached with a blend of curiosity, pragmatism, and enthusiasm. It is a journey that fosters personal growth, deepens technical understanding, and holds the potential to revolutionize the way developers solve problems and interact with computers.

An Album for Each Year

Sun, 16 Jul 2023 19:19:11 +0200

I stumbled upon a fun-sounding challenge from Werner Vogels' blog so I wanted to partake in it personally. The challenge was to list your favourite album for every year of your life with a restriction of only one album per year and no repeats of artists.

Here is my list:

1995: Alice in Chains, Alice in Chains
1996: Type O Negative, October Rust
1997: Deftones, Around the Fur
1998: Neutral Milk Hotel, In the Aeroplane Over the Sea
1999: Sleep, Jerusalem (Dopesmoker)
2000: Godspeed You! Black Emperor - Lift Your Skinny Fists Like Antennas to Heaven
2001: B.R.M.C., Black Rebel Motorcycle Club
2002: Queens of the Stone Age, Songs for the Deaf
2003: Songs: Ohia, The Magnolia Electric Co.
2004: MF DOOM, MM...FOOD
2005: Boris, Pink
2006: Tool, 10,000 Days
2007: Porcupine Tree, Fear of a Blank Planet
2008: Have a Nice Life, Deathconsciousness
2009: Them Crooked Vultures, Them Crooked Vultures
2010: Nails, Unsilent Death
2011: Gillian Welch, The Harrow & the Harvest
2012: Death Grips, The Money Store
2013: Jason Isbell, Southeastern
2014: Swans, To Be Kind
2015: Kendrick Lamar, To Pimp a Butterfly
2016: Gojira, Magma
2017: Slowdive, Slowdive
2018: Anna Von Hausswolff, Dead Magic
2019: Justin Townes Earle, The Saint of Lost Causes
2020: Lianne La Havas, Lianne La Havas
2021: Silk Sonic, An Evening with Silk Sonic
2022: Weyes Blood, And in the Darkness, Hearts Aglow
2023: King Gizzard & the Lizard Wizard, PetroDragonic Apocalypse (as of July 2023)

Challenge turned out to be quite hard for me since most of my favourite artists and albums are from time before my birth, but I managed to find something for each year. Especially difficult aspect was leaving great albums out. Rule of no repeats of artists didn’t make it easier either. Most likely I missed some great artists/albums so this list most likely just represents how I currently feel.

Embrace Authenticity in Your Blog

Sat, 10 Jun 2023 23:38:04 +0200

I’ve been a longtime fan of Julie Evans’ writing and recently stumbled upon a great post of her called Some blogging myths which inspired to me write something about the same topic.

I have always seen blogging or personal websites as this powerful platforms for self-expression, enabling people from all walks of life to share their thoughts, ideas, and experiences with the whole wide world. Beyond that, I’ve never really given too much thought to it.

While it’s true that if you happen to be some sort of artist, or even a blogger, that uses his site for some capitalistic reasons, attracting traffic and engagement is important. This can be seen today in social media. This basically means that people tend to approach blogging nowadays from this mindset. Meaning, that they are only able to write about topics that are popular or, especially in tech world, about topics that they consider themselves being an expert in.

Now I don’t say that you shouldn’t write about these sort of topics if you really happen to be expert in the field in question. But, limiting the topics solely to topics that are likely to generate traffic, you most likely inadvertently stifle your own authenticity. Blog should be a place for you to have freedom to explore diverse subjects that just happens to pique your interest. By doing so, you invite your readers into a genuine and vulnerable space where they can connect with you on a deeper level.

This sort of safe haven fosters a certain trust and connection with your audience. When you write about wide range of topics, even those outside your comfort zone, you invite readers who share those interest or experience to engage with you. This way you create an inclusive environment where your audience can feel heard and understood strengthening the bond between you and them.

While first and foremost blog is mainly about your voice but at the same time it’s also about providing a platform for other to share their stories and experiences. Embracing a wide range of topics sparks conversations that can educate, enlighten, and inspire everyone involved.

I don’t say that if your blog is dedicated to a either popular or very niche topic would be a bad thing. If that’s something that you enjoy, godspeed. The way I see it is that if I would limit the topics written in my blog, I would inadvertently hinder my own personal growth as a writer and individual. Exploring new subjects, especially those that you’re by no mean an expert of, allows you to expand your knowledge, learn from others, and develop new skills. Possibly it can also help you to discover hidden passions and uncover something new about yourself.

When you write about anything that sparks your interest, you also demonstrate that it’s okay to write about anything and to step outside the boundaries of popular opinion or expertise, you empower individuals to express themselves more authentically in their own spheres. This sort of vulnerability and explorations can serve as an inspiration and catalyst for others to break free from limitations and embrace their true selves.

Another Godfather of AI Feeling Lost

Thu, 01 Jun 2023 08:41:57 +0200

Not so long ago, Geoffrey Hinton, another “godfather of AI”, raised his concerns about the future of AI and his own feelings towards it.

Now it seems, another “godfather of AI”, is feeling lost with his work

He is the second of the so-called three “godfathers” of AI, known for their pioneering work in the field, to voice concerns about the direction and the speed at which it is developing. […]

Prof Bengio told the BBC he was concerned about “bad actors” getting hold of AI, especially as it became more sophisticated and powerful.

“It might be military, it might be terrorists, it might be somebody very angry, psychotic. And so if it’s easy to program these AI systems to ask them to do something very bad, this could be very dangerous.

“If they’re smarter than us, then it’s hard for us to stop these systems or to prevent damage,” he added. Prof Bengio admitted those concerns were taking a personal toll on him, as his life’s work, which had given him direction and a sense of identity, was no longer clear to him. “It is challenging, emotionally speaking, for people who are inside [the AI sector],” he said. “You could say I feel lost. But you have to keep going and you have to engage, discuss, encourage others to think with you.”

What I Read in April 2023

Sun, 07 May 2023 13:33:09 +0200

Lots of non-fiction for me this month, mainly revolving around attention span. Very interesting topic overall, especially considering the highly technological dopamine filled world we live in today.

Gloria Mark: Attention Span, Adam Alter: Irresistible, Alex Soojung-Kim-Pang: The Distraction Addiction:

I decided to group these together since all three of these books revolved around technology, addiction, distraction and attention span. I’ve read a lot about these subjects so I didn’t necessarily learn too much new, but still, I would say these books were worth a read.

Basically, the key takeaway from these were the fact that modern digital technologies have changed the “structure of our attention”. We use screens more and more using software and products that constantly hinder our attention with constant bombardment of various stimulus from different sources.

But then again, it’s easy to blame technology for this and the culprit for distraction comes already from the state of mind on how you approach your given tasks.

Is the cause of your distraction social media? Maybe email? Maybe some IM app like Slack? Finding the main reason for your own distraction is very beneficial, since when you know that, it’s easy to limit access to it.

Jenny Odell: How to Do Nothing:

Great book on how this overly capitalistic rat-race that most of us are in can be very detrimental to people. Big reason for this was the fact that nowadays people tend to see that every hour of the day is potentially monetisable. So we’re not “allowed” to do nothing anymore. Which is sad. As a practising buddhist, I really like this message. Pausing and engaging with the world can be very beneficial in finding new and interesting meanings in our everyday experiences. There’s no need to be in some sort of constant hustle.

Geoffrey Hinton Leaves Google and Warns of Danger Ahead

Tue, 02 May 2023 20:23:51 +0200

‘Godfather of AI’ Geoffrey Hinton quits Google and warns over dangers of misinformation

Hinton, 75, said he quit to speak freely about the dangers of AI, and in part regrets his contribution to the field. He was brought on by Google a decade ago to help develop the company’s AI technology, and the approach he pioneered led the way for current systems such as ChatGPT. […]

Hinton’s concern in the short term is something that has already become a reality – people will not be able to discern what is true any more with AI-generated photos, videos and text flooding the internet.

The recent upgrades to image generators such as Midjourney mean people can now produce photo-realistic images – one such image of Pope Francis in a Balenciaga puffer coat went viral in March.

Hinton was also concerned that AI will eventually replace jobs like paralegals, personal assistants and other “drudge work”, and potentially more in the future.

Google and Amazon Struggle to Lay Off Workers in Europe

Thu, 06 Apr 2023 14:24:16 +0200

cries in corporate…

“Difficult” labour laws make it hard for corporations to fire people. Voluntary departures and individual settlements are favoured

In the US, companies can announce widespread job cuts and let go of hundreds if not thousands of workers within months — and many have. Meanwhile, in Europe, mass layoffs among tech companies have stalled because of labor protections that make it virtually impossible to dismiss people in some countries without prior consultations with employee interest groups. […]

Both in France and Germany, where labor laws are among the strongest in the EU, Google is currently in negotiations with works councils — company-specific groups whose elected employee representatives negotiate with management about workforce issues, according to a person familiar with the matter. By law, companies are required to bargain with these councils before implementing layoffs — a sometimes lengthy process that includes information gathering, negotiations and the possibility of recourse. […]

While the different standards of treatment have not created friction among Google employees spread around the world, “people have realized the way things happen in the US versus France and Germany” are different, says Parul Koul, executive chair of the Alphabet Workers Union and a software engineer at Google based in New York.

“It is inspiring for people in the US to see things are different in other places – it’s a blueprint for what people can fight for,” they added.

What I Read in March 2023

Tue, 04 Apr 2023 10:40:15 +0200

Pretty technical month in my reading this time.

Liz Rice: Container Security

I got free copies of this book and the couple of next books from an event so I decided to read them through due to how close they are to my profession. While working as a plumber, container security tends to be a very common topic and also topic that I’m very interested in. Rice is pretty popular public figure in the “cloud native” world who I’ve seen to give great talks about wonderful topics in the past, which made me quite excited about this book.

Generally speaking book itself was good. It offered a nice and relatively brief overview of various techniques that are involved in container security and also giving a nice understanding of containers itself. Personally, I probably would’ve wanted it to be slightly more practical, but nonetheless, I think it’s a great tech book addition to many bookshelves.

Liz Rice: Learning eBPF

Another book from Rice. I’ve been big fan of eBPF for many years so I was quite excited about this one also. Timing of receiving this book was great since I was just about to give a talk on how me and my colleagues at my current employer have used eBPF based technologies in our day-to-day work at Isovalent’s (creators of eBPF) Cilium (Kubernetes CNI) Workshop that we were hosting in Berlin.

I have been already working quite a bit Linux Kernel and eBPF before so there wasn’t necessarily that much new stuff from the book but there were some and lots of refreshers so I really enjoyed it.

eBPF is also a topic quite dear to me, so I already have some posts about in the woodshed. So expect more in that front from me.

John Rosso et al: Production Kubernetes

This was the last book that I got for free and was quite interested about it too. Despite working with production Kubernetes for quite a few years already, I wanted to read this to see if I could learn some new stuff from it. Overall, I think it’s a great book if you happen to work with Kubernetes. For me, most of the stuff was something that I knew already from using Kubernetes for many year, but still I think it offers a great “introduction” to how to operate Kubernetes in production.

John Maeda: The Laws of Simplicity

I’ve had this book on my library for many years but I just haven’t been able to start reading. Especially in tech, simplicity is something that tends to by highly valued, and for a reason. While this book applied mainly to design of various products etc., Maeda himself comes from technical background, so lots of the stuff he writes, can be applied to computer science and programming. I think it was a great book!

E.M. Foster: The Machine Stops (reread)

I have read this sci-fi short-story for many times already and I love it every time. It’s a very psychedelic book to read in 2020s. Book itself was written in 1909, but somehow even back then Foster was able to depict a future where humans are addicted to technology. Very short book but great! I think I originally found it from some Jaron Lanier’s talk about social media so this book made my almost luddite stance towards tech even stronger.

Utah Is First US State to Limit Teen Social Media Access

Sun, 26 Mar 2023 13:32:28 +0200

Utah has become the first US state to require social media firms get parental consent for children to use their apps and verify users are at least 18.

The bills will give parents full access to their children’s online accounts, including posts and private messages. The move comes amidst heightened concern over the impact of social media on children’s mental health. Under the measures enacted on Thursday, a parent or guardian’s explicit consent will be needed before children can create accounts on apps such Instagram, Facebook and TikTok.

I feel that most of us can agree that social media has had a very bad impact on our youth and adults alike. Mental health issues have risen linearly with the usage of social media. People’s attention span has decreased constantly due to the nature of the applications and how they grab your attention. Of course, what I’m referring here is Big Tech social media. So in many ways, banning social media would be a good idea.

But this is dangerous game US is playing. When you’re proposing something like this, the proposal can be easily applied to many other online communities that can be tremendously helpful for many. Great example for something like this is various LGBT communities online which might be the only safe haven for many, especially those living under strict regime or conservative surroundings. Can this sort of bill also ban those? Will teenagers using these sort of “social medias” be outed then?

Similar concerns are raised by Common Sense Media:

But Common Sense Media and other advocacy groups warned some parts of the new legislation could put children at risk.

Ari Z Cohn, a free speech lawyer for TechFreedom, said the bill posed “significant free speech problems”.

“There are so many children who might be in abusive households,” he told the BBC, “who might be LGBT, who could be cut-off from social media entirely.”

So time will tell how this will turn out. Personally I would be more comfortable to see more strict restrictions to be applied for these sort of companies, instead of giving the state/parents a right to go through their children’s messages.

Running staticcheck with eglot and gopls

Fri, 17 Mar 2023 21:33:10 +0100

Another neat finding in Go’s language server. Basically, I wanted to include some sort of way to run some static analyser with my language server. I remember golanci-lint was long the “de facto” tool for this, but seems that staticcheck has grown a lot in popularity. So I wanted to integrate that with my gopls.

Naturally, the first step was installing the tool itself. Fortunately, that can be done super easily with just:

$ go install honnef.co/go/tools/cmd/staticcheck@latest

Next I needed to enable this somehow with gopls. Again fortunately, all the possible settings for gopls can be found here. Including the simple variable for staticcheck. To pass in this setting to eglot, I need to configure variable eglot-workspace-configuration, which basically allows you to configure LSP servers specifically for a given project and given LSP.

To pass in the setting to the gopls, we need to pass in a plist with the configurations we want:

(use-package eglot
  :custom
  (eglot-workspace-configuration '((:gopls . ((staticcheck . t))))))

;; OR

(setq-default eglot-workspace-configuration '((:gopls . ((staticcheck . t)))))

Naturally, you can pass any setting you desire that is available for the language server this way.

What I Read in February 2023

Mon, 06 Mar 2023 20:11:35 +0100

Purely Functional Data Structures by Okasaki, Chris (reread)

I read this book when I was in my deep-deep functional programming phase couple of years back. Coming from imperative and OOP world, I already had a pretty good understanding of algorithms and data structures, but that knowledge was very much grounded in the imperative world. This book got recommended to me on how similarly useful data structures can be implemented in a purely functional way. I wanted to reread due to having some discussions with friends of mine about this same topic, so I wanted to refresh some memory about it.

Overall, I feel that it’s a very fun book to read and, compared to many other data structure books, it’s also quite fun to work with. It’s definitely niche on not so widely usable book compared to many other algorithm and data structure book, but definitely something that can give you a new view on certain problems.

The Buddha’s Teachings and Refuge by Thanissaro Bhikkhu

I have had long fascination towards Thai Forest Buddhism for many years now. Work of Thanissaro Bhikkhu was familiar to me in the form of his translations of numerous suttas. Quite recently I found out that he has also written many books on Buddha’s teaching, varying from fundamentals to interpreting Pali canon. So I wanted to wrap my head around those. Definitely recommend for anyone that cares about either Buddhist practice or just mindfulness in general.

All of his writing and talks are available at https://www.dhammatalks.org/index.html.

Insomniac City by Hayes, Bill

Every once in a while, I have some irregularities in my sleeping schedule. I got recommend this book due to its running theme of insomnia in it, so I during one of many of my sleepless nights, I decided to pick this up and start reading it. Book is about the authors, insomniacs himself, own life in New York and also the event what led to him moving there. Book covers many sleepless nights in New York City and how the author wanders around it finding stories. One of the main focal points in the book is also the author’s relation ship with the late Oliver Sacks. Very heartwarming story, definitely recommended.

Ten Arguments for Deleting Your Social Media Accounts Right Now by Lanier, Jaron and Digital Minimalism by Newport, Cal (both reread)

I decided to bundle both of these books here since they’re mainly about the same topic. Main reason on why I wanted to reread these was the recent events happening on Twitter. Despite I’m not part of any social medias (outside LinkedIn and that is also mainly due to requirements), I’m always quite interested on what’s happening around it. I thinks it’s mainly just hate towards those platforms so I just want to see them burn. Another big reason why I wanted to reread these was the fact that the popularity of Mastodon really started to get some traction. But I think this requires it’s own long form blog post about. So stay tuned on it.

No Longer Human by Dazai, Ozamu

For some reason, I’ve always had a weird fascination towards tragic human lives. Ozamu Dazai’s semi-autobiographical novel No Longer Human definitely tells the story of one. Very dark, grim and heavy book about mental illness and its effects on human nature. Don’t know if I can recommend this book for too many people. While it was very well written, the topics that it explores can be definitely be too much for many. But if you’re up to it, I think it’s worth a read

Generics-aware gopls in Emacs

Fri, 03 Mar 2023 16:51:54 +0100

I needed to write some funky Go code using relatively new Go generics in it, just to quickly notice that my LSP client in Emacs didn’t recognise those like you would expect. Fixing gopls was relatively straight-forward, since my issue seemed to be just an old version. So I just installed gopls while having Go 1.18+ installed. In my case, I had already generics-aware Go version installed but seemed like that I had installed my gopls version before the generics update, so I had to just reinstall it with:

$ go install golang.org/x/tools/gopls@latest

After that gopls was pleased, or at least my eglot didn’t complain about syntactical errors in my code. Pretty much immediately I realised that now my goimports was broken, so it didn’t organise my imports accordingly. I knew that gopls was able to do this stuff instead of using goimports, but I just never was eager to fix something that was already working. But since now it was broken, I decided to find a way to fix it.

What I did was just uninstall goimports from my machine and started relying on gopls for organising my imports. Previously, I had set my Emacs so that when I saved my files, it always ran goimports (when working on Go files, of course). Setting my eglot to do that was relatively simple, but then I noticed that it only formats my code, it doesn’t automatically import the libraries like how goimports does.

Like I mentioned earlier, gopls should be able to work exactly like goimports in this case, so I had to start digging on how I can make my eglot to do this. Basically, how gopls does this, is it uses source.organizeImports action for it. So I needed to run that somehow on save.

Fortunately, eglot exports all these code actions that the LSP can do with a neat function called eglot-code-actions. After some tinkering, I was able to call that before the save:

(use-package go-mode
  :ensure t
  :preface
  (defun tok/gofmt-before-save ()
    (interactive)
    (gofmt-before-save)
    ;; Run `eglot-code-actions' only in buffers where `eglot' is active.
    (when (functionp 'eglot-code-actions)
      (eglot-code-actions nil nil "source.organizeImports" t)))
  :hook (go-mode . (lambda ()
                     ;; Using depth -10 will put this before eglot's
                     ;; willSave notification so that the notification
                     ;; reports the actual contents that will be
                     ;; saved.
                     (add-hook 'before-save-hook 'tok/gofmt-before-save -10 t))))

I decided to use gofmt-before-save, which comes from go-mode here since I noticed that if you would just run eglot-format-buffer formatting doesn’t open a new buffer where it lists all the errors and instead prints them in the LSPs messages. You can probably dig them somehow from there and print in a new buffer, but I liked already the existing behaviour of running gofmt with go-mode so I decided to use that one.

Small fix, but a really good one. Happy generics-aware hacking.

Finally Got My Emacs Setup Just How I Like It

Fri, 24 Feb 2023 23:31:55 +0100

So in recent days, I have stumbled upon some REALLY NICE (at least in my own standards) Emacs tweaks, which I wanted to share with you.

First, something very trivial, I knew that Emacs had some sort of jump to previous location etc., type of feature available, but I never got into using it. Turns out there’s is a built-in keybinding for that or a couple. First, one was C-x C-x, which jumps to the last position and selects the text from your current position. So, e.g. you jump to the beginning of a file and press that combination, it selects the text from the beginning to the last position. Which was cool for me, but I rarely need something like that.

That was enough for me for some time, but I wanted to tweak it slightly. I didn’t care about selection and wanted to centre the screen after the jump. Thankfully, this is relatively trivial to implement in Emacs with a single function:

(defun jump-to-mark-and-center (arg)
  (interactive "*p")
  (goto-char (mark))
  (recenter))

(global-set-key (kbd "C-x C-x") 'jump-to-mark-and-center)

So I bound the old keybinding to that new function, which does exactly what I want. Lovely!

To the second lovely new configuration! I had longed for a feature in Emacs, where I could find a file based on the format of FILENAME:LINENUMBER. So, if I would have a file, let’s say file.cc and I would immediately jump to the line number 14 in that file, I would want to open that file with file.cc:14 like most of the Unix tools print these file locations, but I just couldn’t do it in Emacs.

Thankfully, after some searching throughout the interwebs, I found some nice defadvice that fixes this for me:

(defadvice find-file (around find-file-line-number
                             (filename &optional wildcards)
                             activate)
  (save-match-data
    (let* ((matched (string-match "^\\(.*\\):\\([0-9]+\\):?$" filename))
           (line-number (and matched
                             (match-string 2 filename)
                             (string-to-number (match-string 2 filename))))
           (filename (if matched (match-string 1 filename) filename)))
      ad-do-it
      (when line-number
        ;; goto-line is for interactive use
        (goto-char (point-min))
        (forward-line (1- line-number))))))

And BAM! It works just like that!

And the last lovely new feature! I use mainly vterm inside my Emacs for my terminal needs. I always wanted to use it so that when I’m inside a certain directory in the terminal, I could just open some file in that directory, but unfortunately, by default, vterm only knows the directory where it was opened at.

Thankfully, after reading some documentation about vterm, it turns out you’re able to send certain character codes to emacs from your vterm session. So you’re able to make it so that when you open vterm in directory x and proceed to change the directory inside the vterm with many different cd commands etc. to something like x/many/different/subdirs, when I run something like C-x C-f in that vterm buffer, the minibuffer inside Emacs, would know that I want to file directory in the directory where vterm currently is, instead of the directory where it was initially opened.

This can be done by doing some shell tweaking. I use zsh myself, if you use something else, refer to vterm README.

# Enable the shell to send information to vterm via properly escaped
# sequences.
vterm_printf() {
  printf "\e]%s\e\\" "$1"
}

vterm_prompt_end() {
  vterm_printf "51;A$(whoami)@$(hostname):$(pwd)"
}

 Let vterm know what dir I'm in
setopt PROMPT_SUBST
PROMPT=$PROMPT'%{$(vterm_prompt_end)%}'

If you happen to use screen or tmux, you might need to do some other tweaks in there, but these are mentioned in the vterm README.

In any case, when you define those to your .zshrc, vterm sends the information of the current directory straight to Emacs, so it knows where you’re currently at. Which is great!

To make that even better, I often noticed that I wanted to open files straight from the command line instead of running some Emacs command to open the files. Fortunately, vterm covers this also:

vterm_cmd() {
  local vterm_elisp
  vterm_elisp=""
  while [ $# -gt 0 ]; do
    vterm_elisp="$vterm_elisp""$(printf '"%s" ' "$(printf "%s" "$1" | sed -e 's|\\|\\\\|g' -e 's|"|\\"|g')")"
    shift
  done
  vterm_printf "51;E$vterm_elisp"
}

find_file() {
  vterm_cmd find-file "$(realpath "${@:-.}")"
}

alias e="find_file"

With these functions inside your .zshrc, I can run find_file inside vterm, and it opens the file in your current Emacs session. I just use short alias to run e somefile inside the terminal; it opens a new buffer for the file.

I have used Emacs for a long time, but these recent additions made it so much nicer. Hopefully, these are helpful for you too.

Machine Learning Would Have Advanced Faster With Lisp

Wed, 22 Feb 2023 15:23:28 +0100

Turing Award winner Yann LeCun dropping some much needed wisdom about ML advancements:

Hotter take: ML would have advanced faster if another front-end language had been available and widely adopted instead of Python.

One that is interactive yet fast & compilable, multithreaded (no GIL), isn’t bloated, doesn’t care about white spaces,…

E.g. Julia or some Lisp.

Hot take: Machine Learning would not have been nearly as advanced now were it not for Python. Python’s two main virtues in the context of ML:

Lowering barriers to entry.

As a scripting language, it encourages and enables experimental workflow.

Best Description of AI

Tue, 21 Feb 2023 09:33:54 +0100

Recently stumbled upon this great quote about the description of AI:

AI is magic.

You see a magic trick. You are amazed by the magic.

You are shown how the trick works. You are impressed by the technique.

You learn how to perform the trick. Now it’s not magic, it’s sleight-of-hand, or mirrors, or misdirection.

Unknown

Nice way of saying AI is bullshit.

Adding Lunar Phases to Emacs' Org Agenda

Sun, 19 Feb 2023 20:58:07 +0100

As some of you may know, I’ve been a practising Buddhist for some time, focusing mainly on the teachings of Early Buddhism and Theravada. One aspect of this practice is keeping up with Uposatha, which is also present in other schools of Buddhism. Wikipedia describes Uposatha as follows:

The Uposatha (Sanskrit: Upavasatha) is a Buddhist day of observance, in existence from the Buddha’s time (600 BCE), and still being kept today by Buddhist practitioners. The Buddha taught that the Uposatha day is for “the cleansing of the defiled mind,” resulting in inner calm and joy. On this day, both lay and ordained members of the sangha intensify their practice, deepen their knowledge and express communal commitment through millennia-old acts of lay-monastic reciprocity. On these days, the lay followers make a conscious effort to keep the Five Precepts or (as the tradition suggests) the ten precepts. It is a day for practicing the Buddha’s teachings and meditation.

Uposatha days change from lunar month to lunar month, so there are no set days for those. But generally speaking, Uposatha is observed about once a week in accordance to the lunar phases. Which lunar phases are observed depends on the culture, but there are four phases to this: the new moon, the full moon and two quarter moons between those. Theravada cultures generally observe all four, but for example, in Sri Lanka, they tend to only observe the new and the full moon.

How these Uposatha days are calculated is quite interesting, and there is a paper published on GitHub by Gambhiro Bhikkhu about that.

Personally, I have followed these moon days by adding those to my Apple Calendar via the iCal link provided in the repository above, and I’ve been quite happy with that. Lately, though, I noticed that I tend to easily miss these days since I don’t often have notifications on or my calendar open when these days happen, which made me easily miss those. That got me thinking that it would be pretty cool to have those lunar phases on my emacs since that is most likely always open for me and I’m already mainly doing my time management in org-mode.

So I started digging around on how these could be easily added to my org-agenda, and then I found a built-in command from emacs that is at least the first step there called M-x lunar-phases, which basically prints lunar phases of the last, current and next month. So I started hacking around to try to get these to my Agenda view.

Not long after starting my hacking in this, I found a nice doc piece about some ad-hoc tweaks to be made to the agenda from org-mode, which actually made exactly what I wanted. What it involves is that I need to build an agenda file with these lunar phases so that the Agenda view in emacs picks those up.

So first, I needed to add some arbitrary file to my org-agenda-files. With use-package:

(use-package org
  :custom
  (org-agenda-files '(<your other agenda files> "/path/to/lunar.org")))

Or normally:

(setq org-agenda-files '(<your other agenda files> "/path/to/lunar.org"))

I happen to have all my agenda files in my Documents folder on my macOS, so they just get synced across my devices, but naturally, you can use any path you want.

The lunar.org file itself is pretty simple. Basically, we add a header to it with a single diary sexp calling a function, org-lunar-phases, which we soon define:

* Lunar phase
+CATEGORY: Lunar
%%(org-lunar-phases)

The org-lunar-phases itself either isn’t too complicated. It involves that we pass in the current day to it, which gets passed in in a relatively odd way via the diary sexp we used above, and after that we just parse the lunar-phase-list with the current day, or month and year in this case, since lunar-phase-list returns a list of lunar phases for the next three months.

(require 'cl-lib)

;; Pass current day to `org-lunar-phases', which is annoyingly in a stupid
:: format, (MM DD YYYY).
(with-no-warnings (defvar date))
(defun org-lunar-phases ()
  "Show lunar phase in Agenda buffer."
  (require 'lunar)
  (let* ((phase-list (lunar-phase-list (nth 0 date) (nth 2 date)))
         (phase (cl-find-if (lambda (phase) (equal (car phase) date))
                            phase-list))
         (lunar-phase-names '("● New Moon"
                              "☽ First Quarter Moon"
                              "○ Full Moon"
                              "☾ Last Quarter Moon")))
    (when phase
      ;; Return the phase to the agenda file.
      (setq ret (concat (lunar-phase-name (nth 2 phase)))))))

Naturally, you can get all fancy with those lunar-phase-names. Maybe adding emojis and whatnot if you’re into it. But in all simplicity, that is how you can add lunar phases to your agenda, and it shows as following in there:

10 days-agenda (W08-W09):
Monday     20 February 2023 W08
  Lunar:      ● New Moon
Tuesday    21 February 2023
Wednesday  22 February 2023
Thursday   23 February 2023
Friday     24 February 2023
Saturday   25 February 2023
Sunday     26 February 2023
Monday     27 February 2023 W09
  Lunar:      ☽ First Quarter Moon
Tuesday    28 February 2023
Wednesday   1 March 2023

One thing that I noticed from this was the fact the GitHub link above that I mentioned. That happens to produce a little bit different results for these phases. Every once in a while, some phases differ just a tiny bit. There is a mention that the calculation method used in that paper was related to how these days are calculated in Mahānikāya in Thailand, so there might be some variance compared to the M-x lunar-phases. But personally, I feel it’s close enough and quite beneficial to me.

Telemetry in Go

Wed, 15 Feb 2023 20:29:07 +0100

Update: Go decided to go with opt-in telemetry, which at leastpersonally speaking I’m very happy with. [https://research.swtch.com/telemetry-opt-in](Read more about it from here.)

Last week, Russ Cox from Go team started a discussion about the possibility of starting to collect telemetry from Go usage.

How do software developers understand which parts of their software are being used and whether they are performing as expected? The modern answer is telemetry, which means software sending data to answer those questions back to a collection server.

I believe that open-source software projects need to explore new telemetry designs that help developers get the information they need to work efficiently and effectively, without collecting invasive traces of detailed user activity.

I have written a short series of blog posts about one such design, which I call transparent telemetry, because it collects as little as possible (kilobytes per year from each installation) and then publishes every bit that it collects, for public inspection and analysis.

I’d like to explore using transparent telemetry, or a system like it, in the Go toolchain, which I hope will help Go project developers and users alike. To be clear, I am only suggesting that the instrumentation be added to the Go command-line tools written and distributed by the Go team, such as the go command, the Go compiler, gopls, and govulncheck. I am not suggesting that instrumentation be added by the Go compiler to all Go programs in the world: that’s clearly inappropriate.

Cox also published three part introductory blog post about “transparent telemetry” that is worth a read:

So this whole discussion got me thinking about my feelings towards Go and, possibly, its future. First, I enjoy working with Go. As a language, it’s delightful to work with. It’s safe and fast, and I feel productive in it. But if the Go developers would introduce something like telemetry collecting your Go usage, I would have to re-evaluate the need/desire to use Go in current/new projects. There aren’t many developers that enjoy something like this in their toolchains. Or that would willingly allow something like that.

Sure, collecting telemetry in Visual Studio Code hasn’t affected too much in its popularity. But then again VSCodium is also quite popular, so clearly, some people hate this kind of telemetry, even in their favourite tool. Of course, I’m not yet even talking about the legality of collecting something like that. Because if something like this would be on by default, even if opt-out is offered, looking at GDPR, this can be considered illegal.

Of course, Go has had some “trust issues” for many due to it being language primarily developed, or at least funded, by Google. So naturally, people tend to have certain ideas and feelings about it even without touching it. Understandably so. Many people have already raised criticism in Go, for example, in their usage of Google run closed source Go module proxy mirror (proxy.golang.org), which is set on by default. This is also odd since Go’s import system was made to be decentralized from the get-go. Still, they decided to introduce something like this to increase reliability when importing libraries.

Considering all this, personally, I feel that if they were to introduce something like this to the Go toolchain, especially if it’s set on by default, it’d be a horrible thing for Go making the language to start fighting a big uphill battle, which may never end. I would still like to continue working in it, but if something like this were to happen, I feel that I couldn’t continue working in it if I had a choice.

Shawn Raboutou Projecting Burden of Dreams

Mon, 06 Feb 2023 08:48:28 +0100

So climbing world seems pretty stoked that Shawn has recently been projecting the infamous Burden of Dreams, 9A/V17, in Finland. Boulder is famous for being the first 9A boulder first ascent made by Nalle Hukkataival in 2016 after three years of projecting.

Lately, Shawn has also uploaded his projecting videos of the boulder to his YouTube channel, so check those out!

Carbon Costs of Self Driving Cars

Sat, 04 Feb 2023 21:53:39 +0100

MIT study finds huge carbon cost to self-driving cars

The study found that with a mass global takeup of autonomous vehicles, the powerful onboard computers needed to run them could generate as many greenhouse gas emissions as all the data centres in operation today.

These data centres currently produce around 0.14 gigatonnes of greenhouse gas emissions per year, equivalent to the entire output of Argentina or around 0.3 per cent of global emissions, according to the researchers. […]

The high emissions are the result of the huge computing workload placed on each self-driving vehicle. The researchers’ modelling assumes that the vehicles use a similar algorithm to what is popular today – a multi-task learning deep neural network, so called because it can perform many tasks at once.

These neural networks have to process an onslaught of data, simultaneously analysing the inputs provided by several onboard cameras with high frame rates to allow the car to drive on its own.

We Have This Today With Common Lisp

Fri, 03 Feb 2023 21:39:37 +0100

Despite not using Twitter, I stumble upon some Twitter threads every once in a while, and I have to read them through. This time, it was one of the tweets that John Carmack, for whom I have tremendous respect, wrote:

Programmers are also users, and “funnel drop offs” apply. “Show source” was pivotal for the web, and I wish native code behaved similarly — ctrl-break any application and be in a debugger with full source and the ability to make changes.

Precisely this kind of control-break behaviour where you can immediately after failure get into full-fledged debugging, inspectability with complete source available but also where you can evaluate any expression in any stack frame is available in Common Lisp. Today! Similar behaviour is basically impossible to implement in any modern language that requires compiling and running to do anything, meaning C, C++, Rust, Haskell, Go and so on.

Actually, just recently, Jack Rusher had a great talk in Strange Loop, called “Stop Writing Dead Programs”, which revolved around this exact topic, why we keep using technologies that were meant for batch-processing days and punch cards, and one of the topics in this was the previously mentioned ability to debug from Lisp. Check it out:

So naturally, in the Twitter thread, someone raised Lisp, and Carmack responded to that with:

Unfortunately, heavily customizable languages like lisp are their own barrier to entry, even for those that already know the language. There is a lot to be said for boring languages making it easier to contribute.

Which is fair. But, at the same time, every language can be abused; the same applies to Lisp. Of course, you can go full-on wild west with various macros, which only you can understand, but when it comes to good code (whatever that means), even in Lisp, it should be pretty straightforward and easily understandable. Functions, classes, nothing too fancy. If you need to use something fancy, the problem should REALLY require it. Great example would be reader macros from Common Lisp. Great feature, rarely needed.

It truly saddens me that we live in a world where something that Carmack talks about would be easily achievable, and you could argue when looking at the industries where Lisp used to be heavily used, we had this world, but we just decided to give it away.

Common Lisp would definitely be a fine choice for many problems that we face today, but we just decided to go elsewhere, and now we have to live with it. But maybe it’s not too late. Start using Common Lisp more!

What I Read Between November 2022 and January 2023

Fri, 03 Feb 2023 17:35:26 +0100

Some time ago, I decided to start keeping a book list and wanted to start writing some notes about them. But, unfortunately, life happened, and I forgot that I was supposed to make these notes. Of course, I don’t have anything or anyone enforcing me to do these, but I still want to keep my own promises. So, now when I found some time, I decided to do this small overview of the months that I missed. Hopefully, in the future, I could continue this habit monthly. So here are the books I read between November 2022 and January 2023.

Jason Molina: Riding with the Ghost by Osmon, Erin, ISBN: 9781538112182

Jason Molina has been one of my favourite songwriters for a very long time. Still, I never knew too much about him, so I wanted to pick this book up as a desperate fanboy.

Rautatie by Aho, Juhani

I mainly use Kindle for my reading, which naturally depends on Amazon itself, sadly. Since moving to Berlin, I’ve missed Finnish literature since, naturally, those books are pretty hard to come by here. At the same time, Amazon doesn’t really have too many of these. Then I found out that Project Gutenberg provides lots of books not only in English, but in other languages, too, such as Finnish. For which I wrote a little appreciation post a while ago.

Juhani Aho is one of the first novelists coming from Finland during the time of late 1800s and early 1900s, Rautatie (engl. Railway) being one of his most notable novels. A culturally important book about a man and woman living in the countryside in the late 1800s, where they start hearing rumours about trains without seeing them.

To Live’s to Fly: The Ballad of Late, Great Townes Van Zandt by Kruth, John, ISBN: 9780306816048

In the same category as Jason Molina’s biography, Townes Van Zandt is also one of my favourite songwriters, and I was interested in learning more about him.

The Prophet by Gibran, Kahlil, public domain

Another classic. An excellent and short book with lots of profound teaching and philosophies about life. Inspirational.

Be Here Now by Dass, Ram

When moving to Berlin, I sold everything I owned and moved only with the clothes I had. The sad part is that I had to sell all the vinyls I owned and all the books I owned. The only physical book that I brought with me was this one. Very hippy-dippy book to some, but I love it. I’ve read it multiple times before this and will most likely reread it.

One Hundred Years of Solitude by Márquez, Gabriel García, ISBN: 9780060883287

For some reason, when I started to keep this reading list, I filled it up with classics that I, for some reason, hadn’t read earlier. This is one of them. A fascinating book filled with symbolism, some could say a little bit too much of it even, telling the story of one family living in a small town and their problems with the world and how history keeps repeating itself. Classic for a reason, definitely a great read.

Juoksuhaudantie by Hotakainen, Kari

Despite not being too old book (coming from 2002), many would call it already a classic in Finnish literature. It was awarded the prize for literacy excellence (Finlandia award). When I started reading this book, it really didn’t hit me too much, but despite this, I decided to continue reading, and in the end, I feel that I was rewarded. The story of a delusional man who tries to get his family back, whom he lost due to his own mistakes.

Hävitys by Rauma, Iida

Last year’s Finlandia award winner. Tells the grim and dark story about the effects of bullying in school.

Juha by Aho, Juhani

Another classic by Juhani Aho which many would consider his most important work. Tells the story of a “love” triangle in Sweden occupied Finland.

Chat Control: The EU’s CSEM Scanner Proposal

Wed, 01 Feb 2023 11:13:09 +0100

The EU Commission proposes to oblige providers to search all private chats, messages, and emails automatically for suspicious content – generally and indiscriminately.

All of your chat conversations and emails will be automatically searched for suspicious content. Nothing remains confidential or secret. There is no requirement of a court order or an initial suspicion for searching your messages. It occurs always and automatically.

If an algorithms classifies the content of a message as suspicious, your private or intimate photos may be viewed by staff and contractors of international corporations and police authorities. Also your private nude photos may be looked at by people not known to you, in whose hands your photos are not safe.

Flirts and sexting may be read by staff and contractors of international corporations and police authorities, because text recognition filters looking for “child grooming” frequently falsely flag intimate chats.

You can falsely be reported and investigated for allegedly disseminating child sexual exploitation material. Messaging and chat control algorithms are known to flag completely legal vacation photos of children on a beach, for example. According to Swiss federal police authorities, 80% of all machine-generated reports turn out to be without merit. Similarly in Ireland only 20% of NCMEC reports received in 2020 were confirmed as actual “child abuse material”. 40% of all criminal investigation procedures initiated in Germany for “child pornography” target minors.

On your next trip overseas, you can expect big problems. Machine-generated reports on your communications may have been passed on to other countries, such as the USA, where there is no data privacy – with incalculable results.

Intelligence services and hackers may be able to spy on your private chats and emails. The door will be open for anyone with the technical means to read your messages if secure encryption is removed in order to be able to screen messages.

This is only the beginning. Once the technology for messaging and chat control has been established, it becomes very easy to use them for other purposes. And who guarantees that these incrimination machines will not be used in the future on our smart phones and laptops?

Beastie Against Fascism

Mon, 23 Jan 2023 21:00:26 +0100

Vienna, Austria and Leipzig, Germany representing!

“Beastie Smashing Fascism” -sticker was made by: https://raw.at/materialien. Unfortunately don’t know at the moment who has made the “Run BSD stop fascism” -sticker.

Source

Migrating to Contabo

Sat, 21 Jan 2023 18:49:32 +0100

So yesterday, I, at least it seems, successfully migrated to Contabo. I’ve been looking for a small server to serve my home page and other self-hosted services for some time. Yesterday I decided to ask for some recommendations around the lazyweb and got many good and bad recommendations. One of the better recommendations, it seems, was Contabo.

While I don’t use too much computing power, at least now, I still need some for my sites and services. This basically meant that pricing was obviously the most significant factor. Many other service providers provide similar features as Contabo, but it seems to be on the cheaper end, but still being something that people tend to recommend. Currently, I only need a single instance, and probably in the future, I will need some storage space, and Contabo seemed to provide both at a reasonable price.

One other “nice-to-have” was the support for *BSD. Sure most provider offer something along the lines of Custom ISO so you can use any OS you desire, but this, of course, involves lots of unnecessary stuff that I just didn’t want to start doing it. Contabo fortunately offered FreeBSD 13.1 installation out of the box, which was great! I was considering something like Hetzner for a brief second since people tended to recommend that also but then noticed that they had FreeBSD support until last summer but then decided to silently remove it with which is why I chose to go with Contabo.

There’s probably a bunch of stuff still working a bit wonky, but I bet it’s getting there. Thankfully my site is just a simple Hugo site so setting it up was quite simple. Put an nginx in front of it, and you’re pretty much good to go. Before, I ran this site on GitHub pages, limiting me in many ways.

Since I now have some computing power, I can introduce some services that I might find interesting, i.e. offering commenting and analytics but in a privacy-friendly way (as in no Disqus or Google Analytics), but it remains to be seen do I want to add something like that. One thing I probably want to start looking in to start running some self-hosted service for my videos, photos and such. I feel that what Contabo offers are suitable for it. But let’s see…

Lisa Source Code Release

Fri, 20 Jan 2023 06:10:36 +0100

Apple’s Lisa computer turned 40 and Computer History Museum had received a permission to release the source code for the Lisa software, including its system and applications software.

Despite the Lisa’s failure in the marketplace, it holds a key place in the history of the GUI and PCs more generally as the first GUI-based computer to be released by a personal computer company. Though the Xerox Star 8010 beat the Lisa to market in 1981, the Star was competing with other workstations from Apollo and Sun. Perhaps more importantly, without the Lisa and its incorporation of the PARC-inspired GUI, the Macintosh itself would not have been based on the GUI. Both computers shared key technologies, such as the mouse and the QuickDraw graphics library. The Lisa was a key steppingstone to the Macintosh, and an important milestone in the history of graphical user interfaces and personal computers more generally.

Building GPT From Scratch

Wed, 18 Jan 2023 14:15:32 +0100

Stumbled upon an interesting video by Andrej Karpathy talking about building Generatively Pretrained Transformer (GPT). I definitely need to take some time to start looking through the papers linked in the video on wonder how things work out.

N26 Goes Into Crypto

Tue, 17 Jan 2023 23:03:17 +0100

While moving to Berlin, naturally I needed to open a new bank account. Friend of mine recommended N26 to me, and, all things considered, it has treated me fine. Unfortunately I just stumbled upon the following news: German online bank N26 to expand crypto trading… Am I going to regret my decision?

German online bank N26 said on Tuesday that it would expand the list of countries where its customers may trade cryptocurrencies to include those in Germany and Switzerland. […]

The prices of major cryptocurrencies such as bitcoin and ether have fallen sharply over the past year as a broader downturn in global markets prompted investors to ditch risky assets and following the collapse of Sam Bankman-Fried’s FTX.

N26 said it would roll out the expanded trading gradually over the coming weeks.

Internet Is Empty and Devoid of People

Sat, 07 Jan 2023 06:20:09 +0100

Proving you’re a human on a web flooded with generative AI content

Humans who want to engage in informal, unoptimised, personal interactions have to hide in closed spaces like invite-only Slack channels, Discord groups, email newsletters, small-scale blogs, and digital gardens. Or make themselves illegible and algorithmically incoherent in public venues. […]

We’re about to drown in a sea of pedestrian takes. An explosion of noise that will drown out any signal. Goodbye to finding original human insights or authentic connections under that pile of cruft.

Many people will say we already live in this reality. We’ve already become skilled at sifting through unhelpful piles of “optimised content” designed to gather clicks and advertising impressions. […]

As the forest grows darker, noisier, and less human, I expect to invest more time in in-person relationships and communities. And while I love meatspace, this still feels like a loss.

Am I even real anymore?

We're Here on Earth to Fart Around

Fri, 06 Jan 2023 16:53:21 +0100

Stumbled upon a great PBS interview between David Brancaccio and Kurt Vonnegut and found a great life rule to live by:

DAVID BRANCACCIO: There’s a little sweet moment, I’ve got to say, in a very intense book– your latest– in which you’re heading out the door and your wife says what are you doing? I think you say– I’m getting– I’m going to buy an envelope.

KURT VONNEGUT: Yeah.

DAVID BRANCACCIO: What happens then?

KURT VONNEGUT: Oh, she says well, you’re not a poor man. You know, why don’t you go online and buy a hundred envelopes and put them in the closet? And so I pretend not to hear her. And go out to get an envelope because I’m going to have a hell of a good time in the process of buying one envelope. I meet a lot of people. And, see some great looking babes. And a fire engine goes by. And I give them the thumbs up. And, and ask a woman what kind of dog that is. And, and I don’t know. The moral of the story is, is we’re here on Earth to fart around. And, of course, the computers will do us out of that. And, what the computer people don’t realize, or they don’t care, is we’re dancing animals. You know, we love to move around. And, we’re not supposed to dance at all anymore.

Luddite Teens

Thu, 05 Jan 2023 06:38:27 +0100

I stumbled upon this recent New York Times article which, to be honest, was one of the most heartening things I’ve read in a long time. Especially during these times of discussion about technology addiction.

For the first time, she experienced life in the city as a teenager without an iPhone. She borrowed novels from the library and read them alone in the park. She started admiring graffiti when she rode the subway, then fell in with some teens who taught her how to spray-paint in a freight train yard in Queens. And she began waking up without an alarm clock at 7 a.m., no longer falling asleep to the glow of her phone at midnight. Once, as she later wrote in a text titled the “Luddite Manifesto,” she fantasized about tossing her iPhone into the Gowanus Canal. […]

“I still long to have no phone at all,” she said. “My parents are so addicted. My mom got on Twitter, and I’ve seen it tear her apart. But I guess I also like it, because I get to feel a little superior to them.”

Jason Isbell and George Saunders Have an Epic Conversation

Fri, 30 Dec 2022 19:27:46 +0100

Stumbled upon a great interview/conversation between one of my favourite writers AND one of my favourite songwriters!

Banging My Head Against the Wall With Haskell and C++ FFI

Sun, 18 Dec 2022 15:51:40 +0200

As some of you may know, I have a soft spot for functional programming. So most of the code I tend to write outside my professional work tends to be written in Haskell!

Unfortunately, the world we live in is still living in a world of imperative languages, and due to the sheer amount of that code, we most likely will be living in that world for years to come; I can’t say forever since the planet is not here forever, but we’re talking about a long, long time. It’s also the same for my own professional life.

I’m not very fanatical when it comes to tooling as long as it gets the job done. But still, the interaction between imperative and functional worlds is fascinating. So we are talking about foreign function interface (FFI) here.

Lately, I’ve been banging my head against the wall with the FFI of Haskell since I wanted to write a particular piece of code in mainly Haskell, but I needed something from the world of C++. Could have I written this in just Haskell? Probably, but despite enjoying functional programming more over imperative code, certain data structures, algorithms and libraries work better in the C++ world and sometimes you might need to “extra oomph” when it comes to performance.

Okay, better is a strong word for this, maybe better on how they work in the imperative world and but they don’t translate one to one to fully functional and pure world of Haskell where there are better options for data structures and algorithms. Sure some libraries are only written in C++, so there is nothing that can be done with it. For functional data structures, I definitely recommend Purely Functional Data Structures by Chris Okasaki.

Generally speaking, when it comes to FFI, interacting with C is pretty straightforward, but when C++ comes into play, some extra ceremony is required due to the nature of the language with things like unstable ABI, more extensive language and more complex features. Thankfully, C++ offers an easy way to write C ABI compatible code with the extern "C" keyword, which makes it so that the code can be called from C. But there is a caveat in this. While you can use C++ features inside the function body, when you use extern "C" functions type signature needs to be compatible with C. So no fancy STL features etc., here.

Also, when we talk about C and/or C++, memory management comes into question. So if you want to bind those languages to some memory-managed language like Haskell, you need to ensure that the memory gets handled correctly. C++ offers fancy features like RAII, smart pointers and stuff for making memory management a little bit easier, but that’s not the case in C.

Let’s start by creating a small Cabal project and some sample C++ library that we would like to interact with from Haskell.

common base
  ghc-options: -Wall -Wextra -Wno-orphans -Wno-name-shadowing
  default-language: Haskell2010
  build-depends: base ^>=4.16.3.0

executable arith
  import: base
  main-is: Main.hs
  hs-source-dirs: app

  -- C++ bits
  cxx-options: -std=c++20 -Wall -Werror -Wextra
  cxx-sources: cbits/arith_capi.cc cbits/arith.cc
  include-dirs: cbits
  extra-libraries: stdc++

While otherwise, it’s a reasonably standard Cabal boilerplate, the interesting bits are the lines relating to C++. Basically, what we do here is define some compiler options for the C++ compiler, where the C++ source files are located, and the relevant header files. Commonly, in Haskell, if you’re library/application has had anything related to C or C++ files relevant for those have usually resided in a directory called cbits, but nothing forces you to follow this convention. When that’s done, we can proceed to write some “earth-shattering” C++ library for our application.

// arith.h

#pragma once

struct arith {
  arith() noexcept;

  int add(int x, int y) noexcept;
  int sub(int x, int y) noexcept;
  int mult(int x, int y) noexcept;
  int div(int x, int y) noexcept;
};

// arith.cc

#include "arith.h"

arith::arith() noexcept {}

int arith::add(int x, int y) noexcept { return x + y; }
int arith::sub(int x, int y) noexcept { return x - y; }
int arith::mult(int x, int y) noexcept { return x * y; }
int arith::div(int x, int y) noexcept { return x / y; }

We’ll just define a simple arith struct/class with some elementary arithmetic operations. Nothing too fancy. This will work as our C++ library that we’ll interact with via Haskell. After that’s done, we need to provide some simple C API for this library so that we can interact with the library via the stable C ABI.

// arith_capi.h

#pragma once

#ifdef __cplusplus
extern "C" {
#endif

typedef struct arith arith;

extern arith *arith_new();

extern void arith_delete(arith *p);

extern int arith_add(arith *p, int x, int y);
extern int arith_sub(arith *p, int x, int y);
extern int arith_mult(arith *p, int x, int y);
extern int arith_div(arith *p, int x, int y);

#ifdef __cplusplus
}
#endif

// arith_capi.cc

#include "arith_capi.h"
#include "arith.h"

extern "C" {
  struct arith *arith_new() { return new arith(); }

  void arith_delete(arith *p) { delete p; }

  int arith_add(arith *p, int x, int y) { return p->add(x, y); }

  int arith_sub(arith *p, int x, int y) { return p->sub(x, y); }

  int arith_mult(arith *p, int x, int y) { return p->mult(x, y); }

  int arith_div(arith *p, int x, int y) { return p->div(x, y); }
}

In our C API, you’ll notice that we need to wrap our functions inside extern "C" to ensure that they’re compatible with C ABI. Also since extern "C" is a C++ keyword, we’ll wrap it in #ifdef __cplusplus directive to ensure that it gets only used if we happen to call this via C++. In the actual implementation side, you can notice that we use new and delete to do the memory management. The thing to note here is that using those keywords in “modern C++” is very much frowned upon since the language offers better ways to do that management with features like RAII, smart pointers etc., which basically makes it so that programmer don’t need memory management explicitly like we do here, but instead, they can let the compiler do it for you. We on the other need to use those keywords since we are similar management but from Haskell with its foreign pointers, which makes it so that we are able to leave the memory management to Haskell’s runtime and garbage collector.

Now we have the C++ bits done, we can proceed on how to interact with that via Haskell. All Haskell’s FFI features reside behind GHC’s {-# LANGUAGE ForeignFunctionInterface #-} language extension. So first, we need to include that in our Haskell files (either on top of the file or in the project’s Cabal file), and we can already import some of the needed modules.

{-# LANGUAGE ForeignFunctionInterface #-}

module Main where

import Control.Exception ( mask_ )
import Foreign.Ptr ( FunPtr, Ptr )
import Foreign.C.Types ( CInt(..) )
import Foreign.ForeignPtr ( ForeignPtr, newForeignPtr, withForeignPtr )

What are we importing here?

mask_ is needed to avoid leaking the pointer in case an async exception occurs between allocation and wrapping it in a foreign pointer.
Foreign.Ptr is a module holding pointers to foreign data from which we’re using only FunPtr, a pointer to a function, and Ptr, general pointer to an object.
Foreign.C.Types holds the mappings of C types to Haskell types, we’re now only using only int in C++, so Haskell type CInt corresponds to that.
Foreign.ForeignPtr, as mentioned above, is used for representing an object that is maintained in a foreign language and not Haskell’s own storage manager. The way it differs from vanilla memory type, Ptr, is that we can associate finalizers to it that can be invoked by Haskell’s storage manager, essentially cleaning it from all the references.

After the importing shenanigans, we can proceed on to make foreign imports to our code so that we can actually call the C++ code we just wrote.

data Arith

foreign import ccall unsafe "arith_capi.h arith_new" c_arithNew :: IO (Ptr Arith)
foreign import ccall unsafe "arith_capi.h &arith_delete" c_arithDelete :: FunPtr (Ptr Arith -> IO ())
foreign import ccall unsafe "arith_capi.h arith_add" c_arithAdd :: Ptr Arith -> CInt -> CInt -> IO CInt
foreign import ccall unsafe "arith_capi.h arith_sub" c_arithSub :: Ptr Arith -> CInt -> CInt -> IO CInt
foreign import ccall unsafe "arith_capi.h arith_mult" c_arithMult :: Ptr Arith -> CInt -> CInt -> IO CInt
foreign import ccall unsafe "arith_capi.h arith_div" c_arithDiv :: Ptr Arith -> CInt -> CInt -> IO CInt

So what’s happening here:

We’ll create an empty data type which we can pass in as a tag to the Ptr type
foreign import ccall unsafe "arith_capi.h arith_new" c_arithNew :: IO (Ptr Arith)
- foreign import ccall tells Haskell that we’re importing a C call.
- unsafe tell Haskell that the C call won’t be calling back to Haskell, which essentially makes it produce a little bit less overhead when crossing languages.
- "arith_capi.h arith_new" tells what we’re actually importing to Haskell.
- c_arithNew :: IO (Ptr Arith) defines the name with what we can call the function and its type signature.

Finally, you’re actually able to call these functions.

-- | Create a new foreign object that will be cleaned after it's not in use
-- anymore. It also uses mask_ in case the pointer leaks if an exception happens.
newArith :: IO (ForeignPtr Arith)
newArith = mask_ c_arithNew >>= newForeignPtr c_arithDelete

main :: IO ()
main = newArith >>= \arith -> withForeignPtr arith $ \ptr -> do
  -- Foreign object is now unwrapped to a foreign pointer which you can use in
  -- any FFI function you described above.
  c_arithAdd ptr 1 1 >>= print
  c_arithSub ptr 1 1 >>= print
  c_arithMult ptr 2 2 >>= print
  c_arithDiv ptr 2 2 >>= print

Now you should be able to run your program and interact with C++ in safe manner from the comfortable world of Haskell.

$ cabal run
# ... cabal stuff ...
2
0
4
1

So you can see that while calling C++ from a foreign language is definitely possible, it just requires a bit more ceremony than calling C from these kinds of languages. I initially started to bang my head against the wall with these FFI shenanigans just for the need to use some C++ interfaces that didn’t offer C API, so hopefully, this proves to be beneficial to some. At least, I can use this to freshen my memory in the future since I can guarantee that I’ll forget about it.

The Poetry of Programming

Thu, 08 Dec 2022 10:31:08 +0100

I found this great quote from Richard P. Gabriel, Lisp hacker and a poet, which summarizes nicely my feelings towards programming and how I see it as a creative field more so than technical:

“Writing code certainly feels very similar to writing poetry. When I’m writing poetry, it feels like the center of my thinking is in a particular place, and when I’m writing code the center of my thinking feels in the same kind of place. It’s the same kind of concentration. So, I’m thinking up possibilities, I’m thinking about, well, so how do I reinvent the code, gee, you know, what’s the simplest way to do this.”

Richard P. Gabriel

Blogging as a Form of Free Writing

Wed, 07 Dec 2022 11:02:28 +0100

My blogging, or at least the topics and themes of it, has always been quite volatile, to say the least. I haven’t really ever given too much thought to it. Still, I’ve always just written about topics that interest me at the moment in question. This might be the big reason behind the fact of why the topics in this blog vary pretty heavily, all the way from geeky programming topics, to spirituality, to ranting about corporations. You could describe this kind of writing as some sort of “free writing”, a commonly used technique in creative writing.

Regarding my own creative writing, it has always been mainly in Finnish, but these blatherings that I’ve collected for this blog have been fully English. Like my blogging, I’ve never really given too much thought to my language choice here. Maybe there is a small egomaniac in me, like in every other “artist”, and I initially thought that perhaps someone really outside Finland wants to read my writings here, which is why I chose English. Who knows?

But what is this commonly used technique called “free writing”? What does it involve? Well, like the name suggests… it involves free writing. Usually time-boxed writing and usually quite raw and often unusable material. The good thing about this sort of writing is the possibility of just focusing on producing text alone without the fear of censure and with little concern for conventions and mechanics. No regard for spelling, grammar or other corrections. Free to stray off-topic and let your thoughts lead the way. Although considering this blog, my posts haven’t been this free.

Writers of any kind, prose, lyrics, or poems, often tend to be proponents for this sort of writing where you don’t need to worry about failure, deadlines or other forms of resistance. Often mentioned how it helps to “unblock” your writer’s block. Being especially beneficial when done regularly. Is there data on that? Probably. I just don’t have it. I just know how it affects me, whether it is writing incoherent blogs for my website, writing interesting, or in my case not so interesting, lyrics, or any other kind. Just writing something tends to be the biggest helper when it comes to writing anything. Shocker, I know.

Overall, when it comes to writing, it is a skill like any other, and it needs honing. You might be currently writing your next big novel. However, you can still benefit from writing without any framework, a plan, no knowing where it’ll go, no apathy, no resentment, just words on paper. Just for a short period. Maybe 15 minutes. Maybe 30 minutes. Maybe as long as you can’t think of anything, like writing in a trance. This sort of writing doesn’t focus on the product itself but more on the thought process relating to writing.

After that, try returning to your project, whatever it might be. Maybe opening and limbering this process in your mind before you start “real work” helps you. Perhaps it can direct you to places that you never thought to explore. Some might hate free writing, and some might love it.

In Praise of Public Domain Literature

Thu, 01 Dec 2022 19:04:25 +0100

I’ve been living in Berlin for under a year, which has made a small dent in my reading habits. Not necessarily anything terrible but something I haven’t been used to before. When I’m reading books here, those tend to be written in English or German, to a smaller extent, while I’m learning the language. But my native language, Finnish, is forgotten on that list. I have always read books regularly in English, but at the same time, I’ve also read those in Finnish. But now, when I’m living abroad, the availability of those books is quite limited. Especially because I mainly read in eBook format due to the practicality of those. I had loads of books when I was still living in Finland, but moving with the number of books I had was painful, to say the least. This was the biggest reason I started reading more and more eBooks. Unfortunately, it also happens that the eBook industry in Finland is not as “booming” as it is abroad, which means that most of the new books written in Finnish don’t make it to that format. So when it comes to new books, I’m pretty out of luck.

But I still want to read in Finnish, so how can I fix this? Sure with some tremendous luck, I might find a shop here in Berlin that sells Finnish books, but that is not a guarantee. Then I realized. I’ve been a big fan of Project Gutenberg, a volunteer effort to digitalize and archive many cultural works around the world focusing on works that have become public domain due to their age or other reasons. While I had been familiar with the project earlier, I had always mainly used it for English literature and works. It wasn’t until recently that I realized they also had a vast library of books in many other languages, including Finnish. Everything is published in plain text and when available in various eBook formats also.

So I wanted to make this small post of appreciation for this project and how they helped me in many ways to continue reading great works of literature in my own native language when those are not so readily available. If you’re willing, please support their project by donating or spreading the word.

Reading List

Sun, 13 Nov 2022 22:22:57 +0100

For years, I’ve been a relatively voracious reader, but for one reason or another, I’ve never really given too much thought to my reading. Sure, while I was in school, I did the occasional book report. Although considering my history at school, these probably weren’t given much thought. This has meant that I mainly just read what I enjoy, put the book away, and that’s it. Although, occasionally, on good books, I might’ve returned to them more than once.

Lately, I’ve wanted to start to change this habit, and I’ve decided to do this is to begin maintaining a reading list/queue and writing some brief notes after I’ve read the book. By these notes, I’m not talking about any summary of the topic or anything like that but more so as a way of saving thoughts and ideas about what I’ve read. Of course, this probably also means that not all books I read are worth taking notes, but at least those that I find fascinating/interesting/great will probably have these.

Since I’ve been reading about one book per week for many years, naturally, I can’t remember everything I’ve read. This is why I will probably just reread many of the books I have already read just to include those on my list. Thankfully, I only read good books, so this won’t be a chore… right? I’ve also decided to maintain this reading list online, which can be found here.

Dealing With Injuries

Sat, 04 Jun 2022 20:04:51 +0200

Climbing and training for it have been my “mainstay” for quite some time. Naturally, when I have to stay away from it, it’ll start affecting me in one way or another. Especially these last couple of years have been relatively tough in this sense since, naturally, due to COVID: I’ve had to stay away from training, mainly due to external restrictions, closed gyms, and so on. Unfortunately, alongside this, I’ve had to live with a couple of injuries simultaneously.

First, I had a nasty fall while bouldering outside in 2019. While the fall itself was similar to many other falls that I had already had during that day, this time, I just fell a little bit awkwardly on my pads and dislocated my ankle and had two minor fractures to it at the same time. Thankfully nothing too major that a cast, rest and some rehab couldn’t fix it. But the timing of this was really unfortunate since it was very close to all this COVID non-sense, which meant that gyms in Finland started to close down, which then affected my recovery a little bit since I couldn’t get back to my regular training. Thankfully I was able to recover from my ankle issues quite nicely, albeit mobility is still a little bit worse than in my other ankle, but it is usable, at least.

During 2020 and 2021, COVID in Finland was going in waves, so we had a couple of months with no cases around, and then a couple of months later, there were hundreds of cases around. So during these “good months”, gyms were open, so I could at least train a little bit. But since COVID was still very present, and the training was quite haphazard for me. Meaning during the “busy hours”, I often didn’t want to go to the gym since there were so many people around. So there wasn’t really any regularity to my training which was a shame until around the second half of 2021.

Then, of course, I had to stumble upon a new injury around the end of 2021. This time a pulley injury on my right ring finger’s A3. So again, I had to take a couple of weeks off. Again the timing was quite unfortunate since, during the same time, COVID cases were rising in Finland and gyms had to close down again. Which again hindered my recovery.

During the first half of 2022, I also moved to Berlin, so I needed to find a new gym which basically meant finding a new “community”. Thankfully, I could get back into the groove of training in the months I’ve been living here. Unfortunately, again, after a couple weeks of training, I had a minor injury. This time a minor tear/sprain on my right knee’s LCL. Thankfully this time, it should be slightly more minor than my last two injuries so it should be healed in a couple of weeks, RICE (Rest, Ice, Compression, Elevate) and some physiotherapy/rehab.

Speaking only on my behalf (albeit I do believe that many other “athletes” feel the same), being on the disabled list definitely puts a toll on me. Mainly because while I’m injured, I can’t experience life the way I want. Why is it so? Of course, there are natural factors like pain and damage that will, of course, affect anyone experiencing those one way or another. Also, everything relating to those, like medical appointments.

But I would say climbing (alongside many other things like music) is part of my identity. So when I’m injured, I always feel that I’m losing some aspect of my whole identity. This kind of behaviour can often lead to a situation where the “athlete” tries to train through the injury, and often, making things only worse. Tied to this, being unable to do something that plays a significant part in your life can easily lead to hopelessness and loss of purpose.

Most importantly, I feel that connection plays the most prominent aspect in this. When “an athlete” is injured, they can often feel the loss of connection, especially if the community around your sport/activity is very tightly knit (like it tends to be in climbing, for example). So the fact that you need to sit out training sessions that you usually do with your “community” can significantly affect your mentality.

This one was especially crucial for me in these early months of 2022 since I had just moved to a new country, new town, without too much of connections (outside work at least). I already felt that I had become a part of the local climbing community, and after my injury, I already thought I couldn’t be a part of it. Thankfully in my case, the injury rehab and recovery time was relatively short, so I didn’t need to worry too much, but things could be worse.

But can you get away from this mindset? I think, first and foremost you need to respect your body and try to understand that even though you might not be able to take part in the activity that forms part of your identity, it doesn’t take anything away from it. Instead, try to focus your energy on healing and rehabbing so that eventually, when you’re healed, you can return even stronger. Also, when it comes to communal aspects of this, personally, I believe that isolating away from others is a big mistake. Instead you should try to champion others and give back to it as much as possible.

Now Page

Fri, 03 Jun 2022 23:06:45 +0200

My “now page” can be found here

So I decided to join the ranks of a bunch of “cool people” and create my “now page”. While my direct home page (root of this site) works as what most people could call “about” page, I feel that the “now page” reflects a little bit better on what I’m doing at this very moment. A similar effect could be achieved with random ramblings on some social media platforms, but personally I just want to stay away from those as much as possible.

That being said, most of the posts that I stumbled upon in those platforms - when I was still on those - were often related to relatively small events with a couple of big news here and there. “Now page” reflects the “big picture” a little bit better, which is why it works nicely for sharing with people I haven’t met in a long time.

With this kind of page, I believe it’ll also work to remind me of my current priorities. So if I stumble upon something new and exciting, I can reflect these on my “now page” to see if it fits there or not. Further reading:

https://nownownow.com/about

Symbolics Graphics Reel 1989

Mon, 25 Apr 2022 10:25:58 +0200

This is a collection of promotional and commercial work done by the Symbolics Graphics Division (and customers) showing off the capabilities of the Symbolics LISP Machine.

Table-Driven Testing in C++

Sun, 24 Apr 2022 21:52:29 +0200

I’ve always found tremendous value in testing my software. Especially what might be closest to home for developers - or at least should be - are unit tests. While unit tests are not necessarily the best way of making safe working code (this often requires a little bit more exhaustive testing) but at least they’re very beneficial for your future self and/or co-workers who might be working with your code since with them you can quickly see any new errors that might’ve come from regression.

That being said, often, writing unit tests can be quite cumbersome. I would love to see some mature tooling for randomized testing like QuickCheck in Haskell (and later some other languages too) that would “just work”, but often something like that just isn’t possible, especially when the project reaches a certain degree of complexity. Tests and test suites should be designed on their own as well as your code itself. Unfortunately, people tend to forget this. In these kinds of cases, quite simple table-driven test design can come to help!

I first stumbled upon table-driven test design when I was working with Go, since in there, this seems to be a quite popular way of doing unit tests, and at least, in my opinion, it works quite nicely!

Often while writing unit tests, you would want to write various failing and passing test cases, which often leads to quite a bit of duplication. For example:

TEST(TwoSumTests, PassingTest) {
  std::vector<int> nums{2, 7, 11, 15};
  auto got = twoSum(nums, 9);
  std::vector<int> expected{0, 1};
  EXPECT_EQ(got, expected);
}

TEST(TwoSumTests, FailingTest) {
  std::vector<int> nums{2, 7, 11, 15};
  auto got = twoSum(nums, 9);
  std::vector<int> expected{0, 123};
  EXPECT_NEQ(got, expected);
}

So even with this elementary example, we can see that most of the code in the test case is duplicated and/or boilerplate. So we can do better. For example, with quite a simple table for tests, we can loop through multiple tests without duplication and easily add new tests.

Regarding testing functions, we often care about what is going in and what should go out. Everything else in unit tests is often boilerplate. So where table-driven design help in setting up these input and expected outputs.

typedef struct {
  std::vector<int> nums;
  int target;
  std::vector<int> expected;
} twoSumTestCase;

TEST(TwoSumTests, BasicAssertions) {
  twoSumTestCase tests[] = {
    {
      std::vector<int>{2, 7, 11, 15},
      9,
      std::vector<int>{0, 1},
    },
    {
      std::vector<int>{3, 2, 4},
      6,
      std::vector<int>{1, 2},
    },
    {
      std::vector<int>{3, 3},
      6,
      std::vector<int>{0, 1},
    },
  };
  for (auto t : tests) {
    auto got = twoSum(t.nums, t.target);
    EXPECT_EQ(got, t.expected);
  }
}

So when we run this we can easily run all the tests at once:

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from TwoSumTests
[ RUN      ] TwoSumTests.BasicAssertions
[       OK ] TwoSumTests.BasicAssertions (0 ms)
[----------] 1 test from TwoSumTests (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 1 test.

To demonstrate failing test case, let’s add new test there:

{
  std::vector<int>{3, 3},
  6,
  std::vector<int>{0, 2},
},

We get the following output:

Expected equality of these values:
  got
    Which is: { 0, 1 }
  t.expected
    Which is: { 0, 2 }
[  FAILED  ] TwoSumTests.BasicAssertions (0 ms)
[----------] 1 test from TwoSumTests (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] TwoSumTests.BasicAssertions

 1 FAILED TEST

Extending test cases

Of course, with that information, test logs can be pretty misleading. Thankfully, we can just change the table to our liking. For example, we could add names to the tests:

typedef struct {
  std::string name;
  std::vector<int> nums;
  int target;
  std::vector<int> expected;
} twoSumTestCases;

That we could then use on diagnostic messages in GTest’s macros:

EXPECT_TRUE(false) << "diagnostic message"; // format to your liking

With this kind of formatting, we easily extend these test cases with just playing around a little bit with your test struct, so it could involve enumeration, subtests and much more. Which could help you making your tests/code easier to fix, but also easier for adding new useful and good tests.

Showing Now Playing with Hugo

Wed, 06 Apr 2022 18:50:41 +0200

So I wanted to add a “Now playing” footer to my posts so I can easily share the music I’m listening to. Maybe some people can find something new and exciting with that, so I implemented a very quick and poor man’s implementation for it! I only play with YouTube’s search_query URL parameter and pass in the song to that from the Hugo post’s front matter. I pass in the current song as a slice in the post’s front matter similar to this:

---
nowPlaying: ["DAF", "Liebe auf den Erste Blick"]
---

Then I just parse that in the template:

{{ if .Params.nowPlaying }}
{{ $artist := index .Params.nowPlaying 0 }}
{{ $song := index .Params.nowPlaying 1 }}
{{ $query := querify "search_query" ( printf "%s %s" $artist $song ) "search_type" "videos" }}
<a href="https://www.youtube.com/results?{{ $query | safeURL }}" target="_blank">
  {{ $artist }} - {{ $song }}
</a>
{{ end }}

Imperial Triumphant - Rotted Futures

Sun, 03 Apr 2022 18:04:09 +0200

Stumbled upon this weird experimental avant-garde jazz black metal type of band a while ago and have been loving them ever since!

Moving to Berlin and Hopefully More Regular Updates

Sun, 03 Apr 2022 17:29:04 +0200

So I found myself in Berlin after living many good years in Helsinki. Moving here has been a plan of mine for quite some time, and initially, it was a big reason why I joined my current employer. I had a plan on moving here a lot earlier, but due to all this COVID nonsense around the world, these plans were a little bit postponed. But hey, here we are finally in lovely Kreuzberg.

Time will tell how long I will enjoy my stay here, but I sold my earthly belongings before moving here, so the move was pretty painless. Also, if I don’t gather too much more material things around my life, moving somewhere else would probably be as easy!

Also, this post marks the start of my (hopefully) more frequent posting in a form of these smaller rambles. This is mainly due to the reason I try to stay “off-the-grid” from all these social platforms, and my friends and the family wanted to hear more updates from my side, so I might as well do it here!

Looking forward to my future here in Berlin!

Why Not Kubernetes?

Sun, 06 Feb 2022 11:21:44 +0200

First, I would like to say I think Kubernetes is an excellent platform for its intended purposes. It provides excellent fault tolerance all over the cluster, a fast and easy way to run updates on your deployments, great tools for managing services, volumes, metrics, and more, each has its own lifecycle to manage. Also, implementing your tooling by extending the Kubernetes’ API is a trivial task, so you can easily leverage the great tooling to make your own for whatever you might need.

Today it’s also effortless to spin up a Kubernetes cluster with various installers and managed options. While complex, it’s still a step closer to the idea of “just run my code and make it work”. Also, with containers in the picture, we are pretty close to the magical situation where we can run the same application similarly on the laptop and in one cloud.

For me, issues start rising when we use Kubernetes for something other than its intended purposes. While I don’t have any statistics on this, I have a pretty strong gut feeling that most of the people running Kubernetes are using it as a glorified scheduler for placing containers on nodes as fast as possible. While it’s an excellent and pretty easy tool to use for orchestrating containers. Its fundamental purpose is to orchestrate anything crucial to your infrastructure, like network, storage, and other dependencies.

Kubernetes allows the complete user freedom to run your infrastructure as you see fit. Despite sounding like a cliche, this kind of freedom can bear huge responsibilities. I would dare to say that most developers and system administrators don’t want to make these decisions. What if, at some point in the development, you wish to change your networking interface or maybe some dynamic storage provider? Can you even do such a thing in that stage of development if the decision was made before you even had anything running in Kubernetes?

Kelsey Hightower put it nicely a while back when he described Kubernetes isn’t meant for being a developer platform but a framework for creating platforms. So while it can work as a developer platform, and overall it’s pretty easy to get started, kubectl run and kubectl expose, and you’re good to go. That being said, all the API designs in Kubernetes are created for clusters and how to manage these. So while containers are part of this, much more must be leveraged. So should application developers, startups or small businesses use something like this? Probably not unless they are developing a platform product.

When we get into cluster management, we need to start thinking about managing the lifecycle of everything running inside the cluster. Unfortunately, this is also when things start to get hard. For example, what to do if something inside the cluster dies? What if I need to provision something dynamically? Kubernetes is pretty good at simplifying many of these topics, but all this complexity cannot be simplified away due to the complexity of things happening behind the scenes.

Kubernetes has a high entry threshold, and it’s a very complex project, but still, way too often, I see it marketed as a simple solution for many problems. While you can effortlessly use Kubernetes and get lots of stuff done, eventually, you will hit a wall. Deploying fault-tolerant distributed applications that are scalable against a pool of machines with dependencies in networking, storage, and more that’s a complicated problem.

Kubernetes is built for production workloads and running infrastructure beyond your demo application. For this reason, the complexity of Kubernetes is justified and should be approached with that mindset.

Spotify and the Spread of Misinformation

Thu, 27 Jan 2022 12:57:48 +0200

So Spotify seems to be a shit show at the moment, at least when sharing misinformation on their platform. At the centre of it all seems to be Joe Rogan. Spotify made a widely reported deal with Joe Rogan for exclusive rights to his podcast in 2020. Weirdly enough, Rogan has also lately been under heavy scrutiny for his idiotic statements about COVID.

In January 2022, 270 medical experts submitted an open letter for Spotify to moderate this misinformation on their platform. Letter was largely prompted because of a controversial physician who openly promoted preposterous nonsense on Joe Rogan Experience (JRE).

A few days ago, Neil Young started a protest against Spotify giving them an ultimatum for choosing either Young’s music or JRE. And well… neither Young nor 270 medical experts were enough to change Spotify’s mind about the subject since they decided that they would rather leave JRE on their platform rather than Young.

This is a weird point of view that Spotify has taken in this case since they have already participated in self-regulation before by removing harmful content from their platform. This kind of content has involved music connected one way or another to white supremacy or neo-nazi movements and much more (very much understandably so). Spotify also joined many other streaming platforms by removing another nutjob Alex Jones’ podcast InfoWars from their platform for spreading misinformation. But guess misinformation from dear Joe Rogan is not considered harmful by their standards.

Neil Young also wrote:

I sincerely hope that other artists and record companies will move off the SPOTIFY platform and stop supporting SPOTIFY’s deadly misinformation about COVID.

Unfortunately, while writing this, not many artists have joined him in this endeavour. But this also brings a little unfortunate truth about the situation, Neil Young can leave Spotify without noticing it too much. Financially, the revenue he gets from Spotify would be a very minor source of income, especially because last year, he sold 50% of his publishing rights to the investment fund Hipgnosis, netting him $150m.

Also, because his fan base is definitely - in general - on the older end of the spectrum. Yours truly is probably on the younger end. So most likely, a more significant portion of his revenue comes from physical products and live shows (although the latter is probably true for many artists).

Newer artists and bands can’t really take this kind of stand against large streaming platform since so much of their audience accesses their music via these means. In most cases, they would like to please Spotify, hoping to get in one of their playlists and so promoting their music to a vast audience.

In any case, if more big artists join Young, it definitely can impact Spotify since these kinds of heavyweight artists can pressure these big companies. But only time will tell.

RIP Thich Nhat Hanh

Sat, 22 Jan 2022 18:34:49 +0200

Google Analytics Considered... Illegal?

Wed, 19 Jan 2022 19:48:02 +0200

Some time ago, I wrote a short post about my feelings towards web analytics which were sparked due to a spike in visitors on my site (mainly coming from Hacker News). Due to that surge, I decided to part ways completely from any sort of tracking since, for me, it was mainly an unnecessary dopamine fix rather than anything useful.

Today I stumbled upon big news on the front of the legitimacy of web analytics from the point of view of privacy. Turns out, as most suspected, it’s not so good, at least according to Austria’s data protection authority.

Basically, this case dates back to the invalidation of Privacy Shield data sharing system between the EU and the US because of overreaching US surveillance. Turns out that many companies in the US have largely ignored this invalidation happened in 2020, and despite this, they have still continued to transfer data from the EU to the US. The Austrian DPA held that the use of Google Analytics by an Austrian website provider led to transfers of personal data to Google LLC in the US in violation of Chapter V. of the GDPR.

Future of Google Analytics in EU

In the long run, there will be two options: Either the US changes its surveillance laws to strengthen their tech businesses, or US providers will have to host the data of European users in Europe. This kind of transcontinental transfer is currently (as of the time of writing this) only illegal Austria, but Dutch’s DPA (data protection authority) has stated that Google Analytics “may soon no longer be allowed”.

In any case, this is a great thing for privacy in the EU, and hopefully, many more countries will join Austria in this effort. You can follow what countries have started to follow this at Is Google Analytics ILLEGAL in your country?

Adventures in Linear Types

Sun, 09 Jan 2022 12:32:43 +0200

Lately, I have dedicated a large part of my free time to audio software. I have done this mainly out of interest in the subject due to my history in music. But at the same time, I also thought writing audio software could be a fun passion project or even a small business that I could work on alongside my day job. I don’t see myself replacing my current job with this, but maybe I could dedicate 20% of my work time to it.

The world of audio software is a pretty exciting place. It involves a lot of low-level systems like signals and real-time operations, complex math at times, and something you can feel or at least hear. And what’s great, I don’t have any background in this stuff!

Now I have programmed most of my life and played around with RTOS, but when it comes to writing algorithms for manipulating digital signals, that’s new stuff for me. However, I have experience with the topic from the user point of view since I have been making music for almost as long as I have programmed. This experience involves playing instruments, how effects affect the sound, how mixing and mastering works etc. But what do linear types have to do with any of this?

Signals in the Wild

Like I said earlier, signal processing (not necessarily just audio) is very low-level stuff. So when working with signals in software, you often need to work with C or C++. This is mainly due to the performant and close-to-hardware nature of the languages required to handle and manipulate signals optimally and efficiently.

Digital signal processing is also full of algorithms. The standard workflow for people in this industry seems to be that these applications are prototyped on some high-level language before being produced. Often in languages/tools like MATLAB, Octave, Mathematica, and similar, very heavily math-oriented languages and tools. Julia has appeared to grow in popularity also in this world. These high-level languages are mainly used due to the speed of development.

It is also not uncommon to see FPGA being used in these applications. For several reasons, they are reconfigurable hardware, so you can tailor and deploy computation units and data buses specifically designed for your particular needs. So if you’re working with digital hardware, you can’t go wrong with FPGAs. In this world, VHDL or Verilog comes in handy.

As you can see, overall, the applications tend to involve a lot of different low-level concepts but, at the same time, high-level topics in terms of prototyping. But, as the post’s header might hint, I’m not interested in the prototyping aspects of signal processing since I think those are all well and good. Instead, I’m interested in having a small thought experiment on whether the low-level elements could be improved somehow.

I would consider myself a functional programmer first and foremost, even though I mainly write imperative and/or object-oriented code, at least professionally. Now in my free time and in non-trivial side projects (that are not signal processing related), I like to work with weird languages like Haskell or Common Lisp. Unfortunately, as I mentioned above, almost all the work in this signal-processing world is written in C or C++, emphasizing the latter. However, I completely understand why these languages are used since we talk about real-time programming, so latency needs to be minimized.

“Real-time” can be understood that the program has to produce the correct result but also on a certain amount of time (which varies between systems).

If we use audio processing as an example, typically, you would have some sort of processing function in your code that would work in the audio callback:

process :: BufferRef -> ()

This function would get its callback from a sound card or some input device, e.g. microphone. After it has received its callback, this block of code (whatever might be inside it) would write the corresponding audio data into the given buffer. Which would then be played by the speakers or vice-versa when recording.

This procedure is basically what should happen in real-time lots of times when we are doing audio processing. Audio software is often set up to send these audio callbacks from a high-priority “real-time” thread with a very short latency between the callbacks, ~1-10ms (varies between systems).

To achieve this minimal latency between callbacks, you often can’t rely on garbage collection since you can’t be sure when your program launches it. I dare to say that most of the software benefits from GC significantly, but in the audio, making GC right is very hard. What makes it hard is that if GC launches at the wrong time or the latency between callbacks gets too large, garbage data will leak into the buffer, causing unwanted sounds.

Most other software might only see a slight latency in their computations if they do profiling, so that might not be the end of the world, of course, depending on your context. But in audio, you cannot let that happen since you can literally hear that glitch, which is unforgivable.

When it comes to C or C++, I think everyone knows their foot guns that involve memory management. Thankfully in modern C++, it’s not that bad (as long as you follow core guidelines,) but there is still a lot of unnecessary baggage when it comes to safe code in these languages.

Could there be any way we could use garbage-collected language while doing “real-time” operations and how that could be achieved?

Linear Types

GHC 9.0 introduced support for Linear Haskell, which can be enabled with -XLinearTypes. One of the significant use cases for linear types is implementing latency-sensitive real-time services and analytics jobs. As I mentioned earlier a major issue in this use case is GC pauses, which can happen at arbitrary points for reasonably long periods. The problem is exacerbated as the size of the working set increases. The goal is to partially or entirely eliminate garbage collection by controlling aliasing and making memory management explicit but safe.

So what, then, are linear types. Henry Baker described linear types and their benefits in his paper Lively Linear Lisp — ‘Look Ma, No Garbage!’ and also on “Use-once” variables and linear objects: storage management, reflection and multi-threading. As you can see, we are not talking about a new topic. We are talking about types whose instances must be held in a linear variable. A variable is linear if it’s accessed exactly once in its scope. Same for a linear object, their reference count is always 1. When we have this safety guarantee on the type-level, we can avoid synchronization and GC and also, we could update linear objects in place since that would be referentially transparent.

Avoiding Garbage Collection

So why can we avoid synchronization and GC with linear types? If we would consider the following function as an example:

linearFunc :: a %1-> b

On their own, linear types only gives a type to functions that consume their argument exactly once when their result is consumed precisely once. So, alone, they don’t make your programs any faster or safer for resources. Still, they allow you to write many optimizations and resource-safe abstractions that weren’t possible before.

First, since linear values can only be used once, these values cannot be shared. This means that, in principle, they shouldn’t be subject to GC. But this depends on the value consumer since that may very well do some sort of de-allocation on the spot. One way to mitigate this could be to store these values to heap outside of GC’s control.

While utilizing heap for these values alone would diminish the GC, it would introduce some overhead to your program, which could increase the total running time of your application. But if we continue using real-time systems as an example, this isn’t necessarily bad.

In real-time systems, optimizations often happen only to the worst-case scenarios. This is because you don’t really care about your latencies as long as they stay within the particular window. But you watch that those latencies should never go above your maximum limit, and this is primarily where optimisation utilizing linear types could come in handy.

Practical Linear Types

Linear types are a blessing in GC languages if you intend to do anything safely in the low-level world. I would like to continue this post with some practical examples of how Haskell utilizes these types and how they can make low-level optimizations and resources safer in your Haskell code, but that deserves its own post.

Missing Technical Writing

Tue, 30 Nov 2021 17:32:05 +0200

This blog is a few years old, and during its time, it has had a couple of different style changes (and a couple of domain changes). Initially, this blog started in 2018 when I was in school, and I thought some blogging platform could improve my writing.

Most of the posts back then were some assignments for various school related works. The blog was a clever publishing platform for those and was accepted in many courses. Unfortunately, most of those have since been deleted and forgotten.

During those days, this was very tech-oriented, focusing more on writing various tutorials about different subjects, mainly around distributed computing. Then I landed my first job as a software engineer, where I could put the tutorials I had written to practice. Unfortunately, at about the same time, I started losing my motivation on regular writing, mainly due to time constraints.

Around late 2020, or early 2021, I started to rekindle my writing habit with different styles and topics. Then I began to write down my thoughts about the world around me. So many topics were about music, art, and some tech stuff here and there but not really “technical writing”. I was mainly sharing my opinions about the various topics that interested me. While I enjoyed this little bit more pondering writing style - some could call ranting - for me, it felt that it was missing something.

I missed the technical writing. But now, since I had established a particular style in my writing, it felt out of place to start writing about various geeky subjects revolving around computer science. That being said, all those writings in the tutorial and guidance way or these recent little bits of the more analyzing way of writing haven’t gone to waste. However, I feel that now I know what I want to write.

I think the combination of both will suit me the best. At the same time, I don’t necessarily see myself writing some basic how-to tutorials anymore. But I feel technical analysis with various concrete implementations would be interesting to research and write. My main interests generally lie in music and arts and how these intertwine with computer science. However, my professional life is linked to designing and implementing distributed systems, distributed computing in big data environments, systems engineering, performance, reliability, compilers and programming languages. I feel that those might also be regular visitors to my ramblings.

My professional life has also revolved around these topics. So I feel I can write a lot in conjunction with my work. Especially since working with large distributed systems, you’re bound to stumble upon some weird things, making this analysis-oriented technical writing an excellent tool for thought. I would want to have this semi-regular to regular writing habits that I once had, but that is something I can’t promise. Post at a time!

But since this is my blog, I feel that occasional rambles and shitposts about various topics are in place. Do people want to read those? Probably not, but it’s fun to vent every once in a while. So going forward, I think you can expect a significant portion of more technical commentary with a creative angle alongside a smaller portion of things that entertain and occasionally horrifies me.

Views on Digital Declutter

Mon, 20 Sep 2021 20:01:49 +0200

I’m very prone to procrastination. While I wouldn’t say that I have focus issues, I have noticed that I can easily spend hours on non-essential sites that don’t bring anything to my life. Social media has been one of them. I have always had a pretty weird relationship with social media. I joined Facebook and Instagram a long time ago because many of my friends and family were already there. While I never did post stuff actively, I always noticed that I just ended up mindlessly surfing these, especially on Instagram.

A couple of years back, I became conscious of this and decided to delete my accounts on these platforms without giving too much thought to it. While leaving these platforms was pretty easy for me, I noticed that I had just replaced these with some other platform, YouTube in my case. After which, I started spending countless hours on that platform instead. Back then, I didn’t consider this habit as bad as mindlessly browsing Instagram or Facebook despite it being the same. I think I just rationalized it to myself as being educational or informative in a better way than other platforms.

A year or two passes without Instagram or Facebook completely fine, but I wanted to start using them again for some reason. Maybe I thought I had already been cured of this disease, so I could have a healthy relationship with them from now on. I also had professional reasons behind this since I thought these platforms offer a great way of marketing your art to others, which is true in some cases. However, I quickly noticed similar behaviour when I was last on these platforms. So after a couple of months of trying to get back in, I just felt repulsed by them and decided to leave them again. When it comes to marketing, that is not for me. I understand the benefits of being an artist in social media. Still, since I mainly enjoy that as a passionate hobby, I don’t see the need for being on social media.

So at the time of writing this, I think it has been about six months or so of living without these. Still, I’m conscious of my unnecessarily large usage of YouTube, News etc. While comparing my use with Instagram, I still wouldn’t consider watching YouTube or regularly checking news as bad as mindlessly scrolling through your feeds. I still noticed similar behaviour on those I struggled with, for example, with Instagram. I became conscious about randomly picking up my phone and scrolling through the news even though I had just read them or letting YouTube’s autoplay roll for long periods without giving it too much thought. So I wanted to tackle these habits.

I have noticed that the most extreme methods work the best when fixing some bad habit, at least in my case. So I didn’t want to ease when trying to have a healthy relationship with these applications but instead went cold turkey immediately.

So how has this worked for me? I think great! In the beginning, I noticed how much free time I have when I don’t spend on useless things. Also, initially, I occasionally picked up my phone by instinct. However, I quickly realized I had no applications to spend mindlessly surfing, so I quickly grew out of this habit. At first, I felt boredom slightly when I couldn’t spend time on these apps, but thankfully I realized that this spare time needed to be used elsewhere. Before this, I was already reading relatively a lot, about three to four books per month, but I have almost doubled that number nowadays. I also wrote about time management between multiple passions a while ago, where I pondered how I manage time between, for example, programming and music. After ditching distractive sites entirely, I have felt that the time management between these activities and my work life hasn’t been an issue. Finding time for various pet projects and serious work outside my life is straightforward since I don’t spend my time on useless stuff anymore.

Do I see myself using these applications in the future? Well, I want to read the news and continue to do so, not just constantly. I usually catch up with recent events in the morning, but I don’t desire to install any news apps on my telephone. When it comes to these streaming platforms, YouTube, Netflix, etc., I could live without them. There are lots of good information on these platforms, so if I need to watch some videos, I can allow myself to do so. However, I don’t want them to control my life in a way that I’m uncomfortable with.

Passions and Time Management

Tue, 10 Aug 2021 17:28:55 +0200

I have always enjoyed reading about other people’s productivity hacks and workflows, in general, regarding whatever they might be doing. However, I often stumble upon reading how people maintain an extravagant lifestyle with dozens of different hobbies, interests, and passions with ease. So it makes me wonder how they manage their time to maintain a healthy level of participation in their interests without burning out.

I don’t have dozens of different passions or interests in my life, but my passions tend to be quite large on their own, so when combining those with 40 hour work week, I need to think about my time management thoroughly. These passions I would consider to be writing, music and programming. Fortunately, I currently work in the tech industry, so I can make a good living by doing one of my passions. Writing on its own isn’t necessarily a huge topic/interest in the sense that it only consumes time, but the practice itself is pretty straightforward. On the other hand, music is time-consuming, and it involves many different activities, in my case. Sure, you could argue that writing includes other practices, too, like planning what to write, but music is on another level. I play multiple instruments, which I record for myself and others in my home studio. I enjoy composing tunes, adding some mixing and mastering to this, and you need to sacrifice a lot of time for this. Programming is also something that I enjoy spending my time on. While I do it for a living, what makes me genuinely like it is the projects I work on in my free time, whether it’s my pet projects of various sizes or some open-source projects.

Finding the time and focusing on the task at hand isn’t necessarily an issue for me, but it’s more about maintaining a healthy balance between all these passions that I hold dear to me. Although that being said, I would consider myself a very gifted procrastinator, so focusing on the task at hand can often be tricky. However, focusing becomes no more an issue once I’ve gotten into the flow. A more significant problem here is often finishing projects rather than starting a new one (which I feel is entirely another issue to improve).

When I work on my passions, I tend to focus for days or weeks on one passion, e.g. programming, neglecting my other passions like writing and music. This on its own isn’t necessarily a bad thing since I don’t feel that I’m wasting my time when I’m doing something that I enjoy and can get rewarded in multiple ways. But I would like to maintain an equal balance between my passions.

Tools to the Rescue

A while ago, I started reading about how other people have managed their time with multiple passions/hobbies, and almost unanimously, everybody used various schedules for this. So I have already used a “life management” system for a long time to handle all my to-do lists and schedules related to my home and work life (insert praising words about emacs’ org-mode here).

I’m not going into details about how I manage my life with org-mode, but if you’re interested in the tool, I would recommend going through articles found at Org for GTD and other Task management systems and from Rainer König’s OrgMode tutorial

I realise that I have been missing for a long time in my current setup because I haven’t scheduled when to work on what project. While I’ve split my free-time projects into sub-tasks and occasionally schedule and deadline when to work/finish those, the work has always been very sporadic. The result has often been that I work for an X period on one project and then move on to another, so I usually just forget what I was supposed to do on the earlier project. For me, this often leads to unnecessary postponing or cancelling/removing tasks completely. I also quickly start saying, “I’ll do it tomorrow”, which everyone knows won’t happen.

Starting Light

So I started to approach the whole concept of time management between multiple different interests: to make dedicated timeslots and days for whatever I might be working on. I maintain numerous ongoing projects that don’t necessarily have deadlines but are just larger projects that I want to work on it from time to time. Then these projects have sub-projects which are usually scheduled with deadlines. These projects and sub-projects might include something related to work, home, open-source work, recording, or simply writing something.

I nowadays approach working these by dedicating timeslots for something on a specific day. My work life and day-to-day home stuff take a good portion of my days, but I try to use it as efficiently as possible the rest of the time. So on Monday, I might work on some programming-related endeavours based on my backlog, Tuesday something else and so on. The way I still approach tasks haven’t changed in any way, as in I still manage my tasks and TODOs and keep track of them, but nowadays, I just dedicate specific days to specific interest/passion.

Conclusion

This way, I don’t feel that I’m neglecting the stuff I want to work on. Issues with this kind of approach are the context switching almost daily. However, this kind of switching isn’t necessarily a bad thing. I don’t want to think about work-related topics after I’ve “clocked in the hours”, but I want to do something to relax or move my thoughts elsewhere. I’ve mainly stumbled with issues that when you might focus on programming one day, you focus on music. While this switch on its own hasn’t been too bad in my case, when I get back to, for example, programming after doing something else for several days, it always takes a while to get back to the flow. But I do believe that this is just missing practice in the world of managing multiple different passions.

I have now split my time between multiple passions for several weeks, which is an excellent way to go. It has also taught me about the stuff that I genuinely want to work with since when you write down what you want to do and when it’s easy to spot the stuff that you don’t want to work on or just don’t have an interest in it. So this also works in my case to find the topics that genuinely interest me. Will I continue to manage my time like this for long? Well hopefully. I feel that this way, I can contribute to all the stuff that makes my life interesting, so obviously, I wouldn’t want to miss that.

Music, AI, and the Future

Wed, 28 Jul 2021 16:39:02 +0200

Since the discussion about artificial intelligence has become mainstream, I have started to ponder AI’s possible impacts on our day-to-day lives. While I work in tech, I come from this “culture and arts” background, at least a little bit. I did some theatre when I was young and have worked with music in one way or another for most of my life. This background has got me thinking about how AI could affect these fields.

We have seen multiple interdisciplinary works mixing artificial intelligence with various art forms, like drawings, paintings, music etc. Already drawings and paintings generated with AI present a superb quality in those, in which you cannot distinguish whether these were created by AI or an actual human. On the other hand, music is not quite at that level yet, in my opinion. At least in the form of an entirely generated song by AI. That being said, I have heard great pieces utilizing both human touch and AI, where AI plays a supportive role in the whole work. Similar things can be seen in all creative endeavours where AI could be utilized.

If AI gets used more and more in these creative projects with great success, to me, it raises a question, can human art be entirely replaced with AI? I believe it would be naive to say that it couldn’t. But, considering the possible future where we cannot distinguish humans from computers, how could we determine this kind of smaller medium like song or book on how it was created or who created it? As a consumer of these kinds of mediums, does it matter if some algorithms made your new favourite novel to provide the same feeling that you might get from reading a regular author’s book?

They Are Taking Our Jobs

To put it shortly, AI can replace anyone’s job who happens to handle bits in one way or another. AI can do it way better than you ever can in these jobs. So when we talk about “creative jobs”, how can you do it better than someone else? Are you possibly better at drawing than someone else? Or can you compose better symphonies than someone else? What makes you better? Is it purely a technical thing, or is there something else? When we talk about painting or drawing’s technicality, sure, you could argue that your “pen strokes”, etc., might be better than someone else’s. But does this make it better art?

Already there has been a trend of AI-generated music populating different streaming platforms. Currently, that music has almost always been something simple in which AI can excel. This could be called elevator music or Muzak. This kind of music is most likely something that many people wouldn’t mind because it’s generated with a computer and lacks the human touch. But how would people feel if there were a chart-topping song entirely generated with AI? Again, I believe many people wouldn’t like that, other than a few tech geeks who might think it could be cool (me included).

Could AI then fully replace the human touch in our art forms? We might wait for that to happen for a very long time. Still, as I said earlier, it would be naive to think that this couldn’t happen, especially in the future, where we have reached a certain level of intelligence where we can’t distinguish each other from humans and machines.

So what could this mean for our “blue-collar” artists? What could be the driving force for them to create new art if the audience doesn’t know if it was created by a computer or a human? To me, that seems very grim.

Creative Programming

If we can’t beat them, join ’em? Right? If we think this will be the future, while it might not be a very uplifting thing to consider, it’ll most likely be very realistic. While AI will have dire repercussions on our life in the future, I also believe it can be used for great good. Whether AI is used in health, fighting climate change etc., there are many good use cases. In my opinion, utilizing AI in the arts is also one. Should you create your next song or novel entirely with AI? Possibly not, although GPT-3 has shown some great results on how good text it can write.

I like to write or play music, so I don’t want to replace the artificial process I enjoy so much. So I could utilize it in my creative endeavours by working with it side by side. It could possibly generate some ideas for my next blog post, novel, poem or whatever. For example, AI could be taught with the text of a long list of your favourite authors or songs by your favourite bands. Based on this knowledge, maybe some of the possible ideas it could generate could be finished by a human giving the final piece that human touch.

Conclusion

So while the future might look dark and grim for us, maybe we could make some use of it, so at least we might have a little bit of enjoyment. Thankfully we are a long way from this singularity that many people tend to talk about, but the trend has shown to be moving towards that kind of future. So rather than fighting against it, at least personally, I want to make the best use of our technical achievements in one way or another. Who knows if the next big novel or piece will be created with AI or other great technological invention. That said, I have already found many great ways to utilize AI in my creative projects, so who knows what might come from those.

Code Reading

Wed, 23 Jun 2021 17:13:35 +0200

Code reading has always been an activity I’ve just done without giving any thought to it. But despite this, now, when I look back at this habit, I see it as immensely beneficial. This habit caught my attention when I was reading Peter Seibel’s book Coders at Work, in which there is a section where Peter asks about code reading from his interviewees. His interviewees tended to be unanimous that code reading is very beneficial. Still, while reading his interviews, it left a picture that the practice itself seemed to be lacking even within those heavyweight programmers. The exception in this being Brad Fitzpatrick and, obviously, Donald Knuth. If these programmers speak for this practice but don’t do it in the wild, then who does? Overall, it seems pretty odd to me. Seibel made a great comparison regarding this when he compared programmers to novelists, where if a novelist hasn’t read anyone else’s publications, it would be unheard of.

I’ve always enjoyed reading others’ source code, mainly, let’s face it, to steal some ideas. But by doing this, I’ve received a long list of different lessons, ideas, and patterns, which I’ve utilized frequently in most of my work after these revelations.

Pattern Matching

One of the most significant benefits I’ve learned while reading code is that you can learn various patterns after a while. Sure, every project might seem cluttered and hard to understand for a while, but when you get the gist of it, you start to realize why this or that has been done the way it is. Furthermore, when you’ve understood some of these patterns, it gets much more comfortable to start noticing them in other similar or not-so-similar projects. Fundamentally this means the graph of WTF-per-seconds starts getting less and less.

I have also noticed that pattern matching helps understand the whole project under study itself. It would be best to try comprehending a large open-source project simultaneously but in small pieces. Then, when one of these pieces is understood, it can help tremendously understand the other pieces.

Benefits of Reinventing

It can often be pretty hard to understand the functionality of some parts of an extensive program by looking at the code. So quite often, to get a better grasp of foreign code is to reimplement the way you would write it. This way, you can abstract the bread and butter out of the program and utilize it however you want.

This kind of reimplementing can be quite hard on bigger projects. The best way to reinvent something in those projects is to change something and see changes in the new compilation. For example, try to change some text in some menu or output. This way, you can quickly test how well you understand the foreign code.

Code as a Literature Medium

Many say that code is not literature because you read it differently from prose. In my opinion, this doesn’t necessarily need to be the case. Overall, code is written for humans first and then machines second. An excellent example is Robert C. Martin’s ravings, in which he often recites that the “code should read like prose to be clean”, which I tend to agree with. Another good one is Donald Knuth’s approach to literate programming. However, the latter one is more about embedding code pieces amidst what one could call prose. Nonetheless, this kind of system makes the code much more readable since writing is such a big part.

One thing that I believe makes people think code is not literature is syntax highlighting. I don’t use it. For some reason, I never grew used to coloured text. Of course, I might be a bit biased, but when I turn on syntax highlighting, I tend to focus on the wrong things in the code, making it so that it doesn’t read like prose anymore. Removing syntax highlighting has allowed me to grasp the whole structure better. Is this true, or does it work for everyone? I don’t think so, but that’s how I feel.

Code Reading Club

Based on these thoughts and Seibel’s ideas, I decided to try some code-reading clubs in my workplace. Initially, what I had in mind for this kind of club was choosing one library/program per week/month or whatever and then dissecting the main logic behind it and discussing it. However, I quickly realized that this would most likely work since people have different interests in programming. For example, I am not interested in various GUI applications or other frontend technologies, even though they might have some good ideas behind them.

So a much better approach would most likely be that person chooses one library/program and then dissect it sharing the findings with the rest of the group. This dissection done by someone other than yourself could easily inspire you and others to dive more deeply into the code itself, even though it might be a little bit outside your interests. That being said exploring the world around your circles can be mind-opening since you can easily find new approaches to the same problems that you might face in your work.

I want to give this kind of approach a good try, and I could write some “deep thoughts” about it in the form of a review.

Extravagancy in Tech

Sat, 08 May 2021 11:53:44 +0200

I’ve started to ponder the repercussions of this trend of extravagant architectural choices in the tech industry. But unfortunately, these options seem prevalent in this current era of cloud computing. At least, I seem to stumble upon these regularly when working with a wide variety of distributed systems. Great examples of this kind of trend are various Kubernetes setups in projects where you could easily manage to progress without it or some data infrastructure solution that feels like a sledgehammer for hitting a small nail.

I’m not bashing these technologies since I enjoy working with them, and I work with them daily. They have their purpose, but this purpose is often meant for a larger picture in mind. Now, if we focus on the example of Kubernetes, sure, it can bring many benefits, like easier deployments, reduced complexity on large projects, and often reduced costs. But no one can argue that it can be overkill in many projects. If it’s not needed, it mainly brings unnecessary complexity and reduces productivity in these projects. So it can be a double-edged sword. But I don’t want to focus on these singular technologies in this topic since they feel minor on the grand scale.

Implications on Our Evolution

When we move more to this science-fiction picture of the future, we need to start thinking more about topics such as transhumanism and how we will live with machines that’ll outsmart us. Understandably, issues associated with transhumanism, like the singularity, AI, nanotechnologies, cybernetics, and much more, are challenging to discuss, first of all, on a technological level and on a moral and ethical level. But, on the other hand, it is also hard to say that we will even ever see the rise of these kinds of technologies. It could be that our civilization can see that these inventions are possible, but we cannot implement these. On the other hand, it could also be that technological evolution has also started to get so rapid that we will see a significant turn of events in these topics in the near future. [[https://www.kurzweilai.net/the-law-of-accelerating-returns][Overall technological evolution grows exponentially, so the time between significant inventions gets shorter and shorter]]. So, we can only speculate on how things might turn out.

Whatever the outcome may be, I believe that some degree of optimism is in place. However, I think the singularity is inevitable, and most of the industry’s actions indicate that the path is not good. These actions are the main reason why these over-the-top architectural choices might hint at something that might be inevitably bad.

When I talk about some projects using these “sledgehammer” solutions in projects where they aren’t necessary, I’m overall talking about a small pesky thing. What worries me about this topic is that we are using these kinds of hyped-up tools, which happen to be the month’s flavour in every project; what could this mean, for example, in the development of AI or other future technologies? Could we seem to have endless resources cause of something that cannot be reverted? Bill Joy wrote a great essay about the future not needing us, which makes it scary to think that we run these extravagant systems just mainly because we can. A similar thing applies to data collection and many other issues in privacy. Most big platforms that utilize some tracking tend to collect a lot of data, which often isn’t used thoroughly, so the data is collected to build minimal information about the user. Possibly the rest are saved for later.

Clever Usage of Limited Resources

Back in the olden days, when I wasn’t even born, computers tended to be understandably very limited in terms of resources. Computing has evolved tremendously since allowing us to use these kinds of larger-than-life solutions in environments where they wouldn’t necessarily be needed. Has the quality of systems or programs evolved directly proportional to the increase in computing power? Definitely not. The fact that these kinds of powers are available to us everywhere has possibly increased the number of innovations since more people can start thinking of possible uses for these machines that are all around us because they are in contact with them regularly. Although you could think that since more people are in contact with these machines daily, it would equal more interest in programming, etc. This doesn’t seem to be the case.

Where I’m getting with this is the fact that the quality tends to be going down when we go towards the future; how could this be tackled? Clearly, this kind of wild west design in these crucial systems can’t continue.

Strategic Approach in the Development

When we talk about this extravagancy phenomenon in tech projects, it tends to affect the program/system developers the most. Often, they are not making these decisions since it tends to be someone from the ivory tower who often plans these decisions. Thankfully, these people have at least some background in these systems relatively frequently but not always. So should the developer’s opinions matter more when considering various options for your project? Sun Microsystems had a great idea when they marketed Java to people. Sun was a hardware company that figured out that they had to please programmers first to sell more hardware, which resulted in Java being one of the most widely used languages today. Now, did Java please programmers? Maybe back when people hated C++, but opinions seem to have shifted recently, although both languages still enjoy immense support.

Overall, I think these large systems have their places in many domains, but these domains where their power could use efficiently are very rare. This ends up in a situation where we either have a lot of unnecessary computing power just lying there or used for something unnecessary. Now systems have this unnecessary complexity that mainly hinders the people’s workflow in developing the whole system.

I also think that doing something because “this might be needed in the future” is a bad practice since this tends to end up in an infinite loop of unnecessary work. Since more straightforward solutions tend to be quite often good enough for most projects with much better developer experience and much better efficiency. These solutions also often allow effortless migration to a bigger and better solution if needed. So don’t optimize if it’s not necessary.

Contemplating Web Analytics

Sun, 28 Mar 2021 09:12:55 +0200

I started to rekindle my, unfortunately, lost writing habit a couple of weeks ago. I set up Google Analytics for this page mainly due to its easy use to see simple analytics. I was only interested in visitor count and possibly where my readers’ were coming from. Google Analytics is a massive tool with massive amounts of data going into it. I tried to restrict this collection as much as possible, which suits my personal blog’s needs.

Then my page rose to the front page of Hacker News, and it started to get a lot of traction. Suddenly, thousands of readers came every day to my pesky little page with just a few posts as I followed the visitor counts rising in my Google Analytics view. That got me thinking about the ethics of this kind of tracking. Which then ended up with me deleting my account and data from it.

Discomfort With Tracking

Before I deleted my data and account from Google Analytics, I looked for alternatives. I stumbled upon many other privacy-oriented and GDPR-compliant analytics platforms, which at first seemed promising. Also, having good options for ever-prevalent Google Analytics is a great thing. But despite these features, they don’t remove the uneasiness mining your users’ data causes. Of course, we are talking about spying here. Thankfully there are now some restrictions regarding personally identifiable information (PII), at least in the GDPR, limiting the shadiness quite a lot. But that brings new issues in handling this kind of information since you need to be sure that your software doesn’t leak this information. Thankfully, opting out entirely from collecting PII in your software is an option.

I understand why people might want to add at least simplistic tracking to their sites since it can provide helpful information about your content, companies can see how users use their site, and the list goes on. Especially when you combine Google Analytics, or similar analytics tool, with ads, companies can reap significant benefits from this kind of tracking. But 9 of 10 sites shouldn’t need this. You could argue that most administrators use this tracking only for dopamine fixes and don’t utilize the tracked data. Even though they might use it somehow, how do they inform the user? I dare to say that information about data usage is almost always written in some shallow boilerplate text or in no way at all.

GDPR highlights mainly four things about data usage:

It gives EU citizens the final say on how their data is used. If your company handles PIIs, there are tighter restrictions on handling these. Companies can store/use data only if the person consents to it. User has rights to their data.

Consent is the crucial part here since many sites lack on this front. There has been a lot of discussion about what should be considered consent. GDPR Art. 6.1(f) says that “processing is necessary for the legitimate interests pursued by the controller or by a third party”. Now legitimate interest is relatively shallow, and quite a few authorities in Germany, for example, consider that third-party analytics do not fall under “legitimate interest”. You can utilize consent management platforms to ensure the user’s consent before dropping the tracking code on your page. But this then raises the question of what can be considered consent.

Drew DeVault wrote a great post about web analytics and informed consent. Informed consent is a principle from healthcare, but it still can offer significant elements to be utilized, especially in technology and privacy. Drew split up the essential elements of informed consent in tracking to these three points:

Disclosure of the nature and purpose of the research and its implications (risks and benefits) for the participant and the confidentiality of the collected information. An adequate understanding of these facts on the part of the participant, requiring an accessible explanation in lay terms and an assessment of understanding. The participant must exercise voluntary agreement, without coercion or fear of repercussions (e.g. not being allowed to use your website).

Considering these essential elements of informed consent, we agree that most tracking sites don’t follow these guidelines.

Thankfully trivial tracker blocking is supported already in many browsers, which makes this issue slightly more bearable, and also, you’re able to download external tools to do it. But still, this kind of approach is pretty upside down.

All Kinds of Cookies

Unfortunately, ad-tech companies have tried to make blocking these harder and harder by constantly evolving these cookies to evercookies, supercookies, etc. The way these have worked is that trackers have stored these harder-to-detect and delete cookies in different obscure places in the browser, like Flash storage or HSTS flags. Evercookies were a big thing in early 2010 since many sites were using Flash and Silverlight, and those were very exploitable. Today those technologies aren’t used anymore, but that doesn’t mean the evolution of cookies has stopped. On the other hand, Supercookies work on the network level of your service provider.

Thankfully lately, for example, Firefox has been able to start tackling these. In that post, the Firefox team discloses what they had to do to take some action against this, and it is wild. First, they had to re-architect the whole connection handling in the browser, which was first made to increase user experience by reducing overhead to eliminate these pesky cache-based cookies.

Still, browser fingerprinting could be considered the evilest cookie of them all. Browser fingerprinting identifies everything it can from your system. Like some cookies, this has real use cases, e.g., preventing fraud in financial institutions. Still, principally this is just another intrusive way to track people. Thankfully some modern browsers offer at least some ways to avoid this, but not a full-fledged solution (other than disposable systems).

Future of Cookies

Lately, there has been some news about privacy-friendly substitutes to cookies by tech giants. Cookies have been a relatively significant issue privacy-wise for decades, and since the ad industry is so large, finding a replacement for these has been hard. So only time will tell. We cannot get rid of cookies entirely in the near future. They might change into something else, maybe this kind of API utilizing machine learning to analyze user data. Which I don’t know is better or worse. So cannot wait! tin-foil hat tightens

Conclusion

So what is the conclusion here? Probably nothing. Recently started small-time blogger just got scared from big numbers coming into his site collecting all kinds of data which ended up with him stopping this kind of action, at least on his site. Since for most users/sites, this kind of tracking is just a silly monkey-get-banana dopamine fix.

Don’t track unless you need to; if you do, inform it thoroughly.

Leap of Faith in Email Providers

Wed, 03 Mar 2021 18:21:12 +0200

When talking about the tools of the trade, almost regardless of the industry, email seems to be a vital tool. The same applies to me. Obviously, in the tech industry, everything goes by email. But also in music. If I happen to write, record, mix or master something, I always share these via email.

Unfortunately, email is a crucial part of my workflow, so I care about my productivity while using it. So recently, I started to look for options for my two different GSuite accounts. One was used for my personal domain, and another was for my music publishing company. A big reason behind the migration was that I found GSuite too much for my needs. I don’t necessarily have anything against Google’s product, albeit I agree they have a bit too big of a footprint on the internet, so I at least try to limit my contributions to it.

Requirements for Provider

I only have two requirements for my provider: IMAP/SMTP support and the ability to use my domain(s). Given these requirements, there are probably hundreds of providers that would fit these requirements. But after a while of skimming through different providers, I ended up with FastMail and ProtonMail.

FastMail

FastMail seemed like a good fit when I first looked into it: easily manageable domains and reasonable pricing. I quickly tested it with their offered trial account and was pretty pleased with their product. However, concerns arose when I learned that the company is from Australia. Not that I hate Australia by any means, but their hostile and subversive laws regarding encryption are pretty sketchy. The assistance and access act allows, under Australia’s legislation, police to force companies to create a technical function that would give them access to encrypted messages without the user’s knowledge, which made FastMail pretty much a no-go for me.

ProtonMail

After finding Australia’s laws against encryption, it seemed like a natural choice. I had already heard of them before, and their security stand. Unfortunately, ProtonMail doesn’t support IMAP/SMTP access, at least in the standard way, mainly because of encryption, which is why I didn’t want to go that route when I first heard of them. However, they offer a kind of unorthodox solution via their ProtonMail Bridge. By my understanding, this only handles the authentication to your mail and provider localhost-only endpoints to IMAP4/SMTP. Then you can configure your mail client of choice with these new endpoints.

Attractive solution, and at least for me, it seems to work and doesn’t hinder my workflow that much. Albeit, this conveniently enables vendor lock-in, which is not very good in my books. Still, I’m pretty happy with their product and decided to migrate my emails there.

Honorable Mention: Migadu

Migadu is on the smaller end of the spectrum when talking about email providers, but overall they seemed to have great values. I didn’t go that route (yet?) because I read that they have had some outages in their services in the past. This doesn’t mean that your email has been lost since the global mail system is pretty tolerant of that, but not logging into your mail can be pretty annoying. Also, their bandwidth-based pricing and daily mail limits made them unsuitable for me. I work a lot with email and send and receive a lot of them, so they offered pricing ideal for my needs, but it was a little bit too expensive at that point.

Dishonorable Mention: Self-hosting

No.

Conclusion

FastMail at first seemed like a good fit, but due to Australia’s legislation, it just doesn’t work for me. ProtonMail overall seems like a pretty exciting provider, at least on paper. But the vendor lock-in aspect of their bridge is rather odd, although I understand why they have done it. Still, this seemed minor to me, so I’ll continue to use their service, at least for a while.

Reawakening Long Lost Habit (Or Forming a New One)

Sun, 14 Feb 2021 10:32:10 +0200

A few years ago, I had a habit of semi-regularly writing about various exciting topics. Unfortunately, time passed, and I began to write less and less, and recently I’ve gotten out of the habit altogether. This is a shame in many ways since I’ve always felt writing to be immensely therapeutic.

At the time of writing, this world is also in a very odd place. Most countries are quarantined due to COVID-19, and people stay in their homes. Yours truly included! So to pass the time during these times, I’m trying to reawaken this habit.

Habitual writing has been on my mind for a long time, especially since it has been so present in my life. I’ve also somehow lost a few other healthy habits lately, which have made me think about how I can reawaken them in my daily life. Healthy practices that come to mind that I’ve lost would definitely be workouts and meditation. Although you could argue that the lost habit of working out is mainly related to the current difficult times, I’m not too worried. I believe that eventually when the world calms down in terms of this pandemic, I can relearn that habit quite quickly. But losing the regular meditation practice is really a shame, in my opinion. Like working out, meditation has played a big part in my life for years.

Even though my meditation practices have been irregular lately, the earlier “hard work” has helped me in my everyday life. But recently, I’ve started thinking about how I could relearn this habit. I’ve learned that, at least in my own case, the best way to learn habits has definitely been to do something often but not in an excessive amount. So in meditation, this was easy. Start for 5 or 10 minutes (which is nothing, everyone can find time for this) and just do it. Current times support relearning this since people are primarily working remotely. Hence, it is easy to start your day with this practice. With these simple steps, I feel like I’ve been able to reawaken this practice that was once very present in my life.

This got me thinking about utilizing a similar approach in other habits I’ve forgotten. The habits that came to mind were music and writing. Although some could argue that these are more or less the same thing. For some reason, I’ve struggled to pick up my instruments and write some new music during the pandemic. Many others have the same feelings in their own area of interest. I don’t know the cause for this; maybe the constant staring at the same four walls for over a year is the culprit. Who knows? A similar thing has also happened in my writing.

What really got me wanting to reawaken these habits was when I stumbled upon Richard P. Gabriel’s poetry. Gabriel is a legendary Lisp programmer. As a Lisp programmer myself, I’m always interested in what other like-minded people are up to. Gabriel started a project of writing one poem a day on March 18, 2000 to end a lengthy poetry-writing slump. Gabriel agrees that he is not necessarily a great poet, even though many could argue otherwise, but I think that is non-essential. While forming this habit, you don’t necessarily need to be the new Robert Frost. But since writing poetry (or anything) is a technical skill, constant practice is bound to help you in your journey. I stumbled upon a similar approach while reading Pat Pattison’s Writing Better Lyrics, where he talked about “daily object writing” in terms of getting better at writing. Pattison also noted that forming a habit is the big thing in this, which will eventually improve writing.

This approach is more or less similar to how I learned the healthy habit of regular meditation. How could I apply a similar approach to my composing and writing? Knowing myself, I cannot do this kind of creative work sporadically (or wait for the creative slump to end), or I’ll never do it. If I tried to write one piece and post every day, I feel that doing both daily would be slightly excessive (mainly timewise). So I need to find a healthy balance in practice and not be over-encumbered.

In my case, I believe that some time-boxed, very focused practice on something works the best. So what I intend to do is I’ll focus on a period (half an hour, an hour or so) on the given task, whether it is composing, writing, or programming (another healthy habit that I practice, which thankfully hasn’t been lost, but I always feel I could do more of it). I’ll set a healthy goal for this time box, so I don’t expect to write some new groundbreaking sonata, earth-shattering blog post, or the next big open-source project. Instead, I want to do something in these fields regularly to hone my skills in the given area. Since I’m trying to work on multiple habits, I also understand that I might not always have time to do everything. That’s okay. I can most likely squeeze in a smaller session to have at least some practice. Or if I just simply cannot do anything, that’s fine too. I just don’t want to see myself doing something excessively one day and then slacking off the next day since “I did so much yesterday.” (learned from Pattison).

Productivity has been really close to my heart, even though I occasionally lack significantly in that area. But maybe with small steps, everyone can benefit from a slight boost in their productivity.

Or just procrastinate… As long as you’re happy.

FreeBSD Jails For Fun and Profit

Mon, 16 Nov 2020 19:43:56 +0200

Docker has recently stormed into software development. While its concepts are powerful and valuable, similar tools have been used in systems for decades. FreeBSD’s jails in one of those tools which build upon even older chroot(2). To put it shortly, with these tools, you can make a safe environment separated from the rest of the system.

Jails in FreeBSD are by no means a new tool (introduced in 4.x), but for one reason or another, I haven’t used them that often, which is a shame since they are so powerful. So I wanted to explore this concept in a concise and summarized manner.

Templates

ZFS datasets are a great way of creating templates for jails since, after the template creation, you can easily create new jails with zfs clone or zfs send/receive. Typically, people divide jails to complete and service jails, where the former resembles a real FreeBSD system, and the latter is often dedicated to applications/services. I’ll cover complete jails for now.

Creating templates starts with creating a dataset for your jail and template. Here I’ll make a new dataset for the base installation of FreeBSD 12.2.

$ sudo zfs create -o mountpoint=/vm zroot/vm
$ sudo zfs create zroot/vm/tmpl
$ sudo zfs create zroot/vm/tmpl/12.2

After that, fetch the base installation itself:

$ fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.2-RELEASE/base.txz
# Fetch all the necessary stuff for your template, e.g. lib32 if needed
$ sudo tar -xJvpf base.txz -C /vm/tmpl/12.2

After that, you should write a minimum viable /etc/rc.conf for the template:

$ sudo emacs /vm/tmpl/12.2/etc/rc.conf

# Start or stop services
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
syslogd_flags="-ss"
cron_flags="-J 60"

You can also disable some unnecessary jobs for jails:

$ sudo emacs /vm/tmpl/12.2/etc/periodic.conf

# No output for successful script runs.
daily_show_success="NO"
weekly_show_success="NO"
monthly_show_success="NO"
security_show_success="NO"

# Output to log files which are rotated by default.
daily_output="/var/log/daily.log"
daily_status_security_output="/var/log/daily.log"
weekly_output="/var/log/weekly.log"
weekly_status_security_output="/var/log/weekly.log"
monthly_output="/var/log/monthly.log"
monthly_status_security_output="/var/log/monthly.log"

# No need for those without sendmail
daily_clean_hoststat_enable="NO"
daily_status_mail_rejects_enable="NO"
daily_status_mailq_enable="NO"
daily_queuerun_enable="NO"

# Host does those
daily_status_disks_enable="NO"
daily_status_zfs_zpool_list_enable="NO"
daily_status_network_enable="NO"
daily_status_uptime_enable="NO"
daily_ntpd_leapfile_enable="NO"
weekly_locate_enable="NO"
weekly_whatis_enable="NO"
security_status_chksetuid_enable="NO"
security_status_neggrpperm_enable="NO"
security_status_chkuid0_enable="NO"
security_status_ipfwdenied_enable="NO"
security_status_ipfdenied_enable="NO"
security_status_ipfwlimit_enable="NO"
security_status_ipf6denied_enable="NO"
security_status_tcpwrap_enable="NO"

You also might want to enable ports in your jail:

$ sudo mkdir /vm/tmpl/12.2/usr/ports
$ sudo mkdir -p /vm/tmpl/12.2/var/ports/{distfiles,packages}
$ sudo emacs /vm/tmpl/12.2/etc/make.conf

WRKDIRPREFIX = /var/ports
DISTDIR = /var/ports/distfiles
PACKAGES = /var/ports/packages

Apply system updates to the template:

$ sudo freebsd-update -b /vm/tmpl/12.2 fetch install

Lastly, take a snapshot:

Strictly speaking, a template is a snapshot, not a dataset. The snapshot can be cloned or sent/received to generate new datasets for production jails.

$ sudo zfs snapshot zroot/vm/tmpl/12.2@complete

This creates a snapshot of zroot/vm/tmpl/12.2 named complete. You can then check your current snapshots with the following:

$ sudo zfs list -t snapshot

Creating jails from the template

Now you should create a new jail based on that snapshot. You can do it either with zfs clone or zfs send/receive:

Difference Between the Two

“A clone is a writable volume or file system whose initial contents are the same as the dataset from which it was created. As with snapshots, creating a clone is nearly instantaneous and initially consumes no additional disk space. In addition, you can snapshot a clone.” [1]

“The zfs send command creates a stream representation of a snapshot that is written to standard output. By default, a full stream is generated. You can redirect the output to a file or to a different system. The zfs receive command creates a snapshot whose contents are specified in the stream that is provided on standard input. If a full stream is received, a new file system is created as well. You can send ZFS snapshot data and receive ZFS snapshot data and file systems with these commands. See the examples in the next section.” [2]

$ sudo zfs clone zroot/vm/tmpl/12.2@complete zroot/vm/jail1

# OR

$ sudo sh -c "zfs send zroot/vm/tmpl/12.2@complete | zfs receive zroot/vm/jail1"

Jail configurations

# /etc/rc.conf

cloned_interfaces="lo0"

# PF is used for NAT and port forwarding.
pf_enable="YES"
pflog_enable="YES"

jail_enable="YES"
jail_list="jail1"

## /etc/jail.conf

exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;

host.hostname = $name;
path = "/vm/$name";
exec.consolelog = "/var/log/jail_${name}_console.log";
exec.prestart = "cp /etc/resolv.conf $path/etc";
exec.poststop = "rm $path/etc/resolv.conf";

jail1 {
        ip4.addr = "lo0|127.1.1.1/32";
        ip6.addr = "lo0|fd00:1:1:1::1/64";
        allow.chflags;
        allow.raw_sockets;
}

# /etc/hosts

...

127.1.1.1 jail1
fd00:1:1:1::1 jail1

Jail management

FreeBSD provides nifty built-in tools for jail management:

Start all jails.

$ sudo service jail start

Start a specific jail(s).

$ sudo service jail start jail1

$ sudo jexec jail1

Run a command on a jail.

$ sudo jexec jail1 ifconfig

List running jails.

$ jls
$ jls -v
$ jls -s

So that’s how you can spin up a simple restricted environment on your FreeBSD system. Of course, this topic still has many things to cover, e.g., in-depth networking and configurations.

Notes

[1] Overview of ZFS Clones: https://docs.oracle.com/cd/E19253-01/819-5461/gbcxz/index.html
[2] Sending and Receiving ZFS Data: https://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.html

References

FreeBSD Jails