The best kind of Hackaday posts are those that address seemingly insurmountable problems with creative and elegant solutions. However, todays post deviates from that structure, as it dives into a rather frustrating experience involving software that unexpectedly ceases to function. Many of you are likely familiar with the concept of bit rot. It's that infuriating phenomenon where something that has been working flawlessly suddenly stops for no discernible reason. Well, Ive encountered a hefty dose of this issue, and instead of seeking a creative fix, I opted for a more brute-force approachthink of it as tackling the problem with a virtual chainsaw.

This saga began with a Linux Fu article from 2022, which focused on utilizing a tool called autokey.

The Problem

I rely heavily on autokey for its ability to provide emacs-style keystrokes while navigating web browsers and certain other applications. Essentially, autokey intercepts keystrokes and translates them into different ones, offering a layer of convenience for my typing needs. However, it seems that the current Linux community has taken a strong liking to Wayland, causing autokey to fall out of favor. While its not entirely accurate to say that the community despises autokey, there is an undeniable preference for Wayland over the older X11 system.

My reluctance to switch from X11 stems from the fact that I have yet to discover an equally effective alternative to autokey within the Wayland framework. Unfortunately, as the tide shifts toward Wayland, development for X11 has begun to lag, and I have started to experience some glitches. For instance, autokey is no longer available in the standard repositories for my distribution (KDE Neon). Although I have managed to install the latest version on my own, Ive noticed that my computer tends to hang, particularly after prolonged sleep sessions. On top of that, autokey often stops functioning altogether, despite still running in the background, leading me to restart the application frequently. This issue is further compounded by high memory consumption during these instances.

As with many tech users, Ive learned to live with peculiar system quirks over time. However, these minor frustrations accumulate, leading to a tipping point. It became clear to me that I needed to confront this issue head-on. Yet, the task of debugging autokey seemed daunting, especially considering it could take hours for the problem to arise.

The Chainsaw

Let me be upfront: ideally, tracking down the memory leak would be the sensible approach. My plan would involve building the program with debug symbols, running the code, and meticulously probing to uncover the root of the issue involving the interplay between X11, evdev, and other intricate components. Yet in todays fast-paced world, who really has the time for such detailed debugging?

Instead, I resolved to implement a workaround by launching a wrapper script to manage autokey. I had already removed autokey from my KDE session to prevent it from starting automatically and causing conflicts. Now, I would execute this wrapper in its place.

So, what exactly does this wrapper do? It monitors the memory usage of autokey in real-time. It quickly became apparent that its memory consumption steadily increases. Once the script detects that it has surpassed a certain threshold, it forcefully terminates the application and restarts it. The script is also programmed to relaunch autokey if it crashes, although that occurrence is rare.

Understanding Memory Metrics

The next hurdle was determining how to accurately gauge how much memory a process is utilizing. Is it measured by the number of physical pages allocated? Virtual memory space? What about shared libraries? For my purposes, I simply needed a concrete number that indicates a consistent rise in memory usage that I could monitor.

The /proc file system in Linux contains a directory for each process ID (PID), filled with an abundance of information. Within this directory, there is a comprehensive accounting of memory metrics. For instance, if you inspect /proc/$PID/smaps for a running program, youll discover data such as:

00400000-00420000 r--p 00000000 fd:0e 238814592 /usr/bin/python3.12 Size: 128 kB KernelPageSize: 4 kB MMUPageSize: 4 kB Rss: 128 kB Pss: 25 kB ...

This output includes sections for each executable and shared object, along with a wealth of information. To ascertain the total PSS (proportional set size) for the application, you can utilize a simple command:

cat /proc/$PID/smaps | grep -i pss | awk '{Total+=$2} END { print Total}'

Building the Chainsaw

Equipped with this understanding, I was able to write a script that monitors autokeys memory usage, restarting the application if it begins consuming excessive memory. Additionally, I incorporated optional debugging code for further clarity on the scripts operations. Heres a simplified version of my script:

#!/bin/bash # Run autokey, kill it if it gets too big # what's too big? $MLIMIT MLIMIT=500000 # how often to check (seconds) POLL=10 while true; do   PID=$(pgrep autokey-qt)   if [ ! -z "$PID" ]; then     PSS=$(cat /proc/$PID/smaps | grep -i pss | awk '{Total+=$2} END { print Total}')     if [ "$PSS" -gt "$MLIMIT" ]; then       kill $PID       PID=       sleep 2     fi   fi   if [ -z $PID ]; then     autokey-qt &   fi sleep $POLL done

In practice, removing the command to save the old log may be advisable once debugging is complete, but it proved beneficial in determining how often the process was terminated during troubleshooting. Running this script highlighted that the PSS for autokey was approximately 140,000 and steadily climbing every 10 seconds. Consequently, I set the threshold at 500,000, which has been effective so far.

Rethinking the Chainsaw

While this approach may seem crude, it works effectively and only took a few minutes to implement. Sure, using a chainsaw for a simple task may appear excessive, but theres something to be said for efficiency. I did contemplate simply terminating autokey at regular intervals and restarting it, but my unpredictable work schedule made this impractical; I would need to synchronize it with my screensaver activity. Alternatives to this approach certainly exist, ranging from employing systemd timers to even considering a complete abandonment of autokey. But what are your thoughts? Have you ever resorted to such a workaround for software issues? Share your experiences in the comments section below.