If you follow any of my repos on GitHub you’ll know that my commit style is sparse. I like a handful of well ordered, well documented commits without a whole bunch of churn. Unfortunately, that isn’t very good for communicating progress when you’re in the middle of basically detonating the code and stitching it back together as I am now so I’m here to give you a two week update on my last set of posts.

The Daemon is Probably Good

I’ve pushed a couple of commits to the daemon as I’ve been using it with the old client code. Fixed a more subtle deadlock, and finally got the database trimming code to shut the fuck up on Arch by disabling gdbm file locks (which sorta sucks, but it’s protected from other daemons by the pid file lock and if some other program kills the database in the meantime I consider it fair to have undefined behavior).

All in all, for a change as large as threading I think it’s gone well and I haven’t had any trouble with it recently. Sometime between now and when I release it (likely as 0.9.0) I’m going to give it a few days of constant uptime to coax out any issues.

Curses Client

The client is where I’m focused in the immediate term. I’ve successfully split the curses interface into a number of threads. The old code looked like

signal thread -> gui thread <- input thread

by necessity. They communicated with each other using Queues which I have now sworn off. The new code is now

signal thread |-> config thread
              |-> item thread
              |-> command thread (input)
              |-> gui thread

The config and item thread are entirely independent of the rest of the code which is nice because it means that they can be developed and debugged independently too. These were actually pretty easy to do as well since all locking is internal and as such the callback code makes it a drop in replacement.

The item thread now updates a TagCore object that mirrors the content from the daemon. Later the signal thread performs a "sync" that updates the Tag object (the graphical representation of a tag) for the next screen update. This means that the client will always have the latest tag information, but we still have control over when it's displayed.

The command thread now receives input and resolves it into commands and issues them at which point it will release the GUI thread to update the screen. The nice thing here is that gui thread can be released elsewhere too so you fun stuff like getting new items still works while the user is in an input box (which needs to be locked down), but more importantly stuff like info boxes work so help, error, or completion information can be provided while the user is in an input box.

Client Performance

I've been floored with how well the revamped client code performs. It's made the client start up snappier, it's made the occasional lagging input disappear. Surprisingly, it also fixed the Cairo terminal sluggishness, which I wasn't expecting but now understand. Basically, when the input thread is running as fast a possible grabbing keys and putting them on a queue, you end up with a whole bunch of keys that were issued before the interface was ready for them. Now that there is no input thread, only a command thread that takes input (semantics, semantics), keys between commands are ignored instead of queued. Cairo came into play with this because on the unthreaded code, Cairo made the redraw take longer which meant that commands took longer, which meant that more input got queued instead of handled and suddenly your cursor is moving seconds after you stopped giving it input. Xterm et. al., which do basically nothing but throw characters on the screen (no post-processing), meant that our commands were being done (nearly?) as fast as input. So that's been resolved now and even though Cairo terminals are still noticeably slower, they still feel responsive.

In terms of memory performance, I can guarantee that it hasn't gotten worse but I can't say if it's gotten better either. When I release 0.9.0 I'm going to do some rough memory and CPU performance metrics to quantify this.

Still TODO

Threading is almost done. I'm using it as my daily client now. There are a few caveats. I've dropped a couple of commands because they no longer make sense in their current context. There's also a bunch of little due diligence type things that need to be done, like resolving the fact that hooks are being called from the config and item threads that are manipulating objects without locking. The initial interface doesn't show up without being poked, same with the reader. I also have some HACK comments in the code for stuff that I put in just to get threading bootstrapped and now need attention. I'll most likely get this stuff cleaned up this week.

What's next

After I squash all of these new incompatibilities, I'll release the threading code to git. After that, it's going to be all about completion. I'm going to blow up the command structure. I've already routed the input through readline in the threading code, the rest is just providing information to the library and the user.