What’s on the horizon?

It’s been almost a year since I’ve had the energy to seriously look at canto. Between a job that was soul-crushing with layoffs of a lot of people I consider friends, subsequent unsuccessful job hunting, a five-day two-man startup, and the usual stress of being a husband and father, I needed a hiatus.

… Until a week or two ago …

What’s in tree now

Over the past few days I’ve pushed a couple of nice commits to the canto trees.

  • Much faster Reddit plugin for the daemon
  • Much faster config stuff, so doing stuff like :browser or any other configuration from canto-curses or canto-remote is instantaneous instead of lagging for a second or two.
  • A coarse grain multi-threading implementation for the daemon.


Multi-threading is something that I’ve avoided for a long time in the daemon. Here’s why:

  • The daemon was designed to be a logical abstraction of the feed database that could function over a socket. That is, it’s meant to be a server like screen, not Apache.
  • The daemon already took advantage of threading for fetching feeds to avoid long IO timeouts by necessity, and that’s easily separated.
  • Multi-threading introduces subtle bugs

You can see why I was reticent. However, here’s why I’ve started to implement it.

  • It makes the daemon marginally more responsive.
  • It will allow better and faster client behavior.
  • The daemon is relatively mature.

The big thing here is that it will allow clients to have better behavior. For example, currently in canto-curses if you, say, start it up and then immediately switch a setting (like :filter), you will clear the items… and then proceed to get a bunch more that don’t follow that filter setting. That’s shit, but there are only three ways to handle this. Either you teach the client to ignore the subsequent responses from the daemon (which with a single thread means the daemon is still generating output, and the client is still reading it, just not acting on it), you close the socket which discards the information but also all of your connection state (like events you’re watching), or you bite the bullet and make multiple daemon connections not suck which lets you keep your state on one connection and start a fresh connection to get data with your new settings.

The yields other benefits, like being able to get data out of line. For example, opening the reader requires information that 99% of the time you don’t need hanging around (i.e. 8k of text when all you care about is the 100 byte title — over 10k items that adds up quite quickly). In the current system, the reader has to wait in line until it gets the information it needs. Crappy. If the client and daemon are threaded, the reader makes its request and, even if the daemon’s in the middle of fulfilling a big request for another connection, the reader will get it’s information immediately.

I also mentioned maturity. Canto, as a whole, isn’t even close to perfect. However, the daemon itself is pretty stable and polished and ready for an undertaking like threading.

The interface part that you actually use? Not so much. The command line sucks, there’s no completion, bad or non-existent error handling. There’s also an ultra-powerful text rendering system built into that’s basically impossible to use if you’re not me or at least decent at Python.

This multi-threading change is going to lead to a revamp of the curses interface. I’ll get into that a little later (after I’ve actually coded a lot of it), but the threaded daemon is an important part of it.

2 thoughts on “What’s on the horizon?

  1. Hello, I’ve tried to contact you via email but it did not success, your server rejected my letter. So, I’ll just ask you here.

    I need to write a plugin which will do smth (to create a file) when new rss item is added. Currently I’m using this way:

    I’ve added one more line: “os.sytem(“touch /tmp/rssitems”)” right after line “call_hook(“items_added” into canto_curses/tag.py file. But it’s not working properly, because it’s creates not only when new item is added, but even when I read the items and they are being filtered.

    Please help me at least to find proper spot in your code where I should add my thing to work :)

Leave a Reply

Your email address will not be published. Required fields are marked *