Status June 2016

In a break from the in-depth technical format of the previous posts, in this post I’ve elected to give a brief snapshot on development and just intersperse links to interesting bits and pieces and let the code speak for itself. My reasoning for this shift is that it’s just too time consuming to write in-depth technical articles to what amounts to development snapshots of a final product that probably won’t resemble its early stages very much.

One thing is certain in kernel development, and that’s that there is a ton of work to be done. Even mature kernels like Linux are constantly in flux, and for a kernel like Viridis this means that there are about a thousand things to be done at any given time. Since I’m the sole developer, the currency of the project is my attention and that’s not infinite. This leads to, shall we say, scaffolding where some “subsystems” aren’t anything more than a shadow of what they need to be. For example, the vsalloc implementation I wrote about in the last MM article… utter garbage, and I disclaimed that it was garbage, but considering handing out virtual addresses (at the time) was just the last small hurdle in having a working page allocator, I just needed something in place that would give me a valid answer for the handful of calls I made to it. Or the physical page allocator itself, it currently takes one gigantic alloc of 0.3% of the total memory available and eventually that will be unacceptable. For now though, with no drivers up or programs running it’s just not worth the effort until there’s something to do with the other 99.7% of memory.

Anyway, the result of the scaffolding code is that there are corners of the codebase that just aren’t worth writing about in detail because they’ll cease to exist the minute they become inadequate. Like vsalloc, in the current git it’s been reworked entirely to use a generalized domain allocator that is just all around better… but of course I haven’t written a single sentence about it until now. And that’s just stuff I’ve already written about, which is small fraction of what I’ve actually implemented.

Bringing us up to speed, here’s the stuff I’ve implemented in Viridis and haven’t had time or desire to dissect:

  • APIC. Viridis supports both xapic and x2apic via mmio and MSR.
  • ACPI. Viridis can traverse ACPI tables to detect CPUs (well, technically other APICs).
  • Basic SMP/locking. Viridis can actually wake up threads and transition them to Long Mode in a known location.
  • Fine grained memory allocation. Viridis can hand out arbitrary sized objects like kmalloc()
  • EFI. Viridis can be loaded directly by EFI firmware, transition to Runtime (ExitBootServices) and can use GOP framebuffers directly.

I could write pages about all of these, but I haven’t because that time is better spent implementing more stuff.

Which brings me to another transition. Viridis has transitioned from a toy to an experiment.

Initially I was going to implement this kernel as a sort of self-contained universe with its own quirky userspace, and a toolchain completely separate from GNU, but still very much modeled after Linux even if it wasn’t POSIX compliant. Running native ELF binaries, one memory context per process, similar schedulers, similar movement of data through the system.

Now I think, if I’m going through all of this trouble anyway, why not attempt to do something really novel? If I’ve already decided to throw away any sort of portability, then why not think about a really interesting way to organize the OS and see if you can get some interesting behavior? So I came up with a concept, and now I’m going for it.

I don’t really want to get into more detail on it, because this is still pre-prototype stage and it might be a complete pipe dream (although I’ve thought about it a lot and can’t see any obvious shortcomings), but the core idea is to fit the entire userspace more into the reactive programming paradigm where the system is oriented around data and signaling changes to that data in the fewest instructions possible – and I don’t just mean practically the fewest where you’re jumping between processes and every layer has a buffer it’s reading into and writing from, blocking on this or that, I mean literally the fewest where changes are propagated without kernel intervention if at all possible.

It’s going to require an entirely different concept of what a task is and how it’s scheduled and executed. In other words, I aim to slaughter some sacred cows and create something wholly unlike the Unix and NT descended kernels that dominate computing today. The question is whether that’s possible without making a lethal mistake, like becoming a worthless cooperative multitasking system, or losing an inordinate amount of performance or readability.

Anyway, this and a lot of other related questions, I am asking in code and we’ll see if I get any interesting answers. Worst case, and this is pretty likely, I end up with an even quirkier self contained universe to put on my mantle and admire. Best case, I revolutionize computing learn something interesting that I can write a paper about or apply elsewhere.

Leave a Reply

Your email address will not be published. Required fields are marked *