I am not Web Scale: Open Source Edition

Today, while feeling very overwhelmed and in an effort to not forget anything, I made a list of all the in-flight open-source projects I have running right now. Let me present to you, dear reader, my To-Do list:

This is too many things. I'm maintaining a whole bunch of code effectively by myself. How am I gonna finish all of this stuff, with a baby and a day job?

Then I realized, "Well duh. Nobody can contribute anything if the only hints of what's coming next or what you're working on exist between your ears." So, in an effort to fix that, here's all the stuff I'm working on - I'm mentioning this, because if anyone reading this post is interested in hacking on any of it, I'd love to help them out!

But I'm new to Open Source / not sure how this stuff works!

That's fine, I'll help you out. I don't mind helping developers get involved in open-source - if something on this list is interesting to you, drop me an Email or ping me on Twitter: paul@paulbetts.org

Alright, here we go:

Refit {#refithttpsgithubcompaulcbettsrefit}

Right now, Refit doesn't work on iOS or WinRT, because it generates proxy classes at runtime using System.Reflection.Emit (via Castle.Core). Instead, we need to generate the HTTP client classes at build-time. To do this, we're going to abuse the Roslyn C# compiler, in a BeforeBuild in the user's project:

  1. Find all calls to RestService.For<T> using the Roslyn syntax parser to scan all of the C# files in the parent project. Extract out the T they're using (just as a string)

  2. Search through all of their C# files, looking for interface definitions and method definitions

  3. Generate a C# class using a template, then dump it into their project file

There's another advantage for this even on full CLR platforms, is that the generated class can be extended - all the methods are virtual, and the class is partial, so it's easy to add extra methods to the class.

Stuff to Hack On:

  • If you're interested in Roslyn and MSBuild tricks, this could be a cool project (and it's already partially done!)

ModernHttpClient {#modernhttpclienthttpsgithubcompaulcbettsmodernhttpclientpull55}

ModernHttpClient just became a PCL library in the last PR - the last thing I want to do before I ship 2.0 is implement Download Progress.

Normally I'm pretty against adding "Features" to HttpClient, but doing this via stock HttpClient is really difficult and since we already control the HttpContent streams, this feature is far easier to implement for us than it is for callers.

Stuff to Hack On:

  • Add Download Progress to ModernHttpClient on Xamarin.iOS and Xamarin.Android. HTTP Nerds Requested!

Splat 2.0 {#splat20httpsgithubcompaulcbettssplat}

The big features of Splat 2.0 are all around Logging. v2.0 cleans up a lot of the logging API (since all of my libraries follow SemVer, major version changes are the only place I can break APIs).

Splat 2.0 is actually one of the more interesting things I'm working on, because it adds dual-point logging (i.e. "Spans") that are tracked along async actions. This is super hard to get right, but if I can do it, it could be huge for debugging performance problems. What does this mean? Consider the following code:

using (Profiler.EnterSpan("Loading Items")) {  
    await Task.Run(() => {
        this.Items = LoadAllTheItems();
        Profiler.Log("Loaded {0} items", Items.Count);
    });
}

If you read the code, it's clear that "Loaded {0} Items" should be related to the span "Loading Items" (i.e. if you wanted to filter on 'all the events that happened in the Loading Items span', this log entry should definitely be included). In single-threaded apps, this is easy.

However, in an async/await world, associating that span with the log message is really difficult, especially when you include concepts like Dispatcher Items (i.e. "Run on the UI Thread"). Effectively, you have to hook / replace the Task Schedulers and track associated threads. Check out the progress so far on the span-tracking branch.

Stuff to Hack On:

  • Hooking the Dispatcher is easy in WPF, but trickier on other platforms. It needs to be reimplemented in iOS and Android, and my Cocoa/Android-fu isn't Great. Some platform experts could definitely give me a hint or two!

  • Create some good ways to unit test this madness

ReactiveUI 6.0

ReactiveUI 6.0 is getting closer to shipping a huge new release, it's been in the works for almost 6 months now. The ship thread for ReactiveUI 6.0 lists all of the new stuff that is about to be added, including a much improved ReactiveCommand.

Stuff to Hack On

ReactiveUI is actually one of the best organized of all of the open-source projects I work on, because it has a Milestone with all of the blocking release bugs

Some Highlights from that list:

Akavache 4.0

This is the closest thing I have going to shipping a new release. Basically the only thing that needs to be done is cleaning up the glitched unit tests, as well as getting a Windows Phone 8.1 build. It's also blocked on a tough PR that currently deadlocks a lot.

Akavache 4.0 drops its dependency on ReactiveUI which was a big objection as to why people didn't want to use it in the past ("I'm already using MvvmCross / Caliburn Light!"). Akavache 4.0 also adds a Vacuum method that drops expired keys as well as shrinks your database, as well as a lot of concurrency improvements vs Akavache 3.0 on SQLite3 (if yanno, I can get them to work).

Stuff to Hack On

  • I have some Grandiose Ideas™ involving the DropBox Sync API and Akavache, creating an automatically-synced Akavache backend would be Really Damn Cool. Azure Mobile Services also shipped offline support, which might also be interesting to create as an Akavache backend.

  • Someone contributing a Windows Phone 8.1 backend would be pretty cool.

  • If you're feeling really ambitious, the future of Akavache's SQLite3 backend definitely is Eric Sink's SQLite3 PCL library, the top-half of Frank Krueger's SQLite-net library needs to be ported over.

Previous post:

Next post: