Musing and the future of feather and the Pugs repository

(This blog post will probably only interest Perl 6 hackers, since it talks only about infrastructure.)

One of the central pieces of Perl 6 infrastructure is the Pugs svn repository. It holds not only the Pugs source code, but also lots of other Perl 6 projects:

• Specification
• Test suite
• STD.pm (the standard grammar)
• SMOP
• mildew and mildew-js
• sprixel
• vill
• mp6, kp6, perlito
• elf
• various websites (perl6.org, pugscode.org, perlcabal.org/syn/)
• a host of scripts related to keep things running (rebuild the HTML version of the synopsis; smartlink checking; cronjobs for updating websites etc.)
• various documentation efforts
• An unknown number of projects more or less related to Perl 6

It's huge, but at the same time it's very practical: anybody who is interested can get write access very easily, create a new subfolder for a new project, or can fix a typo in someone else's README file without asking for commit access first.

The pugs repo is also viral: Anybody with commit access and invite new committers. Despite what you might think of it: it actually works in practice, so far I haven't seen a single case of abuse.

The pugs repository is hosted on feather, a server kindly provided by Juerd and his company. It contains three virtualized servers, feather{1,2,3}. feather2 is used for "sensitive" data (for example an IRC bot that has API keys for various github accounts, and the perl6.org website). Only a handful of "trusted" users have an account there. feather3 is used for low security stuff like evalbots which might go astray.

feather1 holds all the rest. That means the pugs repository, commitbit (the software we use for handing out commit bits and resetting svn passwords), various websites, and a whole bunch of Perl 6 developers and users have a shell account there, for trying out Perl 6 and hosting their screen + irssi sessions there.

I was about to write feather1 is maintained by a bunch of volunteers, but that would be a lie. It is "maintained" on an as-needed basis by whoever has time and feels half-way competent. It is an "interesting" mixture of Debian unstable and experimental. And it's becoming unmaintainable.

It seems clear what to do: set up a fourth virtual machine, set up a replacement for feather1, and en passant migrate some things (like websites and the pugs repo) to feather2.

But wait! There is an issue. There's always an issue. Setting up a new machine and migrating services takes time. Lots of time. And nobody wants to do it right now. Quite understandably.

Take the pugs repository for example: you might think it's easy enough to copy /var/svn/... recursively on the new host and set up Apache... except that you need authentication. And authentication is coupled to commitbit. And commitbit runs on Jifty. And if you want to install Jifty, you install half of CPAN. Does anybody know if commitbit is still maintained? and if it installs cleanly on a Debian stable?

So I thought about some changes to the infrastructure to make it easier to maintain. For example we handle commit bits differently for git projects on github: an IRC bot knows the API keys of the project owners, and can add committers to the projects. That's nice, and the IRC frontend is much leaner than the Jifty-based web frontend. But we lose virality. For security reasons the bot has to keep a whitelist of IRC users who are allowed to invite commiters. Since IRC nick names and (git|svn) user names don't always match, such a list has to be maintained manually.

The second question is: should we take the chance and move the pugs repo to git? I prefer git over svn by far, but there are also costs involved. For example Rakudo checks out a copy of the test suite via svn. The test suite is only a subdirectory of the pugs repo, so unless we split the pugs repo, we'd have to find a way to check out only a part of a git repository. I have no idea if that's possible.

Either way, I'd like to hear your thoughts, and learn from your experience:

• Do you know any "viral" git or svn hosting solution (ie where every committer can invite more committers)? (don't say commitbit - it's a PITA to maintain. Something more lightweight would be appreciated)
• If you use the pugs repository now: do you prefer svn or git? how strongly?