Perlgeek.de

Fixing Rakudo Memory Leaks

Moritz Lenz — Thu, Aug 26 13:52:12 2010

Rakudo has been leaking memory for a few month. The other day, after some nagging, Will Coleda identified a memory leak, and Tyler Curtis fixed it.

Now we can again make long-running processes with Rakudo. For example for my talk at YAPC::EU I plotted a resonance curve. For that I needed to start a new Rakudo process for every data point because it would leak so badly that it died after processing a few data points. Now I recalculated a whole curve in one process, with memory usage not exceeding 200MB of virtual mem.

I also had some fun recalculating a mandelbrot fractal in a size that would previously make Rakudo segfault or consume too much memory.

(Rendered with colomon's mandelbrot code)).

Pascal's Triangle in Perl 6

Moritz Lenz — Wed, Aug 25 17:03:32 2010

Today on IRC, Larry Wall showed this piece of Perl 6 code, which he wrote for Rosetta Code:

sub pascal { [1], -> @p { [0, @p Z+ @p, 0] } ... * };
say pascal[^10].perl
# output (reformatted for easy readbility):
# ([1],
#  [1, 1],
#  [1, 2, 1],
#  [1, 3, 3, 1],
#  [1, 4, 6, 4, 1],
#  [1, 5, 10, 10, 5, 1],
#  [1, 6, 15, 20, 15, 6, 1],
#  [1, 7, 21, 35, 35, 21, 7, 1],
#  [1, 8, 28, 56, 70, 56, 28, 8, 1],
#  [1, 9, 36, 84, 126, 126, 84, 36, 9, 1])

That's Pascal's triangle, generated in one line of Perl 6.

The ... is the series operator, which generates lists by feeding the previous value(s) (here always one array) to the generating block on its left, until it reaches the goal on the right (in this case "whatever", which means it returns a lazy, infinite list).

So for example if the previous item was the array [1, 2, 1], the code block evaluates 0, 1, 2, 1 Z+ 1, 2, 1, 0.

Z is the zip operator, Z+ is pairwise addition (ie adding the pairs that the zip operator produced). In our example that leads to 0+1, 1+2, 2+1, 1+0 or 1, 3, 3, 1.

It takes a while to get used to the meta operators and the series operator, but once you've understood them, you can do pretty neat things with them.

Programming Languages Are Not Zero Sum

Moritz Lenz — Mon, Aug 23 10:54:54 2010

From Wikipedia, the free encyclopedia:

In game theory and economic theory, zero-sum describes a situation in which a participant's gain or loss is exactly balanced by the losses or gains of the other participant(s). If the total gains of the participants are added up, and the total losses are subtracted, they will sum to zero.

Being advocate, implementor, tester and co-designer of a new programming language, I often hear objections along the lines of you are killing $other_programming_language, combined with a mixture of fear and resentment. People are afraid that having a new player on the market will decrease market share of their own, favorite programming language.

While I can understand these thinking patterns, there is no reason for concern. The market for programming languages is not a zero-sum situation. While I don't have hard data, I have the impression that the programming job sector is growing, and the US government expects it to grow further, too.

Certainly the growth of world population sets a rapidly increasing baseline, and even if we assume a constant percentage of all people related to programming in some way, the total number of programmers rises, and will continue for quite some time.

(I'm pointing to some resources about programming jobs, and I fully realize that it's not the same as number of overall programmers; but it's easier to get data for jobs, and I do think that the general trend statements are true for both).

So as long as the total number of programmers increases, a decrease in relative market share doesn't automatically mean a loss. In fact the job trends show an increase for "scripting" languages, and while Ruby is certainly the winner in terms of growth, Python, Perl and PHP win too!

Non-job data shows for example a noisy but steady growth of uploads to the Comprehensive Perl Archive Network (CPAN) -- data from a programming language that is often perceived as a loser of ruby's and python's success.

A recent Linux distribution trend analysis fell into the same trap: it shows relative numbers of search terms, and talks about a decline for all distributions except Ubuntu. Again I don't have hard numbers (the mirror infrastructure of most Linux distributions makes it nearly impossible to get accurate download counts), but I haven't seen any evidence that total usage numbers of any of the Linux distributions actually decreased.

The State of Regex Modifiers in Rakudo

Moritz Lenz — Mon, Aug 16 17:16:15 2010

During the last one and a half month, I've been working on making regex modifiers easily available in Rakudo.

The regex compiler itself has to support only a few of the adverbs that can be applied to regexes; those include :ignorecase, :sigspace, :ignoremark and :continue/:pos. NQP-rx, the regex engine that Rakudo uses under the hood, supports those (except :ignoremark), so previously you could write

if 'ABC' ~~ /:i abc/ {
    say "case insensitive match";
}

But not

if 'ABC' ~~ rx:i/abc/ {
    say "case insensitive match";
}

nor m:i/abc/, for that matter.

I've patched Rakudo to actually recognize those adverbs outside of the regex, and also for s/// substitutions.

Another category of adverbs are those that apply to regex calls, not to the compilation of a regex. Among those are :global/:g, :overlap/:ov, :nth($n), :x. I've implemented those for substitutions, but implementing them for m// turns out to be quite a bit harder.

The reason is the return value: each regex match returns a Match object, which can store positional and named parts. S05 says that regex matches with multiple results should return a single match object, with all results as positional parts. It can be distinguished from a normal match object by evaluating it in slice context... which Rakudo doesn't support yet.

Now the subst method and thus s/// are implemented by calling .match(:global, ...), and without slice context, it can't distinguish between multiple matches, and a single match with subcaptures. And so my changes to the global match broke the substitution, and I see no easy way to fix it.

Anyway, here are a few examples of what works today:

$_ = 'ab12fg34';
s:g/\d/X/;
.say; # output: abXXfgXX


$_ = 'Hello, World';
# :ii is the same as :samecase
s:ii/world/perl/;
.say; # output: Hello, Perl

$_ = 'I did not know that that work together';
s:2nd/that/they/;
.say; # output: I did not know that they work together

What is the "Cool" class in Perl 6?

Moritz Lenz — Thu, Aug 12 15:28:39 2010

In Perl, subroutine and operator names determine what happens, usually not the type of the arguments. Instead the arguments are coerced to a type on which the operation makes sense:

say uc 34;      # coerces 34 to a string, and upper-cases it
say 1 + "2";    # converts "2" to a number before adding

To make things more extensible, the uc function re-dispatches to the uc method on its argument. So for the example above to work, we need an uc function in Int. And in Array, so that @a.uc works. And so on.

The original approach was to stuff all these methods into Any, the base class of the object hierarchy. Which kinda worked, but also meant that all user-defined classes ended up having some few hundreds methods to start with. Not good.

These days, the type Cool fills this niche: most built-in types (all that are meant to be used in that polymorphic way) inherit from Cool, so the uc method is actually defined in class Cool, coerces to string, and then re-dispatches to the internal logic that actually does the upper-casing.

The name either stands for Convenient object oriented loopback, or just expresses that that most built-ins are cool with an argument of that type.

If users want to write a type that can be used like a built-in type now just inherit from Cool, and define coercion methods to other built-in types. If the types don't inherit from Cool, they are more light-weight, and less magic. There's more than one way to do it.

Cool is a class (and not a role), because classes are mutable; so if you want to inject behavior into nearly all built-in types, augmenting Cool is an option (though usually considered evil, and should not be done lightly).

What is "Modern Perl"?

Moritz Lenz — Thu, Aug 12 15:17:32 2010

These days you often hear term Modern Perl, as something new(ish), and much improved over the old ways.

But what is it exactly? Well, there's no proper definition, but here is what that term means to me:

It's a set of tools, ideas and attitudes that help you to write better Perl programs, and allows you to have more fun while writing them.

Here are some aspects of Modern Perl

Testing: Most modern Perl modules have extensive test suites, that make development sane and robust
Some built-ins now come with safer forms: the three-argument form of open() allows you to open files safely with arbitrary characters in them, without any extra precautions. Lexical file handles make things safer and easier too.
use strict; use warnings;
Proper OO: with Perl 6 and with Moose in Perl 5, we have good object systems, that require less low-level fiddling than the standard Perl 5 object system
Following Best Practices
(For open source projects) Liberally handing out commit privileges. The source is stored in a version control system anyway, so low-quality changes or vandalism can simply be reverted (but that doesn't happen often in practice).
Caring about marketing: do tell people that you built something cool and useful
Small handy modules such as List::Util and Try::Tiny
Development tools such as Devel::Cover and Devel::NYTProf
(update) perlbrew and local::lib to help maintain your own perl installation and locally installed modules.

All of these techniques help to write scalable Perl programs by making proper encapsulation much easier, or by avoiding common errors, identifying performance bottlenecks etc.

Update: after watching some discussions about this post in various media, I should add a few more tools that I forgot about earlier:

Devel::REPL, Read-Evaluation-Print Loop
Perl::Critic, a code quality and style tester
DDBIx::Class, a modern and very popular object relational mapper
Catalyst, a web framework based on the MVC pattern

Perl 6 Questions on Perlmonks

Moritz Lenz — Tue, Aug 10 11:18:42 2010

Since the Rakudo Star release, there has been a noticeable increase in Perl 6 questions on perlmonks - a good sign, because it means people are using it.

Array interpolation in Rakudo
Rakudo build problems
Perl6, latest Rakudo bug or error on my part? (parsing issue related to missing whitespace).
Perl6, modifying @*INC (another ws parsing issue)
Perl6's LWP::Simple -- seemingly an installation issue
Perl 6, defining a hash with interpolated string

I've assembled this list by looking through the 100 newest nodes in Seekers of Perl Wisdom, which makes it 6% of the questions asked, or on average 1 per day (used to be around 1 per week).

Most of the questions are related to environmental issues (building/installing stuff), or beginner's questions related to syntax.

It's good to see the questions flowing in, and I hope that we'll soon see more questions where I can show off cool Perl 6 features in the answers :-).

Notes from the YAPC::EU 2010 Rakudo hackathon

Moritz Lenz — Fri, Aug 6 16:48:40 2010

At YAPC::EU 2010 we had a long discussion about Perl 6, Rakudo and related matters. Here are some (very incomplete) notes of the ongoing discussions and results.

Attendees

Patrick Michaud, Jonathan Worthington, Carl Mäsak, Moritz Lenz, Gabor Szabo, and a fluctuation of other Perl 6 hackers.

Speed

What can we do to improve Rakudo's performance?

jnthn's grant proposal for a low-level meta object protocol: See http://news.perlfoundation.org/2010/07/hague-grant-application-meta-m.html. Will probably bring the biggest speed improvement of all options we have under our control
Rakudo built-in optimizations: Most Rakudo built-ins are written for correctness first, and without a good feeling for what constructs are fast and what aren't. A thorough review (and preferably profiling) could bring decent speed improvements, as the case of int ranges showed.
Garbage collector: Parrot's GC is... suboptimal. To be gentle.
Optimization framework: We will try to convince people that Tyler Curtis' optimization framework for PAST and POST should be shipped with parrot (probably compile PIRs in ext/, just like NQP-rx does it now). Using that, we can do constant folding
Moving stuff to compile time: Number parsing needs to be moved to compile time.

What do we need to keep hacking?

Brought up by Gabor

Money

We do much volunteer work, but when we get funding, we can devote more time to hacking

Travel/Conferences

We'd like to get together a few times (2? 3? 4?) a year, in real life.

Funding and organization would be very welcome

Short-time funding

It would be nice to have a way to have funding available much more quickly than through the usual grant process, which tends to be longish.

Rakudo Star feedback

Good: It worked. It did what we wanted it to.

Bad:

It lacked a module installer (It shipped proto, but didn't install it).
Compilation takes too much memory. pmichaud will try a hack to split the setting, which would solve that problem.
There was some discussion about the roles + outer scopes bugs, which was way over my head. It seems to be related to the fact that parrot has two outer chains for nested blocks: one at compile time, one at runtime. Since role methods are flattened into classes, there compile time outer block is actually different than where it runs, and that screws up ... forget it, somebody else must describe it.
Lack of modules - doesn't seem to bee a big problem
Lack of features: not a big problem.

Biggest complaints: missing perl6doc. Missing non-blocking IO, binary file support.
Prefix paths with spaces are not supported :(

jnthn: "I actually tried to write a C program that binary patches the perl6 executable to allow spaces in path names. It almost worked."
We will try to advocate compilation to PBC, not PIR - once that's supported.

Proto/Pls

Proto needs to be end-of-life'd.

It confuses people that there are two different project lists, and the lists diverge.

We would like to decentralize the module list somehow. Still open how.

People don't release Perl 6 modules, because there's no need so far, and it's tedious to add the version name in each .pm/.pm6 file. We might need to come up with a clever idea for that.

Backend diversity

Additionally to the parrot backend, we want to run Perl 6 code on other virtual machines.

jnthn will work on a .NET/CLR port. He wants to prototype the new low-level class composition code in .NET anyway, which will provide the basic foundations for running NQP.

pmichaud wants to explore javascript on V8 as a possible backend. "I managed PIR, I'll certainly manage javascript" :-)

Huge time sink, but still worth doing it
Apache runtime library might be worth looking into
risks: stalled refactors are dangerous (see: PHP 6, cardinal (the ruby-on-parrot compiler))

We want to avoid fragmentation into many subprojects
We want to increase the number of possible contributors to rakudo by enabling non-parrot people to contribute.
Code for different backends will be maintained as directories in Rakudo and NQP, not as branches.
pir:: things will be hidden behind an nqp:: abstraction layer

Attracting contributors

Moritz wants to continue with the "weekly" challenges, but runs out of ideas. Add ideas to http://svn.pugscode.org/pugs/misc/helpnow/README.

We will try to apply patches faster, thus encouraging people who already did the first step.

Documentation

in p5 pod for now, so that people can contribute easily
masak and szabgab expressed interest in working on pod6 tools

My first YAPC - YAPC::EU 2010 in Pisa

Moritz Lenz — Fri, Aug 6 16:37:07 2010

This week I attended my first international Perl conference, the YAPC::EU 2010 in Pisa, Italy. I very much enjoyed it.

I arrived a few days earlier, and spent the time visiting Pisa, talking with old and new friends, and did some collaborative hacking. I especially enjoyed meeting people with whom I had had only contact via Internet so far, and found all of them to be very nice in meat space.

On Tuesday, the day before the conference started officially, a group of Perl 6 hackers met and discussed topics around Perl 6 and Rakudo. I recall talking with Patrick Michaud, Jonathan Worthington, Carl Mäsak, Paweł Murias, Gabor Szabo, smash (I can't spell his full name correctly from memory, sorry for that), and we have a very productive discussion (about 5 hours or so). Notes from the discussion will be published later.

On Wednesday the actual conference started, and there were plenty of very interesting talks, and very amusing lightning talks. I generally like the humor that is widespread in the Perl community.

On Thursday I continued to attend nice and informative talks, and also gave a talk on my own. It was about physical modelling with Perl 6, and in general the feedback was very positive, and somebody even commented that while he didn't understand everything I wrote, it reminded him that it was important to learn Perl 6 now. Win \o/. (There was also some criticism, but from somebody who apparently hasn't read the abstract; the "write-only" meme seems to apply to perl bloggers, not code). You can find the slides to my talk here.

Firday was the day of my departure too - sadly I had to leave after the first talk, and missed the rest of the day, including the closing keynote by mst (I did attend a talk of his) and the traditional auction.

I especially enjoyed...

meeting Patrick, Larry, Gloria, Aaron, Gabor and many others for the first time in real-life
a thorough, high-bandwith discussion with Larry, Jonathan, Patrick and Carl about p6 spec questions, and the Perl 6 discussions mentioned earlier
a lightning talk imagine you're in a data center with no connection to the outside, and you accidentally executed chmod -x chmod. What would you do?
good Italian Pizzas (although I had to wait 50 minutes for one of them)
A talk by Tim Bunce, where he demonstrated database access in Perl 6 both with libraries that do native calls, and through the Blizkost project using the Perl 5 DBI/DBD::SQLite modules
the general relaxed attitude in the Perl community, where people help each other, and don't seem to be easily offended (at least in meat space :)
being surrounded by many other geeks
realizing that several Perl hackers brought their family to the conference

It was an overwhelming experience, and I look forward to my next YAPC!

Currying

Moritz Lenz — Sun, Jul 25 11:17:10 2010

NAME

"Perl 5 to 6" Lesson 28 - Currying

SYNOPSIS

  use v6;
  
  my &f := &substr.assuming('Hello, World');
  say f(0, 2);                # He
  say f(3, 2);                # lo
  say f(7);                   # World
  
  say <a b c>.map: * x 2;     # aabbcc
  say <a b c>.map: *.uc;      # ABC
  for ^10 {
      print <R G B>.[$_ % *]; # RGBRGBRGBR
  }

DESCRIPTION

Currying or partial application is the process of generating a function from another function or method by providing only some of the arguments. This is useful for saving typing, and when you want to pass a callback to another function.

Suppose you want a function that lets you extract substrings from "Hello, World" easily. The classical way of doing that is writing your own function:

  sub f(*@a) {
      substr('Hello, World', |@a)
  }

Currying with `assuming`

Perl 6 provides a method assuming on code objects, which applies the arguments passed to it to the invocant, and returns the partially applied function.

  my &f := &substr.assuming('Hello, World');

Now f(1, 2) is the same as substr('Hello, World', 1, 2).

assuming also works on operators, because operators are just subroutines with weird names. To get a subroutine that adds 2 to whatever number gets passed to it, you could write

  my &add_two := &infix:<+>.assuming(2);

But that's tedious to write, so there's another option.

Currying with the Whatever-Star

  my &add_two := * + 2;
  say add_two(4);         # 6

The asterisk, called Whatever, is a placeholder for an argument, so the whole expression returns a closure. Multiple Whatevers are allowed in a single expression, and create a closure that expects more arguments, by replacing each term * by a formal parameter. So * * 5 + * is equivalent to -> $a, $b { $a * 5 + $b }.

  my $c = * * 5 + *;
  say $c(10, 2);                # 52

Note that the second * is an infix operator, not a term, so it is not subject to Whatever-currying.

The process of lifting an expression with Whatever stars into a closure is driven by syntax, and done at compile time. This means that

  my $star = *;
  my $code = $star + 2

does not construct a closure, but instead dies with a message like

  Can't take numeric value for object of type Whatever

Whatever currying is more versatile than .assuming, because it allows to curry something else than the first argument very easily:

  say  ~(1, 3).map: 'hi' x *    # hi hihihi

This curries the second argument of the string repetition operator infix x, so it returns a closure that, when called with a numeric argument, produces the string hi as often as that argument specifies.

The invocant of a method call can also be Whatever star, so

  say <a b c>.map: *.uc;      # ABC

involves a closure that calls the uc method on its argument.

MOTIVATION

Perl 5 could be used for functional programming, which has been demonstrated in Mark Jason Dominus' book Higher Order Perl.

Perl 6 strives to make it even easier, and thus provides tools to make typical constructs in functional programming easily available. Currying and easy construction of closures is a key to functional programming, and makes it very easy to write transformation for your data, for example together with map or grep.

Common Perl 6 data processing idioms

Moritz Lenz — Thu, Jul 22 15:34:26 2010

NAME

"Perl 5 to 6" Lesson 27 - Common Perl 6 data processing idioms

SYNOPSIS

  # create a hash from a list of keys and values:
  # solution 1: slices
  my %hash; %hash{@keys} = @values;
  # solution 2: meta operators
  my %hash = @keys Z=> @values;

  # create a hash from an array, with
  # true value for each array item:
  my %exists = @keys Z=> 1 xx *;

  # limit a value to a given range, here 0..10.
  my $x = -2;
  say 0 max $x min 10;

  # for debugging: dump the contents of a variable,
  # including its name, to STDERR
  note :$x.perl;

  # sort case-insensitively
  say @list.sort: *.lc;

  # mandatory attributes
  class Something {
      has $.required = die "Attribute 'required' is mandatory";
  }
  Something.new(required => 2); # no error
  Something.new()               # BOOM

DESCRIPTION

Learning the specification of a language is not enough to be productive with it. Rather you need to know how to solve specific problems. Common usage patterns, called idioms, helps you not having to re-invent the wheel every time you're faced with a problem.

So here a some common Perl 6 idioms, dealing with data structures.

Hashes

  # create a hash from a list of keys and values:
  # solution 1: slices
  my %hash; %hash{@keys} = @values;
  # solution 2: meta operators
  my %hash = @keys Z=> @values;

The first solution is the same you'd use in Perl 5: assignment to a slice. The second solution uses the zip operator Z, which joins to list like a zip fastener: 1, 2, 3 Z 10, 20, 30 is 1, 10, 2, 20, 3, 30. The Z=> is a meta operator, which combines zip with => (the Pair construction operator). So 1, 2, 3 Z=> 10, 20, 30 evaluates to 1 => 10, 2 => 20, 3 => 30. Assignment to a hash variable turns that into a Hash.

For existence checks, the values in a hash often doesn't matter, as long as they all evaluate to True in boolean context. In that case, a nice way to initialize the hash from a given array or list of keys is

  my %exists = @keys Z=> 1 xx *;

which uses a lazy, infinite list of 1s on the right-hand side, and relies on the fact that Z ends when the shorter list is exhausted.

Numbers

Sometimes you want to get a number from somewhere, but clip it into a predefined range (for example so that it can act as an array index).

In Perl 5 you often end up with things like $a = $b > $upper ? $upper : $b, and another conditional for the lower limit. With the max and min infix operators, that simplifies considerably to

  my $in-range = $lower max $x min $upper;

because $lower max $x returns the larger of the two numbers, and thus clipping to the lower end of the range.

Since min and max are infix operators, you can also clip infix:

 $x max= 0;
 $x min= 10;

Debugging

Perl 5 has Data::Dumper, Perl 6 objects have the .perl method. Both generate code that reproduces the original data structure as faithfully as possible.

:$var generates a Pair ("colonpair"), using the variable name as key (but with sigil stripped). So it's the same as var => $var. note() writes to the standard error stream, appending a newline. So note :$var.perl is quick way of obtaining the value of a variable for debugging; purposes, along with its name.

Sorting

Like in Perl 5, the sort built-in can take a function that compares two values, and then sorts according to that comparison. Unlike Perl 5, it's a bit smarter, and automatically does a transformation for you if the function takes only one argument.

In general, if you want to compare by a transformed value, in Perl 5 you can do:

    # WARNING: Perl 5 code ahead
    my @sorted = sort { transform($a) cmp transform($b) } @values;

    # or the so-called Schwartzian Transform:
    my @sorted = map { $_->[1] }
                 sort { $a->[0] cmp $b->[0] }
                 map { [transform($_), $_] }
                 @values

The former solution requires repetitive typing of the transformation, and executes it for each comparison. The second solution avoids that by storing the transformed value along with the original value, but it's quite a bit of code to write.

Perl 6 automates the second solution (and a bit more efficient than the naiive Schwartzian transform, by avoiding an array for each value) when the transformation function has arity one, ie accepts one argument only:

    my @sorted = sort &transform, @values;

Mandatory Attributes

The typical way to enforce the presence of an attribute is to check its presence in the constructor - or in all constructors, if there are many.

That works in Perl 6 too, but it's easier and safer to require the presence at the level of each attribute:

    has $.attr = die "'attr' is mandatory";

This exploits the default value mechanism. When a value is supplied, the code for generating the default value is never executed, and the die never triggers. If any constructor fails to set it, an exception is thrown.

MOTIVATION

N/A

Gabor: Keep going

Moritz Lenz — Mon, Jul 19 22:36:05 2010

After reading this blog post, I just want to say: keep going.

Gabor does really awesome things for Perl (and especially for Perl 6, which I tend to notice more): beginner's tutorials, screencasts, training courses, writes an IDE and so on. If his primary interest was his own success, his priorities would be quite different.

I hope that Gabor doesn't pay too much attention to the hostilities from parts of the Perl community, and I wish him and us all the best for his current project.

This Week's Contribution to Perl 6 Week 9: Implement Hash.pick for Rakudo

Moritz Lenz — Tue, Jul 13 15:22:47 2010

For this week's contribution to Perl 6 we ask you to implement the Hash.pick method (which does a weighted random selection) for Rakudo.

(Introduction to this series of challenges)

Background

In Perl 6 the List class has a method called pick, which randomly selects one item from a list. It has a few more options too:

<a b c>.pick;       # pick one random element
<a b c>.pick(2);    # pick two distinct, random elements
<a b c>.pick(2, :replace); # pick two random elements, it's ok
                           # if they are the same
<a b c>.pick(*);    # return a random permutation of the elements
<a b c>.pick(*, :replace); # infinite, random stream of elements

This is already implemented through several multi methods in Rakudo.

Now the specification describes such a method for hashes too (actually it talks about Bags, but Rakudo doesn't have Bags yet. Pretend it says "Hash" instead). It assumes that each value in the hash is numeric, and that the value is a weight that determines the probability of picking one value. For example

{a => 1, b => 2}.pick;  # returns 'a' with probability 1/3
                        # and 'b' with probability 2/3

{a => 1, b => 2}.pick(*);  # <a b b> with probability 1/3
                           # <b a b> with probability 1/3
                           # <b b a> with probability 1/3
{a => 1, b => 0.5}.pick(*) # dies, because the weights aren't all integers
{a => 1, b => 0.5}.pick(*, :replace)  # ok

What you can do

Implement Hash.pick. It's ok if your patch doesn't cover all cases. It would be nice if it supported non-integer weights.

Hint: this could be done by storing a list of accumulated weights, and a list of keys.

{a => 1, b => 2.5, c => 1}

# could translate to 
my @keys = ('a', 'b', 'c');
my @accumulated_weights = (1, 3.5, 4.5);

# now pick a random number between 0 and 4.5,
# find the next-highest index in @accumulated_weights
# with a binary search, and then use that to obtain the key.

Of course other schemes are fine too.

Second hint: because it takes quite some time to recompile Rakudo, it is probably easier to implement the actual logic in a function in a normal source file first, and only later move it into src/core/Hash.pm.

Submission

Please submit your source code to the perl6-compiler@perl.org mailing list (and put moritz@faui2k3.org on CC, because the mailing list sometimes lack quite a bit).

Update: there's one submission on the perl6-compiler mailing list already, which looks pretty good.

Want to write shiny SVG graphics with Perl 6? Port Scruffy!

Moritz Lenz — Mon, Jul 12 15:43:27 2010

First let my apologize for waiting so long to come up with a new "weekly" Perl 6 challenge - I'm running out of ideas, and the one I have left needs more time to prepare.

Instead I want to motivate you to help porting the ruby scruffy charting library to Perl 6. It can generate shiny SVG graphics (they are currently broken on their main website, hence the waybackmachine link).

There's already an initial version Perl 6 port called "tufte", but it's not running yet. It needs your help. If you know a little ruby, and want to learn some more Perl 6, join #perl6, ask for a commit bit, and translate some ruby code into Perl 6. And in the end you'll be rewarded with nice SVG charts :-).

This Week's Contribution to Perl 6 Week 8: Implement $*ARGFILES for Rakudo

Moritz Lenz — Tue, Jun 29 00:00:00 2010

For this week's contribution to Perl 6 we ask you to implement the $*ARGFILES special variable (and underlying object) for Rakudo.

(Introduction to this series of challenges)

Background

In Perl 5, there is a "magic" way to iterate over input: while (my $line = <>) { ... }. This reads from standard input if no command line arguments were provided. If there are command line arguments, they are taken as file names, and <> iterates over the contents of these files.

In Perl 6 this magic is performed with the $*ARGFILES special variable. Please implement it!

To do that, you have to understand how file reading works in Perl 6. Once you have a file handle, there are two ways to read lines: either by calling $handle.get, which returns just one line, or by calling $handle.lines, which returns a (lazy) list of all the remaining lines.

You can obtain the command line arguments from @*ARGS, open a file with my $handle = open $filename; for reading; And finally you can use lines('filename') to read all lines from a file.

What you can do

Implement a backend class for $*ARGFILES. You can do that in normal Perl 6 code, no need to change the actual compiler. Once that's done, we will plug it into Rakudo. Your code might look like this:

class IO::ArgFiles {
    has @!filenames;
    method get() { ... }
    method lines() { ... }
    method filename() {
        # return the current filename, or '-' if standard input
        ...
    }
}
my $*ARGFILES = IO::ArgFiles.new(filenames => @*ARGS);
.say for $*ARGFILES.lines();

(Of course this is only a rough skeleton, you might need to change some details, or add some things).

Submission

Please submit your source code to the perl6-compiler@perl.org mailing list (and put moritz@faui2k3.org on CC, because the mailing list sometimes lack quite a bit).

Update:: there have been two submissions, and a mixture of both has been applied.