This page is a central location
for information about past (and future)
attempts at creating a version of
software construction utility,
that can build multiple targets in parallel
(the sort of functionality supported by
make -j option).
This includes copies of the relevant code,
and editorial comments
(see the bottom of the page).
I hope that sharing this information
will provide the right insight or inspiration
for someone to complete the job.
There have been (to my knowledge) four significant attempts at creating a parallel version of Cons. None of these versions has yet provided a complete solution. So that we have some common terminology, I'm calling them, in order, multi, cons-p, pcons, and pcons-2.3.0:
This is Bob Sidebotham's original attempt
at parallelizing Cons.
Bob did most of this work in 1996,
and then set it aside.
||multi.tar.gz||Bob's original tar file he sent me (and another person on the cons-discuss mailing list) containing all his work on parallelizing Cons.|
|multi.email||Bob's original email explaining what's in the multi.tar.gz file.|
|cons.multi||Bob's parallel version of Cons, already extracted from the tar file for easy viewing. (This is slightly different from the version in multi.tar.gz; I pasted back in an enumerate subroutine that was left out.)|
|This is my attempt at picking up and finishing Bob's work on multi. This took place in early 1999 (February through April). I took a lot of wrong turns, among them: merging Bob's two job classes into one (I thought it would be clever to handle serial jobs as an N=1 case of parallelism, but it's much more efficient to keep them separate); breaking signature handling; spending too much energy on a complicated scheme to enumerate dependencies up-front and still handle dynamic dependencies... I added some documentation, though, and extended Bob's job classes to handle multi-line commands.||cons-p.tar.gz||An archive of the last three versions of cons-p, plus the test cases (shell scripts in the manner of versions 2.X of the cons-test suite) for which I was trying to get cons-p to work.|
|cons-p.email||Some fairly lengthy email exchanges among Bob, Rajesh and myself as I struggled to get cons-p working.|
This is John Erickson's parallel Cons
from August 1999.
John took a different approach from Bob's,
spawning parallel jobs within
the recursive dependency descent for a target.
This leads to an elegantly concise set of changes
that don't give ideal parallelism
(you can't keep N compiles running
simultaneously if they're
spread across multiple targets),
but do an excellent job of
keeping each target as parallel as possible.
The only other drawback is that
John didn't keep multi-line commands working;
you had to separate commands with
||pcons.announce||John's original announcement to the Cons mailing list about pcons, with his attached patch.|
|pcons.diff||John's original patch, as attached to his announcement mail.|
|pcons||John's patch applied to Cons 1.6b1 (the current version at the time).|
|pcons.email||An email exchange between John and myself about pcons, differences between his and Bob's approaches, and how we might take it to the next step. We also discussed the idea of using threads.|
This is Thomas Gleerup's
merging and updating of John Erickson's pcons
change with Cons version 2.3.0.
This has the same minor drawback as Erickson's
(very useful but slightly imperfect parallelism),
and half-supports multi-line commands
by automatically converting
newline separators into
||pcons-2.3.0.announce||Thomas's original announcement to the Cons mailing list about his work.|
Thomas' version of pcons,
based on Cons 2.3.0.
Note that this is actually slightly
updated from the version in his
adding the automatic
After re-familiarizing myself with these attempts, I think each has some good ideas that could be pulled together into a good implementation:
Feel free to send me something at firstname.lastname@example.org.
||Last modified: 8 November 2001|