From knight@baldmt.com Fri Aug 20 00:46:12 1999 -0400 Date: Fri, 20 Aug 1999 00:46:09 -0400 (EDT) From: Steven Knight X-Sender: knight@ceirt.pair.com To: jderick@mail.utexas.edu cc: bob_sidebotham@yahoo.com, Rajesh Vaidheeswarran Subject: Re: Experimental Parallel Cons In-Reply-To: <19990819173002.A7573@cs2873-103.austin.rr.com> Message-ID: MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-1605496331-935124369=:15729" Status: RO X-Status: X-Keywords: X-UID: 1 --0-1605496331-935124369=:15729 Content-Type: TEXT/PLAIN; charset=US-ASCII On Thu, 19 Aug 1999 jderick@mail.utexas.edu wrote: > I'm working on a build system with about 7500 source files, and we've been > trying to find ways to keep the build time low. One of the biggest gains > we've been able to identify so far is forking multiple jobs on > multiprocessor machines. I've made some modifications to cons to allow a > Make-like -j flag to be passed in to specify the max number of jobs to be > forked. The modifications are experimental quality, and have not been > thoroughly tested, but I've built some medium sized test projects with > it. > > Cons did not seem particularly well suited for the modification, and as a > result the changes I've made are a bit hackish. For starters, the changes > require that only highest-level targets be specified on the command line > (cons . will most likely not take advantage of parallelism and might give > incorrect results). I've also only made the changes for unix. > > I'd be interested to hear any comments on the modifications, as well as > anyone else's attempts at increasing build speed. jderick-- As Rajesh noted, I've been working on a parallel cons for a while now. (No real activity for the last two months or so, though.) This is based on some work that Bob Sidebotham started a few years ago. You're exactly right that Cons is not currently well-suited for parallelism--one big reason it hasn't happened sooner. Bob's changes took a step in that direction, isolating job creation and monitoring in a separate class. I went a little further and reworked Bob's job class to isolate the differences for Win32 and Unix systems. (As far as I can tell, you can't actually support parallel builds on Win32, but I'm not an NT person.) Bob's parallel-build architecture enumerated the directory descendents of the specified targets in-order, not requiring that the parallel targets be specified. Unfortunately, it ran into the same problem that I think yours does (if I understand you correctly above) in that dependency chains are still single-threaded. Given cons -j4 building two libraries that each have two .c files, I want all four .o files compiled simultaneously, not just the two libraries in parallel--and this should happen whether the command line specifies "." or "lib1 lib2". I've taken some steps towards this by pushing dependency enumeration into subclass methods, so that we can determine most of the build order up-front and not right when the build is about to take place. This works fairly well but loses when it comes to dynamic dependencies--for example, a dependency that you can't know about until you scan a newly-generated .c file. For a really fully-featured parallel build, I also think we eventually need to let the Cons environment control the parallelism: a flag that says, "This build must proceed in the foreground, wait for all outstanding jobs to finish first," and another flag that says, "Only one build in this environment can take place at a time." The other important criteria is that we don't want this to slow down non-parallel builds. Right now, the job classes seem to slow things down more than wanted, but my emphasis is on getting it right first, and then we'll figure out what we have to optimize. I have five test cases that I'm working against. If it's all right with you, I'd like to throw my current draft version and the tests your way. If you have time and inclination, let's stare at each other's code for awhile and see if we can't merge the best ideas from both. On the assumption that this is all right with you, I'm attaching a gzip'ed tar file. (The cons script itself is very much an in-progress development version, with a bunch of commented-out stuff in places where I was in the process of moving functionality into other packages.) If you've gotten any interest from others in helping out (i.e., more than just, "Hey, let me know when it's working"), let me know and let's get them on board, too. --SK [ATTACHMENT DELETED] From knight@baldmt.com Fri Aug 20 14:24:39 1999 -0400 Received: from cs2873-103.austin.rr.com (IDENT:erickson@cs2873-103.austin.rr.com [24.28.73.103]) by ceirt.pair.com (8.9.1/8.6.12) with ESMTP id OAA29152 for ; Fri, 20 Aug 1999 14:24:34 -0400 (EDT) X-Envelope-To: Received: (from erickson@localhost) by cs2873-103.austin.rr.com (8.8.7/8.8.7) id NAA07977 for knight@baldmt.com; Fri, 20 Aug 1999 13:27:51 -0500 Message-ID: <19990820132749.A7959@cs2873-103.austin.rr.com> Date: Fri, 20 Aug 1999 13:27:49 -0500 From: John Erickson To: Steven Knight Subject: Re: Experimental Parallel Cons References: <19990819173002.A7573@cs2873-103.austin.rr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: ; from Steven Knight on Fri, Aug 20, 1999 at 12:46:09AM -0400 Status: RO X-Status: A X-Keywords: X-UID: 3 Steven, I took a quick look at the approach you are taking, and attempted to build the prototype I've been running against with it. I like the idea of finding all dependencies before the build process starts (what it seems like you are doing with enumerate), but as you have mentioned, it seems to have problems with dynamic dependencies. Unfortunately, the project I'm working on involves many dynamic dependencies so it is something I could not afford to neglect. Although I think it would be ideal to separate the dependency generation process from the actual building process (IE build a complete DAG before beginning any jobs), I'm not sure it is possible. I agree that full parallelism would be ideal, but I think that what I have settled for gives nearly identical results. It builds all siblings in parallel, thus if you have two libraries with two .o each, the two .o files that correspond to the same library will be compiled in parallel. In practice, this results in good utilization of processors because there are almost no targets in my project that have less than four (and usually significantly more) dependents. Being able to control parallelism with the environment sounds interesting, what would it be useful for? Of course, it would be ideal to merge parallelism into Cons without affecting non-parallel builds, but I don't think we need to worry too much about it. Worst case, we could have duplicate code inside of Cons or a separate parallel Cons could be maintained. Overall, I think my approach is much more quick and dirty, but it seems to address the important issue of dynamic dependencies which I'm not sure can be done any other way. I'd be interested to hear your thoughts. John > As Rajesh noted, I've been working on a parallel cons for a while > now. (No real activity for the last two months or so, though.) > This is based on some work that Bob Sidebotham started a few years > ago. > > You're exactly right that Cons is not currently well-suited for > parallelism--one big reason it hasn't happened sooner. Bob's > changes took a step in that direction, isolating job creation and > monitoring in a separate class. I went a little further and reworked > Bob's job class to isolate the differences for Win32 and Unix > systems. (As far as I can tell, you can't actually support parallel > builds on Win32, but I'm not an NT person.) > > Bob's parallel-build architecture enumerated the directory descendents > of the specified targets in-order, not requiring that the parallel > targets be specified. Unfortunately, it ran into the same problem > that I think yours does (if I understand you correctly above) in > that dependency chains are still single-threaded. Given cons -j4 > building two libraries that each have two .c files, I want all four > ..o files compiled simultaneously, not just the two libraries in > parallel--and this should happen whether the command line specifies > "." or "lib1 lib2". > > I've taken some steps towards this by pushing dependency enumeration > into subclass methods, so that we can determine most of the build > order up-front and not right when the build is about to take place. > This works fairly well but loses when it comes to dynamic > dependencies--for example, a dependency that you can't know about > until you scan a newly-generated .c file. > > For a really fully-featured parallel build, I also think we eventually > need to let the Cons environment control the parallelism: a flag > that says, "This build must proceed in the foreground, wait for > all outstanding jobs to finish first," and another flag that says, > "Only one build in this environment can take place at a time." > > The other important criteria is that we don't want this to slow > down non-parallel builds. Right now, the job classes seem to slow > things down more than wanted, but my emphasis is on getting it > right first, and then we'll figure out what we have to optimize. > > I have five test cases that I'm working against. If it's all right > with you, I'd like to throw my current draft version and the tests > your way. If you have time and inclination, let's stare at each > other's code for awhile and see if we can't merge the best ideas > from both. On the assumption that this is all right with you, I'm > attaching a gzip'ed tar file. (The cons script itself is very much > an in-progress development version, with a bunch of commented-out > stuff in places where I was in the process of moving functionality > into other packages.) > > If you've gotten any interest from others in helping out (i.e., > more than just, "Hey, let me know when it's working"), let me know > and let's get them on board, too. > > --SK From knight@baldmt.com Fri Aug 20 15:15:49 1999 -0400 Date: Fri, 20 Aug 1999 15:15:49 -0400 (EDT) From: Steven Knight X-Sender: knight@ceirt.pair.com To: John Erickson Subject: Re: Experimental Parallel Cons In-Reply-To: <19990820132749.A7959@cs2873-103.austin.rr.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO X-Status: X-Keywords: X-UID: 2 On Fri, 20 Aug 1999, John Erickson wrote: > I took a quick look at the approach you are taking, and attempted to build > the prototype I've been running against with it. I'd have been really surprised if it had worked... :-) For any real-world project, that is. It works well enough if there aren't any dynamic dependencies, but that isn't the real world... > I like the idea of finding all dependencies before the build process > starts (what it seems like you are doing with enumerate), but as you have > mentioned, it seems to have problems with dynamic dependencies. > Unfortunately, the project I'm working on involves many dynamic > dependencies so it is something I could not afford to neglect. Although I > think it would be ideal to separate the dependency generation process from > the actual building process (IE build a complete DAG before beginning any > jobs), I'm not sure it is possible. It isn't. There are some dependencies that simply don't exist for you to find until the source file is generated. > I agree that full parallelism would be ideal, but I think that what I have > settled for gives nearly identical results. It builds all siblings in > parallel, thus if you have two libraries with two .o each, the two .o > files that correspond to the same library will be compiled in parallel. > In practice, this results in good utilization of processors because there > are almost no targets in my project that have less than four (and usually > significantly more) dependents. Good point. It's possible that I'm trying to over-engineer things, trying to squeeze those few extra cycles by having, say lib1/[abc].c and lib2/a.c going simultaneously with -j4 in force. There is also the issue, though, of making it independent of the way the arguments are specified on the command line. > Being able to control parallelism with the environment sounds > interesting, what would it be useful for? On uniprocessor systems, small -j values can still reduce wall-clock compile time. Some build utilities consume more resources than others, though: the linker typically takes up a lot more memory than individual compilations. Being able to tweak the build process to compile in parallel but link serially can be a win because it avoids swapping out a big link job for a smaller compile. > Of course, it would be ideal to merge parallelism into Cons without > affecting non-parallel builds, but I don't think we need to worry too much > about it. I tend to agree with you. Cons is pretty efficient, so I don't think a few extra percent on non-parallel builds would kill anyone. This is apparently a pretty big issue to Bob and Rajesh and the folks at Fore, though. Since they originated the code, I've been trying to respect that requirement. > Worst case, we could have duplicate code inside of Cons or a > separate parallel Cons could be maintained. Actually, I should go back to the approach Bob started with. He had separate job::parallel and job::serial sub-classes, and you used the right class for your -j value. I merged those into a single job class because I was trying to be clever and thought, "Hey, single-threading is just parallel job control with N == 1!" I should have left well enough alone. > Overall, I think my approach is much more quick and dirty, but it seems to > address the important issue of dynamic dependencies which I'm not sure can > be done any other way. I agree with you that trying to enumerate the DAG ahead of time is a dead end. But maybe it's a step towards another direction. As Bob pointed out in his previous email, the real barrier is that he put the dependency recursive descent inside the build step. I had started breaking that apart as part of trying to enumerate the DAG. Perhaps reapplying that separation to later in the build process will prove fruitful... And of course, there's Bob's suggestion to multi-thread the thing. Your thoughts on that? > I'd be interested to hear your thoughts. And yours. It's great to have someone to bounce ideas off of. Neither Bob nor Rajesh has a lot of cycles these days for it. --SK From knight@baldmt.com Tue Aug 24 23:35:33 1999 -0400 Received: from resnet-50-73.dorm.utexas.edu (IDENT:erickson@resnet-50-73.dorm.utexas.edu [129.116.50.73]) by ceirt.pair.com (8.9.1/8.6.12) with ESMTP id XAA13702 for ; Tue, 24 Aug 1999 23:35:33 -0400 (EDT) X-Envelope-To: Received: (from erickson@localhost) by resnet-50-73.dorm.utexas.edu (8.8.7/8.8.7) id WAA05750 for knight@baldmt.com; Tue, 24 Aug 1999 22:37:19 -0500 Message-ID: <19990824223718.B693@resnet-50-73.dorm.utexas.edu> Date: Tue, 24 Aug 1999 22:37:18 -0500 From: John Erickson To: Steven Knight Subject: Re: Experimental Parallel Cons References: <19990820132749.A7959@cs2873-103.austin.rr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: ; from Steven Knight on Fri, Aug 20, 1999 at 03:15:49PM -0400 Status: RO X-Status: A X-Keywords: X-UID: 5 I like the idea of using perl threads. It seems like it would address all of the issues that we've talked about. I'm not very familiar with them, what are the current limitations? You mentioned earlier that only a subset of perl ports supported them, is there a list somewhere of the platforms supported? Bob mentioned earlier that a mutex would be required for each target. Do you think that any other mutexes might be required? On Fri, Aug 20, 1999 at 03:15:49PM -0400, Steven Knight wrote: > On Fri, 20 Aug 1999, John Erickson wrote: > > I took a quick look at the approach you are taking, and attempted to build > > the prototype I've been running against with it. > > I'd have been really surprised if it had worked... :-) > > For any real-world project, that is. It works well enough if there > aren't any dynamic dependencies, but that isn't the real world... > > > I like the idea of finding all dependencies before the build process > > starts (what it seems like you are doing with enumerate), but as you have > > mentioned, it seems to have problems with dynamic dependencies. > > Unfortunately, the project I'm working on involves many dynamic > > dependencies so it is something I could not afford to neglect. Although I > > think it would be ideal to separate the dependency generation process from > > the actual building process (IE build a complete DAG before beginning any > > jobs), I'm not sure it is possible. > > It isn't. There are some dependencies that simply don't exist for > you to find until the source file is generated. > > > I agree that full parallelism would be ideal, but I think that what I have > > settled for gives nearly identical results. It builds all siblings in > > parallel, thus if you have two libraries with two .o each, the two .o > > files that correspond to the same library will be compiled in parallel. > > In practice, this results in good utilization of processors because there > > are almost no targets in my project that have less than four (and usually > > significantly more) dependents. > > Good point. It's possible that I'm trying to over-engineer things, > trying to squeeze those few extra cycles by having, say lib1/[abc].c > and lib2/a.c going simultaneously with -j4 in force. > > There is also the issue, though, of making it independent of the > way the arguments are specified on the command line. > > > Being able to control parallelism with the environment sounds > > interesting, what would it be useful for? > > On uniprocessor systems, small -j values can still reduce wall-clock > compile time. Some build utilities consume more resources than > others, though: the linker typically takes up a lot more memory > than individual compilations. Being able to tweak the build process > to compile in parallel but link serially can be a win because it > avoids swapping out a big link job for a smaller compile. > > > Of course, it would be ideal to merge parallelism into Cons without > > affecting non-parallel builds, but I don't think we need to worry too much > > about it. > > I tend to agree with you. Cons is pretty efficient, so I don't > think a few extra percent on non-parallel builds would kill anyone. > This is apparently a pretty big issue to Bob and Rajesh and the > folks at Fore, though. Since they originated the code, I've been > trying to respect that requirement. > > > Worst case, we could have duplicate code inside of Cons or a > > separate parallel Cons could be maintained. > > Actually, I should go back to the approach Bob started with. He > had separate job::parallel and job::serial sub-classes, and you > used the right class for your -j value. I merged those into a > single job class because I was trying to be clever and thought, > "Hey, single-threading is just parallel job control with N == 1!" > I should have left well enough alone. > > > Overall, I think my approach is much more quick and dirty, but it seems to > > address the important issue of dynamic dependencies which I'm not sure can > > be done any other way. > > I agree with you that trying to enumerate the DAG ahead of time is > a dead end. But maybe it's a step towards another direction. As > Bob pointed out in his previous email, the real barrier is that he > put the dependency recursive descent inside the build step. I had > started breaking that apart as part of trying to enumerate the DAG. > Perhaps reapplying that separation to later in the build process > will prove fruitful... > > And of course, there's Bob's suggestion to multi-thread the thing. > Your thoughts on that? > > > I'd be interested to hear your thoughts. > > And yours. It's great to have someone to bounce ideas off of. > Neither Bob nor Rajesh has a lot of cycles these days for it. > > --SK > From knight@baldmt.com Wed Aug 25 15:57:46 1999 -0400 Date: Wed, 25 Aug 1999 15:57:46 -0400 (EDT) From: Steven Knight X-Sender: knight@ceirt.pair.com To: John Erickson Subject: Re: Experimental Parallel Cons In-Reply-To: <19990824223718.B693@resnet-50-73.dorm.utexas.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO X-Status: X-Keywords: X-UID: 4 Hey John-- Thanks for the reply. > I like the idea of using perl threads. It seems like it would address all > of the issues that we've talked about. Conceptually, I agree. Threads are, technically, the perfect choice. And I think it would be a lot of fun. I'd sure learn a lot, which is always a cool thing. > I'm not very familiar with them, > what are the current limitations? I don't consider myself a multi-threading guru, but as far as I can tell, the Perl Thread package looks pretty fully-featured. The biggest hassle wouldn't be the thread management itself, it would probably be making sure that everything else in Cons is thread-safe. A lot of end-cases to test for... > You mentioned earlier that only a > subset of perl ports supported them, is there a list somewhere of the > platforms supported? I haven't found such a list, but this is my biggest concern. Neither RedHat 6.0 nor FreeBSD 3.2 ships a thread-enabled Perl. Considering that those are the two biggest distributions in their respective camps, if even one of them shipped native support for Perl Threads, I'd be a lot more comfortable. My own experience is that trying to assemble the right pieces on Linux to enable thread support is non-trivial, even if you know what you're doing. I don't know about BSD. Based on those data points, though, to the extent that we make a parallel Cons dependent on threads, it feels to me like we're cutting out everyone except those who have the technical knowledge and time to configure a fairly technically complex feature into their system. Maybe this is something that we should poll the mailing list about. If there's consensus that it's better to do a parallel Cons "right" by using threads, and then wait for the world's thread support to catch up, then so be it. > Bob mentioned earlier that a mutex would be required for each target. Do > you think that any other mutexes might be required? I'd guess an additional mutex per environment, at least if we go ahead with the idea of letting an environment hook control parallelism. --SK From knight@baldmt.com Wed Aug 25 17:02:44 1999 -0400 Date: Wed, 25 Aug 1999 17:02:44 -0400 (EDT) From: Steven Knight X-Sender: knight@ceirt.pair.com To: John Erickson Subject: Re: Experimental Parallel Cons In-Reply-To: <19990824223718.B693@resnet-50-73.dorm.utexas.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO X-Status: X-Keywords: X-UID: 6 John-- Well, I was doing some more research, and I found the following from a deja.com search. Here's no less than Tom Christiansen himself with a strong vote against threads: >>Forum: comp.lang.perl.misc >>Thread: a thread on threads >>Message 13 of 48 Subject: Re: a thread on threads Date: 1999/06/14 Author: Tom Christiansen [courtesy cc of this posting mailed to cited author] In comp.lang.perl.misc, Greg Bartels writes: :threads were supposed to be "lightweight" which implied :low overhead, so I can have a lot of threads without sucking :up resources, etc. Threads are not lightweight, least of all in Perl. Your Perl program will probably take twice the time to run if compiled with threading than it would otherwise. Remember also that processes in Unix are much faster than are threads in Microsoft. Yes, that's comparing apples with apple trees, but that's how inept Microsoft is at operating system design. I'm afraid that threads are often a problem looking for a solution, not the other way around. --tom -- "Umm, square root of two? Ouch!" --The guy who blew a hole in the Pythagoreans' assertion that all numbers can be represented as a ratio of two integers, so they killed him There are some other postings in this thread that make a compelling case that fork() and good IPC make for a far more efficient solution than threads. Christiansen reposts a really nice, long explanation of the tradeoffs from Chris Torek about ten messages earlier. Of course, there are some strongs opinions voiced in the other direction, too... There are additional postings making the point that Perl Threads haven't been declared stable, so there's not a strong case for making an architectural change towards using them. Check out this thread (no pun intended) and let me know what you think... --SK From knight@baldmt.com Thu Aug 26 16:01:44 1999 -0400 Received: from resnet-50-73.dorm.utexas.edu (IDENT:erickson@resnet-50-73.dorm.utexas.edu [129.116.50.73]) by ceirt.pair.com (8.9.1/8.6.12) with ESMTP id QAA03802 for ; Thu, 26 Aug 1999 16:01:40 -0400 (EDT) X-Envelope-To: Received: (from erickson@localhost) by resnet-50-73.dorm.utexas.edu (8.8.7/8.8.7) id PAA06439 for knight@baldmt.com; Thu, 26 Aug 1999 15:03:03 -0500 Message-ID: <19990826150302.A6380@resnet-50-73.dorm.utexas.edu> Date: Thu, 26 Aug 1999 15:03:02 -0500 From: John Erickson To: Steven Knight Subject: Re: Experimental Parallel Cons References: <19990824223718.B693@resnet-50-73.dorm.utexas.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: ; from Steven Knight on Wed, Aug 25, 1999 at 05:02:44PM -0400 Status: RO X-Status: A X-Keywords: X-UID: 8 Steve, I checked into this thread and read a few other posts on perl threads. There doesn't seem to be much hard evidence against using perl threads (other than speed, which may be a reasonable concern for cons since that is why we are doing threading in the first place). However, I did get the sense that they aren't quite ready for production. It sounded like we could be asking for trouble trying to use them this early in their development. This especially concerns me because the project I'm working on needs to run on several platforms (Linux, NT, Solaris, AIX, HPUX, and a few others). I'm not sure what the best way to procede from here is. Right now, I'm thinking about just making due with my current solution until threads become more stable. John On Wed, Aug 25, 1999 at 05:02:44PM -0400, Steven Knight wrote: > John-- > > Well, I was doing some more research, and I found the following > from a deja.com search. Here's no less than Tom Christiansen > himself with a strong vote against threads: > > >>Forum: comp.lang.perl.misc > >>Thread: a thread on threads > >>Message 13 of 48 > > Subject: Re: a thread on threads > Date: 1999/06/14 > Author: Tom Christiansen > > > [courtesy cc of this posting mailed to cited author] > > In comp.lang.perl.misc, Greg Bartels writes: > :threads were supposed to be "lightweight" which implied > :low overhead, so I can have a lot of threads without sucking > :up resources, etc. > > Threads are not lightweight, least of all in Perl. Your > Perl program will probably take twice the time to run if > compiled with threading than it would otherwise. Remember > also that processes in Unix are much faster than are threads > in Microsoft. Yes, that's comparing apples with apple > trees, but that's how inept Microsoft is at operating system > design. > > I'm afraid that threads are often a problem looking for a > solution, not the other way around. > > --tom > -- > "Umm, square root of two? Ouch!" > --The guy who blew a hole in the Pythagoreans' assertion that all > numbers can > be represented as a ratio of two integers, so they killed him > > There are some other postings in this thread that make a compelling > case that fork() and good IPC make for a far more efficient solution > than threads. Christiansen reposts a really nice, long explanation > of the tradeoffs from Chris Torek about ten messages earlier. > > Of course, there are some strongs opinions voiced in the other > direction, too... > > There are additional postings making the point that Perl Threads > haven't been declared stable, so there's not a strong case for > making an architectural change towards using them. > > Check out this thread (no pun intended) and let me know what you > think... > > --SK > From knight@baldmt.com Mon Aug 30 22:49:42 1999 -0400 Date: Mon, 30 Aug 1999 22:49:42 -0400 (EDT) From: Steven Knight X-Sender: knight@ceirt.pair.com To: John Erickson Subject: Re: Experimental Parallel Cons In-Reply-To: <19990826150302.A6380@resnet-50-73.dorm.utexas.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO X-Status: X-Keywords: X-UID: 7 On Thu, 26 Aug 1999, John Erickson wrote: > I checked into this thread and read a few other posts on perl threads. > There doesn't seem to be much hard evidence against using perl threads > (other than speed, which may be a reasonable concern for cons since that > is why we are doing threading in the first place). Interesting. I think I found the arguments more compelling than you did, but if by "hard evidence," you mean actual measurements, then I have to agree with you. Now that you mention it, though, I also don't know if the slowdown with Perl threads is system time or wall-clock execution time. I'd gladly have Cons chew more CPU cycles if it meant the overall build was more efficient because we're running N compilations simultaneously. > However, I did get the > sense that they aren't quite ready for production. It sounded like we > could be asking for trouble trying to use them this early in their > development. I agree. > This especially concerns me because the project I'm working > on needs to run on several platforms (Linux, NT, Solaris, AIX, HPUX, and a > few others). > > I'm not sure what the best way to procede from here is. Right now, I'm > thinking about just making due with my current solution until threads > become more stable. Absolutely. Even if we end up with some other solution (threaded or not), your parallel version of Cons is the closest to working of any out there. If it helps people be more productive, then by all means use and share it. I'd only request that you make sure people know that it's experimental and that the implementation (and possibly the interface) might change in the production version--whenever that may come about. --SK