After about three weeks of fighting with an optimization problem at work (including an interesting detour into Perl guts), we have finally prevailed. The processing didn't run out of memory and die after the first few iterations.

The solution was to re-think the algorithm yet again. Our first few attempts helped but didn't get us far enough. The most recent change, however, has kept us far enough under the memory limit to ensure that it should run to completion. I'll know for sure in the morning.

It'll be interesting to see what happens when the Perl code is converted to Java. Will it be faster or slower? Use more or less memory? How many more or fewer lines of code will it require?

Time will tell.

Posted by jzawodn at March 03, 2003 07:43 PM

Reader Comments
# George Schlossnagle said:

After all the discussion in the previous posting, it would be interesting to hear more about the refactoring. If you're at liberty to, of course.

on March 3, 2003 09:03 PM
# Dan said:

I, for one, would love to hear what the end results were on the Java rewrite. (And I'm still tempted to write a lower-memory-usage hash implementation for those cases where you only really need an integer, rather than a full perl scalar attached, but that's a separate thing)

on March 4, 2003 07:30 AM
# Morgan Deters said:

Well, in general, expect it to be slower and take more memory for hashes. Then again, that test doesn't try hashes as large as those you're using.

And judging by the usual ratio of 1 : 10000 sloc for mapping from perl into almost any other imperative language, expect to write more code, too. (Okay, maybe that's a bit of an exaggeration...)

on March 5, 2003 10:28 AM
Disclaimer: The opinions expressed here are mine and mine alone. My current, past, or previous employers are not responsible for what I write here, the comments left by others, or the photos I may share. If you have questions, please contact me. Also, I am not a journalist or reporter. Don't "pitch" me.


Privacy: I do not share or publish the email addresses or IP addresses of anyone posting a comment here without consent. However, I do reserve the right to remove comments that are spammy, off-topic, or otherwise unsuitable based on my comment policy. In a few cases, I may leave spammy comments but remove any URLs they contain.