This is a little funny. Yahoo got slashdotted today. It was because of Michael Radwin's PHPCon 2002 talk on Yahoo adopting PHP. The funny thing is that it held up just fine--served by a single FreeBSD server running Apache. The hardware was nothing special. So, why is it that when most sites get "featured" on Slashdot, they crumble?
They generally have two fatal flaws: (1) not enough bandwidth, and (2) dynamic content. We're fortunate enough to have some excellent network connectivity, so we can handle a lot of traffic. The fact that public.yahoo.com was serving static files, no PHP or anything fancy, meant that the CPU had time to spare. During the peak of traffic, the CPU was still over 50% idle much of the time. Running a tail -f against the apache log was quite amusing. It was scrolling really, really fast.
The usual stupidity
Reading over the comments, I noticed that almost everyone suffered from a similar mental disorder: they didn't bother to read slides before commenting. It's really pathetic. Won't the Slashdot freaks ever learn?
Ah, well. I suppose that's what's so great about free speech, huh?
Posted by jzawodn at October 29, 2002 08:18 PM
I quit reading /. about the time I started reading http://jeremy.zawodny.com/blog/ . If it's important, Jeremy will talk about it, or point to it at Scott or John's site. I quit reading user comments at /. about a year ago becuase 90% of /. readers are prepubecent freaks.
How many requests per second is 'scrolling really, really fast'?
Just curious...
I still read ?>. I just read comments @ +5. But it remains a Karma Whore's paradise.
I've stopped reading comments on slashdot ages ago.. waste of time.
As far as getting slashdotted.. a bandwidth limitation would have been my first guess.. and yahoo having enough bandwidth to handle requests from slashdot would have been my guess as well :)
Some answers
YSP was "Yahoo Server Pages," a quick mod_perl based system that we built to benchmark against PHP.
How many requests per second did we serve while being slashdotted? Several hundred.
I stopped reading slashdot when they started doing the ads. I was in favor of the ads and was even ready to subscribe. When they fielded questions in irc, they refered to the 3% who read the comments as insignificant. The same 3% who submit the content, post comments and (what they didn't seem to realize) participate in the chats. I left and didn't look back.
One problem with the /. comment & moderation system that I've noticed is that a comment has to get in fairly early to get modded up. If it's been 15 or 20 minutes since the article went up and you're posting, you quite possibly will never get seen by a moderator (and good luck if you're an AC). So anybody who had time to read the slides wouldn't have had a comment that got visible through a typical moderation filter.
It's mostly chance that I decided to put the slides on public.yahoo.com and not on www.radwin.org. I had no idea I was going to get slashdotted. In retrospect, I'm really glad I made that decision.
radwin.org is running Apache on Linux, but it's a shared server with a bunch of low- and medium-traffic sites (such as www.phish.net) so it probably wouldn't have handled the load as well.
Also, radwin.org is physically located in a colo in NYC, so it probably would've been a slower web-browsing experience for the west coast folks than public.yahoo.com. I'm not sure where the majority of slashdot readers are coming from, but I'd guess that many of them are located in Silicon Valley.
Microsoft Powerpoint's HTML export tends to create a ton of really tiny .gif files, so those increase the load noticeably. It didn't help that I added a bunch of cute clipart to the presentation rather than my usual text-only style.
I am curious if you folks looked at AOLserver?
I've been hacking AOLserver for 2-3 years now and it's a very nice webserving/application serving environment. In the various developer forums for AOLserver there has been a degree of disappointment expressed that your slides didn't mention AOLserver at all.
I would suspect AOLserver/PHP is not the best fit at Yahoo for social, political, technical, and engineering reasons, but then again, AOLserver powers some dynamic websites that have a very high request rate.
It's easy enough to handle a few hundred static requests to second; unless you run out of bandwidth of course.
One of the develooper.com servers were doing ~20Mbit per second recently after slashdot linked to some PDF file we served for a friend of a friend. :-)
When some perl.org stuff gets slashdot'ed, it hasn't been much of a blip on the radar. I suspect that it's because most slashdot readers are clueless non-techie people who are only surfing slashdot for entertainment because they can't surf porn in the office.
Talking about the slashdot effect, I found this document (http://radio.weblogs.com/0103807/documents/opensource_performance.pdf) to be an interesting read.
I've always thought the definition of getting slashdotted was that the server(s) did indeed crumble. Wasn't this more of an attempted slashdotting?
The jargon file definition of slashdotting is linked in the URL up above--it seems to support the crumbling server definition.
My former little site (http://gamefu.org/) was featured on slashdot two times in one week (http://slashdot.org/article.pl?sid=02/04/09/0420239&mode=thread&tid=127 , http://slashdot.org/article.pl?sid=02/04/06/1337250&mode=thread&tid=127 ). The site was dynamic (very crapily written code) and we had bandwidth limits. I really don't see how sites get taken down so easily. Also getting slashdotted is actually when the site gets taken down due to the linking of it form slashdot, so there is really nothing funny about yahoo being linked from slashdot.
MUNDEN -MUNDEN 0 MUNDEN ( MUNDEN * RANDY * RANDY * randy
