Dryline Hosting

Yeah, Mike got Farked, torrented and a few other things all in one day. We're trying to get him a work around right now. He'll be back up.

Well, the FARKing was moderately hard, according to FARK's ticker maybe 20,000 pageviews in the course of a day, with maybe a meg or two of photos on each page. That's like 40 gigs, give or take 10 gigs, tops, right? Torrenting shouldn't matter at all... you do understand how peer to peer works, yes? It has nothing to do with http, operates on a different port, is not a server client relationship, doesn't involve your server at all, etc.

You may not be able to comment on customer issues, but out of curiousity... is pulling the plug SOP over at drylinehosting.com any time a site gets unusually popular? Not trying to ruffle your feathers but, candidly, that seems like a pretty shortsighted thing to do, PR wise. Now hundreds of thousands of people (including the chasers you cater to) get to see a giant advert about how great your hosting is (headlined by 30 point text saying the site is down) instead of the content they were looking for. Talk about irony.

This kind of thing is why I decided to use blogspot for image posting a long time ago. Google never yanks my cord outta the wall just because I'm popular. They just raise my pagerank. (And it's free.) I know that you do not run a billion dollar company or anything and can't give things away for free, but you do compete with billion dollar companies, and when they're perceived as giving better, more reliable service than you, I'm not sure what your market advantage is.

**edit** 12:43AM 01/03:

Just a heads up that, as David pointed out to me in PM, this post came off more than a little bit offensive to him. It was worded poorly, and I did not mean to infer that David's service sucked; it was, actually, a misfired attempt to ask him what the heck was up with pulling a webpage that was getting a lot of traffic. He does take pride in his service, and I suspect that had I worded it better so that he could see what I was getting at as opposed to my post's snarky undertone, he would not have been as upset. I apologize for putting the question to him in this manner -- I could have phrased it much better.
 
Last edited by a moderator:
:confused: Whaaaa???

Im a happy Dryline customer.. please splain Lucy...

Fred
 
Well, the FARKing was moderately hard, according to FARK's ticker maybe 20,000 pageviews in the course of a day, with maybe a meg or two of photos on each page. That's like 40 gigs, give or take 10 gigs, tops, right? Torrenting shouldn't matter at all... you do understand how peer to peer works, yes? It has nothing to do with http, operates on a different port, is not a server client relationship, doesn't involve your server at all, etc.

The farking or other such massive influx of traffic is not about pageviews or bandwidth usage (although it can certainly eat up a person's allocated bandwidth real quick). It's the http requests. There are only so many virtual doors to go through at any given time for EVERYONE on the server. At one point there were over 20k http requests per hour, just from Fark.com and that doesn't include the other traffic to this site (seemed the pictures managed to get direct linked from a number of other places besides fark.com) and the other traffic to the other sites on the server. Just to understand the http requests. Calling up one page can mount MANY http requests depending on what's on it. That's just for ONE viewer.

As I explained to you via PM, it's much like Walmart on black Friday when the doors open. Only so many people can squeeze through until the entrance gets clogged up. And yes, I have a full understanding of how peer to peer works, I mentioned that because I saw someone post it, not anything to do with anything else.

You may not be able to comment on customer issues, but out of curiousity... is pulling the plug SOP over at drylinehosting.com any time a site gets unusually popular? Not trying to ruffle your feathers but, candidly, that seems like a pretty shortsighted thing to do, PR wise. Now hundreds of thousands of people (including the chasers you cater to) get to see a giant advert about how great your hosting is (headlined by 30 point text saying the site is down) instead of the content they were looking for. Talk about irony.
No we do not pull a website for being popular. In fact this is the first time ever we have had to pull a website for monopolizing the server resources (in this case CPU time and memory allocation). We worked to try and accomodate it for several hours until it became apparent that it just wasn't going to happen. We then had a decision to make. Suspend the site or let it take down all the rest of the websites with it. I did what any other shared host would do and spared all the other customers the pain. As for shortsighted? I dunno....shut one site down or lose them all? What would you choose? The giant "advert" you are talking about is a standard suspension page. Any time a site gets suspended (which is almost always because someone didn't pay their bill on time) it automatically goes to that page. I can't change it from one site to the next. It's a limitation of cpanel. Ironic, maybe I don't know. I was just trying to keep the rest of the server up at that point.

This kind of thing is why I decided to use blogspot for image posting a long time ago. Google never yanks my cord outta the wall just because I'm popular. They just raise my pagerank. (And it's free.)
Well since you don't actually get any disk space to store stuff there you can't test that, but if you did I would tell you to start backing your huge hard drive up on there and see how long before they pull your site....errr...BLOG..yeah.

I know that you do not run a billion dollar company or anything and can't give things away for free, but you do compete with billion dollar companies, and when they're perceived as giving better, more reliable service than you, I'm not sure what your market advantage is.
Your right, I don't run a million dollar company. If I did I probably wouldn't be here right now personally answering this question (although I do tend to give a lot of things away free). I'd probably have some peon doing it if it got answered at all. We try to PERSONALLY take care of everyone's needs. Something a billion dollar company could care less about. For some things, no doubt they do have a better service. We aren't all things to everyone. Some sites outgrow standard shared hosting and they need to move on to more dedicated solutions. You might recall Tim had to do the same thing with ST here recently.

Market advantage? I'll let our other customers post their own experiences and that can stand for itself.

And yeah I did pull this thread initially. I acted on instinct rather than better judgement and after discussing it with another mod put it back.
 
Last edited by a moderator:
High traffic sites are not the norm, and no 'normal' web server is built to handle one site suddenly getting insane amounts of traffic. It's like the bathrooms at the office, the gas pumps at a gas station, and the lanes on a highway. They are designed to handle normal traffic flow with a limited ability for higher capacity. When an unusual circumstance occurs where higher than normal traffic is presented, there is going to be a problem: traffic jam. It is no one's fault, just the nature of the situation.

A site that routinely gets high traffic (like every day or once a week) is usually best put on its own dedicated server with non-shared resources. For most web sites, a dedicated server is overkill (like building a gas station with 100 pumps to handle the one day a month with high traffic). It's a trade-off. You can build for the rare high traffic days at high expense, or you can build for the normal days at more affordable costs (with the tradeoff of capacity problems during the rare high traffic days). It's just economics.

A site that gets high amounts of traffic is going to have problems on shared servers - just a fact of life.

A tactic that works well for high traffic events is to thumbnail and link all photos on a page, or break the photos up into a series of pages. Having all of the photos full-sized on one page forces the server to attempt to load them all at once every time someone views the page.
 
Last edited by a moderator:
As a site administrator I can tell you that the Fark effect a.k.a. Slashdot effect can be a serious mess, and the problem can really scale out of control depending on what kind of content is on the page. The pipeline feeding any given server is not infinitely large, and a smart administrator will be thinking about all the other customers relying on the server. It was great when our Mr. T Forecasting School comic got Farked last year, but I was also monitoring server load & bandwidth consumption and was ready to zap the directory if things got bad.

I applaud Ryan's edit and hopefully some info has been learned here. The bottom line is if you expect a Slashdot effect, the only sure way to stay afloat is to get dedicated hosting and keep on top of things. Unfortunately that can cost some $$.

Tim
 
I did want to add that the standard suspended page that displays might have given the wrong impression that Mike hadn't paid his bill or something.

That was certainly not the case at all. It was suspended solely out of trying to keep all the other customers online. The limitations of the cpanel software only allow me one suspended page for everything. He has expressed to me that it might have appeared that way to some people and as such we will be making some changes to that page.

As I told Mike, I will be analyzing the traffic from today over the coming week, see in fact WHERE it was all coming from (I am sure it was not all from fark.com) and see if there is any place where we could make improvements or if in fact it WAS just too much for the server to handle. If there is room for improvement there, it will be made.

I congratulate Mike on hitting a level with his site where he could really make use of a dedicate server. Few personal sites ever hit that level. It's actually quite an accomplishment.
 
Last edited by a moderator:
Well I'd say if I hit this every day I'd probably really need a dedicated server, but I don't sustain that stuff..far from it. And like Dan mentions, it's probably not worth it. It'd be nice if one could spread that out over the month and not get it all that day. Life would be simpler.

It would not surprise me if Fark did this on its own. It's been extremely rare to be on a host and have it survive during the fark(one must realize how huge they've become). Guessing most all news outlets check out the links frequently. I've only been on one server that utterly impressed me for being able to handle it. Prices went up and I was moved from that one. Like was mentioned from the start of this, it's not likely to affect most, and if it does, it certainly won't be most of the time. Since this is my job now, I have to be bugged when I can't stay running during a fark or high hitting deal. If it wasn't my "job" and it was like before, then I'd just say, ok cool, let me know when it is over and back up(oh how I sooooo miss those days sometimes).

I've been sustaining 2k to 3k visitors a day for a while now(with higher peaks in there), and have had no server issues while at dryline. I have a lot of images online now, so that's a healthy number of folks looking at a large selection of images. I may have had problems come up, but it was unavoidable things that I didn't understand, such as local ISP issues. That or problems my ftp created...which I can now say for 100% certainty is my program and not a server glitch(still does it at the new host). Questions were always promptly answered. I would not be nearly as nice to myself as David has been to me.

The best part here is that the current servers got a real good test and it can be looked at. I've seen David talk to a lot of different people about different things relating to hosting, and can't recall seeing him tripped up on anything.

One interesting thing to me is how long it was on and working. It was placed on fark late the previous night, and what time was it shut off? 11:30 a.m.? I'd almost say it was half handling it. Then again I guess the longer it is up the more locations will start to link to it(would love to see what happened with hits/visits).

I actually expected it to go down once I learned it was on fark. It's always been the norm everywhere I've been. In the end, if there is something that can be changed, I'm sure David will do it.
 
As pure interest, how are things handled financially during a peak popularity cycle? Do you just burn through your allotted bandwidth and then the site shuts off, or do you get an extra bill? Or perhaps is there some contact made between David & Mike to discuss matters? Not trying to intrude, but am curious on how things are handled.

Way to be a resource hog Mike ;) Congrats - they really are some great pictures!
 
There are ways to avoid being farked/slashdotted - several customers I have worked with in the past were linked to by slashdot, and you _can_ stay up - it's just a matter of how you configure some things and overall load.

One of my clients had a 2.2 Ghz machine with 1GB of ram and T-1 for upstream. The client went from 6,000 hits a day to over 36,000. The way the site was configured, the T-1 went full before the site actually went down. It's an interesting config, and doesn't work for everyone. If someone wants some pointers, let me know.......

(and no, Akamai was not the answer.)
 
As pure interest, how are things handled financially during a peak popularity cycle? Do you just burn through your allotted bandwidth and then the site shuts off, or do you get an extra bill? Or perhaps is there some contact made between David & Mike to discuss matters? Not trying to intrude, but am curious on how things are handled.

Way to be a resource hog Mike ;) Congrats - they really are some great pictures!


The sites go down before bandwidth limits are reached. The biggest spurt I ever saw was after the November 2004 Auroras after a fark and follow up massive e-mail chain(one that actually was linking to my site.....not bill bob took these hurricane pictures). That was 820,000 visitors in 20 days. It came out to around 30,000,000 file hits and 400 gigs transfer. So, if godaddy says 2,000 gigs transfer, they'd pretty much have to also be saying they could handle very hard hitting, since it would be a requirement(cause it'd be 5 times harder than THAT......which was the worst I'd seen....and someone else will have to tell you what exactly kept that server up and running....I have no clue).

The funny thing is I think that was the only time I went past what I was given for bandwidth. The guy kept it running and said it would cost me, but then forgot to bill the reseller I was going through.
 
Hard bandwidth limits are usually better than the alternative. That is when the host quietly lets a site go over the limits and then charges overages (which are usually charges passed on from the host's connectivity provider). Overages are usually astronomical, and for a typical Fark/Slashdot could stick a site owner with thousands in overage fees. It does happen - JibJab is a good example. I think JibJab's overage fees were in the five figure range.
 
As pure interest, how are things handled financially during a peak popularity cycle? Do you just burn through your allotted bandwidth and then the site shuts off, or do you get an extra bill? Or perhaps is there some contact made between David & Mike to discuss matters? Not trying to intrude, but am curious on how things are handled.

Ed, the service is setup it automatically suspends when the bandwidth limit is reached. This is primarily to prevent any new signups from doing something naughty, like uploading warez etc. We recently upgraded the bandwidth ceilings on all packages and so far NO ONE has hit them.

As far as how we handle it. We have yet to charge for overages. It's been my unwritten policy that if someone just happens to go over a little on their bandwidth, we don't charge for it. If it's happening regularly, we usually just ask them to upgrade to the next plan. Again though, since our last upgrade on bandwidth on all the packages, I don't recall anyone that has hit their ceiling on bandwidth.

EDIT: I wanted top add that if someone who is a customer is expecting a peak in bandwidth usage, like their site getting featured somewhere, if they will let us know ahead of time if they can I can temporarily raise the bandwidth cap even higher so it doesn't automatically shut off during the peak when they need it most.
 
I got some preliminary numbers on the farking, and with Mike's permission I wanted to share so you guys can see some of the numbers we were talking about.

These numbers were JUST for extremeinstability.com and doesn't account for other traffic on the server.

Yesterday 780,591 http requests with 35796 full pages hit.

The majority of it came between 8 and 11 am. With 9 am hour being a real kicker with 105497 http requests in that hour alone. This is when the server completely crashed and I struggled with it to recover after that for some time.


Additionally, there were another 12942 direct requests for the images alone on top of the regular http requests. (Images directly linked elsewhere)


Then there was the residual traffic Umscheid got from the image of his someone direct linked there as well.




 
I think that David Drummond has excelled at looking after every one of his sites during this awkward and inconsiderate "attack" or whatever you call it. He hosts my domain and has always been perpetually attentive and quick to respond to any issues I had that were within his control.

I for one am certainly not going to blast him for pulling the site in question in order to save the rest of the Dryline community. What would you have had him do - leave things as they were and have everybody's e-mail and websites down for days or weeks on end? Personally, I send and expect a lot of emails domestically - and with a sick family member - internationally......and I appreciate the way in which David considered everybody in this dilemma.

At the end of the day - this Fark thing is only a fad and it will pass. Give it 5 years and the site might not even be there anymore. But I know mine will be! ;)

KL
 
Furthermore, the practical benefits of this type of publicity (Fark, etc) are (at least in my experience) not worth the costs. High web site traffic results in a lot of 'oohs and ahhs' and 'that's really cool' from people, but at the end of the day, what does that bring? Not much or anything at all, at least when it happens to me. Certainly doesn't help pay my mortgage, LOL. It's nice to get the exposure, yes, but IMO not worth any bandwidth overages or server headaches! Publicity like this can be more of a curse than a blessing sometimes.
 
site traffic

Hello David, Tim, and the others,

I'm impressed by the numbers of the visitors! I never had such chance. My top day is with 180 visitors per day and I have made virtually everything, which is possible, to get such traffic as Mike, but with no success:(

Please tell me more about the Fark, Slashdot, and Akamai.

Fark, as I understand, is a site where everyone can post an URL, so it will be visible for short time, can I submit the URL of my homepage? I have option to limit the numbers of the connections and the traffic speed (I'm not using cPanel, because it's paid software, I keep the things as simple as possible, doing the entire work in the good old UNIX style like in '70).

What is "slashdot"? slashdot.com? I know this word ("slashdot") from CNN or other media, and now I see that exist domain slashdot.org for some computer news.

The problem with the visitors on my site is from more than 7 years.

Please, Please, Please explain me how this farking is working!

:)

Angel
 
Furthermore, the practical benefits of this type of publicity (Fark, etc) are (at least in my experience) not worth the costs. High web site traffic results in a lot of 'oohs and ahhs' and 'that's really cool' from people, but at the end of the day, what does that bring? Not much or anything at all, at least when it happens to me. Certainly doesn't help pay my mortgage, LOL. It's nice to get the exposure, yes, but IMO not worth any bandwidth overages or server headaches! Publicity like this can be more of a curse than a blessing sometimes.

I think it depends on how you capitalize on the FARKing. The first FARKing I had, lots of people asked about prints, but I didn't have a pricing scheme put up for view and had to email everyone back personally. Most people got cold feet between the time they decided they were interested and the time I emailed them back and I only sold a few prints. The second time I was FARKed (and Reddited), I had a short blurb at the end mentioning pricing, mostly to avoid having people who have no idea about how much art costs and aren't willing to spend more than five bucks bug me with emails. :) That time I ended up making a thousand dollars from print sales. I suspect I could have done a whole lot better if I'd actually bothered to put up a Paypal ordering button or something. The important thing to remember is that FARK doesn't exist to link to sales sites, they exist to link to interesting sites. They didn't mind that I had prices at the bottom, especially since I'd had a bajillion people emailing me the first time, but I doubt they'd link to a site that appeared to be primarily trying to sell things, at least not without paying FARK money for the link.

As for FARK going away any time soon -- Karen, are you kidding?! ;) FARK's been around for six years already and gets like 50 million page views a day!
 
Last edited by a moderator:
Fark-Digg-Slashdot effect


Hi,

I'm back. I read in wikipedia about the slashdot effect and few other examples. It's cool:) and I'm going to blast my infinite 1024 kbps network. Wish me "good luck" ;)

Coming back later (if I don't come back after few hours then I'm blasted:)))).

:)

Angel
 
Hi Ryan,

You have very nice images on your site. About the commercial sites - Yes, I read that on the fark.com, but anyway, if I get 1000 or more visitors per day for an of my images, this is a very good "farking" start for me:) Then I can look forward how can I use the slashdot effect (for which I didn't know so far).

Angel
 
I'm not sure, but I'm thinking that trying to "Fark" one's self is probably not a good idea. I don't believe it will workout as one expects it to. :confused:
 
Hi Kurt,

Why do you think so? Fark.com will delete my URL from the site, because it will see that I'm farking myself (looking at the IP of the fark account and the target URL)? I want to play with this kinds of Internet effects.

Can I put a copyright on the image with my URL? May be not... I will re-read the fark.com terms of use.

Angel
 
Hi Kurt,

Why do you think so? Fark.com will delete my URL from the site, because it will see that I'm farking myself (looking at the IP of the fark account and the target URL)? I want to play with this kinds of Internet effects.

Can I put a copyright on the image with my URL? May be not... I will re-read the fark.com terms of use.

Angel

Hi, Angel! The way FARK works is that users submit stories to them. They get thousands and thousands of submissions a day, and only a handfull are selected to go to the main page. You're free to submit your own site, but they reject the overwhelming majority of most submissions. It's this selectiveness that makes them popular -- people keep coming back to the site because it's a source of interesting content. :)
 
Last edited by a moderator:
ah... bad... thanks for the info, Ryan:) If someone of you know way to get in slashdot storm, please let me know:)

Angel
 
I tell you the person that called for everyone on fark to crash my server has caused just about as much problems as the initial farking has. I've been having to block IPs left and right and it's still going on today. :(
 
I just wanted to follow up in case anyone that was interested missed this on Dryline forums. We just commissioned a new Dual Core Xeon server today that will be a replacement for the current one. More robust and should be able to handle more demands placed on it.
 
Back
Top