Do it right now for the lulz.
So hereās the Official Postmortemā¢ from the first debate. The server was fine. We probably could have managed without the upgrade, but the strains would have been noticed and it was nice to have the backup capacity there.
Obviously there were some configuration issues that led to a bunch of those stupid logged out messages. The forum software is tuned to be, well, a forum. Not a chat room that 150 people are hammering non-stop for 90 minutes. I knew that was an issue before the upgrade and had adjusted for it, but figured the default config would be OK once we doubled capacity. I was wrong. Sorry!
So moving forward Iāll reapply those changes. Iāll also add 2 or 4 more unicorn workers because we now have the capacity to support them.
Yes, the forum has unicorn workers. Four of them. And Iām going to add more. Duh.
I canāt believe some of you didnāt know about our unicorn workers.
As long as no one is forced to eat Skyline, all is well.
Thank you for all the work you put into this site.
+1 Take pride in the fact that while Trump steamrolled the nation tonight, he was no match for the UP server and your hard work everyone!
Why did the crawler traffic drastically increase in March/April?
Makes sense.
We switched back to AdSense, which does a lot more crawling than other networks.
Trump making Unstuck great again imo.
MUGA!
Or is it MUPGA?
Not sure how the Expenditures thread ended up the dumping ground for server status reports, but Iām too lazy to do anything different tonight.
Everything seemed pretty ok for the debate tonight and we were chugging along at about 50% capacity with no issues. Then right around 10:25 Eastern things went super flaky for 5-10 minutes and there were presumably lots of temporary logout messages that no matter how hard I try always sound much more alarming than they should.
We had been steady at around 50-60 active users and ~40 page views per minute all evening, but at that moment we spiked to 110 active users and 100 page views/minute. Both of those numbers slowly tapered off over the next 30 minutes.
Also, on Tuesday we had a surge of unregistered users that doubled our traffic for the day, making it by far our busiest day*. The next day things returned to normal. Why did these things happen and what do they mean?
ĀÆ\_(ć)_/ĀÆ
*Technically we had 1 busier day earlier this month, but it was a fluke of crawler traffic and not real users.
Could be a probe for an attack.
Twitter refs?
Yeah thatās concerning. Should we consider going registered users only and locking registration near election day?
I thought about it more, and thereās way likelier explanations. I think an attack probe would look like a bunch of random unregistered user traffic, but the volume would probably be a lot higher than what zikzak observed.
I agree. And if weāre going to get ddosāed thereās nothing we can really do to prevent it anyway except hang on and hope we have the capacity to weather the storm.
Also @anon38180840 we have a lot of unregistered regās. I would be very reluctant to block them all.
Could we restrict it to like IPs that have already previously visited and X new lurkers such that X wonāt overload our servers?
Theoretically, I suppose. But again, that wonāt prevent anybody with a moderately sized botnet from taking us down if they so choose. Firewall rules on the server youāre trying to protect wonāt do any good because an attacker will just overload the firewall instead of the web server. End result is the same - a system that canāt handle all the traffic being directed at it.
I should probably report here that I am spending a few cents per month on our email service.
But, I wanted to ask @spidercrab, what does our budget look like? what do our costs look like? I just would like a general idea of the numbers in my head, and itās been a little while since Iāve read an update about this (and the forum search software sucks, I really tried to look).
and are we just giving up on the google ads money?
Nobody is giving up on anything, there is just nothing to do. There is no way to contact anybody. There is nobody to appeal to. Google just takes the money. Thatās the end of the story.