Good job - thanks 
1 Like
Thank you guys. We certainly don’t deserve it, but just know a random on the interweb thinks you are neat.
2 Likes
Thanks olink!!!
I wonder if it was something about restarting it. If 2.8.0.beta2 was broken for twitter links, you’d think there would be some chatter about it on the discourse forums.
Sometimes you just gotta pop the cartridge out and blow on it.
1 Like
https://twitter.com/mattyglesias/status/1428337149292711937
https://twitter.com/mattyglesias/status/1428338919800819721
I think it might be cached by the system. It’s not even trying to make a call to onebox for that tweet for some reason. Other tweets work.
That second link was working, and now it’s not. Onebox/twitter is acting super flaky right now.
https://twitter.com/dznyc/status/1428340046319468544
I wonder if we’re getting throttled by twitter somehow.
https://twitter.com/pomeranian99/status/1428083230486454282
And we’re back. At least this time it’s not getting 404 errors when the links don’t preview. It’s making the call but twitter isn’t returning fast enough or something. So that’s better.
I poked around but I can’t find any way to set onebox to wait a little longer for previews.
I have a feeling it might be something like this happening.
Once I’d changed the setting, this meant doing a re-bake of historic posts. Since that rebake, a large chunk of YouTube videos that were previously onebox’d have now changed back to the linked URL
We have one thread of 10,000 posts that consists of solely YouTube video replies and all posts are URLs and not oneboxes.
During the re-bake, all queued jobs processed organically so it’s not jobs stuck in a queue of deleted jobs.
I haven’t seen the same error messages that @marcozambi described, but I believe we are tripping a rate limit too.
This is youtube. But if we’re bombarding twitter with “rebake” posts, that could trigger them to rate-limit us.
Could be this? But if so why did it stop working before?
I’m new to Discourse, what precisely does rebaking a post do?
Baking is converting posts to static markdown that gets saved in the database. Rebaking can be done to update a post if necessary. afaik it needs to be manually triggered and shouldn’t be happening automatically anywhere. You can rebuild individual posts yourself if you want by clicking the little gear bottom right.
From that thread it just happens when you rebuild the site? Our discourse server then has to go out to generate all those twitter previews again. It seems like we did that a week ago or so?
I don’t know if rebaking is what’s causing it. But it sure seems like twitter is rate-limiting us for whatever reason. I don’t know a way around that except maybe to try to get a new IP with our hosting provider.
The key thing to know is our server definitely caches previews. So the network request will go from the user’s browser to our server, then if there isn’t already a cached preview (or failed preview) for that link, our server will go out to twitter. So when testing always use a brand new random twitter post.
We did some kind of major rebuild a week ago or so right? All my threads went from normal to tracking.
Also btw what was happening yesterday was different. Our server was returning 404 for everything. Which is what happens now if you enter a twitter link that doesn’t exist.
https://twitter.com/darrenrovell/status/1428391111056314371
Today when tweets fail our server is returning 200 with a message that it can’t find twitter.com. So something definitely changed with either the restart or upgrading to the latest discourse version.
My guess is in the first scenario twitter is returning 404 (not found), and in the second it’s returning 429 (too many requests). But we’d need to see the logs to make sure.
Not that I am aware of. I did a major migration over a month ago but that shouldn’t have had any impact on thread tracking.
No. Standard rebuilds don’t do any of that. Rebaking posts is more of an in-case-of-emergency-break-glass thing.
Got it.
Do these onebox requests going from our server out to twitter or other sites get logged anywhere? If we can find a 429 response that confirms the rate-limiting.
Probably. Everything gets logged. The log files are so comprehensive they’re useless without major filtering.
@microbet suggested this to me last night as well. Wonder if there is any point canning some/all of the bots. Rupar made about 30 tweets yesterday
Yeah based on that meta.discourse onebox/youtube thread I posted, our situation sounds very similar.
Canning the bots might be a good idea.
@goofyballer we’re going to suspend the bots for a while and see if that helps our tweet problem in case we’re being rate-limited. Can I just ban the accounts or is it better if you disable them?