The Glowfic Constellation

[Throne3d]

Heroku limits it to 500MB memory, pretty sure. It doesn't seem to be adversely affecting anything, because requests go through – it seems to be more like a warning – but there are only errors when someone tries loading a huge post all in one go (like 'halls of stone' on 'per page' of 'all'), because it freezes other requests until it's done processing that one.

Edit: Oh, forgot to mention, it's been giving the 'error' for at least a few days now (that's the first time I actually looked at the logs, so I don't know about before then), and the numbers that it's giving are much lower today than they were last night (might be linked to how many requests are going through at once). I honestly don't think it's having any impact at the moment, though, because the requests are going through fine and it only says 'timed out' if, again, someone has temporarily locked it up by loading a huge thread.

[PlainDealingVillain]

Huh. I'd also been seeing a lot more Application Error pages than usual, so I thought those were related. Probably it's just because I default to all.

[Ezra]

I've been seeing a few application errors this week on my per-page of 25.

[Alicorn]

I've been getting them. I have per-page 100.

[Throne3d]

It usually gives people a bunch of them in a row when the requests time out, again due to someone else having tried to load a large page and causing the app to be locked to that. Since it can only process one request at a time (which is possibly a thing we're looking into? I'm not sure), if someone loads all of 'halls of stone' and it takes something like 30 seconds to load, then in that 30 seconds the Constellation might be hit with 5 requests for other pages, including a request for the 'Unread' page which is slow and takes a while to load, and also three replies to different threads, which it then has to try to process after serving the large page.

Since the subsequent requests then take up extra time, it can end up being over a minute (or something like that) since a request has been sent before it finally gets down to it to respond, which causes it to terminate the request to try to prevent the app from getting stuck in a huge queue of things to process, giving you an Application Error. It seems to process thread replies just fine, without giving a timeout, so you shouldn't (and I haven't so far) lost a tag, even when I get the error, because for some reason those seem like they can be processed at the same time, or maybe they're just given priority – you may, however, receive an 'application error' after posting a reply, but that's because it failed to load the thread to display to you after it processed your reply.

I'm not really sure how to fix this issue other than getting the application extra processing, which costs money. I'm also not clear if that would in fact fix the issue – it should, I think, but it might not because the 'locking' that I mentioned (to prevent a thing called a 'race condition', where you alter a thing from separate places and it ends up being incorrect overall) might prevent them from running at the same time, something like that.

It's probably preferable if people don't load large pages like "halls of stone" all in one go, anyway. Maybe try loading 500 replies at a time, or fewer, and waiting a minute between your requests? I've specifically got my scraper going through Dreamwidth for those large threads so it doesn't freeze up the Constellation when I try making EPUBs of Silmaril. Unfortunately this means the EPUBs won't have the edits made on the Constellation (for typos and such).

[Marri]

Restarted the app, hopefully that helps some. I do think the thread-safe thing I looked at might be a cause; I believe Heroku defaults to use Puma and I should really be on Unicorn, so might swap that out.

[Ezra]

Petty: when I refresh a thread page, the layout seems to go wonky for a bit until the images load. I'm told it'd stop doing that if the image tags mentioned the image sizes?

[Marri]

I have been explicitly not including image sizes, just max-width and max-height, because images are often not given to me square and this prevents distortion. I could probably force the wrapping boxes to be 100x100, but that would cause some extra whitespace bloat. Are you bothered by the wonkiness in a particular direction (horizontal/vertical) or both?

[Kappa]

*whispers creepily* eeeeeeveryone should use squaaaaaaare icooooonssssssss *descends into unintelligible muttering*

[Ezra]

Vertical is what's bothering me.