Alicornutopia

GC.start triggers garbage collection, it doesn't release the memory. My concern with setrlimit is that all it seems to do is shut down the worker when you're out of memory, which would spare me needing to keep an eye on our memory usage and restart the server, but it would cause user errors instead of swap memory which seems suboptimal from a user perspective.

Wait, it shuts down the worker? That seems like a big bug on Ruby’s part, you actually needing more than 256 MB for some requests, or a leak. What it does is make the OS say “no” to Ruby asking for more memory, which should just trigger a more aggressive GC and compaction, and only error out if it really can’t find enough memory for everything.

Ruby has settings to trigger both minor GC and major GC runs, yes.

My point is that setrlimit shouldn’t cause a crash; it should trigger as aggressive a GC as necessary instead. The only reason calling GC.start is inadvisable in the middle of something major is that it stops the world, right? That’s still preferable to a crash in pretty much any application written in a language like Ruby. Either you really do need more than 256 MB in some requests and the only solution is to disallow big requests or do streaming (or switch to one worker and lose the ability to handle multiple requests at once), or you don’t and triggering the GC if necessary is better than an R14.

That’s why I’m surprised you’re actually seeing crashes. That shouldn’t happen.

What generally does happen when Heroku throws an R14? I thought it killed the worked but that doesn’t sound right given what you’ve said here.

Setrlimit is not the same as triggering a GC run, but I am way too tired and dealing-with-family-drama to keep explaining Ruby for now, so further questions will temporarily need to go to Google.

Marri wrote:Setrlimit is not the same as triggering a GC run, but I am way too tired and dealing-with-family-drama to keep explaining Ruby for now, so further questions will temporarily need to go to Google.

Relevant links are here for Ruby's Process::setrlimit() and here for unix's setrlimit(2).

Oh, an R14 is just a warning and swap slowdown; I thought it meant Heroku killed the app.

I failed to find the other answers on Google, perhaps because I don’t know enough about Ruby, but it seems like it doesn’t crash the application but instead throws a NoMemoryError, which I suppose if it happened in the middle of Rails code instead of your own code could cause a crash. Before throwing the error it looks like it does trigger the GC, though; this makes setrlimit equivalent to triggering the GC when it thinks it’s necessary.
Of course, with multiple processes in the list, it can’t say, “I’m out of memory; I’ll trigger a GC and also ask the other one to do a GC”, so that wouldn’t help as much as I was thinking earlier.

Updates from Throne:
- You can now mass add characters (that do not already have a template) to a template when adding or editing it, or mass remove a template's characters when editing it.
- Top header bar now has your user id in the galleries and characters link.
- Galleries in Add Existing are now sorted by name
- Favoriting a post no longer redirects you back to page=unread, just the correct page number

Updates from me:
- Test coverage is at 81.12%. More details for the curious:
- I realized that the code coverage number and favicon turn orange when you break 80%, so that's delightful. Way less angry red things!
- Our mailers are now 100% tested!
- Our libraries (the post scraper) and our async jobs (...we don't have any) and our presenters (JSON code for the forthcoming API / existing piecemeal Javscript calls) and our concerns (shared code, e.g. posts/replies) were already 100% tested.
- Our application helpers are at 87.14% tested and will remain so for the foreseeable future, as they are used almost entirely for rendering views and those I need to either come up with a sane way to test that in my existing RSpec framework (unlikely) or install proper Cucumber view tests (likely but time consuming).
- Our models are at 96.01% tested and are my current burndown target.
- Our controllers are 71.57% tested, not to mention a good 55% some percent of our code all by themselves, so they go last.

Also someone has been loading A Tale of Two Hells, the post on the Constellation with the most replies, in flat mode today. Twice. My memory consumption jumped 120MB the first time and the second time failed outright (it's... likely to just time out until I get streaming working? Which can't be until I have Rails 4 working? Which can't be until I have more tests? THERE'S A REASON I WROTE MORE TESTS TODAY UGH.) I need to look into precomputing the damn thing.

Something seems to have gone wrong with templates pages--the Posts with Template Instances part is working fine, and view=list correctly displays only the characters that are members of the template, but view=icons now seems to display all of an author's characters instead of just the ones with that template, see e.g. here and here.

I think Marri's fixed it?