Alicornutopia

And private threads, etc. There's a reason I asked for the non-private threads instead of everything except passwords. But I suppose that would require the table for users to map the author IDs to the actual authors, so I was still asking for more than I thought I was.

I still think the right solution, from both a memory point of view and a responsiveness point of view, is streaming the replies. It would make client-side caching harder, but you currently say Cache-Control: max-age=0, private, must-revalidate even for logged-out users, so somehow I don’t think you care about that very much.

General principle on memory usage in GCed systems: never-shrinking OS-level memory usage does not mean the application is actually leaking memory; most GCs don't bother to return memory, especially if it's not a large proportion.

If you have some concrete upper limit (e.g. All The Memory Your Computer Actually Has or some hosting service quota), you want to tell the GC about that limit; there is likely some startup option to do so ("heap size" or some such). This is much less hairy than any other kind of GC tuning. (But don't set the limit low for no reason.)

If the application is actually leaking memory, this will manifest as one or more of:
• actually getting an out-of-memory error
• extreme slowdown as the GC desperately tries to squeeze out the last bit of free memory to use (or, if there is no limit and the OS is set to use swap, lots of pages are swapped as the GC does its thing).

Disclaimer: My background/experience for most of this is from Java (JVM), not Ruby.

Yeah, I meant "in addition to non-public threads", which is a very reasonable filter to apply. But yes, to get threads you need users and characters and icons and then all the associations, etc etc.

Rails 3 does not have proper streaming; all they have is "render layout then render the expensive view". To get proper streaming we have to get to Rails 4, which is part of why I've been writing so many tests; unlike Ruby, major Rails versions are deeply unsafe as an upgrade without good tests. (2 - 3 was the worst I know of, which thankfully we skip, but 3 - 4 won't be fun.)

GC tuning post I like, or another one that has good explanations of the various tuning variables, but none seem to actually tell the GC tuning how much my concrete upper limit (512 MB total, so 256MB is) is.

The application is not leaking memory, as near as I can tell; it's got reasonably stable memory usage now except for occasional large jumps, which are pretty trivially mapped to memory-intensive requests like per_page=all and view=flat. It's just not returning memory to the OS. WIth Heroku's ephemeral boxes, this manifests as a combination of swap memory usage and Heroku-triggered R14 Out of Memory errors.

This is why I suggested setting ulimit. I don’t know how much control Heroku gives you, but you could set ulimit -v 500000 (or 524288) before starting the Ruby processes, and then they’ll know they both have to fight for that space.

The man page for ulimit just says “kilobyte”; I don’t know whether that’s actually kilo- or if it’s really kibi-, so the numbers I gave above both assume the latter so you don’t set the limit too high.

I am reasonably confident this is not a thing I can do on a shared Heroku server.

ulimit does not, in general, require root, if that's what you mean.

Raising ulimits takes root; lowering them does not. You’re only lowering them for yourself.

You can also call Process.setrlimit(:RSS, 512*1024*1024) pre-fork or Process.setrlimit(:RSS, 256*1024*1024) post-fork, I think.

Do you have control over how the Ruby app is started, or do you just have some entry point in the Ruby code?

I'm pretty sure Heroku just runs a ruby file if you're running a rails server. So I guess we could try to manually run a shell thing at boot but I'm not sure?

Without knowing much about Ruby, section 2.4 of this article seems to give concrete steps to set memory limits for workers, one of which is Process.setrlimit which Daniel just mentioned. (It's honestly sort of mindboggling to me that Ruby doesn't have a concrete way to force memory to be freed...)

You can call GC.start, but as far as I know it's really very inadvisable to do that while you're doing anything major, and "is partway through a controller in a Rails app" is something major. There used to be a way to trigger it between requests to the app, but it was with a different server system and I don't think there's any way to do it with the current one.

I have no idea about Process.setrlimit though and am not really on-point with the GC stuff.