Daily Reports Chatter

[jalapeno_dude]

To get new threads, can't you just look for the largest value of x such that /posts/x exists? I guess private posts could be a problem, but you could e.g. try the next 10 entries after the current highest one you have before giving up. (I assume you're already querying the site way more times than that each time you make the report, so this shouldn't be a heavy load on the server.)

[Throne3d]

Yeah, I can do that, but also people have threads private and then later publicize them, which wouldn't therefore be on the list. It's a way I've sorta been doing it (when I notice a new thread I usually check between that thread and the last one I remember adding to the list), but it unfortunately doesn't work in all scenarios. It'd be useful if my history broke for some reason, though.

Also it still doesn't solve the issue of the thread descriptors and colorings and flags and such, especially when a third person joins a two-person thread, or the moiety of the world isn't clear, or if the thread has no 'description'. But I'm guessing you didn't mean it'd solve that. :P

[DanielH]

I assume there's a reason you can’t just check the Recently Updated page for most of this, but I don’t know what it is. Is the page not stable?

How difficult would it be to add an actual API?

[Throne3d]

Haven't gotten round to it yet – currently I'm doing things manually and it works without a huge expenditure of effort in the present. The page doesn't flag new threads but I could just save the old ones for that kind of thing. Also I still have to add dreamwidth threads (this was a concern at the time? I don't think there are many dreamwidth threads still around, and I haven't actually checked a lot of this in a while), though that could be separate. The page would also have a chance of being updated while I scrape it, causing me to miss a thread or something, and things keep getting modified in different places on the site (which is great! But means I have to edit things and also means I'm sort of wary of adding more pages to scrape). So, it's doable and I have been thinking about moving over to that for scraping for the report, but I haven't actually gotten around to it yet.

Also it still doesn't solve the issue of thread descriptors and that's the main thing that's been bugging me. :P

Actual APIs, there are… bits and pieces around where AJAX is used? It wouldn't be too hard – still takes work and checking and such though – it was being looked into, it hasn't actually had much progress yet.

[Throne3d]

So, I have now created a Javascript thing to allow me to more easily see if a thread should be added to the report. It should also work for others, and it now no longer relies on my history. This is fortunate because I've just swapped to a laptop and apparently Chrome hasn't synced my history properly! (Oh well, I still have access to my desktop so it wouldn't be all that bad anyway).

For people who understand this, you run the following script – and if you don't understand this, you probably shouldn't be doing this because it could theoretically ~do bad things~ such as post on your account or something.

Code: Select all

javascript:void($.getScript("https://pastebin.com/raw.php?i=d5RxNvdN"));

(Remove "javascript:" if you're pasting it in a console, make sure "javascript:" is there if you make it a bookmark, typical sorts of things when it comes to this stuff.)

If you want to see the actual code it uses, that's here: https://pastebin.com/raw.php?i=d5RxNvdN. This can be modified by me at any time, so again, security risks and beware.

This is mainly as a backup for me if I somehow lose it, but could be used by other people if they are willing to execute it. It makes a nice little popup if there are missing threads. It is not, in fact, malicious, but you still probably shouldn't run it unless you know what you're doing.

For example, with a thread (that is marked private, so I haven't put it on the report) missing, it shows this: https://i.gyazo.com/00192908eda39676261 ... baee85.png.

[Throne3d]

Might be a bit late doing the report on Tuesday – I should have it done before 6pm my time, I think (GMT with BST, UTC+1), but may not get it done until much later if at all; I'm unsure what my schedule will be like that day.

Then I'll probably be a bit more inconsistent about when I put it up, since I'll be at university and busy at certain times more than others. So, apologies in advance? :P

[Ezra]

"Collaboration with the enemy" should have Kappa purple as well as its other two colors.

[Throne3d]

It should indeed, thank you. That should be fixed tomorrow. (I still have not automated it, ugh, and three-person boxes never seem to start off with all three authors nor make it clear they're getting cameos.)

Thank you also to Pedro who has done the report the past couple of days. Yesterday, I had internet issues. Today, I had schedule issues. :P

[Ezra]

Have You Heard doesn't seem to be in the Today, Not Yesterday. Does this indicate a bug?

[Throne3d]

It's because it was (technically) created yesterday (with 10am UTC being the cutoff for each day), and I hadn't forcefully marked it 'new' – and I also didn't add it yesterday, idr why; might be just that it had one post?

I can manually put it there, though. And mark it 'new'. And alter the link. (I should probably make this a thing I can do automatically…)