On 14/02/2023 23:45, William F Pokorny wrote:
> pokorny:tmp$ telnet news.povray.org 119
> Trying 203.29.75.35...
> Connected to news.povray.org.
> Escape character is '^]'.
> 200 news.povray.org DNEWS Version  5.7b1,, S0, posting OK
> article <5e9cce19@news.povray.org>
> ... I get what looks like some large mime dump that I suspect is
> due some attached images... Probably the header is there, but not sure how to see
the header in this case.

You can use 'head <5e9cce19@news.povray.org>' instead of 'article' in this case. And
to get just the body minus the headers you can use 'body'.

> What to make of this? Some of my stuff is on the server and other stuff is not?

I can't say for sure what's there and what's not. To be totally sure I'd need to write
some code to cross-reference posts that are in the database against those in the news
server. Generally speaking if an article is removed from the news server by some
automated process (expire or disk usage) they will remain in the DB. If they are
explicitly removed (by the user or an admin) then they will be gone from both.

At the moment there's no way to specifically search the web view's DB (or more
specifically, no way unless google gets their shit together).

That said, I have improved the search pages by switching to google's paid search
service. The results on the free tier which I was using were pretty terrible.

However that doesn't fix the issue of missing pages or improper dates etc which we've
seen.

I have made another attempt to get google to digest the site by providing them with a
sitemap (tried this in the past and it didn't help, but worth trying again).

If you were really keen to see if a particular message is in the web DB you can grab
the sitemap and download the files it refers to, then grep for the message ID. The
sitemap is currently at https://news.povray.org/gsitemap.xml. Note this is (currently)
static, generated as of a few hours ago. If it works then I'll set up a cronjob to
keep it updated.

In addition to the above I also retain log files containing all content posted to the
server since 2004 (at least in theory - the earliest is 2004 and latest is today, but
I have yet to check that they all line up wrt start/end so there's no missing dates).
But accessing those shouldn't be necessary provided the web DB has everything.

If after checking the above-mentioned resources doesn't find a message you think is
missing it probably wouldn't be too difficult to throw together a simple perl script
to pull every post you made from the log files.

-- Chris

Post a reply to this message