|
 |
[Related note: NNTP users please use nntp.povray.org going forward]
For the last year I have been seeing lots and lots of requests in the
logs for the version 2.2 sources, many of which were made in locations
where it had never existed. It thought it curious but in the overall
scheme of things was not causing problems.
I have just recently found out the actual cause of this: an AI benchmark
that tasks the agent with finding, downloading and building the version
2.2 POV-Ray source (released 1993)!
The reason I found out: a PR lodged with the authors of the benchmark
noting that their agents had been blocked from downloading since we put
the site behind Cloudflare.
I thought it a bit funny that they didn't just ask us to allow an exception
to the Cloudflare WAF rules, which I would have been happy to do since the
files are all static and put little load on the server. I have in fact now
done this - everything under the /ftp path is now except from most of the
rules (but not from rate limiting, for example) and in addition I have put
in cache overrides for the entire tree, telling Cloudflare to cache *everything*
in there, plus use cache reserve to keep it there. This means we see very
little actual traffic for those files now, since almost all requests are
satisfied by the cache.
After my initial reply to the PR I got curious as to what the actual numbers
were so did some log analysis, and the results were too good not to share :-)
If you are interested you can see my response at
https://github.com/harbor-framework/terminal-bench-2/issues/76
but I'll also include the meat of it here.
------------------------------------------------------------------------
A follow-up, now that I finally know why povray.org spent the last year
being asked for a release from 1993. I went through our server logs, and
the numbers turned out to be too good not to share.
First, to be clear: you weren't the cause of our server trouble. From
September 2025 to August 2026, povray.org was being crawled by something
*much* bigger: a not-very-smart scraper working through residential
proxy networks.
- Scale: over 30 million distinct IP addresses hit our web and news
servers in that time. About 15 million of them sent requests in the
form a client uses to talk to a proxy, which no ordinary browser sends
to a web server.
- Rotation: most addresses made only one or two requests, so no
per-address limit could catch it.
- Manners: it ignored robots.txt and followed every link, including the
clearly labelled "Post a reply" link on our forums, which only leads
to a login page. Undeterred, it followed the "if at first you don't
succeed, just ask again" school of thought: /post/ became the
most-requested page on the server by a mile, with 30.6 million
requests for 15.3 million different reply links, four times our home
page.
- URL handling: it joined URLs so badly that it invented its own. That
came to 16.5 million distinct URLs out of our links page in four
weeks, and news thread links glued end to end.
At its worst it took the server down: a kernel panic on 5 August, the
first unplanned reboot in more than 20 years. That is what finally
pushed the sites behind Cloudflare. On 7 August, Cloudflare counted 39
million requests from 18.9 million distinct addresses in a single day;
the busiest hour averaged about 2,500/second, and peaked at an estimated
5,000/second (it appears the authors of the scraper, human or machine,
don't grasp the concept of "moderation").
So your agents just had the bad luck to walk into the bot protection we
put up against all that. For scale, agents seeking the 2.2 source code
came from about 81,000 IP addresses over a whole year -- basically a
rounding error compared to the total of more than 30 million.
POV-Ray 2.2's unexpected second career
--------------------------------------
The three Official-2.2 archives (POVSRC.TAR.Z, POVDOC.TAR.Z,
POVSCN.TAR.Z) used to be fetched a few dozen times a month, mostly by
people feeling nostalgic. Then, on 5 September 2025:
Distinct addresses since then at least 81,000
Served over HTTP (Sep 2025 - Aug 2026) ~5 million, about 2.4 TB
Busiest day (7 Jan 2026) ~118,000 downloads
Served over actual FTP ~355,000, about 170 GB
Before September 2025, per month 0-9 by FTP, a few dozen to
a few hundred by HTTP
Almost all of it was curl/8.5.0 and Wget/1.21.4, which is to say Ubuntu
24.04 containers, from just about every major cloud you can name.
Some agents are smarter than others
-----------------------------------
ftp.povray.org does in fact speak FTP, a protocol whose first RFC
predates the C language. Some agents worked that out, and some kept
hammering HTTP. Share of agents (by address) that got the archives over
real FTP:
Period FTP share What was going on
----------------------- --------- ----------------------------------
Sep 2025 - mid-May 2026 ~31% Both doors open (9,078 FTP vs
22,736 HTTP)
Mid-May - late Aug 2026 ~1% I'd put ftp. behind Cloudflare,
which doesn't do FTP. My fault
7 Sep - 4 Oct 2026 73% HTTPS had a bouncer, so three
quarters went round the back
(14,135 FTP vs 6,525 HTTPS)
Since 5 Oct (the fix) ~21% Front door open; 96% get the
files, nearly all on the first try
Other observations from the logs:
- Thrift: the FTP users averaged about 15 archive downloads per address.
The HTTP crowd averaged about 58.
- Persistence: 80-85% of the agents that tried FTP got the files in the
end. One made 4,704 attempts in a single FTP session. (Many of those
failures are probably at our end: our FTP server rejects data
connections that come from a different address than the control
connection, which cloud NAT does a lot. I'll look at it.)
- Imagination: povray-2.2.tar.gz has been requested from at least half a
dozen directories where it has never existed.
A modest proposal
-----------------
Finding and compiling a 33-year-old C program is one thing. Finding it,
compiling it, and making it render 10x faster, fully autonomously, is
another thing entirely. So here's a cheeky challenge for the next
version of the task:
- the same container (1 CPU, 2 GB);
- a fair scene: illum1.pov at 640x480 with the task's own settings, or
any 2.2-era benchmark scene you like;
- the output must still pass your SSIM check against the reference
image;
- no turning features off and no lowering quality;
- and no, quietly substituting POV-Ray 3.7 does not count.
Bonus points if it doesn't download the source 4,704 times first
(looking at you, agent [IP redacted]!)
The files are open and cached now, so please use HTTPS; it's quicker for
everyone. And next time, just ask. We're friendly, and we've been at
this since 1988, when it was called DKB-Trace and passed around on
bulletin boards, 2400 baud was fast, and almost nobody had heard of the
internet. Odds are most of the people building these agents hadn't been
_born_ yet.
And if your agents enjoyed the 1993 source, the current one is here
(stars welcome):
https://github.com/POV-Ray/povray
A few stars might even get my 63-year-old backside off the couch and
into a new release. There are PRs waiting to be merged, and a Claude
instance that's starting to feel neglected.
Post a reply to this message
|
|
povray