web/scripts
Robin Whittleton 71b675331a Fix AVIF images in iOS16 (and 8+ times speedup of AVIF generation)
iOS16 upholds AVIF standards more strictly than other renderers, which means that the current AVIF images on SE are broken. A fix for generation bug has been added to go-avif (https://github.com/Kagami/go-avif/pull/17) but we can do better.

This PR switches out the AVIF generation to cavif (https://github.com/kornelski/cavif-rs) which not only doesn’t suffer from the iOS16 but drops generation time by more than 8 times. For example, here’s a set of three timed runs in both go-avif and cavif on the title card of https://standardebooks.org/ebooks/jonathan-dymond/an-inquiry-into-the-accordancy-of-war-with-the-principles-of-christianity:

```
root@f58fb0beac90:/standardebooks.org/web/scripts# time ./go-avif -e "illustration.jpg" -o "illustration-go-1.avif" --best

real	0m31.191s
user	1m39.113s
sys	0m1.662s
root@f58fb0beac90:/standardebooks.org/web/scripts# time ./go-avif -e "illustration.jpg" -o "illustration-go-2.avif" --best

real	0m33.097s
user	1m49.846s
sys	0m0.831s
root@f58fb0beac90:/standardebooks.org/web/scripts# time ./go-avif -e "illustration.jpg" -o "illustration-go-3.avif" --best

real	0m36.963s
user	2m4.235s
sys	0m0.711s

root@f58fb0beac90:/standardebooks.org/web/scripts# time ./cavif --quality 50 "illustration.jpg" -o "illustration-cavif-1.avif"
illustration-cavif-1.avif: 98KB (96840B color, 0B alpha, 242B HEIF)

real	0m4.017s
user	0m11.600s
sys	0m0.051s
root@f58fb0beac90:/standardebooks.org/web/scripts# time ./cavif --quality 50 "illustration.jpg" -o "illustration-cavif-2.avif"
illustration-cavif-2.avif: 98KB (96840B color, 0B alpha, 242B HEIF)

real	0m4.083s
user	0m11.857s
sys	0m0.091s
root@f58fb0beac90:/standardebooks.org/web/scripts# time ./cavif --quality 50 "illustration.jpg" -o "illustration-cavif-3.avif"
illustration-cavif-3.avif: 98KB (96840B color, 0B alpha, 242B HEIF)

real	0m4.237s
user	0m12.339s
sys	0m0.093s
```

The executable has been generated via cargo, and has the following qualities:

```
root@f58fb0beac90:/standardebooks.org/web/scripts# file cavif
cavif: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=f972071996c58b7a855d04e4c5804f4a3d94f575, for GNU/Linux 3.2.0, with debug_info, not stripped
```

I’m not sure how cavif’s quality 50 matches go-avif’s best flag, but they produce similar file sizes and similar visual quality.

There is a slight different in apparent colour space. If this is an issue there are tweaks, but according to the documentation it makes files larger so should be avoided. Given that the titlecards will matches the covers I see no reason to go down that path.
2022-09-18 22:01:26 +02:00
..
cavif Fix AVIF images in iOS16 (and 8+ times speedup of AVIF generation) 2022-09-18 22:01:26 +02:00
delete-unconfirmed-newsletter-subscribers Convert newsletter to use Users table as base 2022-07-04 12:09:49 -05:00
deploy-ebook-to-www Fix AVIF images in iOS16 (and 8+ times speedup of AVIF generation) 2022-09-18 22:01:26 +02:00
generate-bulk-downloads chown/chmod files during deploy within their correct scripts, instead of in the parent deploy script 2022-07-13 11:25:06 -05:00
generate-feeds chown/chmod files during deploy within their correct scripts, instead of in the parent deploy script 2022-07-13 11:25:06 -05:00
process-pending-payments Remove logging code 2022-07-24 18:33:07 -05:00
README.md Normalize contents of webroot variables 2022-07-09 20:09:39 -05:00
rebuild-cache Rebuild web caches immediately on ebook updates 2022-07-14 15:28:57 -05:00
reset-php-fpm-opcache Change the way we deploy ebooks from destroying the entire apcu cache and rebuilding on the next web request, to rebuilding in the background 2020-02-29 22:29:19 -06:00
rotate-www-logs Anonymize web logs after rotating and disable explicit download logging in favor of grepping the regular web logs 2022-03-16 13:04:52 -04:00
sync-ebooks Added to allowed characters in tokens 2022-01-12 08:53:31 -06:00
update-patrons-circle Create cookie-based login and authentication system 2022-07-10 20:53:20 -05:00

sync-ebooks

To use, call this script with the directory where your ebooks go as its last argument. For example sync-ebooks /standardebooks.org/ebooks, or if you want to clone them into this repository, sync-ebooks ebooks. If you want progress output, use -v, and if you want detailed Git progress output use -vv. GitHub allows you to make 60 unauthenticated API requests per hour. If you use unauthenticated API requests for other things, this might not be enough, so to resolve that issue, you can create a new OAuth token at https://github.com/settings/tokens/new and pass it via the --token option. You dont need to give the token any permissions.

deploy-ebook-to-www

To use, call this script with the directories of the books you want to deploy as its arguments. For example, to deploy all ebooks after using sync-ebooks, run deploy-ebook-to-www /standardebooks.org/ebooks/*. To deploy only The Time Machine by H. G. Wells, you would run deploy-ebook-to-www /standardebooks.org/ebooks/h-g-wells_the-time-machine. To output progress information, use -v or --verbose.

The default web root is /standardebooks.org/web/www. If it is located elsewhere, specify it with the --webroot option. For instance, deploy-ebook-to-www --webroot /var/www/html /path/to/ebook. Note that there will be php errors if the Git repositories are not in the ebook directory immediately in the web root. Either keep them there or create a symlink.

The default group is se. to use a different one, specify it with the --group option.

The default URL is https://standardebooks.org. To change it, use the --weburl option. For example, deploy-ebook-to-www --weburl "http://localhost:8080". This option will cause deploy-ebook-to-www to use the specified URL in the generated OPDS and RSS files. Care should be taken however; the URL https://standardebooks.org is hard-coded in a few places, even when SITE_URL is changed to a custom URL, so for testing, it may be more convenient to simply use /etc/hosts or a similar mechanism to resolve standardebooks.org to 127.0.0.1.