The subreddit /r/AnimeThemes holds about 10.000 anime openings and endings. This scraper reads each wiki page and parses them. The found themes will be downloaded and compressed whereupon there are stored in the filesystem and it's metadata in the database. Also, an anime a theme belongs to will be fetched from MAL. Furthermore, themes can be grouped by popularity and rank through the anime metadata. Because of this, the scraper can guarantee a seamless song stream without any rate limits for the bot.
We try to reduce our requests to Jikan and AnimeThemes because we don't want to put their free service under heavy load. Requests are delayed with timers and already scraped songs are not fetched every time.
Install the Docker image
docker pull shirodiscordbot/scrapermkdir /root/themesdocker run -d --name scraper --restart=always --env-file /root/env.list --network="host" -v /root/themes/:/src/themes/ shirodiscordbot/scraper