File hosting changes

SpindleyQ's picture

Hello friends,
The Glorious Trainwrecks server has been running into disk space issues for a while. Thus far I've been able to deal with them but it's been clear for a while that a long-term fix is needed. And I've built one!

If you are not interested in the technical details, just be aware that I made some changes to how the site works and if you notice anything weird or any downloads are not working, please let me know! Everything should be backed up in multiple places. I should be able to fix it.

If you ARE interested in the technical details: I am moving most of our hosted games to a service called Backblaze B2. B2 is a static file hosting service that charges me only for the space that I use, and is MASSIVELY cheaper than adding disk space to the server directly. I already have full daily encrypted site backups stored in B2, so I trust it with your games. That said, if I needed to move to a different service in the future, there are lots of options out there, and moving should be very easy to do now that the pieces are in place.

I've written a small PHP script that redirects requests to B2 if it detects that the file has been migrated. nginx is passing all requests for the /files/ directory to this script. I actually keep a 0-byte file around on disk for migrated files so that Drupal's auto file renaming logic should still work. I've done my best to make this totally transparent, but there may still be a bug or two that I haven't caught - please let me know if you see anything weird! I tried to keep things as simple and foolproof as possible.

At this time, migration is a process that has to be triggered manually, so I have full control over what files are migrated. I'm not moving screenshots because I'm pretty sure that would mess up thumbnail generation. Newly uploaded stuff will still be served locally until I need to clear up more space again. Which might not be for a very long time.

Anyway! That was too much information but this took up most of my day today so I wanted to share. I've always wanted Glorious Trainwrecks to be a place where your work could be stored for the long haul; this is just the next step in making sure I can continue to keep that promise to you.