The State of GIF on Tumblr
I recently got into a conversation about GIF uploading with QIL, owner of sukme and a GIF artist here on Tumblr about a post he made regarding his frustrations. He referenced this image and how he wished it could look have uploaded the more awesome version he put on Imgur. Below is what I sent back to him, with certain details sanitized. For instance, TOOL_X and TOOL_Y refer to open-source GIF tools, but I’m not going to name them publicly.QIL,Thanks for your email. Our limits are a bit more complicated that I think you realize, so let me explain:Size limits:We only allow 1MB GIFs to be uploaded. However, there are other limits on the size of the image when it is unpacked. More importantly, there are cases where the resizer that we use (TOOL_X) will produce a larger GIF than the one uploaded, even at the same or smaller resolution. The GIF resize process is pretty complicated and the frame transparency optimization even more so.Time limits:There is a maximum amount of time allowed for GIF resizing before we reject. This is most likely the limit you run into most often. You are astute to notice that we have taken a step backwards in terms of what we support: when we migrated data centers, we moved to more efficient but ultimately slower servers. For most traffic, we simply deploy more of these slower servers and it is a win for everybody. For GIF resizing the straight-line processing power of a single node is the limiting factor. I am pleased to report that we are in the process of rolling out a new generation of web servers that are significantly faster than what we currently have in production. I am not at liberty to disclose specifications or benchmark results as to how much faster, but I’ll just say that it is a lot faster. This should bring us far past our capabilities from Summer 2012.The image you linked to of the slapping, color-cycling image is hitting our time limits. I asked TOOL_X to resize both the Imgur copy and the Tumblr copy to 500x500 (which they already were. Here are the results on my MacBook Pro:
TOOL_X -resize 500x500 suk.me_tumblr.gif suk.me_tumblr.out.gif 18.88s user 0.26s system 98% cpu 19.372 total
TOOL_X -resize 500x500 suk.me_imgur.gif suk.me_imgur.out.gif 258.26s user 0.97s system 99% cpu 4:21.23 totalOuch! 18.88 seconds is bad. 258.26 seconds is insanity.Quality:As I previously mentioned, we use TOOL_X for resizing. There are several other tools out there that can be used on a server, the best of them being TOOL_Y. It is much faster than TOOL_X and produces much more optimized GIFs. However, it has quality problems. Specifically, there are black pixels that should have been transparent. As much as we all would love to switch converters, we simply haven’t found any that are available for batch processing that do as well on the wide variety of GIFs we see as TOOL_X.For the fun of it, I also resized (and optimized) the GIFs using TOOL_Y:TOOL_Y —resize 500x500 -o suk.me_tumblr.out.gs.gif suk.me_tumblr.gif 0.46s user 0.02s system 97% cpu 0.487 total
TOOL_Y —resize 500x500 -o suk.me_imgur.out.gs.gif suk.me_imgur.gif 0.45s user 0.01s system 95% cpu 0.490 totalWell, that kind if says it all… except about image size:-rw-r—r—@ 1 1013814 Oct 24 12:42 suk.me_imgur.gif-rw-r—r— 1 2166779 Oct 24 13:55 suk.me_imgur.out.gif-rw-r—r— 1 1013780 Oct 24 13:56 suk.me_imgur.out.gs.gif-rw-r—r—@ 1 995853 Oct 24 12:43 suk.me_tumblr.gif-rw-r—r— 1 2249449 Oct 24 13:55 suk.me_tumblr.out.gif-rw-r—r— 1 995364 Oct 24 13:56 suk.me_tumblr.out.gs.gifIn other words, the TOOL_X versions took a LOT longer to produce and are actually bigger than the originals. On these particular images, the TOOL_Y versions look great. Sadly, this isn’t always the case.Money:You mention that Imgur allows you to pay to elevate limits. You have my solemn promise that money is not the problem here. If we had a $1M solution to the problem, we would pay it with smiles on our faces. At our scale, there are no simple solutions. We could do better by purchasing faster machines, but we all know that isn’t the real solution. Doing it in a way that can scale to our volume (100M posts/day) and with images as complex as yours — that is the problem.Imgur has another advantage over us: Resizes do not need to be immediately available. Because your followers are supposed to see your post pretty much immediately, that’s a problem! And if some are on mobile, some are hitting the dashboard, some are hitting your blog directly where you may have a theme that uses larger image sizes, we really need to have all of the resizing done as part of the posting process.What’s Next:Well, that’s unfortunately not clear for us. We are rolling out our new generation of nodes. That will help. The TOOL_Y library/app is improving and something we will be keeping an eye on because it is 99% awesome and new tools for GIF are rare these days. There are some big plans around static (non-animated) images that are going to make that side of things better and possibly reduce contention for CPU, further expanding our capacity for speedy GIF resize.I would love to be able to tell you and all of the awesome GIF artists on Tumblr that we have a solution rolling out tomorrow. Sadly, it just isn’t the case. We are still working hard to balance quality and performance so that both producers and consumers of these images have the best experience possible. Things will get better in the near term. Great is going to take a little longer.