Field notes #2
Sprites and CDNs
One of the hardest technical problems behind Aevocam wasn't storing decades of photos. It was making them play back smoothly like YouTube while still giving users the fine control of a security camera interface. Single frame stepping. Pinch to zoom. Hand scrubbing. And keeping it continuously up to date as new photos arrived. It wasn't a given.
Sprites
I'll skip a lot of the early experiments and say that I settled on sprites and CDNs. If you've played old-school video games then you've seen sprites. You start with many hand-painted images of, say, Mario Bros characters running or jumping. Rather than have each frame represented as separate image files, you gang them all together on a single image like stickers on a strip of paper.
It turns out the same trick works surprisingly well for long photo histories. When you first open Aevocam's player, one of the first things your browser receives is a low-resolution sprite sheet. It will contain up to 400 photos. Usually that arrives in a fraction of a second. Then higher resolution versions. Then eventually the full-resolution photos uploaded by the source camera.
That's why you can start scrubbing immediately instead of waiting on a black screen like with YouTube. It's there right from the start.
Content delivery networks
The other piece of the puzzle was using a CDN. This is my first time using AWS CloudFront for mass content like this.
If you aren't familiar with CDNs, requests go to a nearby CloudFront server instead of one distant server. The photos live in an AWS S3 bucket. CloudFront fetches them and delivers them around the world.
Here's the part that blew my mind. You'd think adding another server in the chain would slow things down. It often doesn't. Even before a photo has been cached, CloudFront can often retrieve it from S3 faster than your browser could. Amazon has built an incredibly fast network between its own datacenters. Much faster than the connection most of us have into AWS. I don't know about you, but I find that fascinating.
The results
In my own tests I find the CDN can easily deliver a 300 KB+ high-res photo to my browser in about 200 ms. Not bad by itself. But it does so while delivering hundreds of other photos at the same time.
Sometimes I have to stop and remind myself just how absurd this all is. We're casually throwing around hundreds of megabytes of imagery like it's nothing. Twenty years ago I'd have considered this hopelessly extravagant. Today it's just another Tuesday on the internet.
But I'm also aware that I only got to build it because so many other engineers solved some serious infrastructure problems first. Twenty years ago this would have taken a large dev team to build and cost a lot of money. Today one engineer can put the pieces together in a few intense months. And deploy it on the cheap.