I have really enjoyed collecting street-level imagery over the past year, but with the size of my collection, the time has come to set up a BikeSpace Panoramax instance.

I am currently uploading to two services: Mapillary and Panoramax. Mapillary is owned by Meta, but open-license, unlike Google StreetView. Panoramax is fully open source and open license, and relies on a federated network of instances to host photos.

Panoramax’s federated model makes it possible to have street-level imagery without the resources of a big company to host and maintain it. As long as it’s feasible for individuals or smaller groups to set up and host instances, the platform can still grow over time without compromising its free and open values.

Why Coolify?

To make it easy to start your own instance, the Panoramax team has extensive documentation on how to set up and host the service, with tutorials on how to run it with docker compose. The docker images have a lot of customizability, so you can set up your instance to your needs without needing to maintain a fork of the core software.

To make it even easier, I wanted to try setting up a deployment that could be run using Coolify, an open source and self-hostable alternative to platform-as-a-service (PAAS) providers.

In addition, I had a couple guiding decisions:

  • Design for a user who wants to run a relatively small, invite-only or private instance, without a lot of manual maintenance.
  • Build on the keycloak authentication example from the Panoramax docs to enable the private / invite-only use case.
  • Use S3-compatible object storage to find a decent balance between low maintenance, price, and performance.
  • Include thorough and easy-to-follow documentation on setting up an instance.
  • Build in automatic backups and include documentation on how to restore from backup.

Our set-up and documentation is available on GitHub at bikespace/panoramax-coolify.

It turned out to be the perfect timing to set up the BikeSpace instance, since the panoramax command-line tool and API just added functionality to transfer photos between instances while keeping the sequence and photo IDs the same.

What Does It Cost?

There have been some good blog posts from others who have set up Panoramax instances recently. In May, Matija Nalis wrote a post about setting up the OSM-HR instance for Croatia using donated hardware and a colocation sponsor. In July, one of the maintainers of the Fulda instance in Germany wrote a post about their set-up (in German), which uses a Hetzner VPS and storage box to run an up to 1 terabyte (TB) instance that only costs EUR €8.50 (~USD $9.80) per month.

Some info and stats on the BikeSpace instance, and then the pricing:

  • I have Coolify running on a VPS (Virtual Private Server) hosted by OVHCloud in Canada.
  • The VPS specs are equivalent to the current OVHCloud 2027 VPS-2 with 4 cores, a 75GB ssd, and 8GB of RAM, though I’ve used the 2027 VPS-1 for pricing below, since the BikeSpace VPS is also being used to run several other services.
  • The instance has 80,480 images, all 360-degree photos taken with a GoPro Max.
  • The production S3 bucket has 603GB of photos, including the HD originals and derivates such as thumbnails, standard definition copies, and 360 panorama tiles.
  • The backup S3 bucket, which stores database backups and copies of the HD originals, has 250GB of photos and files.
  • Sales tax in Ontario, Canada is 13%.

Here are the costs, including sales tax 1:

Item Cost (USD) Cost (CAD) Description
VPS $5.01 $7.01 OVHCloud 2027 VPS-1
Production S3 $4.43 $6.20 OVHCloud Object Storage
Backup S3 $1.98 $2.77 BackBlaze B2
Total Monthly $11.42 $15.98  
Annual $137.04 $191.76  

If you scale the costs to 1TB of photos, the first TB is USD $15.64/mo, and then USD $10.63/mo per each additional TB 2:

Item Cost (USD) Cost (CAD) Description
VPS $5.01 $7.01 OVHCloud 2027 VPS-1
Production S3 $7.35 $10.28 OVHCloud Object Storage
Backup S3 $3.28 $4.59 BackBlaze B2
Total Monthly $15.64 $21.88  
Annual $187.68 $262.56  

If I try to price out the set-up from the Fulda instance today, I get USD $8.01/mo for the Hetzner CX23 VPS and USD $4.52/mo for the Storage Box, so the VPS and S3 storage for 1TB above (excluding backup) are pretty much in the same USD $12–13/mo ballpark 3.

Considering the cost of a GoPro Max and the requisite Micro SD card, these running costs seem to be in a feasible range for a person or small group who are interested in running their own instance.

There’s also the RAMmageddon factor: one the one hand, these prices are still in a feasible range despite significant price increases in these services. Plus, capacity seems to be a real challenge for affordable providers like Hetzner: as of the date writing this, the CX23 VPS the Fulda instance is using is marked as “unavailable”. But who knows what the future will hold; hopefully as much good news as bad.

One other cost worth noting: including tax, I paid USD $236 (CAD $330) for a 5TB backup drive 4. In retrospect, I probably could have gotten a smaller one, but this is a one-time cost to factor in if you want to fully implement the 3-2-1 backup strategy.

Other Options I’m Curious About

For self-hosting, Dokploy and Dokku are other open source options for a self-hosted PAAS-like experience, both with seemingly substantial communities behind them. I ran into a number of weird edge cases deploying via docker compose on Coolify, and I wonder if the deployment would be more straightforward to adapt on any of these other open platforms.

For instances below the size where colocation makes sense, it might be possible to run cheap storage using hard drives at home and making them available to the web with something like Tailscale. If you try this, please write about it!

When looking at cloud storage options, block/volume storage always seemed to be in the range of CAD $60+ per TB/per month. I assume that the significant jump up in pricing is because you get an SSD on the other end of block/volume storage, but hard disks on the other end of most S3 storage. I assume that “hard drives” is also why the Hetzner Storage Box product is so cheap, in which case, it would be the only non-S3 option I’ve seen so far to get cloud hard drive storage. If there are other hard drive–based cloud storage products out there, I would be interested to see how they compare.

I’m also curious where the tipping point is when colocation does start to make sense, and you can get better value with second-hand or donated hardware. From the posts I’ve seen, the big OSM-FR Panoramax instance gets a remarkable cost per TB by buying and hosting second-hand hard drives. It would certainly be something I would look into if this instance ever got close to double-digit TBs of images. I suspect that having imagery of every official bike route in Toronto is doable in under 2TB, so this may only something to consider if we grow to having contributors from multiple cities across Canada.

Closing Thoughts

At various points during the process of getting this deployment running 5, I wished that Panoramax was more centralized, with some organization running an instance I can just use and donate to every year. After all, part of the appeal of doing street-level imagery was to spend more time exploring the city on my bike and less time coding.

There’s a strong case in favour of scale and efficiency: with One Big Instance, you would have a much better cost per TB, and server administrators who know a lot more about what they’re doing than I do. But looking at the current situation with Overpass servers, there is a clear danger to becoming the One Big Instance that everyone relies on without also having an organization that has the desire and resources to sustain it.

So I think the core Panoramax contributors are doing the right thing by making sure that OSM-FR doesn’t end up being the One Big Instance and encouraging the explosion of small instances we’ve seen this year. A successful and sustainable federation needs to have multiple centres of gravity. I hope that our example provides one more foothold to help others feel like they can host something themselves.

It reminds me a lot of what Ilya Kreynin said at Civic Tech Toronto in his talk about the nature of Civic Tech projects and the dynamics of doing something together as volunteers 6:

[For organizations run by volunteers], it’s not that money doesn’t matter, but money isn’t gravity. It’s not the main thing that is holding them down… because of that, they like it when they have more people, even if that set of people is doing the same amount of work.

…by giving people the opportunity to contribute, it becomes healthier, more vibrant, more reflective of a community, and it builds those people up. … The size of the people in your organization is an asset, not a liability.

Here’s to growing the Panoramax community, one instance at a time.

  1. I’ve assumed an exchange rate of CAD $1.40 per USD $1.00 for the currencies. ↩

  2. Of course, if your traffic grows alongside your storage, you may need to upgrade your VPS at some point as well, which will add to the marginal cost per TB. ↩

  3. I’m assuming for comparison’s sake that sales tax applies in both cases, though it looks like Hetzner currently doesn’t charge tax to Canadian customers. ↩

  4. Funnily enough, I did this while I was on a bike ride to record imagery in north-east Scarborough. ↩

  5. Such as: getting to know the PHP-infused oddities of how Coolify handles docker compose files, tediously setting up yet another S3 bucket for testing, etc. ↩

  6. Relevant part of the video here: https://youtu.be/mI70wtkUGTI?si=O-0g8FTisobgscJF&t=676 ↩