my dear sysops: where do you store backups?

i’m looking for something below 2EUR/TB, because otherwise it’s just more cost-effective to buy drives and create a small backup server…

the whole thing would be in the ballpark of 30-40TB, so if you want to suggest hetzner storageboxes… uh…

@domi perhaps backblaze? https://www.backblaze.com/cloud-backup/pricing

@domi My backups are on a cheap 5T usb drive connected to an intel nuc (celeron 3050) from 2016.

My offsite backups are on an identical drive, connected to an Atom D525 in my moms house. Yes, that atom box was built when I still lived there and has been running 24/7 for over 15 years (not with the same disks tough).

@Peetz0r doesn’t help much, i want to avoid buying drives –in this economy– if at all possible

@aelin that was my first thought, it’s horribly expensive and they will DELETE your backups if you don’t touch them for 60 days. it’s unusable

@aelin or at least that’s what i read the last time i looked into it

@domi aww sad, but TIL

@aelin https://old.reddit.com/r/backblaze/comments/131c86a/when_do_backed_up_files_get_deleted_the_reality/

i’m getting mixed signals right now. i guess it’s 30 days not 60? but not really? but yes really? i would need to re-read their service agreement

@aelin @domi Their “unlimited cloud backup” doesn’t have linux support. This is intentional (because it would destroy their pricing structure)

@domi Oh crap, I hadn't noticed that even low-end mechanical drives had increased in price too.

...ignore everything I said unless you can get a time machine.

@LilaHexe @aelin well, everything can have linux support if you bash it hard enough. that’s not necessarily a problem

@Peetz0r generally the trick is to buy 2nd-hand 1-2TB drives. they’re around $10 per TB, which I consider to be quite good. everything else doesn’t make sense price-wise

this is REALLY sad because 5 years ago it didn’t make sense to buy anything smaller than 4TB…

@domi I hate that I'm suggesting this, you're probably going to reject this out of principle, but aws glacier.

Tossing your numbers into the aws calc it comes out to roughly 30 $/month + $1.5 for an initial upload.

Keep in mind that a restore would cost you over $150 so

@Mae jfc that restore cost

@domi I am assuming here you're restoring the full 40TiB in a catastrophic end of the world scenario, smaller restores cost less but yeah it's fucking expensive

@domi how you feel about rsync.net ?

@Mae yeah i gathered that much, but restoring 1/4 of it would already put me at $40. so i can’t even make a test restore without throwing out money in the ballpark of three pizzas

@pj rsync.net / rsync.com seem somewhat shady to me (not very transparent about pricing, and the fact that they squat THOSE domains…)

@domi what you can do here is initially upload your shit to a normal s3 tier, and then have a lifecycle rule that moves it to glacier archival after a day, so you can test your backups without going broke, but yeah

@Mae approaching rube-goldberg territory…

@domi I don't think there is anything cheaper than glacier for raw /TiB cost

@domi rsync.net is not rsync.com tho?

@pj plural

@domi yes

@domi I'm pretty sure that they write your data to a tape then bury it out back to save space

@domi groans AWS Deep Glacier (yes, I’m aware)
ahh, already suggested, nevermind

@pj plural as in my comment applies to both…

@domi @pj i find the domain squatting very eh, but i don't see what's not transparent about pricing? at least to me it seems clear X per month per GB, with different brackets for scale

i didn't yet sign up but was gonna once i fixed my setup for backups

@navi @pj status update: i’m blind and didn’t notice the huge “pricing” page

okay that seems a bit better

@navi @pj still. it’s 1000 cents per terabyte. aka $10. which is 5 times as much as my limit

40TB would cost me $400. that’s enough to buy the drives in two months. maybe three in THIS economy

@domi @pj i didn't quite look at op ~w~

sorry i was more trying to understand what seemed shady about the pricing than +1'ing rsync.net for that usecase

@navi @pj is alright. admittedly it took me way too long to parse all of this anyways. i should go to sleep

@domi In my experience anything other than local storage is going to be prohibitively expensive
I even checked my employer and unfortunately we're at a terrible corporate 250GB/$5 pricing

I did find this though, haven't looked deep into it: https://pulsedmedia.com/storage-boxes.php
Looks like terribly slow egress, but the pricing feels wild

@domi @Peetz0r yeah but power usage is way higher with many 1-2tb drives over fewer 4-6-even more-tb drives

@domi I heard Mega S4 (aka S3 storage but they wanted to be fancy bout it) is actually kinda okay?
And 40TB comes to around (had to convert from PLN, in it its 456 zł) 107 euro a month which would be close to 2 EUR/TB/month

@liv @Peetz0r don’t care? i can swap them one by one? at this point we’re way beyond “automatic backups”, it’s a post-apo “what can hack up to make my thing work”

@Remiberry @domi huh, never heard of this finnish company. i use backblaze because last time i looked around i couldnt find anything with competitive pricing, but their software sucks and im tempted to try this

@domi it was $7.50/TB but a KTN-STL3 I only had to pay for once
(I have two on opposite sides of the country)

@izzy well, but then you still pay for upkeep (if the drive fails, etc) and power… not to mention that you probably bought those drives before the prices went up late last year. it would be 4x as expensive now

@domi @aelin it's 1 year now (for files which are no longer visible to the client or if the computer dies).

But you also only get the "unlimited" plan if you can run the (Windows or macOS) client. Otherwise you need to use B2 at $6.95/TB-month.

@domi yeah now an array with 2TB drives is more expensive x-x
hoping to sit out the current price situation by leaving 1/3 as cold spares

@Mae @domi how did you get $150? Last time I checked, just getting data out of aws into the broader non-aws world was like $100 per terabyte of traffic, on top of all the S3 charges.
(And same for azure and gcp, so I was going to use hetzner storage boxes for backups.)

@domi @Peetz0r aaah, i’m still at “i’d like automatic backups because i know i’ll mess it up otherwise”

@IngaLovinde @domi
Youre right, I'm used to dealing with shit that runs in aws already, so this isn't a problem, but I forgot egress. With egress it comes out to over 4k, Jesus Christ

@domi I actually have 5x2TB drives that I could possibly part with... (TOSHIBA P300)

@domi currently I just don’t for a lot of data (due to a lack of places to store them). I think if I had that much data to back up I’d consider getting a tape drive.

@domi That amount of data makes it rather difficult :/
I went with an OVH KS-Stor server with 4x4TB drives for 28€/month.They also have it with 4x6TB, but then it's 48€/month.
Their newer SYS-Stor can do 4x6 and 6x6 configs, but those are 66€ and 86€ per month already and only the smaller one is available at all atm and only in Canada

@domi I'm not aware of any service even close to that pricing tbh

@me then don’t reply.

@MarkAssPandi okay that makes it much better. still not great, that’s more than my whole infra combined, but better

@domi I thought I'd help align expectations ¯\_ (ツ)_/¯

@domi
Hm, my backups cost me significantly more per TB than that. I was looking at my other options for storing a lot of data and the only one which makes sense is building my own backup server.

Which still hurts with disk prices 🥲

@pj @domi and I would have a machine to put it in (Dell H310 with 6/8 slots populated, 5 SATA ports free on the board). It's a Haswell i3 with 12GB of DDR3 ECC RAM.

Last time I checked on it one or two drives needed replacing, but I would have ~12TB free after sorting minitokyo archive (I have roughly 4TB of my personal data, books, music, projects and stuff).

With your drives we can throw in one more for RAIDZ2 and have ~20TB of space for backups, just need to find co-location for that box

@elly @pj

just need to find co-location for that box

unless you’re hosting it under your bed like a real hacker, it’ll cost more than the value it provides

@domi tarsnap is good but way too expensive for this

@domi I’m not sure of the price, but I use some of Infomaniaks other services, which are not at all expensive. They have Swiss Backup which might be relevant to check out: https://www.infomaniak.com/en/swiss-backup

I love having my service at a European Company I can actually call on the phone.

@mosgaard i’m really impressed with their pricing. negatively. it’s yet another service where the price per TB goes up the more you order, which I find completely hecking baffling

@domi I formerly would’ve suggested Hetzner Auctions for that range. Needless to say, prices have gone up

(I did the math and you could get ~2EUR with high utilization of a Hetzner auction box. But you can’t with 2026 pricing, so just build it at home, I guess)

@kouhai got another person suggesting the same thing, it’s definitely on my mind ^^

@domi also free 5x egress so in need of restore it shouldn’t be a problem

@domi @Peetz0r @liv that doesn’t really make sense to me? with german power costs, small power savings can quickly dwarf initial expenditure

@domi couldn't really find their pricing on the Backup, only use the kSuite Pro myself, which is completely decent in price for 3TB, so I kind of thought it would stack up that heavily.

@mosgaard https://www.infomaniak.com/en/swiss-backup/prices

@domi nooooowaaayyy! That's a heavy price model!

@domi tbh one of the reasons why i don't like this idea for myself… is that you lose a lot of the savings if you need to set up raid/parity

@domi
Bash something hard is a phrase I'd use more from now on

@aelin @LilaHexe

@navi @pj @domi while still too much, it’s $0.008/gb on the “borg” plan, so like, $8
and backblaze b2 is like $6.95/tb so not much better
wasabi.com does similar pay-as-you-go, but free egress, and if you reserve capacity upfront instead of pay-as-you-go then there’s extra discounts (they don’t say how big but claim significant discounts, have to reserve at least 25tb for at least 1 year tho, it seems, and a bit of a “contact sales” thing)

but AWS S3 Glacier
that is such a pain to calculate the pricing, but it’s the only thing i know that’s actually in the desired ballpark
the actual storage part’s super cheap but there’s all sorts of extra fees for putting and updating and deleting objects and stuff
flexible retrieval storage is like $0.0036 per GB, deep archive is like $0.00099 per GB, but deep archive has like minimum 180 day storage per object and deleting before is pro-rated
whether it’s even considerable depends a bit on the usage style, if most data stays same after initial sync, and restores are very infrequent, then it might make sense as emergency backup…might mess around w the https://calculator.aws

@Mae @domi maybe punchcards or microfilm 🤔

@domi OVH cold object storage is about that price, scaleway was also suggested.
Another option is your own tapes, if you can get access to a drive. You could make use of the library at hswaw. Still leaves the problem of moving and storing the tapes physically somewhere tho.

@implr what’s the status with the tape library? because if it’s ready then investing in 5-8 tapes and queueing some backups every time i’m in warsaw and can physically take the tapes with me isn’t that bad of an idea..

@domi it works. it has two lto-6 drives in decent condition. idk if there's for 8 more tapes but some of the old stuff could be moved out.
there's not a good shared access story (without having root on a specific machine), but we can figure that out

@implr the “8 tapes” was more like… max 4 in use at once, rest swapped out when i come around”? and it was a ballpark figure, i don’t remember how much data they take

re access; i’m very friendly and i can pinky promise not to break stuff (: or we could pipe something over the network. i’m flexible

@domi @implr there's two drives, each with their own FC, so it should *technically* be possible to plug one of them into a different machine if that's what we wanted to do (requires a second FC HBA tho)

@radex @implr PCIe ports are the only thing i don’t have a shortage of in sakamoto, so. that’s also an idea

@radex @domi I have no idea how arbitration works with multi-host fc devices but yes, it should work. Or we could ebay a junk fc switch

@implr @domi junk fc switch sounds like something that would waste a non-significant amount of electricity :/

@radex @implr tasmota and chill?

@radex @domi behold, złom:
https://allegro.pl/produkt/brocade-200e-fibre-channel-switch-19a662c8-8abe-4218-945d-e79fddffe20f
allegedly 50w
alternatively, Enterprise Złom (also 50w)
https://allegro.pl/produkt/switch-ibm-2498-b24-8gbps-24xport-24-activ-2f61446a-e0a5-447d-932d-bcab5bb26ea9?offerId=18480278415

@implr @domi no jo, ale 50W to jest ~50zł/mies w prądzie. Chociaż indeed można by to wyłączać when unused (razem z samym tapelib który bierze 45W a jest mostly unused I guess?)

dedicated DC zigbee-to-mqtt perhaps?

@radex @domi perhaps
ale najpierw bez, może zadziałała na ringu albo multiporcie, topologie FC są dziwne, pls engage autism and find out

@implr @domi not curious enough to engage autism, ale jeśli domi chce wykminić jak to zrobić i nie będzie u nas sprzeciwu do implementacji to mogę fizycznie poprzepinać kabloze

@implr @domi a, jeszcze jedna opcja do rozważenia to przerzucenie HBA do snowflake i zrobienie tam VMki dedykowanej do dotykania tapelib