Hacker Newsnew | past | comments | ask | show | jobs | submit | hiAndrewQuinn's commentslogin

> [S]ometimes the things I have the most passion and interest for are not the things that are [natural] strengths.

Whic implies that most of the time, they are.

And that's a good thing. If it took you years to grasp assembly and C, whereas e.g. asynchronous TypeScript is bequem for you the same way polynomials were bequem for David Hilbert in grade school, you would probably make more money, contribute more to the economy, and be an overall happier person overall working a job that is about 80-90% asynchronous TypeScript, and maybe 10-20% the interesting stuff you don't have natural talent at.

Exceptions exist to this rule but they face a double filter:

1. How are you so sure you know better than the people waving money in your face?

2. Even if you have a good reason, why are you the right person to be doing this? Wouldn't someone else whose strengths and talents already align be better still? Is it really impossible to find them and put them in that position instead?


This seems to place interest and talent as equivalent. The issue is that while I might be talented at TypeScript, I have zero interest in it. I know this because I got paid an unreasonbly high amount of money as an undergrad to write async TypeScript code and do full-stack development.

The conclusion is that I would not be an "overall happier person" from this, even if I could be great at it. Besides, long-term, I would not be great at it, since my lack of interest implies a lack of motivation to succeed at it. And it was a real waste of time when it came to my goals in systems.

Interest plays a big role beyond talent. I would feel more fulfilled being a mediocre dog-killer than an excellent something-else. Either way, interest feels almost equally as important as talent to me. Interest can sometimes make up for what I lack in talent. To succeed at what I want to do, I am more than happy to put in twice the time as someone with natural talent.


Obsession beats all. Being interested or talented help in different ways but obsession is the drive that shines talent.

I wish I had known about this site when I was writing [1]. If we use warranties as our expected lifecycle, this lets me drop down from $5 per TB-year of storage down to almost $2 per TB-year. What immense savings compared to the cloud!

[1]: https://andrew-quinn.me/digital-resiliency-2025/#postscript-...


Doing a self-audit like this is actually an amazing idea. I consider and re-consider my choices every once in a while, but sitting down and doing an end-to-end write-up would put me a lot more at ease.

Like you, I also considered the implications of mixing TOTP into KeePass, but eventually landed on going all-in on the one database. It does mean raising the bar for keeping it secure, but it was already very high to begin with.

One thing I have considered is combining this all-in-one approach with an additional keyfile, which I could then share OOB to devices, effectively adding a second factor. I like the idea of using Yubikey or similar, but the fear of locking myself out is too great.


I don't get it -- AWS deep archive is $12/TB/yr and provides actual durability and connectivity, not just drive-in-a-shoebox. That seems pretty hard to beat by buying raw storage at retail


AWS connectivity is stupidly expensive in the outgoing direction, so that connectivity may or may not be worth much of anything. Connectivity is also a risk.

Overall glacier is only really suited for backups, and I don't need that much durability for a single backup. And even if durability is a big deal, I can get there cheaper. Especially using a realistic expected life cycle and not the warranty period.


It's a little annoying to change from 3 to 2 to 1 columns when someone does resize, though. I just let people resize the window itself on my blog if they want to compress the text down.


Electricity prices are also token-based, in a sense, yet most people broadly agree this is the best way to price them.


I’m not following your point. What is the disagreement about Llm pricing?


Well, does anyone actually have a copy of the contract from 14 years ago? Usually there are clauses hedging against this kind of thing.

Example: I recently wrote the T&S for my Finnish dictionary app (still working on it), and I make it clear in advance that the license was a one time fee for perpetual use for that major version. [1]

I can do this because the app is almost entirely offline, and because for the parts that are, smart cloud infra decisions means my recurring infra costs are low. If I add in features which imply a bespoke server down the line, of course that would probably be a major version upgrade - and a change in the pricing model to boot. But I'd still keep the old v1 stuff up for the lifers.

[1]: https://taskusanakirja.com/terms-of-service/#91-pricing-and-...


IIRC they advertised themselves as "pay once, use forever" in their marketing. So why shouldn't they uphold that?


An advertisement is not a contract, unfortunately. If we're going to talk legal, we need to talk in legalese.


They are (or were, at the time they had that slogan) an Australian company. I am an Australian citizen. Under Australian Consumer Law, an advertisement is absolutely legally binding.

https://www.accc.gov.au/consumers/advertising-and-promotions...


So if an Australian ad tells something is "the best" or something similar and you can prove it isn't, you can get your money back?


Usually subjective opinion isn't binding (though I'm sure there are exceptions to this across jurisdictions)


The link I shared makes it quite clear that "puffery" that nobody is reasonably expected to take literally does not count.

Being told that the app you paid for would be a one-time payment, and then having the service deliberately degraded to try and force you into a subscription model, is clearly not puffery.


Well, if you actually want to learn, there is always the vast swaths of the Internet. Virtually everything here is free, and nobody will care until you do something impressive with your knowledge.

As for universities, they will likely stay as signaling mechanisms until society finds a more efficient way to signal the things that universities do. This is a worldwide pattern that has emerged, and to the extent you see deviations from it it's usually situations like e.g. getting into Tokyo University is already so incredibly difficult that some employers will just accept your letter of admission itself as a sufficient signal of your value to the firm and hire you and let you skip the whole getting a degree thing.

What does university graduation signal? Some combination of raw intelligence, conscientiousness, and ability to conform (not against the "I have beef with the standard model of physics" nonconformance, so much as the "I will not physically assault the professor for telling me I'm wrong in class" nonconformance). Admission to a selective university signals you had these traits even earlier and with greater strength than your peers.

I'm going to underline something from your own article here, which is that you went to an excellent university and got near the top of your class despite hating it. It is an incredibly rare psychological profile in the wild to be able to war-of-attrition your way through so many elite classes, while having virtually zero interest in the material themselves. Any employer would be drooling at the mouth to hire you because you sound reliable even in a pinch. Alas they cannot tell you apart from the ultranerd who gets all As because she genuinely finds all knowledge presented to her endlessly fascinating - but she's probably a good hire too, for different reasons!

But, almost by definition, you can't really signal that kind of ability if you only ever do things you want to do... And most of the things most people in the world want to do most of the time aren't very economically valuable from the doer's perspective. Everyone wants to eat, nobody wants to grow crops, etc.


There is definitely a lot of information on the internet, but I wouldn’t undervalue the benefits of being in a classroom with a small group of bright people focusing on the same topic every week.

Probably the best class I took for my philosophy degree was a 3 hour metaphysics course, held once every Wednesday. There were maybe 6-7 people in the class, and the discussions we got into were incredibly educational.

I don’t think reading a bunch of books and web pages about metaphysics would have been 10% as insightful. Maybe with talking to an AI, you could get that up to 20%…but still, it’s not the same.


I don't think I'm undervaluing it, I think it's just low expected value in the first place. Just because you had 36-40 hours of a good experience over a 3 or 4 year degree doesn't actually mean the median or even the 90th percentile classroom experience is better than self-education.


…I had more than one good class in my four years of college. It was an example.

Self-education is great but is also leads to a kind of idiosyncratic blindness, because no one really forces themselves to study anything that they aren’t already interested in.


I agree. I would even go so far as to suggest there is far more information availiable on the internet than you can get in a degree. Mostly it's inside pirated textbooks and academic papers.

I've been self-teaching cryptography since I graduated with an engineering degree, and it's amazing how woefully unequipped a degree program alone leaves you compared to the information that's just out there


Before anything one should probably check or at least ballpark their IQ score. The median IQ for mathematics PhD students probably hovers somewhere around 145, about the top 0.2% of the population, correlated with about a 1510/1600 on the SATs, a 34 on the ACTs, etc. Those aren't perfect correlates but you're much more likely to have an SAT or ACT score than a professional IQ score handy.

Math is infamously g-loaded, pure math even more so. An unfortunate fact of life. On the bright side, math is very much a "shoot for the moon and you'll land among the stars" subject to pursue if you even loosely keep industrial or business applications in mind.


I got rage baited by this so hard, cant comprehend thinking this way.

Hung out with PhD's, economists, bankers, trust find kids, scientists, and artists - who maybe weren't top tier enough, but none thought this way.

Literally the weirdest take on a forum filled with dreamers, but every take is valid.


It's not comfortable, but this seems to be what the priors point to. I suspect that pure mathematics is one of the most intelligence-dependent fields; one where hard work, practical solving problems and a large knowledge base is less of a substitute.


> one where hard work, practical solving problems and a large knowledge base is less of a substitute

Collaboration remains an important skill – I had an REU mentor who said that, given the explosion of mathematics that one had to learn to do cutting-edge work in a field, she had to end up "pooling experience..."


> one where hard work, practical solving problems and a large knowledge base is less of a substitute.

I have seen this first hand. I remember when I was in university doing my math major. This one older adult lady (she seemed 40yrs old, and very attractive too), she had decided for some reason or other she wanted to do a major in mathematics. Not for a job or anything but just to do it.

Whereas the rest of us, let’s face it, we just wanted a good job in STEM.

Bless this lady, she was so determined and hard working. She would show up to every lecture, first in, last out, and she would show up to every study session and give it her all.

But unfortunately, she was not good at grasping the concepts nor solving the problems. It was shocking how little she grokked the introductory concepts for the amount of effort she put in. She worked harder than anyone in our group.

I don’t think any of us had the heart to tell her that maybe a math major was not in the cards.

I never saw her on campus in my 3rd year and on so imagine she dropped off.

But I was rooting for her.


I think what happens is, IQ is a sort of speed. They teach the class assuming they can get most people through it at whatever pace they are used to teaching at. If you're quick, you can keep up with a below average effort, and vice versa.

As you get higher and higher up in the stratosphere, the balance between "we need enough students" and "we need to go faster" ends up favouring the few super intelligent people, along with the people who can arrange their lives to put in the hours.

That's not to say you can't learn something if you are slow. You just can't learn it at the pace they are teaching, and you might not have the wherewithal to learn it at your own pace.

So to you it looks like this lady would never learn it, but I would guess if she had a personal tutor they would be able to pace it.


Same lol. By the OP's logic, every student pursuing this field in a university as an undergrad/graduate student should be taking an IQ test before proceeding to the upper level math courses covering these topics. Anything less than the threshold will mean they have to focus on something different.


Two things :

> The median IQ for mathematics PhD students probably hovers somewhere around 145

Does that mean the 145 figure is only a guess on your end ?

Second, as far as I know, an individual's IQ is not something set in stone, and can absolutely be improved with training. I remember reading (that's an anecdote so correct me if I'm wrong) that rewarding a good score with money was able to improve the outcome by up to 20 points. It doesn't sound absurd to me that someone with a slightly above average IQ could get close to 140 after 6, 7 years of high level math training.


>Does that mean the 145 figure is only a guess on your end ?

EDIT: Mea maxima culpa, confusion crept in, my 145 number was supposed to be a much looser guess for actual working full time mathematicians. I miswrote this in the original post as applying to math PhD students, which are much lower. Closer to a 130 median.

ORIGINAL: It's not quite a guess, but I don't have precise data on this exact thing either. Previous studies in this field have consistently found a range of between 140 and 150, and you can probably find those with some Googling if you want to corroborate it yourself. I have a long cached memory of seeing a study where theoretical physics PhD students had an average IQ of 150, which also loosely supports this, since theoretical physics is almost its own form of pure mathematics.

>an individual's IQ is not something set in stone, and can absolutely be improved with training

Most psychological research I've seen says no such thing, unfortunately. Believe me, I would love for that to be the case - one extra point of IQ correlates to roughly $1000 extra income per year in the US, and so if your 20 point claim were really true we could potentially cause a double digit spike in GDP over the next few months just by implementing it in smart ways. But my baseline belief is that study is almost certainly an outlier in a sea of similar studies which support the null hypothesis.


Did a Google search, and the only actual, definitive thing I found was this:

https://pmc.ncbi.nlm.nih.gov/articles/PMC5008436/#tbl1

Note that it's an IQ of 128 vs 125 for humanities. With the small sample size, it's basically noise. And given that this is Oxford, I would expect the average PhD student to have less than these numbers.


Oh, hold on, I may have gotten my numbers mixed up slightly. 145 might actually be for working, full time pure mathematicians - sorry about that, I'm double checking now.

EDIT: My massive bad, it looks like I accidentally bumped everything up a standard deviation in my head somewhere. Jesus. I should update the numbers in the original post. Maybe I should also consider getting a math PhD after all, apparently I'd be ahead of the pack in that case.


Kudos to you for acknowledging your error and correcting it. We certainly do need more high IQ (>128) math PhDs with integrity, like yourself. Please do consider pursuing higher math in a full time capacity.


Maybe, but let say you're right. I still don't understand your suggestion of doing an IQ test before you decide to study math. If you can't go further than a masters, like you said there are still a lot of industries you can go too and have an interesting, lucrative job. And if you do succeed to finish your PhD then that's great news. There are no benefits that I can see in doing an IQ test like you suggested before you make your decision. If you love math and are good at it, chances are you're moderately smart at least. Might as well go as far as you can if that's what you want.


My experience with pure math is that this is not necessary to get job as one, even at good institutions, but you will be terrorized by the arrogance of the ones you mention. Learning to deal with the "brilliant jerk" is a problem in many fields, but the ones I've met in pure math are some of the weirdest (and most vicious)


At least from my point of view (in the industry, not academia) this is actually the opposite. Math graduates tend to be smart and humble and I respect them a lot. Sometimes it almost feels like math and physics are the last "real" degrees left.


Similar - I found math majors to be fairly humble. Yes, there's always the exception, but I found them to be fairly fun folks.

Physics majors, in my experience, had a significantly higher arrogance level.


I'm talking about professors at R1 schools and Ivies to be clear.


Yes, and those are the ones who didn’t make it to researcher.


Who's to say that you can't go into industry and not be a researcher? You don't have to stay in academia to do research. Many companies and industries tend to publish papers and some even work with universities for research.


No I’m saying the top tier mathematicians tend to work in academia because that’s where the most math is done (general trend with exceptions).


I think that is a fair statement to make. Thanks for clarifying :)


I don't think that's what GP was saying, but I could be wrong.


A corollary of this is that many professional mathematicians are not actually competitive in research.

It’s just different leagues of intelligence: social studies undergrad vs math undergrad vs math grad vs competitive researcher.


The definition of professional mathematics is research. That’s what they are trained in and that’s what they are competent at. I don’t understand your comment.


Lots of professors aren’t leading their field in research - they aren’t competitive with those that do.

So yes, they are teachers or administrators or make minor research contributions.


This is what I've observed as well. By my own metrics and grades, I was a somewhat bright math minor (near-perfect score in abstract algebra, etc), would have been middle of the pack as a PhD student, may have been below par if I managed to complete the PhD, and almost certainly would have been deadweight as a pure mathematician myself. That's just how the scaling and competitive dynamics have worked out; it's not really something to feel personally bad about, any more than you might feel personally bad about not having the potential to be a competitive figure skater.

EDIT: Uh, actually, it looks like I may have underestimated myself at basically every point here and would have become a basically okay mathematician based on updated priors.


The silly thing about this is that context is everything. I bet it's extremely easy to be a top-tier figure-skater in, say, a small tropical island nation? In a similar way, I very much doubt that you'd really need to be in the top 0.2% of the population to complete a phd. Do you need to be in the top 0.2% of people to compete as a contributor with absolutely everyone else in the whole world at the same time? Well yeah, but at that point the statement is so obviously true that it doesn't mean much.


You're right in a sense, but I took the context we're working in as somewhat of a given based on the title of the post. Our goal is to work, full time, as a professional pure mathematician; that naturally puts us in the labor market for pure mathematicians. We can't know that market exactly, of course, but it's far from arbitrary. We are in competition with other market participants, and we can study their properties and use that knowledge to guide our actions productively - including making the decision to exit the market if that's what makes sense to us.


> Our goal is to work, full time, as a professional pure mathematician; that naturally puts us in the labor market for pure mathematicians. We can't know that market exactly, of course[...]

For what it's worth, my classmates from college who have completed PhDs, based their postgrad career decisions on completely different factors – mostly their families and partners, and whether they're willing to move around (especially to rural areas) to target an extremely shrinking academic job pool.

EDIT: example that came to mind – I had a classmate who postdoc'd at Chicago, who decided to stay in town and work in finance rather than pursue some tenure-track offers at R1s, because his young one went to a prestigious UChicago Lab School and didn't want to uproot her.


The dumbest people I’ve ever encountered in university were the math and physics majors who thought they could score some easy points by taking humanities classes, because just like you they considered that below their level. I’m sure they were smart on an IQ test but they couldn’t reason their way out of a paper bag, and their writing skills were just laughable.

The smartest ones were usually the philosophy majors. Also some of the weirdest (in a good way) folks.


I didn’t say that and I also have a philosophy degree.


What's much more sensible than taking an IQ test is looking at your experience with math to date.


Yes.

Apart from the fact that IQ tests are racist bunk, there's no need to do some fancy self-discovery journey or anything to determine whether you're cut out for pure math or not: if you have to ask, then it's not for you.


Or, alternatively, you could just skip over "general intelligence" and get to your second measure that's directly validated against college outcomes – SAT score.

(My understanding is that the "general" GRE – not the math subject test – is less of a predictor of completing a PhD, but I think we can come up with a hundred reasons why.)

[1] https://journals.plos.org/plosone/article?id=10.1371/journal...


The interest in pure mathematics probably sets inquiring individuals in a higher intelligence bracket already. If somebody got through high school enjoying and succeeding at geometry and calculus, then they probably could stomach most undergraduate work in the same manner.

It seems a bit like gatekeeping to make people question whether they are smart enough when they will figure out pretty quickly if they have the aptitude or will to do it just by being exposed.


>they will figure out pretty quickly if they have the aptitude or will to do it just by being exposed

We disagree here, most people are not very good at figuring this out for themselves at all ime. It's always wise to compare yourself to known or semi-known metrics before you take the plunge into any given career, to make sure it really does seem like a good fit for you, or to make sure you can justify why you want to do it anyway even if the metrics paint an unflattering story.


I did consider doing a mathematics postgrad qualification but my IQ is high enough to realise I liked actually getting paid decent money.


Yes, exactly! One reason it's valuable to look at these numbers is because it makes you take a step back and say "Wait - I'm in what percentile?"

That naturally leads many people to ask whether making only $200,000 a year as a professor somewhere is really a price you're willing to pay, as opposed to making multiples of that as the smartest guy in the room in any number of private industries. Opportunity cost matters!


If you're making multiples of $200k for being a smart guy, there's a decent chance you're doing something that a lot of people would be ethically uncomfortable with (HFT, ads/surveillance tech, etc.). Being happy with what you do in the world also matters, and $200k/yr is easily enough to support a family on a single income pretty much anywhere already.


Bit more complicated here in the UK with salaries in academia. But in private sector I was earning 1.5x average mathematics professor salary before I even had a mathematics degree. And equity on top of that. Not the smartest guy in the room by far but the most useful.


Imagine if Richard Feynman used his IQ as a metric for deciding whether he should become a physicist. Physics would not be the same.

I am certain that there are mathematicians below, near, and above an IQ of 145 that all have great research productivity. IQ tests do not approximate the creativity, effort, and collaboration required in a mathematician. Not to mention the dubious nature of the 145 claim.

Of course, there are some people that will have a greater aptitude for mathematics than others. But you do not need to be a genius, and this is echoed by Terence Tao [0].

[0] https://terrytao.wordpress.com/career-advice/does-one-have-t...


Just to complement your post, Richard Feynman's quote on the topic:

“I was an ordinary person who studied hard. There are no miracle people. It happens they get interested in this thing and they learn all this stuff, but they’re just people.”

― Richard Feynman


I dunno man but I always believed Feynman was expressing a very “aw shucks” everyman type of sensibility to motivate his students but really he’s a genius who just never saw himself on par with the other genius demigod scientists of his time but still far removed from common people like me for example. Or he knew he was exceptional but he just liked to distinguish himself from the more square academic types by appealing to the regular people.

Either way, I never bought his claim that he was not exceptional.


I think Feynman was bullshitting you, sorry to say. This is just a manifestly crazy claim from a guy who scored literally #1 on the Putnam.


Also possible that Feynman had superb verbal-mathematical ability and bad visual-spatial ability and took a visual-spatial test. It's unusual but not incredibly so. I am the same way.


!!!! IMPORTANT COMMENT !!!!,

because the edit button is now gone:

I misspoke, 145 should have been my loose estimate for the median of actual working mathematicians and taken with many more grains of salt. Mathematics PhD students cluster around a much more attainable average of 128-130. Per [1] this would map to a much easier SAT score of only around 1280.

My general point still stands that you probably want to look at this and consider your potential career in math against this, but the skill curve is less punishing than I initially thought.

[1]: https://www.iqcomparisonsite.com/satiq.aspx


But why would you do an IQ test, when you could just do a math test? Surely a math test is a better indicator of math ability than a test that is merely correlated?


That approach would work too, but generally (not always) math tests require you to already know a fair bit of math to do them. IQ tests can and have been designed with don't even require the use of language, let alone any conceptual machinery that strong.


> "shoot for the moon and you'll land among the stars"

I´ve never been able to wrap my mind around this saying.


Ha, yeah, it's a weird saying. I think it makes more sense if you imagine the sky as like a static painting you shoot yourself into with a cannon or something.


That's not a bad outcome, especially if the world ends up being one large Truman show and you escape! ;)


I was about to ask if there is any way we can bridge the mathematics the same way we did with programming ; I love going-on with an LLM to learn math and apply them directly to problems I'm facing ; and to be honest I often feel like a musician who never actually learned his tuning lately ; especially in topics correlated to balancing, monitoring, and even simply in projection of cost ; It's like I have been over-focused on complexity of algorithms without actually realizing that it's only one part of the problem - so I have a huge potential usage of mathematics and one can really easily leverage them thanks to AI - especially considering tests ; but that's where my road ends without a more sophisticated approach - and even then it's a very dark place to wonder by (eg: lot of time spent for seemingly unknown appliance) it does however start to feel even more attracting for these exact reasons - I feel like having problems to solve with just makes this a lot easier - but I'd love to be able to ground-up things even more - and especially be able to take shortcuts.

I mean a lot of people just run a database but don't know wtf it does - but it still useful to them - maths however need to be understood to be really useful -

Is there not a way to make this lot more navigable ? Are there bridge concepts that are important enough that we can spend some time to learn them ? (there are ofc) - and how deep shall we go ?


Here is a project in that direction: https://www.math.inc/gauss

I think that yes, math will become much more accessible, and pure brain power will become much less important to use and understand math successfully.

I don't like attitudes like that of hiAndrewQuinn. If you like math, just do it, there is no need for an IQ test.


Not true at all, we need mathematicians and scientists from all backgrounds, and creativity comes in many shapes


Unfortunately, I have heard logic like this throughout my life, leading me to decide that because I struggled with (some subset of) math, I am not an intelligent person, which led to forcing myself to do pure math in college to prove my intelligence. This led to many significant and awful mental health issues. While this is a bit of a fallacious logical leap, it's not impossible that other people have went through this because of this sort of information being hammered into their head.

I choose to believe succeeding at anything is mostly about persistence and interest, barring other immense structural factors. I have zero interest after doing difficult pure math classes, so I stopped. I now think I am good at what I do, but everyone's intelligence and interests are different.

I think this sort of quantification of intelligence is really harmful to people. I don't want to exclude people from pursuing their interest because their SAT score wasn't high enough. I have met math PhD students with bad GPAs and poor math class grades in their undergrad.

On a tangent, CS undergraduate programs are insanely competitive and filter in crazy ways, and most of my friends who were passionate about CS (especially systems CS and SWE) did ECE just to avoid the competition and dispassionate culture. Your GPA and SAT scores had to be insane to get into almost any undergrad school for CS.


In a shocking twist of fate, people who do abstract reasoning and patterns oriented exercises all day long score high on a test measuring abstract reasoning and patterns manipulation. /s

You have your cause and effect reversed. You don’t need high IQ score to do maths, doing a ton of maths will naturally increase your IQ score. IQ is not general intelligence.


[deleted]


Your personal experience does not line up with the empirical evidence at all.

Some people are (much) smarter than others. It sucks, but that's life.


In theory a market could arise around providing bespoke algorithmic feeds for one's RSS content, in some kind of adapter model, but I don't know enough about ML engineering to suggest whether that is at all economically feasible. Maybe?


I don't actually know what the bus factor for Tarsnap's infrastructure is. 2 is just the absolute lower bound from what I know of the company itself. It is in all likelihood much higher.

I can't read the founder's mind, but if I were them I would probably have some Kongō Gumi style designs on making it a 1000-year company just because that's a fun intellectual exercise. [1]

[1]: https://www.tofugu.com/japan/oldest-businesses-in-japan/


I think you two may be talking past each other a bit here. Bear in mind I am not a security expert, just a spirited hobbyist; I may be missing something. As stated in my digital resilience audit, I actually use both Tarsnap and restic for different use cases. That said:

Tarsnap's deduplication works on the archive level, not on the particular files etc within the archive. Someone can set up a write-only Tarsnap key and trust the deduplication to work. A compromised machine with a write-only Tarsnap key can't delete Tarsnap archive blobs, it can only keep writing new archive blobs to try to bleed your account dry (which, ironically, the low sync rate helps protect against - not a defense for it, just a funny coincidence).

restic by contrast does do its dedupe at the file level, and what's more it seems to handle its own locks within its own files. Upon starting a backup, I observe restic first creates a lock and uploads it to my S3 compatible backend - my general purpose backups actually use Backblaze B2, not AWS S3 proper, caveat emptor. Then restic later attempts to delete that lock and syncs that change too to my S3 backend. That would require a restic key to have both write access and some kind of delete access to the S3 backend, at a minimum, which is not ideal for ransomware protection.

Many S3 backends including B2 have some kind of bucket-level object lock which prevent the modification/deletion of objects within that bucket for, say, their first 30 days. But this doesn't save us from ransomware either, because restic's own synced lock gets that 30 day protection too.

I can see why one would think you can't get around this without restic itself having something to say about it. Gemini tells me that S3 proper does let you set delete permissions at a granular enough level that you can tell it to only allow delete on locks/, with something like

        # possible hallucination.
        # someone good at s3 please verify
        {
            "Sid": "AllowDeleteLocksOnly",
            "Effect": "Allow",
            "Action": "s3:DeleteObject",
            "Resource": "arn:aws:s3:::backup-bucket/locks/*"
        }
But, I have not tested this myself, and this isn't necessarily true across S3 compatible providers. I don't know how to get this level of granularity in Backblaze, for example, and that's unfortunate because B2 is about a quarter the cost of S3 for hot storage.

The cleanest solution would probably be to have some way for restic to handle locks locally, so that locks never need to hit the S3 backend in the first place. I imagine restic's developers are already aware of that, so this seems likely to be a much harder problem to solve than it first appears. Another option may be to use a dedicated, restic-aware provider like BorgBase. It sounds like they handle their own disks, so they probably already have some kind of workaround in place for this. Of course, as others have mentioned, you may not get as many nines out of BB as you would out of one of the more established general-purpose providers.

P.S.: Thank you both immensely for this debate, it's helped me advance the state of my own understanding a little further.


Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: