Viewing a single comment thread. View all comments

Just_Another_Scott t1_iu2jklj wrote

Their website says about 15 terabytes a night. So take that for what you will.

> Rubin Observatory R&D has led to a new-generation imaging CCD which is highly segmented, low noise, and sensitive from the UV to the near IR. The speed with which Rubin Observatory can cover half the sky will produce about 15 terabytes (TB) per night, leading to a total database over the ten years of operations of order 50 petabytes (PB) for the raw data, and 15 PB for the catalog database. The total data volume after processing will be over 100 PB

So total data requirements exceeds 100 PB over a 10 year period.


TheRealMrChips t1_iu2qkvr wrote

Which seems like a lot, but with 30TB HDDs coming in 2023 and even larger capacities in the next couple years, 100PB won't be all that hard to make happen over 10 years' time, even with redundant storage and backups taking it to 300-400PB of "actual" storage footprint.


bsloss t1_iu4ddam wrote

Back of the napkin math says it would only take 5 to 10 racks of something like this to cover the storage requirements. Not something that comes cheap, but certainly doable and still below the industrial scale of massive data centers run by most big tech companies.