Railfan Technology > Digital Photo storage

Now that I started scanning my slides, Ive been looking into good solutions for securely storing them digitally. I'm scanning at 4800dpi, so they are about 7Mb each ehich adds up fest enough.  I have used external hard drives before, but can;t risk having a drive fail - which they do occationally.  So Ive been looking into Network attached Storage (NAS) as a possible solution.  I like the idea of have a 2 bay enclosure with redundant drives and I like the home and cloud accessability options. 

How do you store your scans?  Do you backup your device eelsewhere?  Anyone running a home server or NAS device?
 

Yes, I run a couple of NAS's. There is a  motto:  "A server is not a backup" which is justified by several things.  So I have a second NAS that backs up the first NAS.   May sound silly, but to me it is worth it due to the easy and fast access to everything I have in the way of digital items.  That may be an overkill for most, so then perhaps you might consider backing up on DVD's.  Maybe even two of each. 
 

My only thoughts to add, is that I stay clear of DVDs etc. In 10 years, I doubt any computer will be on the market with any form of mechanical drives. Friends of mine have suffered DVD failures as well. chris uk.

 I shoot a lot of digital video (big files). I use 8TB external drives.  I always use one primary and one back-up drive in case a drive fails (hasn't happen yet).  External HD's are not that expensive and are a good storage solution for me.

My choice has been Seagate SSD backup drives, either leave them conected or do a backup after scanning, or each week etc. What sold me was they don't have a power supply, which often have a higher failure rate than other computer parts. They run off the computers USB, as long as there's a USB and a company called Seagate it should work ( but that's not a sure thing either ). Which reminds me I just did some scans so I better do a backup.

I've been using small portable external hard drives for quite a few years.  A pair of 4 TB drives makes a set, one to work from, one that is a copy of the first.  Store them in the same room, but one is in a safe, one hidden.  Should really have a copy offsite.

It is tempting to move up to a NAS, but that would still require a backup, either to an external HD or another NAS as pointed out above.  Either way drives need to be replaced every so many years, NAS or external HD.

I recently read something about bit rot and how it can corrupt files over time.  Did not sound like a something to loose sleep over yet, but need to research more.  My takeaway was a concern that backups and maybe even RAID just copies the bad bits.

A very helpful tool for creating user friendly backups - the backup drive folder and file structure looks like the original - is https://www.bvckup2.com/.  Lots of configuration options.  Plug both drives in and the backup will run automatically on the schedule you decide, plus it only backups what changed.

Jeff Eggert

I use two external hard drives and Amazon.  Since I have Amazon Prime the storage is free.

Bob Wilcox
Charlottesville, VA
My Flickr Shots

No RAID's for you guys?

dan Wrote:
-------------------------------------------------------
> No RAID's for you guys?

Sure, my NAS's are RAID's, but not 5.

 

I read that non-organic blueray discs are the most time-stable of all currently available storage media. 

Bit rot shouldn't be a problem if you spray them with WD-40

I also use external drives. KISS.  RAID etc is for critical real time  informational tech, not that necessary for choo choo photos.  I have one external  with computer, one secured upstairs and one out in shed away from house.

I don't think you want your primary backup sitting attached to your computer waiting for something bad to happen.
I found them to be reliable but don't expect them to last forever.

Key is having a process to keep things up to date which I am a little lax on. Of course if you are scanning slides you still have them also.

bob

 

The obvious solution for long term storage is multiple copies.  That's obvious. 

But the question becomes: how to easily make multiple copies and what are the advantages of each solution?

1.  I have a retired gung-ho techie friend I worked with (I'm a retired techie, too) that puts everything into RAID.  He's written lengthy automated Linux scripts to perform backups to the cloud nightly.  He also has written continous disk-monitoring scripts to presumably catch a problem before it's a problem.  It's a great idea, in my book.  But I see 2 problems: 1) If his computer gets a virus, it's duplicated instantly on the RAID and that night, into the cloud.  I presume he has a strong enough internet security product to be able to remove it.  and 2) He's paying a small fortune for the most reliable and fastest Western Digital drives (black) and has a failure about once per year.  I think he's got about 10 WDs spinning at the moment.  Given that his retired wife was a college French professor and she keeps just about everything on the computer, there was a need to ensure security while she was working.  Nowadays, he's downloading videos, etc, and still keeps all the French lessons on the computer too.  Way too much overkill, in my opinion.

2.  USB connected external drives - way too slow, if you ask me.  Yes, there's software that does incremental backups (ie, only files that changed) but that requires one to have the 'base' copy and all incremental backups available!  Get an unrecoverable error on one of the incrementals, and you can no longer trust the validity of what's on those drives.  The comment that the power supply for external drives fails should be noted.  Also, the retail price per gigabit of external drive is in the range of double the price of the hard drive inside!  I had been using an external drive for my monthly off-site clone backups until it failed.  To my chagrin, it was only 2 years old but had a much older hard drive (mfg date) inside.  I had paid dearly for what already was one generation 'behind' in technology.  Also, all the fancy buffers and caches on an external drive are a complete waste of money.  The slowest link in the chain is the USB...even USB 3.1!  When I built my current computer 4-5 years ago (octo processor at 4.7ghz), I intentionally purchased a USB/eSATA front panel device that could handle all sizes and shapes of USB devices as well as an eSATA device.  My last external drive had both USB 3.0 and eSATA capabilities.  I decided to use the eSATA connection first, and several months later discovered that using it via USB was about 10 minutes faster when cloning a half-full 3 TB WD Green (their slowest) hard drive!  I was overjoyed to get about 50% of the cost of that drive on ebay when I sold it.

3.  Cloud storage...I you happen to be on a slow internet server, cloud storage will take forever to move data.  I routinely get a multi-layer Photoshop image via a link to a cloud storage and it consistently takes 30 seconds or more to recieve & download a 10MB .PSD image.  I shudder to think how long it would take to down(or up) load 1.5 TB!

I, personally, have multiple issues with cloud storage and therefore use it only when someone sends me something via the cloud and/or I have to transmit something to someone via the cloud.  First and foremost, anything in the cloud requires a folder on your hard drive that mimics what's in the cloud, sucking up space.  Second, at computer startup/reboot, the cloud software comes in and spends time synchronizing what's on your drive vs what's in the cloud, and then transmits data to/from the cloud to make it match.  I have a friend that asked me to speed up his really slow laptop.  He had 4 different competitor cloud services on his laptop!  From type in the password to things running 'normal' was well over 7-8 minutes!  I offered to remove any or all clouds, but he said he had a need for each of them.  I ended up removing various Windows 10 'useless trash' such as Xbox, Skype, Get Office and Movies & TV as well as a number of other unnecessary Windows 10 'services' that start automatically on his computer.  All that baggage is 'competing' for RAM, although usually 'paged out' once it's started.  It still takes regular machine cycles to keep it 'alive'.  He was overjoyed it was so much faster!  When I upgraded my 3 computers to Win 10, I couldn't get rid of all the useless builtins except the Calculator fast enough!  I also disabled about 50 Windows 10 services as well!  And lastly, I consider the cloud as 'fair game' for any hackers out there.  Back in the '80s and '90, I routinely 'beat' the various IBM mainframe security systems to get my programs to the top of the priority list for execution.  In the PC world, I've seen far too many instances where some new virus hits a computer and gets disseminated throughout an entire large corporation in less than 10 minutes due to it's email tie-in.  These days, I believe absolutely anything connected to the internet can be hacked if someone tries hard enough...and that includes infrastructure systems such as the electric grid.

My solution has evolved through the years to having a pair of SSD slide in/out bays for my C: drive, and 2 slide in/out bays for my long-term storage 3TB drive.  I simply clone (not copy!) the current SSD to a twin weekly and twice monthly (separate copies) and clone the 3TB drive to its twin twice quarterly.  One set of those copies is in a 'bug out bag' and the other is at an offsite location should I come home to an empty or burned down house.  Anything I put 'new' onto the 3TB long term storage drive is kept on the SSD as well until the next quarterly clones have been completed.  So I always have at least 1 backup of whatever is on the 3TB drive.  How important is it to keep current clones of the SSD C: drive?  I weekly copy a small number of files, including my checkbook spread sheet to a USB drive on my keyring.

And for what it's worth, a couple of weeks ago, Windows 10 decided to put problematic version 2004 onto my computer one afternoon as I had failed to increase my 'no Windows updates' setting.  Once I saw what happened, it took less than a minute to swap SSD drives and the update was instantly gone!  I had to copy a handful of files from the v2004 SSD to the replacement drive and I was done!

I should note, too, that I have a generic USB 3.0 'docking bay' I found on Amazon that I can slide in any SATA drive including laptop size/SSD size drives that I've used a number of times on friends' computers to upgrade Windows 7 to 10 as well as migrate data from an old Win XP computer to their new computer.  I also have a slide-in SATA large to small bay converter that I can put an SSD into, then slide the whole thing into a full-size SATA drive on my computer.  Having a number of non-computer savvy friends has thrown a lot of curves at me, including a Win 8.1 computer that when I put the hard drive into my computer it automatically changed my BIOS settings including boot-sequence!

For me, it comes down to how important is the images and downloaded videos on my 3TB drive?  Is copies of software I wrote for my own contracting company 25 years ago and used at client sites have any value?  Are a bunch of rock and roll, clasical, country, and jazz videos a 'big loss' if they got wiped out?  At this point, I think there's less than 3-4 Gig of data on that drive I would be 'broken up' over if it got wiped out.  There's many files out there that other than being copied from drive to drive through the years, haven't been opened or run in well over 10 years!  Maybe 30 years ago, my father told me that 'anything you haven't used in the past 2 years is likely you never will use again'.  I finally decided he was right in my early 60s. 

Like a number of former RR slide image creators, I've been busily scanning my slides as well.  Sad to say, there's slides I know I haven't looked at since I got them home from the processing lab!  Although I thought of myself as a decent photographer 30-40 years, looking at those slides today, many fall into the 'WHY did I KEEP THIS?' and it's in the trash can.  I'm also considering what will happen to those images when I'm gone.  Will anyone even CARE about them?  Fortunately, I've been selling some of the better slides on ebay in the past couple of months.  Let someone ELSE figure out to do with them when THEY are gone!

I'm fortunate, I guess, that I have less than 300 'family' images on slides and digital.  So when I drop dead, there's no big loss of family history.  I suspect that most readers, however, have pictures of their family going back several generations to deal with.  Getting copies of THOSE to family members is important.  For what it's worth, the wife of a good friend asked me to digitize a bunch of family slides of her parents and her siblings before they got married.  I was happy to do so.  I made a 'finished set' of all of them for her and her 2 sisters.  I keep the set on my 3TB drive as well, in case I'm asked to make more copies.  Although that was the 3rd set of 'family images' I've digitized, I finally realized that those are probably the ONLY images that are worth cleaning up/editing/recovering with the tools I have...Lightroom and Photoshop.  Other than railroad images I have/will post on this site and others, I settle for the 'as scanned' version of the image.  Spending even 2 minutes per slide cleaning them up times about 5,000 slides is NOT reasonable.  So, if I ever want to post them some time, I'll clean them up then and save time today.  Besides, who will give a hoot about my 'common railfan'-level RR slides when I'm gone?  Who cares?

How and when you store scanned images and other files is up to you.  There's no one size fits all.  What works for you works for you, and probably won't work for the next person.  Knowing that there ARE options is probably the most important.



 

Thanks for all the great food for thought!  I was tempted to go NAS because all of my family stuff would be on there and accessible from anywhere. I have tons of digital family images since that's how I always took them.  I get so sick of having these pics on my phone, others on my wife's phone, some stuff on my work computer etc etc.  Everything in one place seems better. 

As for slides, I plan to sell what I have after I scan them, hence the importance of a secure storage solution.  I see no need to keep the slide after I scan it. Better to sell it and buy more slides!  Ive gotten better at cleaning them before i scan, so there isn't a lot of touchup needed, so unless I want to post it online I'll leave it as scanned. 

Hi Matt,
I also recently tackled the very question. My decision was to use Lexar 64 and 128GB memory cards, since there price has fallen so much over the last year. So I bought the Lexar card enclosure. It has 4 cards at a time in it. But you could use it as a stand-alone if you wanted. 

At present, am going thru my VHS and SVHS tapes. They have been packed away for close to 40 years. The outcome is really good, compared to what I expected from the tapes, being so old. After that, I may tackle the slides next. 

All the MiniDV Tapes will not be backed up, since they were taped in digital format. They certainly will last longer then I have left In life. 

As for my digital DSLR video, they are already backed up 

Good luck, it's a chore for sure

Dave
In PA

Cloud stroage: never trust someone else with your data.

optical disks: obsolencense, and if they are not mechnically pressed, will not last.

SSD: from what I have read do not hold data long term.

I use multiple external non-SSD hard drives. And I have copies externally in other locations. If you house burns and you have 3 backups all in the same house. Guess what, you now lost your life's work. 

For the average user concerned about data, a RAID array on your main machine or a small NAS appliance and - most importantly - a regular backup to an external drive kept offsite are probably adequate.  The biggest thing is have an offsite, offline backup that you keep up to date.  That way if a lightning strike or a fire or a virus hits and destroys your main data store, you have a backup that is untouched.

My own solution is probably overkill, but when I consider the tens of thousands of hours and dollars I've spent traveling and photographing trains (and other things), the possibility of a good part of my life's work and memories just blinking out makes me terrified. 

My main storage is a NAS-type server with two disks worth of redundancy.  Basically I can lose two disks before the data is gone, so that if one fails, I still have one extra while I'm rebuilding the array.   (It's also based on ZFS, an advanced filesystem that detects and corrects any corruption the underlying disks don't fix.  It also allows me to snapshot things, so I can pull up an older version of a file if I accidentally overwrite something.  Setting it up is not for the faint of heart, however.)  That gets backed up every night to a second array on my workshop machine made of older disks and with only one disk redundancy.  All of the disks are HGST server grade drives - the things are crazy reliable.  I then have two external drives, and every Thursday I bring one home from the office, back the array up to it, and take it back on Friday morning.  The next week I bring the other one home.  Every ~4 years, I upgrade the disks in the main array and the old disks get bumped down to the secondary array.  The secondary array disks then go away, or repurposed.  The external drives get upgraded maybe every 2-3 years, and the old ones build up in my filing cabinet at work as just that extra little bit of security if I need to go get something from years ago.

That way, at any given time, there should be two copies of any data over 24 hours old, but only in the house.  After a week, all data is guaranteed to be off-premises.  After two weeks, there's guaranteed to be four copies - two in the house, two outside the house.


Don't let anybody tell you there's such a thing as archival digital anything yet.  There isn't.  Either the hardware will fail or go obsolete, or the file formats will.  Digital longevity requires constant upkeep and care.

Does anyone use a Polaroid Palette to archive their digital to film?

I don't trust these unproven new-fangled technologies. :^)  At least you can hold it up to the light and see what it is!

​I have 4x5, 120, and 35mm backs for mine though I have only done 35mm E6 process film recently.

Railpax71 Wrote:
-------------------------------------------------------
> Does anyone use a Polaroid Palette to archive
> their digital to film?
>
> I don't trust these unproven new-fangled
> technologies. :^)  At least you can hold it up to
> the light and see what it is!
>
> ​I have 4x5, 120, and 35mm backs for mine though
> I have only done 35mm E6 process film recently.

How often does your diplodocus need feeding...? ;)

Good one! I am a dinosaur in case nobody understands the mention.  However, I really like the effect of the Polaroid ProPallete on a contact printed 4x5. No point to scan the negative. PITB to expose a single image and process but it is archived. I have a stock of frozen TMax 4x5 that I have to use.

PS. to support this endeavor I run Windows XP in a virtual machine on Windows 10. It runs so fast on a core i7 without all the bloatware!



Edited 1 time(s). Last edit at 07/26/20 01:08 by Railpax71.