My solution for ponderous NAS file movement. Don't!

[Gary M. Mugford]

Hi all,

Been a semi-regular whiner for the last couple of years at the speed of copying large amounts of files around my home network. Whether using Nikos' Robust Copy, FastCopy, TeraCopy and even FreeFileSync, the fact is that folders with a gazillion icon files or a total of 1T+, or even healthy percentages of those, all seemed to take slightly longer than the average World Cup to finish. World Cup of what? I'm not defining. At issue was the fact that I was slinging bits usually between local drives and NAS box drives. I LIKE backups. (And yes, I have the required cloud backup component of my backup matrix) And I have lots of video that needs ... movement. Ergo, lots of these copying marathons.

Turns out, I now have a working solution. Ditch the NAS boxes. They were decent D-Link boxes. Three of them, each with two bays, over the last half-decde. One box died, one half-died and I lost two of the four drives ensconced within. So, I bit the bullet, on advice of my hardware tech, and switched to using a ProBox 4-Bay USB 3.0 box that connects through a port in the powered 10-port USB 3.0 hub tied to my main computer. And the drives are shared using basic Windows sharing with the other computers in the house. In fact, I bought two of the 4-bay boxes and retired one NAS, keeping one in circulation for talking to devices other than computers here at the Castle of Confusion. The speed increase moving bits about is breath-taking. An order of magnitude? Or close to it. PLUS the filenames are now available to Locate32 for easy searching (and I assume with Everything from voidtools). AND the drives are subject to Hard Disk Sentinel Pro, which allowed me to spot that two of the drives I had repurposed (reformatted from ext2 to ntsf) were closer to keeling over than being trustworthy. A VERY GOOD TOOL TO HAVE. And it works with these boxes.

By the way, the price I paid for TWO of the ProBox enclosures was less than what I paid for a D-LINK two-drive NAS as recently as 18 months ago and a heckuva a lot cheaper than the QNAP box(es) I was saving up for. And the thing is a WHOLE lot quieter than what it replaced.

I assume there's SOME gotcha's somewhere in my lashed-up system. There are some weasel words at the Mediasonic site regarding stuff the boxes require. But, right now, I'm sure getting a lot of performance for my buck.

End of unpaid advertisement.

GM

[nikos]

a get quite a few supports for NAS drives but frankly I don't know what they are
like a drive with an IP address so it can be referenced from any computer in a local network?
for backups I use removable USB drives that can be easily moved and plugged in without too much networking trouble

[Gary M. Mugford]

Nikos,

Network Attached Storage boxes are containers that you shove a drive (or several) into and the contents become accessible to everything on the network in question. FREQUENTLY, because NAS's were originally intended for small/medium size office use and frequently as central data repositories and/or their backups, the drives were hot-swappable and/or RAID'd. They also largely used some sort of linux file system (ext2/ext3 for example) and the firmware on the things was usually a linux derivative. All to keep the costs in decent shape. The performance was always good, never great, with all of the paranoid redundancies and assorted other security and quality services built in. And as a network appliance, the box was ultimately available if the user desired, on the internet. A good idea. And a bit extravagant for the normal home user. Of which I fail to qualify.

Every member of my family thinks video's are actually one of the inalienable rights given man by whoever made us in his/her/their image. I have LOTS of video, as the family archivist. I have never video'd a thing in my life. It's one of life's little ironies. I also collect Icon collections because I am a fool. Folders with LOTS and LOTS of files in them. So, Large files, check. Folders with Large numbers of files, check. A need to backup and frequently re-organize, check. An ever-present space crunch despite have more than 10 Terabytes of storage, check. It was doable on NAS's. But I am an impatient man. And file copying was ... cutting years off my life expectency.

Switching AWAY from NAS's has been a health godsend (and having had three heart attacks and still dealing with blood clots in my lungs and assorted other things trying to kill me). That I find it cheaper to switch to USB3.0 boxes that MIMIC NAS's in that they are multi-bay enclosures brings a smile to my face. And smiling is ALWAYS good. The speed increase has been remarkable. The data coming FROM the drives now feeds into Hard Disk Sentinel which warns me if the drive gets too hot, or developes too many imperfections or just basically stops doing it's job, which is good for peace of mind. And now Locate32, which is the filename database that I use, ALSO includes the contents of the externally-located files. Previously, neither utility would work with NAS's.

Like you say, a usb3.0 box does lots of good things. But having the equivalent of 10 of them available when I turn on the computer is even better

Switch rather than fight. GM

[dunno]

Thanks for your Info/Rant.

I have a 2TB NAS by Seagate, the thing is so darned slow over the air and only twice as fast with a LAN cable, pathetic, it's ok for Video streaming, just. But as you mentioned file transfer speeds are suicidally slow, and this from a guy who has time on his hands.

My rant if I may....Is the lack of WiFi connectivity between a old HiFi to the router or directly to the Command notebook or tablet, there just isn't a easy cost effective solution available.
What I'm thinking of doing is getting a small fan-less pc/notebook with usb3, as my media hub, connect my HiFi and TV to that, then connect to that "media hub pc" via Remote Desktop, problem solved methinks. A bonus is that I won't have to worry about DLNA or third party apps to stream my entertainment, vlc and foobar will work.

[Gary M. Mugford]

Thanks for your Info/Rant.

I have a 2TB NAS by Seagate, the thing is so darned slow over the air and only twice as fast with a LAN cable, pathetic, it's ok for Video streaming, just. But as you mentioned file transfer speeds are suicidally slow, and this from a guy who has time on his hands.

My rant if I may....Is the lack of WiFi connectivity between a old HiFi to the router or directly to the Command notebook or tablet, there just isn't a easy cost effective solution available.
What I'm thinking of doing is getting a small fan-less pc/notebook with usb3, as my media hub, connect my HiFi and TV to that, then connect to that "media hub pc" via Remote Desktop, problem solved methinks. A bonus is that I won't have to worry about DLNA or third party apps to stream my entertainment, vlc and foobar will work.

The advantage of having a hardware guy (and a best mate at that) is that I was able to get my media centre running and then change in mid-stream when I started to switch horses. I think that analogy still works, although I haven't tried wading through running water on pony-back since ... well I was little enough to BE on a pony's back. I digress.

Patrick hard-wired in a computer that had generationally fallen into disuse (It was Nuklon, my last AMD box, which was a three machines ago and about 10 years old, having been succeeded by Ollie, Popeye and now Quincy). Having the machine down in the viewing room, then allowed me to hard cable in the computer into the big TV. Nuklon was plenty fast enough via wired-cable into the router to access the lone remaining NAS AND the USB shared files through my network. I operate it using a wireless remote keypad with tiny trackball that I bought for 10 bucks from China. The ADDED touch was that Patrick set up a second wireless router that originated on Nuklon and serves out media COPIED THERE to the tablets around the house. That gives me SOME control, and it's easy to grab stuff from the upstairs computer jungle and bring it down to the first floor. Like you, it all sort of works.

And yes, Patrick did some tricks to baffle the fan noise, but Nuklon was never all that noisy to begin with, having HUGE fans that ran at slow speeds in the biggest case I could find at the time ... almost thigh-high. I traded less noise for less space. But a laptop would have been just as good. We have two or three lying around from the days I USED to leave this abode. Sigh. There's always kids to try and tease into using Windows. Haven't had much luck recently, but one or two of them are showing signs of curiosity at the thought of seeing if they can't FIX some of them up properly [G].

[dunno]

Patrick hard-wired in a computer that had generationally fallen into disuse (It was Nuklon, my last AMD box, which was a three machines ago and about 10 years old, having been succeeded by Ollie, Popeye and now Quincy). Having the machine down in the viewing room, then allowed me to hard cable in the computer into the big TV. Nuklon was plenty fast enough via wired-cable into the router to access the lone remaining NAS AND the USB shared files through my network. I operate it using a wireless remote keypad with tiny trackball that I bought for 10 bucks from China. The ADDED touch was that Patrick set up a second wireless router that originated on Nuklon and serves out media COPIED THERE to the tablets around the house. That gives me SOME control, and it's easy to grab stuff from the upstairs computer jungle and bring it down to the first floor. Like you, it all sort of works.

Whatever happened to Plug and Play......I'm astounded that the 0and1 gurus still haven't figured that out, I guess if the gurus can't figure out 0&1's, there's no hope for black holes.

[Kilmatead]

Whatever happened to Plug and Play....

Actually, that reminds me - in the original post above, Gary mentions his (old?) NAS drives (within the enclosure) were hot-swappable... I was wondering if that tech has gotten any better since I played with it (a few years ago, now). I had nothing but trouble with the SATA controllers dropping out, and Windows sort of arbitrarily smoking cigarettes in the loo when it was supposed to be detecting new drives. You'd try it once, and it would end in tears. Do exactly the same sequence again and it would work as advertised. But then the next one would fail. I had friends who would swear upon their mum's graves that swapping rack-servers (hot) was what made going home to their dreary wives bearable. But to me it was never anything but trouble.

Has it improved? (Hot-swapping, that is, not putting up with dreary wives )

[Gary M. Mugford]

Whatever happened to Plug and Play....

Actually, that reminds me - in the original post above, Gary mentions his (old?) NAS drives (within the enclosure) were hot-swappable... I was wondering if that tech has gotten any better since I played with it (a few years ago, now). I had nothing but trouble with the SATA controllers dropping out, and Windows sort of arbitrarily smoking cigarettes in the loo when it was supposed to be detecting new drives. You'd try it once, and it would end in tears. Do exactly the same sequence again and it would work as advertised. But then the next one would fail. I had friends who would swear upon their mum's graves that swapping rack-servers (hot) was what made going home to their dreary wives bearable. But to me it was never anything but trouble.

Has it improved? (Hot-swapping, that is, not putting up with dreary wives )

M'Lord Kilmatead,

Yep, the NAS's were, indeed hot swappable and the one time it was tried here, it worked. Mind, you, rebuilding the drive with NEW backups to it was ... loo-inspecting time. The problem was, as it has been, the directories full of small files. i.e. All those bleepin' icon files I have collected to help make xplorer2 (and my own software) so beautiful. Sure, I could stick them into libraries and squeeze the multi=thousand file folders into a single file. BUT, how would I then use locate32 to find *comic*.ico? Or *iso*.ico? Sigh.

GM

[Tuxman]

I plan to set up a "home NAS" for my digitalized music collection (vinyl rips et cetera) but I think "full-featured" dedicated NAS boxes are a bit pricey. My current intention is to buy some of the higher-ranged ARM (A20?) boards and pin two USB drives on them. NetBSD should handle this use case well. Why spend 200 <your currency here> more than needed?

[Kilmatead]

Why spend 200 bucks more than needed?

When Germans use 'bucks', I'll be hot-swapping gravestones. You been busking on 5th avenue again?

I always seem to get the impression that the NAS bandwagon flopped before it really began, even though a lot of people (who probably didn't need it) subscribed to it. No empirical evidence to back that up - just the impression I get being a self-confessed hardware-junkie.

Then again, networks are the devil's bed, so I can't see much point in "the home", less you have "other people" to concern yourself with (Sartre never knew what he'd unleashed).

I'm currently in the slow process (saving up 'bucks') of converting all my mechanical drives into solid-state (except for a couple of old 10,000 rpm raptors I'll keep for nostalgic purposes) - but it seems like a waste to dedicate a whole array (of SSD's) to my music collection (it is static, after all), so that's going to remain mechanical. Ugh, "mechanical" - even the word sounds so... mechanical.

With any luck the next generation of drives will abandon SATA fully and finally just be PCIe so there's no bus latency at all to think about. Ironically even SATA3.1 has been saturated (capped at 600MBps practical transfer speed).

[Tuxman]

When Germans use 'bucks', I'll be hot-swapping gravestones. You been busking on 5th avenue again?

The common slang words for alien currencies (US Dollars, British Pounds, Fiji Seashells) are not part of my actively used thesaurus. I rarely need to talk about money which is not (intended to be) mine.

I'm currently in the slow process (saving up 'bucks') of converting all my mechanical drives into solid-state (except for a couple of old 10,000 rpm raptors I'll keep for nostalgic purposes) - but it seems like a waste to dedicate a whole array (of SSD's) to my music collection (it is static, after all), so that's going to remain mechanical. Ugh, "mechanical" - even the word sounds so... mechanical.

I actually doubt using SSDs for large-capacity NAS storages are worth the money yet. Why would I trade some multi-TByte selfmade "NAS" just to have a faster write cycle?

[Kilmatead]

I actually doubt using SSDs for large-capacity NAS storages are worth the money yet. Why would I trade some multi-TByte selfmade "NAS" just to have a faster write cycle?

This is true, on the multi-T-scale it's not worth it. That said, the difference between normal write cycle and "faster" is orders-of-magnitude, not just a few belt-loosening notches. Then again, networks by their nature tend to be slow anyway. I wasn't touting them for network storage - it's just the other day I saved myself mucho Fiji Seashells by deciding that a music collection doesn't much need one either, so it was topical (to me).

(And the Seashells in Fiji are so secret [and radioactive] that foreigners aren't allowed to know the local slang for them - we would hurt ourselves with the pronunciation tongue-gymnastics only our significant-others are meant to truly appreciate. )

[Tuxman]

it's just the other day I saved myself mucho Fiji Seashells by deciding that a music collection doesn't much need one either, so it was topical (to me).

Tropical, you mean?

[Gary M. Mugford]

I actually doubt using SSDs for large-capacity NAS storages are worth the money yet. Why would I trade some multi-TByte selfmade "NAS" just to have a faster write cycle?

This is true, on the multi-T-scale it's not worth it. That said, the difference between normal write cycle and "faster" is orders-of-magnitude, not just a few belt-loosening notches. Then again, networks by their nature tend to be slow anyway. I wasn't touting them for network storage - it's just the other day I saved myself mucho Fiji Seashells by deciding that a music collection doesn't much need one either, so it was topical (to me).

(And the Seashells in Fiji are so secret [and radioactive] that foreigners aren't allowed to know the local slang for them - we would hurt ourselves with the pronunciation tongue-gymnastics only our significant-others are meant to truly appreciate. )

Now, we do have some Fijians in town as it happens (Junior Girls Rugby World Championships if I recall, mostly correctly). I have it without good authority that the slang is CLAMS. Thus, we import American secrets transparency to this discussion.

Oops, now that I think of it, I'm not American. We here in Canada use Loonies, Toonies and Dubloonies, roughly in that order, when exercising our rights not to use American slang. Or something like that.

GM [G]

[Kilmatead]

Now, we do have some Fijians in town [...] Junior Girls Rugby World Championships [...] I have it without good authority that the slang is CLAMS.

Methinks you've confused that with the otherwise rhyming "ain't she got great gams" - being that gangsters (as in Tommy-guns, not contemporary pretenders) used to find Canada an easy hiding place. As a disclaimer, I have a lot of drive space being taken up with Mad Men now, and it behoves me to note that the 50's seems to have had a lot of cigarettes, sex, and misogyny, in that order. Definitely more gams than clams.

(And Tux, editing your post when someone has already "quoted" the edited-part-in-question is counter-productive, no? Didn't they teach you that in your fancy school of tropical programmationing? And yes, programmationing is a real word. Sort of. If you squint a bit and don't think too hard about it.)