blog: sparse is spartan

Post by **nikos** » 2011 Jun 05, 07:42

here's the comment area for today's blog found at
http://zabkat.com/blog/05Jun11-sparse-file-creation

Kilmatead · Post by **Kilmatead** » 2011 Jun 05, 09:14

I don't know what real purpose they would serve — apparently they are designed for video and database use.

A little perusing of the voluminous SQL Server docs suggest it to be the NTFS equivalent of malloc().

Or, at least, that's the first thing that comes to mind. (Well, after tea and some nicotine at this time of day.) They also appear to be the real villains behind the practical application of the SizeOnDisk column.

kunkel321 · Post by **kunkel321** » 2011 Jun 05, 15:31

Well... You could've just did a forum search for all of Kilmatead's posts, then saved them to your disk. That'd be a few TB at least! he he he ...

Kilmatead · Post by **Kilmatead** » 2011 Jun 29, 22:37

As a tangentially related investigation (i.e., something to do when bored off my arse), it appears the Sparse attribute is valued at:

FILE_ATTRIBUTE_SPARSE_FILE 512 (0x200)

...and yet does not appear in the x2 Attribute string. I take it you deemed it so pointless so as to just leave it as road-kill (as it's rather difficult for the user to modify themselves)? Much like FILE_ATTRIBUTE_TEMPORARY 256 (0x100), I suppose.

Which leads me to a more pertinent question (for my own nefarious needs): x2's Attribute display format is "RHSAD???J" - I'm assuming the missing letters are "O" (Offline), "C" (Compressed), and 'E' (Encrypted), but - um - what order would they appear in? Not having any of these files it's hard to test for. The x2 docs are a wee bit vague on the ingredients list, and I can't see any logical correlation between attribute letters and their positions, aside from Nikos' dyslexic humour.

Basically, what would a complete attribute string look like? (And yes, I do have a reason for wanting to know this, other than random curiosity.)

Post by **nikos** » 2011 Jun 30, 04:07

{_T('R'),_T('H'),_T('S'),_T('A'),_T('D'),_T('C'),_T('E'),_T('O'),_T('J')};

Kilmatead · Post by **Kilmatead** » 2011 Jun 30, 06:18

Sláinte.

John_Gray · Post by **John_Gray** » 2011 Jul 20, 08:14

Am I right in thinking that a TrueCrypt 'container' would be a good application for a sparse file, because you have to allocate the maximum size you want initially, even though it may only be a few percent full at any one time?

A backup would/should/might back up on the amount of space actually used in the sparse file, not the entire TrueCrypt container.

(I suspect that "it doesn't work that way", but it would have been a neat idea...)

quietbritishjim · Post by **quietbritishjim** » 2011 Jul 25, 21:25

That is indeed how how TrueCrypt containers work, but only if you choose to make them dynamic, rather than static. See this TrueCrypt doc (scroll down to "dynamic"). It contains this warning though:

WARNING: Performance of dynamic (sparse-file-hosted) TrueCrypt volumes is significantly worse than performance of regular volumes. Dynamic (sparse-file-hosted) TrueCrypt volumes are also less secure, because it is possible to tell which volume sectors are unused. Furthermore, if data is written to a dynamic volume when there is not enough free space in its host file system, the encrypted file system may get corrupted.

One other thing not contained in that warning: I'm pretty sure that if you use and then zero back out some data in a sparse file, that data is not just re-marked as a sparse region, but instead continues to take up real space.

However, I have a large dynamic TrueCrypt archive and it works well for me. According to Explorer its size is "150 GB (161,061,273,600 bytes)" and its size on disk is "48.7 GB (52,324,859,904 bytes)".