Click to Rate and Give Feedback
MSDN Library
Technical Articles
System Services
File Services
 A Programmer's Perspective on NTFS ...
Files and I/0 Technical Articles
A Programmer's Perspective on NTFS 2000 Part 1: Stream and Hard Link

Dino Esposito

March 2000

Summary: This article provides an in-depth discussion on NTFS 2000, a new file system in Microsoft Windows 2000. (19 printed pages)

Download NTFSext.exe.


Overview of NTFS 2000
Multiple File Streams
Fundamentals of Streams
Streams Backup and Enumeration
Hard Links
Enjoy NTFS Features


The myth of a fully object-oriented version of Microsoft® Windows NT® has been around for a while, since 1994. Cairo—the code name of that legendary version of the OS—never materialized outside a lab in Redmond. Since Cairo's inception some of its fundamental ideas have been introduced now and again.

The basic idea behind Cairo was that files and folders would become objects and collections of objects. The folder's content is not necessarily bound to the underlying file system storage mechanism and you can access and replicate those objects as independent and stand-alone entities. File and folder objects would expose a programmable API in terms of methods and properties, both standard and defined by the owner or the author.

What we have today, instead, is a file system that registers files and folders in some internal structures, which are duplicated when the files and folders are moved around the disks. Files and folders have a fixed set of features that is too small for the needs of modern applications. As a partial workaround, we've been given over the last few years several techniques for adding extra information to files and folders. Shell and namespace extensions, the desktop.ini file, the FileSystemObject, and the Shell Automation Object Model are just a few examples. However, all these functionalities are just spot and local solutions. They completely miss the point of an organic reengineering of the Windows® file system. Because backward compatibility is a serious issue, Windows is still utilizing an old-fashioned file system built on top of the file allocation table (FAT), the dawning of which dates back to Microsoft MS-DOS® version 2.0! Even with some more recent improvements, such as support for high-capacity hard disks, FAT remains a rather inadequate way of storing file and folder information.

Years of real-world experience demonstrate that the most significant limitations we run into have to do with the additional information programmers need to properly manage and identify files. Recently, I was asked to try to retrieve the real creation date for a Word 97 document. You may think it is an easy task since the creation date is an attribute you can easily retrieve through some API functions. This is only partially true. Try copying the same Word file on a different machine, or even in the same folder, and then compare the creation date of both copies. Surprisingly, they differ! While making a copy, you create a brand new file with the time stamp of when the creation occurs. Working on a copy, you lose potentially valuable information concerning when the file was originally created.

Fortunately, a Word document retains such information internally in the SummaryInformation fields. So, in my case I was able to solve the problem and successfully bill the client. Had it been an Access or a text file, my effort would have failed.

With Windows NT, Microsoft introduced a new file system called NTFS. Among its most notable features is the B-tree structure, which speeds up file retrieval on large folders, file-based security, logging, enhanced file system recoverability, and a much better use of disk space than FAT or FAT32. (By the way, Windows 2000 provides full support and access to FAT32 volumes.)

Since their advent with Windows 3.1, NTFS volumes also have another, often underestimated feature: They support multiple streams of data into a single file. With Windows 2000, the stream support has been reinforced, and other rather handy features have been added to help you work seamlessly with files. Let's look at the major features of NTFS 2000, the version of NTFS that comes with Windows 2000.

Overview of NTFS 2000

If multiple streams of data aren't an exclusive feature of NTFS 2000 volume files, there are several other features that require Windows 2000 to work. They are:

  • File and directory encryption
  • Per-user, per-volume disk quotas
  • Reparse points and hierarchical storage management
  • Mount points
  • Hard links
  • Change Journal

During the Windows 2000 installation, you're asked to specify whether you want your Windows 2000 volume to be converted to NTFS 2000. The use of the NTFS 2000 file system, though, is required only on machines acting as domain controllers. You can convert a FAT partition to NTFS at any time by using the command-line utility convert.exe:

   CONVERT volume /FS:NTFS [/V]

The volume argument specifies the drive letter followed by a colon. It could be also a mount point, or a volume name. The /FS:NTFS option specifies that the volume must be converted to NTFS. Finally, use /V if you want the utility to run in verbose mode. When you run convert.exe it does some initialization and then asks you to reboot. The conversion will take place immediately upon next startup.

In addition to all the features listed above, a remarkable aspect of the Windows 2000 overall folder management is the full and somewhat extended support it provides for the desktop.ini files. In the remainder of this article, I'll focus primarily on streams and hard links. Table 1, however, summarizes the most important points pertaining to the other key NTFS 2000 features.

Table 1. Key Features of NTFS 2000

Feature Description
Encrypted file system Administrators can selectively encrypt files and folders on NTFS 2000. The encryption is transparent to the user. The core Windows API is aware of the encryption attribute for a given file. To preserve it, don't use your own function to move/copy files but rely on system's CopyFile() or MoveFile().
Disk quotas With disk quotas you can assign a maximum of disk space to each user and even be notified when he or she exceeds an intermediate warning level. Quotas are transparent to users who simply see the amount of free disk space.
Reparse points A reparse point is a collection of user-defined data you assign to a file or a folder. The format of this data is understood by the application, which stores the data, and a file system filter, which you install to interpret and process it for that file or folder.
Sparse files A sparse file is a very large file without a lot of data in it. When the sparse file facilities are used, the system does not allocate hard drive space to a file except in regions where it contains something other than zeros.
Mount points A volume mount point is an existing path where you "mount" another volume. Given this, users and applications can refer to the mounted volume by that path. It allows you to unify into one logical file system disparate file systems.
Change Journal The Change Journal is a database that contains a list of every changed file or directory on an NTFS 2000 volume. Each volume has its own journal with records reflecting all the file and folder changes that have occurred.
Desktop.ini Everything inherent to the folder customization is kept in the desktop.ini file. It is a small text file that stores information about the icon and infotip of the folder. With Windows 2000, such a file is fully supported at the UI level.

Multiple File Streams

Under an NTFS file system each file can have multiple streams of data. It's worth pointing out that streams are not a feature of NTFS 2000, but they have been in existence since Windows NT 3.1. When you read the content of a file under a non-NTFS volume (say, a disk partition of a Windows 98 machine) you're able to access only one stream of data. Consequently, you perceive it as the real and "unique" content for that file. Such a main stream has no name and is the only one that non-NTFS file systems can handle. But when you create a file on an NTFS volume, things might be different. Look at Figure 1 to understand the big picture.

Figure 1. The structure of a multi-stream file

A multi-stream file is a sort of collection of single-stream files all embedded in the same file system entry. They definitely look like a unique and atomic unit, yet comprise a number of independent sub-units you can create, delete, and modify separately. There are a number of common programming scenarios where streams are more than handy. However, if you plan to use them, bear in mind that as soon as you copy a multi-stream file to a non-NTFS storage device (such as a CD, a floppy, or a non-NTFS disk partition), all extra streams will be unrecoverable if lost. Unfortunately, such a compatibility issue makes streams much less attractive in practice. For server-side applications designed and destined to run only on NTFS volumes, streams are an excellent tool to leverage to build great and creative solutions.

Fundamentals of Streams

When you copy a multi-stream file on non-NTFS volumes, only the main stream is copied. This means you lose your extra data, because they can't come up again even if you copy the file back to an NTFS disk. For now, let's assume you're working exclusively on NTFS machines and let's see how to create named streams. In Code Sample 1 you can see a Windows Script Host (WSH), Microsoft Visual Basic® Scripting Edition (VBScript) file that demonstrates how to read and write streams from an NTFS file.

To identify a named stream within a file you should follow a particular naming convention and add at the end of the file name a colon followed by the stream name. For example, to access a stream called VersionInfo on a file called test.txt you should use the file name:


Use it with any Microsoft Win32® API function that manipulates files. To access the content of the VersionInfo stream, pass that name on to CreateFile() and then use ReadFile() and WriteFile() to accomplish reading and writing as usual. If you want to check whether a certain stream exists within a file, compose the file stream name as just shown and use CreateFile() to check it for existence:

HANDLE hfile = CreateFile(szFileStreamName, GENERIC_READ, 0, 
if (hfile == NULL)
   MessageBox(hWnd, "Error", NULL, MB_OK);

To work with streams, you don't necessarily need to be an intrepid C++ programmer. You can take advantage of streams also in Visual Basic and even with script code, as Code Sample 1 shows. The key factor that enables this sort of transparency is that all the low-level Win32 API functions, CreateFile() in particular, support stream-based file names on NTFS partitions. If you try to open a file called Test.txt:VersionInfo on a non-NTFS partition, for example on a Windows 98 machine, you'll get a "file not found" error message. Notice that what matters is only the file system of the volume that contains the file, not the Windows platform or the disk partition type hosting the calling application. In other words, you can successfully access a certain named stream on a shared folder on an NTFS partition also from a connected Windows 98 machine. Furthermore, consider that the colon is not a valid character, even for long file names. So when CreateFile() encounters that in a file name, it knows it has a special meaning.

As of Code Sample 1, you can use streams with VBScript too, because the FileSystemObject object model makes intensive use of CreateFile() to open, write, create, and test files. In the sample code, I'm creating a text file with an empty, 0-length main stream and as many named streams as you want. Try running the demo and create a couple of streams. Let's say you call them VersionInfo and VersionInfoEx. There's nothing in the Windows shell that can lead you to suppose the presence of streams within a certain file. In Figure 2 you can see how the test.txt file looks within Windows Explorer.

Figure 2. A file can be 0-length but have named streams.

The Size column shows only the size of the main, unnamed stream and not even in the Properties dialog box can you get more information about streams. Although only on NTFS volumes, the Windows 2000 Properties dialog box gives you a chance to associate summary information to all files, including text files. Click the Summary tab and enter, for example, an author name, as shown in Figure 3.

Incidentally, such a name can be displayed on a specific Author column due to the shell UI enhancements of Windows 2000. Read about that in the premiere issue of MSDN Magazine at

Figure 3. Associating extra information for a .txt file on an NTFS volume

Hey, wait a minute. The summary information is typical data you set for Word or Excel documents but is definitely part of the document itself. How is it possible to associate it with a text file without altering the plain content? Elementary, Watson. The shell does that through streams! Once you apply those changes, try copying the file on another non-NTFS partition. The dialog box shown in Figure 4 will appear.

Figure 4. Windows 2000 forewarns about possible stream data loss.

It proves that the test.txt file contains a stream with document summary information. The system realizes when you're going to copy a file with extra information to a volume that doesn't support that. On non-NTFS partitions only the main unnamed stream is copied and the rest are discarded. For this reason, stream-based files are hardly to be exchanged if the target is not compliant.

Streams Backup and Enumeration

Is there a way—an API function or two—to enumerate all the streams a certain file has? Yes, there is, but it's not as easy and intuitive as it should be. The Win32 backup API functions (BackupRead, BackupWrite, and so forth) could be used to enumerate the streams within a file. However, they are a bit quirky to use and look more like a workaround than an effective and definitive solution.

The idea is that when you want to back up a file or an entire folder, you need to package and store all possible information. For this reason, BackupRead() is your best friend when it comes to trying to enumerate the streams of a file. Let's focus on the function's prototype:

BOOL BackupRead(
  HANDLE hFile,          
  LPBYTE lpBuffer,        
  DWORD nNumberOfBytesToRead,  
  LPDWORD lpNumberOfBytesRead,  
  BOOL bAbort,                  
  BOOL bProcessSecurity,        
  LPVOID *lpContext             

For our purposes, you can ignore here aspects like context and security. The hFile argument must be obtained through a call to CreateFile(), while lpBuffer should point to a WIN32_STREAM_ID data structure:

typedef struct _WIN32_STREAM_ID { 
    DWORD         dwStreamId; 
    DWORD         dwStreamAttributes; 
    DWORD         dwStreamNameSize; 
    WCHAR         cStreamName[ANYSIZE_ARRAY]; 

The first 20 bytes of such a structure represent the header of each stream. The name of the stream begins immediately after the dwStreamNameSize field and the content of the stream follows the name. Because the traditional content of the file can be seen as a stream, although an unnamed stream, to enumerate all the streams, you just need to loop until BackupRead returns False. BackupRead, in fact, is supposed to read all the information associated with a given file or folder:

ZeroMemory(&sid, sizeof(WIN32_STREAM_ID));
DWORD dwStreamHeaderSize = (LPBYTE)&sid.cStreamName - 
      (LPBYTE)&sid+ sid.dwStreamNameSize;
bContinue = BackupRead(hfile, (LPBYTE) &sid, 
   dwStreamHeaderSize, &dwRead, FALSE, FALSE, 

The preceding snippet is the crucial code that reads in the header of the stream. If the operation is successful, you can attempt to read the actual name of the stream:

WCHAR wszStreamName[MAX_PATH]; 
BackupRead(hfile, (LPBYTE) wszStreamName, sid.dwStreamNameSize, 
   &dwRead, FALSE, FALSE, &lpContext);

Before attacking with the next stream, first move forward the backup pointer by calling BackupSeek():

BackupSeek(hfile, sid.Size.LowPart, sid.Size.HighPart, 
   &dw1, &dw2, &lpContext);

In most cases, you can treat streams as if they were regular files—for example, to delete a stream resort to DeleteFile(). If you want to refresh their content, just use ReadFile() and WriteFile(). There's no official and supported way to move or rename streams. In the final part of the article, I'll utilize this code to build an NTFS 2000-specific Windows shell extension that adds a new property page to all files with stream information. In the meantime, let's take a whirlwind tour of another NTFS feature.

Hard Links

Do you know about shortcuts—those little .lnk files, mostly sprinkled around your desktop, that you use to reference something else? Well, no doubt shortcuts are a useful feature, but they also have a few drawbacks. First off, if you have multiple shortcuts pointing to the same target from different folders, actually you have multiple copies of the same—fortunately, rather small—file. More importantly, the target object of a shortcut may change over time. It might be moved, deleted, or simply renamed. What about your shortcuts? Are they capable of detecting those changes and tracking them down, (auto)updating properly? Unfortunately, they can't. The main reason for this is that shortcuts are an application-level feature. From the system's point of view, they are just user-defined files that simply require some extra work when you try to open them. Consider that being a shortcut is a privilege you might decide to assign also to other classes of files. Provided it makes sense, you can create your own class of shortcuts with an extension other than .lnk. What does the job is a registry entry named IsShortcut under the class node. Suppose you want to treat .xyz files as shortcuts. Register the file class by creating an .xyz node under HKEY_CLASSES_ROOT and make it point to another node, usually xyzfile. Then add an empty REG_SZ entry to:


and you're done.

Other operating systems, specifically Posix and OS/2, have a similar feature that acts at the system level. OS/2, in particular, calls them shadows. A hard link is a file system-level shortcut for a given file. By creating a hard link to an existing file, you duplicate neither the file nor a file-based reference (that is, a shortcut) to it. Instead, you add information to its directory entry at the NTFS level. The physical file remains intact in its original location. Simply put, it now has two or more names that you can use to access the same content!

A hard link saves you from maintaining multiple (but needed) copies of the same file, making the system responsible for managing various path names to address a single physical content. This greatly simplifies your work and saves valuable disk space. Furthermore, hard links, as system-level shortcuts, always point to the right target file—no matter if you rename or move it. Because the link is stored at the file system level, all changes apply automatically and transparently. It's worth noting that hard links must be created within the same NTFS volume. You cannot have a hard link on, say, drive C: pointing to a file on drive D:.

If it sounds more familiar, think of a hard link as an alias for a file. You could use any alias to access it and the file gets deleted only when you delete all of its aliases. (Aliases act like a reference count.) Because hard links are aliases, synchronizing the content is simply a non-issue.

CreateHardLink() is the API function used to create hard links. Its prototype looks like this:

BOOL CreateHardLink(
  LPCTSTR lpFileName,                          
  LPCTSTR lpExistingFileName,               
  LPSECURITY_ATTRIBUTES lpSecurityAttributes  

As the companion code for an old MIND article (see "Windows 2000 for Web Developers," MIND, March 1999), I provided a COM object allowing you to create hard links from script code. Code Sample 2 shows a VBScript program that utilizes it to create hard links for a given file. While it's easy to discover how many hard links a file has, there's no facility to enumerate them all. The API function GetFileInformationByHandle() fills out a BY_HANDLE_FILE_INFORMATION structure, whose nNumberOfLinks field informs you about that. Enumerating all the names of the linked files is a bit more difficult. Basically, you must scan the entire volume and, for each file, keep track of the unique ID it's been assigned. When you run into an existing ID you've found a hard link for that file. The file's unique ID is assigned by the system and is stored in the nFileIndexHigh and nFileIndexLow fields of BY_HANDLE_FILE_INFORMATION.

Enjoy NTFS Features

Streams are particularly useful for adding extra information to files without altering or damaging the original format and taking up disk space. Streams, of course, occupy their natural space, but Windows Explorer appears unaware of this. Streams are invisible to Windows Explorer, so while it appears there is plenty of free disk space, in actuality free space is dangerously low. You can add extra (and invisible) information to any file, including text and executables.

On the other hand, hard links are a great resource to centralize shared information. You have just one real repository for information that can be accessed from a variety of different paths. Consider that hard links aren't an entirely new concept for the Windows NT technology. Hard links have been around since the inception of Windows NT, but not until Windows 2000 did Microsoft provide a public function to create them. Each file has at least one link to itself so that GetFileInformationByHandle always returns a number of links greater than zero. You cannot set hard links to a directory, only to files.

A practical problem that streams and hard links share is their very limited support from the shell. In an effort to remedy this, I've written a shell extension to provide information about the streams and the hard links of a given file. Figure 5 illustrates its look and feel.

Figure 5. The Streams tab shows information about streams and hard links.

The source code for the shell extension enumerates the streams using the BackupRead() API function. The content of the selected stream can be deleted with a simple call to DeleteFile(). The Edit Streams button runs the script code in Code Sample 1 by means of which you can add and update streams. Likewise, the Create Hard Link button runs the code in Code Sample 2 to create additional links. The user interface reflects all changes only if you refresh. As a final note, bear in mind that if you delete a hard link (which means deleting a file) the total number of links isn't updated as long as the deleted files remain in the Recycle Bin.


In this article, I've just scratched the surface of NTFS 2000, focusing on key features such as streams and hard links. For a wider perspective of what's new in the Windows 2000 file system, I suggest you refer to the article "A File System for the 21st Century: Previewing the Windows NT 5.0 File System," written by Jeff Richter and Luis Cabrera for Microsoft Systems Journal, November 1998. Interesting topics, in particular sparse streams and reparse points, have not been addressed here, but if you enjoyed this article, please let us know and we'll soon provide a follow-up.

Code Sample 1

' CreateStream.vbs
' Demonstrates streams on NTFS volumes 
' --------------------------------------------------------

Option Explicit

' Some constants
Const L_NotNTFS = "Sorry, the current volume is not NTFS."
Const L_EnterFile = "Enter a file name"
Const L_TestNTFS = "Test NTFS"
Const L_StdFile = "c:\testntfs\test.txt"
Const L_EnterStream = "Enter a stream name"
Const L_StdStream = "VersionInfo"
Const L_EnterTextStream = "Enter the text of the stream"
Const L_StdContent = "1.0"

' Makes sure the current volume is NTFS
if Not IsNTFS() then 
   MsgBox L_NotNTFS
end if

' Query for a file name
dim sFileName
sFileName = InputBox(L_EnterFile, L_TestNTFS, L_StdFile)
if sFileName = "" then WScript.Quit

' Query for the stream to be written
dim sStreamName
sStreamName = InputBox (L_EnterStream, L_TestNTFS, L_StdStream)
if sStreamName = "" then WScript.Quit

' Initializes the FS object model 
dim fso, bExist
set fso = CreateObject("Scripting.FileSystemObject")   

' Creates the file if it doesn't exist
dim ts
if Not fso.FileExists(sFileName) then 
   set ts = fso.CreateTextFile(sFileName)
end if 

' Try to read the current content of the stream
dim sFileStreamName, sStreamText
sFileStreamName = sFileName & ":" & sStreamName
if Not fso.FileExists(sFileStreamName) then 
   sStreamText = L_StdContent
   set ts = fso.OpenTextFile(sFileStreamName)
   sStreamText = ts.ReadAll()
end if 

' Query for the content of the stream to be written
sStreamText = InputBox (L_EnterTextStream, L_TestNTFS, sStreamText)
if sStreamText = "" then WScript.Quit

' Try to write to the stream
set ts = fso.CreateTextFile(sFileStreamName)
ts.Write sStreamText

' Close the app
set ts = Nothing
set fso = Nothing

' ////////////////////////////////////////////////////////
' // Helper functions

' IsNTFS() - Verifies whether the current volume is NTFS
' --------------------------------------------------------
function IsNTFS()
   dim fso, drv
   IsNTFS = False
   set fso = CreateObject("Scripting.FileSystemObject")   
   set drv = fso.GetDrive(fso.GetDriveName(WScript.ScriptFullName)) 
   set fso = Nothing
   if drv.FileSystem = "NTFS" then IsNTFS = True
end function

Code Sample 2

' Hardlinks.vbs
' Demonstrates hard links on NTFS volumes 
' --------------------------------------------------------

Option Explicit

' Some constants
Const L_NoHardLinkCreated = "Unable to create hard link"
Const L_EnterTarget = "Enter the file name to hard-link to"
Const L_HardLinks = "Creating hard link"
Const L_EnterHardLink = "Name of the hard link you want to create"
Const L_CannotCreate = "Make sure that both files are on the same volume and the volume is NTFS"
Const L_NotExist = "Sorry, the file doesn't exist"
Const L_SameName = "Target file and hard link cannot have the same name"

' Determine the existing file to (hard) link to
dim sTargetFile 
if WScript.Arguments.Count >0 then
   sTargetFile = WScript.Arguments(0)
   sTargetFile = InputBox(L_EnterTarget, L_HardLinks, "")
   if sTargetFile = "" then WScript.Quit
end if

' Does the file exist?
dim fso
set fso = CreateObject("Scripting.FileSystemObject")   
if Not fso.FileExists(sTargetFile) then
   MsgBox L_NotExist
end if

' Main loop
while true
   QueryForHardLink sTargetFile

' Close up

' /////////////////////////////////////////////////////////////
' // Helper Functions

' Create the hard link
function QueryForHardLink(sTargetFile)
   ' Extract the hard link name if specified on the command line
   dim sHardLinkName
   if WScript.Arguments.Count >1 then
      sHardLinkName = WScript.Arguments(1)
      dim buf
      buf = L_EnterHardLink & " for" & vbCrLf & sTargetFile
      sHardLinkName = InputBox(buf, L_HardLinks, sTargetFile)
      if sHardLinkName = "" then WScript.Quit   
      if sHardLinkName = sTargetFile then 
         MsgBox L_SameName
         exit function
      end if
   end if 

   ' Verify that both files are on the same volume and the 
   ' volume is NTFS
   if Not CanCreateHardLinks(sTargetFile, sHardLinkName) then 
      MsgBox L_CannotCreate
      exit function
   end if
   ' Creates the hard link
   dim oHL
   set oHL = CreateObject("HardLink.Object.1")
   oHL.CreateNewHardLink sHardLinkName, sTargetFile
end function

' Verify that both files are on the same NTFS disk
function CanCreateHardLinks(sTargetFile, sHardLinkName)
   CanCreateHardLinks = false
   dim fso
   set fso = CreateObject("Scripting.FileSystemObject")
   ' same drive?
   dim d1, d2
   d1 = fso.GetDriveName(sTargetFile)
   d2 = fso.GetDriveName(sHardLinkName)
   if d1 <> d2 then exit function

   ' NTFS drive?
   CanCreateHardLinks = IsNTFS(sTargetFile)
end function

' IsNTFS() - Verifies whether the file's volume is NTFS
' --------------------------------------------------------
function IsNTFS(sFileName)
   dim fso, drv
   IsNTFS = False
   set fso = CreateObject("Scripting.FileSystemObject")   
   set drv = fso.GetDrive(fso.GetDriveName(sFileName)) 
   set fso = Nothing
   if drv.FileSystem = "NTFS" then IsNTFS = True
end function

© 2008 Microsoft Corporation. All rights reserved. Terms of Use  |  Trademarks  |  Privacy Statement
Page view tracker