Multithreading

Aug 6, 2010 at 1:24 AM

Ken,

Im horribly confused on what level of multithreading the library supports.  I've create a new vhd, and have multiple threads writing to the vhd.  Each file open/read/write/etc is protected by a shared object but I'm clearly running into issues.  I presumed the system reset the 'real' file point for the 'real' underlying stream as multiple higherlevel streams where used.  Can you clarify what is expected, specifically an I open file for write on a vhd in thread a, open a seperate file in thread b, and write to both if they are blocked on a shared object (e.g. only one can write at a time).  Or can only one file stream be open on a volume at a time?

Bill

 

Coordinator
Aug 7, 2010 at 11:40 AM

Hi Bill,

There's no locking within DiscUtils - so you should only have a single thread modifying/reading any one virtual disk / file system at a time, which it sounds like is what your locking is aiming for.

It's certainly intended that you can have multiple files open concurrently and interlace reads & writes between the open files (again, so long as only one thread at a time is actually reading or writing at a time).

Note - it's not just writing threads that need to be exclusive - it's not safe to have multiple threads reading at the same time, either.

 

There aren't many (any?) unit tests for interlaced I/O on multiple files, so it's possible there are bugs.

What kind of things are you observing?

 

Cheers,

Ken

Aug 7, 2010 at 4:48 PM

Hi Ken,

I've been running a number of experiments to reverse engineer the threading rules.  Here is what I found, let me know if it's what is expected (from the above it sounds like 'a' is right 'b' is not)

a) If I have an NTFS volume mounted read only.  I can open multiple file handles to files on that volume in differing threads.  I share a lock across all of the threads so only one can be inside any discutils operation at any one time.  If a thread enumerates files (getfiles[] getdirectories[] etc) these also need to be syncronized.  That was my initial error, the threads with open file handles were blocking but not the enumeration thread.  That caused immediate issues ,)

b) The writing case appears to be different.  If I have a writeable volume and I open multiple write handles to the drive, it appears I can not syncronize the writes.  I quickly start getting exceptions thrown which suggest discutils has lost it's position on the underlying volume (guessing there is a place where the real underlying file pointer isn't kept/updated).  This also appears to occur if one thread has a file open and another thread calls any funtion to modify the file system (CreateDirectory(), CreateHardlink(), etc).  These are (like the reads) also syncronized on a shared object.  (Then again, the write to file case may be the root problem, since just multiple writes don't appear to work when I factor out the file system modifications [e.g. dirs, hardlinks, etc]).

The fact that you indicate this should work is good (didn't know if I was doing something that you'd say 'no way that will ever work').  I'll try to see if I can find some commonality in which functions may be the culprit (I didnt, for example, test CreateDirectory seperate from CreateHardlink() , etc yet).

System.IO.IOException occurred
  Message=Unable to complete read of 512 bytes
  Source=DiscUtils
  StackTrace:
       at DiscUtils.Utilities.ReadFully(Stream stream, Int32 count) in C:\Users\Bill Sobel\Documents\Visual Studio 2010\Projects\UtilV Optimizer\DiscUtils\DiscUtils\src\Utilities.cs:line 698
  InnerException: 

  DiscUtils.dll!DiscUtils.Utilities.ReadFully(System.IO.Stream stream, int count) Line 698 C#
  DiscUtils.dll!DiscUtils.Vhd.DynamicStream.Write(byte[] buffer, int offset, int count) Line 350 + 0x16 bytes C#
  DiscUtils.dll!DiscUtils.SubStream.Write(byte[] buffer, int offset, int count) Line 176 + 0x4b bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.NonResidentAttributeBuffer.RawWrite(long position, byte[] data, int dataOffset, int count) Line 459 + 0x61 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.NonResidentAttributeBuffer.Write(long pos, byte[] buffer, int offset, int count) Line 158 + 0x30 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.NtfsAttributeBuffer.Write(long pos, byte[] buffer, int offset, int count) Line 238 + 0x5b bytes C#
  DiscUtils.dll!DiscUtils.BufferStream.Write(byte[] buffer, int offset, int count) Line 167 + 0x51 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.File.FileStream.Write(byte[] buffer, int offset, int count) Line 1170 + 0x4b bytes C#
> DiscUtils.dll!DiscUtils.Ntfs.NtfsFileStream.Write(byte[] buffer, int offset, int count) Line 135 + 0x35 bytes C#

Any suggestions on how to best debug this, or what data I can generate for you to help track this down?

Bill

 

Aug 7, 2010 at 5:04 PM

Another example of the exceptions I get when trying this (in this case diskMagic was 'FILE' and _magic wsa 'INDX')

System.IO.IOException occurred
  Message=Corrupt record
  Source=DiscUtils
  StackTrace:
       at DiscUtils.Ntfs.FixupRecordBase.FromBytes(Byte[] buffer, Int32 offset, Boolean ignoreMagic) in C:\Users\Bill Sobel\Documents\Visual Studio 2010\Projects\UtilV Optimizer\DiscUtils\DiscUtils\src\Ntfs\FixupRecordBase.cs:line 106
  InnerException:

 DiscUtils.dll!DiscUtils.Ntfs.FixupRecordBase.FromBytes(byte[] buffer, int offset, bool ignoreMagic) Line 106 C#
  DiscUtils.dll!DiscUtils.Ntfs.FixupRecordBase.FromBytes(byte[] buffer, int offset) Line 86 + 0x17 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.IndexBlock.IndexBlock(DiscUtils.Ntfs.Index index, DiscUtils.Ntfs.IndexNode parentNode, DiscUtils.Ntfs.IndexEntry parentEntry, DiscUtils.Ntfs.BiosParameterBlock bpb) Line 55 + 0x12 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.Index.GetSubBlock(DiscUtils.Ntfs.IndexNode parentNode, DiscUtils.Ntfs.IndexEntry parentEntry) Line 158 + 0x47 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.Index.FindAllIn(System.IComparable<byte[]> query, DiscUtils.Ntfs.IndexNode node) Line 266 + 0x4c bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.Index.FindAll(System.IComparable<byte[]> query) Line 123 + 0x218 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.IndexView<DiscUtils.Ntfs.FileNameRecord,DiscUtils.Ntfs.FileRecordReference>.FindAll(System.IComparable<byte[]> query) Line 477 + 0x450 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.IndexView<DiscUtils.Ntfs.FileNameRecord,DiscUtils.Ntfs.FileRecordReference>.FindFirst(System.IComparable<byte[]> query) Line 485 + 0x1b3 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.Directory.GetEntryByName(string name) Line 52 + 0xc0 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.NtfsFileSystem.GetDirectoryEntry(DiscUtils.Ntfs.Directory dir, string[] pathEntries, int pathOffset) Line 1887 + 0x5b bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.NtfsFileSystem.GetDirectoryEntry(DiscUtils.Ntfs.Directory dir, string path) Line 1841 + 0x17 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.NtfsFileSystem.GetDirectoryEntry(string path) Line 1835 + 0x32 bytes C#
  DiscUtils.dll!DiscUtils.Ntfs.NtfsFileSystem.OpenFile(string path, System.IO.FileMode mode, System.IO.FileAccess access) Line 789 + 0x10 bytes C#

Aug 7, 2010 at 5:15 PM

Ok, this is probably a good hint...  I removed all of the statements that wrote to a file from my code (e.g. removed my SetLength, Write, SetFileAttributes, SetFileSystemSecurity, SetXTimeUtc(), etc) and just left the open and close call (syncronized on a shared object) in the code.  That is enough to trigger the corruption everytime after just a few attempts.

 d = DestinationNtfsFileSystem.OpenFile(PathName, FileMode.Create, FileAcces.Write);

 d.Close();

Coordinator
Aug 9, 2010 at 10:33 PM

Hi Bill,

I've been trying to repro this, but without much success (I've been using a single thread and interleaving different operations, so I'm confident there's only one thread inside DiscUtils at a time).

Is it possible to post a version of your code that shows the problem?

 

Cheers,

Ken

Aug 12, 2010 at 12:52 AM

Ken, Im sorry to have spun you on this... I tried to create a repro application trimming out everything possible (which was of course not showing the issue).  Finally with quite a bit of debug logging I found an unexpected exception which failed wound up calling .Close() on a discutils stream without locking the right object (I've been using the VirtualDisk object as a lock, this locked the partitions file system).  This explains the corruption I was seeing.  Once changed all of my tests have worked with flying colors.

Best,

Bill