I have a large number of audio files I am running through a processing algorithm to attempt to extract certain bits of data from it (ie: average volume of the entire clip). I have a number of build scripts that previously pulled the input data from a Samba network share, which I've created a network drive mapping to via net use (ie: M: ==> \\server\share0).
Now that I have a new massive 1TB SSD, I can store the files locally and process them very quickly. To avoid having to do a massive re-write of my processing scripts, I removed my network drive mapping, and re-created it using the localhost host name. ie: M: ==> \\localhost\mydata.
When I make use of such a mapping, do I risk incurring significant overhead, such as from the data having to travel through part of Windows' network stack, or does the OS use any shortcuts so it equates more-or-less to direct disk access (ie: does the machine know it's just pulling files from its own hard drive). Increased latency isn't much of a concern of mine, but maximum sustained average throughput is critical.
I ask this because I'm deciding whether or not I should modify all of my processing scripts to work with a different style for network paths.
Extra Question: Does the same apply to Linux hosts: are they smart enough to know they are pulling from a local disk?
答案 0 :(得分：5)
当我使用这样的映射时，我是否有可能产生大量开销，
是。通过使用UNC路径（\\hostname\sharename\filename）而不是本地路径（[\\?\]driveletter:\directoryname\filename），您可以通过服务器消息块协议（SMB / Samba）发出所有流量。这通常会增加磁盘访问和访问时间方面的显着开销。
网络流量如下：
Application -> SMB Client -> Network -> SMB Server -> Target file system
现在通过将文件移动到本地计算机，但仍使用UNC访问它们，流程如下：
Application -> SMB Client -> localhost -> SMB Server -> Target file system
您唯一最小化（未消除，到localhost的SMB流量仍涉及网络层以及所有计算和流量关联）是网络流量。
此外，鉴于SMB专门针对网络流量而定制，其读取可能无法最佳地使用磁盘和操作系统的缓存。例如，它可以在一定大小的块中执行读取，而在读取另一个大小的块时，磁盘的性能会更好。
如果您想要最佳吞吐量和最短访问时间，请尽可能使用中间层，在这种情况下直接访问文件系统：
Application -> Target file system
答案 1 :(得分：4)
确保使用TCP直接文件访问即使使用＆＃34; loopback＆＃34;在Linux和Windows上都有路由，内存分配等开销，是的，环回设备是非物理内核设备，比其他网络设备更快，但不比直接文件访问快。据我所知，在Windows上还有其他的环回优化，如NetDNA和＆＃34; Fast TCP Loopback＆＃34;。
我认为环回设备的瓶颈将是内存（复制）进程。因此，在Linux和Windows上，直接访问文件而不是通过环回设备将总是更快（并且资源消耗低）。
此外，两个操作系统都通过＆＃34;命名管道来解决IPC的协议开销。在Windows和＆＃34; unix域套接字＆＃34;在Linux上，使用它们也比使用环回设备更快。