@ronomon/direct-io - npm Package Compare versions

		@@ -7,3 +7,3 @@ var Node = {
		var Queue = require('@ronomon/queue');
		var binding = require('./binding.node');
		var binding = require('.');

		@@ -10,0 +10,0 @@ const ALIGNED = 1;

package.json

		{
		"name": "@ronomon/direct-io",
		"version": "2.4.3",
		"version": "3.0.0",
		"description": "Direct IO helpers for block devices and regular files on FreeBSD, Linux, macOS and Windows.",
		@@ -8,4 +8,6 @@ "main": "binding.node",
		"benchmark.js",
		"binding.cc",
		"binding.gyp"
		"binding.c",
		"binding.gyp",
		"get-block-device",
		"test.js"
		],
		@@ -56,5 +58,4 @@ "repository": {
		"dependencies": {
		"@ronomon/queue": "^3.0.0",
		"nan": "^2.11.0"
		"@ronomon/queue": "^3.0.1"
		}
		}

119

README.md

		@@ -6,2 +6,12 @@ # direct-io

		* [Installation](#installation)
		* [Direct memory access](#direct-memory-access)
		* [Buffer alignment](#buffer-alignment)
		* [Synchronous writes](#synchronous-writes)
		* [Block device size and sector size](#block-device-size-and-sector-size)
		* [Block device path and permissions](#block-device-path-and-permissions)
		* [Mandatory locks](#mandatory-locks)
		* [Advisory locks](#advisory-locks)
		* [Benchmark](#benchmark)

		## Installation
		@@ -15,7 +25,7 @@

		Direct Memory Access bypasses the filesystem cache and avoids memory copies to
		and from kernel-space, writing and reading directly to and from the disk device
		cache. File I/O is done directly to and from user-space buffers regardless of
		whether the file descriptor is a block device or regular file. To enable Direct
		Memory Access, use the following open flag or method according to the platform:
		Direct memory access bypasses the filesystem cache and avoids memory copies to
		and from kernel space, writing and reading directly to and from the disk device
		cache. File I/O is done directly to and from user space buffers regardless of
		whether the file descriptor is a block device or regular file. To enable direct
		memory access, use the following open flag or method according to the platform:

		@@ -25,3 +35,3 @@ O_DIRECT (FreeBSD, Linux, Windows)
		Provide as a flag to `fs.open()` when opening a block device or regular file to
		enable Direct Memory Access.
		enable direct memory access.

		@@ -42,17 +52,17 @@ On Windows, `O_DIRECT` is supported as of

		Please note that turning data caching off with `F_NOCACHE` [will not purge any
		previously cached pages](https://lists.apple.com/archives/filesystem-dev/2007/Sep/msg00012.html). Subsequent direct reads may continue to return
		cached pages if they exist, and concurrent processes may continue to populate
		the cache through non-direct reads. To ensure direct reads on macOS (for example
		when data scrubbing) you should set `F_NOCACHE` as soon as possible to avoid
		populating the cache yourself.
		Turning data caching off with `F_NOCACHE` [will not purge any previously cached
		pages](https://lists.apple.com/archives/filesystem-dev/2007/Sep/msg00012.html).
		Subsequent direct reads will continue to return cached pages if they exist, and
		concurrent processes may continue to populate the cache through non-direct
		reads. To ensure direct reads on macOS (for example when data scrubbing) you
		should set `F_NOCACHE` as soon as possible to avoid populating the cache.

		Alternatively, if you want to ensure initial boot conditions with a cold disk
		buffer cache, you can purge the entire cache for all files using `sudo purge`.
		Please note that this will affect system performance.
		This will affect system performance.

		## Buffer Alignment

		When writing or reading to and from a block device or regular file using Direct
		Memory Access, you need to make sure that your buffer is aligned correctly or
		When writing or reading to and from a block device or regular file using direct
		memory access, you need to make sure that your buffer is aligned correctly or
		you may receive an `EINVAL` error or be switched back silently to non-DMA mode.
		@@ -63,4 +73,4 @@
		block device. Buffers allocated using Node's `Buffer.alloc()` and related
		methods will typically not meet these alignment requirements. Use
		`getAlignedBuffer()` to create properly aligned buffers:
		methods will not meet these alignment requirements. Use `getAlignedBuffer()` to
		create aligned buffers:

		@@ -73,4 +83,8 @@ getAlignedBuffer(size, alignment) (FreeBSD, Linux, macOS, Windows)
		the block device (typically 512 bytes or 4096 bytes).
		* `alignment` must be greater than 0, a power of two, and a multiple of the
		physical sector size of the block device.
		* `size` must be at most `require('buffer').kMaxLength` bytes.
		* `alignment` must be at least 8 bytes for portability between 32-bit and 64-bit
		systems.
		* `alignment` must be a power of two, and a multiple of the physical sector size
		of the block device.
		* `alignment` must be at most 4194304 bytes for a safe arbitrary upper bound.
		* An alignment of 4096 bytes should be compatible with
		@@ -87,7 +101,8 @@ [Advanced Format](https://en.wikipedia.org/wiki/Advanced_Format) drives as well
		depending on the platform).
		* Buffers are not zero-filled. The contents of newly created aligned Buffers
		are unknown and may contain sensitive data.
		* Buffers are zero-filled with `memset()` when allocated for safety.
		* Buffers are automatically freed using the appropriate call when garbage
		collected in V8 (either `free()` or `_aligned_free()` depending on the
		platform).
		collected in V8, either `free()` or `_aligned_free()` depending on the
		platform.
		* `getAlignedBuffer()` should be used judiciously as the algorithm that realizes
		the alignment constraint can incur significant memory overhead.

		@@ -100,3 +115,3 @@ Further reading:

		Direct Memory Access will write directly to the disk device cache but not
		Direct memory access will write directly to the disk device cache but not
		necessarily to the disk device storage medium. To ensure that your data is
		@@ -106,7 +121,24 @@ flushed from the disk device cache to the disk device storage medium, you should
		open flags. These are the equivalent of calling `fs.fdatasync()` or `fs.fsync()`
		respectively after every write. Please note that some file systems have had bugs
		in the past where calling `fs.fdatasync()` or `fs.fsync()` on a regular file
		would only force a flush if the page cache was dirty, so that bypassing the page
		cache using `O_DIRECT` meant the disk device cache was never actually flushed.
		respectively after every write, but with less system call overhead, and with the
		advantage that they encourage the disk device to do real work during the
		`fs.write()` call, which can be useful when overlapping compute-intensive work
		with IO.

		Some systems implement the `O_DSYNC` and `O_SYNC` open flags by setting the
		Force Unit Access (FUA) flag, which works for SCSI
		[but](https://perspectives.mvdirona.com/2008/04/disks-lies-and-damn-disks/)
		[not](https://blogs.msdn.microsoft.com/oldnewthing/20170510-00/?p=95505) for
		EIDE and SATA drivers.

		Conversely, some systems have had bugs where calling `fs.fdatasync()`
		or `fs.fsync()` on a regular file would force a flush only if the page cache was
		dirty, so that bypassing the page cache using `O_DIRECT` meant the disk device
		cache was never flushed.

		This means that the `O_DSYNC` and `O_SYNC` open flags are not sufficient on
		their own, but should be combined with `fs.fdatasync()` or `fs.fsync()` for
		durability or write barriers. This does not mean that these open flags are not
		useful. As we have already seen, they can reduce the latency of the eventual
		fsync call, eliminating latency spikes.

		O_DSYNC (FreeBSD, Linux, macOS, Windows)
		@@ -135,7 +167,7 @@

		## Block Device Size and Sector Sizes
		## Block Device Size and Sector Size

		Please note that `fs.fstat()` will not work at all on Windows for a block
		device, and will not report the correct size for a block device on other
		platforms. You should use `getBlockDevice()` instead:
		Node's `fs.fstat()` will not work at all for a block device on Windows, and will
		not report the correct size for a block device on other platforms. You should
		use `getBlockDevice()` instead:

		@@ -146,5 +178,5 @@ getBlockDevice(fd, callback) (FreeBSD, Linux, macOS, Windows)

		* `logicalSectorSize` - The size of a logical sector in bytes. Many drives
		advertise a backwards compatible logical sector size of 512 bytes when in fact
		their physical sector size is 4096 bytes.
		* `logicalSectorSize` - The size of a logical sector in bytes. Some drives will
		advertise a backwards compatible logical sector size of 512 bytes while their
		physical sector size is in fact 4096 bytes.

		@@ -156,8 +188,8 @@ * `physicalSectorSize` - The size of a physical sector in bytes. You should use

		* `size` - The total size of the block device in bytes.

		* `serialNumber` - The serial number reported by the device. (FreeBSD, Linux)

		* `size` - The total size of the block device in bytes.
		## Block Device Path and Permissions

		## Opening a Block Bevice

		You will need sudo or administrator privileges to open a block device. You can
		@@ -262,7 +294,10 @@ use `fs.open(path, flags)` to open a block device, where the path you provide

		The performance of various block sizes and open flags can vary across operating
		systems and between hard drives and solid state drives. Use the included
		benchmark script to benchmark various block sizes and open flags on the local
		file system (by default) or on a specific block device or regular file:
		The write performance of various block sizes and open flags can vary across
		operating systems and between hard drives and solid state drives. Use the
		included write benchmark to benchmark various block sizes and open flags on the
		local file system (by default) or on a specific block device or regular file:

		**WARNING: The write benchmark will erase the contents of the specified block
		device or regular file if any.**

		```
		@@ -269,0 +304,0 @@ [sudo] node benchmark.js [device\|file]

binding.cc

binding.gyp

Sorry, the diff of this file is not supported yet

Improved metrics

Worsened metrics

Dependency changes