Working with the fsync commit rate

In this recipe, we will be discussing how to benchmark the fsync speed using open source tools.

Getting ready

Fsync is a system call that flushes the data from system buffers into physical files. In PostgreSQL, whenever a CHECKPOINT operation occurs, it internally initiates the fsync, to flush all the modified system buffers into the respective files. The fsync benchmarking defines the transfer ratio of data from memory to the disk.

How to do it...

To perform fsync benchmarking, we can use a dedicated benchmark test called fs-mark from Phoronix. This fs-mark test was built based on a filesystem benchmarking tool called fs_mark, or fio, which supports several fsync test cases. We can run this fs-mark test case using the following command:

$ phoronix-test-suite benchmark fs-mark FS-Mark 3.3:
 pts/fs-mark-1.0.1
Disk Test Configuration
1: 1000 Files, 1MB Size
 2: 1000 Files, 1MB Size, No Sync/FSync
 3: 5000 Files, 1MB Size, 4 Threads
 4: 4000 Files, 32 Sub Dirs, 1MB Size
 5: Test All Options
 Test:

Note

The preceding command failed to install while testing on the local machine. Once I installed glibc-static via yum install, then the test went smooth.

How it works...

Phoronix installs all the binaries on the local machine when we start benchmarking the corresponding test. In the preceding command, we are benchmarking the test fs-mark, where it installs the tool at ~/.phoronix-test-suite/installed-tests/pts/fs-mark-1.0.1/fs_mark-3.3. Let's go to the location, and let's see what fsync tests it supports:

./fs_mark -help
Usage: fs_mark
 -S Sync Method ( 0:No Sync, 
 1:fsyncBeforeClose, 
 2:sync/1_fsync, 
 3:PostReverseFsync, 
 4:syncPostReverseFsync, 
 5:PostFsync, 
 6:syncPostFsync)

I would encourage you to read the readme file, which exists in the same location, for detailed information about the sync methods. Let's run a simple fs_mark benchmarking by choosing one sync method as shown in the following here:

./fs_mark -w 8096 -S 1 -s 102400 -d /tmp/ -L 3 -n 500
# ./fs_mark -w 8096 -S 1 -s 102400 -d /tmp/ -L 3 -n 500
# Version 3.3, 1 thread(s) starting at Fri Dec 30 04:26:28 2016
# Sync method: INBAND FSYNC: fsync() per file in write loop.
# Directories: no subdirectories used
# File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
# Files info: size 102400 bytes, written with an IO size of 8096 bytes per write
# App overhead is time in microseconds spent in the test not doing file writing related system calls.
FSUse% Count Size Files/sec App Overhead
 39 500 102400 156.4 17903
39 1000 102400 78.9 22906
39 1500 102400 116.2 24269

We ran the preceding test with write files of size 102,400 and block size of 8,096. The number of files it needs to create is 500 and it needs to repeat the test three times by choosing sync method 1, which closes the file after writing the content to disk.