Linux System Programming - Buffered I/O

Introduction
• Input/output (I/O) buffering is a mechanism that
improves the throughput of input and output
operations
• Throughput is the amount of work completed in a
unit of time
• It is implemented directly in hardware and the
corresponding drivers (hence the block
devices found in Unix-like systems), and is also
universal among programming language standard
libraries.

Introduction
• I/O operations often have high latencies; the time
between the initiation of an I/O process and its
completion may be millions of processor clock cycles.
• Most of this latency is due to the hardware itself; for
example, information cannot be read from or written
to a hard disk until the spinning of the disk brings the
target sectors directly under the read/write head.
• This is reduced by having one or more input and
output buffers associated with each device.
• Latency=delay.

Introduction
• The goal of the buffering provided by the standard I/O
library is to use the minimum number of read and write
calls
• A buffer is a memory area that stores data being
transferred between two devices or between a device and
an application
• Buffering is done for three reasons.
– First is to cope with a speed mismatch between producer and
consumer of a data stream.
– The second use of buffering is to provide adaptation for data
that have different data-transfer sizes.
– Third use of buffering is to support copy semantics for the
application I/O

Introduction
• copy semantics
– Data is first copied from user application memory
into kernel memory
– Data from kernel memory is then written to
device
– Prevents application from changing contents of a
buffer before it is done being written

User-Buffered I/O
• User buffered I/O, shortened
to buffering or buffered I/O, refers to the
technique of temporarily storing the results of an
I/O operation in user-space before transmitting it
to the kernel (in the case of writes) or before
providing it to your process (in the case of reads).
• By so buffering the data, you can minimize the
number of system calls and can block-align I/O
operations, which may improve the performance
of your application.

User-Buffered I/O
• Block Size:
– In practice, blocks are usually 512, 1,024, 2,048, 4,096, or
8,192 bytes in size.
– a large performance gain is realized simply by performing
operations with respect to block size.
– This is because the kernel and hardware speak in terms of
blocks
– Thus, using the block size or a value that fits neatly inside
of a block guarantees block-aligned I/O requests and
prevents extraneous(external) work inside the kernel
– The end result is fewer system calls for larger amounts of
data, all aligned on block boundaries

Standard I/O
• The standard C library provides the standard I/O library
(often simply called stdio), which in turn provides a
platform-independent, user-buffering solution
• The three input/output (I/O) connections are
called standard input (stdin), standard output (stdout)
and standard error (stderr).
• Standard input is a stream from which a program reads
its input data.
• Standard output is a stream to which a program writes
its output data
• Standard error is another output stream typically used
by programs to output error messages or diagnostics.

Linux System Programming - Buffered I/O

Standard I/O
File Pointer
• Standard I/O routines do not operate directly on
file descriptors
• Instead, they use their own unique identifier,
known as the file pointer.
• File pointer is a pointer which is used to handle
and keep track on the files being accessed.
• A new data type called “FILE” is used to declare
file pointer.
• This data type is defined in stdio.h file. File
pointer is declared as FILE *fp. Where, ‘fp’ is a file
pointer.

Opening Files
• Files are opened for reading or writing via
fopen():
#include <stdio.h>
FILE * fopen (const char *path, const char
*mode);
• This function opens the file path with the
behavior given by mode and associates a new
stream with it.
• A stream is a sequence of data elements made
available over time

• Modes
• The mode argument describes how to open the given file. It
is one of the following strings:
• r:Open the file for reading. The stream is positioned at the
start of the file.
• r+:Open the file for both reading and writing. The stream is
positioned at the start of the file.
• w:Open the file for writing. If the file exists, it is truncated
to zero length. If the file does not exist, it is created. The
stream is positioned at the start of the file.
• w+:Open the file for both reading and writing. If the file
exists, it is truncated to zero length. If the file does not
exist, it is created. The stream is positioned at the start of
the file.
• a:Open the file for writing in append mode. The file is
created if it does not exist. The stream is positioned at the
end of the file. All writes will append to the file.
• a+:Open the file for both reading and writing in append
mode. The file is created if it does not exist. The stream is
positioned at the end of the file. All writes will append to

Opening Files
• Upon success, fopen() returns a valid FILE
pointer. On failure, it returns NULL and sets
errno appropriately.
Fopen() Open()
fopen series are standard C
library functions
the Open series are defined by
POSIX and are system calls in
UNIX systems.
when using fopen functions,
you must define an object that
refers to a file. It is called "file
handler and is a struct
the Open series uses an int
integer called "file descriptor.

Opening a Stream via File Descriptor
• The function fdopen() converts an already
open file descriptor (fd) to a stream:
#include <stdio.h>
FILE * fdopen (int fd, const char *mode);
• The possible modes are the same as for
fopen() and must be compatible with the
modes originally used to open the file
descriptor

Opening a Stream via File Descriptor
• On success, fdopen() returns a valid file pointer; on failure,
it returns NULL and sets errno appropriately.
• Ex:
FILE *stream;
int fd;
fd = open ("/home/kidd/map.txt", O_RDONLY);
if (fd == -1)
• /* error */
stream = fdopen (fd, "r");
if (!stream)
• /* error */ otherwise appropriate operation will be
executed.

Closing Streams
• The fclose() function closes a given stream:
#include <stdio.h>
int fclose (FILE *stream);
• Any buffered and not-yet-written data is first
flushed. On success, fclose() returns 0.
• On failure, it returns EOF and sets errno
appropriately.

Closing All Streams
• The fcloseall() function closes all streams
associated with the current process, including
standard in, standard out, and standard error:
#define _GNU_SOURCE
#include <stdio.h>
int fcloseall (void);
• Before closing, all streams are flushed. The
function always returns 0; it is Linux-specific.

Reading from a Stream
• The standard C library implements multiple
functions for reading from an open stream,
ranging from the simple to the complex.
• The three of the most popular approaches to
reading:
– reading one character at a time,
– reading an entire line at a time,
– reading binary data

• Reading a Character at a Time
– the ideal I/O pattern is simply reading one character
at a time. The function fgetc() is used to read a single
character from a stream:
#include <stdio.h>
int fgetc (FILE *stream);
• This function reads the next character (an
unsigned char) from the specified stream and
advances the position indicator for the stream.

#include <stdio.h>
int main ()
{
FILE *fp;
int c;
int n = 0;
fp = fopen("file.txt","r");
if(fp == NULL)
{
perror("Error in opening file");
return(-1);
}
do
{
c = fgetc(fp);
if( feof(fp) ) {
break ;
}
printf("%c", c);
}
while(1);
fclose(fp);
return(0);
}

• Putting the character back:
• Standard I/O provides a function for pushing a
character back onto a stream, allowing you to
“peek” at the stream and return the character
if it turns out that you don’t want it:
#include <stdio.h>
int ungetc (int c, FILE *stream);

• Reading an Entire Line:
– The function fgets() reads a string from a given
stream:
#include <stdio.h>
char * fgets (char *str, int size, FILE *stream);
– This function reads up to one less than size bytes
from stream and stores the results in str.
– A null character (0) is stored in the buffer after
the last byte read in.

#include <stdio.h>
int main ()
{
FILE *fp;
char str[60];
/* opening file for reading */ fp =
fopen("file.txt" , "r");
if(fp == NULL)
{
perror("Error opening file");
return(-1);
}
if( fgets (str, 60, fp)!=NULL )
{ /* writing content to stdout */ puts(str);
}
fclose(fp);
return(0);
}

• Reading Binary Data:
– Sometimes, developers want to read and write
complex binary data
– For this, the standard I/O library provides fread():
#include <stdio.h>
size_t fread(void * buffer, size_t size, size_t count, FILE
* stream)
• fread() function is used to read data from a file
and store it in a buffer.

• The fread() takes in a number of parameters.
Let’s look at each of them in detail:
– Buffer: Pointer to the buffer where data will be
stored. A buffer is a region of memory used to
temporarily store data
– Size: The size of each element to be read in bytes
– Count: Number of elements to be read
– Stream: Pointer to the FILE object from where
data is to be read

• Upon successful execution, the function
returns an integer value equivalent to count.
In case of an error or EOF, a value less
than count is returned.

Writing to a Stream
• The standard C library defines several
functions for writing to an open stream
• three of the most popular approaches to
writing:
– writing a single character,
– writing a string of characters, and
– writing binary data.

Writing to a Stream
• Writing a Single Character:
– The counterpart of fgetc() is fputc():
#include <stdio.h>
int fputc (int c, FILE *stream);
– Parameters
• c − This is the character to be written. This is passed as its int
promotion.
• stream − This is the pointer to a FILE object that identifies the
stream where the character is to be written.
– Return Value
• If there are no errors, the same character that has been written is
returned. If an error occurs, EOF is returned and the error
indicator is set.

Writing to a Stream
#include <stdio.h>
int main ()
{
FILE *fp;
int ch;
fp = fopen("file.txt", "w+");
for( ch = 33 ; ch <= 100; ch++ )
{
fputc(ch, fp);
}
fclose(fp);
return(0);
}
Example for writing a string of characters

Writing to a Stream
• Writing a String of Characters:
– The function fputs() is used to write an entire string to a
given stream:
#include <stdio.h>
int fputs (const char *str, FILE *stream);
– Parameters
• str − This is an array containing the null-terminated sequence of
characters to be written.
• stream − This is the pointer to a FILE object that identifies the
stream where the string is to be written.
– Return Value
• This function returns a non-negative value, or else on error it
returns EOF.

Writing to a Stream
#include <stdio.h>
int main() {
const char *buffer = "Hello world!";
fputs (buffer, stdout);
return 0;
}
Example for writing a string of characters

Writing to a Stream
• Writing Binary Data
– To directly store binary data such as C variables, standard I/O
provides fwrite():
#include <stdio.h>
size_t fwrite (void *buf,size_t size,size_t nr,FILE *stream);
– Parameters
• buf− This is the pointer to the array of elements to be written.
• size − This is the size in bytes of each element to be written.
• nr − This is the number of elements, each one with a size of size bytes.
• stream − This is the pointer to a FILE object that specifies an output
stream.
– Return Value
• This function returns the total number of elements successfully
returned as a size_t object, which is an integral data type.

Writing to a Stream
• Example program for writing binary data to a file
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
FILE *file_ptr;
int iCount;
char arr[6] = "hello";
file_ptr = fopen("sample.txt", "wb");
iCount = fwrite(arr, 1, 5, file_ptr);
fclose(file_ptr);
return 0;
}

Seeking a Stream
• The fseek() function, the most common of the standard
I/O seeking interfaces, manipulates the file position of
stream in accordance with offset and whence:
#include <stdio.h>
int fseek (FILE *stream, long offset, int whence);
• Parameters
– stream − This is the pointer to a FILE object that identifies
the stream.
– offset − This is the number of bytes to offset from whence.
– whence − This is the position from where offset is added.
It is specified by one of the following constants −

Seeking a Stream
Sr.No Constant & Description
1 SEEK_SET
Beginning of file
2 SEEK_CUR
Current position of the file pointer
3 SEEK_END
End of file
Whence parameter possible values:
Return Value:
This function returns zero if successful, or else it returns a
non-zero value.

Seeking a Stream
• Other alternatives to fseek() are:
#include <stdio.h>
int fsetpos (FILE *stream, fpos_t *pos);
And
#include <stdio.h>
void rewind (FILE *stream);

Seeking a Stream
#include <stdio.h>
int main ()
{
FILE *fp;
fp = fopen("file.txt","w+");
fputs("This is tutorialspoint.com", fp);
fseek( fp, 7, SEEK_SET );
fputs(" C Programming Language", fp);
fclose(fp);
return(0); }
Example for fseek function

Seeking a Stream
• Obtaining the Current Stream Position:
– Unlike lseek(), fseek() does not return the updated
position. A separate interface is provided for this
purpose. The ftell() function returns the current
stream position in the stream:
#include <stdio.h>
long ftell (FILE *stream);
– On error, it returns −1 and errno is set
appropriately.

Seeking a Stream
• Obtaining the Current Stream Position ex:
#include <stdio.h>
#include<conio.h>
void main ()
{
FILE *f;
int len;
f = fopen("one.txt", "r");
if(f == NULL)
{
perror(“Error opening file”);
return(-1);
}
fseek(f, 0, SEEK_END);
len = ftell(f);
fclose(f);
printf("Size of file: %d bytes", len);
getch(); }

Seeking a Stream
• Alternatively, standard I/O provides fgetpos():
#include <stdioh.h>
int fgetpos (FILE *stream, fpos_t *pos);
Upon success, fgetpos() returns 0, and places
the current stream position of stream in pos

Flushing a Stream
• The standard I/O library provides an interface
for writing out the user buffer to the kernel,
ensuring that all data written to a stream is
flushed via write(). The fflush() function
provides this functionality:
#include <stdio.h>
int fflush (FILE *stream);

Flushing a Stream
• On invocation, any unwritten data in the
stream pointed to by stream is flushed to the
kernel.
• If stream is NULL, all open input streams in the
process are flushed.
• On success,fflush() returns 0. On failure, it
returns EOF, and errno is set appropriately

Flushing a Stream
#include <stdio.h>
int main()
{
char buf[50];
FILE *fp;
fp = fopen("test.txt", "r+");
if (fp)
{
fputs("Test data by Fresh2refres", fp);
fflush(buf); // flushes the buffer to load the data from file
fgets(buf, 20, fp); // It loads 1st 20 characters from file to buffer
puts(buf); // It displays buffer data in output screen
fclose(fp);
return 0;
}
return 1;
}

Errors and End-of-File
• Some of the standard I/O interfaces, such as
fread(), communicate failures back to the caller
poorly, as they provide no mechanism for
differentiating between error and end-of-file.
• With these calls, and on other occasions, it can be
useful to check the status of a given stream to
determine whether it has encountered an error
or reached the end of a file.
• Standard I/O provides two interfaces to this end.

• The function ferror() tests whether the error
indicator is set on stream:
#include <stdio.h>
int ferror (FILE *stream);
• The error indicator is set by standard I/O
interfaces in response to an error condition.
• The function returns a nonzero value if the
indicator is set, and 0 otherwise.

• The function feof() tests whether the EOF
indicator is set on stream:
#include <stdio.h>
int feof (FILE *stream);
• The EOF indicator is set by standard I/O
interfaces when the end of a file is reached.
• This function returns a nonzero value if the
indicator is set, and 0 otherwise.

• The clearerr() function clears the error and the
EOF indicators for stream:
#include <stdio.h>
void clearerr (FILE *stream);
• It has no return value, and cannot fail
• You should make a call to clearerr() only after
checking the error and EOF indicators

Obtaining the Associated File
Descriptor
• Sometimes it is advantageous to obtain the
file descriptor backing a given stream.
• To obtain the file descriptor backing a stream,
use fileno():
#include <stdio.h>
int fileno (FILE *stream);
• Upon success, fileno() returns the file
descriptor associated with stream. On failure,
it returns −1.

Obtaining the Associated File
Descriptor
#include <stdio.h>
void main()
{
FILE *stream; stream = fopen( "file", "r" );
printf( "File number is %dn", fileno( stream ) );
fclose( stream );
}
It produces output similar to the following:
File number is 7

Controlling the Buffering
• Standard I/O implements three types of user
buffering and provides developers with an
interface for controlling the type and size of the
buffer.
• The different types of user buffering serve
different purposes:
– Unbuffered:No buffering - characters are transmitted
to the system as they are written
– Line-buffered:characters are transmitted to the system
as a block when a new-line character is encountered
– Block-buffered:characters are transmitted to the
system as a block when a buffer is filled.

Thread Safety
• In computer programming, thread-safe describes
a program portion or routine that can be called
from multiple programming threads without
unwanted interaction between the threads.
• By using thread-safe routines, the risk that one
thread will interfere and modify data elements of
another thread is eliminated by circumventing
potential data race situations with coordinated
access to shared data.

Thread Safety
• The standard I/O functions are inherently thread-
safe
• Any given thread must acquire the lock and
become the owning thread before issuing any I/O
requests
• Two or more threads operating on the same
stream cannot interleave standard I/O
operations, and thus, within the context of single
function calls, standard I/O operations are
atomic.

Thread Safety
• Standard I/O provides a family of functions for
individually manipulating the lock associated with
a stream.
• Manual File Locking:
– The function flockfile() waits until stream is no longer
locked, bumps the lock count, and then acquires the
lock, becoming the owning thread of the stream, and
returns
#include <stdio.h>
void flockfile (FILE *stream);

Thread Safety
• The function funlockfile() decrements the lock
count associated with stream:
#include <stdio.h>
void funlockfile (FILE *stream);
• If the lock count reaches zero, the current
thread relinquishes ownership of the stream.
• Another thread is now able to acquire the
lock.

Thread Safety
• The ftrylockfile() function is a nonblocking version
of flockfile():
#include <stdio.h>
int ftrylockfile (FILE *stream);
• If stream is currently locked, ftrylockfile() does
nothing and immediately returns a nonzero
value.
• If stream is not currently locked, it acquires the
lock, bumps the lock count, becomes the owning
thread of stream, and returns 0.

Thread Safety
• Unlocked Stream Operations:
– The use of the _unlocked variants of the stream
operations will unlock the streams.
– Fgets_unlocked(buf,20,-----)

Critiques of Standard I/O
• The biggest complaint with standard I/O is the performance
impact from the double copy.
• When reading data, standard I/O issues a read() system call
to the kernel, copying the data from the kernel to the
standard I/O buffer.
• When an application then issues a read request via
standard I/O using, say, fgetc(), the data is copied again, this
time from the standard I/O buffer to the supplied buffer.
• Write requests work in the opposite fashion: the data is
copied once from the supplied buffer to the standard I/O
buffer and then later from the standard I/O buffer to the
kernel via write().

Linux System Programming - Buffered I/O

Recommended

More Related Content

What's hot (20)

Similar to Linux System Programming - Buffered I/O (20)

Recently uploaded (20)

Linux System Programming - Buffered I/O