Basic Python File Operations Tutorial

What are file operations

So far we’ve stored data only in memory (RAM), but memory is volatile. The data is lost when we close the program, or turn off the device.

File operations allow us to work with data that is permanently stored in non-volatile memory, such as a file on a hard disk. We can store data permanently in a file like a spreadsheet and later read from the file, or write to it again.

When we want to perform operations on a file, we need to open it first. When we are done performing operations, we need to close the file so that the resources the file used are freed.

Because of this, any file operation takes place in the following order:

1. Open file
2. Perform operations (read or write)
3. Close file

In this lesson we will use a simple text file to demonstrate the concepts of creating, opening, reading and writing file operations.

How to open a file

Python has a built-in open() function to open files that we want to perform operations on. The function returns the file object, otherwise known as the file handle.

This function takes the file path (location, name & extension) as its first argument.

 handle = open("location/name.ext")

We can specify the absolute or relative path of a file.

# relative path (current directory)
f = open("test.txt")

# absolute path
f = open("C:/PythonProjects/Tutorials/FileOps/test.txt")

How to open a file with a mode

The open() function allows us to specify a mode when opening a file.

We can specify whether we want to read (‘r’), write (‘w’) or append (‘a’) to the file. We can also specify if we want to open the file in text mode (’t’) or binary mode (‘b’).

To specify a mode, we pass one of the mode letters as the function’s second argument.

 handle = open("location/name.ext", "mode_letter")

tip By default, the file is opened in text mode with read privileges.

 f = open("test.txt", "w")

List of file operation modes

The following table lists the file operation modes available to us in Python.

# open in text mode with reading
# privileges, same as 'r' or 'rt'
f = open("test.txt")

# open in text mode with writing
# privileges
f = open("test.txt", "w")

# open in binary mode with writing
# privileges
f = open("test.txt", "wb+")

# specify mode parameter and value
f = open("test.txt", mode="x")

How to specify a file encoding type

The default encoding of a file is platform dependent. In Linux, for example, a file is encoded as ‘utf-8’ but in windows the file is encoded as ‘ANSI’.

We shouldn’t rely on the default encoding because our code will behave differently on different platforms. So, when working with files in text mode, we recommend always specifying the encoding type.

# specify mode and encoding parameters and values
f = open("test.txt", mode="x", encoding="UTF-8")

How to create a file

We can see from the table of operation modes and privileges that we use the open() function with ‘x’, ‘w’ or ‘a’ if we want to create a file.

• x will only create the file.
• w will create the file and open it with writing privileges.
• a will create the file and open it to append at the end of the file.

Mode: x (create)

If we want to create a file for later use without immediately opening it too, we use x as the mode.

 f = open("test.txt", mode="x", encoding="UTF-8")

Mode: w (create and write)

If we want to create a file and open it directly for writing to it, we use w as the mode. If the file already exists it will only open the file with writing privileges.

 f = open("test-write.txt", mode="w", encoding="UTF-8")

Mode: a (create and append)

If we want to create a file and open it directly for appending to it, we use a as the mode. If the file already exists it will only open the file to append to it.

 f = open("test-append.txt", mode="a", encoding="UTF-8")

How to close a file

When we’re finished with operations in a file, we need to close the file to free up resources associated with it.

Python has a garbage collector. At some point in the future, the it will come in and clean up any unreferenced objects. However, it’s good practice to not rely on the garbage collector and explicitly close the file with the close() function.

The function is used on the file handle with dot notation.

# open or create a file
handle = open("location/name.ext")

# use handle to close file
handle.close()

f = open("test.txt", mode="w", encoding="UTF-8")

# perform operations

f.close()

The example above will work fine, but it’s not entirely safe. If an exception occurs while we’re performing operations on the file, the code may exit without reaching the close statement.

tip A safe way to guarantee the file closing is to use it in a try finally block .

try:
    # open file
    # perform operations
finally:
    # close file (guaranteed)

try:
    # open file
    f = open("test.txt", mode='w', encoding='UTF-8')
    # perform operations
finally:
    # close file (guaranteed)
    f.close()

How to write to a file

When writing to a file we need to be careful two of the modes when opening the file.

• Mode w will overwrite anything already in the file.
• Mode a will add to the end of the file.

To actually write to the file we use the write() function on the file handle with dot notation.

# open or create a file
handle = open("location/name.ext", "w")

# use handle to write to file
handle.write("text")

Let’s overwrite any previous content in our file by using w as the mode.

try:
    f = open("test.txt", mode='w', encoding='UTF-8')
    # write to the file, overwriting any previous content
    f.write("Hello World")
finally:
    f.close()

Now let’s append a string to our file by using a as the mode.

try:
    f = open("test.txt", mode='a', encoding='UTF-8')
    # write to the file, appending to previous content
    f.write("\nThis line is appended")
finally:
    f.close()

tip To get a newline, we have to specify it explicitly with \n .

From the examples, the file will now contain two lines.

Hello World
This line is appended

How to read from a file

In Python, there are several ways to read data from a file.

But before we can read the data in a file, we have to open it in one of the reading modes with the open() function.

# open or create a file
handle = open("location/name.ext", "r")

# use handle to close file
handle.write("text")

The simplest way to read a file is to read() it in its entirety.

try:
    f = open("test.txt", mode='r', encoding='UTF-8')
    # read the contents of the whole file
    output = f.read()
finally:
    f.close()

# print output
print(output)

How to read file with a loop

We can read() a file line-by-line using a for loop.

try:
    f = open("test.txt", mode='r', encoding='UTF-8')
    # read the contents of the whole file line-by-line
    for line in f:
        print(line)
finally:
    f.close()

How to read a line with readline()

We can read a file line-by-line with the readline() function.

try:
    f = open("test.txt", mode='r', encoding='UTF-8')
    # read the contents of the whole file
    print(f.readline())
    print(f.readline())
finally:
    f.close()

How to read a file with readlines()

We can read the whole file line-by-line with the readlines() function. This function returns a list collection of the lines.

try:
    f = open("test.txt", mode='r', encoding='UTF-8')
    # read the contents of the whole file
    output = f.readlines()
finally:
    f.close()

print(output)

The list that is returned from the readlines() function can be used as a normal list.

try:
    f = open("test.txt", mode='r', encoding='UTF-8')
    # read the contents of the whole file
    output = f.readlines()
finally:
    f.close()

for item in output:
    print(item)

How to read file at position

We can use read(size) to specify a size of characters we want to read out.

try:
    f = open("test.txt", mode='r', encoding='UTF-8')
    # read the contents of the first 11 characters
    output = f.read(11)
finally:
    f.close()

# print output
print(output)

In the example above we specify that we want to see the first 11 characters in the file.

How to show the current position in file

We can then use the tell() method to see our current position in the file.

try:
    f = open("test.txt", mode='r', encoding='UTF-8')
    # read the contents of the first 11 characters
    print(f.read(11))
    # tell our current position
    print(f.tell())
finally:
    f.close()

The example above shows our current position as 11. That’s where we stopped because we only accessed 11 characters in the file.

We can save this number if we need to for later usage when reopening the file.

How to set the current position in file

We can also adjust our position in the file with the seek() function.

try:
    f = open("test.txt", mode='r', encoding='UTF-8')
    # set our position to 11 characters into the file
    f.seek(11)
    # print some characters from that position
    print(f.read(22))
finally:
    f.close()

In the example above we set our position in the file to 11 characters, the end of the sentence “Hello World”.

When we print 22 characters now, it will start at 11 and print 22 characters from there.

List of file functions

The following table lists some of the file functions available to us in Python.

Summary: Points to remember

• File operations can be done on files stored on a hard disc, such as csv or txt files.
• It’s good practice to open a file with a specific operation mode and encoding type.
• A file’s encoding depends on your operating system. Linux and Windows files sometimes have a different encoding type.
• Set the internal file pointer to a specific position with the seek() function.