Skip to content

Properly Closing Files in Python, Even with Exceptions

Updated: at 02:12 AM

In Python, it is important to properly close open files after accessing them to free up system resources and prevent unwanted behavior such as file corruption or data loss. However, gracefully closing files can be tricky, especially when exceptions are raised while the file is open. This article provides a comprehensive guide on techniques and best practices for ensuring files are closed correctly in Python, even when exceptions occur.

Table of Contents

Open Table of Contents

Introduction

When a file is opened in Python, it consumes system resources to keep the file available for reading and writing. If a file is opened but not explicitly closed after use, these system resources remain allocated even though the file is no longer needed. Over time, failing to close files can degrade performance, slow down your application, or even crash your program.

Additionally, changes made to an open file may not be flushed from internal buffers or fully written to disk until the file is closed. If the program crashes or loses power before closing files, data could be lost or corrupted.

To avoid these issues, it is important to properly close files after accessing them. However, gracefully handling exceptions while ensuring files get closed can be challenging. This guide covers several methods and coding patterns in Python to guarantee files are cleanly closed no matter what exceptions occur.

Using the try/finally Block

The best way to ensure files are closed properly is to use the try/finally block:

file = open("data.txt")

try:
  # read or write to the file
finally:
  file.close()

The code inside try will execute first and let you access the file. The finally block will then run no matter what, even if exceptions occur, closing the file for you.

Here is a more complete example:

file = open("data.txt")

try:
  data = file.read()
  # process data
except Exception as e:
  print("Error:", e)
finally:
  print("Closing file")
  file.close()

If an exception occurs while reading or processing the file, it will get caught, allowing the close operation in finally to run afterwards.

The finally block will execute after try, even if it raises an exception itself. This guarantees the file is closed regardless of any problems that occur.

The with Statement (Context Manager)

Python’s with statement provides another clean way to ensure files are closed properly. This takes advantage of Python’s context manager protocol.

The basic syntax is:

with open("file.txt") as file:
  # read or write to file

This works because the open() function returns a file object that acts as a context manager. The file will automatically be closed when the with block finishes execution, even if exceptions were raised inside the block.

For example:

with open("data.txt") as file:
  data = file.read()

  # process data
  raise Exception("Error!")

print("File has been closed")

Here the exception raised does not prevent the file from being closed when exiting the with statement.

The with block is simpler and cleaner than using try/finally directly, so it should be preferred in most cases.

Defining a Custom Context Manager

For more control, you can define your own context manager class by implementing the context manager methods __enter__ and __exit__.

Here is an example custom context manager to handle opening and closing files:

class FileManager():
  def __init__(self, filename, mode):
    self.filename = filename
    self.mode = mode

  def __enter__(self):
    print("Opening file")
    self.file = open(self.filename, self.mode)
    return self.file

  def __exit__(self, exc_type, exc_value, exc_traceback):
    print("Closing file")
    self.file.close()

# Usage:

with FileManager("data.txt", "r") as f:
  data = f.read()

This allows you to reuse the same logic for opening and closing files across multiple parts of your code.

Handling Exceptions While Closing

In rare cases, an exception may actually occur when attempting to close the file in the finally block or __exit__() method.

For example, if the file was already closed or manually deleted elsewhere.

You should handle exceptions properly when closing files as well:

try:
  f = open("data.txt")

  # read file
finally:
  try:
    f.close()
  except Exception as e:
    print("Warning: could not close file")

Or in a custom context manager:

class FileManager():

  #...

  def __exit__(self, exc_type, exc_value, exc_traceback):
    try:
      self.file.close()
    except Exception as e:
      print("Warning: could not close file")

This ensures the close operation does not crash your program even if exceptions occur.

Alternative Patterns

Some other patterns for closing files include:

The atexit Module

You can register a close file callback using atexit to run when the program terminates:

import atexit

f = open("log.txt", "w")

atexit.register(f.close) # will close on exit

Drawbacks are that it won’t run on exceptions, only normal program exit.

Class Deprecation Context

For classes, you can close files in __del__(), which runs when the object is garbage collected:

class Logger():
  def __init__(self, filename):
    self.file = open(filename, "w")

  def log(self, message):
    self.file.write(message)

  def __del__(self):
    self.file.close()
    print("Closed file")

Drawbacks are that object cleanup is not guaranteed immediately after use.

The weakref Module

You can use weakref.finalize to register a callback when an object is about to be destroyed:

import weakref

f = open("data.txt")

def close_file(file):
  print("Closing file")
  file.close()

weakref.finalize(f, close_file, f)

This helps avoid keeping objects alive just to call close. But finally is usually still preferable.

Best Practices

To properly close files and manage resources, keep these best practices in mind:

Properly closing files avoids issues like resource leaks and improves the stability and robustness of Python programs. Following these patterns helps ensure your application gracefully cleans up resources.

Real-World Examples

Here are some real-world examples of properly closing files in Python applications:

Reading a Configuration File

To read a configuration file, use a context manager to ensure it gets closed:

import json

with open("config.json") as f:
  config = json.load(f)

print(config["host"])
# Config now loaded, file closed automatically

This avoids keeping the config file open longer than necessary.

Writing Logs

For logging, open the file once then close it when done:

import logging

log = logging.getLogger("app")
handler = logging.FileHandler("debug.log")
log.addHandler(handler)

log.debug("App started")

# ...

handler.close() # Close file handler

This closes the log file after logging is complete.

Copying Files

Use try/finally when copying files to ensure both get closed:

try:
  with open("file1.txt") as f1:
    with open("file2.txt", "w") as f2:
      f2.write(f1.read())
finally:
  f1.close()
  f2.close()

This ensures proper closure even if an error occurs during the copy.

Reading Data Streams

For data streams like stdin/stdout, close files after processing:

import sys

try:
  data = sys.stdin.read()
  sys.stdout.write(f"Read {len(data)} bytes")
finally:
  sys.stdin.close()
  sys.stdout.close()

This closes the streams before the program exits.

Conclusion

Failing to properly close files can lead to resource leaks, data corruption, and instability. Python provides several effective techniques to guarantee files are cleanly closed no matter what exceptions occur in your code:

Following best practices like these ensures your Python programs gracefully clean up resources, increasing robustness and improving overall quality. Properly handling file closure is an important aspect of production-ready Python code.