A Practical Guide to Reading and Writing Text Files in Python

Text files are one of the most common file formats used for storing and processing data in Python. Whether you need to read data from a configuration file, parse a CSV file, or write logs to a text file, knowing how to handle text files is an essential skill for any Python developer.

In this comprehensive guide, we will explore practical techniques for reading from and writing to text files in Python, with real-world examples and code snippets you can apply right away.

Table of Contents

Overview of Text Files in Python

A text file is a file that stores information as plaintext characters. Text files contain only textual data - no formatting or binary data. In Python, we can work with text files using the built-in open() function combined with the file object attributes and methods.

Here are some key things to know about text files in Python:

• Text files are simple, portable, and widely supported across platforms and programs. CSV, JSON, XML, and source code files are common text file types.

• Python provides easy ways to open, read, write, and close access to text files. We don’t need to import any external libraries.

• File access modes control how we open a text file. The common modes are 'r' for read-only, 'w' for write-only, and 'a' for append.

• Handle text files carefully to avoid data loss or corruption. Always close files when done and use exception handling.

• Text data may require additional processing when reading and writing, such as decoding, encoding, splitting, or joining lines.

Now let’s look at practical examples of reading and writing text files in Python.

Reading Text Files

To read data from a text file in Python, we use the open() built-in function together with a file access mode 'r' to open the file for reading only. This will return a file object that contains the file’s data.

Here is the basic syntax:

file = open('filename.txt', 'r')

This opens filename.txt and assigns the file object to file. We can then call various methods on the file object to read the file contents.

Reading the Entire File Contents

To read all data from the file into a string variable, we use the read() method:

text = file.read()

This reads the entire contents of file into the string text.

For example:

file = open('read_file.txt', 'r')
text = file.read()
print(text)

file.close()

This will print the full contents of read_file.txt.

Reading Line By Line

For iterating over each line in a file, we can use the readlines() method. This returns a list where each element is a line from the file:

lines = file.readlines()

for line in lines:
  print(line)

This reads each line of text into the list lines, then prints each line.

Alternatively, we can use a loop on the file object directly:

for line in file:
  print(line)

This loops through the file line by line without loading all data into memory.

Reading a Fixed Number of Characters

To read only a set number of characters from the file, we can pass an integer argument to read(num_chars):

text = file.read(100)

This reads the first 100 characters into the variable text.

Reading From a Specific Position

We can also read from a specific position in the file by using seek() to set the file cursor position before reading:

file.seek(50)
text = file.read(100)

This seeks to character 50 first, then reads 100 characters from that position.

Checking For End-of-File

The file object has an eof attribute that is True when the end of the file has been reached. We can check this within loops to stop at end-of-file:

for line in file:
  print(line)
  if file.eof:
    break

This loops line-by-line until end-of-file is reached.

By leveraging these file methods, we can efficiently read and process text file data in the desired way.

Writing to Text Files

To write data to a new text file, we open the file with mode 'w' for write-only access. This will create the file if it does not already exist:

file = open('write_file.txt','w')

For an existing file, opening with mode 'w' will overwrite the existing contents.

We can then use the write() method to write data to the file:

file.write('This is some text')

This writes the string 'This is some text' to the file.

The write() method will not add any newline characters automatically - we need to explicitly add newlines:

file.write('First line\nSecond line\n')

Writes two lines of text with a newline between them.

Some key points for writing text files in Python:

• Use 'w' mode to create new files or overwrite existing files

• Remember to close files with file.close() when writing is complete

• Use write() to write strings

• Add explicit newline \n characters as needed

• Exception handling is important to catch I/O errors

Writing a List to a File

To write each element of a list to new lines in a file, we can loop through the list:

lines = ['First line', 'Second line']

for line in lines:
  file.write(line + '\n')

This loops through and writes each list item to a new file line.

Batch Writing Strings

For better performance with large data, we can write a list of strings all at once using writelines():

lines = ['First line\n', 'Second line\n']

file.writelines(lines)

This writes all strings in the list to the file in batch rather than one at a time.

Appending to a File

To add data to an existing text file without overwriting it, open the file with mode 'a' for append:

file = open('existing_file.txt', 'a')
file.write('This is new text')

This will append 'This is new text' to the end of existing_file.txt.

Handling File Errors

When working with text files in Python, errors can occur that we need to handle properly:

• IOError - Base error for I/O failures. Raised when the file can’t be opened.
• OSError - Error for operating system-related failures.
• EOFError - Raised when readline() is called after reaching end-of-file.

We should wrap our file operations in try/except blocks to catch and handle these errors:

try:
  file = open('file.txt', 'r')
  file.read()
except IOError:
  print('Could not read file')
except:
  print('Unknown error occurred')

This catches IOError and other exceptions, printing custom error messages.

We can access the exception instance for additional context in our exception handling:

try:
  file = open('missing.txt', 'r')
except IOError as e:
  print('Could not open file:', e)

This prints custom IOError details.

Proper error handling ensures our programs are robust and prevent crashes when working with text files.

Practical Examples of Reading and Writing Text Files

Now let’s look at some real-world examples of processing text files in Python.

1. Reading Configuration Files

Application configuration files store program settings and options in a text file. For example:

config.txt:

debug=True
log_level=WARNING
output_dir=/user/data

We can write Python code to open this configuration file, parse the key-value pairs, and assign to program variables:

config = {}

with open('config.txt') as file:
  for line in file:
    key, value = line.strip().split('=')
    config[key] = value

debug = config['debug']
log_level = config['log_level']
output_dir = config['output_dir']

This parses each line into a dictionary we can use to set configurable options.

2. Processing CSV Files

CSV files store tabular data in plaintext comma-separated rows and columns. Python has a csv module to help parse CSV files:

import csv

with open('data.csv') as file:
  reader = csv.reader(file)
  for row in reader:
    print(row)

This prints each row as a list of column values.

We can process CSV data into dictionaries as well:

keys = ['id', 'name', 'count']
dict_reader = csv.DictReader(file, keys)

for line in dict_reader:
  print(line['id'], line['name'])

This parses rows into dicts using keys as headers.

3. Writing Logs to Text Files

Applications often write log messages to a text file to record events or debug info:

import datetime

log_file = open('log.txt', 'a')

def write_log(message):
  now = datetime.datetime.now()
  log_file.write(now.strftime('%Y-%m-%d %H:%M') + ',' + message + '\n')

write_log('Starting program')

The write_log() function prepends a timestamp and appends each message to the log file.

4. Reading and Writing JSON Data

JSON is a common plaintext data format in Python. We can use the json module:

import json

with open('data.json', 'r') as file:
  json_data = json.load(file)

print(json_data['name'])

new_data = {'time': '12:30', 'status': 'success'}

with open('new_data.json', 'w') as file:
  json.dump(new_data, file)

This reads JSON into a Python dict, writes a dict to JSON file.

As you can see, reading and writing text files like CSV, configuration, logs, and JSON is a common task in Python.

Conclusion

In this guide, we covered several practical techniques for reading and writing text files in Python, including:

• Opening files in various read, write, and append access modes
• Reading entire file contents, iterating lines, reading chunks, and seeking positions
• Writing strings, lines, and batches using write() and writelines()
• Parsing and processing text file formats like CSV and JSON
• Handling file I/O errors with try/except blocks
• Applying these concepts to real-world use cases

Python’s built-in text file handling provides a straightforward way to access data from plain text files on disk. With the file methods and additional string processing, we can easily read, parse, and write text file content exactly how we need for a given application.

To learn more about working with files in Python, refer to the official Python Documentation on Text File I/O and the Open() Function.