11.2.1 Using os and shutil for file management

The os and shutil modules in Python are commonly used for managing files and directories. They provide functionality for interacting with the operating system, allowing you to automate file operations such as creating, renaming, moving, copying, and deleting files and directories. Here’s an overview of how to use these modules for file management:

1. Using the os Module

The os module provides a portable way to use operating system-dependent functionality, such as file and directory manipulation.

Common Operations with os Module:

• os.mkdir(): Create a new directory.
• os.makedirs(): Create a directory and any intermediate directories that don’t exist.
• os.remove(): Delete a file.
• os.rmdir(): Remove a directory (must be empty).
• os.rename(): Rename a file or directory.
• os.listdir(): List the files and directories in a directory.
• os.path.join(): Join one or more path components to create a valid file path.
• os.path.exists(): Check if a path exists.

Example 1: Creating and Deleting Directories

import os

# Create a new directory
os.mkdir("new_directory")

# Create a directory and any intermediate directories
os.makedirs("parent_directory/child_directory")

# Check if the directory exists
if os.path.exists("new_directory"):
    print("Directory exists")

# Remove the directory (must be empty)
os.rmdir("new_directory")

Example 2: Listing Files in a Directory

import os

# List files and directories in the current directory
files = os.listdir(".")

print("Files in current directory:", files)

2. Using the shutil Module

The shutil module provides high-level file operations, such as copying, moving, and removing files and directories. It is especially useful for tasks like backing up data or managing large sets of files.

Common Operations with shutil Module:

• shutil.copy(): Copy a file to a new location.
• shutil.copy2(): Copy a file along with metadata (like timestamps).
• shutil.move(): Move a file or directory to a new location.
• shutil.rmtree(): Recursively delete a directory and its contents.
• shutil.make_archive(): Create a compressed archive of a directory.

Example 3: Copying and Moving Files

import shutil

# Copy a file to a new location
shutil.copy("source_file.txt", "destination_file.txt")

# Move a file to a new location
shutil.move("source_file.txt", "new_directory/destination_file.txt")

Example 4: Recursively Deleting a Directory

import shutil

# Recursively delete a directory and all its contents
shutil.rmtree("directory_to_delete")

Example 5: Creating a Compressed Archive

import shutil

# Create a zip archive of a directory
shutil.make_archive("backup", "zip", "directory_to_backup")

3. Combining os and shutil for File Management Automation

You can combine both modules to automate more complex file management tasks, such as organizing files by moving them into directories, renaming files, and creating backups.

Example 6: Organizing Files by Extension

import os
import shutil

# Define the source directory and target directories for each file type
source_directory = "/path/to/source"
image_directory = "/path/to/images"
text_directory = "/path/to/text_files"

# Ensure the target directories exist
os.makedirs(image_directory, exist_ok=True)
os.makedirs(text_directory, exist_ok=True)

# Iterate over files in the source directory
for filename in os.listdir(source_directory):
    file_path = os.path.join(source_directory, filename)
    
    if os.path.isfile(file_path):
        if filename.endswith((".jpg", ".png")):
            shutil.move(file_path, os.path.join(image_directory, filename))
        elif filename.endswith(".txt"):
            shutil.move(file_path, os.path.join(text_directory, filename))

print("Files organized successfully.")

Example 7: Backing Up Files

import os
import shutil
from datetime import datetime

# Define the source directory and backup directory
source_directory = "/path/to/source"
backup_directory = f"/path/to/backups/backup_{datetime.now().strftime('%Y%m%d_%H%M%S')}"

# Create the backup directory
os.makedirs(backup_directory)

# Copy all files from the source directory to the backup directory
for filename in os.listdir(source_directory):
    file_path = os.path.join(source_directory, filename)
    
    if os.path.isfile(file_path):
        shutil.copy(file_path, os.path.join(backup_directory, filename))

print("Backup completed successfully.")

Summary

The os and shutil modules in Python make it easy to manage files and directories efficiently:

• os module provides low-level functions for interacting with the operating system, such as file creation, deletion, and path operations.
• shutil module provides high-level functions like copying, moving, and deleting files and directories, as well as creating archives.

By using these modules, you can automate and streamline file management tasks, making your workflows more efficient and organized.