Skip to main content

Chapter 11 - Organizing Files (Python)

Here's a concise walkthrough of the main ideas in Chapter 11, each with a small example.

Organizing files (intro)​

Chapter goal: automate copying, renaming, moving, deleting, and compressing many files/folders instead of doing it manually in a file explorer.

Example scenario: copy only all PDFs in a tree:

from pathlib import Path
import shutil

root = Path.home() / 'project'
dest = Path.home() / 'pdf_backup'
dest.mkdir(exist_ok=True)

for path in root.rglob('*.pdf'):
    shutil.copy(path, dest)

The shutil module​

Provides high-level file operations: copy, move, and delete (tree).

Copying files and folders​

  • shutil.copy(src, dst) copies a single file.
  • shutil.copytree(src, dst) copies a whole directory tree.
import shutil
from pathlib import Path

h = Path.home()
shutil.copy(h / 'spam/file1.txt', h)                      # same name
shutil.copy(h / 'spam/file1.txt', h / 'spam/file2.txt')  # new name
shutil.copytree(h / 'spam', h / 'spam_backup')           # copy folder

Moving and renaming files and folders​

shutil.move(src, dst) moves and optionally renames a file/folder.

import shutil
from pathlib import Path

h = Path.home()
(h / 'spam2').mkdir(exist_ok=True)

shutil.move(h / 'spam/file1.txt', h / 'spam2')              # move
shutil.move(h / 'spam/file1.txt', h / 'spam2/new_name.txt') # move+rename

Permanently deleting files and folders​

  • os.unlink(path) delete file.
  • os.rmdir(path) delete empty folder.
  • shutil.rmtree(path) delete folder tree.

Dry-run pattern:

import os
from pathlib import Path

for filename in Path.home().glob('*.txt'):
    # os.unlink(filename)   # real delete
    print('Deleting', filename)

Deleting to the recycle bin​

Use send2trash.send2trash(path) to move to recycle bin instead of permanent delete.

import send2trash

send2trash.send2trash('file1.txt')

Walking a directory tree​

os.listdir and Path.iterdir​

  • os.listdir(path) returns a list of names.
  • Path(path).iterdir() returns an iterator of Path objects.
import os
from pathlib import Path

print(os.listdir(Path.home()))
print(list(Path.home().iterdir()))

os.walk: naming and yielded values​

os.walk(top) yields (folder_name, subfolders, filenames) for each directory in the tree.

import os
from pathlib import Path

h = Path.home()
for folder_name, subfolders, filenames in os.walk(h / 'spam'):
    print('The current folder is', folder_name)
    for subfolder in subfolders:
        print(' SUBFOLDER:', subfolder)
    for filename in filenames:
        print(' FILE:', filename)

Example: renaming all files to uppercase​

Inside the os.walk loop, use shutil.move to rename each file.

import os, shutil
from pathlib import Path

h = Path.home()
for folder_name, subfolders, filenames in os.walk(h / 'spam'):
    p = Path(folder_name)
    for filename in filenames:
        shutil.move(p / filename, p / filename.upper())

Compressing files with the zipfile module​

ZIP files act as compressed archives of many files/folders.

Creating and adding to ZIP files​

Open ZipFile in write ('w') or append ('a') mode; use write() with compression.

import zipfile

with open('file1.txt', 'w', encoding='utf-8') as f:
    f.write('Hello' * 10000)

with zipfile.ZipFile('example.zip', 'w') as zf:
    zf.write('file1.txt',
             compress_type=zipfile.ZIP_DEFLATED,
             compresslevel=9)

Reading ZIP files​

Use ZipFile.namelist() and getinfo() to inspect contents and sizes.

import zipfile

zf = zipfile.ZipFile('example.zip')
print(zf.namelist())               # ['file1.txt']
info = zf.getinfo('file1.txt')
print(info.file_size, info.compress_size)
zf.close()

Extracting from ZIP files​

  • extractall([path]) extracts everything.
  • extract(name, [path]) extracts one member.
import zipfile

with zipfile.ZipFile('example.zip') as zf:
    zf.extractall('C:/spam')
    zf.extract('file1.txt', 'C:/some/new/folders')

Project: backup_to_zip (incrementing ZIP backups)​

Program: backup_to_zip.py backs up a folder into numbered ZIPs like spam_1.zip, spam_2.zip, etc.

Step 1: Figure out the ZIP file's name​

Loop number = 1, 2, ... until spam_<number>.zip does not exist.

from pathlib import Path

def backup_to_zip(folder):
    folder = Path(folder)
    number = 1
    while True:
        zip_filename = Path(folder.parts[-1] + '_' + str(number) + '.zip')
        if not zip_filename.exists():
            break
        number += 1
    # ...

Step 2: Create the new ZIP file​

Open ZipFile in write mode, print the name.

import zipfile

print(f'Creating {zip_filename}...')
backup_zip = zipfile.ZipFile(zip_filename, 'w')

Step 3: Walk the directory tree and add files​

Use os.walk over folder, add each file into the ZIP, then close().

import os

for folder_name, subfolders, filenames in os.walk(folder):
    folder_name = Path(folder_name)
    print(f'Adding files in folder {folder_name}...')
    for filename in filenames:
        print(f' Adding file {filename}...')
        backup_zip.write(folder_name / filename)

backup_zip.close()
print('Done.')

Call example:

from pathlib import Path
backup_to_zip(Path.home() / 'spam')

Ideas for other programs​

Suggested variations using os.walk, zipfile, and filters:

  • Archive only files with certain extensions (e.g. .txt, .py).
  • Archive everything except certain extensions.
  • Only archive folders that are very large or recently modified.

Example filter idea:

import os, zipfile
from pathlib import Path

root = Path.home() / 'project'
with zipfile.ZipFile('code_backup.zip', 'w') as zf:
    for folder_name, subfolders, filenames in os.walk(root):
        folder_name = Path(folder_name)
        for filename in filenames:
            if filename.endswith('.py'):
                zf.write(folder_name / filename,
                         compress_type=zipfile.ZIP_DEFLATED)

Summary (chapter recap)​

  • Use os/shutil/send2trash to copy, move, rename, delete safely (with dry runs).
  • Use os.walk to operate on entire directory trees.
  • Use zipfile to compress/extract ZIP archives, combining with file operations for backups and packaging.

Practice questions​

The chapter ends with conceptual questions, for example:

  1. Difference between shutil.copy() and shutil.copytree()?
  2. Which function renames files?
  3. Difference between send2trash delete and shutil delete?
  4. Which ZipFile method corresponds to open() for files?

(Answers in short: copy vs whole tree; shutil.move; recycle-bin vs permanent delete; zipfile.ZipFile() constructor.)

Practice programs​

Four suggested exercises:

  1. Selectively Copying – copy only certain extensions into a new folder.
  2. Deleting Unneeded Files – find and print files over a size threshold (e.g. 100MB).
  3. Renumbering Files – close gaps in numbered filenames (e.g. spam001.txt).
  4. Converting Dates – rename files from MM-DD-YYYY to DD-MM-YYYY using os.walk, regex, and shutil.move.

Example snippet for "Deleting Unneeded Files":

import os
from pathlib import Path

root = Path.home()
for folder, subfolders, filenames in os.walk(root):
    for name in filenames:
        path = Path(folder) / name
        if os.path.getsize(path) > 100 * 1024 * 1024:
            print(path, 'is larger than 100MB')

Overall idea of the chapter​

Chapter 11 shows how to automate common file-management tasks — copying, moving, renaming, deleting, and zipping — using shutil, os, send2trash, and zipfile. The dry-run pattern (print before deleting) is a key safety habit, and os.walk is the workhorse for processing entire directory trees.