Paths & File System Navigation

Database Regeneration - Practical Scripting

Course 1 · Chapter 1 · Paths & File System Navigation

🗂️ Paths & File System Navigation

When you write a Python script that works with files, the biggest challenge isn't the logic—it's figuring out where the files actually are. This chapter teaches the most important concept in practical Python scripting: how to reliably locate and work with files on any computer, in any directory, on any operating system.

⚠️ The Problem We're Solving

Imagine you write a script that reads HTML files. On your Windows computer, you test it:

with open("C:\\Users\\philip\\files\\chapters\\ts1-1.html") as f:
    content = f.read()

It works fine. You copy the script to your Linux web server and run it:

with open("C:\\Users\\philip\\files\\chapters\\ts1-1.html") as f:
    content = f.read()

It crashes. Why? Windows and Linux have different path formats, and the directories are different.

❌ COMMON MISTAKE: Hard-coding file paths. This makes your script work on one computer but nowhere else.

The solution: **make your script find files relative to where it's located**. This is what this chapter teaches.

✨ file: The Magic Variable

__file__ is a special Python variable that automatically contains the path to the current script. It's like Python saying "here's where I am right now."

What file Contains

Create a test file called test.py with this code:

print(__file__)

On Windows: Running from C:\Users\philip\scripts\

C:\Users\philip\scripts\test.py

On Linux: Running from /home/philip/scripts/

/home/philip/scripts/test.py

✅ WHAT IT DOES: __file__ is a string containing the absolute path to the Python script currently running. It automatically adapts to Windows or Linux path format.

🤔 WHY IT MATTERS: Your script now knows where it is. This is the foundation for finding other files relative to its location.

📦 The pathlib Module and Path Objects

Python's pathlib module provides a better way to work with file paths. Instead of string manipulation (which is error-prone), you use Path objects that handle Windows/Linux differences automatically.

Creating a Path Object

from pathlib import Path

# Convert __file__ to a Path object
script_path = Path(__file__)
print(script_path)

✅ WHAT IT DOES: Path(__file__) converts the string path into a Path object. Now you can call methods on it.

🤔 WHY USE Path INSTEAD OF STRINGS:

Path handles Windows backslashes vs Linux forward slashes automatically
Path has useful methods like .parent, .exists(), .open()
Path prevents errors from mixing path separators

Alternative: os.path (Old Way)

import os

script_path = os.path.abspath(__file__)
script_dir = os.path.dirname(script_path)

Old Way (os.path)

import os
import os.path

chapters_dir = os.path.join(
    os.path.dirname(__file__),
    "courses",
    "html"
)

New Way (pathlib)

from pathlib import Path

chapters_dir = Path(__file__).parent / "courses" / "html"

💡 Modern Recommendation

Always use pathlib.Path for new code. It's cleaner, safer, and more readable. The old os.path module still works, but pathlib is the modern standard.

👆 The .parent Property: Going Up Directories

The .parent property gets the directory that contains a file or directory. You can chain multiple .parent calls to go up multiple levels.

Understanding Parent Step-by-Step

Let's say your script is at:

/var/www/myapp/rebuild_typescript_course.py

Here's what .parent does at each step:

Path Traversal with .parent

Stage 1:

Path(__file__)

Result:

/var/www/myapp/rebuild_typescript_course.py

This is the script file itself.

Stage 2:

Path(__file__).parent

Result:

/var/www/myapp

Go up one level to the directory containing the script.

Stage 3:

Path(__file__).parent.parent

Result:

/var/www

Go up two levels.

Stage 4:

Path(__file__).parent.parent.parent

Result:

/var

Go up three levels.

✅ WHAT IT DOES: Each .parent goes up one directory level. You can chain them: .parent.parent.parent goes up three levels.

🤔 WHY THIS MATTERS: Your script doesn't know where it will be placed. By using .parent, your script can find files relative to its own location, regardless of where it's copied to.

🔨 Building Paths with the / Operator

Once you have a directory, you add subdirectories and filenames using the / operator (the slash character). This is much cleaner than string concatenation.

The / Operator for Paths

from pathlib import Path

# Start from the script's directory
script_dir = Path(__file__).parent

# Build a path step by step
chapters_dir = script_dir / "courses" / "Programming" / "TypeScript" / "html"

# Get a specific file
chapter_file = chapters_dir / "ts1-1.html"

print(chapter_file)
# Output: /var/www/myapp/courses/Programming/TypeScript/html/ts1-1.html

✅ WHAT IT DOES: The / operator appends a subdirectory or filename to a path. It works on Windows and Linux automatically.

🤔 WHY USE / INSTEAD OF STRING CONCATENATION:

✅ script_dir / "chapters" — Clean and readable
❌ script_dir + "\chapters" — Breaks on Linux (needs forward slash)
❌ script_dir + "/" + "chapters" — Verbose and fragile

Real Example: rebuild_typescript_course.py

Here's how the actual rebuild script finds chapters:

from pathlib import Path

# The script is at: /var/www/myapp/rebuild_typescript_course.py
# We want chapters at: /var/www/myapp/courses/Programming/TypeScript/html/

CHAPTERS_DIR = Path(__file__).parent / "courses" / "Programming" / "TypeScript" / "html"

# Now we can use it:
html_files = list(CHAPTERS_DIR.glob("ts1-*.html"))
for html_file in html_files:
    print(f"Found: {html_file}")

✅ Checking If Files Exist

The .exists() Method

from pathlib import Path

file_path = Path("/var/www/myapp/courses/TypeScript/html/ts1-1.html")

if file_path.exists():
    print("✅ File found!")
else:
    print("❌ File not found!")

✅ WHAT IT DOES: .exists() returns True if the file or directory exists, False otherwise.

Other Useful Methods

from pathlib import Path

file_path = Path("/var/www/myapp/courses/ts1-1.html")

# Is it a file (not a directory)?
if file_path.is_file():
    print("It's a file")

# Is it a directory (not a file)?
if file_path.is_dir():
    print("It's a directory")

# Get just the filename without the directory
print(file_path.name)  # Output: ts1-1.html

# Get the file extension
print(file_path.suffix)  # Output: .html

# Get the filename without extension
print(file_path.stem)  # Output: ts1-1

🖥️ Windows vs Linux: Path Formats

The Path Format Problem

Windows uses backslashes:

C:\Users\philip\files\chapters\ts1-1.html

Linux uses forward slashes:

/home/philip/files/chapters/ts1-1.html

If you hard-code Windows paths, your script breaks on Linux. If you hard-code Linux paths, your script breaks on Windows.

✅ THE SOLUTION: Use pathlib.Path. It automatically uses the correct separator for your operating system.

from pathlib import Path

# This works the same way on Windows and Linux!
chapters_dir = Path(__file__).parent / "courses" / "html"

# On Windows, it becomes: C:\path\to\courses\html
# On Linux, it becomes: /path/to/courses/html
print(chapters_dir)

⚠️ Common Mistakes & How to Fix Them

❌ MISTAKE 1: Forgetting to import Path

# This will crash!
chapters_dir = Path(__file__).parent / "courses"
# NameError: name 'Path' is not defined

Fix:

from pathlib import Path  # ← Add this!

chapters_dir = Path(__file__).parent / "courses"

❌ MISTAKE 2: Using + instead of / to combine paths

# This doesn't work as expected!
chapters_dir = Path(__file__).parent + "/courses"
# TypeError: unsupported operand type(s) for +: 'PosixPath' and 'str'

Fix:

chapters_dir = Path(__file__).parent / "courses"  # ← Use /

❌ MISTAKE 3: Assuming __file__ is always absolute

In some cases (like running with python -m), __file__ might be relative.

Fix:

from pathlib import Path

# Make sure it's absolute
script_dir = Path(__file__).resolve().parent

🎯 Edge Cases & Special Situations

Edge Case 1: File Doesn't Exist Yet

If you're creating a new file, the path won't exist, but that's OK:

from pathlib import Path

file_path = Path(__file__).parent / "new_file.txt"

if not file_path.exists():
    with open(file_path, "w") as f:
        f.write("Hello")

Edge Case 2: Symbolic Links (Shortcuts)

.resolve() follows symbolic links to the actual file:

from pathlib import Path

# If __file__ is a symlink, follow it to the real file
real_script = Path(__file__).resolve()

Edge Case 3: Relative Paths from Current Directory

Sometimes you want files relative to where the user ran the command, not where the script is:

from pathlib import Path

# User's current working directory
current_dir = Path(".").resolve()

# Script's directory
script_dir = Path(__file__).resolve().parent

🌍 Real-World Example: rebuild_typescript_course.py

Here's how the concepts in this chapter are used in actual production code:

from pathlib import Path

# This script is at: /var/www/myapp/rebuild_typescript_course.py

# Start from the script's location
SCRIPT_DIR = Path(__file__).resolve().parent

# Build the path to chapters folder relative to the script
CHAPTERS_DIR = SCRIPT_DIR / "courses" / "Programming" / "TypeScript" / "html"

# Check if it exists
if not CHAPTERS_DIR.exists():
    print(f"❌ Chapters directory not found: {CHAPTERS_DIR}")
    exit(1)

# Find all ts1-*.html files
html_files = sorted(CHAPTERS_DIR.glob("ts1-*.html"))

for html_file in html_files:
    print(f"📄 Processing: {html_file.name}")
    with open(html_file, "r") as f:
        content = f.read()
        print(f"   Read {len(content)} bytes")

💻 Coding Challenge

Challenge: Create a Path Finder Script

Create a Python script that:

Uses __file__ to find the script's directory
Creates a path to a subdirectory called data using /
Checks if the data directory exists
If it exists, lists all files in it
If it doesn't exist, creates it

Goal: Practice using Path, .parent, .exists(), and the / operator.

→ Solution

🎯 What's Next

Now that you can locate files reliably, the next chapter covers Configuration & Environment Variables — how to load settings like database credentials from .env files so your scripts can adapt to different environments.

Paths & File System Navigation