================================================================================
PRACTICAL PYTHON SCRIPTING - CHALLENGE 3 SOLUTION
Chapter 1: Paths & File System Navigation
Challenge: Complex Path Navigation (rebuild_typescript_course.py pattern)
================================================================================

PROBLEM:
Create a script that mirrors the structure of rebuild_typescript_course.py:
1. Use Path(__file__).parent to find the script location
2. Navigate to a nested directory structure (courses/Programming/TypeScript/html/)
3. Find all chapter HTML files (ts1-*.html)
4. Extract chapter numbers and titles from the HTML
5. Display a summary of what was found

This teaches the actual pattern used in production code.

SOLUTION:
================================================================================

from pathlib import Path
import re

# Find the script's directory
SCRIPT_DIR = Path(__file__).resolve().parent
print(f"Script location: {SCRIPT_DIR}\n")

# Navigate to nested directory using / operator
CHAPTERS_DIR = SCRIPT_DIR / "courses" / "Programming" / "TypeScript" / "html"
print(f"Looking for chapters in: {CHAPTERS_DIR}")

# Validate the path exists
if not CHAPTERS_DIR.exists():
    print(f"❌ Chapters directory not found!")
    print(f"   Expected: {CHAPTERS_DIR}")
    print(f"\nTo test this script:")
    print(f"  1. Create the directory structure:")
    print(f"     mkdir -p {CHAPTERS_DIR}")
    print(f"  2. Add HTML files like ts1-1.html, ts1-2.html, etc.")
    exit(1)

# Find all chapter HTML files using glob()
html_files = sorted(CHAPTERS_DIR.glob("ts1-*.html"))

if not html_files:
    print(f"⚠️ No chapter files found (pattern: ts1-*.html)")
    exit(1)

print(f"✅ Found {len(html_files)} chapter file(s)\n")

# Process each chapter file
chapters = []
for html_file in html_files:
    try:
        # Extract chapter number from filename (ts1-1.html → 1)
        match = re.match(r"ts1-(\d+)", html_file.stem)
        if not match:
            continue

        chapter_num = int(match.group(1))

        # Read the HTML file
        with open(html_file, 'r', encoding='utf-8') as f:
            html_content = f.read()

        # Extract chapter title from <h2> tag
        title_match = re.search(r'<h2>(.*?)</h2>', html_content)
        chapter_title = title_match.group(1) if title_match else f"Chapter {chapter_num}"

        # Store chapter info
        chapters.append({
            'number': chapter_num,
            'title': chapter_title,
            'filename': html_file.name,
            'path': html_file,
            'size': html_file.stat().st_size
        })

        print(f"✅ Chapter {chapter_num}: {chapter_title}")
        print(f"   File: {html_file.name}")
        print(f"   Size: {html_file.stat().st_size:,} bytes\n")

    except Exception as e:
        print(f"❌ Error processing {html_file.name}: {e}\n")

# Display summary
print("=" * 60)
print(f"SUMMARY")
print("=" * 60)
print(f"Total chapters found: {len(chapters)}")
print(f"Location: {CHAPTERS_DIR}\n")

if chapters:
    print("Chapter list:")
    for ch in chapters:
        print(f"  {ch['number']:2d}. {ch['title']}")

    total_size = sum(ch['size'] for ch in chapters)
    print(f"\nTotal content size: {total_size:,} bytes ({total_size / 1024 / 1024:.2f} MB)")

print("=" * 60)

================================================================================

HOW IT WORKS:
================================================================================

1. FINDING THE SCRIPT LOCATION
   SCRIPT_DIR = Path(__file__).resolve().parent

   - __file__ = path to this script
   - .resolve() = convert to absolute path (handles symlinks)
   - .parent = the directory containing the script

2. NAVIGATING TO NESTED DIRECTORIES
   CHAPTERS_DIR = SCRIPT_DIR / "courses" / "Programming" / "TypeScript" / "html"

   - Use / to append each directory level
   - Each / appends one directory
   - Result: /full/path/to/script/courses/Programming/TypeScript/html

3. VALIDATING THE PATH
   if not CHAPTERS_DIR.exists():
       # Handle error

   - Check if the nested directory actually exists
   - Helpful error message if it doesn't
   - Prevents confusing errors later

4. FINDING FILES WITH PATTERNS
   html_files = sorted(CHAPTERS_DIR.glob("ts1-*.html"))

   - .glob("ts1-*.html") finds files matching pattern
   - Patterns: "ts1-*" means starts with "ts1-"
   - "*.html" means ends with ".html"
   - So "ts1-*.html" matches: ts1-1.html, ts1-2.html, etc.
   - sorted() puts them in order

5. EXTRACTING INFORMATION
   - html_file.stem = filename without extension
   - Re.match(r"ts1-(\d+)", stem) = extract chapter number
   - (\d+) captures one or more digits
   - open(html_file, 'r') = read the file
   - re.search() = find text matching a pattern in the content

6. STORING AND REPORTING
   - Store chapter info in a list of dictionaries
   - Display formatted output
   - Calculate statistics (total size, count)

================================================================================

PATH MANIPULATION TECHNIQUES:
================================================================================

Technique 1: Building paths step-by-step
script_dir = Path(__file__).parent

# Add one level at a time
level1 = script_dir / "courses"
level2 = level1 / "Programming"
level3 = level2 / "TypeScript"
level4 = level3 / "html"

# Or all at once:
final = script_dir / "courses" / "Programming" / "TypeScript" / "html"

Both are equivalent!

---

Technique 2: Getting parts of paths
full_path = Path("/var/www/myapp/courses/Programming/TypeScript/html/ts1-1.html")

full_path.parent  # /var/www/myapp/courses/Programming/TypeScript/html
full_path.name    # ts1-1.html
full_path.stem    # ts1-1 (filename without extension)
full_path.suffix  # .html (just the extension)

parts = full_path.parts
# ('/','var','www','myapp','courses','Programming','TypeScript','html','ts1-1.html')

---

Technique 3: Relative paths
script_dir = Path(__file__).parent

# This works:
chapters_dir = script_dir / "courses" / "Programming" / "TypeScript" / "html"

# But what if you know the relative path from script to chapters?
# You can navigate backwards then forwards:
chapters_dir = script_dir / "../debserver/chapters"

# Or more safely:
chapters_dir = script_dir.parent.parent / "debserver" / "chapters"

================================================================================

UNDERSTANDING GLOB PATTERNS:
================================================================================

Glob Pattern        Matches
*                   Any characters (except /)
*.html              file.html, chapter.html, ts1-1.html
ts1-*.html          ts1-1.html, ts1-2.html, ts1-10.html
ts1-[1-3].html      ts1-1.html, ts1-2.html, ts1-3.html (1 through 3)
[abc]*.html         a.html, bfile.html, chart.html (starts with a, b, or c)
**/*.html           Recursively find all .html files in subdirectories

Examples:
source_dir.glob("*.txt")        # Find all .txt files
source_dir.glob("test_*.py")    # Find test_*.py files
source_dir.glob("**/config.json") # Find config.json anywhere inside

================================================================================

REGEX PATTERN EXPLANATION:
================================================================================

Pattern: r"ts1-(\d+)"

r"..." = raw string (backslashes are literal, not escape characters)

ts1-  = Match literally "ts1-"
(\d+) = Capture one or more digits
  \d  = Any digit (0-9)
  +   = One or more times
  ()  = Capture this group for extraction

Examples:
"ts1-1"   → Matches, captures: "1"
"ts1-15"  → Matches, captures: "15"
"ts1-abc" → No match (not digits)
"ts2-1"   → No match (says ts2, not ts1)

In the code:
match = re.match(r"ts1-(\d+)", "ts1-1")
match.group(0)  # "ts1-1" (the whole match)
match.group(1)  # "1" (the first captured group)

================================================================================

FILE OPERATIONS USED:
================================================================================

open(path, 'r', encoding='utf-8')
- Open file for reading ('r')
- Specify UTF-8 encoding (important for compatibility)
- Returns a file object
- Use with 'with' statement to auto-close

f.read()
- Read entire file contents as a string
- For large files, use .readlines() or iterate

html_file.stat()
- Get file statistics (size, timestamps, permissions)
- .stat().st_size = file size in bytes

sorted(list_of_paths)
- Sort Path objects in alphabetical order
- Helpful for processing files in order

================================================================================

ERROR HANDLING:
================================================================================

try/except block handles:
- FileNotFoundError: File doesn't exist
- UnicodeDecodeError: File encoding issue
- Regular expression errors: Malformed pattern
- IOError: Permission denied

Specific handling:
except FileNotFoundError:
    print(f"File not found: {file}")

except UnicodeDecodeError:
    print(f"Could not decode file (wrong encoding): {file}")

except Exception as e:
    print(f"Unexpected error: {e}")

The solution uses "except Exception" which catches all errors. For
production code, you'd handle specific errors. But for learning and
general utility scripts, this is appropriate.

================================================================================

TESTING THE SOLUTION:
================================================================================

To test this script:

1. Create the directory structure:
   mkdir -p courses/Programming/TypeScript/html

2. Create some test HTML files:
   cat > courses/Programming/TypeScript/html/ts1-1.html << 'EOF'
   <h2>Chapter 1: Introduction</h2>
   <p>This is the first chapter.</p>
   EOF

   cat > courses/Programming/TypeScript/html/ts1-2.html << 'EOF'
   <h2>Chapter 2: Advanced Types</h2>
   <p>This is the second chapter.</p>
   EOF

3. Run the script:
   python chapter_finder.py

Expected output:
Script location: /home/philip/scripts

Looking for chapters in: /home/philip/scripts/courses/Programming/TypeScript/html
✅ Found 2 chapter file(s)

✅ Chapter 1: Chapter 1: Introduction
   File: ts1-1.html
   Size: 58 bytes

✅ Chapter 2: Chapter 2: Advanced Types
   File: ts1-2.html
   Size: 56 bytes

============================================================
SUMMARY
============================================================
Total chapters found: 2
Location: /home/philip/scripts/courses/Programming/TypeScript/html

Chapter list:
   1. Chapter 1: Introduction
   2. Chapter 2: Advanced Types

Total content size: 114 bytes (0.00 MB)
============================================================

================================================================================

REAL-WORLD CONNECTION:
================================================================================

This solution demonstrates the actual pattern used in
rebuild_typescript_course.py:

1. Find script location using Path(__file__)
2. Navigate to chapters directory
3. Validate the path exists
4. Find files using glob()
5. Extract information from each file
6. Store results for processing
7. Generate a summary

The difference in rebuild_typescript_course.py:
- Also loads database utilities
- Inserts chapters into a database
- Uses try/except for database errors
- Reports database operation results

But the PATH NAVIGATION pattern is identical! That's why this chapter
is the foundation for everything else.

================================================================================

KEY TAKEAWAYS:
================================================================================

✅ Path(__file__).parent finds the script's directory
✅ .resolve() handles symlinks and creates absolute paths
✅ Use / to build nested paths step-by-step
✅ .exists() validates paths before using them
✅ .glob() finds files matching patterns
✅ .stat() gets file information
✅ sorted() puts results in order
✅ Try/except handles errors gracefully
✅ Path properties (.stem, .name, .suffix) extract parts
✅ This pattern scales from simple scripts to complex utilities

The concepts you've learned here are the FOUNDATION for all practical
Python scripting. Every utility script follows this basic pattern:

1. Find files relative to the script location
2. Validate paths exist
3. Process files
4. Report results

Master this pattern, and you can write any utility script!

================================================================================