Building rebuild_typescript_course.py

Database Regeneration - Practical Scripting
Course 1 ยท Chapter 5 ยท Building rebuild_typescript_course.py

๐Ÿ”„ Building rebuild_typescript_course.py

This chapter brings everything together. rebuild_typescript_course.py is the complete automation script that uses all the tools from chapters 1-4: navigating the filesystem (paths), loading configuration (.env), connecting to databases, and using db_utils to populate the database with generated course chapters. This is a real-world, production-ready script.

๐Ÿ“‹ Overview: What This Script Does

The script automates the complete workflow:

Step 1: Find Chapter Files

Use Path and glob() to find all ts1-*.html files

Step 2: Parse Chapter Data

Read HTML files and extract title from <h2> tags with regex

Step 3: Connect to Database

Load .env and use DatabaseConnection to connect

Step 4: Create Course

Use get_or_create_subtopic() to create TypeScript Fundamentals course

Step 5: Insert Chapters

For each chapter, call insert_chapter_with_content()

Step 6: Report Results

Display summary of what was created

๐Ÿ“ Directory Structure

The script expects this layout:

myapp/
โ”œโ”€โ”€ .env                          โ† Configuration (loaded by script)
โ”œโ”€โ”€ rebuild_typescript_course.py  โ† This script
โ””โ”€โ”€ courses/
    โ””โ”€โ”€ Programming/
        โ””โ”€โ”€ TypeScript/
            โ””โ”€โ”€ html/
                โ”œโ”€โ”€ ts1-1.html    โ† Chapter files
                โ”œโ”€โ”€ ts1-2.html
                โ”œโ”€โ”€ ts1-3.html
                ...
                โ””โ”€โ”€ ts1-12.html
๐Ÿค” WHY THIS STRUCTURE:

Using Path(__file__).parent, the script can navigate relative to itself. No matter where you run it from, it finds the chapters in the correct location.

๐Ÿ”ง Main Components

Imports & Constants

from pathlib import Path
import sys
import re

from db_utils import DatabaseConnection, get_or_create_subtopic, insert_chapter_with_content

# Constants
SUBJECT_ID = 3  # Programming subject
SUBJECT_NAME = "Programming"
SUBTOPIC_NAME = "TypeScript Fundamentals"

# Calculate path to chapters directory
SCRIPT_DIR = Path(__file__).resolve().parent
CHAPTERS_DIR = SCRIPT_DIR / "courses" / "Programming" / "TypeScript" / "html"

Finding Chapter Files

def find_chapters() -> list:
    """Find all chapter HTML files."""
    if not CHAPTERS_DIR.exists():
        print(f"โŒ Chapters directory not found: {CHAPTERS_DIR}")
        return []

    # Find all ts1-*.html files and sort them
    html_files = sorted(CHAPTERS_DIR.glob("ts1-*.html"))

    if not html_files:
        print(f"โš ๏ธ No chapter files found in {CHAPTERS_DIR}")
        return []

    print(f"โœ… Found {len(html_files)} chapter files")
    return html_files

Parsing Chapter Data

def parse_chapter(html_file: Path) -> dict:
    """Extract chapter data from HTML file."""
    try:
        # Read HTML content
        html_content = html_file.read_text(encoding='utf-8')

        # Extract chapter number from filename (ts1-1.html โ†’ 1)
        match = re.match(r"ts1-(\d+)", html_file.stem)
        if not match:
            return None

        chapter_num = int(match.group(1))

        # Extract title from <h2> tag
        title_match = re.search(r'<h2>(.*?)</h2>', html_content)
        chapter_title = title_match.group(1) if title_match else f"Chapter {chapter_num}"

        return {
            'number': chapter_num,
            'title': chapter_title,
            'html': html_content,
            'sort_order': chapter_num * 10
        }

    except Exception as e:
        print(f"โŒ Error parsing {html_file.name}: {e}")
        return None

Main Workflow

def main():
    """Main rebuild workflow."""
    print("=" * 60)
    print("๐Ÿš€ TypeScript Course Rebuild")
    print("=" * 60 + "\n")

    # Step 1: Find chapters
    html_files = find_chapters()
    if not html_files:
        exit(1)

    # Step 2: Connect to database
    print("\n๐Ÿ”Œ Connecting to database...")
    db = DatabaseConnection()
    if not db.connect():
        exit(1)

    try:
        # Step 3: Create or get subtopic
        print("\n๐Ÿ“š Creating course...")
        subtopic_id = get_or_create_subtopic(
            db,
            subject_id=SUBJECT_ID,
            subtopic_name=SUBTOPIC_NAME,
            description="Master TypeScript from fundamentals to advanced patterns"
        )

        if subtopic_id is None:
            print("โŒ Failed to create subtopic")
            exit(1)

        # Step 4: Insert chapters
        print(f"\n๐Ÿ“ Inserting {len(html_files)} chapters...")

        inserted_count = 0
        for html_file in html_files:
            chapter = parse_chapter(html_file)
            if not chapter:
                continue

            page_id = insert_chapter_with_content(
                db,
                subtopic_id=subtopic_id,
                chapter_title=chapter['title'],
                html_content=chapter['html'],
                sort_order=chapter['sort_order']
            )

            if page_id:
                inserted_count += 1

        # Step 5: Report results
        print("\n" + "=" * 60)
        print(f"โœ… REBUILD COMPLETE!")
        print("=" * 60)
        print(f"Course: {SUBTOPIC_NAME}")
        print(f"Chapters inserted: {inserted_count}")

    finally:
        db.disconnect()


if __name__ == "__main__":
    main()

๐Ÿ“Œ Key Patterns Used

Pattern From Chapter Purpose
Path(__file__).parent Chapter 1 Find script's directory
glob("ts1-*.html") Chapter 1 Find all chapter files
load_dotenv() Chapter 2 Load database credentials
DatabaseConnection Chapter 3 Connect to database
get_or_create_subtopic() Chapter 4 Create course
insert_chapter_with_content() Chapter 4 Insert chapters

๐Ÿš€ Running the Script

# From the myapp directory:
python rebuild_typescript_course.py

# Output:
============================================================
๐Ÿš€ TypeScript Course Rebuild
============================================================

โœ… Found 12 chapter files

๐Ÿ”Œ Connecting to database...
โœ… Database connection established

๐Ÿ“š Creating course...
โœ… Created subtopic 'TypeScript Fundamentals' with ID 8

๐Ÿ“ Inserting 12 chapters...
โœ… Created chapter with ID: 101
โœ… Created chapter with ID: 102
โœ… Created chapter with ID: 103
...

============================================================
โœ… REBUILD COMPLETE!
============================================================
Course: TypeScript Fundamentals
Chapters inserted: 12

๐Ÿ’ป Coding Challenges

Challenge 1: Test File Discovery

Create a script that:

  1. Uses Path(__file__) to find its directory
  2. Navigates to courses/Programming/TypeScript/html/
  3. Uses glob() to find all ts1-*.html files
  4. Reports how many found and lists them

Goal: Practice path navigation and file discovery.

โ†’ Solution

Challenge 2: Parse Chapter Metadata

Create a script that:

  1. Finds all chapter HTML files
  2. For each file, extract: chapter number, title from <h2>
  3. Display a table with all chapters
  4. Show chapter count

Goal: Practice HTML parsing with regex.

โ†’ Solution

Challenge 3: Complete Rebuild Script

Create a script that:

  1. Finds all chapter files
  2. Parses metadata from HTML
  3. Connects to database using db_utils
  4. Creates subtopic and inserts all chapters
  5. Reports final count

Goal: Build the complete automation workflow.

โ†’ Solution

๐ŸŽฏ What's Next

Chapter 6, the final chapter, covers Building Your Own Utilities โ€” how to take the patterns you've learned and create your own reusable scripts for any project. You'll also learn best practices for organizing, documenting, and testing your utilities.