# Course Evaluator A Claude Code workflow for evaluating course materials against 10 pedagogical frameworks. No API key required — uses your Claude.ai subscription directly. --- ## ⚠️ Privacy Notice — Check Before Uploading Before uploading any course materials (syllabi, assignment instructions, rubrics, session slides, or any other files), please ensure they contain **no personal names** of teachers, lecturers, coordinators, or guest speakers. Replace any names with a role or discipline descriptor, for example: - "Dr. J. Smith" → "Course Coordinator" or "Law Professor" - "Prof. van X" → "Governance Professor" or "Economics Lecturer" - "Guest: A. Johnson, Ministry of Finance" → "Finance Policy Practitioner" The output reports generated by this tool are intended to be shared more broadly. Keeping names out of source documents ensures they stay out of reports. --- ## Commands ### `setup [course-name]` — create a new course or update context for an existing one 1. If `[course-name]` is provided, create `input/[course-name]/` with all six numbered subfolders if it doesn't already exist. Confirm creation to the user. 2. If no `[course-name]` is provided, list existing course folders in `input/` and ask which one to set up. 3. Ask the user to confirm the syllabus is in `input/[course-name]/1_syllabus/`. If it is, read it. If not, ask the user to drop it in before continuing. 4. Extract all course context fields (see Course Context Fields below). 5. Show the extracted values for review and allow corrections. 6. Write the confirmed values to `input/[course-name]/_context.md`. ### `evaluate` — single material 1. List all course folders in `input/`. If multiple, ask which course. 2. List all files in that course's numbered subfolders. Ask which file to evaluate. 3. Ask what type of material it is: syllabus / objectives / assignment / session / reading / rubric 4. Ask which frameworks to apply (default: all 10) 5. Read `input/[course-name]/_context.md` for course context. If it doesn't exist, prompt to run `setup` first. 6. Read the file content 7. Run the evaluation using the frameworks and course context 8. Write a Word doc report to `output/` using python-docx (see Output section) ### `evaluate --course [course-name]` — whole-course evaluation 1. If `[course-name]` is not provided, list course folders in `input/` and ask which to evaluate. 2. Read `input/[course-name]/_context.md`. If it doesn't exist, prompt to run `setup [course-name]` first. 3. Read all files from the course's numbered subfolders in order: `1_syllabus` → `2_objectives` → `3_assessments` → `4_sessions` → `5_readings` → `6_rubrics`. Skip empty subfolders silently. 4. Evaluate each component individually using all frameworks. 5. After all components, run the synthesis evaluation (see Synthesis section below). 6. Write one Word doc report per component + one synthesis report to `output/`. 7. Write one structured MD summary file to `output/` (see MD Output section). --- ## Course Context Fields *(These fields live in each course's `_context.md` file — not here. This is the template `setup` uses.)* - Course name: - Level: master / professional / mixed - Student profile: - Learning objectives: - Teaching formats: - Assessment methods: - Prior knowledge assumptions: - Course description: --- ## Whole-Course Evaluation Order Process components in this order — constructive alignment works top-down: 1. Syllabus — establishes course intent and overall design logic 2. Objectives — specific, measurable, right cognitive level? 3. Assessments — do they measure what the objectives specify? 4. Sessions — do teaching activities prepare students for the assessments? 5. Readings — do materials support the session activities? 6. Rubrics — do they give students actionable guidance aligned to objectives? After all individual evaluations, run the synthesis (see below). --- ## Frameworks Apply all principles at master's / professional education level. Never reference primary school contexts. ### 1. Constructive Alignment (Biggs) Does the learning objective → teaching activity → assessment triangle hold together? - Are learning objectives clearly stated at the appropriate cognitive level? - Do teaching activities directly prepare students to achieve those objectives? - Does the assessment actually measure what the objectives specify? - Is there a clear line of sight from objective → activity → assessment? ### 2. Bloom's Taxonomy (Bloom) Are learning objectives at the right cognitive level for master's students? - Do objectives go beyond recall — do they require analysis, synthesis, or evaluation? - Are students asked to apply concepts to new situations, not just reproduce them? - Does the assignment require higher-order thinking (critique, design, judge)? - Is the cognitive demand appropriate for master's / professional level? ### 3. Cognitive Load (Sweller) Is the material designed so students can actually process and retain it? - Is complex content introduced with worked examples before students work independently? - Are complex topics broken into manageable steps before integration? - Does the material avoid unnecessary complexity that competes with the learning goal? - Are instructions clear enough that students can focus on content, not decoding the task? ### 4. Prior Knowledge & Expert–Novice (Ausubel / Chi et al.) Does the design account for what students already know — and don't know? - Does the material explicitly connect to students' prior knowledge or professional experience? - Are misconceptions anticipated and addressed? - Is scaffolding appropriate — enough for topic beginners, not condescending to their overall level? - Does the material avoid assuming students think like experts when they may not yet? ### 5. Active Learning & Desirable Difficulties (Various) Are students required to actively process, not passively receive? - Does the design require students to do something with content — not just read or listen? - Are retrieval, elaboration, or application built into the learning sequence? - Do questions or tasks steer the kind of thinking you want students to develop? - Is there spaced practice or returning to key concepts across sessions? ### 6. Feedback & Assessment for Learning (Hattie & Timperley / Black & Wiliam) Does assessment inform and support learning, not just measure it? - Does feedback answer: Where am I going? / How am I doing? / What next? - Is feedback targeted at process and reasoning, not just correctness? - Is there formative assessment before high-stakes evaluation? - Can students actually act on the feedback they receive? ### 7. Scaffolding & Gradual Release (Wood, Bruner & Ross / Vygotsky) Is support calibrated to the student's level and gradually withdrawn? - Is support provided just above the student's current level? - Is there a clear sequence: model → guided practice → independent work? - Is the level of independence expected appropriate for where students are in the course? - Are students required to master foundational steps before tackling complex wholes? ### 8. Self-Regulated Learning (Zimmerman) Are students supported in planning, monitoring, and evaluating their own learning? - Are students required to reflect on their own learning process, not just produce outputs? - Is there explicit support for planning and time management on complex tasks? - Are students encouraged to self-assess accurately? - Do students have enough information about criteria to monitor their own progress? ### 9. Situated & Authentic Learning (Brown, Collins & Duguid) Is learning grounded in realistic, relevant contexts that enable transfer? - Is content connected to realistic professional or policy contexts? - Can students see why this matters beyond the classroom? - Does the assignment require applying knowledge to a real or realistic situation? - Is there a bridge from abstract concept to practical application? ### 10. Peer Learning & Collaborative Knowledge Building (Bloom / Vygotsky) Is peer interaction used as a genuine learning mechanism, not just a social activity? - Are collaborative tasks complex enough to require genuine joint reasoning? - Is peer feedback structured so it is substantive and actionable? - Do students have opportunities to teach or explain to each other? - Is the collaborative element designed to produce shared understanding, not just shared output? --- ## Evaluation Response Structure For each material evaluated, structure the response as: 1. **Overall impression** (2–3 sentences) 2. **Per framework:** verdict (Strong / Adequate / Needs work), specific observations with evidence from the material, concrete suggestions 3. **Top 3 priority improvements** — the changes that would make the biggest difference 4. **Rewrite** — produce an improved version of the single weakest element Be direct and specific. Flag what works and what needs improvement. Where you identify a problem, give a concrete suggestion. Do not be vague or generic. **After completing every evaluation** (single material or full course), always end with this offer: > "This evaluation is a solid first pass. If you'd like, I can run a more critical second pass — going through every bullet point of every framework and applying sharper scrutiny. This often surfaces issues the first pass smooths over. Would you like me to do that?" Do not run the second pass automatically. Wait for the user to say yes. --- ## Synthesis Evaluation (whole-course mode only) After evaluating all components individually, evaluate the course as a coherent whole. Identify specifically: 1. **GAPS** — objectives promised but not covered in sessions or tested in assessments 2. **MISALIGNMENTS** — assessments that test something not prepared in sessions; sessions that don't connect to stated objectives 3. **REDUNDANCIES** — content repeated without clear scaffolding purpose 4. **COHERENCE** — does the course tell a clear learning story from start to finish? 5. **OVERALL VERDICT** — one paragraph summary of pedagogical strengths and the single most important structural improvement Name the specific components where you see each problem. --- ## Output Write all evaluation reports as Word documents (.docx) to the `output/` folder. **Also write one structured Markdown summary file per run** (see MD Output section below) — this feeds the future curriculum evaluator. **Filename format:** - Single material Word doc: `_DONE_evaluation_[filename]_[YYYYMMDD].docx` - Whole-course synthesis Word doc: `_DONE_synthesis_[coursename]_[YYYYMMDD].docx` - Structured MD summary (always, every run): `_DONE_summary_[coursename]_[YYYYMMDD].md` **To generate the Word document**, write and run a Python script using python-docx: ```python from docx import Document from docx.shared import Pt from datetime import date doc = Document() # Title doc.add_heading('Course Material Evaluation', 0) # Metadata doc.add_paragraph(f'Source file: [filename]') doc.add_paragraph(f'Material type: [type]') doc.add_paragraph(f'Date: {date.today().strftime("%Y-%m-%d")}') # Sections — use heading styles for structure doc.add_heading('Overall Impression', level=1) doc.add_paragraph('[overall impression text]') # Per framework doc.add_heading('Framework Evaluations', level=1) # ... one heading per framework, verdict + observations + suggestions doc.add_heading('Top 3 Priority Improvements', level=1) # ... numbered list doc.add_heading('Rewrite', level=1) # ... improved version of weakest element doc.save('output/_DONE_evaluation_[filename]_[YYYYMMDD].docx') ``` Each report must include: title, source file, material type, date generated, full evaluation structured as above. --- ## MD Output (Structured Summary for Curriculum Evaluator) At the end of every run — single material or whole course — write one `.md` file to `output/`. This file is the machine-readable layer: it is designed to be aggregated across courses at program level. It is **not** a duplicate of the Word doc; it is a structured summary. **Format:** YAML frontmatter (parseable) + brief prose body (readable by humans and LLMs). ```markdown --- course_code: [e.g. USG6231] course_name: [full name] level: [master / bachelor / professional / mixed] credits: [e.g. 7.5 EC] period: [e.g. 2] coordinator: [name] evaluation_date: [YYYYMMDD] # What was actually submitted and evaluated this run materials_evaluated: - syllabus # include only what was actually read and evaluated - assignment_memorandum - assignment_student_led_class # future options: objectives, session_slides, readings, rubrics # What is missing and not yet evaluated materials_not_yet_evaluated: - session_slides - readings - rubrics - student_evaluations # placeholder for future integration - teacher_reflections # placeholder for future integration assessment_methods: - type: [e.g. written_exam] weight: [e.g. 50%] format: [individual / pairs / groups_2_3 / etc.] # repeat for each assessment component learning_objectives: - "[exact wording of objective 1]" - "[exact wording of objective 2]" # include all objectives verbatim — used by curriculum evaluator to check # progression, overlap, and gaps across the program # Framework verdicts — use exactly: Strong / Adequate / Needs work framework_verdicts: constructive_alignment: [verdict] blooms_taxonomy: [verdict] cognitive_load: [verdict] prior_knowledge: [verdict] active_learning: [verdict] feedback_assessment: [verdict] scaffolding: [verdict] self_regulated_learning: [verdict] situated_learning: [verdict] peer_learning: [verdict] # Aggregated counts for program-level dashboarding verdicts_strong: [count] verdicts_adequate: [count] verdicts_needs_work: [count] top_priorities: - "[priority 1 — one sentence]" - "[priority 2]" - "[priority 3]" key_gaps: - "[gap 1 — one sentence]" key_misalignments: - "[misalignment 1 — one sentence]" # --- FUTURE FIELDS — leave null until data is available --- student_evaluations: evaluated: false source: null # e.g. "Brightspace course evaluation Q3 2026" date_collected: null overall_satisfaction: null # e.g. 3.8/5 response_rate: null key_themes_positive: [] key_themes_critical: [] notable_quotes: [] teacher_reflections: evaluated: false source: null # e.g. "coordinator end-of-block reflection note" date_collected: null key_observations: [] intended_changes: [] session_slides: evaluated: false files_available: [] notes: null # filled when slides are evaluated readings: evaluated: false notes: null # filled when reading list is evaluated --- ## Summary [2–3 sentence overall impression of the course as evaluated so far.] [1 sentence on what is still missing from the evaluation and what it might change.] ## Framework Highlights **Strongest areas:** [list frameworks rated Strong] **Biggest concerns:** [list frameworks rated Needs work, with one-line explanation each] ## Top Priorities [Repeat the top_priorities list here as a numbered list with brief explanation.] ## Notes for Curriculum Evaluator [Any cross-course observations the curriculum evaluator should know: unusual structure, pilot elements, disciplinary dependencies, intended links to other courses in the program.] ``` Write the MD file using the Write tool (not via the Python script). Populate all fields that are known; set unknown future fields to `null` or `false` as shown. Never omit a field — the curriculum evaluator depends on consistent structure across all courses. --- ## File Format Notes - **Word (.docx)** — preferred for text-heavy materials; Claude Code reads these directly - **PDF** — preferred for exported slides; Claude Code extracts text cleanly - **PowerPoint (.pptx)** — supported; speaker notes are extracted alongside slide text - **PNG / images** — avoid; no extractable text