CodeRunner Question Generator (Minimal Pipeline)

Generate Moodle CodeRunner questions. Two modes depending on your environment.

Mode Detection

Before doing anything else, determine your mode:

Run this command:

echo "jobe-check"

If the command succeeds (you can execute bash): use FULL PIPELINE MODE (Sections 3-9). Jobe computes all expected output via curl. This is the preferred mode.
If the command fails (no bash access, e.g. Claude.ai or ChatGPT): use FALLBACK MODE (see box below). You must mentally trace expected output yourself.

FALLBACK MODE (no bash access)

When you cannot run bash commands (Claude.ai, ChatGPT, API without tools):

Generate the same JSON structure as Section 6, but you MUST include "expected" in each test case by carefully tracing the solution through the test code.

Trace character-by-character. Pay special attention to: trailing spaces, newlines, floating-point precision, array formatting.

Apply all per-language rules from Section 2 and auto-fix rules from Section 8 manually.

Wrap in XML using the template from Section 7.

Add this warning to the user: "These questions were generated WITHOUT Jobe server validation. Before using them, import into Moodle and ensure Validate on save is enabled (it is by default). Moodle will run the solution against all test cases on import and flag any mismatches."

All other sections (question design, type rules, XML template, checklist) still apply.

Full Pipeline mode is strongly preferred. It eliminates the #1 source of errors (hallucinated expected output, which causes 35% of all failures).

1. Question Type Reference

Choose the type FIRST -- it determines everything else.

Type	Student Writes	Test Mechanism	Stdin?	Reliability
`java_class`	A class	Test code calls methods	No	HIGH
`java_method`	Static method(s) ONLY	Test code calls methods	No	HIGH
`java_program`	Full program with `main`	Empty test, stdin input	Yes	LOW -- EOF issues
`python3`	Function(s) or class	Test code calls with `print()`	No	HIGH
`python3_w_input`	Full program	Empty test, stdin input	Yes	MEDIUM -- EOF issues
`c_function`	Function(s) + headers	Test code has `main()`	No	HIGH
`c_program`	Full program with `main`	Empty test, stdin input	Yes	MEDIUM
`cpp_function`	Function(s) + headers	Test code has `main()`	No	MEDIUM -- Werror
`cpp_program`	Full program with `main`	Empty test, stdin input	Yes	MEDIUM -- Werror
`nodejs`	Function(s)	Test code uses `console.log()`	No	HIGH

Default choice: Prefer java_class > java_method > java_program. Prefer function types over program types -- they avoid stdin/EOF problems entirely.

2. Per-Language Rules

These rules are derived from 1000+ validated questions. Every rule exists because questions failed without it.

Java

Rule	Applies To	What To Do
Scanner EOF guard	`java_program`	ALWAYS call `hasNextLine()`/`hasNextInt()` before EVERY `Scanner` read. #1 Java failure cause.
Class name	`java_program`	Class MUST be named `Answer`
Method-only solution	`java_method`	Solution MUST be ONLY the method(s) -- NO class wrapper, NO main(), NO import statements. The buildSource step wraps it in `public class Answer { <methods> main() { testcode } }`. Including `public class` or `main()` causes compilation errors.
Class qualifier in tests	`java_class`	Test code runs in a separate `__Tester__` class. Call static methods as `ClassName.method()`. Create objects as `ClassName obj = new ClassName(); obj.method()`.

Python

Rule	Applies To	What To Do
EOF guard	`python3_w_input`	Use `import sys; data = sys.stdin.read().strip()` with `if data:` guard. Never bare `input()`.
Function-only solution	`python3`	Solution is function/class definitions only. Test code calls them with `print()`.

C

Rule	Applies To	What To Do
`-Werror`	ALL C types	Jobe compiles with `-Werror`. ALL warnings are fatal. No unused variables, no implicit declarations.
`#include` in answer	`c_function`	ALL headers (`<stdio.h>`, `<ctype.h>`, `<string.h>`, `<math.h>`, `<stdlib.h>`) MUST be in the solution, not just test code.
`scanf` return check	`c_program`	Always: `if (scanf("%d", &n) == 1) { ... }`. Unchecked scanf on empty stdin = uninitialized variable.
Test code wrapper	`c_function`	Test code MUST include `#include <stdio.h>` and `int main(void) { ... return 0; }`
Array printing	ALL C types	Use `if (i) printf(" "); printf("%d", arr[i]);` -- NOT `printf("%d ", arr[i])` (trailing space fails).
Function-only solution	`c_function`	Solution is function(s) with #include headers only. NO main(). Test code provides main().

C++

Rule	Applies To	What To Do
`-Werror`	ALL C++ types	Same as C -- all warnings fatal.
`size_t` for `.size()`	ALL C++ types	`for (size_t i = 0; i < vec.size(); i++)` -- `int` vs `size_t` is `-Werror=sign-compare` fatal.
Member init order	`cpp_function`	Declare class members in SAME order as constructor initializer list. `-Werror=reorder` is fatal.
`#include` in answer	`cpp_function`	Include `<vector>`, `<string>`, `<algorithm>`, `<sstream>` etc. in the solution, not just test code.
Test code wrapper	`cpp_function`	Test code MUST include `#include <iostream>`, `using namespace std;`, and `int main() { ... }`
Array printing	ALL C++ types	`if (i) cout << " "; cout << v[i];` -- no trailing space.

Node.js

Rule	Applies To	What To Do
Deterministic output	`nodejs`	Use `JSON.stringify(result)` for objects/arrays. Do NOT use `JSON.stringify(obj, replacer)`.
No npm packages	`nodejs`	Only Node.js built-in features. No require() of external modules.

3. Jobe Server Configuration

Servers

Server	URL	Notes
Primary	`https://jobe2.cosc.canterbury.ac.nz`	Canterbury University public Jobe. No API key needed.
Backup	`http://3.252.250.94`	EC2 backup with CORS proxy.

Try the primary server first. If it fails (timeout or HTTP error), fall back to the backup.

Language IDs for Jobe

Question Types	Jobe `language_id`
`python3`, `python3_w_input`	`python3`
`java_class`, `java_method`, `java_program`	`java`
`c_function`, `c_program`	`c`
`cpp_function`, `cpp_program`	`cpp`
`nodejs`	`nodejs`

Calling Jobe via curl

Use this exact curl pattern for every Jobe call:

curl -s -w "\n%{http_code}" \
  https://jobe2.cosc.canterbury.ac.nz/jobe/index.php/restapi/runs \
  -H "Content-Type: application/json" \
  -d '{
    "run_spec": {
      "language_id": "LANGUAGE_ID",
      "sourcecode": "FULL_SOURCE_CODE",
      "input": "STDIN_OR_EMPTY",
      "sourcefilename": "FILENAME",
      "parameters": {"cputime": 10, "memorylimit": 256000}
    }
  }'

Response format:

{"outcome": 15, "cmpinfo": "", "stdout": "42\n", "stderr": ""}

Outcome codes:

15 = success. Use stdout as the expected output.
11 = compilation error. Read cmpinfo for the error.
12 = runtime error. Read stderr for the error.
13 = time limit exceeded.
17 = memory limit exceeded.
Any other outcome = failure.

Important: Stdin MUST end with \n. If the input does not end with a newline, append one before sending.

Important: Expected output: strip the trailing newline from stdout before using it as expected output. CodeRunner strips trailing newlines before comparing, so the <expected> tag should NO

moodle-coderunner