Question writing guide for the impatient

In PL, a "question" really means "question generator", since implicitly, every question (even those with "zero code" below) has the built-in potential for some randomization.

At baseline, a question is one screen in a web browser where a student answers one or more things.

Each question lives in its own subdirectory under questions/ in your course repo. A hierarchy of directories is fine, but all questions must be somewhere under questions/, and that directory must contain all and only valid questions. A valid question is a subdirectory containing at least the two files info.json and question.html; it may also contain other files.

Roughly, there are 3 "tiers" of content (questions) you can author:

I) zero-code: 100% of the question lives in question.html, which consists of HTML plus markup for pre-existing PL "elements". If you look at the element docs, you'll see that many elements have some simple randomization built-in. for example, the pl-checkbox ("select all that apply") element lets you specify a very large number of both correct and incorrect (distractor) choices, specify whether some of those specific choices MUST appear but let PL choose the rest, dictate the minimum/maximum number of correct/incorrect choices that should be displayed, etc. Built-in elements also know how to grade themselves: eg the pl-checkbox element allows various schemes for computing partial credit.

II) moderate-code: part of the responsibility for generating the question resides in a python funciton generate() inside a file server.py. in this function, you pick appropriate (and maybe constraiend) random values that will be used in the question. then in the HTML file, you substitute the actual values into what the student is given. finally, you can provide a grading function that computes the correct answer based on the values.

III) bespoke: you write custom code in the HTML file and custom code in server.py so that anything that can be displayed/rendered in a browser can be shown as the question, and any user interaction supported by the browser (drag/drop, drawing, etc) can become part of the student response. a special case of this is you encapsulate this logic into a new Element that others can then re-use in mode I or II.

For levels II and III, you need a server.py file in the question's directory.

server.py

The generate(data) entry point is passed a nested hash data. You should fill out the following and return the hash as the return value of this function:

• data["params"]: a hash of things that need to be available for rendering question.html and/or available at grading time. The key data["params"]["foo"] can be referenced in question.html as {{ params.foo }}.

• data["correct_answers"]: for question elements that have a notion of a correct answer, like pl-number-input and pl-string-input, a hash (dict) whose keys are referenced in the elements. For example, if you have a <pl-string-input answer-name="foo">, you should set data["correct_answers"]["foo"] to the correct answer for that string input.

The grade(data) entry point gets handed the same data hash but now data["submitted_answers"] contains what the student submitted. Its keys are the same as the corresponding correct_answers keys, so in our example above, data["submitted_answers"]["foo"] is whatever the student actually typed in that box.

All elements that support "correct answer" functionality are graded for you. For elements whose contents you want to grade yourself, use data["submitted_answers"] to extract the answers, and fill in data["partial_scores"][key] with the partial-credit score (0.0 to 1.0) for getting that answer right. The total of all partial scores for a question should be 1.0 (100%).

Debugging server.py is a pain; I have not been able to get the interactive debugger (pdb.set_trace()) to work, probably because at runtime the grading process is not connected to an interactive terminal. The best thing I have found is to import sys at the top, then use sys.stderr.write(string) to log stuff. When running PL in development mode, the console log will be captured and shown to you when you submit a question for grading. Another possibility is to import logging, then use logging.warn("something") and raise an exception. This will cause a runtime error when the question is submitted for grading, but the embedded terminal display in the PrairieLearn window will show the result of the logging call.

question.html

Regular HTML markup, interspersed with PL elements.

The two main required parts are <pl-question-panel>...</pl-question-panel>, which shows the question and collects responses, and <pl-answer-panel>...</pl-answer-panel>, which shows the correct answer after student submits their answer (if the assessment is so configured).

Bootstrap is loaded, so use its standard classes for formatting things. Don't use hardwired style attributes (<div bgcolor="red">). Don't use tables to lay things out; use Bootstrap's row and col classes. Don't use inline CSS stylesheets unless there is absolutely no other choice; you'll almost certainly make either an aesthetically poor choice or one that harms a11y.

Mustache can be used to dereference params. If something was set up in server.py as data["params"]["foo"] then it can be interpolated in HTML with {{params.foo}}. The namespace of params is flat.

You can iterate over a collection of objects. For example, if you setup data["params"]["anArray"] as follows:

data["params"]["anArray"] = [
  {"first": "armando", "last": "fox"},
  {"first": "dan", "last": "garcia"}
]

then in your template you can loop over it like this:

<ul>
  {{#params.anArray}}
    <li>Full name: {{last}}, {{first}} </li>
  {{/params.anArray}}
</ul>

See the Mustache docs for more behaviors.