Developer Documentation

See Release Notes

  • Bug fixes for general core bugs in 3.11.x will end 14 Nov 2022 (12 months plus 6 months extension).
  • Bug fixes for security issues in 3.11.x will end 13 Nov 2023 (18 months plus 12 months extension).
  • PHP version: minimum PHP 7.3.0 Note: minimum PHP version has increased since Moodle 3.10. PHP 7.4.x is supported too.

Differences Between: [Versions 310 and 311] [Versions 311 and 400] [Versions 311 and 401] [Versions 311 and 402] [Versions 39 and 311]

This file contains helper classes for testing the question engine.

Copyright: 2009 The Open University
License: GNU GPL v3 or later
File Size: 1409 lines (56 kb)
Included or required: 37 times
Referenced: 52 times
Includes or requires: 1 file

Defines 18 classes

testable_question_attempt:: (4 methods):

testable_question_engine_unit_of_work:: (8 methods):

question_test_helper:: (1 method):

test_question_maker:: (14 methods):

testing_db_record_builder:: (1 method):

data_loading_method_test_base:: (1 method):

question_testcase:: (2 methods):

question_contains_tag_with_contents:: (1 method):

question_check_specified_fields_expectation:: (1 method):

question_contains_select_expectation:: (1 method):

question_does_not_contain_tag_with_attributes:: (1 method):

question_contains_tag_with_attribute:: (1 method):

question_contains_tag_with_attributes:: (1 method):

question_pattern_expectation:: (1 method):

question_no_pattern_expectation:: (1 method):

qbehaviour_walkthrough_test_base:: (69 methods):

question_test_recordset:: (7 methods):

testable_core_question_renderer:: (1 method):

Class: testable_question_attempt  - X-Ref

Makes some protected methods of question_attempt public to facilitate testing.

add_step(question_attempt_step $step)   X-Ref
No description

set_min_fraction($fraction)   X-Ref
No description

set_max_fraction($fraction)   X-Ref
No description

set_behaviour(question_behaviour $behaviour)   X-Ref
No description

Class: testable_question_engine_unit_of_work  - X-Ref

Test subclass to allow access to some protected data so that the correct
behaviour can be verified.

get_modified()   X-Ref
No description

get_attempts_added()   X-Ref
No description

get_attempts_modified()   X-Ref
No description

get_steps_added()   X-Ref
No description

get_steps_modified()   X-Ref
No description

get_steps_deleted()   X-Ref
No description

get_metadata_added()   X-Ref
No description

get_metadata_modified()   X-Ref
No description

Class: question_test_helper  - X-Ref

Base class for question type test helpers.

get_question_editing_form($cat, $questiondata)   X-Ref
Set up a form to create a question in $cat. This method also sets cat and contextid on $questiondata object.

param: object $cat the category
param: object $questiondata form initialisation requires question data.
return: moodleform

Class: test_question_maker  - X-Ref

This class creates questions of various types, which can then be used when

get_a_qa($question, $maxmark = 3)   X-Ref
Just make a question_attempt at a question. Useful for unit tests that
need to pass a $qa to methods that call format_text. Probably not safe
to use for anything beyond that.

param: question_definition $question a question.
param: number $maxmark the max mark to set.
return: question_attempt the question attempt.

initialise_a_question($q)   X-Ref
Initialise the common fields of a question of any type.

initialise_question_data($qdata)   X-Ref
No description

get_test_helper($qtype)   X-Ref
Get the test helper class for a particular question type.

param: $qtype the question type name, e.g. 'multichoice'.
return: question_test_helper the test helper class.

call_question_helper_method($methodtemplate, $qtype, $which = null)   X-Ref
Call a method on a qtype_{$qtype}_test_helper class and return the result.

param: string $methodtemplate e.g. 'make_{qtype}_question_{which}';
param: string $qtype the question type to get a test question for.
param: string $which one of the names returned by the get_test_questions
param: unknown_type $which

make_question($qtype, $which = null)   X-Ref
Question types can provide a number of test question defintions.
They do this by creating a qtype_{$qtype}_test_helper class that extends
question_test_helper. The get_test_questions method returns the list of
test questions available for this question type.

param: string $qtype the question type to get a test question for.
param: string $which one of the names returned by the get_test_questions
return: question_definition the requested question object.

get_question_data($qtype, $which = null)   X-Ref
Like {@link make_question()} but returns the datastructure from
get_question_options instead of the question_definition object.

param: string $qtype the question type to get a test question for.
param: string $which one of the names returned by the get_test_questions
return: stdClass the requested question object.

get_question_form_data($qtype, $which = null)   X-Ref
Like {@link make_question()} but returns the data what would be saved from
the question editing form instead of the question_definition object.

param: string $qtype the question type to get a test question for.
param: string $which one of the names returned by the get_test_questions
return: stdClass the requested question object.

make_a_multichoice_single_question()   X-Ref
Makes a multichoice question with choices 'A', 'B' and 'C' shuffled. 'A'
is correct, defaultmark 1.

return: qtype_multichoice_single_question

make_a_multichoice_multi_question()   X-Ref
Makes a multichoice question with choices 'A', 'B', 'C' and 'D' shuffled.
'A' and 'C' is correct, defaultmark 1.

return: qtype_multichoice_multi_question

make_a_matching_question()   X-Ref
Makes a matching question to classify 'Dog', 'Frog', 'Toad' and 'Cat' as
'Mammal', 'Amphibian' or 'Insect'.
defaultmark 1. Stems are shuffled by default.

return: qtype_match_question

make_an_essay_question()   X-Ref
Makes a truefalse question with correct ansewer true, defaultmark 1.

return: qtype_essay_question

set_standard_combined_feedback_fields($q)   X-Ref
Add some standard overall feedback to a question. You need to use these
specific feedback strings for the corresponding contains_..._feedback
methods in {@link qbehaviour_walkthrough_test_base} to works.

param: question_definition|stdClass $q the question to add the feedback to.

set_standard_combined_feedback_form_data($form)   X-Ref
Add some standard overall feedback to a question's form data.

Class: testing_db_record_builder  - X-Ref

Helper for tests that need to simulate records loaded from the database.

build_db_records(array $table)   X-Ref
No description

Class: data_loading_method_test_base  - X-Ref

Helper base class for tests that need to simulate records loaded from the

build_db_records(array $table)   X-Ref
No description

Class: question_testcase  - X-Ref

assert($expectation, $compare, $notused = '')   X-Ref
Tolerance accepted in some unit tests when float operations are involved.

assert_select_options($expectation, $html)   X-Ref
Use this function rather than assert when checking the value of options within a select element.

param: question_contains_select_expectation $expectation The select expectation class
param: string $html The rendered output to check against

Class: question_contains_select_expectation  - X-Ref

__construct($name, $choices, $selected = null, $enabled = null, $message = '')   X-Ref
No description

Class: question_pattern_expectation  - X-Ref

__construct($pattern, $message = '')   X-Ref
No description

Class: qbehaviour_walkthrough_test_base  - X-Ref

Helper base class for question walk-through tests.

The purpose of tests that use this base class is to simulate the entire
interaction of a student making an attempt at a question. Therefore,
these are not really unit tests. They would more accurately be described
as integration tests. However, whether they are unit tests or not,
it works well to implement them in PHPUnit.

Historically, tests like this were made because Moodle did not have anything
like Behat for end-to-end testing. Even though we do now have Behat, it makes
sense to keep these walk-through tests. They run massively faster than Behat
tests, which gives you a much faster feedback loop while doing development.
They also make it quite easy to test things like regrading the attempt after
the question has been edited, which would be possible but very fiddly in Behat.

Ideally, the full set of tests for the question class of a question type would be:

1. A lot of unit tests for each qtype_myqtype_question class method
like grade_response, is_complete_response, is_same_response, ...

2. Several of these walk-through tests, to test the end-to-end interaction
of a student with a question, for example with different behaviours.

3. Just one Behat test, using question preview, to verify that everything
is plugged together correctly and works when used through the UI.

What one would expect to see in one of these walk-through tests is:

// 1. Set up a question: $q.

// 2. A call to $this->start_attempt_at_question($q, ...); with the relevant options.

// 3. Some number of calls to $this->process_submission passing an array of simulated
//    POST data that matches what would be sent back be submitting a form that contains
//    the form fields that are output by rendering the question. This is like clicking
//    the 'Check' button in a question, or navigating to the next page in a quiz.

// 4. A call to $this->finish(); which is the equivalent of clicking
//    'Submit all and finish' in the quiz.

// 5. After each of steps 2-4 above, one would expect to see a certain amount of
//    validation of the state of the question and how the question is rendered,
//    using methods like $this->check_current_state(), $this->check_current_output, etc.

The best way to work out how to write tests like this is probably to look at
some examples in other question types or question behaviours.

In writing these tests, it is worth noting the following points:

a) The easiest mistake to make is at step 3. You need to ensure that your
simulated post data actually matches what gets sent back when the
question is submitted in the browser. Try checking it against the
HTTP POST requests you see in your browser when the question is submitted.
Some question types have a $q->prepare_simulated_post_data() method that
can help with this.

b) In the past, tests like these used to contain even more repetitive code,
and so they were re-factored to add the helper methods like
start_attempt_at_question, process_submission, finish. That change had
good effects, like reducing duplicate code. However, there were down-sides.
The extra layers of indirection hide what is going on, which means these
tests are harder to understand until you know what the helpers are doing.
If you want an interesting exercise, take one of the walk-through tests,
and inline all the helpers. This might be a good way to understand more about
the question engine API. However, having made the everything-inlined code
and learned from the process, you should then just throw it away.

c) The way check_current_output works is weird. When these tests were first written
Moodle used SimpleTest for unit tests and check_current_output os written in a
style that made sense there. When we moved to PHPUnit, a quick and dirty
conversion was done. That was a pragmatic move at the time, and we just have
to live with the result. Sorry. (And: don't copy that style for new things.)

setUp()   X-Ref

tearDown()   X-Ref
No description

start_attempt_at_question($question, $preferredbehaviour,$maxmark = null, $variant = 1)   X-Ref
No description

response_data_to_post($data)   X-Ref
Convert an array of data destined for one question to the equivalent POST data.

param: array $data the data for the quetsion.
return: array the complete post data.

process_submission($data)   X-Ref
No description

process_autosave($data)   X-Ref
No description

finish()   X-Ref
No description

manual_grade($comment, $mark, $commentformat = null)   X-Ref
No description

save_quba(moodle_database $db = null)   X-Ref
No description

load_quba(moodle_database $db = null)   X-Ref
No description

delete_quba()   X-Ref
No description

check_comment($comment, $commentformat)   X-Ref
Asserts if the manual comment for the question is equal to the provided arguments.

param: $comment Comment text
param: $commentformat Comment format

check_current_state($state)   X-Ref
No description

check_current_mark($mark)   X-Ref
No description

render()   X-Ref
Generate the HTML rendering of the question in its current state in
$this->currentoutput so that it can be verified.

check_output_contains_text_input($name, $value = null, $enabled = true)   X-Ref
No description

check_output_contains_text_input_with_class($name, $class = null)   X-Ref
No description

check_output_does_not_contain_text_input_with_class($name, $class = null)   X-Ref
No description

check_output_contains_hidden_input($name, $value)   X-Ref
No description

check_output_contains($string)   X-Ref
No description

check_output_does_not_contain($string)   X-Ref
No description

check_output_contains_lang_string($identifier, $component = '', $a = null)   X-Ref
No description

get_tag_matcher($tag, $attributes)   X-Ref
No description

check_current_output()   X-Ref

param: $condition one or more Expectations. (users varargs).

check_output_contains_selectoptions(...$expectations)   X-Ref
Use this function rather than check_current_output for select expectations where
checking the value of the options is required. check_current_output only checks
that the right number of options are available.

param: question_contains_select_expectation $expectations One or more expectations.

get_question_attempt()   X-Ref
No description

get_step_count()   X-Ref
No description

check_step_count($expectednumsteps)   X-Ref
No description

get_step($stepnum)   X-Ref
No description

get_contains_question_text_expectation($question)   X-Ref
No description

get_contains_general_feedback_expectation($question)   X-Ref
No description

get_does_not_contain_correctness_expectation()   X-Ref
No description

get_contains_correct_expectation()   X-Ref
No description

get_contains_partcorrect_expectation()   X-Ref
No description

get_contains_incorrect_expectation()   X-Ref
No description

get_contains_standard_correct_combined_feedback_expectation()   X-Ref
No description

get_contains_standard_partiallycorrect_combined_feedback_expectation()   X-Ref
No description

get_contains_standard_incorrect_combined_feedback_expectation()   X-Ref
No description

get_does_not_contain_feedback_expectation()   X-Ref
No description

get_does_not_contain_num_parts_correct()   X-Ref
No description

get_contains_num_parts_correct($num)   X-Ref
No description

get_does_not_contain_specific_feedback_expectation()   X-Ref
No description

get_contains_validation_error_expectation()   X-Ref
No description

get_does_not_contain_validation_error_expectation()   X-Ref
No description

get_contains_mark_summary($mark)   X-Ref
No description

get_contains_marked_out_of_summary()   X-Ref
No description

get_does_not_contain_mark_summary()   X-Ref
No description

get_contains_checkbox_expectation($baseattr, $enabled, $checked)   X-Ref
No description

get_contains_mc_checkbox_expectation($index, $enabled = null,$checked = null)   X-Ref
No description

get_contains_radio_expectation($baseattr, $enabled, $checked)   X-Ref
No description

get_contains_mc_radio_expectation($index, $enabled = null, $checked = null)   X-Ref
No description

get_contains_hidden_expectation($name, $value = null)   X-Ref
No description

get_does_not_contain_hidden_expectation($name, $value = null)   X-Ref
No description

get_contains_tf_true_radio_expectation($enabled = null, $checked = null)   X-Ref
No description

get_contains_tf_false_radio_expectation($enabled = null, $checked = null)   X-Ref
No description

get_contains_cbm_radio_expectation($certainty, $enabled = null,$checked = null)   X-Ref
No description

get_contains_button_expectation($name, $value = null, $enabled = null)   X-Ref
No description

get_contains_submit_button_expectation($enabled = null)   X-Ref
Returns an epectation that a string contains the HTML of a button with
name {question-attempt prefix}-submit, and eiter enabled or not.

param: bool $enabled if not null, check the enabled/disabled state of the button. True = enabled.
return: question_contains_tag_with_attributes an expectation for use with check_current_output.

get_does_not_contain_submit_button_expectation()   X-Ref
Returns an epectation that a string does not contain the HTML of a button with
name {question-attempt prefix}-submit.

return: question_contains_tag_with_attributes an expectation for use with check_current_output.

get_tries_remaining_expectation($n)   X-Ref
No description

get_invalid_answer_expectation()   X-Ref
No description

get_contains_try_again_button_expectation($enabled = null)   X-Ref
No description

get_does_not_contain_try_again_button_expectation()   X-Ref
No description

get_contains_select_expectation($name, $choices,$selected = null, $enabled = null)   X-Ref
No description

get_mc_right_answer_index($mc)   X-Ref
No description

get_no_hint_visible_expectation()   X-Ref
No description

get_contains_hint_expectation($hinttext)   X-Ref
No description

get_contains_corruption_notification()   X-Ref
Returns an expectation that a string contains a corrupted question notification.

return: question_pattern_expectation an expectation for use with check_current_output.

get_contains_corrupted_subquestion_message()   X-Ref
Returns an expectation that a string contains a corrupted subquestion message.

return: question_pattern_expectation an expectation for use with check_current_output.

Class: question_test_recordset  - X-Ref

Simple class that implements the {@link moodle_recordset} API based on an
array of test data.

See the {@link question_attempt_step_db_test} class in
question/engine/tests/testquestionattemptstep.php for an example of how
this is used.

__construct(array $table)   X-Ref

param: $table as for {@link testing_db_record_builder::build_db_records()}

__destruct()   X-Ref
No description

current()   X-Ref
No description

key()   X-Ref
No description

next()   X-Ref
No description

valid()   X-Ref
No description

close()   X-Ref
No description

Class: testable_core_question_renderer  - X-Ref

Helper class for tests that help to test core_question_renderer.

number($number)   X-Ref
Test the private number function.

param: null|string $number
return: HTML