Developer Documentation

  • Bug fixes for general core bugs in 3.11.x will end 14 Nov 2022 (12 months plus 6 months extension).
  • Bug fixes for security issues in 3.11.x will end 13 Nov 2023 (18 months plus 12 months extension).
  • PHP version: minimum PHP 7.3.0 Note: minimum PHP version has increased since Moodle 3.10. PHP 7.4.x is supported too.
  • Differences Between: [Versions 310 and 311] [Versions 311 and 400] [Versions 37 and 311] [Versions 38 and 311] [Versions 39 and 311]

    This file contains helper classes for testing the question engine.

    Copyright: 2009 The Open University
    License: GNU GPL v3 or later
    File Size: 1409 lines (56 kb)
    Included or required: 37 times
    Referenced: 8 times
    Includes or requires: 1 file

    Defines 18 classes

    testable_question_attempt:: (4 methods):

    testable_question_engine_unit_of_work:: (8 methods):

    question_test_helper:: (1 method):

    test_question_maker:: (14 methods):

    testing_db_record_builder:: (1 method):

    data_loading_method_test_base:: (1 method):

    question_testcase:: (2 methods):

    question_contains_tag_with_contents:: (1 method):

    question_check_specified_fields_expectation:: (1 method):

    question_contains_select_expectation:: (1 method):

    question_does_not_contain_tag_with_attributes:: (1 method):

    question_contains_tag_with_attribute:: (1 method):

    question_contains_tag_with_attributes:: (1 method):

    question_pattern_expectation:: (1 method):

    question_no_pattern_expectation:: (1 method):

    qbehaviour_walkthrough_test_base:: (69 methods):

    question_test_recordset:: (7 methods):

    testable_core_question_renderer:: (1 method):

    Class: testable_question_attempt  - X-Ref

    Makes some protected methods of question_attempt public to facilitate testing.

    add_step(question_attempt_step $step)   X-Ref
    No description

    set_min_fraction($fraction)   X-Ref
    No description

    set_max_fraction($fraction)   X-Ref
    No description

    set_behaviour(question_behaviour $behaviour)   X-Ref
    No description

    Class: testable_question_engine_unit_of_work  - X-Ref

    Test subclass to allow access to some protected data so that the correct
    behaviour can be verified.

    get_modified()   X-Ref
    No description

    get_attempts_added()   X-Ref
    No description

    get_attempts_modified()   X-Ref
    No description

    get_steps_added()   X-Ref
    No description

    get_steps_modified()   X-Ref
    No description

    get_steps_deleted()   X-Ref
    No description

    get_metadata_added()   X-Ref
    No description

    get_metadata_modified()   X-Ref
    No description

    Class: question_test_helper  - X-Ref

    Base class for question type test helpers.

    get_question_editing_form($cat, $questiondata)   X-Ref
    Set up a form to create a question in $cat. This method also sets cat and contextid on $questiondata object.

    return: moodleform
    param: object $cat the category
    param: object $questiondata form initialisation requires question data.

    Class: test_question_maker  - X-Ref

    This class creates questions of various types, which can then be used when

    get_a_qa($question, $maxmark = 3)   X-Ref
    Just make a question_attempt at a question. Useful for unit tests that
    need to pass a $qa to methods that call format_text. Probably not safe
    to use for anything beyond that.

    return: question_attempt the question attempt.
    param: question_definition $question a question.
    param: number $maxmark the max mark to set.

    initialise_a_question($q)   X-Ref
    Initialise the common fields of a question of any type.

    initialise_question_data($qdata)   X-Ref
    No description

    get_test_helper($qtype)   X-Ref
    Get the test helper class for a particular question type.

    return: question_test_helper the test helper class.
    param: $qtype the question type name, e.g. 'multichoice'.

    call_question_helper_method($methodtemplate, $qtype, $which = null)   X-Ref
    Call a method on a qtype_{$qtype}_test_helper class and return the result.

    param: string $methodtemplate e.g. 'make_{qtype}_question_{which}';
    param: string $qtype the question type to get a test question for.
    param: string $which one of the names returned by the get_test_questions
    param: unknown_type $which

    make_question($qtype, $which = null)   X-Ref
    Question types can provide a number of test question defintions.
    They do this by creating a qtype_{$qtype}_test_helper class that extends
    question_test_helper. The get_test_questions method returns the list of
    test questions available for this question type.

    return: question_definition the requested question object.
    param: string $qtype the question type to get a test question for.
    param: string $which one of the names returned by the get_test_questions

    get_question_data($qtype, $which = null)   X-Ref
    Like {@link make_question()} but returns the datastructure from
    get_question_options instead of the question_definition object.

    return: stdClass the requested question object.
    param: string $qtype the question type to get a test question for.
    param: string $which one of the names returned by the get_test_questions

    get_question_form_data($qtype, $which = null)   X-Ref
    Like {@link make_question()} but returns the data what would be saved from
    the question editing form instead of the question_definition object.

    return: stdClass the requested question object.
    param: string $qtype the question type to get a test question for.
    param: string $which one of the names returned by the get_test_questions

    make_a_multichoice_single_question()   X-Ref
    Makes a multichoice question with choices 'A', 'B' and 'C' shuffled. 'A'
    is correct, defaultmark 1.

    return: qtype_multichoice_single_question

    make_a_multichoice_multi_question()   X-Ref
    Makes a multichoice question with choices 'A', 'B', 'C' and 'D' shuffled.
    'A' and 'C' is correct, defaultmark 1.

    return: qtype_multichoice_multi_question

    make_a_matching_question()   X-Ref
    Makes a matching question to classify 'Dog', 'Frog', 'Toad' and 'Cat' as
    'Mammal', 'Amphibian' or 'Insect'.
    defaultmark 1. Stems are shuffled by default.

    return: qtype_match_question

    make_an_essay_question()   X-Ref
    Makes a truefalse question with correct ansewer true, defaultmark 1.

    return: qtype_essay_question

    set_standard_combined_feedback_fields($q)   X-Ref
    Add some standard overall feedback to a question. You need to use these
    specific feedback strings for the corresponding contains_..._feedback
    methods in {@link qbehaviour_walkthrough_test_base} to works.

    param: question_definition|stdClass $q the question to add the feedback to.

    set_standard_combined_feedback_form_data($form)   X-Ref
    Add some standard overall feedback to a question's form data.

    Class: testing_db_record_builder  - X-Ref

    Helper for tests that need to simulate records loaded from the database.

    build_db_records(array $table)   X-Ref
    No description

    Class: data_loading_method_test_base  - X-Ref

    Helper base class for tests that need to simulate records loaded from the

    build_db_records(array $table)   X-Ref
    No description

    Class: question_testcase  - X-Ref

    assert($expectation, $compare, $notused = '')   X-Ref
    Tolerance accepted in some unit tests when float operations are involved.

    assert_select_options($expectation, $html)   X-Ref
    Use this function rather than assert when checking the value of options within a select element.

    param: question_contains_select_expectation $expectation The select expectation class
    param: string $html The rendered output to check against

    Class: question_contains_select_expectation  - X-Ref

    __construct($name, $choices, $selected = null, $enabled = null, $message = '')   X-Ref
    No description

    Class: question_pattern_expectation  - X-Ref

    __construct($pattern, $message = '')   X-Ref
    No description

    Class: qbehaviour_walkthrough_test_base  - X-Ref

    Helper base class for question walk-through tests.

    The purpose of tests that use this base class is to simulate the entire
    interaction of a student making an attempt at a question. Therefore,
    these are not really unit tests. They would more accurately be described
    as integration tests. However, whether they are unit tests or not,
    it works well to implement them in PHPUnit.

    Historically, tests like this were made because Moodle did not have anything
    like Behat for end-to-end testing. Even though we do now have Behat, it makes
    sense to keep these walk-through tests. They run massively faster than Behat
    tests, which gives you a much faster feedback loop while doing development.
    They also make it quite easy to test things like regrading the attempt after
    the question has been edited, which would be possible but very fiddly in Behat.

    Ideally, the full set of tests for the question class of a question type would be:

    1. A lot of unit tests for each qtype_myqtype_question class method
    like grade_response, is_complete_response, is_same_response, ...

    2. Several of these walk-through tests, to test the end-to-end interaction
    of a student with a question, for example with different behaviours.

    3. Just one Behat test, using question preview, to verify that everything
    is plugged together correctly and works when used through the UI.

    What one would expect to see in one of these walk-through tests is:

    // 1. Set up a question: $q.

    // 2. A call to $this->start_attempt_at_question($q, ...); with the relevant options.

    // 3. Some number of calls to $this->process_submission passing an array of simulated
    //    POST data that matches what would be sent back be submitting a form that contains
    //    the form fields that are output by rendering the question. This is like clicking
    //    the 'Check' button in a question, or navigating to the next page in a quiz.

    // 4. A call to $this->finish(); which is the equivalent of clicking
    //    'Submit all and finish' in the quiz.

    // 5. After each of steps 2-4 above, one would expect to see a certain amount of
    //    validation of the state of the question and how the question is rendered,
    //    using methods like $this->check_current_state(), $this->check_current_output, etc.

    The best way to work out how to write tests like this is probably to look at
    some examples in other question types or question behaviours.

    In writing these tests, it is worth noting the following points:

    a) The easiest mistake to make is at step 3. You need to ensure that your
    simulated post data actually matches what gets sent back when the
    question is submitted in the browser. Try checking it against the
    HTTP POST requests you see in your browser when the question is submitted.
    Some question types have a $q->prepare_simulated_post_data() method that
    can help with this.

    b) In the past, tests like these used to contain even more repetitive code,
    and so they were re-factored to add the helper methods like
    start_attempt_at_question, process_submission, finish. That change had
    good effects, like reducing duplicate code. However, there were down-sides.
    The extra layers of indirection hide what is going on, which means these
    tests are harder to understand until you know what the helpers are doing.
    If you want an interesting exercise, take one of the walk-through tests,
    and inline all the helpers. This might be a good way to understand more about
    the question engine API. However, having made the everything-inlined code
    and learned from the process, you should then just throw it away.

    c) The way check_current_output works is weird. When these tests were first written
    Moodle used SimpleTest for unit tests and check_current_output os written in a
    style that made sense there. When we moved to PHPUnit, a quick and dirty
    conversion was done. That was a pragmatic move at the time, and we just have
    to live with the result. Sorry. (And: don't copy that style for new things.)

    setUp()   X-Ref

    tearDown()   X-Ref
    No description

    start_attempt_at_question($question, $preferredbehaviour,$maxmark = null, $variant = 1)   X-Ref
    No description

    response_data_to_post($data)   X-Ref
    Convert an array of data destined for one question to the equivalent POST data.

    return: array the complete post data.
    param: array $data the data for the quetsion.

    process_submission($data)   X-Ref
    No description

    process_autosave($data)   X-Ref
    No description

    finish()   X-Ref
    No description

    manual_grade($comment, $mark, $commentformat = null)   X-Ref
    No description

    save_quba(moodle_database $db = null)   X-Ref
    No description

    load_quba(moodle_database $db = null)   X-Ref
    No description

    delete_quba()   X-Ref
    No description

    check_comment($comment, $commentformat)   X-Ref
    Asserts if the manual comment for the question is equal to the provided arguments.

    param: $comment Comment text
    param: $commentformat Comment format

    check_current_state($state)   X-Ref
    No description

    check_current_mark($mark)   X-Ref
    No description

    render()   X-Ref
    Generate the HTML rendering of the question in its current state in
    $this->currentoutput so that it can be verified.

    check_output_contains_text_input($name, $value = null, $enabled = true)   X-Ref
    No description

    check_output_contains_text_input_with_class($name, $class = null)   X-Ref
    No description

    check_output_does_not_contain_text_input_with_class($name, $class = null)   X-Ref
    No description

    check_output_contains_hidden_input($name, $value)   X-Ref
    No description

    check_output_contains($string)   X-Ref
    No description

    check_output_does_not_contain($string)   X-Ref
    No description

    check_output_contains_lang_string($identifier, $component = '', $a = null)   X-Ref
    No description

    get_tag_matcher($tag, $attributes)   X-Ref
    No description

    check_current_output()   X-Ref

    param: $condition one or more Expectations. (users varargs).

    check_output_contains_selectoptions(...$expectations)   X-Ref
    Use this function rather than check_current_output for select expectations where
    checking the value of the options is required. check_current_output only checks
    that the right number of options are available.

    param: question_contains_select_expectation $expectations One or more expectations.

    get_question_attempt()   X-Ref
    No description

    get_step_count()   X-Ref
    No description

    check_step_count($expectednumsteps)   X-Ref
    No description

    get_step($stepnum)   X-Ref
    No description

    get_contains_question_text_expectation($question)   X-Ref
    No description

    get_contains_general_feedback_expectation($question)   X-Ref
    No description

    get_does_not_contain_correctness_expectation()   X-Ref
    No description

    get_contains_correct_expectation()   X-Ref
    No description

    get_contains_partcorrect_expectation()   X-Ref
    No description

    get_contains_incorrect_expectation()   X-Ref
    No description

    get_contains_standard_correct_combined_feedback_expectation()   X-Ref
    No description

    get_contains_standard_partiallycorrect_combined_feedback_expectation()   X-Ref
    No description

    get_contains_standard_incorrect_combined_feedback_expectation()   X-Ref
    No description

    get_does_not_contain_feedback_expectation()   X-Ref
    No description

    get_does_not_contain_num_parts_correct()   X-Ref
    No description

    get_contains_num_parts_correct($num)   X-Ref
    No description

    get_does_not_contain_specific_feedback_expectation()   X-Ref
    No description

    get_contains_validation_error_expectation()   X-Ref
    No description

    get_does_not_contain_validation_error_expectation()   X-Ref
    No description

    get_contains_mark_summary($mark)   X-Ref
    No description

    get_contains_marked_out_of_summary()   X-Ref
    No description

    get_does_not_contain_mark_summary()   X-Ref
    No description

    get_contains_checkbox_expectation($baseattr, $enabled, $checked)   X-Ref
    No description

    get_contains_mc_checkbox_expectation($index, $enabled = null,$checked = null)   X-Ref
    No description

    get_contains_radio_expectation($baseattr, $enabled, $checked)   X-Ref
    No description

    get_contains_mc_radio_expectation($index, $enabled = null, $checked = null)   X-Ref
    No description

    get_contains_hidden_expectation($name, $value = null)   X-Ref
    No description

    get_does_not_contain_hidden_expectation($name, $value = null)   X-Ref
    No description

    get_contains_tf_true_radio_expectation($enabled = null, $checked = null)   X-Ref
    No description

    get_contains_tf_false_radio_expectation($enabled = null, $checked = null)   X-Ref
    No description

    get_contains_cbm_radio_expectation($certainty, $enabled = null,$checked = null)   X-Ref
    No description

    get_contains_button_expectation($name, $value = null, $enabled = null)   X-Ref
    No description

    get_contains_submit_button_expectation($enabled = null)   X-Ref
    Returns an epectation that a string contains the HTML of a button with
    name {question-attempt prefix}-submit, and eiter enabled or not.

    return: question_contains_tag_with_attributes an expectation for use with check_current_output.
    param: bool $enabled if not null, check the enabled/disabled state of the button. True = enabled.

    get_does_not_contain_submit_button_expectation()   X-Ref
    Returns an epectation that a string does not contain the HTML of a button with
    name {question-attempt prefix}-submit.

    return: question_contains_tag_with_attributes an expectation for use with check_current_output.

    get_tries_remaining_expectation($n)   X-Ref
    No description

    get_invalid_answer_expectation()   X-Ref
    No description

    get_contains_try_again_button_expectation($enabled = null)   X-Ref
    No description

    get_does_not_contain_try_again_button_expectation()   X-Ref
    No description

    get_contains_select_expectation($name, $choices,$selected = null, $enabled = null)   X-Ref
    No description

    get_mc_right_answer_index($mc)   X-Ref
    No description

    get_no_hint_visible_expectation()   X-Ref
    No description

    get_contains_hint_expectation($hinttext)   X-Ref
    No description

    get_contains_corruption_notification()   X-Ref
    Returns an expectation that a string contains a corrupted question notification.

    return: question_pattern_expectation an expectation for use with check_current_output.

    get_contains_corrupted_subquestion_message()   X-Ref
    Returns an expectation that a string contains a corrupted subquestion message.

    return: question_pattern_expectation an expectation for use with check_current_output.

    Class: question_test_recordset  - X-Ref

    Simple class that implements the {@link moodle_recordset} API based on an
    array of test data.

    See the {@link question_attempt_step_db_test} class in
    question/engine/tests/testquestionattemptstep.php for an example of how
    this is used.

    __construct(array $table)   X-Ref

    param: $table as for {@link testing_db_record_builder::build_db_records()}

    __destruct()   X-Ref
    No description

    current()   X-Ref
    No description

    key()   X-Ref
    No description

    next()   X-Ref
    No description

    valid()   X-Ref
    No description

    close()   X-Ref
    No description

    Class: testable_core_question_renderer  - X-Ref

    Helper class for tests that help to test core_question_renderer.

    number($number)   X-Ref
    Test the private number function.

    return: HTML
    param: null|string $number