diff --git a/docs/en_us/dashboard/source/Overview.rst b/docs/en_us/dashboard/source/Overview.rst index aec4f0254..4986136cd 100644 --- a/docs/en_us/dashboard/source/Overview.rst +++ b/docs/en_us/dashboard/source/Overview.rst @@ -6,14 +6,11 @@ Overview EdX Insights makes information about courses available to course team members who have the Course Staff or Instructor role. EdX Insights provides these -course team members with data about student backgrounds and activities -throughout the course. Using edX Insights can help you validate the choices you -made in designing your course. It can also help you re-evaluate choices and -inform efforts to improve your course and the experience of your learners. - -.. ...data about student activity, background, and performance throughout the course. - -.. be sure to cast this as firstly as a tool to provide the "how are my students doing" info that teams want when performance gets added (Mark Rudnick comment) +course team members with data about student activity, background, and +performance throughout the course. Using edX Insights can help you monitor how +students are doing, and validate the choices you made in designing your +course. It can also help you re-evaluate choices and inform efforts to improve +your course and the experience of your learners. Putting the data provided by edX Insights to work involves: @@ -49,6 +46,8 @@ edX Insights presents. * :ref:`Enrollment_Geography` * :ref:`Engagement_Content` + + * :ref:`Performance_Graded` Each of these chapters contains a section for anecdotal "Analytics in Action". These sections showcase how collected data has revealed information about diff --git a/docs/en_us/dashboard/source/Reference.rst b/docs/en_us/dashboard/source/Reference.rst index 6f6467b00..1f66fd805 100644 --- a/docs/en_us/dashboard/source/Reference.rst +++ b/docs/en_us/dashboard/source/Reference.rst @@ -16,6 +16,8 @@ edX Insights are computed. It contains sections for: * :ref:`Location Computations` * :ref:`Engagement Computations` + +* :ref:`Performance Computations` * :ref:`Error Conditions` @@ -39,10 +41,10 @@ All Computations .. spacer -* Metrics for enrollment and engagement do not rely on account activation. A - user who registers an account and enrolls in a course, but does not activate - the user account, is included in all computations as of the date and time of - enrollment. +* Account activation is not considered by the edX Insights computations. EdX + Insights includes a learner who registers a user account and enrolls in a + course, but does not activate the user account, in all computations as of + the date and time of enrollment. .. _Enrollment Computations: @@ -134,7 +136,7 @@ During edX user account registration, students can provide demographic data about themselves. Demographic distributions are computed every day to reflect changes in course enrollment. -Currently, students make selections from dropdown lists on the edx.org and +Currently, students make selections from drop-down lists on the edx.org and edge.edx.org registration pages to provide demographic data. * Students cannot change the selections that they make after registration is @@ -371,6 +373,125 @@ For information about viewing engagement metrics in edX Insights, see * The y-axis shows the number of unique users. + +.. _Performance Computations: + +***************************** +Performance Computations +***************************** + +* Student answer submission data is available only for problems of these + types. + + * Checkboxes (````) + * Dropdown (````) + * Multiple choice (````) + * Numerical input (````) + * Text input (````) + * Math expression input (````) + + For information about the problem types that can be included in courses and + their settings, see `Creating Exercises and Tools`_. + +* For data to be available for a problem, at least one student must + have submitted an answer for that problem after 6 Mar 2014. + +* Computations are updated daily. + +* Only a student's last submission is included in the computation. Any + attempts prior to the last submission are not included. + +* Computations for graded content include only problems for which students can + click **Check** to submit their responses. If students can only save their + responses without submitting them (that is, if the **Maximum Attempts** for + the problem is set to 0), data is not available for student submission + computations. + +* Only problem activity that occurred after 23 Oct 2013 is included. + +**Graded Content Submissions .csv file** + +The .csv file contains a superset of the data that is included in the +Submission Counts chart and report. The .csv file contains the following columns. + +.. list-table:: + :widths: 20 60 + :header-rows: 1 + + * - Column + - Description + * - ``answer_value`` + - The text label of the answer choice for checkboxes, dropdown, and + multiple choice problems. The value entered by the student for text + input, numerical input, and math expression input problems. + + Answer choices selected by at least one student after 23 Oct 2013, but + not selected after 6 Mar 2014, do not include an ``answer_value`` for + checkboxes and multiple choice problems. The ``value_id`` is available + for these problems. + + * - ``consolidated_variant`` + - TRUE if the problem uses the randomization setting in Studio, but there + is a problem with the Python script that randomizes the values. FALSE + if the Python script is randomizing values or if the problem does not + use the randomization setting. + * - ``correct`` + - TRUE if this answer value is correct. FALSE if this answer value is + incorrect. + * - ``count`` + - The number of times that students entered or selected this answer as + their most recent submission for the problem or problem variant. + + The count reflects the entire problem history. If you change a + problem after it is released, it might not be possible for you to + determine which answers were given before and after you made the change. + + * - ``course_id`` + - The identifier for the course run. + * - ``created`` + - The date and time of the computation. + * - ``module_id`` + - The internal identifier for the problem component. + * - ``part_id`` + - For a problem component that contains multiple questions, the internal + identifier for each question. For a problem component that contains a + single question, the internal identifier of that problem. + * - ``problem_display_name`` + - The display name defined for the problem. + * - ``question_text`` + - The accessible label that appears above the answer choices or + the value entry field for the problem. In the Studio Simple Editor, this + text is surrounded by two pairs of angle brackets (>>Question<<). Blank + for questions that do not have an accessible label. + + For problems that use the randomization setting in Studio, if a + particular answer has not been selected since 6 Mar 2014, the + ``question_text`` is blank for that answer. + + * - ``value_id`` + - The internal identifier for the answer choice provided for checkboxes + and multiple choice problems. Blank for dropdown, numerical input, text + input, and math expression input problems. + * - ``variant`` + - For problems that use the randomization setting in Studio, the unique + identifier for a variant of the problem. Blank for problems that have + this setting defined as **Never** (the default). + + +After you download the .csv file, be aware that different spreadsheet +applications can display the same data in different ways. + +* Not all spreadsheet applications interpret and render UTF-8 encoded + characters correctly. + +* Some spreadsheet applications alter data for display purposes, such as + inserting zeroes into numbers expressed as decimals. For example, the + student answer ".5" can be appear in a spreadsheet as "0.5". + +If you notice characters that do not display as expected, or multiple lines +that have the same ``answer_value`` but different counts, try opening the file +in a different spreadsheet application or a text editor. + .. _Error Conditions: ***************** @@ -386,9 +507,9 @@ every course run. In the following situations, data may not be available in edX Insights. -* EdX changed the method used to track student enrollments on 3 December 2013. - For courses created in Studio prior to 4 December 2013, edX Insights reports - enrollment activity beginning with the enrollment count on 11 November 2013. +* EdX changed the method used to track student enrollments on 3 Dec 2013. + For courses created in Studio prior to 4 Dec 2013, edX Insights reports + enrollment activity beginning with the enrollment count on 11 Nov 2013. * For courses with a very small number of enrolled users, such as newly created courses, data for enrollment activity, enrollment geography, or both, may not @@ -398,3 +519,13 @@ In the following situations, data may not be available in edX Insights. courses and courses that finished running early in the history of the edX platform, data for enrollment activity and student engagement may not be available. + +* Charts are not available for problems that use the randomization setting in + Studio. Because such problems can result in numerous possible submission + variants, both correct and incorrect, edX Insights does not attempt to graph + them. The Submissions report and downloadable .csv file are available for + such problems, and include one row for each problem-variant-answer + combination selected by your students. + + +.. _Creating Exercises and Tools: http://edx.readthedocs.org/projects/edx-partner-course-staff/en/latest/exercises_tools/index.html \ No newline at end of file diff --git a/docs/en_us/dashboard/source/conf.py b/docs/en_us/dashboard/source/conf.py index f42b4c469..ac4540e76 100644 --- a/docs/en_us/dashboard/source/conf.py +++ b/docs/en_us/dashboard/source/conf.py @@ -38,7 +38,7 @@ # General information about the project. project = u'Using edX Insights' -copyright = u'2014, edX' +copyright = u'2015, edX' # The short X.Y version. version = '' diff --git a/docs/en_us/dashboard/source/enrollment/Demographics_Age.rst b/docs/en_us/dashboard/source/enrollment/Demographics_Age.rst index 0e5b54238..b15e1213d 100644 --- a/docs/en_us/dashboard/source/enrollment/Demographics_Age.rst +++ b/docs/en_us/dashboard/source/enrollment/Demographics_Age.rst @@ -1,7 +1,7 @@ .. _Demographics_Age: ############################# -Enrollment Demographics: Age +Age Demographics ############################# How old are my students? Awareness of the ages reported by your students can diff --git a/docs/en_us/dashboard/source/enrollment/Demographics_Education.rst b/docs/en_us/dashboard/source/enrollment/Demographics_Education.rst index 93c43dc97..8cc4b1a70 100644 --- a/docs/en_us/dashboard/source/enrollment/Demographics_Education.rst +++ b/docs/en_us/dashboard/source/enrollment/Demographics_Education.rst @@ -1,7 +1,7 @@ .. _Demographics_Education: ################################### -Enrollment Demographics: Education +Education Demographics ################################### What educational background do my students have? Evaluating the stages of diff --git a/docs/en_us/dashboard/source/enrollment/Demographics_Gender.rst b/docs/en_us/dashboard/source/enrollment/Demographics_Gender.rst index 2abb77914..da8df7bc3 100644 --- a/docs/en_us/dashboard/source/enrollment/Demographics_Gender.rst +++ b/docs/en_us/dashboard/source/enrollment/Demographics_Gender.rst @@ -1,7 +1,7 @@ .. _Demographics_Gender: ################################ -Enrollment Demographics: Gender +Gender Demographics ################################ What is the gender balance in my course? Knowing the male-female ratio in your @@ -26,10 +26,10 @@ computations, see Self-Reported Student Gender Chart ====================================== -The bars on this chart represent the percentage of enrolled learners who -reported a gender of **Female**, **Male**, or **Other**. Moving your cursor -over the chart shows the percentage for that gender, calculated to one decimal -place. +The bars on this chart represent the most recently calculated percentage of +enrolled learners who reported a gender of **Female**, **Male**, or **Other**. +Moving your cursor over the chart shows the percentage for that gender, +calculated to one decimal place. Student gender data is also available for review in tabular format and can be downloaded. @@ -46,10 +46,11 @@ Examples of this chart for two different courses follow. .. MITx/8.MReVx/2T2014/enrollment/demographics/gender/ -These charts are both for science courses. In both courses, the median -student age is 25. Each of these course teams might use this information as a -starting point for an investigation into how students learn about their course -offering and make the decision to enroll in the course. +Both of these charts are for science courses, and those courses have the same +median age, 25. Each of these course teams might use information about the +percentages of enrolled men and women as a starting point for an investigation +into how students learn about their course offering and make the decision to +enroll in the course. See the :ref:`Reference` for a detailed description of how student gender values are computed. diff --git a/docs/en_us/dashboard/source/change_log.rst b/docs/en_us/dashboard/source/front_matter/change_log.rst similarity index 65% rename from docs/en_us/dashboard/source/change_log.rst rename to docs/en_us/dashboard/source/front_matter/change_log.rst index 4880b34f4..73521c08e 100644 --- a/docs/en_us/dashboard/source/change_log.rst +++ b/docs/en_us/dashboard/source/front_matter/change_log.rst @@ -2,22 +2,40 @@ Change Log ############ +**** +2015 +**** + +.. list-table:: + :widths: 10 70 + :header-rows: 1 + + * - Date + - Change + * - 11 Mar 2015 + - Added :ref:`Performance_Graded`. + + +**** +2014 +**** + .. list-table:: :widths: 10 70 :header-rows: 1 * - Date - Change - * - 12/9/14 + * - 9 Dec 2014 - Updated :ref:`Enrollment_Activity` to include enrollment tracks. * - - Expanded the section on possible :ref:`error conditions`. - * - 11/24/14 + * - 24 Nov 2014 - Enrollment history is now available for all courses. See :ref:`Enrollment_Activity` and :ref:`Reference`. - * - 11/05/14 + * - 5 Nov 2014 - Added :ref:`Enrollment_Demographics`. - * - 09/30/14 + * - 30 Sep 2014 - Initial release. diff --git a/docs/en_us/dashboard/source/front_matter/index.rst b/docs/en_us/dashboard/source/front_matter/index.rst index bb47b807e..e8fc3a51d 100644 --- a/docs/en_us/dashboard/source/front_matter/index.rst +++ b/docs/en_us/dashboard/source/front_matter/index.rst @@ -6,6 +6,7 @@ For Your Information :numbered: :maxdepth: 2 + change_log read_me preface browsers \ No newline at end of file diff --git a/docs/en_us/dashboard/source/images/answer_dist_easy.png b/docs/en_us/dashboard/source/images/answer_dist_easy.png new file mode 100644 index 000000000..b585eb77b Binary files /dev/null and b/docs/en_us/dashboard/source/images/answer_dist_easy.png differ diff --git a/docs/en_us/dashboard/source/images/answer_dist_hard.png b/docs/en_us/dashboard/source/images/answer_dist_hard.png new file mode 100644 index 000000000..f86e22acd Binary files /dev/null and b/docs/en_us/dashboard/source/images/answer_dist_hard.png differ diff --git a/docs/en_us/dashboard/source/images/assignment.png b/docs/en_us/dashboard/source/images/assignment.png new file mode 100644 index 000000000..425cc6bf4 Binary files /dev/null and b/docs/en_us/dashboard/source/images/assignment.png differ diff --git a/docs/en_us/dashboard/source/images/assignment_name_issue.png b/docs/en_us/dashboard/source/images/assignment_name_issue.png new file mode 100644 index 000000000..8892de028 Binary files /dev/null and b/docs/en_us/dashboard/source/images/assignment_name_issue.png differ diff --git a/docs/en_us/dashboard/source/images/assignment_type.png b/docs/en_us/dashboard/source/images/assignment_type.png new file mode 100644 index 000000000..fad56f313 Binary files /dev/null and b/docs/en_us/dashboard/source/images/assignment_type.png differ diff --git a/docs/en_us/dashboard/source/images/problem_answers.png b/docs/en_us/dashboard/source/images/problem_answers.png new file mode 100644 index 000000000..c68315af8 Binary files /dev/null and b/docs/en_us/dashboard/source/images/problem_answers.png differ diff --git a/docs/en_us/dashboard/source/images/problem_display_name.png b/docs/en_us/dashboard/source/images/problem_display_name.png new file mode 100644 index 000000000..2e62eab74 Binary files /dev/null and b/docs/en_us/dashboard/source/images/problem_display_name.png differ diff --git a/docs/en_us/dashboard/source/images/problem_missing_question.png b/docs/en_us/dashboard/source/images/problem_missing_question.png new file mode 100644 index 000000000..7ea6297db Binary files /dev/null and b/docs/en_us/dashboard/source/images/problem_missing_question.png differ diff --git a/docs/en_us/dashboard/source/images/problem_part_menu.png b/docs/en_us/dashboard/source/images/problem_part_menu.png new file mode 100644 index 000000000..18991eb9f Binary files /dev/null and b/docs/en_us/dashboard/source/images/problem_part_menu.png differ diff --git a/docs/en_us/dashboard/source/images/problems.png b/docs/en_us/dashboard/source/images/problems.png new file mode 100644 index 000000000..530589c33 Binary files /dev/null and b/docs/en_us/dashboard/source/images/problems.png differ diff --git a/docs/en_us/dashboard/source/index.rst b/docs/en_us/dashboard/source/index.rst index e9d0af54c..143bd692c 100755 --- a/docs/en_us/dashboard/source/index.rst +++ b/docs/en_us/dashboard/source/index.rst @@ -10,9 +10,9 @@ Using edX Insights :numbered: :maxdepth: 2 - change_log front_matter/index Overview enrollment/index engagement/index + performance/index Reference diff --git a/docs/en_us/dashboard/source/performance/Performance_Answers.rst b/docs/en_us/dashboard/source/performance/Performance_Answers.rst new file mode 100644 index 000000000..ed3f72ad7 --- /dev/null +++ b/docs/en_us/dashboard/source/performance/Performance_Answers.rst @@ -0,0 +1,344 @@ +.. _Performance_Graded: + +############################# +Graded Content Submissions +############################# + +How are students answering questions? In edX Insights, graded content +submissions show you the responses that students submit for graded problems +and help you evaluate what they find difficult. To illustrate, this chapter +presents the Analytics in Action section first. + +After the Analytics in Action examples, this chapter describes how you +:ref:`select an assignment type, assignment, and problem` and then :ref:`review student +answers`. + +******************************************************* +Analytics in Action: Interpreting Graded Submissions +******************************************************* + +A review of the distribution of student answer submissions for a graded +problem can lead to discoveries about your students and about your course. + +* You can assess how difficult the problem is for students to answer correctly. + +* You can detect common mistakes. + +* You can understand student misconceptions. + +* You can find errors in problem components. + +In addition, you can use the aggregate data in the graphs of your assignment +types and assignments to identify where students are submitting more incorrect +answers. + +=============================================== +Researching Unexpected Difficulties +=============================================== + +For problem types that provide both the question and a set of possible answers +(checkboxes, dropdown, and multiple choice), submission data helps you assess +how difficult it is for students to identify and submit the correct answer. +With the submissions chart, you can visually contrast the number of students +who select incorrect answers with the number who answer correctly. + +If the proportion of students who answer the problem incorrectly surprises +you, research can reveal a variety of causes. Your investigation might begin +with some of these questions. + +* Is the text of the question and of its possible answers clear? Has it been + translated accurately? + +* Does the course outline include relevant course content before the problem, + or after it? + +* Are all of the course prerequisites presented to potential students? + +* Does the problem rely on student access to a video or textbook? Are there + transcripts for the videos, and can the textbook files be read by a screen + reader? + +* Are students relying on conventional wisdom to answer the question instead + of newly acquired knowledge? + +The results of your investigation can guide changes to future course runs. + +================================= +Investigating Similar Responses +================================= + +For open-ended problem types that provide only the question (numerical, text, +and math expression input), submission data can help you identify similar +responses. In the Submission Counts report you have access to every answer +submitted by a student. The chart, however, presents only the 12 most +frequently submitted responses. Your initial investigation into how students +answer a question can begin with this set. + +For example, the edX Demo course includes a text input problem that has a +correct answer of "Antarctica". The problem is set up to recognize other +variations in capitalization for this English spelling as correct. + +.. image:: ../images/problem_answers.png + :alt: This chart has one bar for each answer submitted for a problem or part, + and the bars are colored differently for correct and incorrect + submissions + +.. edX/DemoX.1/2014/performance/graded_content/assignments/i4x%3A//edX/DemoX.1/sequential/9c1aacbb2795470e8473b059b59c3344/problems/i4x%3A//edX/DemoX.1/problem/466f474fa4d045a8b7bde1b911e095ca/parts/i4x-edX-DemoX_1-problem-466f474fa4d045a8b7bde1b911e095ca_2_1/answer_distribution/ + +When you review the submissions chart for the problem, you see that the two +most frequently submitted answers are both marked correct: Antarctica and +antarctica. You also note that several misspelled variations, including +"Antarctic" and "Antartica", are marked incorrect. + +A review of the Submissions Count report reveals several more variations, +including "antartika", "Antartide", and "el continente Antártico". You realize +that these answers also indicate the continent of Antarctica, but in languages +other than English. Seeing answers such as these in the report might reassure +you that more students understand the question and the relevant course +material than is indicated by the correct answer count. You might then decide +to reconfigure the problem so that correct answers in other languages also +evaluate as correct. Alternatively, you might decide to revise the question to +specify that answers be given in English only. + +===================================================== +Reviewing Answers to Test and Complete Course Setup +===================================================== + +Before the release date of each section, you encourage your beta testers to +answer every question and to submit both correct and incorrect answers. +You then use edX Insights to review the answers that your testers submit for +each problem. You verify that each problem is set up as you intend, and +correct any oversights before students can encounter them. + +In this way, you can use edX Insights to validate the grading configuration, +and to proofread the display names, accessible labels, and text that you have +provided for the graded assignment types, assignments, problems, questions, +and answers. + +For example, the first time you use edX Insights to look at student +performance, you choose the "Homework" assignment type. In the chart of the +homework assignments, you see the nine assignments that you expect. However, +you notice that for your first two homework assignments, you forgot to include +the distinguishing number after the name "Problem Set". + +.. image:: ../images/assignment_name_issue.png + :alt: This chart shows two bars with the same x-axis label, Problem Set + +.. MITx/6.832x/3T2014/performance/graded_content/Homework/ + +In another example, you use edX Insights to check the answers that your beta +testers submitted for one of the questions in a quiz. You notice that the chart +for this question does not have a title. For this problem component, you +neglected to identify the question with an accessible label. + +.. image:: ../images/problem_missing_question.png + :alt: In this chart of the answers to a problem the title bar above the chart + is blank + +.. UTAustinX/UT.8.02x/1T2015/performance/graded_content/assignments/i4x%3A//UTAustinX/UT.8.02x/sequential/d8dd4296856a4745b1afd7d7c4aef7f6/problems/i4x%3A//UTAustinX/UT.8.02x/problem/802cd3308245482b870ca7f36a3cd282/parts/i4x-UTAustinX-UT_8_02x-problem-802cd3308245482b870ca7f36a3cd282_2_1/answer_distribution/ + +You can select **View Live** to see what the problem looks like in the LMS, +and from there select **View Unit in Studio** to add the accessible label to +the problem. + +In this last example, when you see the chart for an assignment you realize +that you did not change the default display name, "Multiple Choice", for any +of the problems that it includes. + +.. image:: ../images/problem_display_name.png + :alt: In this chart of the problems in an assignment, every problem has the + same name + +.. MITx/9.01x/3T2014/performance/graded_content/assignments/i4x%3A//MITx/9.01x/sequential/19aa8209e5474d52aab965f0f1649270/ + +Because students see problem display names in the LMS, you might decide to go +back to Studio and provide identifying display names for the problems before +you publish the subsection. + +.. _Selecting the Assignment Type, Assignment, and Problem: + +******************************************************* +Selecting the Assignment Type, Assignment, and Problem +******************************************************* + +To access data about the answers that students submit for a graded problem +component, you make these selections. + +Step 1: Select a graded course assignment type. + +Step 2: Select an assignment. + +Step 3: Select a problem. + +EdX Insights provides aggregated data for each selection that you make. + +================================== +Step 1: Select an Assignment Type +================================== + +After you select **Performance** and **Graded Content**, edX Insights displays +the assignment types that make up the grading configuration of the course. You +use the drop-down **Select Assignment Type** menu or click an item in the +grading configuration to select the assignment type to investigate. + +.. image:: ../images/assignment_type.png + :alt: Single bar, thermometer-style chart that shows the percentage that each + assignment type contributes to the course grade + +.. edX/DemoX.1/2014/performance/graded_content/ + +For information about defining course assignment types, see `Establishing a +Grading Policy`_. + +============================== +Step 2: Select an Assignment +============================== + +After you select one of the course assignment types, edX Insights displays a +stacked bar chart that summarizes student performance on each assignment of +that type. + +.. image:: ../images/assignment.png + :alt: This chart has one bar for each assignment of the selected type, with + different colors stacked in the bar for correct and incorrect submissions + +.. edX/DemoX.1/2014/performance/graded_content/Homework/ + +The Assignment Submissions report on this page provides the number of +problems in each assignment and the aggregate number of correct and incorrect +submissions received for each one. + +For information about identifying the graded subsections in a course, see `Set +the Assignment Type and Due Date for a Subsection`_. + +You use the drop-down **Select {Assignement Type}** menu or click a bar in the +chart to select the assignment you want to examine further. + +========================= +Step 3: Select a Problem +========================= + +After you select an assignment, edX Insights displays a stacked bar chart that +summarizes student performance on each problem in that assignment. In this +example from the edX Demo course, the selected homework assignment includes +just one problem. + +.. image:: ../images/problems.png + :alt: This chart has one bar for each problem in the selected assignment, + with different colors stacked in the bar for correct and incorrect submissions + +.. edX/DemoX.1/2014/performance/graded_content/assignments/i4x%3A//edX/DemoX.1/sequential/9c1aacbb2795470e8473b059b59c3344/ + +The Problem Submissions report on this page provides the number of questions +in each problem and the aggregate number of correct and incorrect submissions +received for each one. + +For information about adding a unit to a subsection, see `Create a Unit`_. + +You use the drop-down **Select Problem** menu or click a bar in the chart to +select the problem that you want to examine further. + +If the problem that you select includes more than one part (or question), the +first part appears. To select a different part, you use the drop-down +**Submissions for Part {number}** menu. In the Demo course example, the +selected homework problem has three parts. + +.. image:: ../images/problem_part_menu.png + :alt: The Submissions for Part menu with three options for different problem + parts + +.. edX/DemoX.1/2014/performance/graded_content/assignments/i4x%3A//edX/DemoX.1/sequential/9c1aacbb2795470e8473b059b59c3344/problems/i4x%3A//edX/DemoX.1/problem/466f474fa4d045a8b7bde1b911e095ca/parts/i4x-edX-DemoX_1-problem-466f474fa4d045a8b7bde1b911e095ca_2_1/answer_distribution/ + +.. _Review Answers to Graded Problems: + +************************************* +Review Answers to Graded Problems +************************************* + +After you select a problem or problem part, edX Insights displays submission +data in a bar chart and a report that you can view or download. Descriptions +of the chart and report follow. For detailed information about the +computations, see :ref:`Reference`. + +.. note:: Problems that use the randomization setting in Studio result in + many possible submission variants, both correct and incorrect. As a result, + edX Insights does not attempt to present a chart of the responses submitted + for these problems. You can download the Submissions Counts report to + analyze the answers that are of interest. + +================== +Submissions Chart +================== + +The bars on this chart represent the number of enrolled students who submitted +a particular answer to a question in a problem component. The x-axis includes +the most frequently submitted answers, up to a maximum of 12. Due to space +limitations, the answer text that is used to label the x-axis might be +truncated. Moving your cursor over each bar shows a longer version of the +answer. + +To review the problem component in the LMS as a student sees it, select **View +Live** and then at the top of the page use the **View this course as** option +to select Student. The LMS displays the page that contains this problem in +Student View. For more information, see `View Your Live Course`_. + +All submitted answers, and complete answer values, are available for review in +tabular format at the bottom of the page and can also be downloaded. + +Examples of the graded content submissions chart follow. In the first example, +most students selected the correct answer for a multiple choice problem. + +.. image:: ../images/answer_dist_easy.png + :alt: A bar chart showing that most students selected the correct answer + out of four possible choices + +.. edX/DemoX.1/2014/performance/graded_content/assignments/i4x%3A//edX/DemoX.1/sequential/9c1aacbb2795470e8473b059b59c3344/problems/i4x%3A//edX/DemoX.1/problem/466f474fa4d045a8b7bde1b911e095ca/parts/i4x-edX-DemoX_1-problem-466f474fa4d045a8b7bde1b911e095ca_3_1/answer_distribution/ + +The second example shows the graph of the top 12 answers submitted for a +numerical input problem. Most students left the answer for this question +blank, or "(empty)", which was marked incorrect. Other answers that students +submitted, both correct and incorrect, are also graphed. The +Submission Counts report includes a row for every submitted answer. + +.. image:: ../images/answer_dist_hard.png + :alt: A bar chart showing that most students submitted incorrect answers + with bars for the 12 most frequent answers + +.. MITx/EECS.6.002x/3T2013/performance/graded_content/assignments/i4x%3A//MITx/EECS.6.002x/sequential/Homework_10/problems/i4x%3A//MITx/EECS.6.002x/problem/H10P3_An_L_Network/parts/i4x-MITx-EECS_6_002x-problem-H10P3_An_L_Network_3_1/answer_distribution/ + +For more information, see the :ref:`Reference`. + +========================= +Submission Counts Report +========================= + +A report with a row for each problem-answer combination submitted by your +students is available for review or download. The report columns show each +submitted answer, identify the correct answer or answers, and provide the +number of students who submitted that answer. + +To download the Submission Counts report in a comma-separated value +file, select **Download CSV**. + +The report includes one row for each problem-answer combination submitted by a +student. For example, consider a dropdown problem that has five possible +answers. The report or file contains up to five rows, one for each answer +submitted by at least one student. + +For problems that use the randomization feature in Studio, the report has one +row for each problem-variant-answer combination selected by at least one +student. For more information about problem randomization, see +`Randomization`_. + +See the :ref:`Reference` for a detailed description of each column. + +.. _Randomization: http://edx.readthedocs.org/projects/edx-partner-course-staff/en/latest/creating_content/create_problem.html#randomization + +.. _View Your Live Course: http://edx.readthedocs.org/projects/edx-partner-course-staff/en/latest/developing_course/testing_courseware.html?highlight=view%20live#view-your-live-course + +.. _Establishing a Grading Policy: http://edx.readthedocs.org/projects/edx-partner-course-staff/en/latest/building_course/establish_grading_policy.html + +.. _Set the Assignment Type and Due Date for a Subsection: http://edx.readthedocs.org/projects/edx-partner-course-staff/en/latest/developing_course/course_subsections.html#set-the-assignment-type-and-due-date-for-a-subsection + +.. _Create a Unit: http://edx.readthedocs.org/projects/edx-partner-course-staff/en/latest/developing_course/course_units.html#create-a-unit \ No newline at end of file diff --git a/docs/en_us/dashboard/source/performance/index.rst b/docs/en_us/dashboard/source/performance/index.rst new file mode 100644 index 000000000..ffd39e2cf --- /dev/null +++ b/docs/en_us/dashboard/source/performance/index.rst @@ -0,0 +1,39 @@ +.. _Performance Index: + +############################# +Student Performance +############################# + +To assess how students are doing in your course, make a selection from the +**Performance** menu. Student performance data is available in edX Insights +for problem components of these types: + +* Checkboxes + +* Dropdown + +* Multiple choice + +* Numerical input + +* Text input + +* Math expression input + +When you select **Graded Content**, EdX Insights displays the grading +configuration for your course. You can review aggregated performance data for +assignments and problems, and then the answers that students submitted for +assigned questions. + +For performance data to be available for a problem, at least one student must +have submitted an answer for that problem. + +Student submissions are updated every day. The computations use the last +answer submitted by each student and received through the end of the previous +day (23:59 UTC). + +.. toctree:: + :maxdepth: 1 + + Performance_Answers +