Skip to content

Commit

Permalink
webeval: text update and more clarification
Browse files Browse the repository at this point in the history
Signed-off-by: Pavel Pisa <[email protected]>
  • Loading branch information
ppisa committed Sep 25, 2024
1 parent 05d89da commit a884556
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 4 deletions.
4 changes: 2 additions & 2 deletions content/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ This is a joint effort of [Faculty of Electrical Engineering](https://fel.cvut.c
- [FIT: BI-APS Architectures of Computer Systems](/courses/fit/bi-aps/)
- [FEL: B4M35PAP Advanced Computer Architectures](/courses/fel/b4m35pap/)

# Online Tools
# Online Tools and Training

- [WebEvaluator for QtRvSim](/online-tools/webeval/)
- [WebEvaluator for QtRvSim](/online-tools/webeval/) ([direct link](https://eval.comparch.edu.cvut.cz) to the web interface)

# Presentations

Expand Down
10 changes: 8 additions & 2 deletions content/online-tools/webeval/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,15 @@
title: "Web Evaluator for QtRvSim"
---

A web application for submission and evaluation of student (and also of the general public) solutions of [bonus RISC-V tasks](https://cw.fel.cvut.cz/wiki/courses/b35apo/en/homeworks/bonus/start) in [b35apo](../../courses/fel/b35apo/). The application takes a code snippet written in RISC-V assembly or C and runs it in QtRvSim using a custom wrapper written in Python. During the evaluation the correctness of the solution is checked by easily configurable testcases, all written in .toml format. After the evaluation is done, a task is scored based on the runtime of the program in cycles. This allows a bit of competition between the students.
A web application for submission and evaluation of student (and also of the general public) solutions of [bonus RISC-V tasks](https://cw.fel.cvut.cz/wiki/courses/b35apo/en/homeworks/bonus/start) initially prepared for the [B35APO](../../courses/fel/b35apo/) bachelor Computer Architectures course. The code snippet written in RISC-V assembly or C is entered through the web interface, and it is compiled by [QtRvSim](https://github.com/cvut/qtrvsim) internal assembler or GCC and then run in [QtRvSim](https://github.com/cvut/qtrvsim) using a wrapper application written in Python. During the evaluation, the correctness of the solution is checked by easily configurable test cases, all written in .toml format. After the evaluation, a task is scored based on the program's runtime in CPU cycles or other metrics (i.e., cache misses). The soring allows a bit of competition between the students in addition to bonus points received for the correct results. The last and the best-scored student submissions are kept in the database, the same as the obtained scores, listed in global and per event/class/institution scoreboards.

In the current state, the evaluator is able to check the content of the registers and the state of memory at set addresses. A comparison of uart is also possible. Each of the testcases can be configured to be either private or public (if set to private the log does not show the trace of an error), with a separate testcase which is set to be a kind of a benchmark - to allow fair competition between different submissions. The system also allows for a custom Makefile to be included, with additional files to be present during compile time.
In the current state, the evaluator can check the registers' content and the memory region's final state at set addresses. Injection of data into memory region specified by address or symbol defined in the tested program is also possible (sequence to sort, which is unknown to the student, for example). A comparison of UART output to reference text is also possible. Each of the test cases can be configured to be either private or public (if set to private, the log does not show the trace of an error), with a separate test case, which is set to be a kind of benchmark - to allow fair competition between different submissions. The system also allows for a custom Makefile to be included, with additional files to be present during compile time.

The only hash of the e-mail address provided during registration is stored for comparison with the hash of the e-mail address used for password recovery. The address is used to send a password reset e-mail during the password recovery, but it is never stored in the database. We hope no GDPR measures apply to us or other organizations interested in running our evaluation system on their server.

The code can be analyzed and developed locally or [online](https://comparch.edu.cvut.cz/qtrvsim/app) in [QtRvSim](https://comparch.edu.cvut.cz/qtrvsim/app). The cache, pipeline, and other parameters must be set according to the task description in the Web Evaluator.

The tasks are sorted from introductory ones (i.e., simple value addition solved by three instructions inserted before ebreak) to more advanced sorting algorithm implementation in RISC-V assembly or C programs for processing data incoming from UART and printing the results similarly.

## Links

Expand Down

0 comments on commit a884556

Please sign in to comment.