Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent time logging for speech_recognition benchmark #129

Open
jbalma opened this issue Oct 25, 2018 · 0 comments
Open

Inconsistent time logging for speech_recognition benchmark #129

jbalma opened this issue Oct 25, 2018 · 0 comments
Labels
Backlog An issue to be discussed in a future Working Group, but not the immediate next one.

Comments

@jbalma
Copy link

jbalma commented Oct 25, 2018

Currently the directions and scripts for the speech_recognition benchmark assume we time the execution of the run_and_time.sh script like this:

cd pytorch
cd docker
sh build-docker.sh
sh run-dev.sh
time sh run_and_time.sh | tee speech_ds2.out

However, the other benchmarks use the date command (instead of time) from within the run_and_time.sh script itself. For example, the timing for the translation benchmark's run_and_time.sh script contains this timing block:


Start timing

START=$(date +%s)
START_FMT=$(date +%Y-%m-%d\ %r)
echo "STARTING TIMING RUN AT ${START_FMT}"

Run benchmark (training)

SEED=${1:-1}

echo "Running benchmark with seed ${SEED}"
. run_training.sh ${SEED} ${TARGET_UNCASED_BLEU_SCORE}

RET_CODE=$?; if [[ ${RET_CODE} != 0 ]]; then exit ${RET_CODE}; fi

End timing

END=$(date +%s)
END_FMT=$(date +%Y-%m-%d\ %r)

echo "ENDING TIMING RUN AT ${END_FMT}"

Report result

result=$(( ${END} - ${START} ))
result_name="transformer"

We should update the speech_recognition benchmark's run_and_time.sh script to match the others for consistency , but it's probably not critical for the 0.5 release.

-Jacob

@bitfort bitfort added the Next Meeting Item to be discussed in the next Working Group label Oct 29, 2018
@petermattson petermattson added the Backlog An issue to be discussed in a future Working Group, but not the immediate next one. label Oct 29, 2018
@bitfort bitfort removed the Next Meeting Item to be discussed in the next Working Group label Oct 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backlog An issue to be discussed in a future Working Group, but not the immediate next one.
Projects
None yet
Development

No branches or pull requests

3 participants