feat(docker): re-organize the autoware docker containers #4072

oguzkaganozt · 2023-12-20T11:42:49Z

Description

Re-organize the Autoware Docker Containers for:

Size reduction for all (devel, prebuilt, runtime) container images
Introduce ccache into prebuilt image Autoware full build now takes only 8 mins from 1.5 hours(with CUDA)
Use a custom base image based on ros2:humble rather than using plain Ubuntu:22.04
Provide a runtime image rather than having a prebuilt image
Provide helper scripts to run Autoware on docker environment
Modify Ansible scripts to enable modular images with distinct dependencies

Related PRs:

Tests performed

Built containers from scratch using new dockerfile and ansible scripts
Tested runtime image on x86 and arm64 machines

Notes for reviewers

Some changes might interfere with some Github Actions

Interface changes

No interface changes

Effects on system behavior

No system effect, CI-CD and container runtime improvements made

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.
The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

The PR follows the pull request guidelines.
The PR has been properly tested.
The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

There are no open discussions or they are tracked via tickets.
The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

.github/workflows/docker-build-and-push-main.yaml

ansible/roles/cuda/tasks/main.yaml

ansible/roles/devel/README.md

ansible/roles/gdown/README.md

ansible/roles/geographiclib/tasks/main.yaml

ansible/roles/nvidia_docker/tasks/main.yaml

docker/autoware-openadk/Dockerfile

docker/autoware-openadk/etc/ros_entrypoint.sh

docker/run.sh

docker/autoware-openadk/docker-bake.hcl

ansible/playbooks/openadk.yaml

mitsudome-r · 2024-01-22T07:46:45Z

@oguzkaganozt Could you also create PR to autoware-documentation to make sure that install instructions for Docker is valid with this PR?

Requested changes have been made

ansible/roles/devel/tasks/main.yaml

setup-dev-env.sh

mitsudome-r · 2024-02-15T14:06:38Z

The code looks okay to me.

I have noticed that you have tested the action and it seems passing: https://github.com/autowarefoundation/autoware/actions/runs/7905813792

We can test with autoware.universe and other repositories to see if they can use the docker image for the CIs.

mitsudome-r · 2024-02-15T14:23:32Z

I just noticed that update-docker-manifest is failing. It might be due to docker images that were pushed manually for CES demo... https://github.com/autowarefoundation/autoware/actions/runs/7903383046/job/21571162050

esteve · 2024-02-16T12:32:11Z

@oguzkaganozt let me know if this is this ready for another review. I saw that there are some conflicts with main, shall I wait until those are resolved to review this PR again? Thanks.

oguzkaganozt · 2024-02-19T15:41:51Z

@oguzkaganozt let me know if this is this ready for another review. I saw that there are some conflicts with main, shall I wait until those are resolved to review this PR again? Thanks.

@esteve Should be resolved, please review now 👍

.devcontainer/Dockerfile

.devcontainer/devcontainer.json

.github/actions/docker-build-and-push/action.yaml

ansible/playbooks/openadk.yaml

ansible/roles/dev_tools/README.md

ansible/playbooks/openadk.yaml

ansible/roles/tensorrt/README.md

docker/README.md

docker/build.sh

xmfcx · 2024-03-06T13:36:48Z

I would like to merge this once this last wave of reviews are resolved.

Thanks @oguzkaganozt @ambroise-arm @esteve @mitsudome-r for your reviews and support for this task!

Signed-off-by: oguzkaganozt <[email protected]>

oguzkaganozt

Made some last changes according to reviews

xmfcx · 2024-03-06T15:13:47Z

@ambroise-arm could you confirm if you are happy with the changes?

xmfcx · 2024-03-06T15:25:45Z

I've confirmed that the the images generated by this PR performs well on the Autoware Universe side.

ci: update build-and-test and -differential jobs autoware.universe#6471 (comment)

ambroise-arm · 2024-03-06T14:37:51Z

.devcontainer/cuda/Dockerfile

@@ -0,0 +1,14 @@
+FROM ghcr.io/autowarefoundation/autoware-openadk:latest-devel-cuda


If apart from this line the rest of the file is exactly the same as in base/Dockerfile then I think it would be better to keep a single Dockerfile under .devcontainer/ and instead have an ARG TAG or something like that in order to use FROM ghcr.io/autowarefoundation/autoware-openadk:$TAG and then pass it with build.args (https://containers.dev/implementors/json_reference/#image-specific) in the respective devcontainer.json

@oguzkaganozt will look into this tomorrow in a follow up PR.

Let's merge this as it is today.

.devcontainer/Dockerfile

.devcontainer/devcontainer.json

ansible/playbooks/openadk.yaml

ansible/roles/tensorrt/README.md

docker/build.sh

Signed-off-by: M. Fatih Cırıt <[email protected]>

ambroise-arm · 2024-03-06T16:07:21Z

.devcontainer/base/devcontainer.json

+  "hostRequirements": {
+    "gpu": true
+  },


This can be removed for this non-GPU file. Although it is not critical because if I remember correctly it just issues a warning in the logs, but doesn't block anything.

resolving following #4072 (comment)

I'm not good at VSCode with Containers, I don't know if this affects anything related to RViz. If you think we can remove it safely, I can remove it.

It won't change how things get setup. From https://containers.dev/implementors/json_reference/#min-host-reqs: "you will be presented with a warning if the requirements are not met". Removing this won't affect whether rviz works or not.

@ambroise-arm I see, I will let @oguzkaganozt know this, let's remove it as well in the next PR.

xmfcx

Now it's time for the big merge 🥁🥁🥁🥁

xmfcx · 2024-03-06T16:18:21Z

Thanks everyone again for your invaluable comments and efforts!

oguzkaganozt · 2024-03-07T09:26:41Z

I would like to thank everyone again for all of the precious reviews and comments !

Signed-off-by: Kotaro Yoshimoto <[email protected]>

HansRobo · 2024-03-08T05:22:29Z

@oguzkaganozt
Thank you for this nice pull-request.
build-main and build-main-self-hosted workflows seem to have been left behind at autoware-universe instead of autoware-openadk.
https://github.com/autowarefoundation/autoware/actions/runs/8187823403
https://github.com/autowarefoundation/autoware/actions/runs/8187794780

And I tried a quick fix, but the build step in build-main does not work correctly.
https://github.com/HansRobo/autoware/actions/runs/8198820592

.github/workflows/docker-build-and-push-main.yaml

oguzkaganozt added meeting:openadkit-wg Open AD Kit working group type:ci Continuous Integration (CI) processes and testing. type:containers Docker containers, containerization of components, or container orchestration. labels Dec 20, 2023

oguzkaganozt requested review from esteve, xmfcx, kaspermeck-arm, mitsudome-r and ambroise-arm December 20, 2023 11:42

oguzkaganozt self-assigned this Dec 20, 2023

oguzkaganozt linked an issue Dec 20, 2023 that may be closed by this pull request

Re-organize the Autoware Docker Containers #3999

Closed

15 tasks

oguzkaganozt marked this pull request as draft December 21, 2023 09:41

ambroise-arm reviewed Jan 11, 2024

View reviewed changes

docker/autoware-openadk/docker-bake.hcl Outdated Show resolved Hide resolved

esteve previously requested changes Jan 11, 2024

View reviewed changes

ansible/playbooks/openadk.yaml Outdated Show resolved Hide resolved

oguzkaganozt requested a review from ambroise-arm January 18, 2024 10:53

mitsudome-r marked this pull request as ready for review January 22, 2024 07:40

oguzkaganozt requested a review from esteve January 30, 2024 08:38

oguzkaganozt force-pushed the 3999-re-organize-the-autoware-docker-containers branch from 0f500ad to 9f1f3ff Compare January 30, 2024 09:13

oguzkaganozt changed the title ~~re organize the autoware docker containers~~ feat(docker): re organize the autoware docker containers Jan 30, 2024

mitsudome-r reviewed Feb 1, 2024

View reviewed changes

ansible/roles/devel/tasks/main.yaml Outdated Show resolved Hide resolved

oguzkaganozt requested a review from mitsudome-r February 1, 2024 11:41

mitsudome-r mentioned this pull request Feb 5, 2024

docs(installation): update the installation instructions autowarefoundation/autoware-documentation#516

Merged

4 tasks

mitsudome-r reviewed Feb 8, 2024

View reviewed changes

setup-dev-env.sh Outdated Show resolved Hide resolved

oguzkaganozt force-pushed the 3999-re-organize-the-autoware-docker-containers branch from 0183ec2 to 5fd8757 Compare February 19, 2024 12:39

ambroise-arm reviewed Mar 5, 2024

View reviewed changes

ambroise-arm reviewed Mar 6, 2024

View reviewed changes

docker/build.sh Show resolved Hide resolved

oguzkaganozt and others added 2 commits March 6, 2024 16:43

Update readme

83c88b9

Signed-off-by: oguzkaganozt <[email protected]>

style(pre-commit): autofix

70d844e

xmfcx mentioned this pull request Mar 6, 2024

Remove the platform flag from the docker build.sh #4522

Closed

oguzkaganozt and others added 2 commits March 6, 2024 17:14

Add two flavors for VS containers

66f2552

Signed-off-by: oguzkaganozt <[email protected]>

style(pre-commit): autofix

3df3bf9

oguzkaganozt requested a review from ambroise-arm March 6, 2024 14:26

oguzkaganozt commented Mar 6, 2024

View reviewed changes

ambroise-arm reviewed Mar 6, 2024

View reviewed changes

M. Fatih Cırıt added 2 commits March 6, 2024 19:00

fix comment for ansible dev_tools

09811ae

Signed-off-by: M. Fatih Cırıt <[email protected]>

reorder prefix and suffix

fe9a840

Signed-off-by: M. Fatih Cırıt <[email protected]>

ambroise-arm reviewed Mar 6, 2024

View reviewed changes

ambroise-arm approved these changes Mar 6, 2024

View reviewed changes

xmfcx approved these changes Mar 6, 2024

View reviewed changes

xmfcx merged commit da22bdc into main Mar 6, 2024
11 of 13 checks passed

xmfcx deleted the 3999-re-organize-the-autoware-docker-containers branch March 6, 2024 16:17

xmfcx mentioned this pull request Mar 6, 2024

ci: update build-and-test and -differential jobs autowarefoundation/autoware.universe#6471

Merged

4 tasks

tier4-autoware-public-bot bot mentioned this pull request Mar 7, 2024

chore: sync upstream tier4/autoware#5

Open

HansRobo added a commit to HansRobo/autoware that referenced this pull request Mar 8, 2024

fix(ci): missing correction in autowarefoundation#4072

c42966a

Signed-off-by: Kotaro Yoshimoto <[email protected]>

xmfcx mentioned this pull request Mar 8, 2024

ci(build-main): update for openadk #4528

Merged

4 tasks

oguzkaganozt mentioned this pull request Mar 8, 2024

chore(docker): use nvidia/cuda image as cuda_base_image #4035

Closed

7 tasks

xmfcx reviewed Jun 17, 2024

View reviewed changes

.github/workflows/docker-build-and-push-main.yaml Show resolved Hide resolved

xmfcx mentioned this pull request Jun 17, 2024

docker-build-and-push should not push an image for every commit to main #4878

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(docker): re-organize the autoware docker containers #4072

feat(docker): re-organize the autoware docker containers #4072

oguzkaganozt commented Dec 20, 2023 •

edited by xmfcx

Loading

mitsudome-r commented Jan 22, 2024

mitsudome-r commented Feb 15, 2024

mitsudome-r commented Feb 15, 2024

esteve commented Feb 16, 2024

oguzkaganozt commented Feb 19, 2024

xmfcx commented Mar 6, 2024

oguzkaganozt left a comment

xmfcx commented Mar 6, 2024

xmfcx commented Mar 6, 2024

ambroise-arm Mar 6, 2024

xmfcx Mar 6, 2024 •

edited

Loading

ambroise-arm Mar 6, 2024

ambroise-arm Mar 6, 2024

ambroise-arm Mar 6, 2024

xmfcx Mar 6, 2024

ambroise-arm Mar 6, 2024 •

edited

Loading

xmfcx Mar 6, 2024

xmfcx left a comment

xmfcx commented Mar 6, 2024

oguzkaganozt commented Mar 7, 2024

HansRobo commented Mar 8, 2024

		@@ -0,0 +1,14 @@
		FROM ghcr.io/autowarefoundation/autoware-openadk:latest-devel-cuda

feat(docker): re-organize the autoware docker containers #4072

feat(docker): re-organize the autoware docker containers #4072

Conversation

oguzkaganozt commented Dec 20, 2023 • edited by xmfcx Loading

Description

Related links

Tests performed

Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author

In-review checklist for the PR reviewers

Post-review checklist for the PR author

mitsudome-r commented Jan 22, 2024

mitsudome-r commented Feb 15, 2024

mitsudome-r commented Feb 15, 2024

esteve commented Feb 16, 2024

oguzkaganozt commented Feb 19, 2024

xmfcx commented Mar 6, 2024

oguzkaganozt left a comment

Choose a reason for hiding this comment

xmfcx commented Mar 6, 2024

xmfcx commented Mar 6, 2024

ambroise-arm Mar 6, 2024

Choose a reason for hiding this comment

xmfcx Mar 6, 2024 • edited Loading

Choose a reason for hiding this comment

ambroise-arm Mar 6, 2024

Choose a reason for hiding this comment

ambroise-arm Mar 6, 2024

Choose a reason for hiding this comment

ambroise-arm Mar 6, 2024

Choose a reason for hiding this comment

xmfcx Mar 6, 2024

Choose a reason for hiding this comment

ambroise-arm Mar 6, 2024 • edited Loading

Choose a reason for hiding this comment

xmfcx Mar 6, 2024

Choose a reason for hiding this comment

xmfcx left a comment

Choose a reason for hiding this comment

xmfcx commented Mar 6, 2024

oguzkaganozt commented Mar 7, 2024

HansRobo commented Mar 8, 2024

oguzkaganozt commented Dec 20, 2023 •

edited by xmfcx

Loading

xmfcx Mar 6, 2024 •

edited

Loading

ambroise-arm Mar 6, 2024 •

edited

Loading