Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Arm32 dockerfile (using template) #76

Draft
wants to merge 18 commits into
base: master
Choose a base branch
from
Draft

Arm32 dockerfile (using template) #76

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
ead47a9
Lower gcache size causing [ERROR] WSREP: Value 2147483648 too large f…
cvb941 May 20, 2019
204b125
arm32v6 image dockerfile and binaries
cvb941 May 20, 2019
0fe480e
set bind address to 0.0.0.0 when launching mysqld
cvb941 May 20, 2019
3408efa
Print error log before ending (for debug purposes)
cvb941 May 20, 2019
4f28c67
Dockerfile template
cvb941 May 22, 2019
38be180
Set gcache higher (to galera default) but still low enough it doesn't…
cvb941 May 22, 2019
a8bd654
Remove docker files
cvb941 May 22, 2019
0e815bf
Merge branch 'multiarch-images'
cvb941 Jun 27, 2019
b2079e2
Re-add docker files
cvb941 Jun 27, 2019
53380a1
Merge branch 'multiarch-images'
cvb941 Jun 27, 2019
0e63543
Changes for the linuxserver mariadb base image
cvb941 Jun 27, 2019
f71725b
revert socket file change
cvb941 Jun 28, 2019
d7f6486
Add gosu to apt install + prolong healthcheck to allow slow devices (…
cvb941 Jun 28, 2019
4eafe0b
specify tag on linuxserver mariadb image
cvb941 Jun 28, 2019
9d1b62a
update build.sh
cvb941 Jun 28, 2019
e97d1c9
Add dockerfile template warning comment
cvb941 Jun 28, 2019
3489c2c
Merge branch 'master' into multiarch-images
cvb941 Jun 28, 2019
7b4fa8d
Templatize sst method
cvb941 Jun 28, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions Dockerfile-10.1 → Dockerfile-10.1.amd64
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# DO NOT EDIT THIS FILE - See Dockerfile.template

FROM mariadb:10.1

# Download blocked from http://www.quicklz.com/qpress-11-linux-x64.tar
COPY bin/qpress-11-linux-x64.tar /tmp/qpress.tar
COPY bin/qpress-11-linux-amd64.tar /tmp/qpress.tar

RUN set -x \
&& apt-get update \
Expand All @@ -12,21 +13,23 @@ RUN set -x \
percona-toolkit \
percona-xtrabackup \
pv \
gosu \
&& tar -C /usr/local/bin -xf /tmp/qpress.tar qpress \
&& chmod +x /usr/local/bin/qpress \
&& rm -rf /tmp/* /var/cache/apk/* /var/lib/apt/lists/*

COPY conf.d/* /etc/mysql/conf.d/
COPY *.sh /usr/local/bin/
COPY bin/galera-healthcheck /usr/local/bin/galera-healthcheck
COPY primary-component.sql /
COPY conf.d/* /etc/mysql/conf.d/
COPY *.sh /usr/local/bin/
COPY bin/galera-healthcheck-amd64 /usr/local/bin/galera-healthcheck
COPY primary-component.sql /

# Fix permissions
RUN chown -R mysql:mysql /etc/mysql && chmod -R go-w /etc/mysql
RUN mkdir /var/run/mysqld && chown -R mysql:mysql /var/run/mysqld && chmod -R go-w /var/run/mysqld

EXPOSE 3306 4444 4567 4567/udp 4568 8080 8081

HEALTHCHECK CMD /usr/local/bin/healthcheck.sh
HEALTHCHECK --retries=5 --start-period=30s CMD /usr/local/bin/healthcheck.sh

ENV SST_METHOD=xtrabackup-v2

Expand Down
36 changes: 36 additions & 0 deletions Dockerfile-10.1.arm32v6
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# DO NOT EDIT THIS FILE - See Dockerfile.template

FROM linuxserver/mariadb:arm32v7-110.3.16mariabionic-ls21

COPY bin/qpress-11-linux-arm32v6.tar /tmp/qpress.tar

RUN set -x \
&& apt-get update \
&& apt-get install -y --no-install-recommends --no-install-suggests \
curl \
netcat \
pigz \
percona-toolkit \
percona-xtrabackup \
pv \
gosu \
&& tar -C /usr/local/bin -xf /tmp/qpress.tar qpress \
&& chmod +x /usr/local/bin/qpress \
&& rm -rf /tmp/* /var/cache/apk/* /var/lib/apt/lists/*

COPY conf.d/* /etc/mysql/conf.d/
COPY *.sh /usr/local/bin/
COPY bin/galera-healthcheck-arm32v6 /usr/local/bin/galera-healthcheck
COPY primary-component.sql /

# Fix permissions
RUN chown -R mysql:mysql /etc/mysql && chmod -R go-w /etc/mysql
RUN mkdir /var/run/mysqld && chown -R mysql:mysql /var/run/mysqld && chmod -R go-w /var/run/mysqld

EXPOSE 3306 4444 4567 4567/udp 4568 8080 8081

HEALTHCHECK --retries=5 --start-period=30s CMD /usr/local/bin/healthcheck.sh

ENV SST_METHOD=xtrabackup-v2

ENTRYPOINT ["start.sh"]
17 changes: 10 additions & 7 deletions Dockerfile → Dockerfile-10.2.amd64
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# DO NOT EDIT THIS FILE - See Dockerfile.template

FROM mariadb:10.2

# Download blocked from http://www.quicklz.com/qpress-11-linux-x64.tar
COPY bin/qpress-11-linux-x64.tar /tmp/qpress.tar
COPY bin/qpress-11-linux-amd64.tar /tmp/qpress.tar

RUN set -x \
&& apt-get update \
Expand All @@ -12,21 +13,23 @@ RUN set -x \
percona-toolkit \
percona-xtrabackup \
pv \
gosu \
&& tar -C /usr/local/bin -xf /tmp/qpress.tar qpress \
&& chmod +x /usr/local/bin/qpress \
&& rm -rf /tmp/* /var/cache/apk/* /var/lib/apt/lists/*

COPY conf.d/* /etc/mysql/conf.d/
COPY *.sh /usr/local/bin/
COPY bin/galera-healthcheck /usr/local/bin/galera-healthcheck
COPY primary-component.sql /
COPY conf.d/* /etc/mysql/conf.d/
COPY *.sh /usr/local/bin/
COPY bin/galera-healthcheck-amd64 /usr/local/bin/galera-healthcheck
COPY primary-component.sql /

# Fix permissions
RUN chown -R mysql:mysql /etc/mysql && chmod -R go-w /etc/mysql
RUN mkdir /var/run/mysqld && chown -R mysql:mysql /var/run/mysqld && chmod -R go-w /var/run/mysqld

EXPOSE 3306 4444 4567 4567/udp 4568 8080 8081

HEALTHCHECK CMD /usr/local/bin/healthcheck.sh
HEALTHCHECK --retries=5 --start-period=30s CMD /usr/local/bin/healthcheck.sh

ENV SST_METHOD=mariabackup

Expand Down
17 changes: 10 additions & 7 deletions Dockerfile-10.3 → Dockerfile-10.3.amd64
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# DO NOT EDIT THIS FILE - See Dockerfile.template

FROM mariadb:10.3

# Download blocked from http://www.quicklz.com/qpress-11-linux-x64.tar
COPY bin/qpress-11-linux-x64.tar /tmp/qpress.tar
COPY bin/qpress-11-linux-amd64.tar /tmp/qpress.tar

RUN set -x \
&& apt-get update \
Expand All @@ -12,21 +13,23 @@ RUN set -x \
percona-toolkit \
percona-xtrabackup \
pv \
gosu \
&& tar -C /usr/local/bin -xf /tmp/qpress.tar qpress \
&& chmod +x /usr/local/bin/qpress \
&& rm -rf /tmp/* /var/cache/apk/* /var/lib/apt/lists/*

COPY conf.d/* /etc/mysql/conf.d/
COPY *.sh /usr/local/bin/
COPY bin/galera-healthcheck /usr/local/bin/galera-healthcheck
COPY primary-component.sql /
COPY conf.d/* /etc/mysql/conf.d/
COPY *.sh /usr/local/bin/
COPY bin/galera-healthcheck-amd64 /usr/local/bin/galera-healthcheck
COPY primary-component.sql /

# Fix permissions
RUN chown -R mysql:mysql /etc/mysql && chmod -R go-w /etc/mysql
RUN mkdir /var/run/mysqld && chown -R mysql:mysql /var/run/mysqld && chmod -R go-w /var/run/mysqld

EXPOSE 3306 4444 4567 4567/udp 4568 8080 8081

HEALTHCHECK CMD /usr/local/bin/healthcheck.sh
HEALTHCHECK --retries=5 --start-period=30s CMD /usr/local/bin/healthcheck.sh

ENV SST_METHOD=mariabackup

Expand Down
36 changes: 36 additions & 0 deletions Dockerfile.template
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# DO NOT EDIT THIS FILE - See Dockerfile.template

FROM __BASEIMAGE_ARCH__

COPY bin/qpress-11-linux-__DOCKER_ARCH__.tar /tmp/qpress.tar

RUN set -x \
&& apt-get update \
&& apt-get install -y --no-install-recommends --no-install-suggests \
curl \
netcat \
pigz \
percona-toolkit \
percona-xtrabackup \
pv \
gosu \
&& tar -C /usr/local/bin -xf /tmp/qpress.tar qpress \
&& chmod +x /usr/local/bin/qpress \
&& rm -rf /tmp/* /var/cache/apk/* /var/lib/apt/lists/*

COPY conf.d/* /etc/mysql/conf.d/
COPY *.sh /usr/local/bin/
COPY bin/galera-healthcheck-__DOCKER_ARCH__ /usr/local/bin/galera-healthcheck
COPY primary-component.sql /

# Fix permissions
RUN chown -R mysql:mysql /etc/mysql && chmod -R go-w /etc/mysql
RUN mkdir /var/run/mysqld && chown -R mysql:mysql /var/run/mysqld && chmod -R go-w /var/run/mysqld

EXPOSE 3306 4444 4567 4567/udp 4568 8080 8081

HEALTHCHECK --retries=5 --start-period=30s CMD /usr/local/bin/healthcheck.sh

ENV SST_METHOD=__SST_METHOD__

ENTRYPOINT ["start.sh"]
0 bin/galera-healthcheck → bin/galera-healthcheck-amd64
View file
Open in desktop
File renamed without changes.
Binary file added bin/galera-healthcheck-arm32v6
View file
Open in desktop
Binary file not shown.
0 bin/qpress-11-linux-x64.tar → bin/qpress-11-linux-amd64.tar
View file
Open in desktop
File renamed without changes.
Binary file added bin/qpress-11-linux-arm32v6.tar
View file
Open in desktop
Binary file not shown.
31 changes: 31 additions & 0 deletions build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
for mariadb_ver in "10.1" "10.2" "10.3"; do

for docker_arch in "amd64" "arm32v6"; do

# For arm we have only mariadb 10.1, at least for now
if [ ${docker_arch} == "arm32v6" ] && [ ${mariadb_ver} != "10.1" ]; then
break
fi


cp Dockerfile.template Dockerfile-${mariadb_ver}.${docker_arch}

# Architecture-specific base image
case ${docker_arch} in
amd64 ) sed -i "s|__BASEIMAGE_ARCH__|mariadb:${mariadb_ver}|g" Dockerfile-${mariadb_ver}.${docker_arch} ;;
arm32v6 ) sed -i "s|__BASEIMAGE_ARCH__|linuxserver/mariadb:arm32v7-110.3.16mariabionic-ls21|g" Dockerfile-${mariadb_ver}.${docker_arch} ;;
esac

# Architecture-specific binaries
sed -i "s|__DOCKER_ARCH__|${docker_arch}|g" Dockerfile-${mariadb_ver}.${docker_arch}

# SST method based on mariadb version
if [ ${mariadb_ver} == "10.1" ]; then
sed -i "s|__SST_METHOD__|xtrabackup-v2|g" Dockerfile-${mariadb_ver}.${docker_arch}
else
sed -i "s|__SST_METHOD__|mariabackup|g" Dockerfile-${mariadb_ver}.${docker_arch}
fi

done

done
2 changes: 1 addition & 1 deletion conf.d/30-galera.cnf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ innodb_lock_schedule_algorithm=FCFS

# Subsequent `wsrep_provider_options` will replace the ones specified below
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_provider_options="gcache.size=2048M; gcache.keep_pages_size=1024M; gcache.recover=yes;"
wsrep_provider_options="gcache.size=128M; gcache.keep_pages_size=1024M; gcache.recover=yes;"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also fork this file for amd64 vs armv6 because 128M is quite small for a machine without tiny disks so I'd like to keep the default for amd64 the same.

#wsrep_sst_method=xtrabackup-v2 # This is set with SST_METHOD env variable
#wsrep_slave_threads=4

Expand Down
2 changes: 1 addition & 1 deletion mysqld.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,5 +355,5 @@ fi
# Start mysqld
echo "${LOG_MESSAGE} ---------------------------------------------------------------"
echo "${LOG_MESSAGE} Starting with options: $OPT $START"
exec mysqld $OPT $START
exec mysqld $OPT $START --bind-address=0.0.0.0
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why this is needed, but I think it belongs in one of the .cnf files instead.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is already here: https://github.com/colinmollenhour/mariadb-galera-swarm/blob/master/conf.d/30-galera.cnf#L2

Are you overwriting this file locally, perhaps?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue I guess is that, at least in the jsurf image and linuxserver image, the bind-address setting gets overwritten by /etc/mysql/mariadb.conf.d/50-server.cnf file, with 127.0.0.1.

The breakdown:
mysqladmin --help command lists the following files

Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf

Out of those, only the /etc/mysql/my.cnf file exists, with the following ending:

# Import all .cnf files from configuration directory
!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mariadb.conf.d/

The first directory is the one the dockerfile populates, the second is presumably mariadb default.

Grepping for the bind-address yields the following output:

root@5d24286d9f20:/etc/mysql# grep -r "bind-address"
conf.d/30-galera.cnf:bind-address=0.0.0.0
mariadb.conf.d/50-server.cnf:bind-address               = 127.0.0.1

As the /etc/mysql/mariadb.conf.d/ gets included later, I believe it overwrites the /etc/mysql/conf.d/ settings.

I guess we should then overwrite the mariadb.conf.d/50-server.cnf through the dockerfile instead of my command argument solution.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'd like to avoid making major changes to the config to avoid impacting existing users, so probably just resetting the environment from the jsurf image to match that of the official image would be the best soluition. E.g. RUN rm /etc/mysql/conf.d/* or similar.


1 change: 1 addition & 0 deletions start.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -423,4 +423,5 @@ test -s /var/run/galera-healthcheck-1.pid && kill $(cat /var/run/galera-healthch
test -s /var/run/galera-healthcheck-2.pid && kill $(cat /var/run/galera-healthcheck-2.pid)

echo "Goodbye"
cat /var/log/mysql/error.log
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused as to why this is needed since it is supposed to be set up so that errors go to stderr which is of course captured by docker... Is there a local cnf file overriding the log path perhaps?

I think this would be really confusing to see the full log after the "Goodbye" since it would not be chronologically correct, so if this does need to remain for some reason every line should be prefixed with something like: "=== post-shutdown - error.log:" so it is obvious where it is coming from. E.g. replace "cat" with:

awk -v prefix="=== post-shutdown - error.log: " '{print prefix $0}'

exit $RC