doil_into_container dic_335 add php container to handle doil commands

.github/TESTING.md

@@ -11,11 +11,6 @@ Before a release doil needs to be tested deeply. For that we implemented a manda
 * [ ] Update should inform you, if root has no access to docker-compose
 * [ ] Update should inform you, if your operating system isn't Darwin or Linux
 * [ ] Update should inform you, if you run a docker version lower than 19.03
-* [ ] Update should inform you, if your php version is lower than 7.4
-* [ ] Update should inform you, if you haven't installed php module zip
-* [ ] Update should inform you, if you haven't installed php module dom
-* [ ] Update should inform you, if you haven't installed composer
-* [ ] Update should inform you, if you haven't installed git
 * [ ] Change mail_password in setup/conf/doil.conf before update. After update, you should be able to login to http://doil/mails with www-data and the password you set.
 * [ ] Update should only run, if a new version is detected
 * [ ] Update runs completely without errors
@@ -28,11 +23,6 @@ Before a release doil needs to be tested deeply. For that we implemented a manda
 * [ ] Setup should inform you, if root has no access to docker-compose
 * [ ] Setup should inform you, if your operating system isn't Darwin or Linux
 * [ ] Setup should inform you, if you run a docker version lower than 19.03
-* [ ] Setup should inform you, if your php version is lower than 7.4
-* [ ] Setup should inform you, if you haven't installed php module zip
-* [ ] Setup should inform you, if you haven't installed php module dom
-* [ ] Setup should inform you, if you haven't installed composer
-* [ ] Setup should inform you, if you haven't installed git
 * [ ] Change mail_password in setup/conf/doil.conf before setup. After setup, you should be able to login to http://doil/mails with www-data and the password you set.
 * [ ] Setup runs completely without errors
 
@@ -109,8 +99,8 @@ Before a release doil needs to be tested deeply. For that we implemented a manda
 
 * [ ] Currently installed instances can be listed
 * [ ] Currently running instances can be listed
-* [ ] doil can be uninstalled (instances remain)
-* [ ] doil can be uninstalled with --prune (no instances remain)
+* [ ] doil can be uninstalled via script with --not-all (instances remain) 
+* [ ] doil can be uninstalled via script with --all (no instances remain)
 * [ ] Version can be displayed
 * [ ] Each command has its help page
 

README.md

@@ -3,13 +3,17 @@
 **doil** provides you with a simple way to create and manage development and
 testing environments for ILIAS. It will create and provision a docker container
 according to your requirements, pull the ILIAS version you want to use and even
-install it if possible.
+install it if possible. Also, **doil** provides you with a mail server wich collects 
+all mails from your instances, so you can test the ILIAS emailing.
 
 ## Installation
 
 1. download and unpack the [latest release](https://github.com/conceptsandtraining/doil/releases)
 1. cd into the unpacked directory
 1. if you run doil on a remote host ensure to change the host name in `setup/conf/doil.conf` to your host name
+1. adjust your mail password in `setup/conf/doil.conf`
+1. if you run global instances make sure to adjust 'global_instances_path' in `setup/conf/doil.conf` to specify
+   where to place them, default is '/srv/instances'. Attention, paths with 'home' are not allowed here.
 1. execute `sudo ./setup/install.sh` in order to install **doil**
 1. you can remove the downloaded folder afterwards
 1. check `doil help` for available commands and further instructions
@@ -35,8 +39,11 @@ Otherwise, use the update script.
 
 1. checkout the newest master branch or copy and extract the zip
 2. cd into the unpacked directory
-3. execute sudo ./setup/update.sh in order to update **doil**
-4. you can remove the downloaded folder afterwards
+3. if you run global instances make sure to adjust 'global_instances_path' in `setup/conf/doil.conf` to specify 
+where to place them, default is '/srv/instances'. Attention, paths with 'home' are not allowed here. The update
+will move all global instances to the set path.
+4. execute sudo ./setup/update.sh in order to update **doil**
+5. you can remove the downloaded folder afterward
 
 
 ## Dependencies
@@ -46,15 +53,15 @@ however **doil** needs [Docker](https://www.docker.com/) in order to work:
 
 * docker version >= 19.03
 * docker-compose version >= 1.25.0 (ensure that root has access too)
-* php version => 7.4
-* php*.*-zip
-* php*.*-dom
-* composer version = depending on installed php version
 * git
 * .ssh folder in your home directory. **doil** will mount it into the container. **doil** needs this to have access to any private git repositories that may be used. 
 
 Additional dependencies, but these are installed automatically during setup.
 
+* php version => 7.4
+* php*.*-zip
+* php*.*-dom
+* composer version = depending on installed php version
 * symfony/console
 * symfony/filesystem
 * symfony/process
@@ -143,8 +150,8 @@ The user who installed doil on the machine is already registered at doil. To add
 another user simply use `doil system:user add <username>`. You can manage the users
 with following commands:
 
-* `doil user:add <username>` adds a user
-* `doil user:delete <username>` deletes a user
+* `doil user:add <username>` adds a user (needs sudo privileges)
+* `doil user:delete <username>` deletes a user (needs sudo privileges)
 * `doil user:list` lists the available users
 
 See `doil user:<command> --help` for more information
@@ -258,11 +265,19 @@ via Dockers volumes and can be accessed from the host system.
 
 **doil** comes with some helpers which are useful if you want to hack on **doil**:
 
-* `doil system:uninstall` will remove doil from your system. users, instance and config remain 
-* `doil system:uninstall --prune` will remove doil completely from your system for all users
 * `doil -V|--version` displays the version
 * `doil` displays the main help page
 
+Also, there is a script to uninstall **doil** in the doil repo. If you have already deleted
+the folder easily clone it again from [doil](https://github.com/conceptsandtraining/doil).
+
+Cd into the cloned folder and execute the script.
+
+`sudo ./setup/uninstall`
+
+The script will ask you if you want to remove **doil** completely from your system or if you
+want to keep your instances, images ...
+
 
 ### SaltStack
 

app/src/App.php

@@ -9,7 +9,7 @@
 
 class App extends Application
 {
-    const NAME = "Doil Version 20230531 - build 2023-05-31";
+    const NAME = "Doil Version 20230711 - build 2023-07-11";
 
     public function __construct(Command ...$commands)
     {
@@ -19,4 +19,4 @@ public function __construct(Command ...$commands)
             $this->add($command);
         }
     }
-}
+}

app/src/Commands/Instances/CreateCommand.php

@@ -87,7 +87,7 @@ public function configure() : void
             ->addOption("use-global-repo", "u", InputOption::VALUE_NONE, "Determines if the repo is global one or not")
             ->addOption("branch", "b", InputOption::VALUE_OPTIONAL, "Sets the branch to use")
             ->addOption("phpversion", "p", InputOption::VALUE_OPTIONAL, "Sets the php version to use")
-            ->addOption("target", "t", InputOption::VALUE_OPTIONAL, "Sets the target destination for the instance. If the folder does not exist, it will be created")
+            ->addOption("target", "t", InputOption::VALUE_OPTIONAL, "Sets the target destination for the instance. If the folder does not exist, it will be created. Will be ignored while creating global instances")
             ->addOption("xdebug", "x", InputOption::VALUE_NONE, "Determines if xdebug should be installed or not")
             ->addOption("global", "g", InputOption::VALUE_NONE, "Determines if an instance is global or not")
             ->addOption("skip-readme", "s", InputOption::VALUE_NONE, "Doesn't create the README.md file")
@@ -122,14 +122,6 @@ public function execute(InputInterface $input, OutputInterface $output) : int
             return Command::FAILURE;
         }
 
-        if ($suffix == "global" && stristr($instance_path, "/home/") !== false) {
-            $this->writer->error(
-                $output,
-                "Global instances must not be created below /home directory."
-            );
-            return Command::FAILURE;
-        }
-
         $this->writer->beginBlock($output, "Creating instance " . $options['name']);
 
         if (isset($options["repo_path"]) && ! $this->filesystem->exists($options["repo_path"])) {
@@ -150,8 +142,8 @@ public function execute(InputInterface $input, OutputInterface $output) : int
             $this->writer->beginBlock($output, "Update debian image");
             $this->docker->pull("debian", self::DEBIAN_TAG);
             $this->writer->endBlock();
-            $this->writer->beginBlock($output, "Install base image");
-            $this->docker->build("/usr/local/share/doil/templates/base/Dockerfile", "base_global");
+            $this->writer->beginBlock($output, "Install base image. This will take a while. Please be patient");
+            $this->docker->build("/usr/local/share/doil/templates/base", "base_global");
             $this->writer->endBlock();
         }
 
@@ -258,7 +250,7 @@ public function execute(InputInterface $input, OutputInterface $output) : int
         $this->writer->endBlock();
 
         // building minion image
-        $this->writer->beginBlock($output, "Building minion image. This will take a while. Please be patient");
+        $this->writer->beginBlock($output, "Building minion image");
         $this->docker->runContainer($instance_name);
         $usr_id = (string) $this->posix->getUserId();
         $group_id = (string) $this->posix->getGroupId();
@@ -539,17 +531,6 @@ protected function gatherOptionData(InputInterface $input, OutputInterface $outp
         call_user_func($this->checkPHPVersion(), $php_version);
         $options["phpversion"] = $php_version;
 
-        // Target
-        if (is_null($target)) {
-            $question = new Question("Please enter a target where doil should install " . $options["name"] . ". Leave blank for current directory. : ");
-            $question->setNormalizer($this->normalizeTarget());
-            $question->setValidator($this->checkTarget());
-            $target = $helper->ask($input, $output, $question);
-        }
-        call_user_func($this->normalizeTarget(), $target);
-        call_user_func($this->checkTarget(), $target);
-        $options["target"] = $target;
-
         // Install xdebug
         if (!$xdebug && $one_option_missed) {
             $question = new ConfirmationQuestion(
@@ -570,6 +551,14 @@ protected function gatherOptionData(InputInterface $input, OutputInterface $outp
         }
         $options["global"] = $global;
 
+        if ($global) {
+            $target = explode("=", $this->filesystem->getLineInFile("/etc/doil/doil.conf", "global_instances_path") ?? "")[1];
+            if (! $target) {
+                $target = "";
+            }
+            call_user_func($this->checkGlobalTarget(), $target);
+        }
+
         // Skip readme
         if (!$skip_readme && $one_option_missed) {
             $question = new ConfirmationQuestion(
@@ -580,6 +569,17 @@ protected function gatherOptionData(InputInterface $input, OutputInterface $outp
         }
         $options["skip_readme"] = $skip_readme;
 
+        // Target
+        if (is_null($target) && !$global) {
+            $question = new Question("Please enter a target where doil should install " . $options["name"] . ". Leave blank for current directory. : ");
+            $question->setNormalizer($this->normalizeTarget());
+            $question->setValidator($this->checkTarget());
+            $target = $helper->ask($input, $output, $question);
+        }
+        call_user_func($this->normalizeTarget(), $target);
+        call_user_func($this->checkTarget(), $target);
+        $options["target"] = $target;
+
         return $options;
     }
 
@@ -641,6 +641,17 @@ protected function checkTarget() : Closure
             return $t;
         };
     }
+    protected function checkGlobalTarget() : Closure
+    {
+        return function(string $t) {
+            if (is_null($t) || $t == "") {
+                throw new RuntimeException("Missing config entry 'global_instances_path'. Please add this entry to /etc/doil/doil.conf to create global instances.");
+            }
+            if (stristr($t, "/home/") !== false) {
+                throw new RuntimeException("Global instances must not be created below /home directory. Please change the entry in /etc/doil/doil.conf.");
+            }
+        };
+    }
 
     protected function getBranches(OutputInterface $output, string $path, string $url) : array
     {

app/src/Commands/Instances/DeleteCommand.php

@@ -20,7 +20,6 @@ class DeleteCommand extends Command
 {
     protected const SALT_MAIN = "/usr/local/lib/doil/server/salt/";
     protected const POSTFIX = "/usr/local/lib/doil/server/mail/";
-    protected const PROXY_SITES = "/usr/local/lib/doil/server/proxy/conf/nginx/sites/";
     protected const PROXY_PATH = "/usr/local/lib/doil/server/proxy/";
 
     protected static $defaultName = "instances:delete";
@@ -144,12 +143,6 @@ protected function deleteInstance(
         $this->docker->removeContainer($instance . "_" . $suffix);
 
         $this->docker->executeCommand(self::SALT_MAIN, "doil_saltmain", "salt-key", "-d", "$instance.$suffix", "-y", "-q");
-
-        $proxy_path = self::PROXY_SITES . $instance . ".conf";
-        if ($this->filesystem->exists($proxy_path)) {
-            $this->filesystem->remove($proxy_path);
-        }
-
         $this->docker->executeCommand(self::PROXY_PATH, "doil_proxy", "/bin/bash", "-c", "/etc/init.d/nginx reload &>/dev/null");
 
         if ($this->docker->hasVolume($instance)) {
@@ -160,7 +153,7 @@ protected function deleteInstance(
             $this->docker->removeImage($id);
         }
 
-        $this->docker->executeCommand(self::POSTFIX, "doil_postfix", "/bin/bash", "-c", "/root/delete-postbox-configuration.sh $instance &>/dev/null");
+        $this->docker->executeCommand(self::POSTFIX, "doil_mail", "/bin/bash", "-c", "/root/delete-postbox-configuration.sh $instance &>/dev/null");
 
         $this->writer->endBlock();
 

app/src/Commands/Instances/StatusCommand.php

@@ -34,7 +34,8 @@ public function execute(InputInterface $input, OutputInterface $output) : int
         $doil_instances = array_filter($doil_instances, function($a) {
             if (
                 strstr($a, "CONTAINER ID") ||
-                strstr($a, "doil") ||
+                strstr($a, "doil_mail") ||
+                strstr($a, "doil_proxy") ||
                 strstr($a, "doil_saltmain") ||
                 strstr($a, "_local") ||
                 strstr($a, "_global")

app/src/Commands/Instances/UpCommand.php

@@ -109,7 +109,7 @@ protected function startInstance(OutputInterface $output, string $path, string $
 
     protected function hasDockerComposeFile(string $path, OutputInterface $output) : bool
     {
-        if (file_exists($path . "/docker-compose.yml")) {
+        if ($this->filesystem->exists($path . "/docker-compose.yml")) {
             return true;
         }
 

app/src/Commands/Mail/ChangePasswordCommand.php

@@ -54,17 +54,17 @@ public function execute(InputInterface $input, OutputInterface $output) : int
         }
 
         $password = $input->getArgument("password");
-        $hash = $this->docker->getShadowHashForInstance("doil.postfix", $password);
+        $hash = $this->docker->getShadowHashForInstance("doil.mail", $password);
         $hash = $this->escapeDollarChar($hash);
-        $this->docker->setGrain("doil.postfix", "roundcube_password", $hash);
+        $this->docker->setGrain("doil.mail", "roundcube_password", $hash);
         sleep(1);
 
-        $this->writer->beginBlock($output, "Apply state change-roundcube-password to doil_postfix");
-        $this->docker->applyState("doil.postfix", "change-roundcube-password");
+        $this->writer->beginBlock($output, "Apply state change-roundcube-password to doil_mail");
+        $this->docker->applyState("doil.mail", "change-roundcube-password");
         $this->writer->endBlock();
 
         $this->writer->beginBlock($output, "Apply changes to image");
-        $this->docker->commit("doil_postfix", "doil_postfix");
+        $this->docker->commit("doil_mail", "doil_mail");
         $this->writer->endBlock();
 
         return Command::SUCCESS;

app/src/Commands/Mail/LoginCommand.php

@@ -32,7 +32,7 @@ public function execute(InputInterface $input, OutputInterface $output) : int
             $this->docker->startContainerByDockerCompose(self::MAIL_PATH);
         }
 
-        $this->docker->loginIntoContainer(self::MAIL_PATH, "doil_postfix");
+        $this->docker->loginIntoContainer(self::MAIL_PATH, "doil_mail");
         return Command::SUCCESS;
     }
 }

app/src/Commands/Mail/RestartCommand.php

@@ -41,7 +41,7 @@ public function execute(InputInterface $input, OutputInterface $output) : int
         $this->docker->startContainerByDockerCompose(self::MAIL_PATH);
         sleep(3);
         $this->docker->executeDockerCommand(
-            "doil_postfix",
+            "doil_mail",
             "supervisorctl start startup"
         );
         $this->writer->endBlock();

app/src/Commands/Mail/UpCommand.php

@@ -40,7 +40,7 @@ public function execute(InputInterface $input, OutputInterface $output) : int
         $this->docker->startContainerByDockerCompose(self::MAIL_PATH);
         sleep(3);
         $this->docker->executeDockerCommand(
-            "doil_postfix",
+            "doil_mail",
             "supervisorctl start startup"
         );
         $this->writer->endBlock();