Disable backend autoconfig via default creds (#1778)

* Disable backends autoconfig on server/config.yml init

* Clarify that configuring backends is optional

README.md

@@ -17,7 +17,7 @@
 `dstack` is a streamlined alternative to Kubernetes, specifically designed for AI. It simplifies container orchestration
 for AI workloads both in the cloud and on-prem, speeding up the development, training, and deployment of AI models.
 
-`dstack` is easy to use with any cloud providers as well as on-prem servers.
+`dstack` is easy to use with any cloud provider as well as on-prem servers.
 
 #### Accelerators
 
@@ -33,16 +33,18 @@ for AI workloads both in the cloud and on-prem, speeding up the development, tra
 
 ## Installation
 
-Before using `dstack` through CLI or API, set up a `dstack` server.
+> Before using `dstack` through CLI or API, set up a `dstack` server. If you already have a running `dstack` server, you only need to [set up the CLI](#set-up-the-cli).
 
-### Configure backends
+### (Optional) Configure backends
 
 To use `dstack` with your own cloud accounts, create the `~/.dstack/server/config.yml` file and 
-[configure backends](https://dstack.ai/docs/reference/server/config.yml).
+[configure backends](https://dstack.ai/docs/reference/server/config.yml). Alternatively, you can configure backends via the control plane UI after you start the server.
+
+You can skip backends configuration if you intend to run containers  only on your on-prem servers. Use [SSH fleets](https://dstack.ai/docs/concepts/fleets#ssh-fleets) for that.
 
 ### Start the server
 
-Once backends are configured, proceed to start the server:
+Once the backends are configured, proceed to start the server:
 
 <div class="termy">
 
@@ -59,7 +61,7 @@ The server is running at http://127.0.0.1:3000/
 </div>
 
 For more details on server configuration options, see the
-[server deployment](https://dstack.ai/docs/guides/server-deployment.md) guide.
+[server deployment guide](https://dstack.ai/docs/guides/server-deployment.md).
 
 ### Set up the CLI
 
@@ -75,15 +77,8 @@ $ dstack config --url http://127.0.0.1:3000 \
 Configuration is updated at ~/.dstack/config.yml
 ```
 
-### Create SSH fleets
-
-If you want the `dstack` server to run containers on your on-prem servers,
-use [fleets](https://dstack.ai/docs/concepts/fleets#ssh-fleets).
-
 ## How does it work?
 
-> Before using `dstack`, [install](https://dstack.ai/docs/installation) the server and configure backends.
-
 ### 1. Define configurations
 
 `dstack` supports the following configurations:

docs/docs/installation/index.md

@@ -9,10 +9,12 @@ To use the open-source version of `dstack` with your own cloud accounts or on-pr
 > If you don't want to host the `dstack` server (or want to access GPU marketplace),
 > skip installation and proceed to [dstack Sky :material-arrow-top-right-thin:{ .external }](https://sky.dstack.ai){:target="_blank"}.
 
-## Configure backends
+### (Optional) Configure backends
 
 To use `dstack` with your own cloud accounts, create the `~/.dstack/server/config.yml` file and 
-[configure backends](../reference/server/config.yml.md).
+[configure backends](../reference/server/config.yml.md). Alternatively, you can configure backends via the control plane UI after you start the server.
+
+You can skip backends configuration if you intend to run containers  only on your on-prem servers. Use [SSH fleets](../concepts/fleets.md#ssh-fleets) for that.
 
 ## Start the server
 
@@ -55,7 +57,7 @@ Once backends are configured, proceed and start the server:
     </div>
 
 > For more details on server configuration options, see the
-> [server deployment](../guides/server-deployment.md) guide.
+> [server deployment guide](../guides/server-deployment.md).
 
 ## Set up the CLI
 
@@ -91,14 +93,10 @@ Configuration is updated at ~/.dstack/config.yml
 
 This configuration is stored in `~/.dstack/config.yml`.
 
-## Create SSH fleets
-
-If you want the `dstack` server to run containers on your on-prem clusters,
-create [SSH fleets](../concepts/fleets.md#ssh-fleets).
-
 ## What's next?
 
 1. Check the [server/config.yml reference](../reference/server/config.yml.md) on how to configure backends
-2. Follow [quickstart](../quickstart.md)
-3. Browse [examples](/examples)
-4. Join the community via [Discord :material-arrow-top-right-thin:{ .external }](https://discord.gg/u8SmfwPpMd)
+2. Check [SSH fleets](../concepts/fleets.md#ssh-fleets) to learn about running on your on-prem servers
+3. Follow [quickstart](../quickstart.md)
+4. Browse [examples](/examples)
+5. Join the community via [Discord :material-arrow-top-right-thin:{ .external }](https://discord.gg/u8SmfwPpMd)

src/dstack/_internal/server/services/config.py

@@ -347,7 +347,9 @@ class _BackendAPIConfig(CoreModel):
 
 class ProjectConfig(CoreModel):
     name: Annotated[str, Field(description="The name of the project")]
-    backends: Annotated[List[BackendConfig], Field(description="The list of backends")]
+    backends: Annotated[
+        Optional[List[BackendConfig]], Field(description="The list of backends")
+    ] = None
 
 
 EncryptionKeyConfig = Annotated[AnyEncryptionKeyConfig, Field(..., discriminator="type")]
@@ -373,7 +375,14 @@ def load_config(self) -> bool:
         return self.config is not None
 
     async def init_config(self, session: AsyncSession):
-        self.config = await self._init_config(session=session, init_backends=True)
+        """
+        Initializes the default server/config.yml.
+        The default config is empty or contains an existing `main` project config.
+        """
+        # The backends auto init feature via default creds is currently disabled
+        # so that the backend configuration is always explicit.
+        # Details: https://github.com/dstackai/dstack/issues/1384
+        self.config = await self._init_config(session=session, init_backends=False)
         if self.config is not None:
             self._save_config(self.config)
 
@@ -386,7 +395,7 @@ async def sync_config(self, session: AsyncSession):
 
     async def apply_encryption(self):
         if self.config is None:
-            logger.warning("server/config.yml not loaded. Skipping encryption configuration.")
+            logger.info("No server/config.yml. Skipping encryption configuration.")
             return
         if self.config.encryption is not None:
             encryption_services.init_encryption_keys(self.config.encryption.keys)
@@ -419,7 +428,7 @@ async def _apply_project_config(
                 session=session, project_name=project_config.name
             )
         backends_to_delete = backends_services.list_available_backend_types()
-        for backend_config in project_config.backends:
+        for backend_config in project_config.backends or []:
             config_info = config_to_internal_config(backend_config)
             backend_type = BackendType(config_info.type)
             try:

src/tests/_internal/server/services/test_config.py

@@ -21,6 +21,7 @@ class TestServerConfigManager:
     class TestInitConfig:
         @pytest.mark.asyncio
         @pytest.mark.parametrize("test_db", ["sqlite", "postgres"], indirect=True)
+        @pytest.mark.skip(reason="Backends auto init is currently disabled")
         async def test_inits_backend(self, test_db, session: AsyncSession, tmp_path: Path):
             await create_project(session=session, name="main")
             config_filepath = tmp_path / "config.yml"