Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup availability check structure and modules #109

Merged
merged 23 commits into from
Jan 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b966cd6
Setup availability check structure and modules
zacksiri Jan 9, 2025
fef4c5b
Use node instead of cluster member
zacksiri Jan 9, 2025
f473489
Setup test case and bypass for uplink nodes listing
zacksiri Jan 9, 2025
94ee75c
Correct query generation for availability
zacksiri Jan 9, 2025
e92cdb2
Add resource schema to be the output of metrics query
zacksiri Jan 10, 2025
466f585
Return tuple for availability check
zacksiri Jan 10, 2025
39b2f36
Compute available resources
zacksiri Jan 10, 2025
663398c
Setup bypass for lxd resources call
zacksiri Jan 10, 2025
2133931
Add test case for availability router
zacksiri Jan 10, 2025
76226c6
Update test case for availability
zacksiri Jan 10, 2025
58d42ce
Delete cache for monitors metrics
zacksiri Jan 10, 2025
39168a6
Integrate opsmo into uplink
zacksiri Jan 27, 2025
e1ce165
Update elixir and integrate opsmo crpm model into the supervision tree
zacksiri Jan 27, 2025
9f39b52
Setup opsmo model for inference
zacksiri Jan 28, 2025
9b6cf5e
Now take requirement passed via router into consideration
zacksiri Jan 29, 2025
51d966a
clean up formatting
zacksiri Jan 29, 2025
521cebc
Add mix opsmo.embed command
zacksiri Jan 29, 2025
9b0390d
Update cache key
zacksiri Jan 29, 2025
72ef09b
run embed before test
zacksiri Jan 29, 2025
a66ba7a
Set mix env when running opsmo embed
zacksiri Jan 29, 2025
9e0dcb7
add libring override
zacksiri Jan 29, 2025
a4f37b3
Improve error handling on availability router
zacksiri Jan 30, 2025
0b568af
Tweak test
zacksiri Jan 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 28 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,19 @@ jobs:
name: Dependencies
runs-on: warp-ubuntu-2204-x64-2x
container:
image: alpine:3.18
image: alpine:3.20

services:
postgres:
image: postgres:15.4
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

steps:
- name: Cancel Previous Runs
Expand All @@ -39,7 +51,7 @@ jobs:
deps
_build
priv/plts
key: ${{ runner.os }}-uplink-${{ hashFiles('mix.lock') }}-v2
key: ${{ runner.os }}-uplink-${{ hashFiles('mix.lock') }}-v4

- name: Install Dependencies
if: steps.mix-cache.outputs.cache-hit != 'true'
Expand All @@ -50,14 +62,21 @@ jobs:
mix deps.get
mix deps.compile
MIX_ENV=test mix deps.compile
MIX_ENV=test mix opsmo.embed
mix dialyzer --plt

env:
POSTGRES_HOST: postgres
POSTGRES_USERNAME: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_PORT: ${{ job.services.postgres.ports[5432] }}

static_code_analysis:
name: Static Code Analysis
needs: deps
runs-on: warp-ubuntu-2204-x64-2x
container:
image: alpine:3.18
image: alpine:3.20

steps:
- name: Cancel Previous Runs
Expand Down Expand Up @@ -85,7 +104,7 @@ jobs:
deps
_build
priv/plts
key: ${{ runner.os }}-uplink-${{ hashFiles('mix.lock') }}-v2
key: ${{ runner.os }}-uplink-${{ hashFiles('mix.lock') }}-v4

- name: Check Code Format
run: mix format --check-formatted
Expand All @@ -98,7 +117,7 @@ jobs:
needs: deps
runs-on: warp-ubuntu-2204-x64-2x
container:
image: alpine:3.18
image: alpine:3.20

services:
postgres:
Expand Down Expand Up @@ -138,10 +157,12 @@ jobs:
deps
_build
priv/plts
key: ${{ runner.os }}-uplink-${{ hashFiles('mix.lock') }}-v2
key: ${{ runner.os }}-uplink-${{ hashFiles('mix.lock') }}-v4

- name: Run Tests
run: mix test --trace --slowest 10
run: |
mix test --trace --slowest 10

env:
POSTGRES_HOST: postgres
POSTGRES_USERNAME: postgres
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/deployment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
- name: Setup Pakman
uses: upmaru/pakman@v8
with:
alpine: v3.18
alpine: v3.20

- name: Bootstrap Configuration
run: |
Expand Down Expand Up @@ -64,7 +64,7 @@ jobs:
- name: Setup Pakman
uses: upmaru/pakman@v8
with:
alpine: v3.18
alpine: v3.20

- name: Merge Artifact
run: |
Expand Down
4 changes: 2 additions & 2 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
elixir 1.14.4-otp-25
erlang 25.3.2.7
erlang 26.2.5
elixir 1.16.3-otp-26
caddy 2.7.4
7 changes: 7 additions & 0 deletions config/config.exs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@ config :uplink, Uplink.Cache,
backend: :shards
]

config :opsmo, :mode, :inference

config :opsmo,
models: %{
"crpm" => "0.3.7"
}

config :uplink, Uplink.Internal, port: 4080

config :uplink, Uplink.Router, port: 4040
Expand Down
2 changes: 1 addition & 1 deletion config/test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,4 @@ config :uplink, :drivers, aws_s3: Uplink.Drivers.Bucket.AwsMock
# config :plug, :validate_header_keys_during_test, false
# Print only warnings and errors during test
# Disable logging in tests
config :logger, level: :warn
config :logger, level: :warning
40 changes: 20 additions & 20 deletions instellar.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ dependencies:
- inotify-tools
- s6
- uplink-openrc
stack: alpine/3.18

stack: alpine/3.20

build:
destinations:
- '_build/prod/rel/uplink/*'
destinations:
- "_build/prod/rel/uplink/*"
command: |
export MIX_ENV=prod

Expand All @@ -25,24 +25,24 @@ build:

mix release

run:
run:
name: uplink
commands:
- name: migrate
binary: uplink
call: 'eval Uplink.Release.Tasks.migrate'
- name: console
binary: uplink
call: remote
- name: logs
binary: tail
path: /usr/bin
call: -f -n 100 /var/log/uplink/current
- name: migrate
binary: uplink
call: "eval Uplink.Release.Tasks.migrate"
- name: console
binary: uplink
call: remote
- name: logs
binary: tail
path: /usr/bin
call: -f -n 100 /var/log/uplink/current
services:
- name: web
binary: uplink
start:
call: 'start'
- name: web
binary: uplink
start:
call: "start"

hook:
post-install: |
Expand All @@ -63,7 +63,7 @@ kits:
main: true
name: lite
max_instances_count: 1
ports:
ports:
- name: web
target: 4040
variables:
Expand Down
41 changes: 23 additions & 18 deletions lib/uplink/application.ex
Original file line number Diff line number Diff line change
Expand Up @@ -26,24 +26,29 @@ defmodule Uplink.Application do

topologies = Application.get_env(:libcluster, :topologies, [])

children = [
{Uplink.Cache, []},
{Cluster.Supervisor, [topologies, [name: Uplink.ClusterSupervisor]]},
{Task.Supervisor, name: Uplink.TaskSupervisor},
{Plug.Cowboy, plug: Uplink.Internal, scheme: :http, port: internal_port},
{Pogo.DynamicSupervisor,
name: @pipeline_supervisor, scope: :uplink, sync_interval: sync_interval},
{Uplink.Monitors, []},
{
Plug.Cowboy,
plug: Uplink.Router,
scheme: :https,
port: port,
key: {:RSAPrivateKey, key},
cert: cert
},
{Uplink.Data.Provisioner, []}
]
children =
[
{Uplink.Cache, []},
{Cluster.Supervisor, [topologies, [name: Uplink.ClusterSupervisor]]},
{Task.Supervisor, name: Uplink.TaskSupervisor},
{Plug.Cowboy,
plug: Uplink.Internal, scheme: :http, port: internal_port},
{Pogo.DynamicSupervisor,
name: @pipeline_supervisor,
scope: :uplink,
sync_interval: sync_interval},
{Uplink.Monitors, []},
{
Plug.Cowboy,
plug: Uplink.Router,
scheme: :https,
port: port,
key: {:RSAPrivateKey, key},
cert: cert
},
{Uplink.Data.Provisioner, []}
]
|> Opsmo.append_model_spec(Opsmo.CRPM)

opts = [strategy: :one_for_one, name: Uplink.Supervisor]
Supervisor.start_link(children, opts)
Expand Down
Loading