HTTP Azure Function Trigger not scaling up

[rahulpr17]

Environment:

• Kubernetes distribution - AKS - 3 node cluster
• Kubernetes version - v1.15.0
• Osiris version: Latest
• Install method: Azure Core Tools

What happened?
I was trying to deploy an HTTP Trigger to keda. I have installed Osiris components for the same. It helped me to scale to zero when no request is coming, but it is not scaling up from 1 instance. I have removed all replica-constraints from deploy.yaml file still no effects. Can you help me with any supportive links?

What you expected to happen?
I expect the nodes to be scaled up when doing a load test for 100 users. But it always show 1 instance
How to reproduce it (as minimally and precisely as possible):
Deploy a Http trigger to Keda and load test >100 user
Anything else that we need to know?

[krancour]

Honestly, I was a little surprised to discover that Azure Functions Core Tools now includes Osiris since Osiris isn't GA. In fact, there hasn't even been a single release, GA or otherwise. You reported using latest, and since we haven't had a release, I equate "latest" with "head of the master branch." I had to try this for myself and discovered that the Osiris that's bundled with Azure Functions Core Tools lags the head of master by a little bit-- but enough that I'm sure you're missing some bugs that were incidentally fixed when we refactored things a bit to include HTTP/2 support. And this is assuming your Azure Functions Core Tools itself is the latest release. If you installed using an older version, your Osiris could be even older.

I'm curious what kind of luck you have if you install Osiris that is built from the head of the master branch using instructions here:

https://github.com/deislabs/osiris#install-osiris

[rahulpr17]

I have tried the above link and installed osiris using the helm package. But the result is same. No scaling upwards. Do you have any document and working sample apart from the above link which i can use and confirm if its working fine?

[krancour]

There's just the hello-osiris app.

Can you share steps to reproduce? Is there an ingress controller or anything involved here that isn't assumed by the install / hello-osiris instructions?

Do you see anything in the activator logs to indicate it is / isn't receiving traffic for scaled down deployments? Or anything in the activator logs that explicitly indicates activation is failing?

[cwoolum]

In your service, you not only need the enabled and deployment annotations but you also need osiris.deislabs.io/loadBalancerHostname or Osiris won't know how to respond to messages.

@krancour, is there a reason that annotation isn't included in the usage section of the main page since this doesn't seem to work without it? Am I missing another use case where it's not needed?

[krancour]

In your service, you not only need the enabled and deployment annotations but you also need osiris.deislabs.io/loadBalancerHostname or Osiris won't know how to respond to messages.

This isn't universally true. It may be true depending on how the service is being addressed. It's the reason I asked this:

Is there an ingress controller or anything involved here that isn't assumed by the install / hello-osiris instructions?

is there a reason that annotation isn't included in the usage section... ?

It shouldn't be required to make the example work. That said, the annotations absolutely should be documented somewhere and they currently are not. PRs are welcome.

[rahulpr17]

Hi Thanks for looking into the issue. I am copying the deployment.yaml file which i was using to deploy the service. The service is a basic service which we create from azure function http trigger.

data:
AzureWebJobsStorage: ''
FUNCTIONS_WORKER_RUNTIME: ZG90bmV0

apiVersion: v1
kind: Secret
metadata:
name: init-sample
namespace: default

apiVersion: v1
kind: Service
metadata:
name: init-sample-http
namespace: default
annotations:
osiris.deislabs.io/enabled: true
osiris.deislabs.io/deployment: init-sample-http
spec:
selector:
app: init-sample-http
ports:

• protocol: TCP
  port: 80
  targetPort: 80
  type: LoadBalancer

apiVersion: apps/v1
kind: Deployment
metadata:
name: init-sample-http
namespace: default
labels:
app: init-sample-http
annotations:
osiris.deislabs.io/enabled: true
spec:
selector:
matchLabels:
app: init-sample-http
template:
metadata:
labels:
app: init-sample-http
spec:
containers:
- name: init-sample-http
image: docker4rahul/init-sample
ports:
- containerPort: 80
env:
- name: AzureFunctionsJobHost__functions__0
value: Function1
envFrom:
- secretRef:
name: init-sample

Below is the screen shot of the deployment for osisris available in the namespace.

keda 1/1 1 1 18d
osiris-osiris-edge-activator 1/1 1 1 18d
osiris-osiris-edge-endpoints-controller 1/1 1 1 18d
osiris-osiris-edge-endpoints-hijacker 1/1 1 1 18d
osiris-osiris-edge-proxy-injector 1/1 1 1 18d
osiris-osiris-edge-zeroscaler 1/1 1 1 18d

Please let me know if i am missing any components here.

[krancour]

@rahulpr17 logs from the activator would be helpful.

[aletc1]

In your service, you not only need the enabled and deployment annotations but you also need osiris.deislabs.io/loadBalancerHostname or Osiris won't know how to respond to messages.

I can confirm that this solution worked for me (Docker Desktop / Kubernetes):

"osiris.deislabs.io/loadBalancerHostname": "locahost"

[JimPaine]

I am hitting this issue today. Did it go anywhere? It seems like we might want to re-open the issue on the function core tools repo? As like I am, I assume everyone in the thread is using

func kubernetes deploy ...

To generate the deployment.yaml and any miss configuration of annotations should be fixed there?

[krancour]

Always check versions... Osiris had some breaking changes wrt some annotations recently. (This is allowed as it hasn't even had a single release yet.) If other projects took a dependency on Osiris and they are using edge Osiris and generating manifests that don't account for the recent changes, then that could be the issue.