Pool Manager creates more pods than poolsize

[AnatoliyYakimov]

Fission/Kubernetes version

fission version
client:
  fission/core:
    BuildDate: "2021-06-21T10:38:48Z"
    GitCommit: d06ad04341377bec6b8c905a440956b7d659d8fb
    Version: 1.13.1
server:
  fission/core:
    BuildDate: "2021-06-21T09:13:11Z"
    GitCommit: 401cbd597b63f5646e3e4be7df38870fdd617485
    Version: 1.13.1

minikube version
minikube version: v1.21.0
commit: 76d74191d82c47883dc7e1319ef7cebd3e00ee11

kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:59:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.7", GitCommit:"132a687512d7fb058d0f5890f07d4121b3f0a2e2", GitTreeState:"clean", BuildDate:"2021-05-12T12:32:49Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes platform (e.g. Google Kubernetes Engine)
Minikube
Describe the bug
Poolmanager creates more pods than poolsize.
fission env list

NAME        IMAGE              BUILDER_IMAGE POOLSIZE MINCPU MAXCPU MINMEMORY MAXMEMORY EXTNET GRACETIME
helidon-env helidon-env:latest               1        0      0      32Mi      128Mi     false  0

To Reproduce

1. Create any env with poolsize 1, helidon-env in my case
2. Fission creates new unspecialized pool pod
3. Create any function related to this env, upper-case in my case
4. call fission fn test --name upper-case --body "some message"
5. Insted of reusing existing pod, Fission will create a new one
   Expected result
   Fission respecialize existing pod and use it

Actual result

Screenshots/Dump file

$ fission support dump

fission-dump_1625568047.zip
Additional context

Im using custom environment for Java based on Helidon Project.

[AnatoliyYakimov]

When i have 2 functions useng same env with poolsize = 1, im getting 4 pods:

[sanketsudake]

Hi @AnatoliyYakimov,

Fission ensure we have a number of warm pods ready, as per pool size. I see we see two pods with a replicaset, which means they are still with the pool manager. One of them is in terminating and another in running state.

No of specialized pods(function is loaded into a pod and its serving function) can vary depending on the number of requests the function is serving. Typically when the pod is specialized, we keep it until the point it is serving requests. After idle time of 2 minutes, we clean up the pod.

However, all said above there can be few situations, where you would see more no of pods in a momentary situation, as Fission doesn't have the strict guarantee.