Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test TestCLIAuthenticate #730

Open
adombeck opened this issue Jan 15, 2025 · 4 comments
Open

Flaky test TestCLIAuthenticate #730

adombeck opened this issue Jan 15, 2025 · 4 comments
Labels
bug Something isn't working

Comments

@adombeck
Copy link
Contributor

See https://github.com/ubuntu/authd/actions/runs/12791887605/job/35661039860?pr=729:

📦 github.com/ubuntu/authd/pam/integration-tests
-test.shuffle 1736955896452403547
coverage: 16.1% of statements in ./...
❌ TestCLIAuthenticate (8.44s)
❌ TestCLIAuthenticate/Authenticate_user_switching_to_local_broker (11.92s)
      cli_test.go:216: 
          	Error Trace:	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:268
          	            				/home/runner/work/authd/authd/pam/integration-tests/cli_test.go:216
          	Error:      	Received unexpected error:
          	            	exit status 1
          	Test:       	TestCLIAuthenticate/Authenticate_user_switching_to_local_broker
          	Messages:   	Failed to run tape "switch_local_broker": exit status 1: File: /tmp/TestCLIAuthenticateAuthenticate_user_switching_to_local_broker2433591295/001/switch_local_broker
          	            	Output .txt switch_local_broker.txt
          	            	Output .gif switch_local_broker.gif
          	            	Set Height 800
          	            	Set FontFamily Monospace
          	            	Set TypingSpeed 5ms
          	            	Set Width 800
          	            	Set FontSize 13
          	            	Set Padding 0
          	            	Set Margin 0
          	            	Set Shell bash
          	            	Set WaitTimeout 10000ms
          	            	Env AUTHD_PAM_RUNNER_TEST_NAME=TestCLIAuthenticate/Authenticate_user_switching_to_local_broker
          	            	Env AUTHD_PAM_CONNECTION_TIMEOUT=3000
          	            	Env AUTHD_TESTS_CLI_AUTHENTICATE_TESTS_SOCK=/tmp/authd-daemon4tests1326301395/authd.socket
          	            	Hide 
          	            	Wait Screen (^|\n)>[\n]*\z
          	            	Type ./pam_authd login socket=${AUTHD_TESTS_CLI_AUTHENTICATE_TESTS_SOCK}
          	            	Enter 1
          	            	Wait Screen (^|[\n]+)[^\n]*Username: user name\n[^\n]*[\n]*\z
          	            	Show 
          	            	Hide 
          	            	Wait Screen (^|[\n]+)[^\n]*Username:[^\n]*\n[^\n]*[\n]*\z
          	            	Type user-integration-switch-broker
          	            	Wait Screen (^|[\n]+)[^\n]*Username: user-integration-switch-broker\n[^\n]*[\n]*\z
          	            	Show 
          	            	Hide 
          	            	Enter 1
          	            	Wait Screen Select your provider
          	            	Wait Screen 2. ExampleBroker
          	            	Show 
          	            	Hide 
          	            	Type 2
          	            	Wait Screen Gimme your password:\n>[\n]*\z
          	            	failed to execute command: timeout waiting for "Screen Gimme your password:\\n>[\\n]*\\z" to match Gimme your password:\n>[\n]*\z; last value was: > ./pam_authd login socket=${AUTHD_TESTS_CLI_AUTHENTICATE_TESTS_SOCK}
          	            	  Select your authentication method                                                                                                                                                                                     
          	            	                                                                                                                                                                                                                        
          	            	> 1. Password authentication                                                                                                                                                                                            
          	            	  2. Send URL to [email protected]                                                                                                                                                               
          	            	  3. Use your fido device foo                                                                                                                                                                                           
          	            	  4. Use your phone +33...                                                                                                                                                                                              
          	            	  5. Use your phone +1...                                                                                                                                                                                               
          	            	  6. Use a QR code                                                                                                                                                                                                      
          	            	  7. Authentication code                                                                                                                                                                                                
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	                                                                                                                                                                                                                        
          	            	recording failed
      helpers_test.go:230: Could not read artifact "/tmp/TestCLIAuthenticateAuthenticate_user_switching_to_local_broker2433591295/001/switch_local_broker.gif": open /tmp/TestCLIAuthenticateAuthenticate_user_switching_to_local_broker2433591295/001/switch_local_broker.gif: no such file or directory
      helpers_test.go:147: (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.103: Starting authenticate
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.103: Called with arguments: '/tmp/TestCLIAuthenticate3200197394/003/authd-pam' 'debug=true' 'connection_timeout=3000' 'logfile=/tmp/TestCLIAuthenticateAuthenticate_user_switching_to_local_broker2433591295/002/authd-pam-test-client.log' '--exec-debug' '--exec-log' '/tmp/TestCLIAuthenticateAuthenticate_user_switching_to_local_broker2433591295/002/authd-pam-test-client.log' '--exec-env' 'GOCOVERDIR=/home/runner/work/authd/authd/coverage/raw' 'socket=/tmp/authd-daemon4tests1326301395/authd.socket'
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.104: Setting up connection at unix:abstract=authd-authd-cli-0c6e047ca843fa173aab15b16787d887 (0c6e047ca843fa173aab15b16787d887)
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.104: Server started, connectable address unix:abstract=authd-authd-cli-0c6e047ca843fa173aab15b16787d887
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.104: Launching '/tmp/TestCLIAuthenticate3200197394/003/authd-pam -flags 32768 authenticate debug=true connection_timeout=3000 logfile=/tmp/TestCLIAuthenticateAuthenticate_user_switching_to_local_broker2433591295/002/authd-pam-test-client.log socket=/tmp/authd-daemon4tests1326301395/authd.socket'
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.105: Launched child 28111
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.116: Accepting connection from PID 28111 on connection 0c6e047ca843fa173aab15b16787d887
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.116: authenticate: called method GetData(('authd.already-authenticated-flag',))
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.117: authenticate: called method GetItem((1,))
          time=2025-01-15T15:47:19.117Z level=DEBUG msg="Connecting to unix:///tmp/authd-daemon4tests1326301395/authd.socket"
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.119: authenticate: called method UnsetData(('authd.authentication-broker-id',))
          time=2025-01-15T15:47:19.123Z level=DEBUG msg="adapter.supportedUILayoutsReceived{layouts:[]*authd.UILayout{(*authd.UILayout)(0xc0002385b0), (*authd.UILayout)(0xc000238620), (*authd.UILayout)(0xc000238690)}}"
          time=2025-01-15T15:47:19.124Z level=DEBUG msg="adapter.brokersListReceived{brokers:[]*authd.ABResponse_BrokerInfo{(*authd.ABResponse_BrokerInfo)(0xc0004ce0f0), (*authd.ABResponse_BrokerInfo)(0xc0004ce370)}}"
          time=2025-01-15T15:47:19.125Z level=DEBUG msg="adapter.BrokerListReceived{}, brokers: []*authd.ABResponse_BrokerInfo{(*authd.ABResponse_BrokerInfo)(0xc0004ce0f0), (*authd.ABResponse_BrokerInfo)(0xc0004ce370)}"
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.125: authenticate: called method GetItem((2,))
          time=2025-01-15T15:47:19.125Z level=DEBUG msg=adapter.userRequired{}
          time=2025-01-15T15:47:19.126Z level=DEBUG msg=adapter.ChangeStage{Stage:0}
          time=2025-01-15T15:47:19.126Z level=DEBUG msg="*adapter.userSelectionModel: Focus"
          time=2025-01-15T15:47:19.165Z level=DEBUG msg="Key: \"u\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.169Z level=DEBUG msg="Key: \"s\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.177Z level=DEBUG msg="Key: \"e\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.184Z level=DEBUG msg="Key: \"r\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.193Z level=DEBUG msg="Key: \"-\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.202Z level=DEBUG msg="Key: \"i\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.217Z level=DEBUG msg="Key: \"n\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.223Z level=DEBUG msg="Key: \"t\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.240Z level=DEBUG msg="Key: \"e\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.249Z level=DEBUG msg="Key: \"g\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.257Z level=DEBUG msg="Key: \"r\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.265Z level=DEBUG msg="Key: \"a\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.274Z level=DEBUG msg="Key: \"t\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.282Z level=DEBUG msg="Key: \"i\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.289Z level=DEBUG msg="Key: \"o\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.303Z level=DEBUG msg="Key: \"n\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.312Z level=DEBUG msg="Key: \"-\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.322Z level=DEBUG msg="Key: \"s\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.334Z level=DEBUG msg="Key: \"w\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.347Z level=DEBUG msg="Key: \"i\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.356Z level=DEBUG msg="Key: \"t\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.363Z level=DEBUG msg="Key: \"c\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.369Z level=DEBUG msg="Key: \"h\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.380Z level=DEBUG msg="Key: \"-\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.388Z level=DEBUG msg="Key: \"b\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.399Z level=DEBUG msg="Key: \"r\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.408Z level=DEBUG msg="Key: \"o\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.417Z level=DEBUG msg="Key: \"k\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.426Z level=DEBUG msg="Key: \"e\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.433Z level=DEBUG msg="Key: \"r\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.447Z level=DEBUG msg="Key: \"enter\" in stage \"userSelection\""
          time=2025-01-15T15:47:19.448Z level=DEBUG msg="adapter.userSelected{username:\"user-integration-switch-broker\"}"
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.449: authenticate: called method GetItem((2,))
          (pam_authd_authd-cli-authenticate:28098): authd-pam-exec-DEBUG: 15:47:19.450: authenticate: called method SetItem((2, 'user-integration-switch-broker'))
          time=2025-01-15T15:47:19.451Z level=DEBUG msg="adapter.UsernameSelected{}, user: \"user-integration-switch-broker\""
          time=2025-01-15T15:47:19.453Z level=DEBUG msg=adapter.brokerSelectionRequired{}
          time=2025-01-15T15:47:19.453Z level=DEBUG msg=adapter.ChangeStage{Stage:1}
          time=2025-01-15T15:47:19.453Z level=DEBUG msg="*adapter.authModeSelectionModel: Reset"
          time=2025-01-15T15:47:19.453Z level=DEBUG msg="*adapter.brokerSelectionModel: Focus"
          time=2025-01-15T15:47:19.477Z level=DEBUG msg="Key: \"2\" in stage \"brokerSelection\""
          time=2025-01-15T15:47:19.478Z level=DEBUG msg="adapter.brokerSelected{brokerID:\"2221040704\"}"
          time=2025-01-15T15:47:19.478Z level=DEBUG msg="adapter.BrokerSelected{BrokerID:\"2221040704\"}"
          time=2025-01-15T15:47:19.481Z level=DEBUG msg="adapter.SessionStarted{brokerID:\"2221040704\", sessionID:\"2221040704-b6bdbc19-74cd-48d0-93d2-790dce50a372\", encryptionKey:\"MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA7VivFOU3pznbsgQ4chKpzPdpHfrHeHzcXB303hUIGI7PJQ/C7dS0o/OvitdeKUilPudwBClBbtwoErWzxrOWbKdmeVLkhQWZcZZUhGjvuW8FLrmnT4T/kNNf6nqWbQv0qEgMU5QYEP1EJwaNpuFJ+iwn3fo6NF0UXfuaPuK5N4hcvdp0QdTxxWQd5gAklSmKzZpbAFD5ABtQDMoDM6UfD9JD/VISUulb/Lut6FWMU6/ObAmyyllsSS6tN0urSBZWEWv0/+t2amqZlyKOSUmD+jzqJ675fvUFXf5z/CTpac0i7IZNcr928hUzXd6zTUiZu1v69Q/LQID0mpfzkvzAwwIDAQAB\"}"
          time=2025-01-15T15:47:19.486Z level=DEBUG msg=adapter.GetAuthenticationModesRequested{}
          time=2025-01-15T15:47:19.486Z level=DEBUG msg="*adapter.brokerSelectionModel: Blur"
          time=2025-01-15T15:47:19.486Z level=DEBUG msg="*adapter.authenticationModel: Reset"
          time=2025-01-15T15:47:19.486Z level=DEBUG msg="*adapter.authModeSelectionModel: Focus"
          time=2025-01-15T15:47:19.489Z level=DEBUG msg="authModes[id:\"password\" label:\"Password authentication\" id:\"entry_or_wait_for_user-integration-switch-broker_gmail.com\" label:\"Send URL to [email protected]\" id:\"fidodevice1\" label:\"Use your fido device foo\" id:\"phoneack1\" label:\"Use your phone +33...\" id:\"phoneack2\" label:\"Use your phone +1...\" id:\"qrcodeandcodewithtypo\" label:\"Use a QR code\" id:\"totp_with_button\" label:\"Authentication code\"]"
          time=2025-01-15T15:47:19.489Z level=DEBUG msg="adapter.authModesReceived{authModes:[]*authd.GAMResponse_AuthenticationMode{(*authd.GAMResponse_AuthenticationMode)(0xc000244cd0), (*authd.GAMResponse_AuthenticationMode)(0xc000244f00), (*authd.GAMResponse_AuthenticationMode)(0xc000244f50), (*authd.GAMResponse_AuthenticationMode)(0xc000244fa0), (*authd.GAMResponse_AuthenticationMode)(0xc000244ff0), (*authd.GAMResponse_AuthenticationMode)(0xc000245040), (*authd.GAMResponse_AuthenticationMode)(0xc000245090)}}"
          time=2025-01-15T15:47:19.490Z level=DEBUG msg="adapter.authModeSelected{id:\"password\"}"
          time=2025-01-15T15:47:19.490Z level=DEBUG msg="adapter.AuthModeSelected{ID:\"password\"}"
          time=2025-01-15T15:47:19.493Z level=DEBUG msg=adapter.UILayoutReceived{layout:(*authd.UILayout)(0xc000154af0)}
          time=2025-01-15T15:47:19.493Z level=DEBUG msg=adapter.ChangeStage{Stage:3}
          time=2025-01-15T15:47:19.493Z level=DEBUG msg="*adapter.authenticationModel: Focus"
          time=2025-01-15T15:47:19.493Z level=DEBUG msg="adapter.formModel: Focus"
          time=2025-01-15T15:47:19.493Z level=DEBUG msg="*adapter.authenticationModel: adapter.startAuthentication{}: current model {Gimme your password: [0xc0002d3508] 0 false}, focused true"
          time=2025-01-15T15:47:19.498Z level=DEBUG msg="adapter.authModeSelectionModel: adapter.authModeSelectionFocused{}"
@adombeck adombeck added the bug Something isn't working label Jan 15, 2025
@adombeck
Copy link
Contributor Author

Cc @3v1n0

@adombeck adombeck mentioned this issue Jan 15, 2025
Merged
@adombeck
Copy link
Contributor Author

Another instance: https://github.com/ubuntu/authd/actions/runs/12793676722/job/35667048798?pr=723

@3v1n0
Copy link
Collaborator

3v1n0 commented Jan 16, 2025 •
edited
Loading

Mh it's quite weird though, since the authentication has started and in theory we had switch the focus to the auth view...

Ah, ok I see authModeSelectionModel is happening later...

@3v1n0
Copy link
Collaborator

3v1n0 commented Mar 7, 2025

This is similar too: https://github.com/ubuntu/authd/actions/runs/13636599692/job/38116829238?pr=818

For some reason the [ Skip ] button doesn't get prompted, even if it seems to be there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@3v1n0 @adombeck