Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add test cases for assistant conversation history #1010

Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: conversation_history_spec.js

Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/
import { BASE_PATH } from '../../../utils/constants';
import '@cypress/skip-test/support';

const conversations = [];
const constructConversations = () => {
cy.sendMessage({
input: {
type: 'input',
content: 'What are the indices in my cluster?',
contentType: 'text',
},
}).then((result) => {
if (result.status !== 200) {
throw result.body;
}
conversations.push(result.body);
});
};

const clearConversations = () =>
conversations.map(({ conversationId }) =>
cy.deleteConversation(conversationId)
);

if (Cypress.env('DASHBOARDS_ASSISTANT_ENABLED')) {
describe('Assistant conversation history spec', () => {
before(() => {
// Set welcome screen tracking to false
localStorage.setItem('home:welcome:show', 'false');
// Hide new theme modal
localStorage.setItem('home:newThemeModal:show', 'false');
// Visit OSD
cy.visit(`${BASE_PATH}/app/home`);
// Common text to wait for to confirm page loaded, give up to 60 seconds for initial load
cy.get(`input[placeholder="Ask question"]`, { timeout: 60000 }).should(
'be.length',
1
);

// Open chat flyout
cy.get('body')
.then(($body) => $body.find('.llm-chat-flyout').length !== 0)
.then((chatFlyoutOpened) => {
if (!chatFlyoutOpened) {
cy.get('img[aria-label="toggle chat flyout icon"]').click();
}
});
});
after(() => {
// Close Chat bot
cy.get('body')
.then(($body) => $body.find('.llm-chat-flyout').length !== 0)
.then((chatFlyoutOpened) => {
if (chatFlyoutOpened) {
cy.get('img[aria-label="toggle chat flyout icon"]').click();
}
});
});
describe('panel operations', () => {
it('should show created conversation in the history list', () => {
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

cy.get('.llm-chat-flyout').contains('Conversations');
conversations.forEach(({ conversationId }) => {
cy.get(
`div[data-test-subj="chatHistoryItem-${conversationId}"]`
).should('exist');
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conversations here should be an empty array, is that intentional?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, this case should be moved under history item operations.

cy.contains('What are the indices in my cluster?');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a conversation from basic_spec.js right? Maybe we should not test that in this spec.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is for waiting assistant has been loaded successfully. I think we can keep it.

cy.get('.llm-chat-flyout button[aria-label="history"]').click();
});

it('should toggle history list', () => {
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
cy.get('.llm-chat-flyout-body')
.contains('Conversations')
.should('be.visible');

cy.get('.llm-chat-flyout button[aria-label="history"]').click();
cy.get('textarea[placeholder="Ask me anything..."]').should('exist');
cy.get('.llm-chat-flyout-body')
.contains('Conversations')
.should('not.be.visible');
});

it('should back to chat panel', () => {
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
cy.get('.llm-chat-flyout')
.contains('Conversations')
.should('be.visible');

cy.get('.llm-chat-flyout-body').contains('Back').click();
cy.get('textarea[placeholder="Ask me anything..."]').should('exist');
});

it('should hide back button in fullscreen mode', () => {
cy.get('.llm-chat-flyout button[aria-label="fullScreen"]').click();
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

cy.get('.llm-chat-flyout')
.contains('Conversations')
.should('be.visible');
cy.get('textarea[placeholder="Ask me anything..."]').should('exist');
cy.get('.llm-chat-flyout-body')
.contains('Back', { timeout: 3000 })
.should('not.exist');

// Back to default mode
cy.get('.llm-chat-flyout button[aria-label="fullScreen"]').click();
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
});
});
describe('history item operations', () => {
before(() => {
// Create conversations data
constructConversations();
});

after(() => {
// Clear created conversations in tests
clearConversations();
});

it('should load conversation in chat panel', () => {
cy.skipOn(conversations.length === 0);
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

const conversationToLoad = conversations[0];

cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToLoad.conversationId}"]`
)
.contains(conversationToLoad.title)
.click();
cy.get('textarea[placeholder="Ask me anything..."]').should('exist');
cy.get('div.llm-chat-bubble-panel-input').contains(
conversationToLoad.title
);
});

it('should able to update conversation title', () => {
cy.skipOn(conversations.length === 0);
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

const conversationToUpdate = conversations[0];
const newTitle = 'New title';

cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToUpdate.conversationId}"] button[aria-label="Edit conversation name"]`
).click();
cy.contains('Edit conversation name');

cy.get('input[aria-label="Conversation name input"').type(newTitle);
cy.get('button[data-test-subj="confirmModalConfirmButton"]')
.contains('Confirm name')
.click();

conversationToUpdate.title = newTitle;
cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToUpdate.conversationId}"]`
).contains(conversationToUpdate.title);
cy.contains('Edit conversation name', { timeout: 3000 }).should(
'not.exist'
);

// Reset to chat panel
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
});

it('should able to delete conversation', () => {
cy.skipOn(conversations.length === 0);
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

const conversationToDelete = conversations[0];

cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToDelete.conversationId}"] button[aria-label="Delete conversation"]`
).click();
cy.get('div[data-test-subj="confirmModalTitleText"]').contains(
'Delete conversation'
);

cy.get('button[data-test-subj="confirmModalConfirmButton"]')
.contains('Delete conversation')
.click();

cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToDelete.conversationId}"]`
).should('not.exist');
conversations.shift();

// Reset to chat panel
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
});
});
});
}
16 changes: 14 additions & 2 deletions cypress/utils/plugins/dashboards-assistant/commands.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@
*/

import FlowTemplateJSON from '../../../fixtures/plugins/dashboards-assistant/flow-template.json';
import { BACKEND_BASE_PATH } from '../../constants';
import { ML_COMMONS_API } from './constants';
import { BACKEND_BASE_PATH, BASE_PATH } from '../../constants';
import { ML_COMMONS_API, ASSISTANT_API } from './constants';
import clusterSettings from '../../../fixtures/plugins/dashboards-assistant/cluster_settings.json';
import { apiRequest } from '../../helpers';

Cypress.Commands.add('addAssistantRequiredSettings', () => {
cy.request('PUT', `${BACKEND_BASE_PATH}/_cluster/settings`, clusterSettings);
Expand Down Expand Up @@ -152,3 +153,14 @@ Cypress.Commands.add('stopDummyServer', () => {
}
});
});

Cypress.Commands.add('sendMessage', (body) =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this be too generic? trying to think if there might be a time where we want to avoid a conflict but I believe assistant is the only thing with a message or conversation

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change to sendAssistantMessage.

apiRequest(`${BASE_PATH}${ASSISTANT_API.SEND_MESSAGE}`, 'POST', body)
);

Cypress.Commands.add('deleteConversation', (conversationId) =>
apiRequest(
`${BASE_PATH}${ASSISTANT_API.CONVERSATION}/${conversationId}`,
'DELETE'
)
);
7 changes: 7 additions & 0 deletions cypress/utils/plugins/dashboards-assistant/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,10 @@ export const ML_COMMONS_API = {
CREATE_MODEL: `${ML_COMMONS_API_PREFIX}/models/_register`,
CREATE_AGENT: `${ML_COMMONS_API_PREFIX}/agents/_register`,
};

export const ASSISTANT_API_BASE = '/api/assistant';

export const ASSISTANT_API = {
SEND_MESSAGE: `${ASSISTANT_API_BASE}/send_message`,
CONVERSATION: `${ASSISTANT_API_BASE}/conversation`,
};
Loading