Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add test cases for assistant conversation history #1010

Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/
import { BASE_PATH } from '../../../utils/constants';
import '@cypress/skip-test/support';

if (Cypress.env('DASHBOARDS_ASSISTANT_ENABLED')) {
describe('Assistant conversation history spec', () => {
before(() => {
// Set welcome screen tracking to false
localStorage.setItem('home:welcome:show', 'false');
// Hide new theme modal
localStorage.setItem('home:newThemeModal:show', 'false');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to get their current value first and then set the back to the original value after these tests are complete in case there are tests specific to these values.

// Visit OSD
cy.visit(`${BASE_PATH}/app/home`);
// Common text to wait for to confirm page loaded, give up to 60 seconds for initial load
cy.get(`input[placeholder="Ask question"]`, { timeout: 60000 }).should(
'be.length',
1
);

// Open chat flyout
cy.get('body')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some of these look like we can add test subjects to the source code so that it is easier to find the values.

Copy link
Collaborator Author

@wanglam wanglam Jan 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step is for check if flyout already opened before. Since we are visiting a new page. I think we can remove this check logic here. The flyout must be closed by default.

.then(($body) => $body.find('.llm-chat-flyout').length !== 0)
.then((chatFlyoutOpened) => {
if (!chatFlyoutOpened) {
cy.get('img[aria-label="toggle chat flyout icon"]').click();
}
});
});
after(() => {
// Close Chat bot
cy.get('body')
.then(($body) => $body.find('.llm-chat-flyout').length !== 0)
.then((chatFlyoutOpened) => {
if (chatFlyoutOpened) {
cy.get('img[aria-label="toggle chat flyout icon"]').click();
}
});
});
describe('panel operations', () => {
it('should toggle history list', () => {
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
cy.get('.llm-chat-flyout-body')
.contains('Conversations')
.should('be.visible');

cy.get('.llm-chat-flyout button[aria-label="history"]').click();
cy.get('textarea[placeholder="Ask me anything..."]').should('exist');
cy.get('.llm-chat-flyout-body')
.contains('Conversations')
.should('not.be.visible');
});

it('should back to chat panel', () => {
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
cy.get('.llm-chat-flyout')
.contains('Conversations')
.should('be.visible');

cy.get('.llm-chat-flyout-body').contains('Back').click();
cy.get('textarea[placeholder="Ask me anything..."]').should('exist');
});

it('should hide back button in fullscreen mode', () => {
cy.get('.llm-chat-flyout button[aria-label="fullScreen"]').click();
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

cy.get('.llm-chat-flyout')
.contains('Conversations')
.should('be.visible');
cy.get('textarea[placeholder="Ask me anything..."]').should('exist');
cy.get('.llm-chat-flyout-body')
.contains('Back', { timeout: 3000 })
.should('not.exist');

// Back to default mode
cy.get('.llm-chat-flyout button[aria-label="fullScreen"]').click();
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
});
});
describe('history item operations', () => {
const conversations = [];

before(() => {
// Create conversations data
cy.sendMessage({
input: {
type: 'input',
content: 'What are the indices in my cluster?',
contentType: 'text',
},
}).then((result) => {
if (result.status !== 200) {
throw result.body;
}
conversations.push(result.body);
});
});

after(() => {
// Clear created conversations in tests
conversations.map(({ conversationId }) =>
cy.deleteConversation(conversationId)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a bulk delete API that we can use? are will there be no need to implement that API.

);
});

it('should show created conversation in the history list', () => {
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

cy.get('.llm-chat-flyout').contains('Conversations');
conversations.forEach(({ conversationId }) => {
cy.get(
`div[data-test-subj="chatHistoryItem-${conversationId}"]`
).should('exist');
});
cy.contains('What are the indices in my cluster?');
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
});

it('should load conversation in chat panel', () => {
cy.skipOn(conversations.length === 0);
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

const conversationToLoad = conversations[0];

cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToLoad.conversationId}"]`
)
.contains(conversationToLoad.title)
.click();
cy.get('textarea[placeholder="Ask me anything..."]').should('exist');
cy.get('div.llm-chat-bubble-panel-input').contains(
conversationToLoad.title
);
});

it('should able to update conversation title', () => {
cy.skipOn(conversations.length === 0);
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

const conversationToUpdate = conversations[0];
const newTitle = 'New title';

cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToUpdate.conversationId}"] button[aria-label="Edit conversation name"]`
).click();
cy.contains('Edit conversation name');

cy.get('input[aria-label="Conversation name input"').type(newTitle);
cy.get('button[data-test-subj="confirmModalConfirmButton"]')
.contains('Confirm name')
.click();

conversationToUpdate.title = newTitle;
cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToUpdate.conversationId}"]`
).contains(conversationToUpdate.title);
cy.contains('Edit conversation name', { timeout: 3000 }).should(
'not.exist'
);

// Reset to chat panel
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
});

it('should able to delete conversation', () => {
cy.skipOn(conversations.length === 0);
cy.get('.llm-chat-flyout button[aria-label="history"]').click();

const conversationToDelete = conversations[0];

cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToDelete.conversationId}"] button[aria-label="Delete conversation"]`
).click();
cy.get('div[data-test-subj="confirmModalTitleText"]').contains(
'Delete conversation'
);

cy.get('button[data-test-subj="confirmModalConfirmButton"]')
.contains('Delete conversation')
.click();

cy.get(
`div[data-test-subj="chatHistoryItem-${conversationToDelete.conversationId}"]`
).should('not.exist');
conversations.shift();

// Reset to chat panel
cy.get('.llm-chat-flyout button[aria-label="history"]').click();
});
});
});
}
16 changes: 14 additions & 2 deletions cypress/utils/plugins/dashboards-assistant/commands.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@
*/

import FlowTemplateJSON from '../../../fixtures/plugins/dashboards-assistant/flow-template.json';
import { BACKEND_BASE_PATH } from '../../constants';
import { ML_COMMONS_API } from './constants';
import { BACKEND_BASE_PATH, BASE_PATH } from '../../constants';
import { ML_COMMONS_API, ASSISTANT_API } from './constants';
import clusterSettings from '../../../fixtures/plugins/dashboards-assistant/cluster_settings.json';
import { apiRequest } from '../../helpers';

Cypress.Commands.add('addAssistantRequiredSettings', () => {
cy.request('PUT', `${BACKEND_BASE_PATH}/_cluster/settings`, clusterSettings);
Expand Down Expand Up @@ -152,3 +153,14 @@ Cypress.Commands.add('stopDummyServer', () => {
}
});
});

Cypress.Commands.add('sendMessage', (body) =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this be too generic? trying to think if there might be a time where we want to avoid a conflict but I believe assistant is the only thing with a message or conversation

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change to sendAssistantMessage.

apiRequest(`${BASE_PATH}${ASSISTANT_API.SEND_MESSAGE}`, 'POST', body)
);

Cypress.Commands.add('deleteConversation', (conversationId) =>
apiRequest(
`${BASE_PATH}${ASSISTANT_API.CONVERSATION}/${conversationId}`,
'DELETE'
)
);
7 changes: 7 additions & 0 deletions cypress/utils/plugins/dashboards-assistant/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,10 @@ export const ML_COMMONS_API = {
CREATE_MODEL: `${ML_COMMONS_API_PREFIX}/models/_register`,
CREATE_AGENT: `${ML_COMMONS_API_PREFIX}/agents/_register`,
};

export const ASSISTANT_API_BASE = '/api/assistant';

export const ASSISTANT_API = {
SEND_MESSAGE: `${ASSISTANT_API_BASE}/send_message`,
CONVERSATION: `${ASSISTANT_API_BASE}/conversation`,
};
Loading