Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ai collab explicit #22836

Open
wants to merge 30 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
5f912d8
Adds explicit strategy with a few unresovled type errors
seanimam Oct 15, 2024
ab3b543
New SharedTree exports for aiCollab library
seanimam Oct 17, 2024
822092e
Updates ai-collab to include the explicit strategy and export it unde…
seanimam Oct 17, 2024
478c5b2
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
seanimam Oct 17, 2024
4b5bc91
Adds back accidentlaly removed dependencies from tree
seanimam Oct 18, 2024
c8b6684
fixes broken tests for implicit strategy not accounting for new objec…
seanimam Oct 18, 2024
3466402
WIP adding test for explicit-strategy
seanimam Oct 18, 2024
62b3783
comments out TreeNode typeguarding as we don't have a working solutio…
seanimam Oct 18, 2024
ecd9746
Adds agentEditingReducer tests, 2 failing currently
seanimam Oct 18, 2024
ccfe5b6
Adds remaining tests with two failing
seanimam Oct 21, 2024
2be8309
small import warning fix
seanimam Oct 21, 2024
a01a52a
In progress adding planning step and other new updates from taylors fork
seanimam Oct 21, 2024
a29d48a
Updates package with new planning step and removal of json streaming …
seanimam Oct 22, 2024
0cf8cfe
Adds jsdoc, resolves build errors, work in progress integration tests
seanimam Oct 25, 2024
232d335
skips integ tests
seanimam Oct 25, 2024
e5e3b2d
Adds more documented bug cases to integration testing
seanimam Oct 25, 2024
b60ecf3
API Updated to accept a treeNode instead of the whole tree
seanimam Oct 28, 2024
c3c50d8
Small integ test update
seanimam Oct 28, 2024
1cd8d8b
Merge branch 'main' of https://github.com/seanimam/FluidFramework int…
seanimam Oct 28, 2024
3c1934c
Merge conflict resolutions and small api update to take TSchema type …
seanimam Oct 28, 2024
6c3b76d
Adds a new README.md file to the /src directory and fixes a SharedTre…
seanimam Oct 29, 2024
f3bd57b
README update
seanimam Oct 29, 2024
b0e2984
Adds more jsdoc comments to internal explicit strategy functions and …
seanimam Oct 29, 2024
c9d79b5
Adds more JSDOC comments and renames ambigious function
seanimam Oct 29, 2024
8f1f36f
Removes old unused functions from promptGeneration, removes exports f…
seanimam Oct 31, 2024
bac3094
Updates some JSOC, removes commented imports from tests, exports and …
seanimam Oct 31, 2024
6eb6623
Another round of JSDoc updates
seanimam Oct 31, 2024
7bfba91
Removes SharedTree simple schema omitFromJson capability from FieldSc…
seanimam Oct 31, 2024
a79e33c
adds comments for undocumented aiCollabApi members and adds partial-f…
seanimam Oct 31, 2024
b675572
Adds jsdoc for remaining aiCollab API members
seanimam Oct 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/dds/tree/api-report/tree.alpha.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ export class FieldSchema<out Kind extends FieldKind = FieldKind, out Types exten
export interface FieldSchemaMetadata<TCustomMetadata = unknown> {
readonly custom?: TCustomMetadata;
readonly description?: string | undefined;
readonly omitFromJson?: boolean | undefined;
Copy link
Contributor

@Josmithr Josmithr Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't add this without a proper design review, especially since this is affecting a public API.

I am also personally not a fan of this pattern - I think we should be really strict about what goes in this property bag and should limit to things to very core concepts.

I would prefer to extend the toJsonSchema transformation to allow the user to provide a callback that can be used to adjust the output based on their own custom metadata. This was the intention when custom was added here.

The idea is that this would closely resemble JSON.stringify's replacer function parameter.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify

I've had this work on my plate for a little while now, but have been consumed by website work 🫤

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can our demo work okay without these changes? I'd prefer to have a separate PR add the relevant capabilities to the toJsonSchema transformation logic with the appropriate test coverage and everything. Would probably be easiest (if possible) to omit these changes for now and move forward with everything else.

Copy link
Contributor Author

@seanimam seanimam Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can work. I've removed the omitFromJson changes in commit 7bfba91ce95a5532c21588b8e7e4b3b750edeecc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes our demo can work without this. I've removed the omitFromJson addition in commit 7bfba91

Copy link
Contributor

@Josmithr Josmithr Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Let me know if/when you need the filtering functionality / want to discuss design options. I'm hoping that I'll be able to get back to SharedTree work next sprint, but I also thought I would get to it this sprint, so... 🤷‍♂️

}

// @public
Expand Down
1 change: 1 addition & 0 deletions packages/dds/tree/api-report/tree.beta.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ export class FieldSchema<out Kind extends FieldKind = FieldKind, out Types exten
export interface FieldSchemaMetadata<TCustomMetadata = unknown> {
readonly custom?: TCustomMetadata;
readonly description?: string | undefined;
readonly omitFromJson?: boolean | undefined;
}

// @public
Expand Down
1 change: 1 addition & 0 deletions packages/dds/tree/api-report/tree.legacy.alpha.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ export class FieldSchema<out Kind extends FieldKind = FieldKind, out Types exten
export interface FieldSchemaMetadata<TCustomMetadata = unknown> {
readonly custom?: TCustomMetadata;
readonly description?: string | undefined;
readonly omitFromJson?: boolean | undefined;
}

// @public
Expand Down
1 change: 1 addition & 0 deletions packages/dds/tree/api-report/tree.legacy.public.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ export class FieldSchema<out Kind extends FieldKind = FieldKind, out Types exten
export interface FieldSchemaMetadata<TCustomMetadata = unknown> {
readonly custom?: TCustomMetadata;
readonly description?: string | undefined;
readonly omitFromJson?: boolean | undefined;
}

// @public
Expand Down
1 change: 1 addition & 0 deletions packages/dds/tree/api-report/tree.public.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ export class FieldSchema<out Kind extends FieldKind = FieldKind, out Types exten
export interface FieldSchemaMetadata<TCustomMetadata = unknown> {
readonly custom?: TCustomMetadata;
readonly description?: string | undefined;
readonly omitFromJson?: boolean | undefined;
}

// @public
Expand Down
17 changes: 17 additions & 0 deletions packages/dds/tree/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,23 @@ export {
getJsonSchema,
type LazyItem,
type Unenforced,
type SimpleNodeSchemaBase,
type SimpleTreeSchema,
type SimpleNodeSchema,
type SimpleFieldSchema,
type SimpleLeafNodeSchema,
type SimpleMapNodeSchema,
type SimpleArrayNodeSchema,
type SimpleObjectNodeSchema,
normalizeFieldSchema,
isTreeNodeSchemaClass,
normalizeAllowedTypes,
getSimpleSchema,
numberSchema,
stringSchema,
booleanSchema,
handleSchema,
nullSchema,
Comment on lines +174 to +178
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these needed? Can we not use the schemaFactory.number, etc. pattern? Not super concerned about exposing these as @internal, just wondering if it's strictly necessary.

type ReadonlyArrayNode,
} from "./simple-tree/index.js";
export {
Expand Down
2 changes: 2 additions & 0 deletions packages/dds/tree/src/simple-tree/api/getSimpleSchema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ const simpleSchemaCache = new WeakMap<TreeNodeSchema, SimpleTreeSchema>();
*
* @privateRemarks In the future, we may wish to move this to a more discoverable API location.
* For now, while still an experimental API, it is surfaced as a free function.
*
* @internal
*/
export function getSimpleSchema(schema: ImplicitFieldSchema): SimpleTreeSchema {
return getOrCreate(simpleSchemaCache, schema, () => toSimpleTreeSchema(schema));
Expand Down
11 changes: 10 additions & 1 deletion packages/dds/tree/src/simple-tree/api/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,16 @@ export {
} from "./schemaCreationUtilities.js";
export { treeNodeApi, type TreeNodeApi } from "./treeNodeApi.js";
export { createFromInsertable, cursorFromInsertable } from "./create.js";
export type { SimpleTreeSchema } from "./simpleSchema.js";
export type {
SimpleTreeSchema,
SimpleNodeSchema,
SimpleFieldSchema,
SimpleLeafNodeSchema,
SimpleMapNodeSchema,
SimpleArrayNodeSchema,
SimpleObjectNodeSchema,
SimpleNodeSchemaBase,
} from "./simpleSchema.js";
export {
type JsonSchemaId,
type JsonSchemaType,
Expand Down
16 changes: 12 additions & 4 deletions packages/dds/tree/src/simple-tree/api/simpleSchema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@

import type { ValueSchema } from "../../core/index.js";
import type { NodeKind } from "../core/index.js";
import type { FieldKind } from "../schemaTypes.js";
import type { FieldKind, FieldSchemaMetadata } from "../schemaTypes.js";

/**
* Base interface for all {@link SimpleNodeSchema} implementations.
*
* @internal
* @sealed
*/
export interface SimpleNodeSchemaBase<TNodeKind extends NodeKind> {
Expand All @@ -24,6 +25,7 @@ export interface SimpleNodeSchemaBase<TNodeKind extends NodeKind> {
/**
* A {@link SimpleNodeSchema} for an object node.
*
* @internal
* @sealed
*/
export interface SimpleObjectNodeSchema extends SimpleNodeSchemaBase<NodeKind.Object> {
Expand All @@ -36,6 +38,7 @@ export interface SimpleObjectNodeSchema extends SimpleNodeSchemaBase<NodeKind.Ob
/**
* A {@link SimpleNodeSchema} for an array node.
*
* @internal
* @sealed
*/
export interface SimpleArrayNodeSchema extends SimpleNodeSchemaBase<NodeKind.Array> {
Expand All @@ -51,6 +54,7 @@ export interface SimpleArrayNodeSchema extends SimpleNodeSchemaBase<NodeKind.Arr
/**
* A {@link SimpleNodeSchema} for a map node.
*
* @internal
* @sealed
*/
export interface SimpleMapNodeSchema extends SimpleNodeSchemaBase<NodeKind.Map> {
Expand All @@ -66,6 +70,7 @@ export interface SimpleMapNodeSchema extends SimpleNodeSchemaBase<NodeKind.Map>
/**
* A {@link SimpleNodeSchema} for a leaf node.
*
* @internal
* @sealed
*/
export interface SimpleLeafNodeSchema extends SimpleNodeSchemaBase<NodeKind.Leaf> {
Expand All @@ -81,6 +86,8 @@ export interface SimpleLeafNodeSchema extends SimpleNodeSchemaBase<NodeKind.Leaf
* @remarks This definition is incomplete, and references child types by identifiers.
* To be useful, this generally needs to be used as a part of a complete {@link SimpleTreeSchema}, which
* contains backing {@link SimpleTreeSchema.definitions} for each referenced identifier.
*
* @internal
*/
export type SimpleNodeSchema =
| SimpleLeafNodeSchema
Expand All @@ -95,6 +102,7 @@ export type SimpleNodeSchema =
* To be useful, this generally needs to be used as a part of a complete {@link SimpleTreeSchema}, which
* contains backing {@link SimpleTreeSchema.definitions} for each referenced identifier.
*
* @internal
* @sealed
*/
export interface SimpleFieldSchema {
Expand All @@ -112,9 +120,9 @@ export interface SimpleFieldSchema {
readonly allowedTypes: ReadonlySet<string>;

/**
* {@inheritDoc FieldSchemaMetadata.description}
* {@inheritDoc FieldSchemaMetadata}
*/
readonly description?: string | undefined;
readonly metadata?: FieldSchemaMetadata | undefined;
}

/**
Expand All @@ -123,7 +131,7 @@ export interface SimpleFieldSchema {
* @remarks Contains the complete set of schema {@link SimpleTreeSchema.definitions} required to resolve references,
* which are represented inline with identifiers.
*
* @sealed
* @internal
seanimam marked this conversation as resolved.
Show resolved Hide resolved
*/
export interface SimpleTreeSchema extends SimpleFieldSchema {
/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,10 @@ function convertObjectNodeSchema(schema: SimpleObjectNodeSchema): JsonObjectNode
const properties: Record<string, JsonFieldSchema> = {};
const required: string[] = [];
for (const [key, value] of Object.entries(schema.fields)) {
if (value.metadata?.omitFromJson === true) {
// Don't emit JSON Schema for fields which specify they should be excluded.
continue;
}
const allowedTypes: JsonSchemaRef[] = [];
for (const allowedType of value.allowedTypes) {
allowedTypes.push(createSchemaRef(allowedType));
Expand All @@ -150,8 +154,8 @@ function convertObjectNodeSchema(schema: SimpleObjectNodeSchema): JsonObjectNode
};

// Don't include "description" property at all if it's not present in the input.
if (value.description !== undefined) {
output.description = value.description;
if (value.metadata?.description !== undefined) {
output.description = value.metadata.description;
Comment on lines +153 to +154
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're deliberately not doing anything with the rest of the metadata here, right?

Copy link
Contributor

@Josmithr Josmithr Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The global metadata bag isn't necessarily JSON compatible. The description property is explicitly respected here, but other flags are not necessarily appropriate or compatible. So this seems right to me.

}

properties[key] = output;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,9 @@ export function toSimpleTreeSchema(schema: ImplicitFieldSchema): SimpleTreeSchem

// Include the "description" property only if it's present on the input.
if (normalizedSchema.metadata?.description !== undefined) {
output.description = normalizedSchema.metadata.description;
output.metadata = {
description: normalizedSchema.metadata.description,
};
}

return output;
Expand Down Expand Up @@ -140,9 +142,9 @@ function fieldSchemaToSimpleSchema(schema: FieldSchema): SimpleFieldSchema {
allowedTypes,
};

// Don't include "description" property at all if it's not present.
if (schema.metadata?.description !== undefined) {
result.description = schema.metadata.description;
// Don't include "metadata" property at all if it's not present.
if (schema.metadata !== undefined) {
result.metadata = schema.metadata;
}

// eslint-disable-next-line @typescript-eslint/no-explicit-any
Expand Down
8 changes: 8 additions & 0 deletions packages/dds/tree/src/simple-tree/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,14 @@ export {
type NodeChangedData,
TreeBeta,
type TreeChangeEventsBeta,
type SimpleNodeSchemaBase,
type SimpleTreeSchema,
type SimpleNodeSchema,
type SimpleFieldSchema,
type SimpleLeafNodeSchema,
type SimpleMapNodeSchema,
type SimpleArrayNodeSchema,
type SimpleObjectNodeSchema,
type JsonSchemaId,
type JsonSchemaType,
type JsonObjectNodeSchema,
Expand Down Expand Up @@ -121,6 +128,7 @@ export {
type InsertableField,
type Insertable,
type UnsafeUnknownSchema,
normalizeAllowedTypes,
} from "./schemaTypes.js";
export {
getTreeNodeForField,
Expand Down
19 changes: 19 additions & 0 deletions packages/dds/tree/src/simple-tree/leafNodeSchema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,27 @@ function makeLeaf<Name extends string, const T extends ValueSchema>(
}

// Leaf schema shared between all SchemaFactory instances.
/**
* @internal
*/
export const stringSchema = makeLeaf("string", ValueSchema.String);

/**
* @internal
*/
export const numberSchema = makeLeaf("number", ValueSchema.Number);

/**
* @internal
*/
export const booleanSchema = makeLeaf("boolean", ValueSchema.Boolean);

/**
* @internal
*/
export const nullSchema = makeLeaf("null", ValueSchema.Null);

/**
* @internal
*/
export const handleSchema = makeLeaf("handle", ValueSchema.FluidHandle);
11 changes: 11 additions & 0 deletions packages/dds/tree/src/simple-tree/schemaTypes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import { isLazy, type FlexListToUnion, type LazyItem } from "./flexList.js";

/**
* Returns true if the given schema is a {@link TreeNodeSchemaClass}, or otherwise false if it is a {@link TreeNodeSchemaNonClass}.
* @internal
*/
export function isTreeNodeSchemaClass<
Name extends string,
Expand Down Expand Up @@ -230,6 +231,12 @@ export interface FieldSchemaMetadata<TCustomMetadata = unknown> {
* used as the `description` field.
*/
readonly description?: string | undefined;

/**
* Whether or not to include the field in JSON output generated by `getJsonSchema` (experimental).
* @defaultValue `false`
*/
readonly omitFromJson?: boolean | undefined;
}

/**
Expand Down Expand Up @@ -328,6 +335,8 @@ export class FieldSchema<

/**
* Normalizes a {@link ImplicitFieldSchema} to a {@link FieldSchema}.
*
* @internal
*/
export function normalizeFieldSchema(schema: ImplicitFieldSchema): FieldSchema {
return schema instanceof FieldSchema
Expand All @@ -340,6 +349,8 @@ export function normalizeFieldSchema(schema: ImplicitFieldSchema): FieldSchema {
*
* @remarks Note: this must only be called after all required schemas have been declared, otherwise evaluation of
* recursive schemas may fail.
*
* @internal
*/
export function normalizeAllowedTypes(
types: ImplicitAllowedTypes,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -172,12 +172,21 @@ describe("simpleSchemaToJsonSchema", () => {
"foo": {
kind: FieldKind.Optional,
allowedTypes: new Set<string>(["test.number"]),
description: "A number representing the concept of Foo.",
metadata: { description: "A number representing the concept of Foo." },
},
"bar": {
kind: FieldKind.Required,
allowedTypes: new Set<string>(["test.string"]),
description: "A string representing the concept of Bar.",
metadata: { description: "A string representing the concept of Bar." },
},
"id": {
kind: FieldKind.Identifier,
allowedTypes: new Set<string>(["test.string"]),
metadata: {
description: "Unique identifier for the test object.",
// IDs should be generated by the system. Hide from the JSON Schema.
omitFromJson: true,
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not 100% sure but this (the comment in particular) feels specific to the ai-collab example app use case; in the general case, I expect one would want identifiers to be included in the output JSON. We should definitely have something here that validates omitFromJson, but I think for test purposes it might be better to just make this a required string named "shouldNotBeInJson" with no special comments?

},
},
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ describe("getSimpleSchema", () => {
},
],
]),
metadata: { description: "An optional string." },
allowedTypes: new Set(["com.fluidframework.leaf.string"]),
description: "An optional string.",
};
assert.deepEqual(actual, expected);
});
Expand Down
5 changes: 4 additions & 1 deletion packages/framework/ai-collab/.eslintrc.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@ module.exports = {
parserOptions: {
project: ["./tsconfig.json", "./src/test/tsconfig.json"],
},
rules: {},
rules: {
"unicorn/switch-case-braces": "off",
"unicorn/no-array-for-each": "off",
},
seanimam marked this conversation as resolved.
Show resolved Hide resolved
overrides: [
{
// Rules only for test files
Expand Down
Loading
Loading