Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test/internal merge tree client #22697

Draft
wants to merge 31 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
2cc6221
Remove Exposure of Client
anthony-murphy Sep 17, 2024
2129908
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Sep 25, 2024
1c687be
update type tests
anthony-murphy Sep 25, 2024
8d7ef70
fix tests
anthony-murphy Sep 25, 2024
620f950
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Sep 27, 2024
00be065
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 1, 2024
2ae3b9e
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 2, 2024
19691c2
Merge branch 'main' into test/internal-merge-tree-client
anthony-murphy Oct 3, 2024
8b0ecb3
Merge branch 'main' into test/internal-merge-tree-client
anthony-murphy Oct 4, 2024
e374f9a
Merge branch 'main' into test/internal-merge-tree-client
anthony-murphy Oct 4, 2024
1cb5de6
Merge branch 'main' into test/internal-merge-tree-client
anthony-murphy Oct 7, 2024
488b0d2
Merge branch 'main' into test/internal-merge-tree-client
anthony-murphy Oct 7, 2024
441a6a9
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 8, 2024
36ae191
remove some deprecations
anthony-murphy Oct 8, 2024
bc72f79
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 8, 2024
20eb3df
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 10, 2024
f1d2579
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 15, 2024
6444ced
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 17, 2024
887dfca
Add IMergeTreeOptionsInternal
anthony-murphy Oct 17, 2024
f096b9c
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 18, 2024
966a8cd
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 25, 2024
ac52ee6
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 29, 2024
c32cefb
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 29, 2024
a898922
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 30, 2024
9b443b7
fix some imports
anthony-murphy Oct 31, 2024
636b4de
fix up event typing
anthony-murphy Oct 31, 2024
25c42b4
improve the sequence interval interface
anthony-murphy Oct 31, 2024
45b0bc8
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
anthony-murphy Oct 31, 2024
ffc81d4
generate docs
anthony-murphy Oct 31, 2024
541b635
re-enable type tests
anthony-murphy Oct 31, 2024
62564f0
add missing file
anthony-murphy Oct 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 0 additions & 124 deletions packages/dds/merge-tree/api-report/merge-tree.legacy.alpha.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,6 @@
// @alpha
export function appendToMergeTreeDeltaRevertibles(deltaArgs: IMergeTreeDeltaCallbackArgs, revertibles: MergeTreeDeltaRevertible[]): void;

// @alpha @sealed @deprecated
export interface AttributionPolicy {
attach: (client: Client) => void;
detach: () => void;
// (undocumented)
isAttached: boolean;
serializer: IAttributionCollectionSerializer;
}

// @alpha (undocumented)
export abstract class BaseSegment implements ISegment {
constructor(properties?: PropertySet);
Expand Down Expand Up @@ -87,104 +78,6 @@ export abstract class BaseSegment implements ISegment {
wasMovedOnInsert?: boolean | undefined;
}

// @alpha @deprecated (undocumented)
export class Client extends TypedEventEmitter<IClientEvents> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tylerbutler do you know why these type removals don't break the type tests? i would expect those to need suppression, but they don't for some reason.

Copy link
Contributor Author

@anthony-murphy anthony-murphy Oct 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess it makes sense if the look at the imports

import type * as old from "@fluidframework/merge-tree-previous/internal";

import type * as current from "../../index.js";

as the types still exist off internal, just not legacy, so are still available off import type * as current from "../../index.js";

however, i would expect this to be a breaking change, as something that was exported at one layer, is no longer exported.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • @CraigMacomber as type test collaborator.
    I think there was a task out there to test against the entrypoints directly. I think that may have been mutated to read the tags so that all pulled into a single file without any duplication. I still think that using the entrypoints directly has merit. Especially relevant if we hand craft any entrypoints, which would allow deviation from tags.

constructor(specToSegment: (spec: IJSONSegment) => ISegment, logger: ITelemetryLoggerExt, options?: IMergeTreeOptions & PropertySet, getMinInFlightRefSeq?: () => number | undefined);
// (undocumented)
addLongClientId(longClientId: string): void;
annotateMarker(marker: Marker, props: PropertySet): IMergeTreeAnnotateMsg | undefined;
annotateRangeLocal(start: number, end: number, props: PropertySet): IMergeTreeAnnotateMsg | undefined;
// (undocumented)
applyMsg(msg: ISequencedDocumentMessage, local?: boolean): void;
// (undocumented)
applyStashedOp(op: IMergeTreeOp): void;
createLocalReferencePosition(segment: ISegment | "start" | "end", offset: number | undefined, refType: ReferenceType, properties: PropertySet | undefined, slidingPreference?: SlidingPreference, canSlideToEndpoint?: boolean): LocalReferencePosition;
// (undocumented)
createTextHelper(): IMergeTreeTextHelper;
findReconnectionPosition(segment: ISegment, localSeq: number): number;
// (undocumented)
getClientId(): number;
// (undocumented)
getCollabWindow(): CollaborationWindow;
// (undocumented)
getContainingSegment<T extends ISegment>(pos: number, sequenceArgs?: Pick<ISequencedDocumentMessage, "referenceSequenceNumber" | "clientId">, localSeq?: number): {
segment: T | undefined;
offset: number | undefined;
};
// (undocumented)
getCurrentSeq(): number;
// (undocumented)
getLength(): number;
// (undocumented)
getLongClientId(shortClientId: number): string;
// (undocumented)
getMarkerFromId(id: string): ISegment | undefined;
// (undocumented)
getOrAddShortClientId(longClientId: string): number;
getPosition(segment: ISegment | undefined, localSeq?: number): number;
// (undocumented)
getPropertiesAtPosition(pos: number): PropertySet | undefined;
// (undocumented)
getRangeExtentsOfPosition(pos: number): {
posStart: number | undefined;
posAfterEnd: number | undefined;
};
// (undocumented)
protected getShortClientId(longClientId: string): number;
insertAtReferencePositionLocal(refPos: ReferencePosition, segment: ISegment): IMergeTreeInsertMsg | undefined;
insertSegmentLocal(pos: number, segment: ISegment): IMergeTreeInsertMsg | undefined;
// (undocumented)
load(runtime: IFluidDataStoreRuntime, storage: IChannelStorageService, serializer: IFluidSerializer): Promise<{
catchupOpsP: Promise<ISequencedDocumentMessage[]>;
}>;
localReferencePositionToPosition(lref: ReferencePosition): number;
// (undocumented)
localTransaction(groupOp: IMergeTreeGroupMsg): void;
// (undocumented)
readonly logger: ITelemetryLoggerExt;
// (undocumented)
longClientId: string | undefined;
obliterateRangeLocal(start: number | InteriorSequencePlace, end: number | InteriorSequencePlace): IMergeTreeObliterateMsg | IMergeTreeObliterateSidedMsg;
peekPendingSegmentGroups(): SegmentGroup | undefined;
// (undocumented)
peekPendingSegmentGroups(count: number): SegmentGroup | SegmentGroup[] | undefined;
posFromRelativePos(relativePos: IRelativePosition): number;
regeneratePendingOp(resetOp: IMergeTreeOp, segmentGroup: SegmentGroup | SegmentGroup[]): IMergeTreeOp;
removeLocalReferencePosition(lref: LocalReferencePosition): LocalReferencePosition | undefined;
removeRangeLocal(start: number, end: number): IMergeTreeRemoveMsg;
resolveRemoteClientPosition(remoteClientPosition: number, remoteClientRefSeq: number, remoteClientId: string): number | undefined;
rollback?(op: unknown, localOpMetadata: unknown): void;
searchForMarker(startPos: number, markerLabel: string, forwards?: boolean): Marker | undefined;
serializeGCData(handle: IFluidHandle, handleCollectingSerializer: IFluidSerializer): void;
// (undocumented)
readonly specToSegment: (spec: IJSONSegment) => ISegment;
// (undocumented)
startOrUpdateCollaboration(longClientId: string | undefined, minSeq?: number, currentSeq?: number): void;
// (undocumented)
summarize(runtime: IFluidDataStoreRuntime, handle: IFluidHandle, serializer: IFluidSerializer, catchUpMsgs: ISequencedDocumentMessage[]): ISummaryTreeWithStats;
// (undocumented)
updateMinSeq(minSeq: number): void;
// (undocumented)
protected walkAllSegments<TClientData>(action: (segment: ISegment, accum?: TClientData) => boolean, accum?: TClientData): boolean;
// (undocumented)
walkSegments<TClientData>(handler: ISegmentAction<TClientData>, start: number | undefined, end: number | undefined, accum: TClientData, splitRange?: boolean): void;
// (undocumented)
walkSegments<undefined>(handler: ISegmentAction<undefined>, start?: number, end?: number, accum?: undefined, splitRange?: boolean): void;
}

// @alpha @deprecated (undocumented)
export class CollaborationWindow {
// (undocumented)
clientId: number;
// (undocumented)
collaborating: boolean;
currentSeq: number;
// (undocumented)
loadFrom(a: CollaborationWindow): void;
localSeq: number;
minSeq: number;
}

// @alpha
export function discardMergeTreeDeltaRevertible(revertibles: MergeTreeDeltaRevertible[]): void;

Expand Down Expand Up @@ -244,16 +137,6 @@ export interface IAttributionCollectionSpec<T> {
}>;
}

// @alpha @deprecated
export interface IClientEvents {
// (undocumented)
(event: "normalize", listener: (target: IEventThisPlaceHolder) => void): void;
// (undocumented)
(event: "delta", listener: (opArgs: IMergeTreeDeltaOpArgs, deltaArgs: IMergeTreeDeltaCallbackArgs, target: IEventThisPlaceHolder) => void): void;
// (undocumented)
(event: "maintenance", listener: (args: IMergeTreeMaintenanceCallbackArgs, deltaArgs: IMergeTreeDeltaOpArgs | undefined, target: IEventThisPlaceHolder) => void): void;
}

// @alpha (undocumented)
export interface IJSONMarkerSegment extends IJSONSegment {
// (undocumented)
Expand Down Expand Up @@ -302,12 +185,6 @@ export interface IMergeTreeAnnotateMsg extends IMergeTreeDelta {
type: typeof MergeTreeDeltaType.ANNOTATE;
}

// @alpha @deprecated (undocumented)
export interface IMergeTreeAttributionOptions {
policyFactory?: () => AttributionPolicy;
track?: boolean;
}

// @alpha (undocumented)
export interface IMergeTreeDelta {
type: MergeTreeDeltaType;
Expand Down Expand Up @@ -392,7 +269,6 @@ export type IMergeTreeOp = IMergeTreeDeltaOp | IMergeTreeGroupMsg;

// @alpha (undocumented)
export interface IMergeTreeOptions {
attribution?: IMergeTreeAttributionOptions;
// (undocumented)
catchUpBlobName?: string;
mergeTreeEnableObliterate?: boolean;
Expand Down
18 changes: 9 additions & 9 deletions packages/dds/merge-tree/src/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,11 @@ import { MergeTreeTextHelper } from "./MergeTreeTextHelper.js";
import { DoublyLinkedList, RedBlackTree } from "./collections/index.js";
import { UnassignedSequenceNumber, UniversalSequenceNumber } from "./constants.js";
import { LocalReferencePosition, SlidingPreference } from "./localReference.js";
import { IMergeTreeOptions, MergeTree, errorIfOptionNotTrue } from "./mergeTree.js";
import {
MergeTree,
errorIfOptionNotTrue,
type IMergeTreeOptionsInternal,
} from "./mergeTree.js";
import type {
IMergeTreeClientSequenceArgs,
IMergeTreeDeltaCallbackArgs,
Expand Down Expand Up @@ -100,9 +104,7 @@ export interface IIntegerRange {
* Emitted before this client's merge-tree normalizes its segments on reconnect, potentially
* ordering them. Useful for DDS-like consumers built atop the merge-tree to compute any information
* they need for rebasing their ops on reconnection.
* @legacy
* @alpha
* @deprecated This functionality was not meant to be exported and will be removed in a future release
* @internal
*/
export interface IClientEvents {
(event: "normalize", listener: (target: IEventThisPlaceHolder) => void): void;
Expand All @@ -125,9 +127,7 @@ export interface IClientEvents {
}

/**
* @deprecated This functionality was not meant to be exported and will be removed in a future release
* @legacy
* @alpha
* @internal
*/
export class Client extends TypedEventEmitter<IClientEvents> {
public longClientId: string | undefined;
Expand Down Expand Up @@ -155,7 +155,7 @@ export class Client extends TypedEventEmitter<IClientEvents> {
constructor(
public readonly specToSegment: (spec: IJSONSegment) => ISegment,
public readonly logger: ITelemetryLoggerExt,
options?: IMergeTreeOptions & PropertySet,
options?: IMergeTreeOptionsInternal & PropertySet,
private readonly getMinInFlightRefSeq: () => number | undefined = (): undefined =>
undefined,
) {
Expand Down Expand Up @@ -322,7 +322,7 @@ export class Client extends TypedEventEmitter<IClientEvents> {
accum: TClientData,
splitRange?: boolean,
): void;
public walkSegments<undefined>(
public walkSegments(
handler: ISegmentAction<undefined>,
start?: number,
end?: number,
Expand Down
1 change: 1 addition & 0 deletions packages/dds/merge-tree/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ export {
AttributionPolicy,
IMergeTreeAttributionOptions,
IMergeTreeOptions,
IMergeTreeOptionsInternal,
getSlideToSegoff,
} from "./mergeTree.js";
export {
Expand Down
28 changes: 14 additions & 14 deletions packages/dds/merge-tree/src/mergeTree.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ import { assert, Heap, IComparer } from "@fluidframework/core-utils/internal";
import { DataProcessingError, UsageError } from "@fluidframework/telemetry-utils/internal";

import { IAttributionCollectionSerializer } from "./attributionCollection.js";
// eslint-disable-next-line import/no-deprecated
import { Client } from "./client.js";
import { DoublyLinkedList, ListNode } from "./collections/index.js";
import {
Expand Down Expand Up @@ -177,11 +176,6 @@ export interface IMergeTreeOptions {
*/
newMergeTreeSnapshotFormat?: boolean;

/**
* Options related to attribution
*/
attribution?: IMergeTreeAttributionOptions;

/**
* Enables support for the obliterate operation -- a stronger form of remove
* which deletes concurrently inserted segments
Expand Down Expand Up @@ -212,6 +206,17 @@ export interface IMergeTreeOptions {
*/
mergeTreeEnableSidedObliterate?: boolean;
}

/**
* @internal
*/
export interface IMergeTreeOptionsInternal extends IMergeTreeOptions {
/**
* Options related to attribution
*/
attribution?: IMergeTreeAttributionOptions;
}

export function errorIfOptionNotTrue(
options: IMergeTreeOptions | undefined,
option: keyof IMergeTreeOptions,
Expand All @@ -222,9 +227,7 @@ export function errorIfOptionNotTrue(
}

/**
* @legacy
* @alpha
* @deprecated This functionality was not meant to be exported and will be removed in a future release
* @internal
*/
export interface IMergeTreeAttributionOptions {
/**
Expand All @@ -250,9 +253,7 @@ export interface IMergeTreeAttributionOptions {
/**
* Implements policy dictating which kinds of operations should be attributed and how.
* @sealed
* @legacy
* @alpha
* @deprecated This functionality was not meant to be exported and will be removed in a future release
* @internal
*/
export interface AttributionPolicy {
/**
Expand All @@ -262,7 +263,6 @@ export interface AttributionPolicy {
*
* This must be done in an eventually consistent fashion.
*/
// eslint-disable-next-line import/no-deprecated
attach: (client: Client) => void;
/**
* Disables tracking attribution information on segments.
Expand Down Expand Up @@ -527,7 +527,7 @@ export class MergeTree {

private readonly obliterates = new Obliterates(this);

public constructor(public options?: IMergeTreeOptions) {
public constructor(public options?: IMergeTreeOptionsInternal) {
this._root = this.makeBlock(0);
this._root.mergeTree = this;
this.attributionPolicy = options?.attribution?.policyFactory?.();
Expand Down
6 changes: 1 addition & 5 deletions packages/dds/merge-tree/src/mergeTreeNodes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,6 @@ export interface ISegment extends IMergeNodeCommon, Partial<IRemovalInfo>, Parti
* This is defined if and only if the insertion of the segment is pending ack, i.e. `seq` is UnassignedSequenceNumber.
* Once the segment is acked, this field is cleared.
*
* See {@link CollaborationWindow.localSeq} for more information on the semantics of localSeq.
*/
localSeq?: number;
/**
Expand All @@ -274,7 +273,6 @@ export interface ISegment extends IMergeNodeCommon, Partial<IRemovalInfo>, Parti
* will be updated to the seq at which that client removed this segment.
*
* Like {@link ISegment.localSeq}, this field is cleared once the local removal of the segment is acked.
* See {@link CollaborationWindow.localSeq} for more information on the semantics of localSeq.
*/
localRemovedSeq?: number;
/**
Expand Down Expand Up @@ -896,9 +894,7 @@ export class Marker extends BaseSegment implements ReferencePosition, ISegment {
}

/**
* @deprecated This functionality was not meant to be exported and will be removed in a future release
* @legacy
* @alpha
* @internal
*/
export class CollaborationWindow {
clientId = LocalClientId;
Expand Down
7 changes: 6 additions & 1 deletion packages/dds/merge-tree/src/test/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,12 @@ export {
runMergeTreeOperationRunner,
TestOperation,
} from "./mergeTreeOperationRunner.js";
export { LRUSegment, MergeTree } from "../mergeTree.js";
export {
LRUSegment,
MergeTree,
IMergeTreeOptions,
IMergeTreeOptionsInternal,
} from "../mergeTree.js";
export { MergeTreeTextHelper } from "../MergeTreeTextHelper.js";
export { SnapshotLegacy } from "../snapshotlegacy.js";
export {
Expand Down
4 changes: 2 additions & 2 deletions packages/dds/merge-tree/src/test/snapshot.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ import {
createInsertOnlyAttributionPolicy,
createPropertyTrackingAttributionPolicyFactory,
} from "../attributionPolicy.js";
import { IMergeTreeOptions } from "../mergeTree.js";
import { IMergeTreeOptionsInternal } from "../mergeTree.js";
import { SnapshotV1 } from "../snapshotV1.js";

import { TestString, loadSnapshot } from "./snapshot.utils.js";

function makeSnapshotSuite(options?: IMergeTreeOptions): void {
function makeSnapshotSuite(options?: IMergeTreeOptionsInternal): void {
describe("from an empty initial state", () => {
let str: TestString;
beforeEach(() => {
Expand Down
8 changes: 4 additions & 4 deletions packages/dds/merge-tree/src/test/snapshot.utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import { ISummaryTree } from "@fluidframework/driver-definitions";
import { ISequencedDocumentMessage } from "@fluidframework/driver-definitions/internal";
import { MockStorage } from "@fluidframework/test-runtime-utils/internal";

import { IMergeTreeOptions } from "../mergeTree.js";
import { type IMergeTreeOptionsInternal } from "../mergeTree.js";
import { ISegment } from "../mergeTreeNodes.js";
import { IMergeTreeOp, ReferenceType } from "../ops.js";
import { PropertySet } from "../properties.js";
Expand All @@ -25,7 +25,7 @@ import { TestSerializer } from "./testSerializer.js";
// Reconstitutes a MergeTree client from a summary
export async function loadSnapshot(
summary: ISummaryTree,
options?: IMergeTreeOptions,
options?: IMergeTreeOptionsInternal,
): Promise<TestClient> {
const services = MockStorage.createFromSummary(summary);
const client2 = new TestClient(options);
Expand All @@ -52,7 +52,7 @@ export class TestString {

constructor(
id: string,
private readonly options?: IMergeTreeOptions,
private readonly options?: IMergeTreeOptionsInternal,
initialState: string = "",
) {
this.client = createClientsAtInitialState({ initialState, options }, id)[id];
Expand Down Expand Up @@ -109,7 +109,7 @@ export class TestString {
}

// Ensures the MergeTree client's contents successfully roundtrip through a snapshot.
public async checkSnapshot(options?: IMergeTreeOptions): Promise<void> {
public async checkSnapshot(options?: IMergeTreeOptionsInternal): Promise<void> {
this.applyPendingOps();
const expectedAttributionKeys = this.client.getAllAttributionSeqs();
const summary = this.getSummary();
Expand Down
Loading
Loading