Skip to content

Commit

Permalink
Introducing DDS doc feedback (microsoft#3523)
Browse files Browse the repository at this point in the history
  • Loading branch information
tylerbutler committed Sep 4, 2020
1 parent 2da5d83 commit 89514ba
Show file tree
Hide file tree
Showing 6 changed files with 88 additions and 42 deletions.
4 changes: 4 additions & 0 deletions docs/_includes/links.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
<!-- Links -->

<!-- Packages -->

[Aqueduct]: {{< relref "/apis/aqueduct.md" >}}
[undo-redo]: {{< relref "/apis/undo-redo.md" >}}

<!-- Classes and interfaces -->

[ContainerRuntimeFactoryWithDefaultDataStore]: {{< relref "/apis/aqueduct/containerruntimefactorywithdefaultdatastore.md" >}}
[DataObject]: {{< relref "/apis/aqueduct/dataobject.md" >}}
[DataObjectFactory]: {{< relref "/apis/aqueduct/dataobjectfactory.md" >}}
Expand All @@ -16,10 +18,12 @@
[SharedMatrix]: {{< relref "SharedMatrix" >}}
[SharedNumberSequence]: {{< relref "SharedNumberSequence" >}}
[SharedObjectSequence]: {{< relref "/apis/sequence/sharedobjectsequence.md" >}}
[SharedSequence]: {{< relref "SharedSequence" >}}
[SharedString]: {{< relref "SharedString" >}}
[Quorum]: {{< relref "/apis/protocol-base/quorum.md" >}}

<!-- Sequence methods -->

[sequence.insert]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-insert-Method" >}}
[sequence.getItems]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
[sequence.remove]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
Expand Down
4 changes: 4 additions & 0 deletions docs/content/docs/advanced/feature-detection-iprovide.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,12 @@ object of the correct type or `undefined`.
<!-- Links -->
<!-- Packages -->
[Aqueduct]: {{< relref "/apis/aqueduct.md" >}}
[undo-redo]: {{< relref "/apis/undo-redo.md" >}}
<!-- Classes and interfaces -->
[ContainerRuntimeFactoryWithDefaultDataStore]: {{< relref "/apis/aqueduct/containerruntimefactorywithdefaultdatastore.md" >}}
[DataObject]: {{< relref "/apis/aqueduct/dataobject.md" >}}
[DataObjectFactory]: {{< relref "/apis/aqueduct/dataobjectfactory.md" >}}
Expand All @@ -104,10 +106,12 @@ object of the correct type or `undefined`.
[SharedMatrix]: {{< relref "SharedMatrix" >}}
[SharedNumberSequence]: {{< relref "SharedNumberSequence" >}}
[SharedObjectSequence]: {{< relref "/apis/sequence/sharedobjectsequence.md" >}}
[SharedSequence]: {{< relref "SharedSequence" >}}
[SharedString]: {{< relref "SharedString" >}}
[Quorum]: {{< relref "/apis/protocol-base/quorum.md" >}}
<!-- Sequence methods -->
[sequence.insert]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-insert-Method" >}}
[sequence.getItems]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
[sequence.remove]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
Expand Down
4 changes: 4 additions & 0 deletions docs/content/docs/concepts/dataobject-aqueduct.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,10 +168,12 @@ The Aqueduct library contains more than just DataObject and DataObjectFactory. T
<!-- Links -->

<!-- Packages -->

[Aqueduct]: {{< relref "/apis/aqueduct.md" >}}
[undo-redo]: {{< relref "/apis/undo-redo.md" >}}

<!-- Classes and interfaces -->

[ContainerRuntimeFactoryWithDefaultDataStore]: {{< relref "/apis/aqueduct/containerruntimefactorywithdefaultdatastore.md" >}}
[DataObject]: {{< relref "/apis/aqueduct/dataobject.md" >}}
[DataObjectFactory]: {{< relref "/apis/aqueduct/dataobjectfactory.md" >}}
Expand All @@ -183,10 +185,12 @@ The Aqueduct library contains more than just DataObject and DataObjectFactory. T
[SharedMatrix]: {{< relref "SharedMatrix" >}}
[SharedNumberSequence]: {{< relref "SharedNumberSequence" >}}
[SharedObjectSequence]: {{< relref "/apis/sequence/sharedobjectsequence.md" >}}
[SharedSequence]: {{< relref "SharedSequence" >}}
[SharedString]: {{< relref "SharedString" >}}
[Quorum]: {{< relref "/apis/protocol-base/quorum.md" >}}

<!-- Sequence methods -->

[sequence.insert]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-insert-Method" >}}
[sequence.getItems]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
[sequence.remove]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
Expand Down
105 changes: 67 additions & 38 deletions docs/content/docs/concepts/dds.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,22 +20,20 @@ etc. This prefix indicates that the object is shared between multiple clients.

{{% /callout %}}

When a DDS is changed by another client, it raises an [event](#events) locally. Your code can listen to these events so
that you know when data is changed and can react appropriately. For example, you may need to recalculate a derived value
when some data in a DDS changes.

When a DDS is changed by any client, it raises an [event](#events) locally. Your code can listen to these events so that
you know when data is changed and can react appropriately. For example, you may need to recalculate a derived value when
some data in a DDS changes.

## Merge behavior

In a distributed system like Fluid, it is critical to understand how to merge changes from multiple clients because it
enables you to "preserve user intent" when users are collaborating on data. This means that the merge behavior should
match what users intend or expect as they are editing data.
In a distributed system like Fluid, it is critical to understand how changes from multiple clients are merged.
Understanding the merge logic enables you to "preserve user intent" when users are collaborating on data. This means
that the merge behavior should match what users intend or expect as they are editing data.

In Fluid, the merge behavior is defined by the DDS. The simplest merge strategy, employed by key-value distributed data
structures like SharedMap, is _last writer wins_ (LWW). With this merge strategy, when multiple clients write different
values to the same key, the value that was written last will overwrite the others. Refer to the API documentation for
each DDS for more details about the merge strategy it uses.

values to the same key, the value that was written last will overwrite the others. Refer to the
[API documentation]({{< relref "/apis" >}}) for each DDS for more details about the merge strategy it uses.

## Performance characteristics

Expand All @@ -60,14 +58,13 @@ Many of the most commonly used DDSes are optimistic, including [SharedMap][], [S

### Consensus-based data structures

Clients must always assume their local DDS state is stale since there may be changes from remote clients that they have
not yet received. In cases where you can only safely update data if you have the latest in-hand, Fluid provides
consensus-based data structures. These data structures are different from optimistic DDSes because they wait to get
confirmation from the Fluid service before apply operations -- even local operations.
Consensus-based DDSes are different from optimistic DDSes because they wait for confirmation from the Fluid service
before applying operations -- even local operations. These data structures offer additional behavior guarantees and can
be used when you need atomicity or synchronous behavior.

However, there are cases where you require behavioral guarantees that cannot be implemented in an optimistic way. The
cost is performance; optimistic DDSes are part of what makes Fluid so fast, so using optimistic DDSes is almost always
preferred, but you can trade performance for behavioral guarantees.
These behavioral guarantees cannot be implemented in an optimistic way. The cost is performance; optimistic DDSes are
part of what makes Fluid so fast, so using optimistic DDSes is almost always preferred, but you can trade performance
for behavioral guarantees.

#### Why consensus-based DDSes are useful

Expand All @@ -93,7 +90,6 @@ sequence number to the operation before applying it locally. With this approach,
local changes until they get back a sequenced op from the server. Once they do, they apply the ops in order, which
results in consistent behavior across all remote clients.


## Creating and storing distributed data structures

A distributed data structure object is created using its type's static `create` method.
Expand All @@ -105,7 +101,6 @@ const myMap = SharedMap.create(this.runtime);
You must pass in an `IFluidDataStoreRuntime` that the DDS will be managed by. We'll cover the runtime in more detail in
the [Encapsulating data with DataObject](./dataobject-aqueduct.md) section.


### Storing a DDS within another DDS

Distributed data structures can store primitive values like numbers and strings and _JSON serializable_ objects. For
Expand All @@ -128,7 +123,6 @@ myMap.set("counter", myCounter.handle);
That's all you need to know about handles in order to use DDSes effectively. If you want to learn more about handles,
see [Fluid handles](../advanced/handles.md) in the Advanced section.


## Events

When a distributed data structure is changed by the Fluid runtime, it raises events. You can listen to these events so
Expand All @@ -137,13 +131,12 @@ recalculate a derived value when some data in a DDS changes.

```ts
myMap.on("valueChanged", () => {
recalculate();
recalculate();
});
```

Refer to the sections below for more details about the events raised by each DDS.


## Picking the right data structure

Because distributed data structures can be stored within each other, you can combine DDSes to create collaborative data
Expand All @@ -159,52 +152,86 @@ length, width, etc.
When users edit this data, what pieces of the data can be edited simultaneously? This is an important question to answer
because it influences how you structure the data in your DDSes.

Let's assume for a moment that all of the data about a shape is stored as a single JSON object in a SharedMap. Recall
that the [SharedMap uses a last writer wins merge strategy](#merge-behavior). This means that if two users are editing
the data at the same time, then the one who made the most recent change will overwrite the changes made by the other
user.
Let's assume for a moment that all of the data about a shape is stored as a single object that looks like this:

This may be perfectly fine for your needs. However, if your scenario requires users to edit individual properties of the
shape, then the SharedMap LWW merge strategy probably won't give you the behavior you want.
```json
{
"x": 0,
"y": 0,
"height": 60,
"width": 40
}
```

If we want to make this data collaborative using Fluid, the most direct -- _but ultimately flawed_ -- approach is to
store our shape object in a SharedMap. Our SharedMap would look something like this:

```json
{
"aShape": {
"x": 0,
"y": 0,
"height": 60,
"width": 40
}
}
```

Recall that the [SharedMap uses a last writer wins merge strategy](#merge-behavior). This means that if two users are
editing the data at the same time, then the one who made the most recent change will overwrite the changes made by the
other user.

Imagine that you are collaborating with a colleague, and you change the shape's width while your colleague changes the
shape's height. This will generate two operations: a `set` operation for you, and another `set` operation for your
colleague. Both operations will be sequenced by the Fluid service, but only one will 'win,' because the SharedMap's
merge behavior is LWW. Since we're storing the shape as an object, both `set` operations _set the whole object_.

This results in someone's changes being "lost" from a user's perspective. This may be perfectly fine for your needs.
However, if your scenario requires users to edit individual properties of the shape, then the SharedMap LWW merge
strategy probably won't give you the behavior you want.

However, you could address this problem by storing individual shape properties in SharedMap keys. Instead of storing a
JSON object with all the data, you can break it apart and store the length in one SharedMap key, the color in another,
JSON object with all the data, you can break it apart and store the length in one SharedMap key, the width in another,
etc. With this data model, users can change individual properties of the shape without overwriting other users' changes.

You likely have more than one shape in your data model, so you could create a SharedMap to store all your shapes, then
store the SharedMaps representing each shape within that SharedMap.

To learn more about how you use DDSes to create more complex Fluid objects, see the [Encapsulating data with
DataObject](./dataobject-aqueduct.md) section.

### Key-value data

These DDSes are used for storing key-value data. They are all optimistic, and use a last-writer-wins merge policy.
These DDSes are used for storing key-value data. They are all optimistic and use a last-writer-wins merge policy.

- [SharedMap][] -- a basic key-value distributed data structure.
- [SharedDirectory][] -- a SharedMap with and API more suited to hierarchical data.
- [SharedDirectory][] -- a SharedMap with an API more suited to hierarchical data.
- [SharedCell][] -- a "single-object SharedMap"; useful for wrapping objects.

### Sequences

These DDSes are used for storing sequential data.
These DDSes are used for storing sequential data. They are all optimistic.

- [SharedNumberSequence][]
- [SharedObjectSequence][]
- [SharedMatrix][]
- [SharedNumberSequence][] -- a distributed sequence of numbers.
- [SharedObjectSequence][] -- a distributed sequence of objects.
- [SharedMatrix][] -- a distributed data structure to efficiently use two-dimensional tabular data.

### Specialized data structures

- [SharedCounter][]
- [SharedString][]
- [Ink][]
- [SharedCounter][] -- a distributed counter.
- [SharedString][] -- specialized data structure for handling collaborative text.
- [Ink][] -- specialized daa structure for ink data.

<!-- AUTO-GENERATED-CONTENT:START (INCLUDE:path=_includes/links.md) -->
<!-- Links -->

<!-- Packages -->

[Aqueduct]: {{< relref "/apis/aqueduct.md" >}}
[undo-redo]: {{< relref "/apis/undo-redo.md" >}}

<!-- Classes and interfaces -->

[ContainerRuntimeFactoryWithDefaultDataStore]: {{< relref "/apis/aqueduct/containerruntimefactorywithdefaultdatastore.md" >}}
[DataObject]: {{< relref "/apis/aqueduct/dataobject.md" >}}
[DataObjectFactory]: {{< relref "/apis/aqueduct/dataobjectfactory.md" >}}
Expand All @@ -216,10 +243,12 @@ These DDSes are used for storing sequential data.
[SharedMatrix]: {{< relref "SharedMatrix" >}}
[SharedNumberSequence]: {{< relref "SharedNumberSequence" >}}
[SharedObjectSequence]: {{< relref "/apis/sequence/sharedobjectsequence.md" >}}
[SharedSequence]: {{< relref "SharedSequence" >}}
[SharedString]: {{< relref "SharedString" >}}
[Quorum]: {{< relref "/apis/protocol-base/quorum.md" >}}

<!-- Sequence methods -->

[sequence.insert]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-insert-Method" >}}
[sequence.getItems]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
[sequence.remove]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
Expand Down
9 changes: 5 additions & 4 deletions docs/content/docs/concepts/server.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,12 @@ We sometimes refer to this as "op routing;" this is the source of the name _Rout
## Summaries

Summaries are a serialized form of a Fluid document, created by consolidating all operations and serializing the data
model. Summaries are used to improve load performance. When a Fluid document is loaded, the service may send a summary to
the client so that the client does not need to replay all ops locally to get to the current state.
model. Summaries are used to improve load performance. When a Fluid document is loaded, the service may send a summary
to the client so that the client does not need to replay all ops locally to get to the current state.

Summaries are created on one of the clients, called the "leader", and sent to the service like any other operation. To
learn more about summaries and how they are created, see the [advanced Summarizer topic](../advanced/summarizer.md).
One of the connected clients is chosen to generate the summary. Once the summary is created it is sent to the service
like any other operation. To learn more about summaries and how they are created, see the [advanced Summarizer
topic](../advanced/summarizer.md).


## Drivers
Expand Down
4 changes: 4 additions & 0 deletions docs/content/docs/get-started/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,10 +210,12 @@ At this point we can run our app. The [full code for this application is availa
<!-- Links -->

<!-- Packages -->

[Aqueduct]: {{< relref "/apis/aqueduct.md" >}}
[undo-redo]: {{< relref "/apis/undo-redo.md" >}}

<!-- Classes and interfaces -->

[ContainerRuntimeFactoryWithDefaultDataStore]: {{< relref "/apis/aqueduct/containerruntimefactorywithdefaultdatastore.md" >}}
[DataObject]: {{< relref "/apis/aqueduct/dataobject.md" >}}
[DataObjectFactory]: {{< relref "/apis/aqueduct/dataobjectfactory.md" >}}
Expand All @@ -225,10 +227,12 @@ At this point we can run our app. The [full code for this application is availa
[SharedMatrix]: {{< relref "SharedMatrix" >}}
[SharedNumberSequence]: {{< relref "SharedNumberSequence" >}}
[SharedObjectSequence]: {{< relref "/apis/sequence/sharedobjectsequence.md" >}}
[SharedSequence]: {{< relref "SharedSequence" >}}
[SharedString]: {{< relref "SharedString" >}}
[Quorum]: {{< relref "/apis/protocol-base/quorum.md" >}}

<!-- Sequence methods -->

[sequence.insert]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-insert-Method" >}}
[sequence.getItems]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
[sequence.remove]: {{< relref "/apis/sequence/sharedsequence.md#sequence-sharedsequence-getitems-Method" >}}
Expand Down

0 comments on commit 89514ba

Please sign in to comment.