Empty compression messages

[andre4i]

Description

We've identified an issue with compression, that the trailing empty ops added to the batch to preserve the batch shape and reserve sequence numbers are not really empty, as the content is still present in the deserializedContent property. This leads to compression creating a large payload which does not end up getting chunked.

This is not actually an issue. When compression is not supported by the loader, the outbox will send the deserializedContent property. When it is, a message will be crafted to only contain contents, metadata, compression and referenceSequenceNumber. So if compression is enabled and supported, the message size will actually be small as the deserializedContent value is discarded. This is still a good change, it will make memory consumption smaller when compression is enabled and supported.

Example of a compressed batch:

with actual content in the messages which are supposed to be empty:

This made the batch exceed 2.5 MB

[vladsud]

It would be great to also have end-to-end test, that validates that we are sending substantially fewer bytes over the wire (compared to initial DDS payload) when dealing with large non-random payloads. Including hitting cases of compression, compression + chunking.

[andre4i]

Ok, we need to keep this for back-compat reasons (old loader submit function requires an object passed as opposed to string contents). We could work around it, but it would add a performance hit as we'd be doing JSON.parse twice for the same message when compression happens this is fine to do for this scenario.

I'll add a note to enhance some of the end-to-end tests to verify the resulting payload size and add more confidence.. (ADO:3499)

[msfluid-bot]

⯅ @fluid-example/bundle-size-tests: +252 Bytes

Metric Name	Baseline Size	Compare Size	Size Diff
aqueduct.js	433.72 KB	433.84 KB	⯅ +126 Bytes
connectionState.js	680 Bytes	680 Bytes	■ No change
containerRuntime.js	228.83 KB	228.95 KB	⯅ +126 Bytes
loader.js	152.93 KB	152.93 KB	■ No change
map.js	43.78 KB	43.78 KB	■ No change
matrix.js	135.95 KB	135.95 KB	■ No change
odspDriver.js	91.79 KB	91.79 KB	■ No change
odspPrefetchSnapshot.js	43.51 KB	43.51 KB	■ No change
sharedString.js	156.5 KB	156.5 KB	■ No change
Total Size	1.36 MB	1.36 MB	⯅ +252 Bytes

---

Baseline commit: 81b5ef9

Generated by 🚫 dangerJS against 34f4181