Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make copying optional when receiving data #428

Merged
merged 10 commits into from
Oct 31, 2016

Conversation

erikprice
Copy link
Contributor

@erikprice erikprice commented Jun 28, 2016

For context, see #424, #426, and #427.

Starting the conversation again on making copying of received frame data optional.

After looking deeper into the calls to handleFrameWithData:opCode: - there are numerous cases, where reading from that data would be incredibly unsafe, since we mutate the data and NSData is not thread-safe.I can see the value from not copying the incoming data, but at the same time using un thread-safe data can lead to crazy crazy problems on the receiving end and at the current moment.

Can you elaborate a bit on where the mutation occurs? I scanned the code, and although the data is passed to numerous delegate methods (any one of which could mutate the data), I don't see where the library itself does so. But I'm probably just not seeing it.

@nlutsenko
Copy link
Contributor

Can you elaborate a bit on where the mutation occurs? I scanned the code, and although the data is passed to numerous delegate methods (any one of which could mutate the data), I don't see where the library itself does so. But I'm probably just not seeing it.

Absolutely, here is the spot where we mutate the currentFrameData.

There is an interesting thing here - it looks like we are safe at not doing any copying for the data, unless it's a control frame, because then we are going to continue adding more data to the current frame data.
I wonder if we remove it completely and only pass in a copy of non-current frame data, or current frame data directly - whether that will work ok or not.

@erikprice
Copy link
Contributor Author

Is it really as simple as the small change in 6ba8526?

@nlutsenko
Copy link
Contributor

Yes, this is what it looks like. Would love to hear if this solution actually works on a big scale and doesn't cause any problems.

@nlutsenko
Copy link
Contributor

Please rebase and let's test it heavily together.

@erikprice
Copy link
Contributor Author

Thanks for your help with this. We've updated our app to use this new branch. We'll test in dev and QA for some period of time, and if all goes well there, the changes will end up in our next App Store release. If you leave this PR open, I will follow up here in about a month or so, assuming we don't run into problems before then.

@erikprice
Copy link
Contributor Author

Also, I assumed this project prefers merge instead of rebase (I merged before I noticed your response), but if you want me to rebase the branch against master instead, just let me know.

@nlutsenko
Copy link
Contributor

merge vs rebase

The way we do it is weird :) As much as possible I rebase the branches, so everything stays in sync and is well-tested on the latest master.
The moment we are landing something onto the master if it contains more than 1 commit and they are concrete - it's merge, otherwise - it's squash/rebase and push.

If you leave this PR open, I will follow up here in about a month or so, assuming we don't run into problems before then.

Sounds perfect! I am also going to try doing some stress testing on this, so we attack this from both ends. If you encounter any issues - please follow-up here. Going to remove review-needed and just keep the PR opened so we can make sure everything is stable before merging.

@erikprice
Copy link
Contributor Author

Absolutely, here is the spot where we mutate the currentFrameData.

Permalink, for posterity (mainly for my own reference).

@erikprice
Copy link
Contributor Author

Just wanted to update this issue: it's been a little over a month, and these changes have gone through our QA and been shipped in two App Store releases. So far, we haven't seen any reason to suspect that they cause problems.

Copy link
Contributor

@nlutsenko nlutsenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this is good to go!
Amazing performance and memory consumption improvement!
Thank you for bearing with me through the process!

@nlutsenko nlutsenko self-assigned this Oct 31, 2016
@nlutsenko nlutsenko merged commit 877ac74 into facebookincubator:master Oct 31, 2016
@nlutsenko nlutsenko added this to the 0.6.0 milestone Oct 31, 2016
Mikefluff pushed a commit to Mikefluff/SocketRocket that referenced this pull request Nov 10, 2016
* Reduce memory usage

Reduce memory usage by discarding, not resetting, the frame data buffer.
Let delegate control copying.

* Delegate methods expect `self`

* Add `-sendWithoutCopyingData:error:`

* Add `-webSocket:shouldCopyReceivedData:`

* Fix error messages

* Remove `-webSocket:shouldCopyReceivedData:`

* Revert "Fix error messages"

This reverts commit 4d5f5f0.

* Fix typo

* Copy only control frames

* Fix error message
madlymad pushed a commit to madlymad/SocketRocket that referenced this pull request Jul 6, 2017
* Reduce memory usage

Reduce memory usage by discarding, not resetting, the frame data buffer.
Let delegate control copying.

* Delegate methods expect `self`

* Add `-sendWithoutCopyingData:error:`

* Add `-webSocket:shouldCopyReceivedData:`

* Fix error messages

* Remove `-webSocket:shouldCopyReceivedData:`

* Revert "Fix error messages"

This reverts commit 4d5f5f0.

* Fix typo

* Copy only control frames

* Fix error message
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants