Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate multi-line-bash-related sandbox tests into runtime tests and fix multi-line issue #3128

Merged
merged 44 commits into from
Jul 27, 2024

Conversation

xingyaoww
Copy link
Contributor

@xingyaoww xingyaoww commented Jul 26, 2024

What is the problem that this fixes or functionality that this introduces? Does it fix any open issues?

We are almost done with the arch refactor for Runtime (#2404). Right now we need to make sure the new runtime has feature parity with existing default ServerRuntime before we are ready to switch over to the new architecture.

This also fixes #3143


Give a summary of what the PR does, explaining any non-trivial design decisions

  • Move all test_sandbox.py tests into test_runtime.py to test both Runtime implementation (ServerRuntime, EventStreamRuntime). It turns out there's too many things to change, it will be easier for code reviewer if i break this down into separate PRs.
  • This PR migrates all the multi-line bash command tests from test_sandbox to test_runtime.
  • Support multi-line bash command handling for EventStreamRuntime
  • Set PS2 to empty, so there's no weird ">" after multi-line bash command execution
  • Use bashlex for bash parsing to handle multi-line commands;
  • Add test cases for multi-line commands, specifically reproduce the issue in [Bug]: Multi-line bash commands time out? #3143
  • Add runtime tests removed in Remove remaining global config #3099.
  • Use env var to change the DEBUG in logger (otherwise it is a bit cumbersome to debug)
  • Merge additional bash parsing unit tests from (test) rewrite of split_bash_parsing and added unit test #3148

Other references

@xingyaoww xingyaoww changed the title [WIP]migrate sandbox tests [WIP] Migrate sandbox tests into runtime tests Jul 26, 2024
@xingyaoww xingyaoww added the test Test cases and testing framework changes label Jul 26, 2024
@xingyaoww xingyaoww changed the title [WIP] Migrate sandbox tests into runtime tests Migrate multi-line-bash-related sandbox tests into runtime tests Jul 27, 2024
@xingyaoww xingyaoww requested a review from neubig July 27, 2024 14:58
@xingyaoww xingyaoww changed the title Migrate multi-line-bash-related sandbox tests into runtime tests Migrate multi-line-bash-related sandbox tests into runtime tests and fix multi-line issue Jul 27, 2024
@xingyaoww xingyaoww marked this pull request as ready for review July 27, 2024 14:58
Copy link
Contributor

@neubig neubig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks for the quick fix!

@neubig neubig enabled auto-merge (squash) July 27, 2024 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Test cases and testing framework changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Multi-line bash commands time out?
3 participants