Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Only test free-threading with faster macOS M1 #116814

Merged
merged 1 commit into from
Mar 14, 2024

Conversation

hugovk
Copy link
Member

@hugovk hugovk commented Mar 14, 2024

We used to only have a single macOS Intel job in the CI.

But we have recently quadrupled it:

Plus we occasionally have a couple testing the JIT, but they only trigger when relevant files are modified.

Unfortunately, we're limited to a maximum of 5 concurrent macOS jobs:

This means at busy times, jobs can be queued for a long time waiting for free macOS runners. In the meantime, the other jobs have finished, but we need to sit around until one of the five runners becomes free.

We've run in to this once per week for the last three weeks, during European evening/US daytime.

  • Last night: ~105m
  • Last week: ~100m
  • The week before: ~90m

To mitigate, let's remove some macOS jobs.

We can't remove them from the regular build, because they're needed to maintain PEP 11's Tier 1 support. So let's start by removing one macOS test job for the free-threading build.

Here's macOS job times for five recent builds. macos-13 is Intel, macOS-14 is M1.

macOS / (macos-13)    11m 8s
macOS / (macos-13)    13m 40s
macOS / (macos-13)    9m 2s
macOS / (macos-13)    9m 52s
macOS / (macos-13)    9m 9s

macOS / (macos-14)    6m 36s
macOS / (macos-14)    6m 38s
macOS / (macos-14)    6m 50s
macOS / (macos-14)    7m 29s
macOS / (macos-14)    8m 46s
macOS (free-threading) / (macos-13)    14m 23s
macOS (free-threading) / (macos-13)    14m 47s
macOS (free-threading) / (macos-13)    18m 46s
macOS (free-threading) / (macos-13)    26m 22s
macOS (free-threading) / (macos-13)    28m 53s

macOS (free-threading) / (macos-14)    10m 3s
macOS (free-threading) / (macos-14)    7m 36s
macOS (free-threading) / (macos-14)    7m 39s
macOS (free-threading) / (macos-14)    7m 52s
macOS (free-threading) / (macos-14)    8m 46s

M1 is much faster (~8 mins), so let's remove the slower free-threading Intel job (~20 mins).

If the problem persists, we can remove the free-threading M1 job as well.

And hopefully GitHub will be able to increase the number of concurrent macOS jobs in the future, especially as the free-threading work matures. 🤞

"macos-14", # M1
"macos-13", # Intel
]
os: ${{fromJson(inputs.os-matrix)}}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't pass arrays into reusable workflows, only strings, booleans or numbers. So we pass the array as a string, then use fromJson to convert the string into a list.

@hugovk
Copy link
Member Author

hugovk commented Mar 14, 2024

Before After

@ambv ambv enabled auto-merge (squash) March 14, 2024 14:14
@ambv ambv merged commit 415cd06 into python:main Mar 14, 2024
34 of 35 checks passed
@hugovk hugovk deleted the ci-free-threading-m1-only branch March 14, 2024 16:32
vstinner pushed a commit to vstinner/cpython that referenced this pull request Mar 20, 2024
Only test free-threading with faster macOS M1
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024
Only test free-threading with faster macOS M1
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
Only test free-threading with faster macOS M1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants