Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update puppeteer, re-enable mac CI, split e2e CI #910

Merged
merged 8 commits into from
Aug 14, 2024
Merged

Update puppeteer, re-enable mac CI, split e2e CI #910

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
aceb9e1
Update puppeteer, re-enable mac CI, split e2e CI
rly Aug 13, 2024
b19f5ad
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 13, 2024
7eac1e5
Remove accidental linux ci addition
rly Aug 13, 2024
6aa700c
Merge branch 'main' into puppeteer
CodyCBakerPhD Aug 14, 2024
208240a
Remove accidental test data submodules
rly Aug 14, 2024
bf82567
restore single deployment file
CodyCBakerPhD Aug 14, 2024
3e28204
Delete .github/workflows/e2e_tests_on_pull_request.yml
CodyCBakerPhD Aug 14, 2024
00b7e8f
Merge pull request #911 from NeurodataWithoutBorders/restore_single_d…
rly Aug 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions .github/workflows/deploy_tests_on_pull_request.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,6 @@ concurrency:

jobs:

DevTests:
uses: ./.github/workflows/testing_dev.yml
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

LiveServices:
uses: ./.github/workflows/testing_dev_with_live_services.yml
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
DANDI_STAGING_API_KEY: ${{ secrets.DANDI_STAGING_API_KEY }}

BuildTests:
uses: ./.github/workflows/testing_flask_build_and_dist.yml

Expand Down
21 changes: 21 additions & 0 deletions .github/workflows/e2e_tests_on_pull_request.yml
View file
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you recombine this with the main deployment again since it doesn't look like anything changed with how they are called?

The suggestion to peel out e2e step was on the lower workflow level, so that 'dev tests' only run vitetest and pytest, and puppeteer calls on pipelines with example data could be a separate npm run call

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohhh I see.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just something to keep in mind if this happens again. Fingers crossed it doesn't though

Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: E2E Tests

on:
pull_request:

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:

DevTests:
uses: ./.github/workflows/testing_dev.yml
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

LiveServices:
uses: ./.github/workflows/testing_dev_with_live_services.yml
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
DANDI_STAGING_API_KEY: ${{ secrets.DANDI_STAGING_API_KEY }}
2 changes: 1 addition & 1 deletion .github/workflows/example_data_cache.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
fail-fast: false
matrix:
python-version: ["3.12"]
os: [ubuntu-latest, windows-latest] #,macos-latest, macos-13]
os: [ubuntu-latest, windows-latest, macos-latest, macos-13]

steps:

Expand Down
15 changes: 5 additions & 10 deletions .github/workflows/testing_dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,11 @@ jobs:
- os: ubuntu-latest
label: environments/environment-Linux.yml

# Both Mac versions for dev testing started failing around July 25, 2024
# A similar type of issue to one previously seen
# manifesting as hanging/freezing/stalling during postinstall step of electron
# Last time, manually updating the package-lock.json file was enough to fix the issue
# But that didn't work this time
# - os: macos-latest # Mac arm64 runner
# label: environments/environment-MAC-apple-silicon.yml
#
# - os: macos-13 # Mac x64 runner
# label: environments/environment-MAC-intel.yml
- os: macos-latest # Mac arm64 runner
label: environments/environment-MAC-apple-silicon.yml

- os: macos-13 # Mac x64 runner
label: environments/environment-MAC-intel.yml

# - os: windows-latest
# label: environments/environment-Windows.yml
Expand Down
15 changes: 5 additions & 10 deletions .github/workflows/testing_dev_with_live_services.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,11 @@ jobs:
- os: ubuntu-latest
label: environments/environment-Linux.yml

# Both Mac versions for dev testing started failing around July 25, 2024
# A similar type of issue to one previously seen
# manifesting as hanging/freezing/stalling during postinstall step of electron
# Last time, manually updating the package-lock.json file was enough to fix the issue
# But that didn't work this time
# - os: macos-latest # Mac arm64 runner
# label: environments/environment-MAC-apple-silicon.yml
#
# - os: macos-13 # Mac x64 runner
# label: environments/environment-MAC-intel.yml
- os: macos-latest # Mac arm64 runner
label: environments/environment-MAC-apple-silicon.yml

- os: macos-13 # Mac x64 runner
label: environments/environment-MAC-intel.yml

# - os: windows-latest
# label: environments/environment-Windows.yml
Expand Down
25 changes: 7 additions & 18 deletions .github/workflows/testing_flask_build_and_dist.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,14 @@ jobs:
include:
# No linux in this matrix since installation instructions use dev mode instead of distributable

# Both Mac versions for dev testing started failing around July 25, 2024
# A similar type of issue to one previously seen
# manifesting as hanging/freezing/stalling during postinstall step of electron
# Last time, manually updating the package-lock.json file was enough to fix the issue
# But that didn't work this time
# - python-version: "3.9"
# os: macos-latest # Mac arm64 runner
# label: environments/environment-MAC-apple-silicon.yml
# prefix: /Users/runner/miniconda3/envs/nwb-guide
#
# - python-version: "3.9"
# os: macos-13 # Mac x64 runner
# label: environments/environment-MAC-intel.yml
# prefix: /Users/runner/miniconda3/envs/nwb-guide

- python-version: "3.9"
os: windows-latest
- os: macos-latest # Mac arm64 runner
label: environments/environment-MAC-apple-silicon.yml

- os: macos-13 # Mac x64 runner
label: environments/environment-MAC-intel.yml

- os: windows-latest
label: environments/environment-Windows.yml
prefix: C:\Miniconda3\envs\nwb-guide

steps:
- uses: actions/checkout@v4
Expand Down
16 changes: 5 additions & 11 deletions .github/workflows/testing_pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,21 +21,15 @@ jobs:
# - os: ubuntu-latest
# label: environments/environment-Linux.yml

# Both Mac versions for dev testing started failing around July 25, 2024
# A similar type of issue to one previously seen
# manifesting as hanging/freezing/stalling during postinstall step of electron
# Last time, manually updating the package-lock.json file was enough to fix the issue
# But that didn't work this time
# - os: macos-latest # Mac arm64 runner
# label: environments/environment-MAC-apple-silicon.yml
#
# - os: macos-13 # Mac x64 runner
# label: environments/environment-MAC-intel.yml
- os: macos-latest # Mac arm64 runner
label: environments/environment-MAC-apple-silicon.yml

- os: macos-13 # Mac x64 runner
label: environments/environment-MAC-intel.yml

- os: windows-latest
label: environments/environment-Windows.yml


steps:
- uses: actions/checkout@v4
- run: git fetch --prune --unshallow --tags
Expand Down
1 change: 1 addition & 0 deletions behavior_testing_data
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these submodules? I can't follow them anywhere

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! That was not intentional.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That said, did you actually make some GitHub repositories with our example testing data? And maybe git LFS? Because we've all wanted that for a really long time lol

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I accidentally added the testing data git repos as submodules. How would you want these repos/submodules to work?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I accidentally added the testing data git repos as submodules.

But that's what I'm confused about - there aren't any git repos currently (well, technically git-annex via GIN)

How would you want these repos/submodules to work?

Identically to the GIN repos, but better in all the ways that GitHub is better than GIN as a platform (and likely using Git LFS for any files that are larger than a few MB)

Submodule behavior_testing_data added at b695ad
1 change: 1 addition & 0 deletions ephy_testing_data
Open in desktop
Submodule ephy_testing_data added at 0c6c55
1 change: 1 addition & 0 deletions ophys_testing_data
Open in desktop
Submodule ophys_testing_data added at e71732
Loading
Loading