Skip to content

Commit

Permalink
Add missing docker build to slow gradcheck label-triggered build (pyt…
Browse files Browse the repository at this point in the history
…orch#61941)

Summary:
Currently, when adding the label, it fails like: https://app.circleci.com/pipelines/github/pytorch/pytorch/352569/workflows/d213cbad-edd6-4fe0-a79c-d46f8c0aae85/jobs/14856158

Pull Request resolved: pytorch#61941

Reviewed By: suo

Differential Revision: D29827084

Pulled By: albanD

fbshipit-source-id: 134828d36e51324e6b6539dd4bc5f1eebfb89a03
  • Loading branch information
albanD authored and facebook-github-bot committed Jul 22, 2021
1 parent 5186fa2 commit 25be031
Show file tree
Hide file tree
Showing 5 changed files with 23 additions and 3 deletions.
3 changes: 3 additions & 0 deletions .circleci/cimodel/data/pytorch_build_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@
("3.6", [
("shard_test", [X(True)]),
("slow_gradcheck", [
# If you update this slow gradcheck, you should
# also update docker_definitions.py to make sure
# the docker image match the config used here
(True, [
('shard_test', [XImportant(True)]),
]),
Expand Down
9 changes: 8 additions & 1 deletion .circleci/cimodel/data/simple/docker_definitions.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,18 @@
"pytorch-linux-bionic-rocm4.2-py3.6",
]

# This entry should be an element from the list above
# This should contain the image matching the "slow_gradcheck" entry in
# pytorch_build_data.py
SLOW_GRADCHECK_IMAGE_NAME = "pytorch-linux-xenial-cuda10.2-cudnn7-py3-gcc7"

def get_workflow_jobs():
def get_workflow_jobs(only_slow_gradcheck=False):
"""Generates a list of docker image build definitions"""
ret = []
for image_name in IMAGE_NAMES:
if only_slow_gradcheck and image_name is not SLOW_GRADCHECK_IMAGE_NAME:
continue

parameters = OrderedDict({
"name": quote(f"docker-{image_name}"),
"image_name": quote(image_name),
Expand Down
3 changes: 3 additions & 0 deletions .circleci/config.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions .circleci/generate_config_yml.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,10 @@ def gen_build_workflows_tree():
binary_build_definitions.get_nightly_uploads,
]

slow_gradcheck_jobs = pytorch_build_definitions.get_workflow_jobs(only_slow_gradcheck=True)
slow_gradcheck_jobs = [
pytorch_build_definitions.get_workflow_jobs,
cimodel.data.simple.docker_definitions.get_workflow_jobs,
]

return {
"workflows": {
Expand All @@ -172,7 +175,7 @@ def gen_build_workflows_tree():
},
"slow_gradcheck_build": {
"when": r"<< pipeline.parameters.run_slow_gradcheck_build >>",
"jobs": slow_gradcheck_jobs,
"jobs": [f(only_slow_gradcheck=True) for f in slow_gradcheck_jobs],
},
}
}
Expand Down
4 changes: 4 additions & 0 deletions .github/pytorch-circleci-labels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,9 @@ labels_to_circle_params:
- run_build
ci/master:
parameter: run_master_build
set_to_false:
- run_build
ci/slow-gradcheck:
parameter: run_slow_gradcheck_build
set_to_false:
- run_build

0 comments on commit 25be031

Please sign in to comment.