Skip to content

Commit

Permalink
[SPARK-48507][INFRA] Use Hadoop 3.3.6 winutils in build_sparkr_window
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
The pr aims to use `Hadoop 3.3.6` winutils in `build_sparkr_window`.

### Why are the changes needed?
Let's use the latest version.
https://github.com/cdarlint/winutils/tree/master

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
N/A

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#46846 from panbingkun/SPARK-48507.

Authored-by: panbingkun <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
  • Loading branch information
panbingkun authored and HyukjinKwon committed Jun 3, 2024
1 parent 67ad55c commit 7e8b60b
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/build_sparkr_window.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
repository: cdarlint/winutils
- name: Move Hadoop winutil into home directory
run: |
Move-Item -Path hadoop-3.3.5 -Destination ~\
Move-Item -Path hadoop-3.3.6 -Destination ~\
- name: Checkout Spark repository
uses: actions/checkout@v4
- name: Cache Maven local repository
Expand Down Expand Up @@ -79,7 +79,7 @@ jobs:
shell: cmd
- name: Run SparkR tests
run: |
set HADOOP_HOME=%USERPROFILE%\hadoop-3.3.5
set HADOOP_HOME=%USERPROFILE%\hadoop-3.3.6
set PATH=%HADOOP_HOME%\bin;%PATH%
.\bin\spark-submit2.cmd --driver-java-options "-Dlog4j.configurationFile=file:///%CD:\=/%/R/log4j2.properties" --conf spark.hadoop.fs.defaultFS="file:///" R\pkg\tests\run-all.R
shell: cmd
Expand Down

0 comments on commit 7e8b60b

Please sign in to comment.