Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always save CSV with UTF-8 encoding #67

Merged
merged 4 commits into from
Jan 4, 2022
Merged

Always save CSV with UTF-8 encoding #67

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5b3dd75
Update test output to include director with 'ō' in name
mx-psi Jan 4, 2022
2494f9d
Set encoding to UTF-8 explicitly
mx-psi Jan 4, 2022
d7c91ae
Use csv-diff fork to specify encoding
mx-psi Jan 4, 2022
27a4489
Merge branch 'main' into mx-psi/new-tests
mx-psi Jan 4, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/endtoend.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ jobs:
- name: Install fa-scrapper
run: poetry install
- name: Install csv-diff
run: python3 -m pip install csv-diff
# Install csv-diff with https://github.com/simonw/csv-diff/pull/19
run: python3 -m pip install git+https://github.com/mikecoop83/csv-diff@c3d32f758343a2ba3737d612e6e906fd9d77322b
- name: Run fa-scrapper
env:
TEST_ACCOUNT_ID: ${{ secrets.TEST_ACCOUNT_ID }}
Expand All @@ -35,5 +36,5 @@ jobs:
- name: Check output
env:
FA_LANG: ${{ matrix.lang }}
run: csv-diff output.csv testdata/expected-${FA_LANG}.csv
run: csv-diff --encoding "utf-8" output.csv testdata/expected-${FA_LANG}.csv
shell: bash
4 changes: 3 additions & 1 deletion fa_scraper/fa_scraper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,9 @@ def save_to_csv(
):
"""Saves films in a csv file"""

with open(filename, "w", newline="") as csvfile:
# Set to UTF-8 to work around Windows error
# "'charmap' codec can't encode character".
with open(filename, "w", encoding="utf-8", newline="") as csvfile:
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
for d in dicts:
Expand Down
1 change: 1 addition & 0 deletions testdata/expected-en.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
Title,Year,Directors,WatchedDate,Rating,Rating10
Attack on Titan: Chronicle,2020,"Masashi Koizuka, Tetsurō Araki",2022-01-03,4.0,8
Full Metal Jacket,1987,Stanley Kubrick,2021-11-10,2.0,4
The Lion King,1994,"Rob Minkoff, Roger Allers",2021-11-10,3.5,7
Mad Max: Fury Road,2015,George Miller,2021-11-10,5.0,10
1 change: 1 addition & 0 deletions testdata/expected-es.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
Title,Year,Directors,WatchedDate,Rating,Rating10
Shingeki no Kyojin: Chronicle,2020,"Masashi Koizuka, Tetsurō Araki",2022-01-04,4.0,8
La chaqueta metálica,1987,Stanley Kubrick,2021-11-11,2.0,4
El rey león,1994,"Rob Minkoff, Roger Allers",2021-11-11,3.5,7
Mad Max: Furia en la carretera,2015,George Miller,2021-11-11,5.0,10