Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

contrib added webscraper contrib #524

Merged
merged 2 commits into from
Nov 9, 2023
Merged

contrib added webscraper contrib #524

merged 2 commits into from
Nov 9, 2023

Conversation

zilto
Copy link
Collaborator

@zilto zilto commented Nov 9, 2023

[Summary of contribution]

For new dataflows:

Do you have the following?

  • Added a new folder for my dataflow under my username directory.
    • Added a README.md file under my dataflow directory that follows the standard headings and is filled out.
    • Added a init.py file under my dataflow directory that contains the Hamilton code.
    • Added a requirements.txt under my dataflow directory that contains the required packages outside of Hamilton.
    • Added tags.json under my dataflow directory to curate my dataflow.
    • Added valid_configs.jsonl under my dataflow directory to specify the valid configurations.
    • Added a dag.png that shows one possible configuration of my dataflow.

For existing dataflows -- what has changed?

How I tested this

I have a notebook to test the DAG can be executed in full

Notes

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Dataflow documentation has been updated if adding/changing functionality.

@zilto zilto added the enhancement New feature or request label Nov 9, 2023
Copy link
Contributor

sweep-ai bot commented Nov 9, 2023

Apply Sweep Rules to your PR?

  • Apply: All new business logic should have corresponding unit tests.
  • Apply: Refactor large functions to be more modular.

@skrawcz skrawcz added the contrib Used for code related to contrib package label Nov 9, 2023

# Configuration Options
## Config.when
This module doesn't receive any configuration.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This module doesn't receive any configuration.
This module doesn't require any configuration.

Tweaks wording in zilo/webscraper README.
@skrawcz skrawcz merged commit 3cee074 into main Nov 9, 2023
2 checks passed
@skrawcz skrawcz deleted the contrib/webscraper branch November 9, 2023 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contrib Used for code related to contrib package enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants