Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add fast-crawl sheet for legislation.gov URLs on launch #74

Open
anjackson opened this issue Jan 11, 2021 · 1 comment
Open

Add fast-crawl sheet for legislation.gov URLs on launch #74

anjackson opened this issue Jan 11, 2021 · 1 comment
Assignees
@anjackson

Comments

@anjackson
Copy link
Contributor

anjackson commented Jan 11, 2021 •
edited
Loading

Ahead of properly implementing things in ukwa/w3act#653, it'll be necessary to add the faster sheet (fast-10ps) to the launcher.

Then, extend the system to support the three new fields:

  • no cookies
  • crawl rate
  • parallel queues
@anjackson
Copy link
Contributor Author

For the 2019-prod branch, see 7466f82#diff-bb59927db1a8b1f5eb54e2262aca4887d5a1ad71d95e815009b27070457e7fa4R255

@anjackson anjackson self-assigned this Feb 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@anjackson anjackson

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@anjackson