Skip to content

Commit

Permalink
Update linkScraper.py
Browse files Browse the repository at this point in the history
  • Loading branch information
ryma2fhir authored Jul 24, 2024
1 parent d636b2e commit 50d56fa
Showing 1 changed file with 0 additions and 3 deletions.
3 changes: 0 additions & 3 deletions IGPageContentValidator/linkScraper.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,10 @@ def RequestData(url):
def ListOfLinks(url):
soup = RequestData(url)
websites = []
print("webpages to check")
for link in soup.find_all('a'):
site = link.get('href')
if isinstance(site, str) and site[0:6]=='/guide':
print(site)
websites.append(site)
print('\n\n')
list_set = set(websites)
unique_websites = list(list_set)
return unique_websites
Expand Down

0 comments on commit 50d56fa

Please sign in to comment.