Add new "Basic" version of the filterlist

fmhy · Aug 18, 2024 · b156238 · b156238
1 parent f9664de
commit b156238
Show file tree

Hide file tree

Showing 19 changed files with 2,173 additions and 351 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -2,4 +2,4 @@
 
 ## Adding a website:
 
-Fork the project, add a website in sitelist.txt and run build.py then submit a pull request. 
+Fork the project, add a website in sitelist.txt (or in sitelist-plus.txt if the domain fits in the Plus category) and run build.py then submit a pull request. 
diff --git a/README.md b/README.md
@@ -1,13 +1,19 @@
 # FMHYFilterlist
-FMHY Filterlist, Blocks unsafe sites listed in unsafe sites.
+FMHY Filterlist, Blocks unsafe and optionally also not recommended sites listed in [FMHY unsafe sites](https://fmhy.net/unsafesites).
 
-[Check out the website](https://fmhy.github.io/FMHYFilterlist/site/index.html)
-## How to use
-### Ublock origin (Easy):
+> [!TIP]
+> [Check out the website](https://fmhy.github.io/FMHYFilterlist/site/index.html)
+
+> [!NOTE]
+> The "Basic" (recommended to use for inclusion in other blocklists or projects/browsers/etc.. as malicious sites filter, or for non-home network filtering) version blocks only the malicious sites, but the "Plus" (recommended to use for most users) version also blocks not recommended/potentially unsafe sites and apps
+## How to use (Plus version) <a name="howtouse-plus"></a>
+> [!NOTE]
+> Plus version of the filterlist blocks apps and websites like Avast/AVG/Avira/Opera/Kik Messenger/McAfee/CNet/Softonic, if you don't want these services to be blocked for some reason, use the "[Basic](#howtouse-basic)" version of the filterlist instead
+### uBlock Origin (Easy):
 1. [Go to the website](https://fmhy.github.io/FMHYFilterlist/site/index.html)
 2. Click the "1 Click Install For Ublock Origin" button
 3. Click the subscribe button
-### Ublock origin (Manual)
+### uBlock Origin (Manual)
 1. Open ublock origin and select the cog ( to go to the dashboard ).
 2. Click filter lists 
 3. Scroll down to "Import" click it and type this url: ```https://raw.githubusercontent.com/fmhy/FMHYFilterlist/main/filterlist.txt``` and click "Apply Changes"
@@ -27,9 +33,30 @@ Blocklist formats:
 
 If your blocker doesn't support any of these formats, feel free to create an issue for adding support for your blocker.
 
+## How to use (Basic version) <a name="howtouse-basic"></a>
+> [!NOTE]
+> Basic version of the filterlist doesn't block apps and websites like Avast/AVG/Avira/Opera/Kik Messenger/McAfee/CNet/Softonic, if you do want services like this to be blocked, please use the "[Plus](#howtouse-plus)" version of the filterlist instead
+### uBlock Origin (Manual)
+1. Open ublock origin and select the cog ( to go to the dashboard ).
+2. Click filter lists 
+3. Scroll down to "Import" click it and type this url: ```https://raw.githubusercontent.com/fmhy/FMHYFilterlist/main/filterlist-basic.txt``` and click "Apply Changes"
+### Brave Browser:
+1. Open a new tab in brave and type ```brave://settings/```
+2. Click Shields/Content Filtering/ Scroll down to add custom filter lists
+3. Enter this url ```https://raw.githubusercontent.com/fmhy/FMHYFilterlist/main/filterlist-basic.txt``` and click add.
+### Other blockers:
+Add blocklist url with right syntax/formatting according to documentation of your blocker.
+Blocklist formats:
+1. Adblock Plus syntax (example uses: Pi-hole) - ```https://raw.githubusercontent.com/fmhy/FMHYFilterlist/main/filterlist-basic-abp.txt```
+2. uBlock Origin syntax - ```https://raw.githubusercontent.com/fmhy/FMHYFilterlist/main/filterlist-basic.txt```
+3. List of domains - ```https://raw.githubusercontent.com/fmhy/FMHYFilterlist/main/filterlist-basic-domains.txt```
+4. Hosts file - ```https://raw.githubusercontent.com/fmhy/FMHYFilterlist/main/filterlist-basic-hosts.txt```
+5. List of domains with wildcards - ```https://raw.githubusercontent.com/fmhy/FMHYFilterlist/main/filterlist-basic-wildcard-domains.txt```
+6. List of urls with wildcards - ```https://raw.githubusercontent.com/fmhy/FMHYFilterlist/main/filterlist-basic-wildcard-urls.txt```
+
 ## How to contribute.
 
-Fork the project, add a website in sitelist.txt and run build.py then submit a pull request. 
+Fork the project, add a website in sitelist.txt (or in sitelist-plus.txt if the domain fits in the Plus category) and run build.py then submit a pull request. 
 
 ## Want to be a collaborator?
 

diff --git a/build.py b/build.py
@@ -1,6 +1,6 @@
 import datetime
 
-def generate_blocklist(header_file, sitelist_file, output_file, format, filterlist_file_ublock_only=''):
+def generate_blocklist(header_file, sitelist_file, output_file, format, plus_version: bool, sitelist_file_plus='', filterlist_file_ublock_only=''):
   """
   Combines and modifies content of two files into a new file, 
   replacing placeholders, counting entries and adding text characters for proper format support.
@@ -14,6 +14,11 @@ def generate_blocklist(header_file, sitelist_file, output_file, format, filterli
     for line in f_filtered:
       if not line.startswith('!'):
         num_entries += 1
+  if plus_version == True:
+    with open(sitelist_file_plus, 'r') as f_filtered_plus:
+      for line in f_filtered_plus:
+        if not line.startswith('!'):
+          num_entries += 1
   if format == 'ublock':
     with open(filterlist_file_ublock_only, 'r') as f_filtered_ublock:
       for line in f_filtered_ublock:
@@ -26,6 +31,16 @@ def generate_blocklist(header_file, sitelist_file, output_file, format, filterli
 
     # Replace placeholders in header lines
     for i, line in enumerate(header_lines):
+      if line.startswith('Title:'):
+        if plus_version == True:
+          header_lines[i] = line.replace('${filterlist_edition}', 'Plus')
+        elif plus_version == False:
+          header_lines[i] = line.replace('${filterlist_edition}', 'Basic')
+      if line.startswith('Description:'):
+        if plus_version == True:
+          header_lines[i] = line.replace('${sites_blocked}', 'malicious and not recommended')
+        elif plus_version == False:
+          header_lines[i] = line.replace('${sites_blocked}', 'malicious')
       if line.startswith('Last modified:'):
         header_lines[i] = line.replace('${current_time}', current_time)
       elif line.startswith('Entries:'):
@@ -57,23 +72,56 @@ def generate_blocklist(header_file, sitelist_file, output_file, format, filterli
           domains_lines[i] = line.replace('!', '#')
       line.format()
 
+    # Add proper filterlist formatting to the plus version extra domains
+    if plus_version == True:
+      with open(sitelist_file_plus, 'r') as f_domains_plus:
+        domains_lines_plus = f_domains_plus.readlines()
+        for i, line in enumerate(domains_lines_plus):
+          if not line.startswith('!'):
+            if format == 'ublock' or format == 'abp':
+              domains_lines_plus[i] = line.replace("\n", '')
+              domains_lines_plus[i] = ('||' + domains_lines_plus[i] + '^' + "\n")
+            elif format == 'hosts':
+              domains_lines_plus[i] = ('0.0.0.0 ' + domains_lines_plus[i])
+            elif format == 'wildcard_domains':
+              domains_lines_plus[i] = ('*.' + domains_lines_plus[i])
+            elif format == 'wildcard_urls':
+              domains_lines_plus[i] = line.replace("\n", '')
+              domains_lines_plus[i] = ('*://*.' + domains_lines_plus[i] + '/*' + "\n")
+          if line.startswith('!'):
+            if format == 'hosts' or format == 'domains' or format == 'wildcard_domains' or format == 'wildcard_urls':
+             domains_lines_plus[i] = line.replace('!', '#')
+          line.format()
+
     # Write modified header and filtered content
     if format == 'abp':
       f_combined.writelines("[Adblock Plus]\n")
     f_combined.writelines(header_lines)
     f_combined.writelines("\n")
     f_combined.writelines(domains_lines)
+    if plus_version == True:
+      if format == 'domains' or format == 'hosts' or format == 'wildcard_domains':
+        f_combined.writelines("\n")
+      f_combined.writelines(domains_lines_plus)
     if format == 'ublock':
       f_combined.writelines(open(filterlist_file_ublock_only, 'r'))
 
   print("Generated filterlist with " + format + " format as " + output_file + "!")
 
 # Generate Header to make filterlist
 print("Generating filterlist.")
-generate_blocklist("header.txt", "sitelist.txt", "filterlist.txt", "ublock", "filters-ublock-only.txt")
-generate_blocklist("header.txt", "sitelist.txt", "filterlist-abp.txt", "abp")
-generate_blocklist("header.txt", "sitelist.txt", "filterlist-domains.txt", "domains")
-generate_blocklist("header.txt", "sitelist.txt", "filterlist-wildcard-domains.txt", "wildcard_domains")
-generate_blocklist("header.txt", "sitelist.txt", "filterlist-wildcard-urls.txt", "wildcard_urls")
-generate_blocklist("header.txt", "sitelist.txt", "filterlist-hosts.txt", "hosts")
+# Generate Plus version filterlist files
+generate_blocklist("header.txt", "sitelist.txt", "filterlist.txt", "ublock", True, "sitelist-plus.txt", "filters-ublock-only.txt")
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-abp.txt", "abp", True, "sitelist-plus.txt")
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-domains.txt", "domains", True, "sitelist-plus.txt")
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-wildcard-domains.txt", "wildcard_domains", True, "sitelist-plus.txt")
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-wildcard-urls.txt", "wildcard_urls", True, "sitelist-plus.txt")
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-hosts.txt", "hosts", True, "sitelist-plus.txt")
+# Generate Basic version filterlist files
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-basic.txt", "ublock", False, filterlist_file_ublock_only="filters-ublock-only.txt")
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-basic-abp.txt", "abp", False)
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-basic-domains.txt", "domains", False)
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-basic-wildcard-domains.txt", "wildcard_domains", False)
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-basic-wildcard-urls.txt", "wildcard_urls", False)
+generate_blocklist("header.txt", "sitelist.txt", "filterlist-basic-hosts.txt", "hosts", False)
 print("Build Finished")
diff --git a/filterlist-abp.txt b/filterlist-abp.txt
@@ -1,11 +1,11 @@
 [Adblock Plus]
-! Title: FMHY Unsafe sites filterlist
-! Description: Blocks sites in the fmhy unsafe sites filterlists
-! Last modified: 08 Aug 2024 17:22 UTC
+! Title: FMHY Unsafe sites filterlist - Plus
+! Description: Blocks malicious and not recommended sites listed in the fmhy unsafe sites page
+! Last modified: 18 Aug 2024 20:35 UTC
 ! Homepage: https://github.com/fmhy/FMHYFilterlist
 ! License: https://github.com/fmhy/FMHYFilterlist/blob/main/LICENSE
 ! Issues: https://github.com/fmhy/FMHYFilterlist/issues
-! Entries: 289
+! Entries: 298
 ! Format: abp
 
 ! Unsafe game sites
@@ -128,13 +128,6 @@
 ||app-valley.vip^
 ||appvalley-apk.com^
 ||tutubox.io^
-||www.cnet.com^
-||cnet.com^
-||download.com^
-||www.zdnet.com^
-||zdnet.com^
-||en.softonic.com^
-||softonic.com^
 ||ezcrack.info^
 ||ccm.net^
 ||downloads.digitaltrends.com^
@@ -168,45 +161,8 @@
 ||downloadly.ir^
 ||downloadly.net^
 ||downloadlyir.com^
-||www.opera.com^
-||opera.com^
-||www.mcafee.com^
-||mcafee.com^
-||www.mcafeemobilesecurity.com^
-||mcafeemobilesecurity.com^
-||avast.com^
-||www.avg.com^
-||avg.com^
-||www.gendigital.com^
-||gendigital.com^
-||us.norton.com^
-||norton.com^
-||login.norton.com^
-||www.nortonlifelock.com^
-||nortonlifelock.com^
-||www.avira.com^
-||avira.com^
-||my.avira.com^
-||www.ccleaner.com^
-||ccleaner.com^
-||www.ccleanercloud.com^
-||ccleanercloud.com^
-||www.privateinternetaccess.com^
-||privateinternetaccess.com^
-||www.expressvpn.com^
-||expressvpn.com^
-||zenmate.com^
-||www.cyberghostvpn.com^
-||cyberghostvpn.com^
-||www.kape.com^
-||kape.com^
-||www.vpnmentor.com^
-||vpnmentor.com^
-||www.wizcase.com^
-||wizcase.com^
 ||bluesmods.com^
 ||bluekik.pro^
-||kik.com^
 ||tlauncher.org^
 ||tlauncher.co.com^
 ||polymc.org^
@@ -250,18 +206,22 @@
 ||z-library-proxy.com^
 ! Fake Windows Activators
 ||activator-kmspico.net^
+||activators.pro^
 ||activators4windows.com^
 ||aktivator-windows.net^
 ||ativadorkmspico.com^
 ||dazactivator.com^
 ||get-kmspico.com^
 ||getkmspico.com^
 ||getkmspico.medium.com^
+||km-spico.ru^
 ||kms-activator.net^
 ||kms-auto.net^
 ||kms-auto.site^
 ||kms-pc.com^
 ||kms-tool.com^
+||kmsauto.biz^
+||kmsauto.net.ru^
 ||kmsauto.org^
 ||kmsauto.su^
 ||kmsautoplus.su^
@@ -270,11 +230,13 @@
 ||kmspico-activator-free.com^
 ||kmspico-activator.in^
 ||kmspico-mafia.com^
+||kmspico-official.medium.com^
 ||kmspico-official.xyz^
 ||kmspico-oficial.com^
 ||kmspico-software.net^
 ||kmspico-tools2.site^
 ||kmspico.co^
+||kmspico.dev^
 ||kmspico.icu^
 ||kmspico.io^
 ||kmspico.micaesoft.com^
@@ -305,5 +267,55 @@
 ||oficial-kmspico.com^
 ||win7loader.com^
 ||windows-7.net^
+||windows-activators.com^
 ||windows64.net^
 ||windowsbit.net^
+! This section of the filterlist contains not recommended/potentially unsafe sites and software domains used in the Plus version
+! Software / App Sites
+||www.cnet.com^
+||cnet.com^
+||download.cnet.com^
+||download.com^
+||www.zdnet.com^
+||zdnet.com^
+||en.softonic.com^
+||softonic.com^
+! Software / Apps
+||www.opera.com^
+||opera.com^
+||www.mcafee.com^
+||mcafee.com^
+||www.mcafeemobilesecurity.com^
+||mcafeemobilesecurity.com^
+||mcafeewebadvisor.com^
+||avast.com^
+||www.avg.com^
+||avg.com^
+||www.gendigital.com^
+||gendigital.com^
+||us.norton.com^
+||norton.com^
+||login.norton.com^
+||www.nortonlifelock.com^
+||nortonlifelock.com^
+||www.avira.com^
+||avira.com^
+||my.avira.com^
+||www.ccleaner.com^
+||ccleaner.com^
+||www.ccleanercloud.com^
+||ccleanercloud.com^
+||www.privateinternetaccess.com^
+||privateinternetaccess.com^
+||www.expressvpn.com^
+||expressvpn.com^
+||zenmate.com^
+||www.cyberghostvpn.com^
+||cyberghostvpn.com^
+||www.kape.com^
+||kape.com^
+||www.vpnmentor.com^
+||vpnmentor.com^
+||www.wizcase.com^
+||wizcase.com^
+||kik.com^
Original file line number	Diff line number	Diff line change
Expand Up		@@ -2,4 +2,4 @@

		## Adding a website:

		Fork the project, add a website in sitelist.txt and run build.py then submit a pull request.
		Fork the project, add a website in sitelist.txt (or in sitelist-plus.txt if the domain fits in the Plus category) and run build.py then submit a pull request.