Just copy geoip.dat from repo temporarily

Add geoip.dat due to GeoLite2 restriction

.github/workflows/build.yml

@@ -1,10 +1,10 @@
 name: Build v2ray rules dat files
 on:
   schedule:
-    - cron: "0 19 * * *"
+    - cron: "0 22 * * *"
   push:
     branches:
-     - master
+      - master
 
 jobs:
   build:
@@ -12,36 +12,50 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - name: Set up Go 1.13
+      - name: Setup Go 1.13
         uses: actions/setup-go@v1
         with:
           go-version: 1.13
         id: go
 
-      - name: Set GOPATH
+      - name: Set $GOPATH and more variables
         run: |
-          echo "::set-env name=GOPATH::$(dirname $GITHUB_WORKSPACE)"
-          echo "::set-env name=GREEN::\033[0;32m"
-          echo "::set-env name=NC::\033[0m"
+          echo "::set-env name=NAME::Released on $(date +%Y%m%d%H%M)"
+          echo "::set-env name=TAG_NAME::$(date +%Y%m%d%H%M)"
           echo "::set-env name=GEOIP_REPO::github.com/v2ray/geoip"
           echo "::set-env name=GEOSITE_REPO::github.com/v2ray/domain-list-community"
+          echo "::set-env name=GOOGLE_URL::https://raw.githubusercontent.com/felixonmars/dnsmasq-china-list/master/google.china.conf"
+          echo "::set-env name=APPLE_URL::https://raw.githubusercontent.com/felixonmars/dnsmasq-china-list/master/apple.china.conf"
           echo "::set-env name=GFWLIST_URL::https://cokebar.github.io/gfwlist2dnsmasq/gfwlist_domain.txt"
-          echo "::set-env name=Profiles_URL::https://raw.githubusercontent.com/ConnersHua/Profiles/master/Shadow/Pro.conf"
+          echo "::set-env name=Blocked_DOMAINS_URL::https://raw.githubusercontent.com/wongsyrone/domain-block-list/master/domains.txt"
           echo "::set-env name=CHINA_DOMAINS_URL::https://raw.githubusercontent.com/felixonmars/dnsmasq-china-list/master/accelerated-domains.china.conf"
+          echo "::set-env name=Profiles_URL::https://raw.githubusercontent.com/ConnersHua/Profiles/master/Shadow/Pro.conf"
+          echo "::set-env name=GOPATH::$(dirname $GITHUB_WORKSPACE)"
           echo "::add-path::$(dirname $GITHUB_WORKSPACE)/bin"
         shell: bash
 
-      - name: Get GeoLite2
-        run: |
-          curl -sSL -O https://geolite.maxmind.com/download/geoip/database/GeoLite2-Country-CSV.zip
-          unzip GeoLite2-Country-CSV.zip
-          rm -f GeoLite2-Country-CSV.zip
-          mv GeoLite2* geoip
+      - name: Checkout branch named hidden of this repo
+        uses: actions/checkout@v2
+        with:
+          ref: hidden
 
-      - name: Generate geoip.dat file
+#       - name: Get GeoLite2 file
+#         run: |
+#           curl -sSL -O https://geolite.maxmind.com/download/geoip/database/GeoLite2-Country-CSV.zip
+#           unzip GeoLite2-Country-CSV.zip
+#           rm -f GeoLite2-Country-CSV.zip
+#           mv GeoLite2* geoip
+
+#       - name: Generate geoip.dat file
+#         run: |
+#           go get -u -v -insecure $GEOIP_REPO
+#           geoip --country=./geoip/GeoLite2-Country-Locations-en.csv --ipv4=./geoip/GeoLite2-Country-Blocks-IPv4.csv --ipv6=./geoip/GeoLite2-Country-Blocks-IPv6.csv
+#           mkdir -p ./publish
+#           mv ./geoip.dat ./publish/
+
+      - name: Copy geoip.dat
         run: |
-          go get -u -v -insecure $GEOIP_REPO
-          geoip --country=./geoip/GeoLite2-Country-Locations-en.csv --ipv4=./geoip/GeoLite2-Country-Blocks-IPv4.csv --ipv6=./geoip/GeoLite2-Country-Blocks-IPv6.csv
+          curl -sSL -O https://github.com/Loyalsoldier/v2ray-rules-dat/raw/master/geoip.dat
           mkdir -p ./publish
           mv ./geoip.dat ./publish/
 
@@ -49,86 +63,63 @@ jobs:
         run: |
           go get -u -v -insecure $GEOSITE_REPO
 
-      - name: Get and add gfwlist into geolocation-!cn
+      - name: Get and add gfwlist domains into temp-proxy.txt file
         run: |
-          cd $GOPATH/src/$GEOSITE_REPO/data
-          curl -sSL $GFWLIST_URL > gfwlist
-          echo "include:gfwlist" >> geolocation-\!cn
+          curl -sSL $GFWLIST_URL > temp-proxy.txt
+      
+      - name: Get and add google domains into temp-proxy.txt file
+        run: |
+          curl -sSL $GOOGLE_URL | awk -F '/' '{print $2}' >> temp-proxy.txt
 
-      - name: Get and add proxy domains from @ConnersHua/Profiles into geolocation-!cn
+      - name: Get and add apple domains into temp-proxy.txt file
         run: |
-          cd $GOPATH/src/$GEOSITE_REPO/data
-          curl -sSL $Profiles_URL | awk '/^DOMAIN.+PROXY/' | cut -d ',' -f 2 > profileproxy
-          echo "include:profileproxy" >> geolocation-\!cn
+          curl -sSL $APPLE_URL | awk -F '/' '{print $2}' >> temp-proxy.txt
 
-      - name: Get and add direct domains from @ConnersHua/Profiles into cn
+      - name: Get and add proxy domains from @ConnersHua/Profiles into temp-proxy.txt file
         run: |
-          cd $GOPATH/src/$GEOSITE_REPO/data
-          curl -sSL $Profiles_URL | awk '/^DOMAIN.+DIRECT/' | cut -d ',' -f 2 > profiledirect
-          echo "include:profiledirect" >> cn
+          curl -sSL $Profiles_URL | awk -F ',' '/^DOMAIN(,|\-SUFFIX,).+PROXY/ {print $2}' >> temp-proxy.txt
+          
+      - name: Get and add blocked domains from @wongsyrone/domain-block-list into temp-proxy.txt file
+        run: |
+          curl -sSL $Blocked_DOMAINS_URL >> temp-proxy.txt
 
-      - name: Get and add chinalist into cn
+      - name: Get and add direct domains from @ConnersHua/Profiles into temp-direct.txt file
         run: |
-          cd $GOPATH/src/$GEOSITE_REPO/data
-          curl -sSL $CHINA_DOMAINS_URL | awk -F '/' '{print $2}' > chinalist
-          echo "include:chinalist" >> cn
+          curl -sSL $Profiles_URL | awk -F ',' '/^DOMAIN(,|\-SUFFIX,).+DIRECT/ {print $2}' > temp-direct.txt
 
-      - name: Get and add reject domains from @ConnersHua/Profiles into category-ads-all
+      - name: Get and add chinalist domains into temp-direct.txt file
+        run: |
+          curl -sSL $CHINA_DOMAINS_URL | awk -F '/' '{print $2}' >> temp-direct.txt
+
+      - name: Get and add rejected domains from @ConnersHua/Profiles into category-ads-all
         run: |
           cd $GOPATH/src/$GEOSITE_REPO/data
-          curl -sSL $Profiles_URL | awk '/^DOMAIN.+REJECT/' | cut -d ',' -f 2 > profilereject
+          curl -sSL $Profiles_URL | awk -F ',' '/^DOMAIN(,|\-SUFFIX,).+REJECT/ {print $2}' > profilereject
           echo "include:profilereject" >> category-ads-all
 
+      - name: Add proxy and direct domains from the branch named hidden of this repo to appropriate temp files
+        run: |
+          cat proxy.txt >> temp-proxy.txt
+          cat direct.txt >> temp-direct.txt
+
+      - name: Remove repeated domains and write domains to appropriate list
+        run: |
+          cat temp-proxy.txt | sort --ignore-case -u > $GOPATH/src/$GEOSITE_REPO/data/proxylist
+          cat temp-direct.txt | sort --ignore-case -u > $GOPATH/src/$GEOSITE_REPO/data/directlist
+
+      - name: Add lists into appropriate category
+        run: |
+          cd $GOPATH/src/$GEOSITE_REPO/data
+          echo "include:proxylist" >> geolocation-\!cn
+          echo "include:directlist" >> cn
+
       - name: Build geosite.dat file
         run: |
           domain-list-community
           mv ./dlc.dat ./publish/geosite.dat
 
-      - name: List above process results
-        run: |
-          echo ">>>>>>>>>>>>>>>>>>>>>>>>"
-          
-          echo "list files in data folder"
-          ls -lah $GOPATH/src/$GEOSITE_REPO/data
-          
-          echo ">>>>>>>>>>>>>>>>>>>>>>>>"
-
-          echo "list last 10 lines of cn file"
-          tail -n 10 $GOPATH/src/$GEOSITE_REPO/data/cn
-          
-          echo ">>>>>>>>>>>>>>>>>>>>>>>>"
-
-          echo "list last 10 lines of geolocation-!cn file"
-          tail -n 10 $GOPATH/src/$GEOSITE_REPO/data/geolocation-\!cn
-
-          echo ">>>>>>>>>>>>>>>>>>>>>>>>"
-
-          echo "list last 10 lines of category-ads-all file"
-          tail -n 10 $GOPATH/src/$GEOSITE_REPO/data/category-ads-all
-
-          echo ">>>>>>>>>>>>>>>>>>>>>>>>"
-
-          echo "list files in publish folder"
-          ls -lah ./publish
-
-          echo ">>>>>>>>>>>>>>>>>>>>>>>>"
-
-          echo -e "${GREEN}完成啦！🌈${NC}"
-
-      - name: Set release variables
-        run: |
-          echo "::set-env name=NAME::Released on $(date +%Y%m%d%H%M)"
-          echo "::set-env name=TAG_NAME::$(date +%Y%m%d%H%M)"
-
       - name: Release dat files
         uses: Ricky-Hao/action-release@master
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           ASSET_PATH: publish
-
-      - name: Upload dat files
-        uses: actions/upload-artifact@master
-        if: success()
-        with:
-          name: dat_files
-          path: publish

README.md

@@ -1,6 +1,6 @@
 # 简介
 
-**v2ray** 加强版规则文件，可代替 v2ray 官方 `geoip.dat` 和 `geosite.dat` 文件。利用 GitHub Actions 每天自动构建，保证最新。
+**v2ray** 加强版规则文件，可代替 v2ray 官方 `geoip.dat` 和 `geosite.dat` 规则文件。利用 GitHub Actions 北京时间每天早上 6 点自动构建，保证规则最新。
 
 ## 规则文件生成方式
 
@@ -13,22 +13,35 @@
 ### geosite.dat
 
 - 通过仓库 [@v2ray/domain-list-community](https://github.com/v2ray/domain-list-community) 生成
-- **优点**：由于项目每天自动构建，所以更新速度比官方 `geosite.dat` 要快得多
-- **加入大量中国大陆域名**：通过仓库 [@felixonmars/dnsmasq-china-list](https://github.com/felixonmars/dnsmasq-china-list) 生成 `chinalist` 类别域名并加入到 `geosite:cn` 类别中
-- **加入最新 GFWList**：通过仓库 [@cokebar/gfwlist2dnsmasq](https://github.com/cokebar/gfwlist2dnsmasq) 生成 `gfwlist` 类别域名并加入到 `geosite:geolocation-!cn` 类别中
+- **加入大量中国大陆域名**：通过仓库 [@felixonmars/dnsmasq-china-list](https://github.com/felixonmars/dnsmasq-china-list) 生成并加入到 `geosite:cn` 类别中
+- **加入最新 GFWList 域名**：通过仓库 [@cokebar/gfwlist2dnsmasq](https://github.com/cokebar/gfwlist2dnsmasq) 生成并加入到 `geosite:geolocation-!cn` 类别中
+- **加入 Greatfire.org 检测到的屏蔽域名**：通过仓库 [@wongsyrone/domain-block-list](https://github.com/wongsyrone/domain-block-list) 获取 greatfire.org 检测到的屏蔽域名，并加入到 `geosite:geolocation-!cn` 类别中
 - **加入更多直连、代理、广告域名**：通过仓库 [@ConnersHua/Profiles](https://github.com/ConnersHua/Profiles/tree/master) 获取更多直连、代理、广告域名，并分别加入到 `geosite:cn`、`geosite:geolocation-!cn` 和 `geosite:category-ads-all` 类别中
+- **加入自定义直连和代理域名**：由于上游域名列表更新缓慢或缺失某些被屏蔽的域名，所以引入自定义域名列表，主要为了解决在 DNS 解析 `A` 和 `AAAA` 记录时的 DNS 泄漏问题。[`hidden` 分支](https://github.com/Loyalsoldier/v2ray-rules-dat/tree/hidden)里有两个文件 `direct.txt` 和 `proxy.txt`，分别放置自定义的直连、代理域名，最终分别加入到 `geosite:cn` 和 `geosite:geolocation-!cn` 类别中
+- **优点**：
+  - 由于项目每天自动构建，所以更新速度比官方 `geosite.dat` 要快得多
+  - 对从上述仓库和文件中获取的直连、代理域名进行简单去重，尽量减小规则文件体积
 
-## 使用方式
+## 规则文件下载及使用方式
 
-1. [点击进入规则文件发布页面](https://github.com/Loyalsoldier/v2ray-rules-dat/releases)，点击下载位于最上方的 `geoip.dat` 和 `geosite.dat`
+**下载地址**：
+
+- **geoip.dat**：[https://github.com/Loyalsoldier/v2ray-rules-dat/releases/latest/download/geoip.dat](https://github.com/Loyalsoldier/v2ray-rules-dat/releases/latest/download/geoip.dat)
+- **geosite.dat**：[https://github.com/Loyalsoldier/v2ray-rules-dat/releases/latest/download/geosite.dat](https://github.com/Loyalsoldier/v2ray-rules-dat/releases/latest/download/geosite.dat)
+
+**使用方式**：
+
+1. 通过上面下载地址或[点击进入规则文件发布页面](https://github.com/Loyalsoldier/v2ray-rules-dat/releases)，下载 `geoip.dat` 和 `geosite.dat`
 2. 把下载下来的 `geoip.dat` 和 `geosite.dat` 文件放入到 v2ray 软件的规则文件目录，替换掉原来的 `geoip.dat` 和 `geosite.dat`
-3. 修改 v2ray 配置文件。配置参考下面👇
+3. 修改 v2ray 配置文件，配置参考下面 👇👇👇
+
+## 参考配置
 
 ### geoip.dat
 
 跟 v2ray 官方 `geoip.dat` 配置方式相同。
 
-Routing 配置方式：
+**Routing 配置方式**：
 
 ```json
 {
@@ -51,7 +64,7 @@ Routing 配置方式：
 
 跟 v2ray 官方 `geosite.dat` 配置方式相同。
 
-Routing 配置方式：
+**Routing 配置方式**：
 
 ```json
 {
@@ -83,7 +96,7 @@ Routing 配置方式：
 }
 ```
 
-DNS 配置方式：
+**DNS 配置方式**：
 
 ```json
 {
@@ -109,3 +122,13 @@ DNS 配置方式：
   }
 }
 ```
+
+## 致谢
+
+- [@v2ray/geoip](https://github.com/v2ray/geoip)
+- [@v2ray/domain-list-community](https://github.com/v2ray/domain-list-community)
+- [@felixonmars/dnsmasq-china-list](https://github.com/felixonmars/dnsmasq-china-list)
+- [@cokebar/gfwlist2dnsmasq](https://github.com/cokebar/gfwlist2dnsmasq)
+- [@wongsyrone/domain-block-list](https://github.com/wongsyrone/domain-block-list)
+- [@ConnersHua/Profiles](https://github.com/ConnersHua/Profiles/tree/master)
+- [MaxMind 免费 IP](https://dev.maxmind.com/geoip/geoip2/geolite2/)