转载

Spring Boot集成ip2region实现ip白名单

1.什么是ip2region？

ip2region v2.0 - 是一个离线IP地址定位库和IP定位数据管理框架，10微秒级别的查询效率，提供了众多主流编程语言的 xdb 数据生成和查询客户端实现。

Ip2region 特性

1、标准化的数据格式

每个 ip 数据段的 region 信息都固定了格式：国家|区域|省份|城市|ISP，只有中国的数据绝大部分精确到了城市，其他国家部分数据只能定位到国家，后前的选项全部是0。

2、数据去重和压缩

xdb 格式生成程序会自动去重和压缩部分数据，默认的全部 IP 数据，生成的 ip2region.xdb 数据库是 11MiB，随着数据的详细度增加数据库的大小也慢慢增大。

3、极速查询响应

即使是完全基于 xdb 文件的查询，单次查询响应时间在十微秒级别，可通过如下两种方式开启内存加速查询：

  1. vIndex 索引缓存 ：使用固定的 512KiB 的内存空间缓存 vector index 数据，减少一次 IO 磁盘操作，保持平均查询效率稳定在10-20微秒之间。
  2. xdb 整个文件缓存：将整个 xdb 文件全部加载到内存，内存占用等同于 xdb 文件大小，无磁盘 IO 操作，保持微秒级别的查询效率。

4、IP 数据管理框架

v2.0 格式的 xdb 支持亿级别的 IP 数据段行数，region 信息也可以完全自定义，例如：你可以在 region 中追加特定业务需求的数据，例如：GPS信息/国际统一地域信息编码/邮编等。也就是你完全可以使用 ip2region 来管理你自己的 IP 定位数据。

2.代码工程

实验目标

根据来源ip 判断属于那个国家，然后做过滤

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>springboot-demo</artifactId>
        <groupId>com.et</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>ipfilter</artifactId>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-autoconfigure</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <!--    ip lib-->
        <dependency>
            <groupId>org.lionsoul</groupId>
            <artifactId>ip2region</artifactId>
            <version>2.6.4</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>
        <dependency>
            <groupId>com.alibaba.fastjson2</groupId>
            <artifactId>fastjson2</artifactId>
            <version>2.0.40</version>
        </dependency>
    </dependencies>
</project>

controller

package com.et.ipfilter.controller;

import com.et.ipfilter.util.AddressUtils;
import com.et.ipfilter.util.IPOfflineUtil;
import com.et.ipfilter.util.IPOnlineUtil;
import lombok.extern.slf4j.Slf4j;
import org.springframework.util.StringUtils;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.HashMap;
import java.util.Map;

@RestController
@Slf4j
public class HelloWorldController {
    @RequestMapping("/hello")
    public Map<String, Object> showHelloWorld(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse,String ip){

        //fisrt get ip addr by offline file
        //String ip = IPOfflineUtil.getIpAddr(httpServletRequest);
        //analyze address
        String addr = IPOfflineUtil.getAddr(ip);
        if(StringUtils.isEmpty(addr)) {
            //get addr by online service
            ip = IPOfflineUtil.getIpAddr(httpServletRequest);
            addr= AddressUtils.getRealAddressByIP(ip);
            log.info("IP >> {},Address >> {}", ip, addr);
         // you can filter by country or province
        }
        Map<String, Object> map = new HashMap<>();
        map.put("msg", "HelloWorld");
        map.put("ipaddr", addr);
        return map;
    }
}

dto

package com.et.ipfilter.dto;

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;

    @Data
    @NoArgsConstructor
    @AllArgsConstructor
    public  class CountryInfo {
        private String query;
        private String status;
        private String country;
        private String countryCode;
        private String region;
        private String regionName;
        private String city;
        private String zip;
        private String lat;
        private String lon;
        private String timezone;
        private String isp;
        private String org;
        private String as;

}

Util

离线查询ip来源，

下载离线IP定位库

离线数据库在项目的data文件夹下，名称为ip2region.db，其他2个文件是用于生成离线库的，可不用下载。 
https://github.com/lionsoul2014/ip2region/tree/master/data/ip2region.xdb
下载到离线数据库后，我们需要读取这个数据库，我们可以放在项目的resources目录 
package com.et.ipfilter.util;

import lombok.extern.slf4j.Slf4j;
import org.lionsoul.ip2region.xdb.Searcher;
import org.springframework.util.StringUtils;

import javax.servlet.http.HttpServletRequest;

/**
 * ip query
 */
@Slf4j
public class IPOfflineUtil {

    private static final String UNKNOWN = "unknown";

    protected IPOfflineUtil() {
    }


    public static String getIpAddr(HttpServletRequest request) {
        String ip = request.getHeader("X-Forwarded-For");

        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("Proxy-Client-IP");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("WL-Proxy-Client-IP");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_X_FORWARDED_FOR");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_X_FORWARDED");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_X_CLUSTER_CLIENT_IP");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_CLIENT_IP");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_FORWARDED_FOR");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_FORWARDED");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("HTTP_VIA");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getHeader("REMOTE_ADDR");
        }
        if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) {
            ip = request.getRemoteAddr();
        }


        int index = ip.indexOf(",");
        if (index != -1) {
            ip = ip.substring(0, index);
        }
        return "0:0:0:0:0:0:0:1".equals(ip) ? "127.0.0.1" : ip;
    }

    public static String getAddr(String ip) {
        String dbPath = "D:\\IdeaProjects\\ETFramework\\ipfilter\\src\\main\\resources\\ip2region\\ip2region.xdb";
        // 1、from dbPath load all xdb to memory。
        byte[] cBuff;
        try {
            cBuff = Searcher.loadContentFromFile(dbPath);
        } catch (Exception e) {
            log.info("failed to load content from `%s`: %s\n", dbPath, e);
            return null;
        }

        // 2、usr cBuff create a query object base on memory。
        Searcher searcher;
        try {
            searcher = Searcher.newWithBuffer(cBuff);
        } catch (Exception e) {
            log.info("failed to create content cached searcher: %s\n", e);
            return null;
        }
        // 3、query
        try {
            String region = searcher.search(ip);
            return region;
        } catch (Exception e) {
            log.info("failed to search(%s): %s\n", ip, e);
        }
        return null;
    }
}
在线查询访问ip 
package com.et.ipfilter.util;

import com.et.ipfilter.dto.CountryInfo;
import org.springframework.http.ResponseEntity;
import org.springframework.web.client.RestTemplate;

import javax.annotation.PostConstruct;

/**
 * @author liuhaihua
 * @version 1.0
 * @ClassName IPOnlineUtil
 * @Description todo
 * @date 2024/08/09/ 9:58
 */

public class IPOnlineUtil {
    private static RestTemplate restTemplate;

    private final RestTemplate template;

    public static final String IP_API_URL = "http://ip-api.com/json/";

    public IPOnlineUtil(RestTemplate restTemplate) {
        this.template = restTemplate;
    }

    /**
     * init RestTemplate
     */
    @PostConstruct
    public void init() {
        setRestTemplate(this.template);
    }

    /**
     * init RestTemplate
     */
    private static void setRestTemplate(RestTemplate template) {
        restTemplate = template;
    }

    /**
     * get country by ip
     *
     * @param ip
     * @return
     */
    public static CountryInfo getCountryByIpOnline(String ip) {
        ResponseEntity<CountryInfo> entity = restTemplate.getForEntity(
                IP_API_URL + ip + "?lang=zh-CN",
                CountryInfo.class
        );
        return entity.getBody();


    }
}
IP 查询工具类 
package com.et.ipfilter.util;

import org.lionsoul.ip2region.xdb.Searcher;
import java.io.IOException;
 
public class SearcherIPUtils {
    public static String  getCachePosition(String ip) {
        return SearcherIPUtils.getCachePosition("src/main/resources/ip2region/ip2region.xdb", ip, true);
    }
 
    public static String  getPosition(String dbPath,String ip,boolean format) {
        // 1、create searcher object
        Searcher searcher = null;
        try {
            searcher = Searcher.newWithFileOnly(dbPath);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
 
        // 2、query
        try {
            String region = searcher.search(ip);
            if (format){
                return region;
            }
            String[] split = region.split("\\|");
            String s = split[0] + split[2] + split[3];
            return s;
        } catch (Exception e) {
            throw new RuntimeException(e);
        }

    }
 
    /**
     * @Description :
     * @Author : mabo
    */
    public static String getIndexCachePosition(String dbPath, String ip, boolean format) {
        Searcher searcher = null;
        byte[] vIndex;
        try {
            vIndex = Searcher.loadVectorIndexFromFile(dbPath);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
        try {
            searcher = Searcher.newWithVectorIndex(dbPath, vIndex);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
        try {
            String region = searcher.search(ip);
            if (format){
                return region;
            }
            String[] split = region.split("\\|");
            String s = split[0] + split[2] + split[3];
            return s;
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
 
    /**
     * @Description :
     * @Author : mabo
     */
    public static String  getCachePosition(String dbPath,String ip,boolean format) {
        byte[] cBuff;
        try {
            cBuff = Searcher.loadContentFromFile(dbPath);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
 
        Searcher searcher;
        try {
            searcher = Searcher.newWithBuffer(cBuff);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
        try {
            String region = searcher.search(ip);
            if (format){
                return region;
            }
            String[] split = region.split("\\|");
            String s = split[0] + split[2] + split[3];
            return s;
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
 
    }
}

package com.et.ipfilter.util;

import com.alibaba.fastjson2.JSONObject;
import lombok.extern.slf4j.Slf4j;
import org.springframework.util.StringUtils;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.ConnectException;
import java.net.SocketTimeoutException;
import java.net.URL;
import java.net.URLConnection;

@Slf4j
public class AddressUtils {

    public static final String IP_URL = "http://whois.pconline.com.cn/ipJson.jsp";
 
    public static final String UNKNOWN = "XX XX";
 
    public static String getRealAddressByIP(String ip) {
        String address = UNKNOWN;
         //if ip is inner ip,return
        if (internalIp(ip)) {
            return "inner IP";
        }
        if (true) {
            try {
                String rspStr = sendGet(IP_URL, "ip=" + ip + "&json=true" ,"GBK");
                if (StringUtils.isEmpty(rspStr)) {
                    log.error("get addr exception {}" , ip);
                    return UNKNOWN;
                }
                JSONObject obj = JSONObject.parseObject(rspStr);
                String region = obj.getString("pro");
                String city = obj.getString("city");
                return String.format("%s %s" , region, city);
            } catch (Exception e) {
                log.error("get addr exception {}" , ip);
            }
        }
        return address;
    }
 
    public static String sendGet(String url, String param, String contentType) {
        StringBuilder result = new StringBuilder();
        BufferedReader in = null;
        try {
            String urlNameString = url + "?" + param;
            log.info("sendGet - {}" , urlNameString);
            URL realUrl = new URL(urlNameString);
            URLConnection connection = realUrl.openConnection();
            connection.setRequestProperty("accept" , "*/*");
            connection.setRequestProperty("connection" , "Keep-Alive");
            connection.setRequestProperty("user-agent" , "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;SV1)");
            connection.connect();
            in = new BufferedReader(new InputStreamReader(connection.getInputStream(), contentType));
            String line;
            while ((line = in.readLine()) != null) {
                result.append(line);
            }
            log.info("recv - {}" , result);
        } catch (ConnectException e) {
            log.error("invoke HttpUtils.sendGet ConnectException, url=" + url + ",param=" + param, e);
        } catch (SocketTimeoutException e) {
            log.error("invoke HttpUtils.sendGet SocketTimeoutException, url=" + url + ",param=" + param, e);
        } catch (IOException e) {
            log.error("invoke HttpUtils.sendGet IOException, url=" + url + ",param=" + param, e);
        } catch (Exception e) {
            log.error("invoke HttpsUtil.sendGet Exception, url=" + url + ",param=" + param, e);
        } finally {
            try {
                if (in != null) {
                    in.close();
                }
            } catch (Exception ex) {
                log.error("invoke in.close Exception, url=" + url + ",param=" + param, ex);
            }
        }
        return result.toString();
    }
 
 
    /**
     * check the ip is inner?
     *
     * @param ip
     * @return
     */
    public static boolean internalIp(String ip) {
        byte[] addr = textToNumericFormatV4(ip);
        return internalIp(addr) || "127.0.0.1".equals(ip);
    }
 
    /**
     * check the ip is inner?
     *
     * @param addr
     * @return
     */
    private static boolean internalIp(byte[] addr) {
        if (null==addr|| addr.length < 2) {
            return true;
        }
        final byte b0 = addr[0];
        final byte b1 = addr[1];
        // 10.x.x.x/8
        final byte SECTION_1 = 0x0A;
        // 172.16.x.x/12
        final byte SECTION_2 = (byte) 0xAC;
        final byte SECTION_3 = (byte) 0x10;
        final byte SECTION_4 = (byte) 0x1F;
        // 192.168.x.x/16
        final byte SECTION_5 = (byte) 0xC0;
        final byte SECTION_6 = (byte) 0xA8;
        switch (b0) {
            case SECTION_1:
                return true;
            case SECTION_2:
                return (b1 >= SECTION_3 && b1 <= SECTION_4);
            case SECTION_5:
                return (b1 == SECTION_6);
            default:
                return false;
        }
    }
 
    /**
     * change IPv4 to byte
     *
     * @param text
     * @return byte
     */
    private static byte[] textToNumericFormatV4(String text) {
        if (text.length() == 0) {
            return new byte[0];
        }
 
        byte[] bytes = new byte[4];
        String[] elements = text.split("\\.", -1);
        try {
            long l;
            int i;
            switch (elements.length) {
                case 1:
                    l = Long.parseLong(elements[0]);
                    if ((l < 0L) || (l > 4294967295L)) {
                        return new byte[0];
                    }
                    bytes[0] = (byte) (int) (l >> 24 & 0xFF);
                    bytes[1] = (byte) (int) ((l & 0xFFFFFF) >> 16 & 0xFF);
                    bytes[2] = (byte) (int) ((l & 0xFFFF) >> 8 & 0xFF);
                    bytes[3] = (byte) (int) (l & 0xFF);
                    break;
                case 2:
                    l = Integer.parseInt(elements[0]);
                    if ((l < 0L) || (l > 255L)) {
                        return new byte[0];
                    }
                    bytes[0] = (byte) (int) (l & 0xFF);
                    l = Integer.parseInt(elements[1]);
                    if ((l < 0L) || (l > 16777215L)) {
                        return new byte[0];
                    }
                    bytes[1] = (byte) (int) (l >> 16 & 0xFF);
                    bytes[2] = (byte) (int) ((l & 0xFFFF) >> 8 & 0xFF);
                    bytes[3] = (byte) (int) (l & 0xFF);
                    break;
                case 3:
                    for (i = 0; i < 2; ++i) {
                        l = Integer.parseInt(elements[i]);
                        if ((l < 0L) || (l > 255L)) {
                            return new byte[0];
                        }
                        bytes[i] = (byte) (int) (l & 0xFF);
                    }
                    l = Integer.parseInt(elements[2]);
                    if ((l < 0L) || (l > 65535L)) {
                        return new byte[0];
                    }
                    bytes[2] = (byte) (int) (l >> 8 & 0xFF);
                    bytes[3] = (byte) (int) (l & 0xFF);
                    break;
                case 4:
                    for (i = 0; i < 4; ++i) {
                        l = Integer.parseInt(elements[i]);
                        if ((l < 0L) || (l > 255L)) {
                            return new byte[0];
                        }
                        bytes[i] = (byte) (int) (l & 0xFF);
                    }
                    break;
                default:
                    return new byte[0];
            }
        } catch (NumberFormatException e) {
            return new byte[0];
        }
        return bytes;
    }
 
}

以上只是一些关键代码，所有代码请参见下面代码仓库

代码仓库

3.测试

  • 启动 springBoot应用
  • 访问http://127.0.0.1:8088/hello?ip=52.220.113.16
  • 可以看到该ip属于新加坡，后续可以根据国家限制访问

4.引用

 
正文到此结束
所属分类： springboot-demo

热门推荐

相关文章

阿里云首购8折
springboot-demo
说给你听

本文目录
随机标签
书籍教程
springboot-demo Java入门教程 bootstrap3 CSS Apache基础教程 php ionic 教程 Python mysql教程 eclipse Ubuntu VPS系统配置 AngularJS 教程 MongoDB教程 Struts2教程 Redis教程 Git教程 Spring教程 Jenkins进阶系列 openfire参考指南 Java设计模式 HBase教程 Maven教程 hibernate教程 Docker 教程 memcached教程 Quartz指南 Hive教程 SpringCloud java实例教程 ANTLR教程 Ant教程 XStream教程 Hazelcast教程 深入浅出MyBatis Elastic-Job-Lite ibaties教程 SVN教程 rabittmq教程 WebService CXF学习 solr教程 Hadoop教程 JPA教程 ActiveMQ中文指南 Java内存模型 dubbo教程 springcloud-demo Linux入门视频教程
近期评论
  1. 1 Spring Boot集成Akka Cluster实现在分布式节点中执行任务
  2. 2 Spring Boot集成Akka Cluster快速入门Demo
  3. 3 Spring Boot集成Akka remoting快速入门Demo
  4. 4 Spring Boot集成Akka Stream快速入门Demo
  5. 5 Spring Boot集成akka actor快速入门Demo
  6. 6 maven父子工程多模块如何管理统一的版本号？
  7. 7 Spring Boot集成Mockito快速入门Demo
  8. 8 Spring Boot集成Hazelcast Jet快速入门DEMO
  9. 9 Spring Boot集成Tess4J实现OCR
  10. 10 Spring Boot实现大文件分片下载
网站信息
  • 文章总数：155,600 篇
  • 文件总数：468,947 个
  • 标签总数：2,356 个
  • 分类总数：91 个
  • 留言数量：2,548 条
  • 在线人数：642 人
  • 运行天数：4,343天
  • 最后更新：2024年09月17日00点
Loading...

其他链接

关于本站

本站定位：个人技术类博客

本站作用：写博客、记日志、闲聊扯淡鼓捣技术。

问题交流

[HBLOG]公众号 HBLOG HBLOG