有一次在生产环境,突然出现了很多笔还款单被挂起,后来排查原因,发现是内部系统调用时出现了Hystrix调用异常。在开发过程中,因为核心线程数设置的比较大,没有出现这种异常。放到了测试环境,偶尔有出现这种情况,后来在网上查找解决方案,网上的方案是调整maxQueueSize属性就好了,当时调整了一下,确实有所改善。可没想到在生产环境跑了一段时间后却又出现这种了情况,此时我第一想法就是去查看maxQueueSize属性,可是maxQueueSize属性是设置值了。当时就比较纳闷了,为什么maxQueueSize属性不起作用,后来通过查看官方文档发现Hystrix还有一个queueSizeRejectionThreshold属性,这个属性是控制队列最大阈值的,而Hystrix默认只配置了5个,因此就算我们把maxQueueSize的值设置再大,也是不起作用的。两个属性必须同时配置
application.yml:
hystrix: threadpool: default: coreSize: 200 #并发执行的最大线程数,默认10 maxQueueSize: 1000 #BlockingQueue的最大队列数,默认值-1 queueSizeRejectionThreshold: 800 #即使maxQueueSize没有达到,达到queueSizeRejectionThreshold该值后,请求也会被拒绝,默认值5 复制代码
/** * @Author: XiongFeng * @Description: * @Date: Created in 11:12 2018/6/11 */ public class RepaymentHelperTest extends FundApplicationTests { @Autowired RepaymentHelper repaymentHelper; @Autowired private RouterFeign routerFeign; @Test public void hystrixTest() throws InterruptedException { for (int i = 0; i < 135; i++) { new Thread(new Runnable() { @Override public void run() { job(); } }).start(); } Thread.currentThread().join(); } public void job() { String repaymentNo = "xf1002"; String transNo = "T4324324234"; String reqNo = "xf1002"; String begintime = "20180831130030"; String endtime = "20180831130050"; TransRecQueryReqDto transRecQueryReqDto = new TransRecQueryReqDto(); transRecQueryReqDto.setTransNo(transNo); transRecQueryReqDto.setBeginTime(begintime); transRecQueryReqDto.setEndTime(endtime); transRecQueryReqDto.setReqNo(reqNo); Resp<List<TransRecDto>> queryTransRecListResp = routerFeign.queryTransRec(new Req<>(repaymentNo, "2018080200000002", null, null, transRecQueryReqDto)); System.out.println(String.format("获取结果为:【%s】", JsonUtil.toJson(queryTransRecListResp))); } } 复制代码
@FeignClient(value = "${core.name}", fallbackFactory = RouterFeignBackFactory.class, path = "/router") public interface RouterFeign { /** * 代扣结果查询 * @param transRecQueryReqDtoReq * @return */ @PostMapping("/queryTransRec") Resp<List<TransRecDto>> queryTransRec(@RequestBody Req<TransRecQueryReqDto> transRecQueryReqDtoReq); } 复制代码
/** * @Author: XiongFeng * @Description: * @Date: Created in 16:04 2018/5/24 */ @Api("还款服务") @RefreshScope @RestController @RequestMapping("/router") public class TestController { private static Logger logger = LoggerFactory.getLogger(TestController.class); // 计数器 private static AtomicInteger count = new AtomicInteger(1); @ApiOperation(value = "代扣结果查询") @PostMapping("/queryTransRec") Resp<List<TransRecDto>> queryTransRec(@RequestBody Req<TransRecQueryReqDto> transRecQueryReqDtoReq) throws InterruptedException { System.out.println(String.format("查询支付结果......计数: %s", count.getAndAdd(1))); Thread.sleep(500); return Resp.success(RespStatus.SUCCESS.getDesc(), null); } 复制代码
hystrix: threadpool: default: coreSize: 10 maxQueueSize: 1000 queueSizeRejectionThreshold: 20 复制代码
hystrix: threadpool: default: coreSize: 10 maxQueueSize: 15 queueSizeRejectionThreshold: 2000 复制代码
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7d6d472b rejected from java.util.concurrent.ThreadPoolExecutor@17f8bcb7[Running, pool size = 3, active threads = 3, queued tasks = 15, completed tasks = 0] 复制代码
hystrix: threadpool: default: coreSize: 10 maxQueueSize: 1500 复制代码
java.util.concurrent.RejectedExecutionException: Rejected command because thread-pool queueSize is at rejection threshold. 复制代码
hystrix: threadpool: default: coreSize: 10 queueSizeRejectionThreshold: 1000 复制代码
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@23d268ea rejected from java.util.concurrent.ThreadPoolExecutor@66d0e2f4[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) 复制代码
hystrix: threadpool: default: coreSize: 10 maxQueueSize: 1500 queueSizeRejectionThreshold: 1000 复制代码