diff --git a/README-zh.md b/README-zh.md index 85b8afe5a..c818308eb 100644 --- a/README-zh.md +++ b/README-zh.md @@ -9,13 +9,15 @@ Spring Cloud Alibaba 致力于提供微服务开发的一站式解决方案。 依托 Spring Cloud Alibaba,您只需要添加一些注解和少量配置,就可以将 Spring Cloud 应用接入阿里微服务解决方案,通过阿里中间件来迅速搭建分布式应用系统。 +此外,阿里云同时还提供了 Spring Cloud Alibaba 企业版 [微服务解决方案](https://www.aliyun.com/product/aliware/mse?spm=github.spring.com.topbar),包括无侵入服务治理(全链路灰度,无损上下线,离群实例摘除等),企业级 Nacos 注册配置中心和企业级云原生网关等众多产品。 + 参考文档 请查看 [WIKI](https://github.com/alibaba/spring-cloud-alibaba/wiki) 。 为 Spring Cloud Alibaba 贡献代码请参考 [如何贡献](https://github.com/alibaba/spring-cloud-alibaba/wiki/%E5%A6%82%E4%BD%95%E8%B4%A1%E7%8C%AE%E4%BB%A3%E7%A0%81) 。 ## 主要功能 -* **服务限流降级**:默认支持 WebServlet、WebFlux, OpenFeign、RestTemplate、Spring Cloud Gateway, Dubbo 和 RocketMQ 限流降级功能的接入,可以在运行时通过控制台实时修改限流降级规则,还支持查看限流降级 Metrics 监控。 +* **服务限流降级**:默认支持 WebServlet、WebFlux、OpenFeign、RestTemplate、Spring Cloud Gateway、Dubbo 和 RocketMQ 限流降级功能的接入,可以在运行时通过控制台实时修改限流降级规则,还支持查看限流降级 Metrics 监控。 * **服务注册与发现**:适配 Spring Cloud 服务注册与发现标准,默认集成了 Ribbon 的支持。 * **分布式配置管理**:支持分布式系统中的外部化配置,配置更改时自动刷新。 * **消息驱动能力**:基于 Spring Cloud Stream 为微服务应用构建消息驱动能力。 @@ -27,6 +29,8 @@ Spring Cloud Alibaba 致力于提供微服务开发的一站式解决方案。 更多功能请参考 [Roadmap](https://github.com/alibaba/spring-cloud-alibaba/blob/master/Roadmap-zh.md)。 +除了上述所具有的功能外,针对企业级用户的场景,Spring Cloud Alibaba 配套的企业版微服务治理方案 [微服务引擎MSE](https://www.aliyun.com/product/aliware/mse?spm=github.spring.com.topbar) 还提供了企业级微服务治理中心,包括全链路灰度、服务预热、无损上下线和离群实例摘除等更多更强大的治理能力,同时还提供了企业级 Nacos 注册配置中心,企业级云原生网关等多种产品及解决方案。 + ## 组件 **[Sentinel](https://github.com/alibaba/Sentinel)**:把流量作为切入点,从流量控制、熔断降级、系统负载保护等多个维度保护服务的稳定性。 @@ -41,22 +45,23 @@ Spring Cloud Alibaba 致力于提供微服务开发的一站式解决方案。 **[Alibaba Cloud OSS](https://www.aliyun.com/product/oss)**: 阿里云对象存储服务(Object Storage Service,简称 OSS),是阿里云提供的海量、安全、低成本、高可靠的云存储服务。您可以在任何应用、任何时间、任何地点存储和访问任意类型的数据。 -**[Alibaba Cloud SchedulerX](https://help.aliyun.com/document_detail/43136.html)**: 阿里中间件团队开发的一款分布式任务调度产品,提供秒级、精准、高可靠、高可用的定时(基于 Cron 表达式)任务调度服务。 +**[Alibaba Cloud SchedulerX](https://cn.aliyun.com/aliware/schedulerx)**: 阿里中间件团队开发的一款分布式任务调度产品,提供秒级、精准、高可靠、高可用的定时(基于 Cron 表达式)任务调度服务。 **[Alibaba Cloud SMS](https://www.aliyun.com/product/sms)**: 覆盖全球的短信服务,友好、高效、智能的互联化通讯能力,帮助企业迅速搭建客户触达通道。 更多组件请参考 [Roadmap](https://github.com/alibaba/spring-cloud-alibaba/blob/master/Roadmap-zh.md)。 ## 如何构建 - -* master 分支对应的是 Spring Cloud Greenwich,最低支持 JDK 1.8。 +* 2020.0 分支对应的是 Spring Cloud 2020,最低支持 JDK 1.8。 +* master 分支对应的是 Spring Cloud Hoxton,最低支持 JDK 1.8。 +* greenwich 分支对应的是 Spring Cloud Greenwich,最低支持 JDK 1.8。 * finchley 分支对应的是 Spring Cloud Finchley,最低支持 JDK 1.8。 * 1.x 分支对应的是 Spring Cloud Edgware,最低支持 JDK 1.7。 Spring Cloud 使用 Maven 来构建,最快的使用方式是将本项目 clone 到本地,然后执行以下命令: - - ./mvnw install - +```bash +./mvnw install +``` 执行完毕后,项目将被安装到本地 Maven 仓库。 ## 如何使用 @@ -64,19 +69,19 @@ Spring Cloud 使用 Maven 来构建,最快的使用方式是将本项目 clone ### 如何引入依赖 如果需要使用已发布的版本,在 `dependencyManagement` 中添加如下配置。 - - - - - com.alibaba.cloud - spring-cloud-alibaba-dependencies - 2.2.5.RELEASE - pom - import - - - - +```xml + + + + com.alibaba.cloud + spring-cloud-alibaba-dependencies + 2.2.7.RELEASE + pom + import + + + +``` 然后在 `dependencies` 中添加自己所需使用的依赖即可使用。 ## 演示 Demo @@ -111,7 +116,7 @@ Example 列表: * 2.0.x 版本适用于 Spring Boot 2.0.x * 2.1.x 版本适用于 Spring Boot 2.1.x * 2.2.x 版本适用于 Spring Boot 2.2.x - +* 2021.x 版本适用于 Spring Boot 2.4.x ## 社区交流 @@ -121,9 +126,11 @@ spring-cloud-alibaba@googlegroups.com,欢迎通过此邮件列表讨论与 spr ### 钉钉群 -![DingQR](https://img.alicdn.com/tfs/TB1jXikzAL0gK0jSZFtXXXQCXXa-1002-323.png) - -如图片有问题,访问 https://img.alicdn.com/tfs/TB1jXikzAL0gK0jSZFtXXXQCXXa-1002-323.png +* Spring Cloud Alibaba 开源交流群(1群):21914947 +* Spring Cloud Alibaba 开源交流群(2群,已满):21992595 +* Spring Cloud Alibaba 开源交流群(3群):35153903 +* Spring Cloud Alibaba 开源交流群(4群,已满):30301472 +* Spring Cloud Alibaba 开源交流群(5群):34930571 ## 社区相关开源 diff --git a/README.md b/README.md index df9a1d1aa..3f8ec48a5 100644 --- a/README.md +++ b/README.md @@ -16,17 +16,19 @@ With Spring Cloud Alibaba, you only need to add some annotations and a small amo ## Features -* **Flow control and service degradation**:Flow control for HTTP services is supported by default. You can also customize flow control and service degradation rules using annotations. The rules can be changed dynamically. -* **Service registration and discovery**:Service can be registered and clients can discover the instances using Spring-managed beans, auto integration Ribbon. -* **Distributed configuration**:support for externalized configuration in a distributed system, auto refresh when configuration changes. -* **Event-driven**:support for building highly scalable event-driven microservices connected with shared messaging systems. -* **Distributed Transaction**:support for distributed transaction solution with high performance and ease of use. -* **Alibaba Cloud Object Storage**:massive, secure, low-cost, and highly reliable cloud storage services. Support for storing and accessing any type of data in any application, anytime, anywhere. -* **Alibaba Cloud SchedulerX**:accurate, highly reliable, and highly available scheduled job scheduling services with response time within seconds. -* **Alibaba Cloud SMS**: A messaging service that covers the globe, Alibaba SMS provides convenient, efficient, and intelligent communication capabilities that help businesses quickly contact their customers. +* **Flow control and service degradation**: Flow control for HTTP services is supported by default. You can also customize flow control and service degradation rules using annotations. The rules can be changed dynamically. +* **Service registration and discovery**: Service can be registered and clients can discover the instances using Spring-managed beans, auto integration Ribbon. +* **Distributed configuration**: Support for externalized configuration in a distributed system, auto refresh when configuration changes. +* **Event-driven**: Support for building highly scalable event-driven microservices connected with shared messaging systems. +* **Distributed Transaction**: Support for distributed transaction solution with high performance and ease of use. +* **Alibaba Cloud Object Storage**: Massive, secure, low-cost, and highly reliable cloud storage services. Support for storing and accessing any type of data in any application, anytime, anywhere. +* **Alibaba Cloud SchedulerX**: Accurate, highly reliable, and highly available scheduled job scheduling services with response time within seconds. +* **Alibaba Cloud SMS**: A messaging service that covers the globe, Alibaba SMS provides convenient, efficient, and intelligent communication capabilities that help businesses quickly contact their customers. For more features, please refer to [Roadmap](https://github.com/alibaba/spring-cloud-alibaba/blob/master/Roadmap.md). +In addition to the above-mentioned features, for the needs of enterprise users' scenarios, [Microservices Engine (MSE)](https://www.aliyun.com/product/aliware/mse?spm=github.spring.com.topbar) of Spring Cloud Alibaba's enterprise version provides an enterprise-level microservices governance center, which includes more powerful governance capabilities such as Grayscale Release, Service Warm-up, Lossless Online and Offline and Outlier Ejection. At the same time, it also provides a variety of products and solutions such as enterprise-level Nacos registration / configuration center, enterprise-level cloud native gateway. + ## Components @@ -34,51 +36,52 @@ For more features, please refer to [Roadmap](https://github.com/alibaba/spring-c **[Nacos](https://github.com/alibaba/Nacos)**: An easy-to-use dynamic service discovery, configuration and service management platform for building cloud native applications. -**[RocketMQ](https://rocketmq.apache.org/)**:A distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability. +**[RocketMQ](https://rocketmq.apache.org/)**: A distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability. -**[Dubbo](https://github.com/apache/dubbo)**:A high-performance, Java based open source RPC framework. +**[Dubbo](https://github.com/apache/dubbo)**: A high-performance, Java based open source RPC framework. -**[Seata](https://github.com/seata/seata)**:A distributed transaction solution with high performance and ease of use for microservices architecture. +**[Seata](https://github.com/seata/seata)**: A distributed transaction solution with high performance and ease of use for microservices architecture. -**[Alibaba Cloud ACM](https://www.aliyun.com/product/acm)**:An application configuration center that enables you to centralize the management of application configurations, and accomplish real-time configuration push in a distributed environment. +**[Alibaba Cloud ACM](https://www.aliyun.com/product/acm)**: An application configuration center that enables you to centralize the management of application configurations, and accomplish real-time configuration push in a distributed environment. **[Alibaba Cloud OSS](https://www.aliyun.com/product/oss)**: An encrypted and secure cloud storage service which stores, processes and accesses massive amounts of data from anywhere in the world. **[Alibaba Cloud SMS](https://www.aliyun.com/product/sms)**: A messaging service that covers the globe, Alibaba SMS provides convenient, efficient, and intelligent communication capabilities that help businesses quickly contact their customers. -**[Alibaba Cloud SchedulerX](https://www.aliyun.com/aliware/schedulerx?spm=5176.10695662.784137.1.4b07363dej23L3)**:accurate, highly reliable, and highly available scheduled job scheduling services with response time within seconds.. +**[Alibaba Cloud SchedulerX](https://www.aliyun.com/aliware/schedulerx?spm=5176.10695662.784137.1.4b07363dej23L3)**: Accurate, highly reliable, and highly available scheduled job scheduling services with response time within seconds.. For more features please refer to [Roadmap](https://github.com/alibaba/spring-cloud-alibaba/blob/master/Roadmap.md). ## How to build - -* **master branch**: Corresponds to Spring Cloud Greenwich & Spring Boot 2.x. JDK 1.8 or later versions are supported. -* **finchley branch**: Corresponds to Spring Cloud Finchley & Spring Boot 2.x. JDK 1.8 or later versions are supported. +* **2020.0 branch**: Corresponds to Spring Cloud 2020 & Spring Boot 2.4.x. JDK 1.8 or later versions are supported. +* **master branch**: Corresponds to Spring Cloud Hoxton & Spring Boot 2.2.x. JDK 1.8 or later versions are supported. +* **greenwich branch**: Corresponds to Spring Cloud Greenwich & Spring Boot 2.1.x. JDK 1.8 or later versions are supported. +* **finchley branch**: Corresponds to Spring Cloud Finchley & Spring Boot 2.0.x. JDK 1.8 or later versions are supported. * **1.x branch**: Corresponds to Spring Cloud Edgware & Spring Boot 1.x, JDK 1.7 or later versions are supported. Spring Cloud uses Maven for most build-related activities, and you should be able to get off the ground quite quickly by cloning the project you are interested in and typing: - - ./mvnw install - +```bash +./mvnw install +``` ## How to Use ### Add maven dependency These artifacts are available from Maven Central and Spring Release repository via BOM: - - - - - com.alibaba.cloud - spring-cloud-alibaba-dependencies - 2.2.5.RELEASE - pom - import - - - - +```xml + + + + com.alibaba.cloud + spring-cloud-alibaba-dependencies + 2.2.7.RELEASE + pom + import + + + +``` add the module in `dependencies`. @@ -118,6 +121,7 @@ As the interfaces and annotations of Spring Boot 1 and Spring Boot 2 have been c * 2.0.x for Spring Boot 2.0.x * 2.1.x for Spring Boot 2.1.x * 2.2.x for Spring Boot 2.2.x +* 2020.x for Spring Boot 2.4.x ## Code of Conduct This project is a sub-project of Spring Cloud, it adheres to the Contributor Covenant [code of conduct](https://github.com/spring-cloud/spring-cloud-build/blob/master/docs/src/main/asciidoc/code-of-conduct.adoc). By participating, you are expected to uphold this code. Please report unacceptable behavior to spring-code-of-conduct@pivotal.io. @@ -135,7 +139,7 @@ Add yourself as an @author to the .java files that you modify substantially (mor Add some Javadocs and, if you change the namespace, some XSD doc elements. -A few unit tests would help a lot as well — someone has to do it. +A few unit tests would help a lot as well —— someone has to do it. If no-one else is using your branch, please rebase it against the current master (or other target branch in the main project). @@ -144,4 +148,4 @@ When writing a commit message please follow these conventions, if you are fixing ## Contact Us Mailing list is recommended for discussing almost anything related to spring-cloud-alibaba. -spring-cloud-alibaba@googlegroups.com:You can ask questions here if you encounter any problem when using or developing spring-cloud-alibaba. +spring-cloud-alibaba@googlegroups.com: You can ask questions here if you encounter any problem when using or developing spring-cloud-alibaba. diff --git a/pom.xml b/pom.xml index 20d4a9c65..d47f54403 100644 --- a/pom.xml +++ b/pom.xml @@ -90,7 +90,7 @@ 4.0.1 - 2.0.2 + 4.6.1 3.7.0 @@ -181,8 +181,13 @@ org.apache.rocketmq - rocketmq-spring-boot-starter - ${rocketmq.starter.version} + rocketmq-client + ${rocketmq.version} + + + org.apache.rocketmq + rocketmq-acl + ${rocketmq.version} diff --git a/spring-cloud-alibaba-dependencies/pom.xml b/spring-cloud-alibaba-dependencies/pom.xml index 894a3ea2d..d29e83344 100644 --- a/spring-cloud-alibaba-dependencies/pom.xml +++ b/spring-cloud-alibaba-dependencies/pom.xml @@ -19,11 +19,11 @@ 2022.0-SNAPSHOT - 1.8.0 + 1.8.1 1.3.0 - 1.4.1 + 2.0.3 0.8.0 - 1.0.10 + 1.0.11 2.2.1 diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/nacos-config.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/nacos-config.adoc index 4f7de3127..0c4c6305b 100644 --- a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/nacos-config.adoc +++ b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/nacos-config.adoc @@ -260,7 +260,7 @@ Nacos 内部有 https://nacos.io/zh-cn/docs/concepts.html[Namespace 的概念]: [quote] 用于进行租户粒度的配置隔离。不同的命名空间下,可以存在相同的 Group 或 Data ID 的配置。Namespace 的常用场景之一是不同环境的配置的区分隔离,例如开发测试环境和生产环境的资源(如配置、服务)隔离等。 -在没有明确指定 `${spring.cloud.nacos.config.namespace}` 配置的情况下, 默认使用的是 Nacos 上 Public 这个namespae。如果需要使用自定义的命名空间,可以通过以下配置来实现: +在没有明确指定 `${spring.cloud.nacos.config.namespace}` 配置的情况下, 默认使用的是 Nacos 上 Public 这个namespace。如果需要使用自定义的命名空间,可以通过以下配置来实现: [source,properties] ---- spring.cloud.nacos.config.namespace=b3404bc0-d7dc-4855-b519-570ed34b62d7 @@ -281,7 +281,7 @@ NOTE: 该配置必须放在 bootstrap.properties 文件中。并且在添加配 === 支持自定义扩展的 Data Id 配置 -Nacos Config 从 0.2.1 版本后,可支持自定义 Data Id 的配置。关于这部分详细的设计可参考 https://github.com/spring-cloud-incubator/spring-cloud-alibaba/issues/141[这里]。 +Nacos Config 从 0.2.1 版本后,可支持自定义 Data Id 的配置。关于这部分详细的设计可参考 https://github.com/alibaba/spring-cloud-alibaba/issues/141[这里]。 一个完整的配置案例如下所示: [source,properties] @@ -348,7 +348,7 @@ Nacos Config 目前提供了三种配置能力从 Nacos 拉取相关的配置 * B: 通过 `spring.cloud.nacos.config.ext-config[n].data-id` 的方式支持多个扩展 Data Id 的配置 * C: 通过内部相关规则(应用名、应用名+ Profile )自动生成相关的 Data Id 配置 -当三种方式共同使用时,他们的一个优先级关系是:A < B < C +当三种方式共同使用时,他们的一个优先级关系是: A < B < C === Nacos Config 对外暴露的 Endpoint diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/rocketmq-new.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/rocketmq-new.adoc new file mode 100644 index 000000000..80b6a280b --- /dev/null +++ b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/rocketmq-new.adoc @@ -0,0 +1,378 @@ +== Spring Cloud Alibaba RocketMQ Binder (NEW) + +=== RocketMQ 介绍 + +https://rocketmq.apache.org[RocketMQ] 是一款开源的分布式消息系统,基于高可用分布式集群技术,提供低延时的、高可靠的消息发布与订阅服务。同时,广泛应用于多个领域,包括异步通信解耦、企业解决方案、金融支付、电信、电子商务、快递物流、广告营销、社交、即时通信、移动应用、手游、视频、物联网、车联网等。 + +具有以下特点: + +* 能够保证严格的消息顺序 + +* 提供丰富的消息拉取模式 + +* 高效的订阅者水平扩展能力 + +* 实时的消息订阅机制 + +* 亿级消息堆积能力 + +=== RocketMQ 基本使用 + +* 下载 RocketMQ + +下载 https://www.apache.org/dyn/closer.cgi?path=rocketmq/4.3.2/rocketmq-all-4.3.2-bin-release.zip[RocketMQ最新的二进制文件],并解压 + +解压后的目录结构如下: + +``` +apache-rocketmq +├── LICENSE +├── NOTICE +├── README.md +├── benchmark +├── bin +├── conf +└── lib +``` + +* 启动 NameServer + +```bash +nohup sh bin/mqnamesrv & +tail -f ~/logs/rocketmqlogs/namesrv.log +``` + +* 启动 Broker + +```bash +nohup sh bin/mqbroker -n localhost:9876 & +tail -f ~/logs/rocketmqlogs/broker.log +``` + +* 发送、接收消息 + +发送消息: + +```bash +sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer +``` + +发送成功后显示:`SendResult [sendStatus=SEND_OK, msgId= ...` + +接收消息: + +```bash +sh bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer +``` + +接收成功后显示:`ConsumeMessageThread_%d Receive New Messages: [MessageExt...` + +* 关闭 Server + +```bash +sh bin/mqshutdown broker +sh bin/mqshutdown namesrv +``` + +=== Spring Cloud Stream 介绍 + +Spring Cloud Stream 是一个用于构建基于消息的微服务应用框架。它基于 SpringBoot 来创建具有生产级别的单机 Spring 应用,并且使用 `Spring Integration` 与 Broker 进行连接。 + +Spring Cloud Stream 提供了消息中间件配置的统一抽象,推出了 publish-subscribe、consumer groups、partition 这些统一的概念。 + +Spring Cloud Stream 内部有两个概念:Binder 和 Binding。 + +* Binder: 跟外部消息中间件集成的组件,用来创建 Binding,各消息中间件都有自己的 Binder 实现。 + +比如 `Kafka` 的实现 `KafkaMessageChannelBinder`,`RabbitMQ` 的实现 `RabbitMessageChannelBinder` 以及 `RocketMQ` 的实现 `RocketMQMessageChannelBinder`。 + +* Binding: 包括 Input Binding 和 Output Binding。 + +Binding 在消息中间件与应用程序提供的 Provider 和 Consumer 之间提供了一个桥梁,实现了开发者只需使用应用程序的 Provider 或 Consumer 生产或消费数据即可,屏蔽了开发者与底层消息中间件的接触。 + +.Spring Cloud Stream +image::https://docs.spring.io/spring-cloud-stream/docs/current/reference/htmlsingle/images/SCSt-overview.png[] + +使用 Spring Cloud Stream 完成一段简单的消息发送和消息接收代码: + +```java +MessageChannel messageChannel = new DirectChannel(); + +// 消息订阅 +((SubscribableChannel) messageChannel).subscribe(new MessageHandler() { + @Override + public void handleMessage(Message message) throws MessagingException { + System.out.println("receive msg: " + message.getPayload()); + } +}); + +// 消息发送 +messageChannel.send(MessageBuilder.withPayload("simple msg").build()); +``` + +这段代码所有的消息类都是 `spring-messaging` 模块里提供的。屏蔽具体消息中间件的底层实现,如果想用更换消息中间件,在配置文件里配置相关消息中间件信息以及修改 binder 依赖即可。 + +**Spring Cloud Stream 底层基于这段代码去做了各种抽象。** + + +=== 如何使用 Spring Cloud Alibaba RocketMQ Binder + +如果要在您的项目中引入 RocketMQ Binder,需要引入如下 maven 依赖: + +```xml + + com.alibaba.cloud + spring-cloud-stream-binder-rocketmq + +``` + +或者可以使用 Spring Cloud Stream RocketMQ Starter: + +```xml + + com.alibaba.cloud + spring-cloud-starter-stream-rocketmq + +``` + +=== Spring Cloud Alibaba RocketMQ Binder 实现 + +这是 Spring Cloud Stream RocketMQ Binder 的实现架构: + +.SCS RocketMQ Binder +image::https://img.alicdn.com/tfs/TB1v8rcbUY1gK0jSZFCXXcwqXXa-1236-773.png[] + +RocketMQ Binder 的重构优化去除了对 https://github.com/apache/rocketmq-spring[RocketMQ-Spring]框架的依赖 。 +RocketMQ Binder 核心类 `RocketMQMessageChannelBinder` 实现了 Spring Cloud Stream 规范,内部会构建 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQInboundChannelAdapter.java[RocketMQInboundChannelAdapter] 和 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProducerMessageHandler.java[RocketMQProducerMessageHandler]。 + +`RocketMQProducerMessageHandler` 会基于 Binding 配置通过 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProduceFactory.java[RocketMQProduceFactory]构造 RocketMQ Producer,其内部会把 `spring-messaging` 模块内 `org.springframework.messaging.Message` 消息类转换成 RocketMQ 的消息类 `org.apache.rocketmq.common.message.Message`,然后发送出去。 + +`RocketMQInboundChannelAdapter` 也会基于 Binding 配置通过 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQConsumerFactory.java[RocketMQConsumerFactory]构造 DefaultMQPushConsumer,其内部会启动 RocketMQ Consumer 接收消息。 + +NOTE: 与 https://github.com/apache/rocketmq-spring[RocketMQ-Spring] 框架的兼容需要手动处理 + +目前 Binder 支持在 `Header` 中设置相关的 key 来进行 RocketMQ Message 消息的特性设置。 + +比如 `TAGS`、`KEYS`、`TRANSACTIONAL_ARGS` 等 RocketMQ 消息对应的标签,详情见 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/contants/RocketMQConst.java[com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst] + +```java +MessageBuilder builder = MessageBuilder.withPayload(msg) + .setHeader(RocketMQHeaders.TAGS, "binder") + .setHeader(RocketMQHeaders.KEYS, "my-key"); +Message message = builder.build(); +output().send(message); +``` +NOTE: 更多使用请参考样例: https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/SenderService.java[com.alibaba.cloud.examples.SenderService] + +=== MessageSource 支持 + +SCS RocketMQ Binder 支持 `MessageSource`,可以进行消息的拉取,例子如下: + +```java +@SpringBootApplication +@EnableBinding(MQApplication.PolledProcessor.class) +public class MQApplication { + + private final Logger logger = + LoggerFactory.getLogger(MQApplication.class); + + public static void main(String[] args) { + SpringApplication.run(MQApplication.class, args); + } + + @Bean + public ApplicationRunner runner(PollableMessageSource source, + MessageChannel dest) { + return args -> { + while (true) { + boolean result = source.poll(m -> { + String payload = (String) m.getPayload(); + logger.info("Received: " + payload); + dest.send(MessageBuilder.withPayload(payload.toUpperCase()) + .copyHeaders(m.getHeaders()) + .build()); + }, new ParameterizedTypeReference() { }); + if (result) { + logger.info("Processed a message"); + } + else { + logger.info("Nothing to do"); + } + Thread.sleep(5_000); + } + }; + } + + public static interface PolledProcessor { + + @Input + PollableMessageSource source(); + + @Output + MessageChannel dest(); + + } + +} +``` + + + +=== 配置选项 + +==== RocketMQ Binder Properties + +spring.cloud.stream.rocketmq.binder.name-server:: +RocketMQ NameServer 地址(老版本使用 namesrv-addr 配置项)。 ++ +Default: `127.0.0.1:9876`. +spring.cloud.stream.rocketmq.binder.access-key:: +阿里云账号 AccessKey。 ++ +Default: null. +spring.cloud.stream.rocketmq.binder.secret-key:: +阿里云账号 SecretKey。 ++ +Default: null. +spring.cloud.stream.rocketmq.binder.enable-msg-trace:: +是否为 Producer 和 Consumer 开启消息轨迹功能 ++ +Default: `true`. +spring.cloud.stream.rocketmq.binder.customized-trace-topic:: +消息轨迹开启后存储的 topic 名称。 ++ +Default: `RMQ_SYS_TRACE_TOPIC`. + + +==== RocketMQ Consumer Properties + +下面的这些配置是以 `spring.cloud.stream.rocketmq.bindings..consumer.` 为前缀的 RocketMQ Consumer 相关的配置。 +更多见 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQConsumerProperties.java[com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties]。 + +enable:: +是否启用 Consumer。 ++ +默认值: `true`. +subscription:: +Consumer 基于 TAGS 订阅,多个 tag 以 `||` 分割。更多见 `com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties.subscription` ++ +默认值: empty. +messageModel:: +Consumer 消费模式。如果想让每一个的订阅者都能接收到消息,可以使用广播模式。更多见 `org.apache.rocketmq.common.protocol.heartbeat.MessageModel` ++ +默认值: `CLUSTERING`. +consumeFromWhere:: +Consumer 从哪里开始消费。更多见 `org.apache.rocketmq.common.consumer.ConsumeFromWhere` ++ +默认值: `CONSUME_FROM_LAST_OFFSET`. + +#下面的这些配置是 Consumer Push 模式相关的配置。# + `spring.cloud.stream.rocketmq.bindings..consumer.push.` + +orderly:: +是否同步消费消息模式 ++ +默认值: `false`. +delayLevelWhenNextConsume:: +异步消费消息模式下消费失败重试策略: +* -1,不重复,直接放入死信队列 +* 0,broker 控制重试策略 +* >0,client 控制重试策略 ++ +默认值: `0`. +suspendCurrentQueueTimeMillis:: +同步消费消息模式下消费失败后再次消费的时间间隔。 ++ +默认值: `1000`. + +其他更多参数见 `com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties.Push` + +#下面的这些配置是 Consumer Pull 模式相关的配置。# +`spring.cloud.stream.rocketmq.bindings..consumer.pull.` + +pullThreadNums:: +消费时拉取的线程数 ++ +默认值: `20`. +pollTimeoutMillis:: +拉取时的超时毫秒数 ++ +默认值: `1000 * 5`. + +其他更多参数见 `com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties.Pull`. + +NOTE: 更多参数见 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQConsumerProperties.java[com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties] + +==== RocketMQ Provider Properties + +下面的这些配置是以 `spring.cloud.stream.rocketmq.bindings..producer.` 为前缀的 RocketMQ Producer 相关的配置。更多见 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQProducerProperties.java[com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties] + +enable:: +是否启用 Producer。 ++ +默认值: `true`. +group:: +Producer group name。 ++ +默认值: empty. +maxMessageSize:: +消息发送的最大字节数。 ++ +默认值: `8249344`. +producerType:: +消息生产者类型,普通或者事务。更多见 `com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties.ProducerType`. ++ +默认值: `Normal`. +transactionListener:: +事务消息监听器的beanName,在 `producerType=Trans` 时才有效;必须是实现 `org.apache.rocketmq.client.producer.TransactionListener` 接口的Spring Bean。 + +sendType:: +消息发送类型(同步、异步、单向)。更多见`com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties.SendType`. ++ +默认值: `Sync`. +sendCallBack:: +消息发送后回调函数的beanName,在 `sendType=Async` 时才有效;必须是实现 `org.apache.rocketmq.client.producer.SendCallback` 接口的Spring Bean。 +vipChannelEnabled:: +是否在 Vip Channel 上发送消息。 ++ +默认值: `true`. +sendMessageTimeout:: +发送消息的超时时间(毫秒)。 ++ +默认值: `3000`. +compressMessageBodyThreshold:: +消息体压缩阀值(当消息体超过 4k 的时候会被压缩)。 ++ +默认值: `4096`. +retryTimesWhenSendFailed:: +在同步发送消息的模式下,消息发送失败的重试次数。 ++ +默认值: `2`. +retryTimesWhenSendAsyncFailed:: +在异步发送消息的模式下,消息发送失败的重试次数。 ++ +默认值: `2`. +retryAnotherBroker:: +消息发送失败的情况下是否重试其它的 broker。 ++ +默认值: `false`. + +NOTE: 生产者其他更多参数请见: +https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQProducerProperties.java[com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties] + +=== 阿里云 MQ 服务 + +使用阿里云 MQ 服务需要配置 AccessKey、SecretKey 以及云上的 NameServer 地址。 + +NOTE: 0.1.2 & 0.2.2 & 0.9.0 才支持该功能 + +```properties +spring.cloud.stream.rocketmq.binder.access-key=YourAccessKey +spring.cloud.stream.rocketmq.binder.secret-key=YourSecretKey +spring.cloud.stream.rocketmq.binder.name-server=NameServerInMQ +``` + +NOTE: topic 和 group 请以 实例id% 为前缀进行配置。比如 topic 为 "test",需要配置成 "实例id%test" + +.NameServer 的获取(配置中请去掉 http:// 前缀) +image::https://spring-cloud-alibaba.oss-cn-beijing.aliyuncs.com/MQ.png[] diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/rocketmq.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/rocketmq.adoc index 340da4d8b..7823c649f 100644 --- a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/rocketmq.adoc +++ b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/rocketmq.adoc @@ -249,6 +249,11 @@ spring.cloud.stream.rocketmq.binder.customized-trace-topic:: 消息轨迹开启后存储的 topic 名称。 + Default: `RMQ_SYS_TRACE_TOPIC`. ++ +spring.cloud.stream.rocketmq.binder.access-channel:: +商业版rocketmq消息轨迹topic自适应,值为CLOUD ++ +Default: null. ==== RocketMQ Consumer Properties @@ -346,6 +351,7 @@ NOTE: 0.1.2 & 0.2.2 & 0.9.0 才支持该功能 spring.cloud.stream.rocketmq.binder.access-key=YourAccessKey spring.cloud.stream.rocketmq.binder.secret-key=YourSecretKey spring.cloud.stream.rocketmq.binder.name-server=NameServerInMQ +spring.cloud.stream.rocketmq.binder.access-channel=CLOUD ``` NOTE: topic 和 group 请以 实例id% 为前缀进行配置。比如 topic 为 "test",需要配置成 "实例id%test" diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/sentinel.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/sentinel.adoc index 6dd603354..2a45f6d13 100644 --- a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/sentinel.adoc +++ b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/sentinel.adoc @@ -222,7 +222,7 @@ spring.cloud.sentinel.datasource.ds1.file.rule-type=flow #spring.cloud.sentinel.datasource.ds1.file.converter-class=org.springframework.cloud.alibaba.cloud.examples.JsonFlowRuleListConverter #spring.cloud.sentinel.datasource.ds1.file.rule-type=flow -spring.cloud.sentinel.datasource.ds2.nacos.server-addr=localhost:8848 +spring.cloud.sentinel.datasource.ds2.nacos.server-addr=127.0.0.1:8848 spring.cloud.sentinel.datasource.ds2.nacos.data-id=sentinel spring.cloud.sentinel.datasource.ds2.nacos.group-id=DEFAULT_GROUP spring.cloud.sentinel.datasource.ds2.nacos.data-type=json diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/sidecar.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/sidecar.adoc index 2a8832f52..07f448cff 100644 --- a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/sidecar.adoc +++ b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/sidecar.adoc @@ -3,7 +3,7 @@ `Spring Cloud Alibaba Sidecar` 是一个用来快速**完美整合** Spring Cloud 与 *异构微服务* 的框架,灵感来自 https://github.com/spring-cloud/spring-cloud-netflix/tree/master/spring-cloud-netflix-sidecar[Spring -Cloud Netflix Sidecar] 。目前支持的服务发现组件: +Cloud Netflix Sidecar],目前支持的服务发现组件: * Nacos * Consul @@ -14,7 +14,7 @@ Cloud Netflix Sidecar] 。目前支持的服务发现组件: 非Spring Cloud应用,统称异构微服务。比如你的遗留项目,或者非JVM应用。 -==== ``完美整合''的三层含义 +==== "完美整合"的三层含义 * 享受服务发现的优势 * 有负载均衡 @@ -89,7 +89,7 @@ spring: cloud: nacos: discovery: - server-addr: localhost:8848 + server-addr: 127.0.0.1:8848 gateway: discovery: locator: diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/spring-cloud-alibaba.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/spring-cloud-alibaba.adoc index 13e0e76ee..80f858ea0 100644 --- a/spring-cloud-alibaba-docs/src/main/asciidoc-zh/spring-cloud-alibaba.adoc +++ b/spring-cloud-alibaba-docs/src/main/asciidoc-zh/spring-cloud-alibaba.adoc @@ -19,6 +19,8 @@ include::sentinel.adoc[] include::dubbo.adoc[] +include::rocketmq-new.adoc[] + include::rocketmq.adoc[] include::ans.adoc[] diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc/nacos-config.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc/nacos-config.adoc index 5aa199c11..c9f97b464 100644 --- a/spring-cloud-alibaba-docs/src/main/asciidoc/nacos-config.adoc +++ b/spring-cloud-alibaba-docs/src/main/asciidoc/nacos-config.adoc @@ -280,7 +280,7 @@ NOTE: This configuration must be in the bootstrap.properties file, and the value === Support Custom Data Id -As of Spring Cloud Alibaba Nacos Config, data id can be self-defined. For detailed design of this part, refer to https://github.com/spring-cloud-incubator/spring-cloud-alibaba/issues/141[Github issue]. +As of Spring Cloud Alibaba Nacos Config, data id can be self-defined. For detailed design of this part, refer to https://github.com/alibaba/spring-cloud-alibaba/issues/141[Github issue]. The following is a complete sample: [source,properties] diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc/rocketmq.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc/rocketmq.adoc index 540d42c41..4362375d9 100644 --- a/spring-cloud-alibaba-docs/src/main/asciidoc/rocketmq.adoc +++ b/spring-cloud-alibaba-docs/src/main/asciidoc/rocketmq.adoc @@ -57,7 +57,7 @@ Send messages: sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer ``` -Output when the message is successfuly sent: `SendResult [sendStatus=SEND_OK, msgId= ...` +Output when the message is successfully sent: `SendResult [sendStatus=SEND_OK, msgId= ...` Receive messages: @@ -246,7 +246,11 @@ spring.cloud.stream.rocketmq.binder.customized-trace-topic:: The trace topic for message trace. + Default: `RMQ_SYS_TRACE_TOPIC`. - ++ +spring.cloud.stream.rocketmq.binder.access-channel:: +The commercial version of rocketmq message trajectory topic is adaptive,the value is CLOUD ++ +Default: null. ==== RocketMQ Consumer Properties diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc/schedulerx.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc/schedulerx.adoc index 38c79c50e..ea68fd8a9 100644 --- a/spring-cloud-alibaba-docs/src/main/asciidoc/schedulerx.adoc +++ b/spring-cloud-alibaba-docs/src/main/asciidoc/schedulerx.adoc @@ -82,7 +82,7 @@ Job Description: Empty Custom Parameters: Empty ---- -The job above is a “Simple Single-Server Job”, and speficied a Cron expression of "0 * * * * ?" . This means that the job will be executed once and once only in every minute. +The job above is a “Simple Single-Server Job”, and specified a Cron expression of "0 * * * * ?" . This means that the job will be executed once and once only in every minute. For more job types, refer to https://help.aliyun.com/document_detail/43136.html[SchedulerX Documentation]. diff --git a/spring-cloud-alibaba-docs/src/main/asciidoc/sentinel.adoc b/spring-cloud-alibaba-docs/src/main/asciidoc/sentinel.adoc index da132203e..0dfac8014 100644 --- a/spring-cloud-alibaba-docs/src/main/asciidoc/sentinel.adoc +++ b/spring-cloud-alibaba-docs/src/main/asciidoc/sentinel.adoc @@ -225,7 +225,7 @@ spring.cloud.sentinel.datasource.ds1.file.rule-type=flow #spring.cloud.sentinel.datasource.ds1.file.converter-class=JsonFlowRuleListConverter #spring.cloud.sentinel.datasource.ds1.file.rule-type=flow -spring.cloud.sentinel.datasource.ds2.nacos.server-addr=localhost:8848 +spring.cloud.sentinel.datasource.ds2.nacos.server-addr=127.0.0.1:8848 spring.cloud.sentinel.datasource.ds2.nacos.data-id=sentinel spring.cloud.sentinel.datasource.ds2.nacos.group-id=DEFAULT_GROUP spring.cloud.sentinel.datasource.ds2.nacos.data-type=json diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/pom.xml b/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/pom.xml index 09b031af6..891376567 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/pom.xml +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/pom.xml @@ -24,12 +24,6 @@ com.alibaba.cloud spring-cloud-starter-alibaba-nacos-config - - - org.springframework.cloud - spring-cloud-starter-bootstrap - - diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/readme-zh.md b/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/readme-zh.md index 363a4cb50..1b26a025b 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/readme-zh.md +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/readme-zh.md @@ -46,7 +46,7 @@ spring: # 2.4.0 新增配置 spring.config.import config: import: - - optional:nacos:localhost:8848 + - optional:nacos:127.0.0.1:8848 ``` 3. 在 nacos 创建 test.yml diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/src/main/resources/application.yml b/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/src/main/resources/application.yml index 6047c5ab3..cdc55e868 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/src/main/resources/application.yml +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-config-2.4.x-example/src/main/resources/application.yml @@ -7,9 +7,12 @@ spring: nacos: config: group: DEFAULT_GROUP - server-addr: localhost:8848 + server-addr: 127.0.0.1:8848 config: import: - optional:nacos:test.yml - optional:nacos:test01.yml?group=group_02 - optional:nacos:test02.yml?group=group_03&refreshEnabled=false +logging: + level: + com.alibaba.nacos: debug diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/readme-zh.md b/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/readme-zh.md index c58f67833..76167ca4b 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/readme-zh.md +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/readme-zh.md @@ -61,9 +61,9 @@ 内容如下 - user.id=1 - user.name=james - user.age=17 + user.id=1 + user.name=james + user.age=17 ### 应用启动 @@ -117,7 +117,7 @@ Nacos Client 从 Nacos Server 端获取数据时,调用的是此接口 `Config * `spring.profiles.active` 即为当前环境对应的 profile,详情可以参考 [Spring Boot文档](https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-profiles.html#boot-features-profiles) - **注意,当 activeprofile 为空时,对应的连接符 `-` 也将不存在,dataId 的拼接格式变成 `${prefix}`.`${file-extension}`** + **注意,当 active profile 为空时,对应的连接符 `-` 也将不存在,dataId 的拼接格式变成 `${prefix}`.`${file-extension}`** * `file-extension` 为配置内容的数据格式,可以通过配置项 `spring.cloud.nacos.config.file-extension`来配置。 目前只支持 `properties` 类型。 @@ -153,7 +153,7 @@ Spring Boot 应用支持通过 Endpoint 来暴露相关信息,Nacos Config Sta Spring Boot 1.x 可以通过访问 http://127.0.0.1:18084/nacos_config 来查看 Nacos Endpoint 的信息。 -Spring Boot 2.x 可以通过访问 http://127.0.0.1:18084/actuator/nacos-config 来访问。 +Spring Boot 2.x 可以通过访问 http://127.0.0.1:18084/actuator/nacosconfig 来访问。 ![actuator](https://cdn.nlark.com/lark/0/2018/png/54319/1536986344822-279e1edc-ebca-4201-8362-0ddeff240b85.png) diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/readme.md b/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/readme.md index 7584f72a7..16441654f 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/readme.md +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/readme.md @@ -46,7 +46,7 @@ Before we start the demo, let's learn how to connect Nacos Config to a Spring Cl 2. Unzip the downloaded file and go to the nacos/bin folder(), And according to the actual situation of the operating system, execute the following command。[see reference for more detail](https://nacos.io/en-us/docs/quick-start.html)。 1. Linux/Unix/Mac , execute `sh startup.sh -m standalone` - 2. Windows , execute `cmd startup.cmd` + 2. Windows , execute `cmd startup.cmd -m standalone` 3. Execute the following command to add a configuration to Nacos Server. @@ -62,9 +62,9 @@ Before we start the demo, let's learn how to connect Nacos Config to a Spring Cl content is - user.id=1 - user.name=james - user.age=17 + user.id=1 + user.name=james + user.age=17 ### Start Application @@ -119,7 +119,7 @@ In Nacos Config Starter, the splicing format of dataId is as follows * `spring.profiles.active` is the profile corresponding to the current environment. For details, please refer to [Spring Boot Doc](https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-profiles.html#boot-features-profiles) - **Note: when the activeprofile is empty, the corresponding connector `-` will also not exist, and the splicing format of the dataId becomes `${prefix}`.`${file-extension}`** + **Note: when the active profile is empty, the corresponding connector `-` will also not exist, and the splicing format of the dataId becomes `${prefix}`.`${file-extension}`** * `file-extension` is the data format of the configuration content, which can be configured by the configuration item `spring.cloud.nacos.config.file-extension`. Currently only the `properties` type is supported. @@ -139,7 +139,7 @@ By default, Nacos Config Starter adds a listening function to all Nacos configur If you need to dynamically refresh a bean, please refer to the Spring and Spring Cloud specifications. It is recommended to add `@RefreshScope` or `@ConfigurationProperties ` annotations to the class. -Please refer to[ContextRefresher Java Doc](http://static.javadoc.io/org.springframework.cloud/spring-cloud-context/2.0.0.RELEASE/org/springframework/cloud/context/refresh/ContextRefresher.html) for more details. +Please refer to [ContextRefresher Java Doc](http://static.javadoc.io/org.springframework.cloud/spring-cloud-context/2.0.0.RELEASE/org/springframework/cloud/context/refresh/ContextRefresher.html) for more details. @@ -156,8 +156,8 @@ Spring Boot 1.x: Add configuration management.security.enabled=false Spring Boot 2.x: Add configuration management.endpoints.web.exposure.include=* To view the endpoint information, visit the following URLS: -Spring Boot1.x: Nacos Config Endpoint URL is http://127.0.0.1:18083/nacos_config. -Spring Boot2.x: Nacos Config Endpoint URL is http://127.0.0.1:18083/actuator/nacos-config. +Spring Boot 1.x: Nacos Config Endpoint URL is http://127.0.0.1:18084/nacos_config. +Spring Boot 2.x: Nacos Config Endpoint URL is http://127.0.0.1:18084/actuator/nacosconfig. ![actuator](https://cdn.nlark.com/lark/0/2018/png/54319/1536986344822-279e1edc-ebca-4201-8362-0ddeff240b85.png) diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/src/main/resources/bootstrap.properties b/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/src/main/resources/bootstrap.properties index 4d880443f..691cba2d1 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/src/main/resources/bootstrap.properties +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-config-example/src/main/resources/bootstrap.properties @@ -18,21 +18,21 @@ spring.cloud.nacos.password=nacos #spring.cloud.nacos.config.shared-data-ids=common.properties,base-common.properties ## recommended. -spring.cloud.nacos.config.shared-configs[0].data-id= test2.yaml +spring.cloud.nacos.config.shared-configs[0].data-id=test2.yaml spring.cloud.nacos.config.shared-configs[0].refresh=true ## the default value is 'DEFAULT_GROUP' , if not specified. -spring.cloud.nacos.config.shared-configs[0].group= GROUP_APP1 +spring.cloud.nacos.config.shared-configs[0].group=GROUP_APP1 ## not recommended. #spring.cloud.nacos.config.ext-config[0]=ext.properties ## recommended. -spring.cloud.nacos.config.extension-configs[0].data-id= extension1.properties +spring.cloud.nacos.config.extension-configs[0].data-id=extension1.properties spring.cloud.nacos.config.extension-configs[0].refresh=true -spring.cloud.nacos.config.extension-configs[1].data-id= test1.yml -spring.cloud.nacos.config.extension-configs[1].refresh= true +spring.cloud.nacos.config.extension-configs[1].data-id=test1.yml +spring.cloud.nacos.config.extension-configs[1].refresh=true -#spring.cloud.nacos.config.refresh-enabled=true +spring.cloud.nacos.config.refresh-enabled=true diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-consumer-example/src/main/resources/application.properties b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-consumer-example/src/main/resources/application.properties index cd4cc01d0..c1071da86 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-consumer-example/src/main/resources/application.properties +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-consumer-example/src/main/resources/application.properties @@ -2,6 +2,7 @@ spring.application.name=service-consumer server.port=18083 management.endpoints.web.exposure.include=* spring.cloud.nacos.discovery.server-addr=127.0.0.1:8848 +spring.cloud.nacos.discovery.fail-fast=true spring.cloud.nacos.username=nacos spring.cloud.nacos.password=nacos diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-provider-example/src/main/java/com/alibaba/cloud/examples/ProviderApplication.java b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-provider-example/src/main/java/com/alibaba/cloud/examples/ProviderApplication.java index aea78447d..038fe452f 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-provider-example/src/main/java/com/alibaba/cloud/examples/ProviderApplication.java +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-provider-example/src/main/java/com/alibaba/cloud/examples/ProviderApplication.java @@ -77,7 +77,12 @@ public class ProviderApplication { @GetMapping("/divide") public String divide(@RequestParam Integer a, @RequestParam Integer b) { - return String.valueOf(a / b); + if (b == 0) { + return String.valueOf(0); + } + else { + return String.valueOf(a / b); + } } @GetMapping("/zone") diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-provider-example/src/main/resources/application.properties b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-provider-example/src/main/resources/application.properties old mode 100644 new mode 100755 index 16c0acc9d..228372852 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-provider-example/src/main/resources/application.properties +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-provider-example/src/main/resources/application.properties @@ -1,6 +1,7 @@ server.port=18082 spring.application.name=service-provider spring.cloud.nacos.discovery.server-addr=127.0.0.1:8848 +spring.cloud.nacos.discovery.enabled=true #spring.cloud.nacos.discovery.instance-enabled=true spring.cloud.nacos.username=nacos @@ -8,3 +9,4 @@ spring.cloud.nacos.password=nacos management.endpoints.web.exposure.include=* management.endpoint.health.show-details=always + diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-spring-cloud-config-client-example/src/main/resources/bootstrap.yml b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-spring-cloud-config-client-example/src/main/resources/bootstrap.yml index 9abeb329b..a53218009 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-spring-cloud-config-client-example/src/main/resources/bootstrap.yml +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-spring-cloud-config-client-example/src/main/resources/bootstrap.yml @@ -6,7 +6,7 @@ spring: username: nacos password: nacos discovery: - server-addr: localhost:8848 + server-addr: 127.0.0.1:8848 config: discovery: enabled: true diff --git a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-spring-cloud-config-server-example/src/main/resources/application.yml b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-spring-cloud-config-server-example/src/main/resources/application.yml index cca1b4465..8fc3f8bf9 100644 --- a/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-spring-cloud-config-server-example/src/main/resources/application.yml +++ b/spring-cloud-alibaba-examples/nacos-example/nacos-discovery-example/nacos-discovery-spring-cloud-config-server-example/src/main/resources/application.yml @@ -9,7 +9,7 @@ spring: username: nacos password: nacos discovery: - server-addr: localhost:8848 + server-addr: 127.0.0.1:8848 config: server: git: diff --git a/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-consume-example/src/main/resources/application.properties b/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-consume-example/src/main/resources/application.properties index 2779db522..e2e77bd0b 100644 --- a/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-consume-example/src/main/resources/application.properties +++ b/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-consume-example/src/main/resources/application.properties @@ -3,20 +3,20 @@ spring.cloud.stream.rocketmq.binder.name-server=127.0.0.1:9876 spring.cloud.stream.bindings.input1.destination=test-topic spring.cloud.stream.bindings.input1.content-type=text/plain spring.cloud.stream.bindings.input1.group=test-group1 -spring.cloud.stream.rocketmq.bindings.input1.consumer.orderly=true +spring.cloud.stream.rocketmq.bindings.input1.consumer.push.orderly=true spring.cloud.stream.bindings.input2.destination=test-topic spring.cloud.stream.bindings.input2.content-type=text/plain spring.cloud.stream.bindings.input2.group=test-group2 -spring.cloud.stream.rocketmq.bindings.input2.consumer.orderly=false -spring.cloud.stream.rocketmq.bindings.input2.consumer.tags=tagStr +spring.cloud.stream.rocketmq.bindings.input2.consumer.push.orderly=false +spring.cloud.stream.rocketmq.bindings.input2.consumer.subscription=tagStr spring.cloud.stream.bindings.input2.consumer.concurrency=20 spring.cloud.stream.bindings.input2.consumer.maxAttempts=1 spring.cloud.stream.bindings.input3.destination=test-topic spring.cloud.stream.bindings.input3.content-type=application/json spring.cloud.stream.bindings.input3.group=test-group3 -spring.cloud.stream.rocketmq.bindings.input3.consumer.tags=tagObj +spring.cloud.stream.rocketmq.bindings.input3.consumer.subscription=tagObj spring.cloud.stream.bindings.input3.consumer.concurrency=20 spring.cloud.stream.bindings.input4.destination=TransactionTopic diff --git a/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/SenderService.java b/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/SenderService.java index cd9e50939..860c01467 100644 --- a/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/SenderService.java +++ b/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/SenderService.java @@ -20,8 +20,8 @@ import java.util.stream.Collectors; import java.util.stream.Stream; import com.alibaba.cloud.examples.RocketMQProduceApplication.MySource; +import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst; import org.apache.rocketmq.common.message.MessageConst; -import org.apache.rocketmq.spring.support.RocketMQHeaders; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.messaging.Message; @@ -62,7 +62,7 @@ public class SenderService { MessageBuilder builder = MessageBuilder.withPayload(msg) .setHeader(MessageHeaders.CONTENT_TYPE, MimeTypeUtils.APPLICATION_JSON); builder.setHeader("test", String.valueOf(num)); - builder.setHeader(RocketMQHeaders.TAGS, "binder"); + builder.setHeader(RocketMQConst.USER_TRANSACTIONAL_ARGS, "binder"); Message message = builder.build(); source.output2().send(message); } diff --git a/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/TransactionListenerImpl.java b/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/TransactionListenerImpl.java index e58beb221..446570ec9 100644 --- a/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/TransactionListenerImpl.java +++ b/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/TransactionListenerImpl.java @@ -16,43 +16,39 @@ package com.alibaba.cloud.examples; -import org.apache.rocketmq.spring.annotation.RocketMQTransactionListener; -import org.apache.rocketmq.spring.core.RocketMQLocalTransactionListener; -import org.apache.rocketmq.spring.core.RocketMQLocalTransactionState; +import org.apache.rocketmq.client.producer.LocalTransactionState; +import org.apache.rocketmq.client.producer.TransactionListener; +import org.apache.rocketmq.common.message.Message; +import org.apache.rocketmq.common.message.MessageExt; -import org.springframework.messaging.Message; +import org.springframework.stereotype.Component; /** * @author Jim */ -@RocketMQTransactionListener(txProducerGroup = "myTxProducerGroup", corePoolSize = 5, - maximumPoolSize = 10) -public class TransactionListenerImpl implements RocketMQLocalTransactionListener { +@Component("myTransactionListener") +public class TransactionListenerImpl implements TransactionListener { @Override - public RocketMQLocalTransactionState executeLocalTransaction(Message msg, - Object arg) { - Object num = msg.getHeaders().get("test"); + public LocalTransactionState executeLocalTransaction(Message msg, Object arg) { + Object num = msg.getProperty("test"); if ("1".equals(num)) { - System.out.println( - "executer: " + new String((byte[]) msg.getPayload()) + " unknown"); - return RocketMQLocalTransactionState.UNKNOWN; + System.out.println("executer: " + new String(msg.getBody()) + " unknown"); + return LocalTransactionState.UNKNOW; } else if ("2".equals(num)) { - System.out.println( - "executer: " + new String((byte[]) msg.getPayload()) + " rollback"); - return RocketMQLocalTransactionState.ROLLBACK; + System.out.println("executer: " + new String(msg.getBody()) + " rollback"); + return LocalTransactionState.ROLLBACK_MESSAGE; } - System.out.println( - "executer: " + new String((byte[]) msg.getPayload()) + " commit"); - return RocketMQLocalTransactionState.COMMIT; + System.out.println("executer: " + new String(msg.getBody()) + " commit"); + return LocalTransactionState.COMMIT_MESSAGE; } @Override - public RocketMQLocalTransactionState checkLocalTransaction(Message msg) { - System.out.println("check: " + new String((byte[]) msg.getPayload())); - return RocketMQLocalTransactionState.COMMIT; + public LocalTransactionState checkLocalTransaction(MessageExt msg) { + System.out.println("check: " + new String(msg.getBody())); + return LocalTransactionState.COMMIT_MESSAGE; } } diff --git a/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/resources/application.properties b/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/resources/application.properties index 772bf456e..428991930 100644 --- a/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/resources/application.properties +++ b/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/resources/application.properties @@ -9,8 +9,9 @@ spring.cloud.stream.rocketmq.bindings.output1.producer.sync=true spring.cloud.stream.bindings.output2.destination=TransactionTopic spring.cloud.stream.bindings.output2.content-type=application/json -spring.cloud.stream.rocketmq.bindings.output2.producer.transactional=true +spring.cloud.stream.rocketmq.bindings.output2.producer.producerType=Trans spring.cloud.stream.rocketmq.bindings.output2.producer.group=myTxProducerGroup +spring.cloud.stream.rocketmq.bindings.output2.producer.transactionListener=myTransactionListener spring.cloud.stream.bindings.output3.destination=pull-topic spring.cloud.stream.bindings.output3.content-type=text/plain diff --git a/spring-cloud-alibaba-examples/seata-example/account-service/src/main/resources/application.properties b/spring-cloud-alibaba-examples/seata-example/account-service/src/main/resources/application.properties index d5f73adfe..d67018432 100644 --- a/spring-cloud-alibaba-examples/seata-example/account-service/src/main/resources/application.properties +++ b/spring-cloud-alibaba-examples/seata-example/account-service/src/main/resources/application.properties @@ -1,6 +1,6 @@ spring.application.name=account-service server.port=18084 -spring.cloud.nacos.discovery.server-addr=localhost:8848 +spring.cloud.nacos.discovery.server-addr=127.0.0.1:8848 spring.datasource.name="accountDataSource" spring.datasource.type=com.alibaba.druid.pool.DruidDataSource diff --git a/spring-cloud-alibaba-examples/seata-example/business-service/src/main/resources/application.properties b/spring-cloud-alibaba-examples/seata-example/business-service/src/main/resources/application.properties index bc40bece3..a0f6550ab 100644 --- a/spring-cloud-alibaba-examples/seata-example/business-service/src/main/resources/application.properties +++ b/spring-cloud-alibaba-examples/seata-example/business-service/src/main/resources/application.properties @@ -1,6 +1,6 @@ server.port=18081 spring.application.name=business-service -spring.cloud.nacos.discovery.server-addr=localhost:8848 +spring.cloud.nacos.discovery.server-addr=127.0.0.1:8848 # The following configuration can be omitted. #feign.hystrix.enabled=true diff --git a/spring-cloud-alibaba-examples/seata-example/order-service/src/main/resources/application.properties b/spring-cloud-alibaba-examples/seata-example/order-service/src/main/resources/application.properties index fb4a4b7c0..4fef9e1b4 100644 --- a/spring-cloud-alibaba-examples/seata-example/order-service/src/main/resources/application.properties +++ b/spring-cloud-alibaba-examples/seata-example/order-service/src/main/resources/application.properties @@ -1,6 +1,6 @@ spring.application.name=order-service server.port=18083 -spring.cloud.nacos.discovery.server-addr=localhost:8848 +spring.cloud.nacos.discovery.server-addr=127.0.0.1:8848 spring.datasource.name="orderDataSource" spring.datasource.type=com.alibaba.druid.pool.DruidDataSource diff --git a/spring-cloud-alibaba-examples/seata-example/storage-service/src/main/resources/application.properties b/spring-cloud-alibaba-examples/seata-example/storage-service/src/main/resources/application.properties index a2bd463a0..c9671852f 100644 --- a/spring-cloud-alibaba-examples/seata-example/storage-service/src/main/resources/application.properties +++ b/spring-cloud-alibaba-examples/seata-example/storage-service/src/main/resources/application.properties @@ -1,6 +1,6 @@ spring.application.name=storage-service server.port=18082 -spring.cloud.nacos.discovery.server-addr=localhost:8848 +spring.cloud.nacos.discovery.server-addr=127.0.0.1:8848 spring.datasource.name="storageDataSource" spring.datasource.type=com.alibaba.druid.pool.DruidDataSource diff --git a/spring-cloud-alibaba-examples/sentinel-example/sentinel-circuitbreaker-example/src/main/resources/bootstrap.yml b/spring-cloud-alibaba-examples/sentinel-example/sentinel-circuitbreaker-example/src/main/resources/bootstrap.yml index 7e7d1440a..6d1b2332f 100644 --- a/spring-cloud-alibaba-examples/sentinel-example/sentinel-circuitbreaker-example/src/main/resources/bootstrap.yml +++ b/spring-cloud-alibaba-examples/sentinel-example/sentinel-circuitbreaker-example/src/main/resources/bootstrap.yml @@ -6,6 +6,6 @@ spring: cloud: nacos: config: - server-addr: localhost:8848 + server-addr: 127.0.0.1:8848 name: sentinel-circuitbreaker-rules.yml file-extension: yml diff --git a/spring-cloud-alibaba-examples/sentinel-example/sentinel-core-example/readme-zh.md b/spring-cloud-alibaba-examples/sentinel-example/sentinel-core-example/readme-zh.md index c5a8111f9..ba4b92177 100644 --- a/spring-cloud-alibaba-examples/sentinel-example/sentinel-core-example/readme-zh.md +++ b/spring-cloud-alibaba-examples/sentinel-example/sentinel-core-example/readme-zh.md @@ -65,7 +65,7 @@ 1. 首先需要获取 Sentinel 控制台,支持直接下载和源码构建两种方式。 - 1. 直接下载:[下载 Sentinel 控制台](http://edas-public.oss-cn-hangzhou.aliyuncs.com/install_package/demo/sentinel-dashboard.jar) + 1. 直接下载:[下载 Sentinel 控制台](https://edas-public.oss-cn-hangzhou.aliyuncs.com/install_package/demo/sentinel-dashboard.jar) 2. 源码构建:进入 Sentinel [Github 项目页面](https://github.com/alibaba/Sentinel),将代码 git clone 到本地自行编译打包,[参考此文档](https://github.com/alibaba/Sentinel/tree/master/sentinel-dashboard)。 2. 启动控制台,执行 Java 命令 `java -jar sentinel-dashboard.jar`完成 Sentinel 控制台的启动。 @@ -206,7 +206,7 @@ Sentinel starter 整合了目前存在的几类 ReadableDataSource。只需要 spring.cloud.sentinel.datasource.ds1.file.file=classpath: degraderule.json spring.cloud.sentinel.datasource.ds1.file.data-type=json -spring.cloud.sentinel.datasource.ds2.nacos.server-addr=localhost:8848 +spring.cloud.sentinel.datasource.ds2.nacos.server-addr=127.0.0.1:8848 spring.cloud.sentinel.datasource.ds2.nacos.dataId=sentinel spring.cloud.sentinel.datasource.ds2.nacos.groupId=DEFAULT_GROUP spring.cloud.sentinel.datasource.ds2.nacos.data-type=json diff --git a/spring-cloud-alibaba-examples/sentinel-example/sentinel-core-example/readme.md b/spring-cloud-alibaba-examples/sentinel-example/sentinel-core-example/readme.md index 1856a4a05..c1c946338 100644 --- a/spring-cloud-alibaba-examples/sentinel-example/sentinel-core-example/readme.md +++ b/spring-cloud-alibaba-examples/sentinel-example/sentinel-core-example/readme.md @@ -181,7 +181,7 @@ If you want to define `FileRefreshableDataSource` and `NacosDataSource`, see the spring.cloud.sentinel.datasource.ds1.file.file=classpath: degraderule.json spring.cloud.sentinel.datasource.ds1.file.data-type=json -spring.cloud.sentinel.datasource.ds2.nacos.server-addr=localhost:8848 +spring.cloud.sentinel.datasource.ds2.nacos.server-addr=127.0.0.1:8848 spring.cloud.sentinel.datasource.ds2.nacos.dataId=sentinel spring.cloud.sentinel.datasource.ds2.nacos.groupId=DEFAULT_GROUP spring.cloud.sentinel.datasource.ds2.nacos.data-type=json diff --git a/spring-cloud-alibaba-examples/sentinel-example/sentinel-dubbo-example/readme-zh.md b/spring-cloud-alibaba-examples/sentinel-example/sentinel-dubbo-example/readme-zh.md index ca7459db7..b721036bd 100644 --- a/spring-cloud-alibaba-examples/sentinel-example/sentinel-dubbo-example/readme-zh.md +++ b/spring-cloud-alibaba-examples/sentinel-example/sentinel-dubbo-example/readme-zh.md @@ -73,7 +73,7 @@ Provider端在application.properties文件中定义dubbo相关的配置,比如 定义具体的服务: - @Service( + @DubboService( version = "${foo.service.version}", application = "${dubbo.application.id}", protocol = "${dubbo.protocol.id}", @@ -111,7 +111,7 @@ Consumer端在服务调用之前,先定义限流规则。 根据Provider端中发布的定义,使用Dubbo的@Reference注解注入服务对应的Bean: - @Reference(version = "${foo.service.version}", application = "${dubbo.application.id}", + @DubboReference(version = "${foo.service.version}", application = "${dubbo.application.id}", path = "dubbo://localhost:12345", timeout = 30000) private FooService fooService; diff --git a/spring-cloud-alibaba-examples/sentinel-example/sentinel-feign-example/readme-zh.md b/spring-cloud-alibaba-examples/sentinel-example/sentinel-feign-example/readme-zh.md index 542d9ef78..06bc694e7 100644 --- a/spring-cloud-alibaba-examples/sentinel-example/sentinel-feign-example/readme-zh.md +++ b/spring-cloud-alibaba-examples/sentinel-example/sentinel-feign-example/readme-zh.md @@ -114,6 +114,8 @@ public class EchoController { - 启动nacos 注册中心 +- 启动sentinel + - 启动服务提供方: 1. IDE直接启动:找到主类 `ProviderApplication`,执行 main 方法启动应用。 @@ -123,3 +125,5 @@ public class EchoController { 1. IDE直接启动:找到主类 `ConsumerApplication`,执行 main 方法启动应用。 2. 打包编译后启动:首先执行 `mvn clean package` 将工程编译打包,然后执行 `java -jar sentinel-feign-consumer-example.jar`启动应用。 + +- 启动之后,Sentinel Dashboard可能看不见service-consumer服务的详细信息,多请求几次接口即可。 diff --git a/spring-cloud-alibaba-examples/sentinel-example/sentinel-feign-example/sentinel-feign-consumer-example/src/main/resources/application.yml b/spring-cloud-alibaba-examples/sentinel-example/sentinel-feign-example/sentinel-feign-consumer-example/src/main/resources/application.yml index 219c38d02..c981dca1e 100644 --- a/spring-cloud-alibaba-examples/sentinel-example/sentinel-feign-example/sentinel-feign-consumer-example/src/main/resources/application.yml +++ b/spring-cloud-alibaba-examples/sentinel-example/sentinel-feign-example/sentinel-feign-consumer-example/src/main/resources/application.yml @@ -8,6 +8,9 @@ spring: nacos: discovery: server-addr: 127.0.0.1:8848 + sentinel: + transport: + dashboard: 127.0.0.1:8081 feign: sentinel: diff --git a/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-client-sample/pom.xml b/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-client-sample/pom.xml index c37db212c..1a675b438 100644 --- a/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-client-sample/pom.xml +++ b/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-client-sample/pom.xml @@ -10,7 +10,6 @@ 4.0.0 - com.alibaba.cloud spring-cloud-dubbo-client-sample Spring Cloud Dubbo Client Sample diff --git a/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-client-sample/src/main/resources/bootstrap.yaml b/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-client-sample/src/main/resources/bootstrap.yaml index 6801b6c4b..19d22ad11 100644 --- a/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-client-sample/src/main/resources/bootstrap.yaml +++ b/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-client-sample/src/main/resources/bootstrap.yaml @@ -12,7 +12,11 @@ spring: allow-bean-definition-overriding: true cloud: nacos: - username: nacos - password: nacos discovery: - server-addr: 127.0.0.1:8848 \ No newline at end of file + username: nacos + password: nacos + server-addr: 127.0.0.1:8848 + namespace: public + +server: + port: 8080 \ No newline at end of file diff --git a/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-server-sample/src/main/resources/bootstrap.yaml b/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-server-sample/src/main/resources/bootstrap.yaml index 650b9861c..db7e38c0d 100644 --- a/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-server-sample/src/main/resources/bootstrap.yaml +++ b/spring-cloud-alibaba-examples/spring-cloud-alibaba-dubbo-examples/spring-cloud-dubbo-server-sample/src/main/resources/bootstrap.yaml @@ -1,4 +1,6 @@ dubbo: + cloud: + subscribed-services: ${spring.application.name} scan: base-packages: com.alibaba.cloud.dubbo.bootstrap protocol: diff --git a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/config/DataSourcePropertiesConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/config/DataSourcePropertiesConfiguration.java index a0a5526f3..330a0a04b 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/config/DataSourcePropertiesConfiguration.java +++ b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/config/DataSourcePropertiesConfiguration.java @@ -132,7 +132,6 @@ public class DataSourcePropertiesConfiguration { if (!ObjectUtils.isEmpty(field.get(this))) { return field.getName(); } - return null; } catch (IllegalAccessException e) { // won't happen diff --git a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/config/NacosDataSourceProperties.java b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/config/NacosDataSourceProperties.java index 2cc75df17..a41ddfbfe 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/config/NacosDataSourceProperties.java +++ b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/config/NacosDataSourceProperties.java @@ -31,6 +31,8 @@ public class NacosDataSourceProperties extends AbstractDataSourceProperties { private String serverAddr; + private String contextPath; + private String username; private String password; @@ -58,7 +60,7 @@ public class NacosDataSourceProperties extends AbstractDataSourceProperties { if (StringUtils.isEmpty(serverAddr)) { serverAddr = this.getEnv().getProperty( "spring.cloud.sentinel.datasource.nacos.server-addr", - "localhost:8848"); + "127.0.0.1:8848"); } } @@ -70,6 +72,14 @@ public class NacosDataSourceProperties extends AbstractDataSourceProperties { this.serverAddr = serverAddr; } + public String getContextPath() { + return contextPath; + } + + public void setContextPath(String contextPath) { + this.contextPath = contextPath; + } + public String getUsername() { return username; } diff --git a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/converter/SentinelConverter.java b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/converter/SentinelConverter.java index 4f4cfaf57..b9e2fc644 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/converter/SentinelConverter.java +++ b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/converter/SentinelConverter.java @@ -85,9 +85,8 @@ public abstract class SentinelConverter }); for (Object obj : sourceArray) { - String item = null; try { - item = objectMapper.writeValueAsString(obj); + String item = objectMapper.writeValueAsString(obj); Optional.ofNullable(convertRule(item)) .ifPresent(convertRule -> ruleCollection.add(convertRule)); } diff --git a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/factorybean/NacosDataSourceFactoryBean.java b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/factorybean/NacosDataSourceFactoryBean.java index 34ad98ea7..bbb0b6aad 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/factorybean/NacosDataSourceFactoryBean.java +++ b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/main/java/com/alibaba/cloud/sentinel/datasource/factorybean/NacosDataSourceFactoryBean.java @@ -35,6 +35,8 @@ public class NacosDataSourceFactoryBean implements FactoryBean private String serverAddr; + private String contextPath; + private String username; private String password; @@ -60,9 +62,17 @@ public class NacosDataSourceFactoryBean implements FactoryBean properties.setProperty(PropertyKeyConst.SERVER_ADDR, this.serverAddr); } else { + properties.setProperty(PropertyKeyConst.ENDPOINT, this.endpoint); + } + + if (!StringUtils.isEmpty(this.contextPath)) { + properties.setProperty(PropertyKeyConst.CONTEXT_PATH, this.contextPath); + } + if (!StringUtils.isEmpty(this.accessKey)) { properties.setProperty(PropertyKeyConst.ACCESS_KEY, this.accessKey); + } + if (!StringUtils.isEmpty(this.secretKey)) { properties.setProperty(PropertyKeyConst.SECRET_KEY, this.secretKey); - properties.setProperty(PropertyKeyConst.ENDPOINT, this.endpoint); } if (!StringUtils.isEmpty(this.namespace)) { properties.setProperty(PropertyKeyConst.NAMESPACE, this.namespace); @@ -89,6 +99,14 @@ public class NacosDataSourceFactoryBean implements FactoryBean this.serverAddr = serverAddr; } + public String getContextPath() { + return contextPath; + } + + public void setContextPath(String contextPath) { + this.contextPath = contextPath; + } + public String getUsername() { return username; } diff --git a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/test/java/com/alibaba/cloud/sentinel/datasource/NacosDataSourceFactoryBeanTests.java b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/test/java/com/alibaba/cloud/sentinel/datasource/NacosDataSourceFactoryBeanTests.java index ca364f30e..8de459f97 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/test/java/com/alibaba/cloud/sentinel/datasource/NacosDataSourceFactoryBeanTests.java +++ b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/test/java/com/alibaba/cloud/sentinel/datasource/NacosDataSourceFactoryBeanTests.java @@ -37,7 +37,9 @@ public class NacosDataSourceFactoryBeanTests { private String groupId = "DEFAULT_GROUP"; - private String serverAddr = "localhost:8848"; + private String serverAddr = "127.0.0.1:8848"; + + private String contextPath = "/my-nacos"; private String accessKey = "ak"; @@ -56,6 +58,7 @@ public class NacosDataSourceFactoryBeanTests { factoryBean.setDataId(dataId); factoryBean.setGroupId(groupId); factoryBean.setServerAddr(serverAddr); + factoryBean.setContextPath(contextPath); factoryBean.setConverter(converter); NacosDataSource nacosDataSource = mock(NacosDataSource.class); @@ -69,6 +72,7 @@ public class NacosDataSourceFactoryBeanTests { assertThat(factoryBean.getDataId()).isEqualTo(dataId); assertThat(factoryBean.getGroupId()).isEqualTo(groupId); assertThat(factoryBean.getServerAddr()).isEqualTo(serverAddr); + assertThat(factoryBean.getContextPath()).isEqualTo(contextPath); } @Test diff --git a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/test/java/com/alibaba/cloud/sentinel/datasource/NacosDataSourcePropertiesTests.java b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/test/java/com/alibaba/cloud/sentinel/datasource/NacosDataSourcePropertiesTests.java index f897d6ed8..cde739d06 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/test/java/com/alibaba/cloud/sentinel/datasource/NacosDataSourcePropertiesTests.java +++ b/spring-cloud-alibaba-starters/spring-cloud-alibaba-sentinel-datasource/src/test/java/com/alibaba/cloud/sentinel/datasource/NacosDataSourcePropertiesTests.java @@ -31,11 +31,13 @@ public class NacosDataSourcePropertiesTests { public void testNacosWithAddr() { NacosDataSourceProperties nacosDataSourceProperties = new NacosDataSourceProperties(); nacosDataSourceProperties.setServerAddr("127.0.0.1:8848"); + nacosDataSourceProperties.setContextPath("/my-nacos"); nacosDataSourceProperties.setRuleType(RuleType.FLOW); nacosDataSourceProperties.setDataId("sentinel"); nacosDataSourceProperties.setGroupId("custom-group"); nacosDataSourceProperties.setDataType("xml"); + assertThat(nacosDataSourceProperties.getContextPath()).isEqualTo("/my-nacos"); assertThat(nacosDataSourceProperties.getGroupId()).isEqualTo("custom-group"); assertThat(nacosDataSourceProperties.getDataId()).isEqualTo("sentinel"); assertThat(nacosDataSourceProperties.getDataType()).isEqualTo("xml"); diff --git a/spring-cloud-alibaba-starters/spring-cloud-circuitbreaker-sentinel/src/test/java/com/alibaba/cloud/circuitbreaker/sentinel/ReactiveSentinelCircuitBreakerIntegrationTest.java b/spring-cloud-alibaba-starters/spring-cloud-circuitbreaker-sentinel/src/test/java/com/alibaba/cloud/circuitbreaker/sentinel/ReactiveSentinelCircuitBreakerIntegrationTest.java index 247b7efbf..0856b4772 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-circuitbreaker-sentinel/src/test/java/com/alibaba/cloud/circuitbreaker/sentinel/ReactiveSentinelCircuitBreakerIntegrationTest.java +++ b/spring-cloud-alibaba-starters/spring-cloud-circuitbreaker-sentinel/src/test/java/com/alibaba/cloud/circuitbreaker/sentinel/ReactiveSentinelCircuitBreakerIntegrationTest.java @@ -44,15 +44,15 @@ import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; import org.springframework.web.reactive.function.client.WebClient; +import static com.alibaba.cloud.circuitbreaker.sentinel.ReactiveSentinelCircuitBreakerIntegrationTest.Application; import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; /** * @author Ryan Baxter */ @RunWith(SpringRunner.class) -@SpringBootTest(webEnvironment = RANDOM_PORT, - classes = ReactiveSentinelCircuitBreakerIntegrationTest.Application.class, - properties = { "spring.cloud.discovery.client.health-indicator.enabled=false" }) +@SpringBootTest(webEnvironment = RANDOM_PORT, classes = Application.class, properties = { + "spring.cloud.discovery.client.health-indicator.enabled=false" }) @DirtiesContext public class ReactiveSentinelCircuitBreakerIntegrationTest { @@ -60,7 +60,7 @@ public class ReactiveSentinelCircuitBreakerIntegrationTest { private int port = 0; @Autowired - private ReactiveSentinelCircuitBreakerIntegrationTest.Application.DemoControllerService service; + private Application.DemoControllerService service; @Before public void setup() { diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/NacosConfigProperties.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/NacosConfigProperties.java index 22f71e0f1..8829e6155 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/NacosConfigProperties.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/NacosConfigProperties.java @@ -53,6 +53,7 @@ import static com.alibaba.nacos.api.PropertyKeyConst.ENDPOINT_PORT; import static com.alibaba.nacos.api.PropertyKeyConst.MAX_RETRY; import static com.alibaba.nacos.api.PropertyKeyConst.NAMESPACE; import static com.alibaba.nacos.api.PropertyKeyConst.PASSWORD; +import static com.alibaba.nacos.api.PropertyKeyConst.RAM_ROLE_NAME; import static com.alibaba.nacos.api.PropertyKeyConst.SECRET_KEY; import static com.alibaba.nacos.api.PropertyKeyConst.SERVER_ADDR; import static com.alibaba.nacos.api.PropertyKeyConst.USERNAME; @@ -106,7 +107,7 @@ public class NacosConfigProperties { .resolvePlaceholders("${spring.cloud.nacos.config.server-addr:}"); if (StringUtils.isEmpty(serverAddr)) { serverAddr = environment.resolvePlaceholders( - "${spring.cloud.nacos.server-addr:localhost:8848}"); + "${spring.cloud.nacos.server-addr:127.0.0.1:8848}"); } this.setServerAddr(serverAddr); } @@ -205,6 +206,11 @@ public class NacosConfigProperties { */ private String secretKey; + /** + * access key for namespace. + */ + private String ramRoleName; + /** * context path for nacos config server. */ @@ -359,6 +365,14 @@ public class NacosConfigProperties { this.secretKey = secretKey; } + public String getRamRoleName() { + return ramRoleName; + } + + public void setRamRoleName(String ramRoleName) { + this.ramRoleName = ramRoleName; + } + public String getEncode() { return encode; } @@ -428,19 +442,19 @@ public class NacosConfigProperties { * @return string */ @Deprecated - @DeprecatedConfigurationProperty( - reason = "replaced to NacosConfigProperties#sharedConfigs and not use it at the same time.", - replacement = PREFIX + ".shared-configs[x]") + @DeprecatedConfigurationProperty(reason = "replaced to NacosConfigProperties#sharedConfigs and not use it at the same time.", replacement = PREFIX + + ".shared-configs[x]") public String getSharedDataids() { - return null == getSharedConfigs() ? null : getSharedConfigs().stream() - .map(Config::getDataId).collect(Collectors.joining(COMMAS)); + return null == getSharedConfigs() ? null + : getSharedConfigs().stream().map(Config::getDataId) + .collect(Collectors.joining(COMMAS)); } /** * recommend to use {@link NacosConfigProperties#sharedConfigs} and not use it at the * same time . * @param sharedDataids the dataids for configurable multiple shared configurations , - * multiple separated by commas . + * multiple separated by commas . */ @Deprecated public void setSharedDataids(String sharedDataids) { @@ -458,9 +472,8 @@ public class NacosConfigProperties { * @return string */ @Deprecated - @DeprecatedConfigurationProperty( - reason = "replaced to NacosConfigProperties#sharedConfigs and not use it at the same time.", - replacement = PREFIX + ".shared-configs[x].refresh") + @DeprecatedConfigurationProperty(reason = "replaced to NacosConfigProperties#sharedConfigs and not use it at the same time.", replacement = PREFIX + + ".shared-configs[x].refresh") public String getRefreshableDataids() { return null == getSharedConfigs() ? null : getSharedConfigs().stream().filter(Config::isRefresh) @@ -506,9 +519,8 @@ public class NacosConfigProperties { * @return extensionConfigs */ @Deprecated - @DeprecatedConfigurationProperty( - reason = "replaced to NacosConfigProperties#extensionConfigs and not use it at the same time .", - replacement = PREFIX + ".extension-configs[x]") + @DeprecatedConfigurationProperty(reason = "replaced to NacosConfigProperties#extensionConfigs and not use it at the same time .", replacement = PREFIX + + ".extension-configs[x]") public List getExtConfig() { return this.getExtensionConfigs(); } @@ -551,6 +563,7 @@ public class NacosConfigProperties { properties.put(NAMESPACE, Objects.toString(this.namespace, "")); properties.put(ACCESS_KEY, Objects.toString(this.accessKey, "")); properties.put(SECRET_KEY, Objects.toString(this.secretKey, "")); + properties.put(RAM_ROLE_NAME, Objects.toString(this.ramRoleName, "")); properties.put(CLUSTER_NAME, Objects.toString(this.clusterName, "")); properties.put(MAX_RETRY, Objects.toString(this.maxRetry, "")); properties.put(CONFIG_LONG_POLL_TIMEOUT, @@ -603,10 +616,10 @@ public class NacosConfigProperties { + ", enableRemoteSyncConfig=" + enableRemoteSyncConfig + ", endpoint='" + endpoint + '\'' + ", namespace='" + namespace + '\'' + ", accessKey='" + accessKey + '\'' + ", secretKey='" + secretKey + '\'' - + ", contextPath='" + contextPath + '\'' + ", clusterName='" + clusterName - + '\'' + ", name='" + name + '\'' + '\'' + ", shares=" + sharedConfigs - + ", extensions=" + extensionConfigs + ", refreshEnabled=" - + refreshEnabled + '}'; + + ", ramRoleName='" + ramRoleName + '\'' + ", contextPath='" + contextPath + + '\'' + ", clusterName='" + clusterName + '\'' + ", name='" + name + '\'' + + '\'' + ", shares=" + sharedConfigs + ", extensions=" + extensionConfigs + + ", refreshEnabled=" + refreshEnabled + '}'; } public static class Config { diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/client/NacosPropertySource.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/client/NacosPropertySource.java index 91e7fd043..4b4d07356 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/client/NacosPropertySource.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/client/NacosPropertySource.java @@ -16,8 +16,10 @@ package com.alibaba.cloud.nacos.client; +import java.util.ArrayList; import java.util.Collections; import java.util.Date; +import java.util.LinkedHashMap; import java.util.List; import java.util.Map; @@ -80,12 +82,36 @@ public class NacosPropertySource extends MapPropertySource { return (Map) propertySource.getSource(); } } - // If it is multiple, it will be returned as it is, and the internal elements - // cannot be directly retrieved, so the user needs to implement the retrieval - // logic by himself - return Collections.singletonMap( - String.join(NacosConfigProperties.COMMAS, dataId, group), - propertySources); + + Map sourceMap = new LinkedHashMap<>(); + List> otherTypePropertySources = new ArrayList<>(); + for (PropertySource propertySource : propertySources) { + if (propertySource == null) { + continue; + } + if (propertySource instanceof MapPropertySource) { + // If the Nacos configuration file uses "---" to separate property name, + // propertySources will be multiple documents, and every document is a + // map. + // see org.springframework.boot.env.YamlPropertySourceLoader#load + MapPropertySource mapPropertySource = (MapPropertySource) propertySource; + Map source = mapPropertySource.getSource(); + sourceMap.putAll(source); + } + else { + otherTypePropertySources.add(propertySource); + } + } + + // Other property sources which is not instanceof MapPropertySource will be put as + // it is, + // and the internal elements cannot be directly retrieved, + // so the user needs to implement the retrieval logic by himself + if (!otherTypePropertySources.isEmpty()) { + sourceMap.put(String.join(NacosConfigProperties.COMMAS, dataId, group), + otherTypePropertySources); + } + return sourceMap; } public String getGroup() { diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/endpoint/NacosConfigHealthIndicator.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/endpoint/NacosConfigHealthIndicator.java index affd0139f..37e3c2463 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/endpoint/NacosConfigHealthIndicator.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/endpoint/NacosConfigHealthIndicator.java @@ -32,6 +32,16 @@ public class NacosConfigHealthIndicator extends AbstractHealthIndicator { private final ConfigService configService; + /** + * status up . + */ + private final String STATUS_UP = "UP"; + + /** + * status down . + */ + private final String STATUS_DOWN = "DOWN"; + public NacosConfigHealthIndicator(ConfigService configService) { this.configService = configService; } @@ -43,10 +53,10 @@ public class NacosConfigHealthIndicator extends AbstractHealthIndicator { // Set the status to Builder builder.status(status); switch (status) { - case "UP": + case STATUS_UP: builder.up(); break; - case "DOWN": + case STATUS_DOWN: builder.down(); break; default: diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/logging/NacosLoggingListener.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/logging/NacosLoggingListener.java new file mode 100644 index 000000000..41d5842ec --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/java/com/alibaba/cloud/nacos/logging/NacosLoggingListener.java @@ -0,0 +1,54 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.nacos.logging; + +import com.alibaba.nacos.client.logging.NacosLogging; + +import org.springframework.boot.context.event.ApplicationEnvironmentPreparedEvent; +import org.springframework.context.ApplicationEvent; +import org.springframework.context.event.GenericApplicationListener; +import org.springframework.core.Ordered; +import org.springframework.core.ResolvableType; + +/** + * Reload nacos log configuration file, after + * {@link org.springframework.boot.context.logging.LoggingApplicationListener}. + * + * @author mai.jh + */ +public class NacosLoggingListener implements GenericApplicationListener { + + @Override + public boolean supportsEventType(ResolvableType resolvableType) { + Class type = resolvableType.getRawClass(); + if (type != null) { + return ApplicationEnvironmentPreparedEvent.class.isAssignableFrom(type); + } + return false; + } + + @Override + public void onApplicationEvent(ApplicationEvent applicationEvent) { + NacosLogging.getInstance().loadConfiguration(); + } + + @Override + public int getOrder() { + return Ordered.HIGHEST_PRECEDENCE + 21; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/resources/META-INF/additional-spring-configuration-metadata.json b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/resources/META-INF/additional-spring-configuration-metadata.json index f6f3fe414..603c6cf43 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/resources/META-INF/additional-spring-configuration-metadata.json +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/resources/META-INF/additional-spring-configuration-metadata.json @@ -3,7 +3,7 @@ { "name": "spring.cloud.nacos.server-addr", "type": "java.lang.String", - "defaultValue": "localhost:8848", + "defaultValue": "127.0.0.1:8848", "description": "nacos server address." }, { diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/resources/META-INF/spring.factories b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/resources/META-INF/spring.factories index dc58fcddd..ae5b0662f 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/resources/META-INF/spring.factories +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/main/resources/META-INF/spring.factories @@ -8,6 +8,8 @@ com.alibaba.cloud.nacos.diagnostics.analyzer.NacosConnectionFailureAnalyzer org.springframework.boot.env.PropertySourceLoader=\ com.alibaba.cloud.nacos.parser.NacosJsonPropertySourceLoader,\ com.alibaba.cloud.nacos.parser.NacosXmlPropertySourceLoader +org.springframework.context.ApplicationListener=\ +com.alibaba.cloud.nacos.logging.NacosLoggingListener # ConfigData Location Resolvers org.springframework.boot.context.config.ConfigDataLocationResolver=\ diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosConfigAutoConfigurationTest.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosConfigAutoConfigurationTest.java index 59309e3d5..3a25e8f29 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosConfigAutoConfigurationTest.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosConfigAutoConfigurationTest.java @@ -38,7 +38,7 @@ public class NacosConfigAutoConfigurationTest { assertThat(BeanFactoryUtils.beanNamesForTypeIncludingAncestors(context, NacosConfigProperties.class).length).isEqualTo(1); assertThat(context.getBean(NacosConfigProperties.class).getServerAddr()) - .isEqualTo("localhost:8848"); + .isEqualTo("127.0.0.1:8848"); context.close(); } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosConfigPropertiesServerAddressBothLevelTests.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosConfigPropertiesServerAddressBothLevelTests.java index d0e134892..32224ea6a 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosConfigPropertiesServerAddressBothLevelTests.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosConfigPropertiesServerAddressBothLevelTests.java @@ -28,7 +28,6 @@ import org.springframework.context.annotation.Configuration; import org.springframework.test.context.junit4.SpringRunner; import static org.assertj.core.api.Assertions.assertThat; -import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; /** * @author lyuzb @@ -36,9 +35,9 @@ import static org.springframework.boot.test.context.SpringBootTest.WebEnvironmen @RunWith(SpringRunner.class) @SpringBootTest( classes = NacosConfigPropertiesServerAddressBothLevelTests.TestConfig.class, - properties = { "spring.cloud.nacos.config.server-addr=321,321,321,321:8848", - "spring.cloud.nacos.server-addr=123.123.123.123:8848" }, - webEnvironment = RANDOM_PORT) + properties = { "spring.cloud.nacos.config.server-addr=321.321.321.321:8848", + "spring.cloud.nacos.server-addr=123.123.123.123:8848" } +) public class NacosConfigPropertiesServerAddressBothLevelTests { @Autowired @@ -46,7 +45,7 @@ public class NacosConfigPropertiesServerAddressBothLevelTests { @Test public void testGetServerAddr() { - assertThat(properties.getServerAddr()).isEqualTo("321,321,321,321:8848"); + assertThat(properties.getServerAddr()).isEqualTo("321.321.321.321:8848"); } @Configuration diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosFileExtensionTest.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosFileExtensionTest.java index 229610514..6e2e36504 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosFileExtensionTest.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-config/src/test/java/com/alibaba/cloud/nacos/NacosFileExtensionTest.java @@ -67,7 +67,7 @@ public class NacosFileExtensionTest { throws Throwable { if ("test-name.yaml".equals(args[0]) && "DEFAULT_GROUP".equals(args[1])) { - return "user:\n name: hello\n age: 12"; + return "user:\n name: hello\n age: 12\n---\nuser:\n gender: male"; } return ""; } @@ -88,6 +88,7 @@ public class NacosFileExtensionTest { Assert.assertEquals(environment.getProperty("user.name"), "hello"); Assert.assertEquals(environment.getProperty("user.age"), "12"); + Assert.assertEquals(environment.getProperty("user.gender"), "male"); } @Configuration diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/NacosDiscoveryProperties.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/NacosDiscoveryProperties.java index 80f26326f..5b5ae9421 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/NacosDiscoveryProperties.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/NacosDiscoveryProperties.java @@ -214,6 +214,12 @@ public class NacosDiscoveryProperties { */ private boolean failureToleranceEnabled; + /** + * Throw exceptions during service registration if true, otherwise, log error + * (defaults to true). + */ + private boolean failFast = true; + @Autowired private InetUtils inetUtils; @@ -501,6 +507,14 @@ public class NacosDiscoveryProperties { this.failureToleranceEnabled = failureToleranceEnabled; } + public boolean isFailFast() { + return failFast; + } + + public void setFailFast(boolean failFast) { + this.failFast = failFast; + } + @Override public boolean equals(Object o) { if (this == o) { @@ -531,6 +545,7 @@ public class NacosDiscoveryProperties { && Objects.equals(secretKey, that.secretKey) && Objects.equals(heartBeatInterval, that.heartBeatInterval) && Objects.equals(heartBeatTimeout, that.heartBeatTimeout) + && Objects.equals(failFast, that.failFast) && Objects.equals(ipDeleteTimeout, that.ipDeleteTimeout); } @@ -540,7 +555,8 @@ public class NacosDiscoveryProperties { watchDelay, logName, service, weight, clusterName, group, namingLoadCacheAtStart, metadata, registerEnabled, ip, networkInterface, port, secure, accessKey, secretKey, heartBeatInterval, heartBeatTimeout, - ipDeleteTimeout, instanceEnabled, ephemeral, failureToleranceEnabled); + ipDeleteTimeout, instanceEnabled, ephemeral, failureToleranceEnabled, + failFast); } @Override @@ -559,7 +575,8 @@ public class NacosDiscoveryProperties { + heartBeatInterval + ", heartBeatTimeout=" + heartBeatTimeout + ", ipDeleteTimeout=" + ipDeleteTimeout + ", instanceEnabled=" + instanceEnabled + ", ephemeral=" + ephemeral - + ", failureToleranceEnabled=" + failureToleranceEnabled + '}'; + + ", failureToleranceEnabled=" + failureToleranceEnabled + '}' + + ", ipDeleteTimeout=" + ipDeleteTimeout + ", failFast=" + failFast + '}'; } public void overrideFromEnv(Environment env) { @@ -569,7 +586,7 @@ public class NacosDiscoveryProperties { .resolvePlaceholders("${spring.cloud.nacos.discovery.server-addr:}"); if (StringUtils.isEmpty(serverAddr)) { serverAddr = env.resolvePlaceholders( - "${spring.cloud.nacos.server-addr:localhost:8848}"); + "${spring.cloud.nacos.server-addr:127.0.0.1:8848}"); } this.setServerAddr(serverAddr); } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/NacosServiceManager.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/NacosServiceManager.java index c672bbac0..31f68cd99 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/NacosServiceManager.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/NacosServiceManager.java @@ -112,9 +112,14 @@ public class NacosServiceManager { } public void nacosServiceShutDown() throws NacosException { - this.namingService.shutDown(); - namingService = null; - namingMaintainService = null; + if (Objects.nonNull(this.namingService)) { + this.namingService.shutDown(); + this.namingService = null; + } + if (Objects.nonNull(this.namingMaintainService)) { + this.namingMaintainService.shutDown(); + this.namingMaintainService = null; + } } @EventListener diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/NacosDiscoveryClientConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/NacosDiscoveryClientConfiguration.java index d2f2f8ba6..af722dda2 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/NacosDiscoveryClientConfiguration.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/NacosDiscoveryClientConfiguration.java @@ -20,7 +20,6 @@ import com.alibaba.cloud.nacos.ConditionalOnNacosDiscoveryEnabled; import com.alibaba.cloud.nacos.NacosDiscoveryProperties; import com.alibaba.cloud.nacos.NacosServiceManager; -import org.springframework.beans.factory.ObjectProvider; import org.springframework.boot.autoconfigure.AutoConfigureAfter; import org.springframework.boot.autoconfigure.AutoConfigureBefore; import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; @@ -32,7 +31,6 @@ import org.springframework.cloud.client.discovery.DiscoveryClient; import org.springframework.cloud.client.discovery.simple.SimpleDiscoveryClientAutoConfiguration; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; -import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler; /** * @author xiaojing @@ -58,10 +56,8 @@ public class NacosDiscoveryClientConfiguration { @ConditionalOnProperty(value = "spring.cloud.nacos.discovery.watch.enabled", matchIfMissing = true) public NacosWatch nacosWatch(NacosServiceManager nacosServiceManager, - NacosDiscoveryProperties nacosDiscoveryProperties, - ObjectProvider taskExecutorObjectProvider) { - return new NacosWatch(nacosServiceManager, nacosDiscoveryProperties, - taskExecutorObjectProvider); + NacosDiscoveryProperties nacosDiscoveryProperties) { + return new NacosWatch(nacosServiceManager, nacosDiscoveryProperties); } } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/NacosWatch.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/NacosWatch.java index bcb590351..181c6b455 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/NacosWatch.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/NacosWatch.java @@ -35,6 +35,7 @@ import com.alibaba.nacos.api.naming.pojo.Instance; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.DisposableBean; import org.springframework.beans.factory.ObjectProvider; import org.springframework.cloud.client.discovery.event.HeartbeatEvent; import org.springframework.context.ApplicationEventPublisher; @@ -45,8 +46,9 @@ import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler; /** * @author xiaojing * @author yuhuangbin + * @author pengfei.lu */ -public class NacosWatch implements ApplicationEventPublisherAware, SmartLifecycle { +public class NacosWatch implements ApplicationEventPublisherAware, SmartLifecycle, DisposableBean { private static final Logger log = LoggerFactory.getLogger(NacosWatch.class); @@ -66,6 +68,14 @@ public class NacosWatch implements ApplicationEventPublisherAware, SmartLifecycl private final ThreadPoolTaskScheduler taskScheduler; + public NacosWatch(NacosServiceManager nacosServiceManager, + NacosDiscoveryProperties properties) { + this.nacosServiceManager = nacosServiceManager; + this.properties = properties; + this.taskScheduler = getTaskScheduler(); + } + + @Deprecated public NacosWatch(NacosServiceManager nacosServiceManager, NacosDiscoveryProperties properties, ObjectProvider taskScheduler) { @@ -191,4 +201,8 @@ public class NacosWatch implements ApplicationEventPublisherAware, SmartLifecycl } + @Override + public void destroy() { + this.stop(); + } } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/actuate/health/NacosDiscoveryHealthIndicator.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/actuate/health/NacosDiscoveryHealthIndicator.java index 5a925ec30..32e523bb1 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/actuate/health/NacosDiscoveryHealthIndicator.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/actuate/health/NacosDiscoveryHealthIndicator.java @@ -31,6 +31,16 @@ import org.springframework.boot.actuate.health.HealthIndicator; */ public class NacosDiscoveryHealthIndicator extends AbstractHealthIndicator { + /** + * status up. + */ + private static final String STATUS_UP = "UP"; + + /** + * status down. + */ + private static final String STATUS_DOWN = "DOWN"; + private final NamingService namingService; public NacosDiscoveryHealthIndicator(NamingService namingService) { @@ -44,10 +54,10 @@ public class NacosDiscoveryHealthIndicator extends AbstractHealthIndicator { // Set the status to Builder builder.status(status); switch (status) { - case "UP": + case STATUS_UP: builder.up(); break; - case "DOWN": + case STATUS_DOWN: builder.down(); break; default: diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/logging/NacosLoggingListener.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/logging/NacosLoggingListener.java new file mode 100644 index 000000000..91822ace6 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/discovery/logging/NacosLoggingListener.java @@ -0,0 +1,54 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.nacos.discovery.logging; + +import com.alibaba.nacos.client.logging.NacosLogging; + +import org.springframework.boot.context.event.ApplicationEnvironmentPreparedEvent; +import org.springframework.context.ApplicationEvent; +import org.springframework.context.event.GenericApplicationListener; +import org.springframework.core.Ordered; +import org.springframework.core.ResolvableType; + +/** + * Reload nacos log configuration file, after + * {@link org.springframework.boot.context.logging.LoggingApplicationListener}. + * + * @author mai.jh + */ +public class NacosLoggingListener implements GenericApplicationListener { + + @Override + public boolean supportsEventType(ResolvableType resolvableType) { + Class type = resolvableType.getRawClass(); + if (type != null) { + return ApplicationEnvironmentPreparedEvent.class.isAssignableFrom(type); + } + return false; + } + + @Override + public void onApplicationEvent(ApplicationEvent applicationEvent) { + NacosLogging.getInstance().loadConfiguration(); + } + + @Override + public int getOrder() { + return Ordered.HIGHEST_PRECEDENCE + 21; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/registry/NacosServiceRegistry.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/registry/NacosServiceRegistry.java index eff8e3071..27ff3bc58 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/registry/NacosServiceRegistry.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/registry/NacosServiceRegistry.java @@ -28,7 +28,6 @@ import com.alibaba.nacos.api.naming.pojo.Instance; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import org.springframework.beans.factory.annotation.Autowired; import org.springframework.cloud.client.serviceregistry.Registration; import org.springframework.cloud.client.serviceregistry.ServiceRegistry; @@ -49,11 +48,12 @@ public class NacosServiceRegistry implements ServiceRegistry { private final NacosDiscoveryProperties nacosDiscoveryProperties; - @Autowired - private NacosServiceManager nacosServiceManager; + private final NacosServiceManager nacosServiceManager; - public NacosServiceRegistry(NacosDiscoveryProperties nacosDiscoveryProperties) { + public NacosServiceRegistry(NacosServiceManager nacosServiceManager, + NacosDiscoveryProperties nacosDiscoveryProperties) { this.nacosDiscoveryProperties = nacosDiscoveryProperties; + this.nacosServiceManager = nacosServiceManager; } @Override @@ -76,11 +76,15 @@ public class NacosServiceRegistry implements ServiceRegistry { instance.getIp(), instance.getPort()); } catch (Exception e) { - log.error("nacos registry, {} register failed...{},", serviceId, - registration.toString(), e); - // rethrow a RuntimeException if the registration is failed. - // issue : https://github.com/alibaba/spring-cloud-alibaba/issues/1132 - rethrowRuntimeException(e); + if (nacosDiscoveryProperties.isFailFast()) { + log.error("nacos registry, {} register failed...{},", serviceId, + registration.toString(), e); + rethrowRuntimeException(e); + } + else { + log.warn("Failfast is false. {} register failed...{},", serviceId, + registration.toString(), e); + } } } @@ -155,8 +159,10 @@ public class NacosServiceRegistry implements ServiceRegistry { public Object getStatus(Registration registration) { String serviceName = registration.getServiceId(); + String group = nacosDiscoveryProperties.getGroup(); try { - List instances = namingService().getAllInstances(serviceName); + List instances = namingService().getAllInstances(serviceName, + group); for (Instance instance : instances) { if (instance.getIp().equalsIgnoreCase(nacosDiscoveryProperties.getIp()) && instance.getPort() == nacosDiscoveryProperties.getPort()) { diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/registry/NacosServiceRegistryAutoConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/registry/NacosServiceRegistryAutoConfiguration.java index c51c085ed..2f7273592 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/registry/NacosServiceRegistryAutoConfiguration.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/java/com/alibaba/cloud/nacos/registry/NacosServiceRegistryAutoConfiguration.java @@ -20,6 +20,7 @@ import java.util.List; import com.alibaba.cloud.nacos.ConditionalOnNacosDiscoveryEnabled; import com.alibaba.cloud.nacos.NacosDiscoveryProperties; +import com.alibaba.cloud.nacos.NacosServiceManager; import com.alibaba.cloud.nacos.discovery.NacosDiscoveryAutoConfiguration; import org.springframework.beans.factory.ObjectProvider; @@ -50,8 +51,9 @@ public class NacosServiceRegistryAutoConfiguration { @Bean public NacosServiceRegistry nacosServiceRegistry( + NacosServiceManager nacosServiceManager, NacosDiscoveryProperties nacosDiscoveryProperties) { - return new NacosServiceRegistry(nacosDiscoveryProperties); + return new NacosServiceRegistry(nacosServiceManager, nacosDiscoveryProperties); } @Bean diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/resources/META-INF/additional-spring-configuration-metadata.json b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/resources/META-INF/additional-spring-configuration-metadata.json index 12a9c531c..2e06a71ac 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/resources/META-INF/additional-spring-configuration-metadata.json +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/resources/META-INF/additional-spring-configuration-metadata.json @@ -2,7 +2,7 @@ { "name": "spring.cloud.nacos.server-addr", "type": "java.lang.String", - "defaultValue": "localhost:8848", + "defaultValue": "127.0.0.1:8848", "description": "nacos server address." }, { diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/resources/META-INF/spring.factories b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/resources/META-INF/spring.factories index 6d42f47f5..c12ffb14a 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/resources/META-INF/spring.factories +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/main/resources/META-INF/spring.factories @@ -9,3 +9,5 @@ org.springframework.boot.autoconfigure.EnableAutoConfiguration=\ com.alibaba.cloud.nacos.NacosServiceAutoConfiguration org.springframework.cloud.bootstrap.BootstrapConfiguration=\ com.alibaba.cloud.nacos.discovery.configclient.NacosDiscoveryClientConfigServiceBootstrapConfiguration +org.springframework.context.ApplicationListener=\ + com.alibaba.cloud.nacos.discovery.logging.NacosLoggingListener diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/test/java/com/alibaba/cloud/nacos/NacosDiscoveryPropertiesServerAddressBothLevelTests.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/test/java/com/alibaba/cloud/nacos/NacosDiscoveryPropertiesServerAddressBothLevelTests.java index 9e1ce8ae3..9a54b2ae1 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/test/java/com/alibaba/cloud/nacos/NacosDiscoveryPropertiesServerAddressBothLevelTests.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/test/java/com/alibaba/cloud/nacos/NacosDiscoveryPropertiesServerAddressBothLevelTests.java @@ -29,18 +29,17 @@ import org.springframework.cloud.client.serviceregistry.AutoServiceRegistrationC import org.springframework.context.annotation.Configuration; import org.springframework.test.context.junit4.SpringRunner; +import static com.alibaba.cloud.nacos.NacosDiscoveryPropertiesServerAddressBothLevelTests.TestConfig; import static org.assertj.core.api.Assertions.assertThat; -import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; /** * @author lyuzb */ @RunWith(SpringRunner.class) -@SpringBootTest( - classes = NacosDiscoveryPropertiesServerAddressBothLevelTests.TestConfig.class, - properties = { "spring.cloud.nacos.discovery.server-addr=321.321.321.321:8848", - "spring.cloud.nacos.server-addr=123.123.123.123:8848" }, - webEnvironment = RANDOM_PORT) +@SpringBootTest(classes = TestConfig.class, properties = { + "spring.application.name=app", + "spring.cloud.nacos.discovery.server-addr=321.321.321.321:8848", + "spring.cloud.nacos.server-addr=123.123.123.123:8848" }) public class NacosDiscoveryPropertiesServerAddressBothLevelTests { @Autowired diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/test/java/com/alibaba/cloud/nacos/NacosDiscoveryPropertiesServerAddressTopLevelTests.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/test/java/com/alibaba/cloud/nacos/NacosDiscoveryPropertiesServerAddressTopLevelTests.java index db90b57a7..fb24e6e80 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/test/java/com/alibaba/cloud/nacos/NacosDiscoveryPropertiesServerAddressTopLevelTests.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-nacos-discovery/src/test/java/com/alibaba/cloud/nacos/NacosDiscoveryPropertiesServerAddressTopLevelTests.java @@ -29,19 +29,17 @@ import org.springframework.cloud.client.serviceregistry.AutoServiceRegistrationC import org.springframework.context.annotation.Configuration; import org.springframework.test.context.junit4.SpringRunner; +import static com.alibaba.cloud.nacos.NacosDiscoveryPropertiesServerAddressTopLevelTests.TestConfig; import static org.assertj.core.api.Assertions.assertThat; -import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; /** * @author lyuzb * */ @RunWith(SpringRunner.class) -@SpringBootTest( - classes = NacosDiscoveryPropertiesServerAddressTopLevelTests.TestConfig.class, - properties = { "spring.cloud.nacos.server-addr=123.123.123.123:8848" }, - webEnvironment = RANDOM_PORT) - +@SpringBootTest(classes = TestConfig.class, properties = { + "spring.application.name=app", + "spring.cloud.nacos.server-addr=123.123.123.123:8848" }) public class NacosDiscoveryPropertiesServerAddressTopLevelTests { @Autowired diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/SentinelWebAutoConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/SentinelWebAutoConfiguration.java index b899320cf..ff37a0a03 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/SentinelWebAutoConfiguration.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/SentinelWebAutoConfiguration.java @@ -36,7 +36,6 @@ import org.springframework.boot.context.properties.EnableConfigurationProperties import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.util.StringUtils; -import org.springframework.web.servlet.config.annotation.InterceptorRegistry; import org.springframework.web.servlet.config.annotation.WebMvcConfigurer; /** @@ -65,23 +64,6 @@ public class SentinelWebAutoConfiguration implements WebMvcConfigurer { @Autowired private Optional requestOriginParserOptional; - @Autowired - private Optional sentinelWebInterceptorOptional; - - @Override - public void addInterceptors(InterceptorRegistry registry) { - if (!sentinelWebInterceptorOptional.isPresent()) { - return; - } - SentinelProperties.Filter filterConfig = properties.getFilter(); - registry.addInterceptor(sentinelWebInterceptorOptional.get()) - .order(filterConfig.getOrder()) - .addPathPatterns(filterConfig.getUrlPatterns()); - log.info( - "[Sentinel Starter] register SentinelWebInterceptor with urlPatterns: {}.", - filterConfig.getUrlPatterns()); - } - @Bean @ConditionalOnProperty(name = "spring.cloud.sentinel.filter.enabled", matchIfMissing = true) @@ -118,4 +100,11 @@ public class SentinelWebAutoConfiguration implements WebMvcConfigurer { return sentinelWebMvcConfig; } + @Bean + @ConditionalOnProperty(name = "spring.cloud.sentinel.filter.enabled", + matchIfMissing = true) + public SentinelWebMvcConfigurer sentinelWebMvcConfigurer() { + return new SentinelWebMvcConfigurer(); + } + } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/SentinelWebMvcConfigurer.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/SentinelWebMvcConfigurer.java new file mode 100644 index 000000000..c9c518b7a --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/SentinelWebMvcConfigurer.java @@ -0,0 +1,57 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.sentinel; + +import java.util.Optional; + +import com.alibaba.csp.sentinel.adapter.spring.webmvc.SentinelWebInterceptor; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.web.servlet.config.annotation.InterceptorRegistry; +import org.springframework.web.servlet.config.annotation.WebMvcConfigurer; + +/** + * @author: chao.wu + */ +public class SentinelWebMvcConfigurer implements WebMvcConfigurer { + + private static final Logger log = LoggerFactory + .getLogger(SentinelWebMvcConfigurer.class); + + @Autowired + private SentinelProperties sentinelProperties; + + @Autowired + private Optional sentinelWebInterceptorOptional; + + @Override + public void addInterceptors(InterceptorRegistry registry) { + if (!sentinelWebInterceptorOptional.isPresent()) { + return; + } + SentinelProperties.Filter filterConfig = sentinelProperties.getFilter(); + registry.addInterceptor(sentinelWebInterceptorOptional.get()) + .order(filterConfig.getOrder()) + .addPathPatterns(filterConfig.getUrlPatterns()); + log.info( + "[Sentinel Starter] register SentinelWebInterceptor with urlPatterns: {}.", + filterConfig.getUrlPatterns()); + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/custom/SentinelProtectInterceptor.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/custom/SentinelProtectInterceptor.java index fbd468b26..5fdc0c0d4 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/custom/SentinelProtectInterceptor.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/custom/SentinelProtectInterceptor.java @@ -86,13 +86,23 @@ public class SentinelProtectInterceptor implements ClientHttpRequestInterceptor Tracer.trace( new IllegalStateException("RestTemplate ErrorHandler has error")); } + return response; } catch (Throwable e) { - if (!BlockException.isBlockException(e)) { - Tracer.trace(e); + if (BlockException.isBlockException(e)) { + return handleBlockException(request, body, execution, (BlockException) e); } else { - return handleBlockException(request, body, execution, (BlockException) e); + Tracer.traceEntry(e, hostEntry); + if (e instanceof IOException) { + throw (IOException) e; + } + else if (e instanceof RuntimeException) { + throw (RuntimeException) e; + } + else { + throw new IOException(e); + } } } finally { @@ -103,7 +113,6 @@ public class SentinelProtectInterceptor implements ClientHttpRequestInterceptor hostEntry.exit(); } } - return response; } private ClientHttpResponse handleBlockException(HttpRequest request, byte[] body, diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/endpoint/SentinelHealthIndicator.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/endpoint/SentinelHealthIndicator.java index 996a483b7..b502fc543 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/endpoint/SentinelHealthIndicator.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/endpoint/SentinelHealthIndicator.java @@ -25,7 +25,7 @@ import com.alibaba.csp.sentinel.datasource.AbstractDataSource; import com.alibaba.csp.sentinel.heartbeat.HeartbeatSenderProvider; import com.alibaba.csp.sentinel.transport.HeartbeatSender; import com.alibaba.csp.sentinel.transport.config.TransportConfig; -import com.alibaba.csp.sentinel.util.function.Tuple2; +import com.alibaba.csp.sentinel.transport.endpoint.Endpoint; import org.springframework.beans.factory.support.DefaultListableBeanFactory; import org.springframework.boot.actuate.health.AbstractHealthIndicator; @@ -84,8 +84,7 @@ public class SentinelHealthIndicator extends AbstractHealthIndicator { // Check health of Dashboard boolean dashboardUp = true; - List> consoleServerList = TransportConfig - .getConsoleServerList(); + List consoleServerList = TransportConfig.getConsoleServerList(); if (CollectionUtils.isEmpty(consoleServerList)) { // If Dashboard isn't configured, it's OK and mark the status of Dashboard // with UNKNOWN. diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/feign/SentinelFeign.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/feign/SentinelFeign.java index 2624aae63..b80227614 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/feign/SentinelFeign.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/feign/SentinelFeign.java @@ -79,22 +79,22 @@ public final class SentinelFeign { @Override public InvocationHandler create(Target target, Map dispatch) { - GenericApplicationContext gctx = (GenericApplicationContext) Builder.this.applicationContext; BeanDefinition def = gctx.getBeanDefinition(target.type().getName()); - /** - * Due to the change of the initialization sequence, BeanFactory.getBean will cause a circular dependency. - * So FeignClientFactoryBean can only be obtained from BeanDefinition + /* + * Due to the change of the initialization sequence, + * BeanFactory.getBean will cause a circular dependency. So + * FeignClientFactoryBean can only be obtained from BeanDefinition */ - FeignClientFactoryBean feignClientFactoryBean = (FeignClientFactoryBean) def.getAttribute("feignClientsRegistrarFactoryBean"); + FeignClientFactoryBean feignClientFactoryBean = (FeignClientFactoryBean) def + .getAttribute("feignClientsRegistrarFactoryBean"); Class fallback = feignClientFactoryBean.getFallback(); Class fallbackFactory = feignClientFactoryBean.getFallbackFactory(); String beanName = feignClientFactoryBean.getContextId(); - if (!StringUtils.hasText(beanName)) { - beanName = feignClientFactoryBean.getName(); + beanName = (String) getFieldValue(feignClientFactoryBean, "name"); } Object fallbackInstance; @@ -113,6 +113,7 @@ public final class SentinelFeign { return new SentinelInvocationHandler(target, dispatch, fallbackFactoryInstance); } + return new SentinelInvocationHandler(target, dispatch); } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/feign/SentinelInvocationHandler.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/feign/SentinelInvocationHandler.java index 92436d9e2..9910724bf 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/feign/SentinelInvocationHandler.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/main/java/com/alibaba/cloud/sentinel/feign/SentinelInvocationHandler.java @@ -111,7 +111,7 @@ public class SentinelInvocationHandler implements InvocationHandler { catch (Throwable ex) { // fallback handle if (!BlockException.isBlockException(ex)) { - Tracer.trace(ex); + Tracer.traceEntry(ex, entry); } if (fallbackFactory != null) { try { diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/test/java/com/alibaba/cloud/sentinel/SentinelAutoConfigurationTests.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/test/java/com/alibaba/cloud/sentinel/SentinelAutoConfigurationTests.java index 56c5bf6bd..2b4c51e0f 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/test/java/com/alibaba/cloud/sentinel/SentinelAutoConfigurationTests.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sentinel/src/test/java/com/alibaba/cloud/sentinel/SentinelAutoConfigurationTests.java @@ -31,7 +31,8 @@ import com.alibaba.csp.sentinel.slots.block.RuleConstant; import com.alibaba.csp.sentinel.slots.block.flow.FlowRule; import com.alibaba.csp.sentinel.slots.block.flow.FlowRuleManager; import com.alibaba.csp.sentinel.transport.config.TransportConfig; -import com.alibaba.csp.sentinel.util.function.Tuple2; +import com.alibaba.csp.sentinel.transport.endpoint.Endpoint; +import com.alibaba.csp.sentinel.transport.endpoint.Protocol; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; @@ -134,8 +135,10 @@ public class SentinelAutoConfigurationTests { Map map = sentinelEndpoint.invoke(); assertThat(map.get("logUsePid")).isEqualTo(Boolean.TRUE); - assertThat(map.get("consoleServer").toString()).isEqualTo( - Arrays.asList(Tuple2.of("localhost", 8080), Tuple2.of("localhost", 8081)) + assertThat(map.get("consoleServer").toString()) + .isEqualTo(Arrays + .asList(new Endpoint(Protocol.HTTP, "localhost", 8080), + new Endpoint(Protocol.HTTP, "localhost", 8081)) .toString()); assertThat(map.get("clientPort")).isEqualTo("9999"); assertThat(map.get("heartbeatIntervalMs")).isEqualTo(20000L); @@ -185,8 +188,10 @@ public class SentinelAutoConfigurationTests { @Test public void testSentinelSystemProperties() { assertThat(LogBase.isLogNameUsePid()).isEqualTo(true); - assertThat(TransportConfig.getConsoleServerList().toString()).isEqualTo( - Arrays.asList(Tuple2.of("localhost", 8080), Tuple2.of("localhost", 8081)) + assertThat(TransportConfig.getConsoleServerList().toString()) + .isEqualTo(Arrays + .asList(new Endpoint(Protocol.HTTP, "localhost", 8080), + new Endpoint(Protocol.HTTP, "localhost", 8081)) .toString()); assertThat(TransportConfig.getPort()).isEqualTo("9999"); assertThat(TransportConfig.getHeartbeatIntervalMs().longValue()) diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/CustomHealthCheckHandler.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/CustomHealthCheckHandler.java new file mode 100644 index 000000000..69598f1c0 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/CustomHealthCheckHandler.java @@ -0,0 +1,26 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.sidecar; + +/** + * @author yuhuangbin + */ +public interface CustomHealthCheckHandler { + + void handler(String applicationName, SidecarInstanceInfo sidecarInstanceInfo); + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarHealthChecker.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarHealthChecker.java index eaf426392..e5673aaa8 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarHealthChecker.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarHealthChecker.java @@ -25,6 +25,8 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; import reactor.core.scheduler.Schedulers; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.actuate.health.HealthIndicator; import org.springframework.boot.actuate.health.Status; import org.springframework.core.env.ConfigurableEnvironment; @@ -37,7 +39,7 @@ public class SidecarHealthChecker { private static final Logger log = LoggerFactory.getLogger(SidecarHealthChecker.class); - private final Map sidecarInstanceCacheMap = new ConcurrentHashMap<>(); + private final Map sidecarInstanceCacheMap = new ConcurrentHashMap<>(); private final SidecarDiscoveryClient sidecarDiscoveryClient; @@ -47,6 +49,9 @@ public class SidecarHealthChecker { private final ConfigurableEnvironment environment; + @Autowired + private ObjectProvider customHealthCheckHandlerObjectProvider; + public SidecarHealthChecker(SidecarDiscoveryClient sidecarDiscoveryClient, HealthIndicator healthIndicator, SidecarProperties sidecarProperties, ConfigurableEnvironment environment) { @@ -64,9 +69,10 @@ public class SidecarHealthChecker { Status status = healthIndicator.health().getStatus(); - instanceCache(applicationName, ip, port, status); + SidecarInstanceInfo sidecarInstanceInfo = instanceCache(applicationName, ip, + port, status); if (status.equals(Status.UP)) { - if (needRegister(applicationName, ip, port, status)) { + if (needRegister(applicationName, sidecarInstanceInfo)) { this.sidecarDiscoveryClient.registerInstance(applicationName, ip, port); log.info( @@ -84,30 +90,37 @@ public class SidecarHealthChecker { buildCache(ip, port, status)); } + try { + customHealthCheckHandlerObjectProvider + .ifAvailable(customHealthCheckHandler -> customHealthCheckHandler + .handler(applicationName, sidecarInstanceInfo)); + } + catch (Exception e) { + // ignore + } }, 0, sidecarProperties.getHealthCheckInterval(), TimeUnit.MILLISECONDS); } - private void instanceCache(String applicationName, String ip, Integer port, - Status status) { - sidecarInstanceCacheMap.putIfAbsent(applicationName, - buildCache(ip, port, status)); + private SidecarInstanceInfo instanceCache(String applicationName, String ip, + Integer port, Status status) { + SidecarInstanceInfo sidecarInstanceInfo = buildCache(ip, port, status); + sidecarInstanceCacheMap.putIfAbsent(applicationName, sidecarInstanceInfo); + return sidecarInstanceInfo; } - private boolean needRegister(String applicationName, String ip, Integer port, - Status status) { - SidecarInstanceCache cacheRecord = sidecarInstanceCacheMap.get(applicationName); - SidecarInstanceCache cache = buildCache(ip, port, status); - - if (!Objects.equals(cache, cacheRecord)) { + private boolean needRegister(String applicationName, + SidecarInstanceInfo sidecarInstanceInfo) { + SidecarInstanceInfo cacheRecord = sidecarInstanceCacheMap.get(applicationName); + if (!Objects.equals(sidecarInstanceInfo, cacheRecord)) { // modify the cache info - sidecarInstanceCacheMap.put(applicationName, cache); + sidecarInstanceCacheMap.put(applicationName, sidecarInstanceInfo); return true; } return false; } - private SidecarInstanceCache buildCache(String ip, Integer port, Status status) { - SidecarInstanceCache cache = new SidecarInstanceCache(); + private SidecarInstanceInfo buildCache(String ip, Integer port, Status status) { + SidecarInstanceInfo cache = new SidecarInstanceInfo(); cache.setIp(ip); cache.setPort(port); cache.setStatus(status); diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarInstanceCache.java b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarInstanceInfo.java similarity index 94% rename from spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarInstanceCache.java rename to spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarInstanceInfo.java index 0c18f1aa4..14892268a 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarInstanceCache.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-alibaba-sidecar/src/main/java/com/alibaba/cloud/sidecar/SidecarInstanceInfo.java @@ -23,7 +23,7 @@ import org.springframework.boot.actuate.health.Status; /** * @author yuhuangbin */ -public class SidecarInstanceCache { +public class SidecarInstanceInfo { private String ip; @@ -63,7 +63,7 @@ public class SidecarInstanceCache { if (o == null || getClass() != o.getClass()) { return false; } - SidecarInstanceCache that = (SidecarInstanceCache) o; + SidecarInstanceInfo that = (SidecarInstanceInfo) o; return Objects.equals(ip, that.ip) && Objects.equals(port, that.port) && Objects.equals(status, that.status); } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/pom.xml b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/pom.xml index b404252b1..925bb2735 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/pom.xml +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/pom.xml @@ -17,6 +17,11 @@ + + com.alibaba.cloud + spring-cloud-alibaba-commons + + org.springframework.boot diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/autoconfigure/DubboServiceRegistrationAutoConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/autoconfigure/DubboServiceRegistrationAutoConfiguration.java index b0897e6ca..8edb3c589 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/autoconfigure/DubboServiceRegistrationAutoConfiguration.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/autoconfigure/DubboServiceRegistrationAutoConfiguration.java @@ -18,6 +18,7 @@ package com.alibaba.cloud.dubbo.autoconfigure; import java.util.Collection; import java.util.HashSet; +import java.util.Iterator; import java.util.List; import java.util.Map; import java.util.Set; @@ -50,6 +51,7 @@ import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.cloud.client.ServiceInstance; import org.springframework.cloud.client.serviceregistry.Registration; import org.springframework.cloud.client.serviceregistry.ServiceRegistry; +import org.springframework.cloud.consul.discovery.ConsulDiscoveryProperties; import org.springframework.cloud.consul.serviceregistry.ConsulRegistration; import org.springframework.cloud.netflix.eureka.EurekaInstanceConfigBean; import org.springframework.cloud.netflix.eureka.serviceregistry.EurekaAutoServiceRegistration; @@ -61,6 +63,7 @@ import org.springframework.context.annotation.Conditional; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Import; import org.springframework.context.event.EventListener; +import org.springframework.util.StringUtils; import static com.alibaba.cloud.dubbo.autoconfigure.DubboServiceRegistrationAutoConfiguration.CONSUL_AUTO_SERVICE_AUTO_CONFIGURATION_CLASS_NAME; import static com.alibaba.cloud.dubbo.autoconfigure.DubboServiceRegistrationAutoConfiguration.EUREKA_CLIENT_AUTO_CONFIGURATION_CLASS_NAME; @@ -77,12 +80,11 @@ import static org.springframework.util.ObjectUtils.isEmpty; @Configuration(proxyBeanMethods = false) @Import({ DubboServiceRegistrationEventPublishingAspect.class, DubboBootstrapStartCommandLineRunner.class }) -@ConditionalOnProperty(value = "spring.cloud.service-registry.auto-registration.enabled", - matchIfMissing = true) +@ConditionalOnProperty(value = "spring.cloud.service-registry.auto-registration.enabled", matchIfMissing = true) @AutoConfigureAfter(name = { EUREKA_CLIENT_AUTO_CONFIGURATION_CLASS_NAME, CONSUL_AUTO_SERVICE_AUTO_CONFIGURATION_CLASS_NAME, - "org.springframework.cloud.client.serviceregistry.AutoServiceRegistrationAutoConfiguration" }, - value = { DubboMetadataAutoConfiguration.class }) + "org.springframework.cloud.client.serviceregistry.AutoServiceRegistrationAutoConfiguration" }, value = { + DubboMetadataAutoConfiguration.class }) public class DubboServiceRegistrationAutoConfiguration { /** @@ -260,6 +262,9 @@ public class DubboServiceRegistrationAutoConfiguration { @AutoConfigureOrder class ConsulConfiguration { + @Autowired + private ConsulDiscoveryProperties consulDiscoveryProperties; + @EventListener(DubboBootstrapStartedEvent.class) public void attachURLsIntoMetadataBeforeReRegist( DubboBootstrapStartedEvent event) { @@ -284,16 +289,37 @@ public class DubboServiceRegistrationAutoConfiguration { })); } + @EventListener(ServiceInstancePreRegisteredEvent.class) + public void onServiceInstancePreRegistered( + ServiceInstancePreRegisteredEvent event) { + Registration registration = event.getSource(); + attachURLsIntoMetadata((ConsulRegistration) registration); + } + private void attachURLsIntoMetadata(ConsulRegistration consulRegistration) { - NewService newService = consulRegistration.getService(); Map serviceMetadata = dubboServiceMetadataRepository .getDubboMetadataServiceMetadata(); - if (!isEmpty(serviceMetadata)) { - List tags = newService.getTags(); - for (Map.Entry entry : serviceMetadata.entrySet()) { - tags.add(entry.getKey() + "=" + entry.getValue()); + if (isEmpty(serviceMetadata)) { + return; + } + NewService newService = consulRegistration.getService(); + // properties `tagsAsMetadata` in tagsAsMetadata is deprecated, and default + // value is true. + for (Map.Entry entry : serviceMetadata.entrySet()) { + attAsTag(newService.getTags(), entry.getKey(), entry.getValue()); + } + } + + private void attAsTag(List tags, String key, String value) { + Iterator iter = tags.iterator(); + while (iter.hasNext()) { + String tag = iter.next(); + String[] tmp = tag.split("="); + if (StringUtils.pathEquals(tmp[0], key)) { + iter.remove(); } } + tags.add(key + "=" + value); } } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/env/DubboCloudProperties.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/env/DubboCloudProperties.java index 577d19e59..5135c5a8c 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/env/DubboCloudProperties.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/env/DubboCloudProperties.java @@ -51,6 +51,10 @@ public class DubboCloudProperties { private String registryType = DUBBO_CLOUD_REGISTRY_PROPERTY_VALUE; + private int maxReSubscribeMetadataTimes = 1000; + + private int reSubscribeMetadataIntervial = 5; + public String getSubscribedServices() { return subscribedServices; } @@ -91,4 +95,20 @@ public class DubboCloudProperties { this.registryType = registryType; } + public int getMaxReSubscribeMetadataTimes() { + return maxReSubscribeMetadataTimes; + } + + public void setMaxReSubscribeMetadataTimes(int maxReSubscribeMetadataTimes) { + this.maxReSubscribeMetadataTimes = maxReSubscribeMetadataTimes; + } + + public int getReSubscribeMetadataIntervial() { + return reSubscribeMetadataIntervial; + } + + public void setReSubscribeMetadataIntervial(int reSubscribeMetadataIntervial) { + this.reSubscribeMetadataIntervial = reSubscribeMetadataIntervial; + } + } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/RevisionResolver.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/RevisionResolver.java new file mode 100644 index 000000000..8ffc18457 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/RevisionResolver.java @@ -0,0 +1,93 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.dubbo.metadata; + +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.util.Map; + +import org.apache.dubbo.common.logger.Logger; +import org.apache.dubbo.common.logger.LoggerFactory; + +import org.springframework.cloud.client.ServiceInstance; + +import static com.alibaba.cloud.dubbo.metadata.repository.DubboServiceMetadataRepository.EXPORTED_SERVICES_REVISION_PROPERTY_NAME; +import static java.nio.charset.StandardCharsets.UTF_8; + +/** + * Copy from org.apache.dubbo.metadata.RevisionResolver. + * + * @author theonefx + */ +public final class RevisionResolver { + + /** + * The param key in url. + */ + public static final String SCA_REVSION_KEY = "sca_revision"; + + private static final String EMPTY_REVISION = "0"; + + private static final Logger logger = LoggerFactory.getLogger(RevisionResolver.class); + + private static final char[] hexDigits = { '0', '1', '2', '3', '4', '5', '6', '7', '8', + '9', 'A', 'B', 'C', 'D', 'E', 'F' }; + + private static MessageDigest mdInst; + + static { + try { + mdInst = MessageDigest.getInstance("MD5"); + } + catch (NoSuchAlgorithmException e) { + logger.error("Failed to calculate metadata revision", e); + } + } + + private RevisionResolver() { + + } + + public static String getEmptyRevision() { + return EMPTY_REVISION; + } + + public static String calRevision(String metadata) { + mdInst.update(metadata.getBytes(UTF_8)); + byte[] md5 = mdInst.digest(); + + int j = md5.length; + char[] str = new char[j * 2]; + int k = 0; + for (byte byte0 : md5) { + str[k++] = hexDigits[byte0 >>> 4 & 0xf]; + str[k++] = hexDigits[byte0 & 0xf]; + } + return new String(str); + } + + public static String getRevision(ServiceInstance instance) { + Map metadata = instance.getMetadata(); + String revision = metadata.get(EXPORTED_SERVICES_REVISION_PROPERTY_NAME); + + if (revision == null) { + revision = RevisionResolver.getEmptyRevision(); + } + return revision; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/ServiceInfo.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/ServiceInfo.java new file mode 100644 index 000000000..add184c4f --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/ServiceInfo.java @@ -0,0 +1,232 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.dubbo.metadata; + +import java.io.Serializable; +import java.lang.reflect.Method; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Objects; +import java.util.Set; +import java.util.SortedSet; +import java.util.TreeMap; +import java.util.TreeSet; + +import org.apache.dubbo.common.URL; +import org.apache.dubbo.common.compiler.support.ClassUtils; +import org.apache.dubbo.common.utils.StringUtils; + +import static org.apache.dubbo.common.constants.CommonConstants.GROUP_CHAR_SEPARATOR; +import static org.apache.dubbo.common.constants.CommonConstants.GROUP_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.INTERFACE_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.METHODS_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.PID_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.TIMESTAMP_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY; + +/** + * Copy from org.apache.dubbo.metadata.MetadataInfo.ServiceInfo. + * + * @author theonefx + */ +public class ServiceInfo implements Serializable { + + private static final long serialVersionUID = -258557978718735302L; + + private String name; + + private String group; + + private String version; + + private String protocol; + + private String path; // most of the time, path is the same with the interface name. + + private Map params; + + // params configured on consumer side, + private transient Map consumerParams; + + // service + group + version + private transient String serviceKey; + + // service + group + version + protocol + private transient String matchKey; + + private transient URL url; + + private static final Set IGNORE_KEYS = new HashSet<>(); + static { + IGNORE_KEYS.add(TIMESTAMP_KEY); + IGNORE_KEYS.add(PID_KEY); + IGNORE_KEYS.add(INTERFACE_KEY); + IGNORE_KEYS.add(METHODS_KEY); + } + + public ServiceInfo(URL url) { + this(url.getServiceInterface(), url.getParameter(GROUP_KEY), + url.getParameter(VERSION_KEY), url.getProtocol(), url.getPath(), null); + + this.url = url; + Map params = new TreeMap<>(); + url.getParameters().forEach((k, v) -> { + if (IGNORE_KEYS.contains(k)) { + return; + } + params.put(k, v); + }); + this.params = params; + } + + public ServiceInfo(String name, String group, String version, String protocol, + String path, Map params) { + this.name = name; + this.group = group; + this.version = version; + this.protocol = protocol; + this.path = path; + this.params = params == null ? new HashMap<>() : params; + + this.serviceKey = URL.buildKey(name, group, version); + this.matchKey = buildMatchKey(); + } + + public String getMatchKey() { + if (matchKey != null) { + return matchKey; + } + buildMatchKey(); + return matchKey; + } + + private String buildMatchKey() { + matchKey = getServiceKey(); + if (StringUtils.isNotEmpty(protocol)) { + matchKey = getServiceKey() + GROUP_CHAR_SEPARATOR + protocol; + } + return matchKey; + } + + public String getServiceKey() { + if (serviceKey != null) { + return serviceKey; + } + this.serviceKey = URL.buildKey(name, group, version); + return serviceKey; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public String getGroup() { + return group; + } + + public void setGroup(String group) { + this.group = group; + } + + public String getVersion() { + return version; + } + + public void setVersion(String version) { + this.version = version; + } + + public String getPath() { + return path; + } + + public void setPath(String path) { + this.path = path; + } + + public Map getParams() { + if (params == null) { + return Collections.emptyMap(); + } + return params; + } + + public void setParams(Map params) { + this.params = params; + } + + public String getParameter(String key) { + if (consumerParams != null) { + String value = consumerParams.get(key); + if (value != null) { + return value; + } + } + return params.get(key); + } + + public String toDescString() { + return this.getMatchKey() + getMethodSignaturesString() + getParams(); + } + + private String getMethodSignaturesString() { + SortedSet methodStrings = new TreeSet<>(); + + Method[] methods = ClassUtils.forName(name).getMethods(); + for (Method method : methods) { + methodStrings.add(method.toString()); + } + return methodStrings.toString(); + } + + public URL getUrl() { + return url; + } + + @Override + public boolean equals(Object obj) { + if (obj == null) { + return false; + } + if (!(obj instanceof ServiceInfo)) { + return false; + } + + ServiceInfo serviceInfo = (ServiceInfo) obj; + return this.getMatchKey().equals(serviceInfo.getMatchKey()) + && this.getParams().equals(serviceInfo.getParams()); + } + + @Override + public int hashCode() { + return Objects.hash(getMatchKey(), getParams()); + } + + @Override + public String toString() { + return "service{" + "name='" + name + "'," + "group='" + group + "'," + + "version='" + version + "'," + "protocol='" + protocol + "'," + + "params=" + params + "," + "consumerParams=" + consumerParams + "}"; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/repository/DubboServiceMetadataRepository.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/repository/DubboServiceMetadataRepository.java index a5be34df8..776513f35 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/repository/DubboServiceMetadataRepository.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/metadata/repository/DubboServiceMetadataRepository.java @@ -16,9 +16,11 @@ package com.alibaba.cloud.dubbo.metadata.repository; +import java.util.ArrayList; import java.util.Collections; import java.util.LinkedHashMap; import java.util.LinkedHashSet; +import java.util.LinkedList; import java.util.List; import java.util.Map; import java.util.Objects; @@ -31,6 +33,8 @@ import com.alibaba.cloud.dubbo.env.DubboCloudProperties; import com.alibaba.cloud.dubbo.http.matcher.RequestMetadataMatcher; import com.alibaba.cloud.dubbo.metadata.DubboRestServiceMetadata; import com.alibaba.cloud.dubbo.metadata.RequestMetadata; +import com.alibaba.cloud.dubbo.metadata.RevisionResolver; +import com.alibaba.cloud.dubbo.metadata.ServiceInfo; import com.alibaba.cloud.dubbo.metadata.ServiceRestMetadata; import com.alibaba.cloud.dubbo.registry.event.SubscribedServicesChangedEvent; import com.alibaba.cloud.dubbo.service.DubboMetadataService; @@ -41,6 +45,8 @@ import com.alibaba.cloud.dubbo.util.JSONUtils; import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.type.TypeFactory; import org.apache.dubbo.common.URL; +import org.apache.dubbo.common.constants.CommonConstants; +import org.apache.dubbo.common.utils.CollectionUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -95,9 +101,10 @@ public class DubboServiceMetadataRepository public static final String DUBBO_METADATA_SERVICE_URLS_PROPERTY_NAME = METADATA_SERVICE_URLS_PROPERTY_NAME; /** - * The {@link String#format(String, Object...) pattern} of dubbo protocols port. + * The key of dubbo metadata revision. copyed from + * ServiceInstanceMetadataUtils.EXPORTED_SERVICES_REVISION_PROPERTY_NAME. */ - public static final String DUBBO_PROTOCOLS_PORT_PROPERTY_NAME_PATTERN = "dubbo.protocols.%s.port"; + public static String EXPORTED_SERVICES_REVISION_PROPERTY_NAME = "dubbo.metadata.revision"; private final Logger logger = LoggerFactory.getLogger(getClass()); @@ -115,14 +122,11 @@ public class DubboServiceMetadataRepository */ private final MultiValueMap allExportedURLs = new LinkedMultiValueMap<>(); - // =================================== Registration - // =================================== // + // ======================== Registration ======================== // - // ==================================================================================== - // // + // ============================================================== // - // =================================== Subscription - // =================================== // + // ======================== Subscription ======================== // /** * A Map to store REST metadata temporary, its' key is the special service name for a * Dubbo service, the value is a JSON content of JAX-RS or Spring MVC REST metadata @@ -132,24 +136,20 @@ public class DubboServiceMetadataRepository private ApplicationEventPublisher applicationEventPublisher; - // ==================================================================================== - // // + // =============================================================== // - // =================================== REST Metadata - // ================================== // + // ======================== REST Metadata ======================== // private volatile Set subscribedServices = emptySet(); /** * Key is application name Value is Map<RequestMetadata, * DubboRestServiceMetadata>. */ - private Map> dubboRestServiceMetadataRepository = newHashMap(); + private final Map> dubboRestServiceMetadataRepository = newHashMap(); - // ==================================================================================== - // // + // =============================================================== // - // =================================== Dependencies - // =================================== // + // ======================== Dependencies ========================= // @Autowired private DubboCloudProperties dubboCloudProperties; @@ -178,8 +178,7 @@ public class DubboServiceMetadataRepository @Autowired private DubboMetadataServiceExporter dubboMetadataServiceExporter; - // ==================================================================================== - // // + // =============================================================== // private static Map getMap(Map> repository, String key) { @@ -187,12 +186,7 @@ public class DubboServiceMetadataRepository } private static V getOrDefault(Map source, K key, V defaultValue) { - V value = source.get(key); - if (value == null) { - value = defaultValue; - source.put(key, value); - } - return value; + return source.computeIfAbsent(key, k -> defaultValue); } private static Map newHashMap() { @@ -237,9 +231,6 @@ public class DubboServiceMetadataRepository dispatchEvent(new SubscribedServicesChangedEvent(this, oldSubscribedServices, newSubscribedServices)); - // clear old one, help GC - oldSubscribedServices.clear(); - return newSubscribedServices.stream(); } @@ -249,13 +240,14 @@ public class DubboServiceMetadataRepository @Override public void afterSingletonsInstantiated() { - initializeMetadata(); + // inited by DubboCloudRegistry.preInit() @theonefx + // initializeMetadata(); } /** * Initialize the metadata. */ - private void initializeMetadata() { + public void initializeMetadata() { doGetSubscribedServices().forEach(this::initializeMetadata); if (logger.isInfoEnabled()) { logger.info("The metadata of Dubbo services has been initialized"); @@ -302,12 +294,36 @@ public class DubboServiceMetadataRepository addDubboMetadataServiceURLsMetadata(metadata, dubboMetadataServiceURLs); addDubboProtocolsPortMetadata(metadata); + addRevision(metadata); return Collections.unmodifiableMap(metadata); } + private void addRevision(Map metadata) { + metadata.put(EXPORTED_SERVICES_REVISION_PROPERTY_NAME, calAndGetRevision()); + } + + public String calAndGetRevision() { + if (CollectionUtils.isEmptyMap(allExportedURLs)) { + return RevisionResolver.getEmptyRevision(); + } + else { + List descs = new ArrayList<>(allExportedURLs.size()); + for (Map.Entry> entry : allExportedURLs.entrySet()) { + entry.getValue().stream().map(ServiceInfo::new) + .map(ServiceInfo::toDescString).forEach(descs::add); + } + + descs.sort(String::compareTo); + + return RevisionResolver.calRevision(descs.toString()); + } + } + private void removeDubboMetadataServiceURLs(List dubboMetadataServiceURLs) { - dubboMetadataServiceURLs.forEach(this::unexportURL); + dubboMetadataServiceURLs.stream().map(URL::getServiceKey).distinct() + .forEach(allExportedURLs::remove); + // dubboMetadataServiceURLs.forEach(this::unexportURL); } private void addDubboMetadataServiceURLsMetadata(Map metadata, @@ -362,7 +378,7 @@ public class DubboServiceMetadataRepository String ipAddress = hostInfo.getIpAddress(); // To use InetUtils to set IP if they are different // issue : - // https://github.com/spring-cloud-incubator/spring-cloud-alibaba/issues/589 + // https://github.com/alibaba/spring-cloud-alibaba/issues/589 if (!Objects.equals(url.getHost(), ipAddress)) { actualURL = url.setHost(ipAddress); } @@ -372,7 +388,7 @@ public class DubboServiceMetadataRepository public void unexportURL(URL url) { String key = url.getServiceKey(); // NPE issue : - // https://github.com/spring-cloud-incubator/spring-cloud-alibaba/issues/591 + // https://github.com/alibaba/spring-cloud-alibaba/issues/591 List urls = allExportedURLs.get(key); if (!isEmpty(urls)) { urls.remove(url); @@ -407,8 +423,34 @@ public class DubboServiceMetadataRepository public List getExportedURLs(String serviceInterface, String group, String version) { - String serviceKey = URL.buildKey(serviceInterface, group, version); - return allExportedURLs.getOrDefault(serviceKey, Collections.emptyList()); + if (group != null) { + List urls = new LinkedList<>(); + if (CommonConstants.ANY_VALUE.equals(group)) { + String serviceKey = URL.buildKey(serviceInterface, group, version); + String expectKey = serviceKey.substring(2); + for (String key : allExportedURLs.keySet()) { + if (key.endsWith(expectKey)) { + urls.addAll(allExportedURLs.get(key)); + } + } + } + else { + String[] groups = group.split(CommonConstants.COMMA_SEPARATOR); + for (String expectKey : groups) { + String serviceKey = URL.buildKey(serviceInterface, expectKey, + version); + List urlList = allExportedURLs.get(serviceKey); + if (urlList != null) { + urls.addAll(urlList); + } + } + } + return urls; + } + else { + String serviceKey = URL.buildKey(serviceInterface, null, version); + return allExportedURLs.getOrDefault(serviceKey, Collections.emptyList()); + } } /** diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/AbstractServiceSubscribeHandler.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/AbstractServiceSubscribeHandler.java new file mode 100644 index 000000000..aa642c3c6 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/AbstractServiceSubscribeHandler.java @@ -0,0 +1,90 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.dubbo.registry; + +import java.util.Collections; +import java.util.List; +import java.util.concurrent.atomic.AtomicBoolean; + +import org.apache.dubbo.common.URL; +import org.apache.dubbo.registry.NotifyListener; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import static org.apache.dubbo.common.URLBuilder.from; +import static org.apache.dubbo.common.constants.RegistryConstants.CATEGORY_KEY; +import static org.apache.dubbo.common.constants.RegistryConstants.EMPTY_PROTOCOL; +import static org.apache.dubbo.common.utils.CollectionUtils.isEmpty; + +/** + * @author theonefx + */ +public abstract class AbstractServiceSubscribeHandler { + + protected final Logger logger = LoggerFactory.getLogger(getClass()); + + protected final URL url; + + protected final NotifyListener listener; + + protected final DubboCloudRegistry registry; + + public AbstractServiceSubscribeHandler(URL url, NotifyListener listener, + DubboCloudRegistry registry) { + this.url = url; + this.listener = listener; + this.registry = registry; + } + + protected void notifyAllSubscribedURLs(URL url, List subscribedURLs, + NotifyListener listener) { + + if (isEmpty(subscribedURLs)) { + // Add the EMPTY_PROTOCOL URL + listener.notify(Collections.singletonList(emptyURL(url))); + // if (isDubboMetadataServiceURL(url)) { + // if meta service change, and serviceInstances is zero, will clean up + // information about this client + // String serviceName = url.getParameter(GROUP_KEY); + // repository.removeMetadataAndInitializedService(serviceName, url); + // } + } + else { + // Notify all + listener.notify(subscribedURLs); + } + } + + private URL emptyURL(URL url) { + // issue : When the last service provider is closed, the client still periodically + // connects to the last provider.n + // fix https://github.com/alibaba/spring-cloud-alibaba/issues/1259 + return from(url).setProtocol(EMPTY_PROTOCOL).removeParameter(CATEGORY_KEY) + .build(); + } + + private final AtomicBoolean inited = new AtomicBoolean(false); + + public void init() { + if (inited.compareAndSet(false, true)) { + doInit(); + } + } + + protected abstract void doInit(); + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/DubboCloudRegistry.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/DubboCloudRegistry.java index 2101a217b..643acdf1a 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/DubboCloudRegistry.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/DubboCloudRegistry.java @@ -16,25 +16,27 @@ package com.alibaba.cloud.dubbo.registry; +import java.util.ArrayList; import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; import java.util.HashSet; -import java.util.LinkedList; import java.util.List; import java.util.Map; -import java.util.Objects; import java.util.Set; -import java.util.function.Supplier; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicBoolean; import java.util.stream.Collectors; +import com.alibaba.cloud.commons.lang.StringUtils; +import com.alibaba.cloud.dubbo.metadata.RevisionResolver; import com.alibaba.cloud.dubbo.metadata.repository.DubboServiceMetadataRepository; import com.alibaba.cloud.dubbo.registry.event.ServiceInstancesChangedEvent; -import com.alibaba.cloud.dubbo.service.DubboGenericServiceFactory; import com.alibaba.cloud.dubbo.service.DubboMetadataService; import com.alibaba.cloud.dubbo.service.DubboMetadataServiceProxy; import com.alibaba.cloud.dubbo.util.DubboMetadataUtils; import com.alibaba.cloud.dubbo.util.JSONUtils; import org.apache.dubbo.common.URL; -import org.apache.dubbo.common.URLBuilder; import org.apache.dubbo.registry.NotifyListener; import org.apache.dubbo.registry.support.FailbackRegistry; import org.slf4j.Logger; @@ -44,25 +46,14 @@ import org.springframework.cloud.client.ServiceInstance; import org.springframework.cloud.client.discovery.DiscoveryClient; import org.springframework.context.ApplicationListener; import org.springframework.context.ConfigurableApplicationContext; -import org.springframework.core.Ordered; -import org.springframework.core.annotation.Order; -import org.springframework.util.CollectionUtils; -import static java.lang.String.format; import static java.util.Collections.emptyList; -import static java.util.stream.StreamSupport.stream; -import static org.apache.dubbo.common.URLBuilder.from; +import static org.apache.dubbo.common.constants.CommonConstants.CONSUMER; import static org.apache.dubbo.common.constants.CommonConstants.GROUP_KEY; -import static org.apache.dubbo.common.constants.CommonConstants.PID_KEY; -import static org.apache.dubbo.common.constants.CommonConstants.PROTOCOL_KEY; import static org.apache.dubbo.common.constants.CommonConstants.PROVIDER; import static org.apache.dubbo.common.constants.CommonConstants.PROVIDER_SIDE; import static org.apache.dubbo.common.constants.CommonConstants.SIDE_KEY; -import static org.apache.dubbo.common.constants.CommonConstants.TIMESTAMP_KEY; -import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY; import static org.apache.dubbo.common.constants.RegistryConstants.CATEGORY_KEY; -import static org.apache.dubbo.common.constants.RegistryConstants.EMPTY_PROTOCOL; -import static org.apache.dubbo.common.utils.CollectionUtils.isEmpty; import static org.apache.dubbo.registry.Constants.ADMIN_PROTOCOL; import static org.apache.dubbo.registry.client.metadata.ServiceInstanceMetadataUtils.METADATA_SERVICE_URLS_PROPERTY_NAME; import static org.springframework.util.StringUtils.hasText; @@ -71,22 +62,14 @@ import static org.springframework.util.StringUtils.hasText; * Dubbo Cloud {@link FailbackRegistry} is based on Spring Cloud {@link DiscoveryClient}. * * @author Mercy + * @author theonefx */ -public class DubboCloudRegistry extends FailbackRegistry { - - /** - * The parameter name of {@link #servicesLookupInterval}. - */ - public static final String SERVICES_LOOKUP_INTERVAL_PARAM_NAME = "dubbo.services.lookup.interval"; +public class DubboCloudRegistry extends FailbackRegistry + implements ApplicationListener { protected static final String DUBBO_METADATA_SERVICE_CLASS_NAME = DubboMetadataService.class .getName(); - /** - * Caches the IDs of {@link ApplicationListener}. - */ - private static final Set REGISTER_LISTENERS = new HashSet<>(); - protected final Logger logger = LoggerFactory.getLogger(getClass()); private final DiscoveryClient discoveryClient; @@ -97,275 +80,362 @@ public class DubboCloudRegistry extends FailbackRegistry { private final JSONUtils jsonUtils; - private final DubboGenericServiceFactory dubboGenericServiceFactory; - private final DubboMetadataUtils dubboMetadataUtils; + private final ConfigurableApplicationContext applicationContext; + + private final ReSubscribeManager reSubscribeManager; + + private final AtomicBoolean inited = new AtomicBoolean(false); + /** - * The interval in second of lookup service names(only for Dubbo-OPS). + * {subscribedURL : ServiceSubscribeHandler}. */ - private final long servicesLookupInterval; + private final Map urlSubscribeHandlerMap = new ConcurrentHashMap<>(); - private final ConfigurableApplicationContext applicationContext; + /** + * {appName: MetadataServiceSubscribeHandler}. + */ + private final Map metadataSubscribeHandlerMap = new ConcurrentHashMap<>(); - private final String currentApplicationName; + /** + * {appName : {revision: [instances]}}. + */ + private final Map>> serviceRevisionInstanceMap = new ConcurrentHashMap<>(); public DubboCloudRegistry(URL url, DiscoveryClient discoveryClient, DubboServiceMetadataRepository repository, DubboMetadataServiceProxy dubboMetadataConfigServiceProxy, - JSONUtils jsonUtils, DubboGenericServiceFactory dubboGenericServiceFactory, - ConfigurableApplicationContext applicationContext) { + JSONUtils jsonUtils, ConfigurableApplicationContext applicationContext) { super(url); - this.servicesLookupInterval = url - .getParameter(SERVICES_LOOKUP_INTERVAL_PARAM_NAME, 60L); this.discoveryClient = discoveryClient; this.repository = repository; this.dubboMetadataConfigServiceProxy = dubboMetadataConfigServiceProxy; this.jsonUtils = jsonUtils; - this.dubboGenericServiceFactory = dubboGenericServiceFactory; this.applicationContext = applicationContext; this.dubboMetadataUtils = getBean(DubboMetadataUtils.class); - this.currentApplicationName = dubboMetadataUtils.getCurrentApplicationName(); + this.reSubscribeManager = new ReSubscribeManager(this); + } + + private void preInit() { + if (inited.compareAndSet(false, true)) { + Set subscribeApps = getServices(null); + + for (String appName : subscribeApps) { + List instances = discoveryClient.getInstances(appName); + + Map> map = serviceRevisionInstanceMap + .computeIfAbsent(appName, k -> new HashMap<>()); + + for (ServiceInstance instance : instances) { + String revision = RevisionResolver.getRevision(instance); + List list = map.computeIfAbsent(revision, + k -> new ArrayList<>()); + list.add(instance); + } + + if (map.size() == 0) { + logger.debug("APP {} preInited, instance siez is zero!!", appName); + } + else { + map.forEach((revision, list) -> logger.debug( + "APP {} revision {} preInited, instance size = {}", appName, + revision, list.size())); + } + } + + metadataSubscribeHandlerMap.forEach((url, handler) -> handler.init()); + urlSubscribeHandlerMap.forEach((url, handler) -> handler.init()); + repository.initializeMetadata(); + + // meke sure everything prepared, then can listening + // ServiceInstanceChangeEvent + applicationContext.addApplicationListener(this); + + logger.info("DubboCloudRegistry preInit Done."); + } } - private T getBean(Class beanClass) { + protected T getBean(Class beanClass) { return this.applicationContext.getBean(beanClass); } - protected boolean shouldRegister(URL url) { + protected boolean shouldNotRegister(URL url) { String side = url.getParameter(SIDE_KEY); boolean should = PROVIDER_SIDE.equals(side); // Only register the Provider. - if (!should) { - if (logger.isDebugEnabled()) { - logger.debug("The URL[{}] should not be registered.", url.toString()); + if (logger.isDebugEnabled()) { + if (!should) { + logger.debug("The URL should NOT!! be registered & unregistered [{}] .", + url); + } + else { + logger.debug("The URL should be registered & unregistered [{}] .", url); } } - return should; + return !should; } @Override public final void doRegister(URL url) { - if (!shouldRegister(url)) { - return; + synchronized (this) { + preInit(); + if (shouldNotRegister(url)) { + return; + } + repository.exportURL(url); } - repository.exportURL(url); } @Override public final void doUnregister(URL url) { - if (!shouldRegister(url)) { - return; + synchronized (this) { + preInit(); + if (shouldNotRegister(url)) { + return; + } + repository.unexportURL(url); } - repository.unexportURL(url); } @Override public final void doSubscribe(URL url, NotifyListener listener) { - - if (isAdminURL(url)) { - // TODO in future - if (logger.isWarnEnabled()) { - logger.warn("This feature about admin will be supported in the future."); + synchronized (this) { + preInit(); + if (isAdminURL(url)) { + // TODO in future + if (logger.isWarnEnabled()) { + logger.warn( + "This feature about admin will be supported in the future."); + } + } + else if (isDubboMetadataServiceURL(url) && containsProviderCategory(url)) { + // for DubboMetadataService + String appName = getServiceName(url); + MetadataServiceSubscribeHandler handler = new MetadataServiceSubscribeHandler( + appName, url, listener, this, dubboMetadataUtils); + if (inited.get()) { + handler.init(); + } + metadataSubscribeHandlerMap.put(appName, handler); + } + else if (isConsumerServiceURL(url)) { + // for general Dubbo Services + GenearalServiceSubscribeHandler handler = new GenearalServiceSubscribeHandler( + url, listener, this, repository, jsonUtils, + dubboMetadataConfigServiceProxy); + if (inited.get()) { + handler.init(); + } + urlSubscribeHandlerMap.put(url, handler); } - } - else if (isDubboMetadataServiceURL(url)) { // for DubboMetadataService - subscribeDubboMetadataServiceURLs(url, listener); - } - else { // for general Dubbo Services - subscribeURLs(url, listener); } } - private void subscribeURLs(URL url, NotifyListener listener) { - - // Sync subscription - subscribeURLs(url, getServices(url), listener); - - // Async subscription - registerServiceInstancesChangedListener(url, - - new ApplicationListener() { + /** + * Process ServiceInstanceChangedEvent, refresh dubbo reference and metadata info. + */ + @Override + public void onApplicationEvent(ServiceInstancesChangedEvent event) { - private final URL url2subscribe = url; + String appName = event.getServiceName(); - @Override - @Order - public void onApplicationEvent(ServiceInstancesChangedEvent event) { - Set serviceNames = getServices(url); + List instances = filter(event.getServiceInstances() != null + ? event.getServiceInstances() : Collections.emptyList()); - String serviceName = event.getServiceName(); + Set subscribedServiceNames = getServices(null); - if (serviceNames.contains(serviceName)) { - subscribeURLs(url, serviceNames, listener); - } - } + if (!subscribedServiceNames.contains(appName)) { + return; + } - @Override - public String toString() { - return "ServiceInstancesChangedEventListener:" - + url.getServiceKey(); - } - }); - } + if (instances.size() == 0) { + logger.warn("APP {} instance changed, size changed zero!!!", appName); + } + else { + logger.info("APP {} instance changed, size changed to {}", appName, + instances.size()); + } + // group by revision + Map> newGroup = instances.stream() + .collect(Collectors.groupingBy(RevisionResolver::getRevision)); - private void subscribeURLs(URL url, Set serviceNames, - NotifyListener listener) { + synchronized (this) { - List subscribedURLs = new LinkedList<>(); + Map> oldGroup = serviceRevisionInstanceMap + .computeIfAbsent(appName, k -> new HashMap<>()); - serviceNames.forEach(serviceName -> { + if (serviceInstanceNotChanged(oldGroup, newGroup)) { + logger.debug("APP {} instance changed, but nothing different", appName); + return; + } - subscribeURLs(url, subscribedURLs, serviceName, - () -> getServiceInstances(serviceName)); + try { - }); + // ensure that the service metadata is correct + refreshServiceMetadataInfo(appName, instances); - // Notify all - notifyAllSubscribedURLs(url, subscribedURLs, listener); - } + // then , refresh general service associated with current application + refreshGeneralServiceInfo(appName, oldGroup, newGroup); - private void registerServiceInstancesChangedListener(URL url, - ApplicationListener listener) { - String listenerId = generateId(url); - if (REGISTER_LISTENERS.add(listenerId)) { - applicationContext.addApplicationListener(listener); + // mark process successful + reSubscribeManager.onRefreshSuccess(event); + } + catch (Exception e) { + logger.error(String.format( + "APP %s instance changed, handler faild, try resubscribe", + appName), e); + reSubscribeManager.onRefreshFail(event); + } } } - private void subscribeURLs(URL subscribedURL, List subscribedURLs, - String serviceName, - Supplier> serviceInstancesSupplier) { - List serviceInstances = serviceInstancesSupplier.get(); - subscribeURLs(subscribedURL, subscribedURLs, serviceName, serviceInstances); - } + private void refreshGeneralServiceInfo(String appName, + Map> oldGroup, + Map> newGroup) { + + Set urls2refresh = new HashSet<>(); - private void subscribeURLs(URL subscribedURL, List subscribedURLs, - String serviceName, List serviceInstances) { + // compare with local + for (String revision : oldGroup.keySet()) { - if (CollectionUtils.isEmpty(serviceInstances)) { - if (logger.isWarnEnabled()) { - logger.warn(format("There is no instance in service[name : %s]", - serviceName)); + if (!newGroup.containsKey(revision)) { + // all instances of this list with revision has losted + urlSubscribeHandlerMap.forEach((url, handler) -> { + if (handler.relatedWith(appName, revision)) { + handler.removeAppNameWithRevision(appName, revision); + urls2refresh.add(url); + } + }); + logger.debug("Subscription app {} revision {} has all losted", appName, + revision); } } - List exportedURLs = getExportedURLs(subscribedURL, serviceName, - serviceInstances); + for (Map.Entry> entry : newGroup.entrySet()) { + String revision = entry.getKey(); + List instanceList = entry.getValue(); - /** - * Add the exported URLs from {@link MetadataService} - */ - subscribedURLs.addAll(exportedURLs); - } - - private List getExportedURLs(URL subscribedURL, String serviceName, - List serviceInstances) { + if (!oldGroup.containsKey(revision)) { + // this instance list of revision not exists + // should acquire urls + urlSubscribeHandlerMap.forEach( + (url, handler) -> handler.init(appName, revision, instanceList)); + } - List validServiceInstances = filter(serviceInstances); + urlSubscribeHandlerMap.forEach((url, handler) -> { + if (handler.relatedWith(appName, revision)) { + urls2refresh.add(url); + } + }); - // If there is no valid ServiceInstance, return empty result - if (isEmpty(validServiceInstances)) { - if (logger.isWarnEnabled()) { - logger.warn( - "There is no instance from service[name : {}], and then Dubbo Service[key : {}] will not be " - + "available , please make sure the further impact", - serviceName, subscribedURL.getServiceKey()); + if (logger.isDebugEnabled()) { + logger.debug("Subscription app {} revision {} changed, instance list {}", + appName, revision, + instanceList.stream().map( + instance -> instance.getHost() + ":" + instance.getPort()) + .collect(Collectors.toList())); } - return emptyList(); } - List subscribedURLs = cloneExportedURLs(subscribedURL, serviceInstances); - - // clear local service instances, help GC - validServiceInstances.clear(); + serviceRevisionInstanceMap.put(appName, newGroup); - return subscribedURLs; + if (urls2refresh.size() == 0) { + logger.debug("Subscription app {}, no urls will be refreshed", appName); + } + else { + logger.debug("Subscription app {}, the following url will be refresh:{}", + appName, urls2refresh.stream().map(URL::getServiceKey) + .collect(Collectors.toList())); + + for (URL url : urls2refresh) { + GenearalServiceSubscribeHandler handler = urlSubscribeHandlerMap.get(url); + if (handler == null) { + logger.warn("Subscription app {}, can't find handler for service {}", + appName, url.getServiceKey()); + continue; + } + handler.refresh(); + } + } } - /** - * Clone the subscribed URLs based on the template URLs. - * @param subscribedURL the URL to be subscribed - * @param serviceInstances the list of {@link ServiceInstance service instances} - * @return non-null - */ - private List cloneExportedURLs(URL subscribedURL, + private void refreshServiceMetadataInfo(String serviceName, List serviceInstances) { + MetadataServiceSubscribeHandler handler = metadataSubscribeHandlerMap + .get(serviceName); - List clonedExportedURLs = new LinkedList<>(); - - serviceInstances.forEach(serviceInstance -> { - - String host = serviceInstance.getHost(); - - getTemplateExportedURLs(subscribedURL, serviceInstances).stream() - .map(templateURL -> templateURL.removeParameter(TIMESTAMP_KEY)) - .map(templateURL -> templateURL.removeParameter(PID_KEY)) - .map(templateURL -> { - String protocol = templateURL.getProtocol(); - Integer port = repository.getDubboProtocolPort(serviceInstance, - protocol); - if (Objects.equals(templateURL.getHost(), host) - && Objects.equals(templateURL.getPort(), port)) { // use - // templateURL - // if - // equals - return templateURL; - } - - if (port == null) { - if (logger.isWarnEnabled()) { - logger.warn( - "The protocol[{}] port of Dubbo service instance[host : {}] " - + "can't be resolved", - protocol, host); - } - return null; - } - else { - URLBuilder clonedURLBuilder = from(templateURL) // remove the - // parameters from - // the template - // URL - .setHost(host) // reset the host - .setPort(port); // reset the port - - return clonedURLBuilder.build(); - } - - }).filter(Objects::nonNull).forEach(clonedExportedURLs::add); - }); - return clonedExportedURLs; + if (handler == null) { + logger.warn("Subscription app {}, can't find metadata handler", serviceName); + return; + } + handler.refresh(serviceInstances); } - private List getTemplateExportedURLs(URL subscribedURL, - List serviceInstances) { + private boolean serviceInstanceNotChanged(Map> oldGroup, + Map> newGroup) { + if (newGroup.size() != oldGroup.size()) { + return false; + } - DubboMetadataService dubboMetadataService = getProxy(serviceInstances); + for (Map.Entry> entry : newGroup.entrySet()) { + String appName = entry.getKey(); + List newInstances = entry.getValue(); - List templateExportedURLs = emptyList(); + if (!oldGroup.containsKey(appName)) { + return false; + } - if (dubboMetadataService != null) { - templateExportedURLs = getExportedURLs(dubboMetadataService, subscribedURL); - } - else { - if (logger.isWarnEnabled()) { - logger.warn( - "The metadata of Dubbo service[key : {}] still can't be found, it could effect the further " - + "Dubbo service invocation", - subscribedURL.getServiceKey()); + List oldInstances = oldGroup.get(appName); + if (newInstances.size() != oldInstances.size()) { + return false; } + boolean matched = newInstances.stream().allMatch(newInstance -> { + + for (ServiceInstance oldInstance : oldInstances) { + if (instanceSame(newInstance, oldInstance)) { + return true; + } + } + + return false; + }); + if (!matched) { + return false; + } } - return templateExportedURLs; + return true; } - private DubboMetadataService getProxy(List serviceInstances) { - return dubboMetadataConfigServiceProxy.getProxy(serviceInstances); + private boolean instanceSame(ServiceInstance newInstance, + ServiceInstance oldInstance) { + if (!StringUtils.equals(newInstance.getInstanceId(), + oldInstance.getInstanceId())) { + return false; + } + if (!StringUtils.equals(newInstance.getHost(), oldInstance.getHost())) { + return false; + } + if (!StringUtils.equals(newInstance.getServiceId(), oldInstance.getServiceId())) { + return false; + } + if (!StringUtils.equals(newInstance.getScheme(), oldInstance.getScheme())) { + return false; + } + if (oldInstance.getPort() != newInstance.getPort()) { + return false; + } + + if (!oldInstance.getMetadata().equals(newInstance.getMetadata())) { + return false; + } + + return true; } private List filter(Collection serviceInstances) { @@ -380,40 +450,14 @@ public class DubboCloudRegistry extends FailbackRegistry { private Set getServices(URL url) { Set subscribedServices = repository.getSubscribedServices(); + if (subscribedServices.contains("*")) { + subscribedServices = new HashSet<>(discoveryClient.getServices()); + } // TODO Add the filter feature return subscribedServices; } - private void notifyAllSubscribedURLs(URL url, List subscribedURLs, - NotifyListener listener) { - - if (isEmpty(subscribedURLs)) { - // Add the EMPTY_PROTOCOL URL - subscribedURLs.add(emptyURL(url)); - - // if (isDubboMetadataServiceURL(url)) { - // if meta service change, and serviceInstances is zero, will clean up - // information about this client - // String serviceName = url.getParameter(GROUP_KEY); - // repository.removeMetadataAndInitializedService(serviceName, url); - // } - } - - if (logger.isDebugEnabled()) { - logger.debug("The subscribed URL[{}] will notify all URLs : {}", url, - subscribedURLs); - } - - // Notify all - listener.notify(subscribedURLs); - } - - private List getServiceInstances(Iterable serviceNames) { - return stream(serviceNames.spliterator(), false).map(this::getServiceInstances) - .flatMap(Collection::stream).collect(Collectors.toList()); - } - - private List getServiceInstances(String serviceName) { + List getServiceInstances(String serviceName) { return hasText(serviceName) ? doGetServiceInstances(serviceName) : emptyList(); } @@ -430,106 +474,14 @@ public class DubboCloudRegistry extends FailbackRegistry { return serviceInstances; } - private String generateId(URL url) { - return url.toString(); - } - - private URL emptyURL(URL url) { - // issue : When the last service provider is closed, the client still periodically - // connects to the last provider.n - // fix https://github.com/alibaba/spring-cloud-alibaba/issues/1259 - return from(url).setProtocol(EMPTY_PROTOCOL).removeParameter(CATEGORY_KEY) - .build(); - } - - private List getExportedURLs(DubboMetadataService dubboMetadataService, - URL subscribedURL) { - String serviceInterface = subscribedURL.getServiceInterface(); - String group = subscribedURL.getParameter(GROUP_KEY); - String version = subscribedURL.getParameter(VERSION_KEY); - // The subscribed protocol may be null - String subscribedProtocol = subscribedURL.getParameter(PROTOCOL_KEY); - String exportedURLsJSON = dubboMetadataService.getExportedURLs(serviceInterface, - group, version); - return jsonUtils.toURLs(exportedURLsJSON).stream() - .filter(exportedURL -> subscribedProtocol == null - || subscribedProtocol.equalsIgnoreCase(exportedURL.getProtocol())) - .collect(Collectors.toList()); - } - - private void subscribeDubboMetadataServiceURLs(URL subscribedURL, - NotifyListener listener) { - - // Sync subscription - subscribeDubboMetadataServiceURLs(subscribedURL, listener, - getServiceName(subscribedURL)); - - // Sync subscription - if (containsProviderCategory(subscribedURL)) { - registerServiceInstancesChangedListener(subscribedURL, - new ApplicationListener() { - - private final URL url2subscribe = subscribedURL; - - @Override - @Order(Ordered.LOWEST_PRECEDENCE - 1) - public void onApplicationEvent( - ServiceInstancesChangedEvent event) { - String sourceServiceName = event.getServiceName(); - String serviceName = getServiceName(subscribedURL); - - if (Objects.equals(sourceServiceName, serviceName)) { - subscribeDubboMetadataServiceURLs(subscribedURL, listener, - sourceServiceName); - } - } - - @Override - public String toString() { - return "ServiceInstancesChangedEventListener:" - + subscribedURL.getServiceKey(); - } - }); - } - } - + // the group of DubboMetadataService is current application name private String getServiceName(URL subscribedURL) { return subscribedURL.getParameter(GROUP_KEY); } - private void subscribeDubboMetadataServiceURLs(URL subscribedURL, - NotifyListener listener, String serviceName) { - - String serviceInterface = subscribedURL.getServiceInterface(); - String version = subscribedURL.getParameter(VERSION_KEY); - String protocol = subscribedURL.getParameter(PROTOCOL_KEY); - - List serviceInstances = getServiceInstances(serviceName); - - List urls = dubboMetadataUtils.getDubboMetadataServiceURLs(serviceInstances, - serviceInterface, version, protocol); - - notifyAllSubscribedURLs(subscribedURL, urls, listener); - } - - // private void subscribeDubboMetadataServiceURLs(URL subscribedURL, - // NotifyListener listener, Set serviceNames) { - // - // String serviceInterface = subscribedURL.getServiceInterface(); - // String version = subscribedURL.getParameter(VERSION_KEY); - // String protocol = subscribedURL.getParameter(PROTOCOL_KEY); - // - // List serviceInstances = getServiceInstances(serviceNames); - // - // List urls = dubboMetadataUtils.getDubboMetadataServiceURLs(serviceInstances, - // serviceInterface, version, protocol); - // - // notifyAllSubscribedURLs(subscribedURL, urls, listener); - // } - private boolean containsProviderCategory(URL subscribedURL) { String category = subscribedURL.getParameter(CATEGORY_KEY); - return category == null ? false : category.contains(PROVIDER); + return category != null && category.contains(PROVIDER); } @Override @@ -550,4 +502,32 @@ public class DubboCloudRegistry extends FailbackRegistry { return DUBBO_METADATA_SERVICE_CLASS_NAME.equals(url.getServiceInterface()); } + protected boolean isConsumerServiceURL(URL url) { + return CONSUMER.equals(url.getProtocol()); + } + + public List getServiceInstances(Map> providers) { + List instances = new ArrayList<>(); + + providers.forEach((appName, revisions) -> { + Map> revisionMap = serviceRevisionInstanceMap + .get(appName); + if (revisionMap == null) { + return; + } + for (String revision : revisions) { + List list = revisionMap.get(revision); + if (list != null) { + instances.addAll(list); + } + } + }); + + return instances; + } + + public Map>> getServiceRevisionInstanceMap() { + return serviceRevisionInstanceMap; + } + } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/GenearalServiceSubscribeHandler.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/GenearalServiceSubscribeHandler.java new file mode 100644 index 000000000..9641c5ce1 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/GenearalServiceSubscribeHandler.java @@ -0,0 +1,279 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.dubbo.registry; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import com.alibaba.cloud.dubbo.metadata.RevisionResolver; +import com.alibaba.cloud.dubbo.metadata.repository.DubboServiceMetadataRepository; +import com.alibaba.cloud.dubbo.service.DubboMetadataService; +import com.alibaba.cloud.dubbo.service.DubboMetadataServiceProxy; +import com.alibaba.cloud.dubbo.util.JSONUtils; +import org.apache.dubbo.common.URL; +import org.apache.dubbo.common.URLBuilder; +import org.apache.dubbo.registry.NotifyListener; +import org.apache.dubbo.rpc.RpcContext; + +import org.springframework.cloud.client.ServiceInstance; + +import static com.alibaba.cloud.dubbo.metadata.RevisionResolver.SCA_REVSION_KEY; +import static java.util.Collections.emptyList; +import static org.apache.dubbo.common.URLBuilder.from; +import static org.apache.dubbo.common.constants.CommonConstants.GROUP_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.PID_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.PROTOCOL_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.TIMESTAMP_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY; + +/** + * @author theonefx + */ +public class GenearalServiceSubscribeHandler extends AbstractServiceSubscribeHandler { + + /** + * the provider which can provide service of the url. {appName, [revisions]} + */ + private final Map> providers = new HashMap<>(); + + private final Map urlTemplateMap = new HashMap<>(); + + private final JSONUtils jsonUtils; + + private final DubboServiceMetadataRepository repository; + + private final DubboMetadataServiceProxy dubboMetadataConfigServiceProxy; + + public GenearalServiceSubscribeHandler(URL url, NotifyListener listener, + DubboCloudRegistry registry, DubboServiceMetadataRepository repository, + JSONUtils jsonUtils, + DubboMetadataServiceProxy dubboMetadataConfigServiceProxy) { + super(url, listener, registry); + this.repository = repository; + this.jsonUtils = jsonUtils; + this.dubboMetadataConfigServiceProxy = dubboMetadataConfigServiceProxy; + } + + public boolean relatedWith(String appName, String revision) { + Set list = providers.get(appName); + if (list != null && list.size() > 0) { + if (list.contains(revision)) { + return true; + } + } + return false; + } + + public void removeAppNameWithRevision(String appName, String revision) { + Set list = providers.get(appName); + if (list != null) { + list.remove(revision); + if (list.size() == 0) { + providers.remove(appName); + } + } + } + + public void addAppNameWithRevision(String appName, String revision) { + Set set = providers.computeIfAbsent(appName, k -> new HashSet<>()); + set.add(revision); + } + + public synchronized void doInit() { + logger.debug("Subscription interface {}, GenearalServiceSubscribeHandler init", + url.getServiceKey()); + Map>> map = registry + .getServiceRevisionInstanceMap(); + for (Map.Entry>> entry : map + .entrySet()) { + String appName = entry.getKey(); + Map> revisionMap = entry.getValue(); + + for (Map.Entry> revisionEntity : revisionMap + .entrySet()) { + String revision = revisionEntity.getKey(); + List instances = revisionEntity.getValue(); + init(appName, revision, instances); + } + } + refresh(); + } + + public void init(String appName, String revision, + List instanceList) { + List urls = getTemplateExportedURLs(url, revision, instanceList); + if (urls != null && urls.size() > 0) { + addAppNameWithRevision(appName, revision); + setUrlTemplate(appName, revision, urls); + } + } + + public synchronized void refresh() { + List urls = getProviderURLs(); + notifyAllSubscribedURLs(url, urls, listener); + } + + private List getProviderURLs() { + List instances = registry.getServiceInstances(providers); + + logger.debug("Subscription interfece {}, providers {}, total {}", + url.getServiceKey(), providers, instances.size()); + + if (instances.size() == 0) { + return Collections.emptyList(); + } + + return cloneExportedURLs(instances); + } + + void setUrlTemplate(String appName, String revision, List urls) { + if (urls == null || urls.size() == 0) { + return; + } + String key = getAppRevisionKey(appName, revision); + if (urlTemplateMap.containsKey(key)) { + return; + } + urlTemplateMap.put(key, urls.get(0)); + } + + private String getAppRevisionKey(String appName, String revision) { + return appName + "@" + revision; + } + + /** + * Clone the subscribed URLs based on the template URLs. + * @param serviceInstances the list of + * {@link org.springframework.cloud.client.ServiceInstance service instances} + */ + List cloneExportedURLs(List serviceInstances) { + + List urlsCloneTo = new ArrayList<>(); + serviceInstances.forEach(serviceInstance -> { + + String host = serviceInstance.getHost(); + String appName = serviceInstance.getServiceId(); + String revision = RevisionResolver.getRevision(serviceInstance); + + URL template = urlTemplateMap.get(getAppRevisionKey(appName, revision)); + + Stream.of(template) + .map(templateURL -> templateURL.removeParameter(TIMESTAMP_KEY)) + .map(templateURL -> templateURL.removeParameter(PID_KEY)) + .map(templateURL -> { + String protocol = templateURL.getProtocol(); + Integer port = repository.getDubboProtocolPort(serviceInstance, + protocol); + + // reserve tag + String tag = null; + List urls = jsonUtils.toURLs(serviceInstance.getMetadata() + .get("dubbo.metadata-service.urls")); + if (urls != null && urls.size() > 0) { + Map parameters = urls.get(0).getParameters(); + tag = parameters.get("dubbo.tag"); + } + + if (Objects.equals(templateURL.getHost(), host) + && Objects.equals(templateURL.getPort(), port)) { // use + // templateURL + // if + // equals + return templateURL; + } + + if (port == null) { + if (logger.isWarnEnabled()) { + logger.warn( + "The protocol[{}] port of Dubbo service instance[host : {}] " + + "can't be resolved", + protocol, host); + } + return null; + } + else { + URLBuilder clonedURLBuilder = from(templateURL) // remove the + // parameters from + // the template + // URL + .setHost(host) // reset the host + .setPort(port) // reset the port + .addParameter("dubbo.tag", tag); // reset the tag + + return clonedURLBuilder.build(); + } + + }).filter(Objects::nonNull).forEach(urlsCloneTo::add); + }); + return urlsCloneTo; + } + + private List getTemplateExportedURLs(URL subscribedURL, String revision, + List serviceInstances) { + + DubboMetadataService dubboMetadataService = getProxy(serviceInstances); + + List templateExportedURLs = emptyList(); + + if (dubboMetadataService != null) { + templateExportedURLs = getExportedURLs(dubboMetadataService, revision, + subscribedURL); + } + else { + if (logger.isWarnEnabled()) { + logger.warn( + "The metadata of Dubbo service[key : {}] still can't be found, it could effect the further " + + "Dubbo service invocation", + subscribedURL.getServiceKey()); + } + + } + + return templateExportedURLs; + } + + private DubboMetadataService getProxy(List serviceInstances) { + return dubboMetadataConfigServiceProxy.getProxy(serviceInstances); + } + + private List getExportedURLs(DubboMetadataService dubboMetadataService, + String revision, URL subscribedURL) { + String serviceInterface = subscribedURL.getServiceInterface(); + String group = subscribedURL.getParameter(GROUP_KEY); + String version = subscribedURL.getParameter(VERSION_KEY); + + RpcContext.getContext().setAttachment(SCA_REVSION_KEY, revision); + String exportedURLsJSON = dubboMetadataService.getExportedURLs(serviceInterface, + group, version); + + // The subscribed protocol may be null + String subscribedProtocol = subscribedURL.getParameter(PROTOCOL_KEY); + return jsonUtils.toURLs(exportedURLsJSON).stream() + .filter(exportedURL -> subscribedProtocol == null + || subscribedProtocol.equalsIgnoreCase(exportedURL.getProtocol())) + .collect(Collectors.toList()); + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/MetadataServiceSubscribeHandler.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/MetadataServiceSubscribeHandler.java new file mode 100644 index 000000000..014131cd1 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/MetadataServiceSubscribeHandler.java @@ -0,0 +1,76 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.dubbo.registry; + +import java.util.List; + +import com.alibaba.cloud.dubbo.util.DubboMetadataUtils; +import org.apache.dubbo.common.URL; +import org.apache.dubbo.registry.NotifyListener; + +import org.springframework.cloud.client.ServiceInstance; + +import static org.apache.dubbo.common.constants.CommonConstants.PROTOCOL_KEY; +import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY; + +/** + * @author theonefx + */ +public class MetadataServiceSubscribeHandler extends AbstractServiceSubscribeHandler { + + private final String appName; + + private final DubboMetadataUtils dubboMetadataUtils; + + public MetadataServiceSubscribeHandler(String appName, URL url, + NotifyListener listener, DubboCloudRegistry registry, + DubboMetadataUtils dubboMetadataUtils) { + super(url, listener, registry); + this.appName = appName; + this.dubboMetadataUtils = dubboMetadataUtils; + } + + @Override + public void doInit() { + logger.debug("Subscription app {} MetadataService handler init", appName); + List serviceInstances = registry.getServiceInstances(appName); + subscribeDubboMetadataServiceURLs(url, listener, serviceInstances); + } + + public void refresh(List serviceInstances) { + logger.debug("Subscription app {}, instance changed, new size = {}", appName, + serviceInstances.size()); + subscribeDubboMetadataServiceURLs(url, listener, serviceInstances); + } + + private void subscribeDubboMetadataServiceURLs(URL subscribedURL, + NotifyListener listener, List serviceInstances) { + + logger.debug("Subscription app {}, service instance changed to size {}", appName, + serviceInstances.size()); + + String serviceInterface = subscribedURL.getServiceInterface(); + String version = subscribedURL.getParameter(VERSION_KEY); + String protocol = subscribedURL.getParameter(PROTOCOL_KEY); + + List urls = dubboMetadataUtils.getDubboMetadataServiceURLs(serviceInstances, + serviceInterface, version, protocol); + + notifyAllSubscribedURLs(subscribedURL, urls, listener); + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/ReSubscribeManager.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/ReSubscribeManager.java new file mode 100644 index 000000000..5de52dbf6 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/ReSubscribeManager.java @@ -0,0 +1,125 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.dubbo.registry; + +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ScheduledThreadPoolExecutor; +import java.util.concurrent.TimeUnit; + +import com.alibaba.cloud.dubbo.env.DubboCloudProperties; +import com.alibaba.cloud.dubbo.registry.event.ServiceInstancesChangedEvent; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.cloud.client.ServiceInstance; + +/** + * @author theonefx + */ +public class ReSubscribeManager { + + private final Logger logger = LoggerFactory.getLogger(ReSubscribeManager.class); + + private final Map reConnectJobMap = new ConcurrentHashMap<>(); + + private final ScheduledThreadPoolExecutor reConnectPool = new ScheduledThreadPoolExecutor( + 5); + + private final DubboCloudRegistry registry; + + private final DubboCloudProperties properties; + + public ReSubscribeManager(DubboCloudRegistry registry) { + this.registry = registry; + this.properties = registry.getBean(DubboCloudProperties.class); + + reConnectPool.setKeepAliveTime(10, TimeUnit.MINUTES); + reConnectPool.allowCoreThreadTimeOut(true); + } + + public void onRefreshSuccess(ServiceInstancesChangedEvent event) { + reConnectJobMap.remove(event.getServiceName()); + } + + public void onRefreshFail(ServiceInstancesChangedEvent event) { + String serviceName = event.getServiceName(); + + int count = 1; + + if (event instanceof FakeServiceInstancesChangedEvent) { + count = ((FakeServiceInstancesChangedEvent) event).getCount() + 1; + } + + if (count >= properties.getMaxReSubscribeMetadataTimes()) { + logger.error( + "reSubscribe failed too many times, serviceName = {}, count = {}", + serviceName, count); + return; + } + + ReSubscribeMetadataJob job = new ReSubscribeMetadataJob(serviceName, count); + reConnectPool.schedule(job, properties.getReSubscribeMetadataIntervial(), + TimeUnit.SECONDS); + } + + private final class ReSubscribeMetadataJob implements Runnable { + + private final String serviceName; + + private final int errorCounts; + + private ReSubscribeMetadataJob(String serviceName, int errorCounts) { + this.errorCounts = errorCounts; + this.serviceName = serviceName; + } + + @Override + public void run() { + if (!reConnectJobMap.containsKey(serviceName) + || reConnectJobMap.get(serviceName) != this) { + return; + } + List list = registry.getServiceInstances(serviceName); + FakeServiceInstancesChangedEvent event = new FakeServiceInstancesChangedEvent( + serviceName, list, errorCounts); + registry.onApplicationEvent(event); + } + + } + + private static final class FakeServiceInstancesChangedEvent + extends ServiceInstancesChangedEvent { + + private static final long serialVersionUID = -2832478604601472915L; + + private final int count; + + private FakeServiceInstancesChangedEvent(String serviceName, + List serviceInstances, int count) { + super(serviceName, serviceInstances); + this.count = count; + } + + public int getCount() { + return count; + } + + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/ServiceInstanceChangeListener.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/ServiceInstanceChangeListener.java new file mode 100644 index 000000000..123214831 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/ServiceInstanceChangeListener.java @@ -0,0 +1,35 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.dubbo.registry; + +import com.alibaba.cloud.dubbo.registry.event.ServiceInstancesChangedEvent; + +import org.springframework.context.ApplicationListener; +import org.springframework.core.Ordered; + +/** + * The interface of ServiceInstanceChange event Listener. + * + * @author theonefx + * @see ServiceInstancesChangedEvent + * @see Ordered + * @see ApplicationListener + */ +public interface ServiceInstanceChangeListener + extends ApplicationListener, Ordered { + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/SpringCloudRegistryFactory.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/SpringCloudRegistryFactory.java index f8b7896dc..2b3b676a3 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/SpringCloudRegistryFactory.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/registry/SpringCloudRegistryFactory.java @@ -89,7 +89,7 @@ public class SpringCloudRegistryFactory extends AbstractRegistryFactory { DubboCloudProperties dubboCloudProperties = applicationContext .getBean(DubboCloudProperties.class); - Registry registry = null; + Registry registry; switch (dubboCloudProperties.getRegistryType()) { case SPRING_CLOUD_REGISTRY_PROPERTY_VALUE: @@ -100,7 +100,7 @@ public class SpringCloudRegistryFactory extends AbstractRegistryFactory { default: registry = new DubboCloudRegistry(url, discoveryClient, dubboServiceMetadataRepository, dubboMetadataConfigServiceProxy, - jsonUtils, dubboGenericServiceFactory, applicationContext); + jsonUtils, applicationContext); break; } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/DubboGenericServiceFactory.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/DubboGenericServiceFactory.java index d5f1ddeef..7e6f5e5e0 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/DubboGenericServiceFactory.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/DubboGenericServiceFactory.java @@ -78,6 +78,9 @@ public class DubboGenericServiceFactory { String interfaceName = serviceClass.getName(); ReferenceBean referenceBean = build(interfaceName, version, serviceName, emptyMap()); + if (DubboMetadataService.class == serviceClass) { + referenceBean.setRouter("-default,revisionRouter"); + } return referenceBean.get(); } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/DubboMetadataServiceProxy.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/DubboMetadataServiceProxy.java index b6f65fa67..c653c52ec 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/DubboMetadataServiceProxy.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/DubboMetadataServiceProxy.java @@ -125,6 +125,7 @@ public class DubboMetadataServiceProxy implements BeanClassLoaderAware, Disposab private DubboMetadataService createProxyIfAbsent(URL dubboMetadataServiceURL) { String serviceName = dubboMetadataServiceURL.getParameter(APPLICATION_KEY); String version = dubboMetadataServiceURL.getParameter(VERSION_KEY); + // Initialize DubboMetadataService with right version return createProxyIfAbsent(serviceName, version); } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/MetadataServiceRevisionRouterFactory.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/MetadataServiceRevisionRouterFactory.java new file mode 100644 index 000000000..9d47c53b7 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/service/MetadataServiceRevisionRouterFactory.java @@ -0,0 +1,75 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.dubbo.service; + +import java.util.ArrayList; +import java.util.List; + +import com.alibaba.cloud.commons.lang.StringUtils; +import org.apache.dubbo.common.URL; +import org.apache.dubbo.rpc.Invocation; +import org.apache.dubbo.rpc.Invoker; +import org.apache.dubbo.rpc.RpcException; +import org.apache.dubbo.rpc.cluster.Router; +import org.apache.dubbo.rpc.cluster.RouterFactory; +import org.apache.dubbo.rpc.cluster.router.AbstractRouter; + +import org.springframework.util.CollectionUtils; + +import static com.alibaba.cloud.dubbo.metadata.RevisionResolver.SCA_REVSION_KEY; + +/** + * @author theonefx + */ +public class MetadataServiceRevisionRouterFactory implements RouterFactory { + + @Override + public Router getRouter(URL url) { + return new AbstractRouter() { + @Override + public List> route(List> invokers, URL url, + Invocation invocation) throws RpcException { + if (CollectionUtils.isEmpty(invokers)) { + return invokers; + } + + if (!DubboMetadataService.class.getName() + .equalsIgnoreCase(url.getServiceInterface())) { + return invokers; + } + + String revision = invocation.getAttachment(SCA_REVSION_KEY); + + if (StringUtils.isEmpty(revision)) { + return invokers; + } + + List> list = new ArrayList<>(invokers.size()); + + for (Invoker invoker : invokers) { + if (StringUtils.equals(revision, + invoker.getUrl().getParameter(SCA_REVSION_KEY))) { + list.add(invoker); + } + } + + return list; + } + }; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/util/DubboMetadataUtils.java b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/util/DubboMetadataUtils.java index af3c59460..acb4c2a14 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/util/DubboMetadataUtils.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/java/com/alibaba/cloud/dubbo/util/DubboMetadataUtils.java @@ -21,12 +21,14 @@ import java.util.Map; import java.util.Objects; import java.util.stream.Collectors; +import com.alibaba.cloud.dubbo.metadata.RevisionResolver; import com.alibaba.cloud.dubbo.service.DubboMetadataService; import org.apache.dubbo.common.URL; import org.springframework.cloud.client.ServiceInstance; import org.springframework.core.env.Environment; +import static com.alibaba.cloud.dubbo.metadata.RevisionResolver.SCA_REVSION_KEY; import static java.lang.String.format; import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY; import static org.apache.dubbo.registry.client.metadata.ServiceInstanceMetadataUtils.METADATA_SERVICE_URLS_PROPERTY_NAME; @@ -74,7 +76,11 @@ public class DubboMetadataUtils { public List getDubboMetadataServiceURLs(ServiceInstance serviceInstance) { Map metadata = serviceInstance.getMetadata(); String dubboURLsJSON = metadata.get(METADATA_SERVICE_URLS_PROPERTY_NAME); - return jsonUtils.toURLs(dubboURLsJSON); + List urls = jsonUtils.toURLs(dubboURLsJSON); + String revision = RevisionResolver.getRevision(serviceInstance); + urls = urls.stream().map(url -> url.addParameter(SCA_REVSION_KEY, revision)) + .collect(Collectors.toList()); + return urls; } /** diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/resources/META-INF/dubbo/org.apache.dubbo.rpc.cluster.RouterFactory b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/resources/META-INF/dubbo/org.apache.dubbo.rpc.cluster.RouterFactory new file mode 100644 index 000000000..f56d67e6d --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-dubbo/src/main/resources/META-INF/dubbo/org.apache.dubbo.rpc.cluster.RouterFactory @@ -0,0 +1 @@ +revisionRouter=com.alibaba.cloud.dubbo.service.MetadataServiceRevisionRouterFactory \ No newline at end of file diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/pom.xml b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/pom.xml index 38be941fa..6af5a64a8 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/pom.xml +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/pom.xml @@ -50,9 +50,12 @@ org.apache.rocketmq - rocketmq-spring-boot-starter + rocketmq-client + + + org.apache.rocketmq + rocketmq-acl - org.springframework.boot spring-boot-starter-test diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQBinderConstants.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQBinderConstants.java deleted file mode 100644 index 47e4b9e7f..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQBinderConstants.java +++ /dev/null @@ -1,51 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq; - -import static org.apache.rocketmq.spring.support.RocketMQHeaders.PREFIX; - -/** - * @author Jim - * @author Xiejiashuai - */ -public final class RocketMQBinderConstants { - - /** - * Header key for RocketMQ Transactional Args. - */ - public static final String ROCKET_TRANSACTIONAL_ARG = "TRANSACTIONAL_ARG"; - - /** - * Default NameServer value. - */ - public static final String DEFAULT_NAME_SERVER = "127.0.0.1:9876"; - - /** - * Default group for SCS RocketMQ Binder. - */ - public static final String DEFAULT_GROUP = PREFIX + "binder_default_group_name"; - - /** - * RocketMQ re-consume times. - */ - public static final String ROCKETMQ_RECONSUME_TIMES = PREFIX + "RECONSUME_TIMES"; - - private RocketMQBinderConstants() { - throw new AssertionError("Must not instantiate constant utility class"); - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQBinderUtils.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQBinderUtils.java deleted file mode 100644 index dfb2e4ec8..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQBinderUtils.java +++ /dev/null @@ -1,89 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq; - -import java.util.Arrays; -import java.util.List; - -import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQBinderConfigurationProperties; -import org.apache.commons.lang3.StringUtils; -import org.apache.rocketmq.spring.autoconfigure.RocketMQProperties; - -import org.springframework.util.CollectionUtils; - -/** - * @author Jim - */ -public final class RocketMQBinderUtils { - - private RocketMQBinderUtils() { - - } - - public static RocketMQBinderConfigurationProperties mergeProperties( - RocketMQBinderConfigurationProperties rocketBinderConfigurationProperties, - RocketMQProperties rocketMQProperties) { - RocketMQBinderConfigurationProperties result = new RocketMQBinderConfigurationProperties(); - if (StringUtils.isEmpty(rocketMQProperties.getNameServer())) { - result.setNameServer(rocketBinderConfigurationProperties.getNameServer()); - } - else { - result.setNameServer( - Arrays.asList(rocketMQProperties.getNameServer().split(";"))); - } - if (rocketMQProperties.getProducer() == null - || StringUtils.isEmpty(rocketMQProperties.getProducer().getAccessKey())) { - result.setAccessKey(rocketBinderConfigurationProperties.getAccessKey()); - } - else { - result.setAccessKey(rocketMQProperties.getProducer().getAccessKey()); - } - if (rocketMQProperties.getProducer() == null - || StringUtils.isEmpty(rocketMQProperties.getProducer().getSecretKey())) { - result.setSecretKey(rocketBinderConfigurationProperties.getSecretKey()); - } - else { - result.setSecretKey(rocketMQProperties.getProducer().getSecretKey()); - } - if (rocketMQProperties.getProducer() == null || StringUtils - .isEmpty(rocketMQProperties.getProducer().getCustomizedTraceTopic())) { - result.setCustomizedTraceTopic( - rocketBinderConfigurationProperties.getCustomizedTraceTopic()); - } - else { - result.setCustomizedTraceTopic( - rocketMQProperties.getProducer().getCustomizedTraceTopic()); - } - if (rocketMQProperties.getProducer() != null - && rocketMQProperties.getProducer().isEnableMsgTrace()) { - result.setEnableMsgTrace(Boolean.TRUE); - } - else { - result.setEnableMsgTrace( - rocketBinderConfigurationProperties.isEnableMsgTrace()); - } - return result; - } - - public static String getNameServerStr(List nameServerList) { - if (CollectionUtils.isEmpty(nameServerList)) { - return null; - } - return String.join(";", nameServerList); - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQMessageChannelBinder.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQMessageChannelBinder.java index a83f43cd6..7e1cea97e 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQMessageChannelBinder.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQMessageChannelBinder.java @@ -16,34 +16,18 @@ package com.alibaba.cloud.stream.binder.rocketmq; -import java.util.Collection; -import java.util.Collections; -import java.util.HashMap; -import java.util.Map; -import java.util.Set; - -import com.alibaba.cloud.stream.binder.rocketmq.consuming.RocketMQListenerBindingContainer; -import com.alibaba.cloud.stream.binder.rocketmq.integration.RocketMQInboundChannelAdapter; -import com.alibaba.cloud.stream.binder.rocketmq.integration.RocketMQMessageHandler; -import com.alibaba.cloud.stream.binder.rocketmq.integration.RocketMQMessageSource; -import com.alibaba.cloud.stream.binder.rocketmq.metrics.InstrumentationManager; +import com.alibaba.cloud.stream.binder.rocketmq.custom.RocketMQBeanContainerCache; +import com.alibaba.cloud.stream.binder.rocketmq.extend.ErrorAcknowledgeHandler; +import com.alibaba.cloud.stream.binder.rocketmq.integration.inbound.RocketMQInboundChannelAdapter; +import com.alibaba.cloud.stream.binder.rocketmq.integration.inbound.pull.DefaultErrorAcknowledgeHandler; +import com.alibaba.cloud.stream.binder.rocketmq.integration.inbound.pull.RocketMQMessageSource; +import com.alibaba.cloud.stream.binder.rocketmq.integration.outbound.RocketMQProducerMessageHandler; import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQBinderConfigurationProperties; import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties; import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQExtendedBindingProperties; import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties; import com.alibaba.cloud.stream.binder.rocketmq.provisioning.RocketMQTopicProvisioner; -import com.alibaba.cloud.stream.binder.rocketmq.provisioning.selector.PartitionMessageQueueSelector; -import com.alibaba.cloud.stream.binder.rocketmq.support.JacksonRocketMQHeaderMapper; -import com.alibaba.cloud.stream.binder.rocketmq.support.RocketMQHeaderMapper; -import com.fasterxml.jackson.databind.ObjectMapper; -import org.apache.rocketmq.acl.common.AclClientRPCHook; -import org.apache.rocketmq.acl.common.SessionCredentials; -import org.apache.rocketmq.client.producer.DefaultMQProducer; -import org.apache.rocketmq.common.UtilAll; -import org.apache.rocketmq.remoting.RPCHook; -import org.apache.rocketmq.spring.autoconfigure.RocketMQProperties; -import org.apache.rocketmq.spring.core.RocketMQTemplate; -import org.apache.rocketmq.spring.support.RocketMQUtil; +import com.alibaba.cloud.stream.binder.rocketmq.utils.RocketMQUtils; import org.springframework.cloud.stream.binder.AbstractMessageChannelBinder; import org.springframework.cloud.stream.binder.BinderSpecificPropertiesProvider; @@ -55,16 +39,19 @@ import org.springframework.cloud.stream.provisioning.ConsumerDestination; import org.springframework.cloud.stream.provisioning.ProducerDestination; import org.springframework.integration.StaticMessageHeaderAccessor; import org.springframework.integration.acks.AcknowledgmentCallback; -import org.springframework.integration.acks.AcknowledgmentCallback.Status; import org.springframework.integration.channel.AbstractMessageChannel; import org.springframework.integration.core.MessageProducer; +import org.springframework.integration.support.DefaultErrorMessageStrategy; +import org.springframework.integration.support.ErrorMessageStrategy; import org.springframework.messaging.MessageChannel; import org.springframework.messaging.MessageHandler; import org.springframework.messaging.MessagingException; -import org.springframework.util.CollectionUtils; import org.springframework.util.StringUtils; /** + * A {@link org.springframework.cloud.stream.binder.Binder} that uses RocketMQ as the + * underlying middleware. + * * @author Jim */ public class RocketMQMessageChannelBinder extends @@ -72,120 +59,45 @@ public class RocketMQMessageChannelBinder extends implements ExtendedPropertiesBinder { - private RocketMQExtendedBindingProperties extendedBindingProperties = new RocketMQExtendedBindingProperties(); - - private final RocketMQBinderConfigurationProperties rocketBinderConfigurationProperties; - - private final RocketMQProperties rocketMQProperties; - - private final InstrumentationManager instrumentationManager; + private final RocketMQExtendedBindingProperties extendedBindingProperties; - private Map topicInUse = new HashMap<>(); + private final RocketMQBinderConfigurationProperties binderConfigurationProperties; - public RocketMQMessageChannelBinder(RocketMQTopicProvisioner provisioningProvider, + public RocketMQMessageChannelBinder( + RocketMQBinderConfigurationProperties binderConfigurationProperties, RocketMQExtendedBindingProperties extendedBindingProperties, - RocketMQBinderConfigurationProperties rocketBinderConfigurationProperties, - RocketMQProperties rocketMQProperties, - InstrumentationManager instrumentationManager) { - super(null, provisioningProvider); + RocketMQTopicProvisioner provisioningProvider) { + super(new String[0], provisioningProvider); this.extendedBindingProperties = extendedBindingProperties; - this.rocketBinderConfigurationProperties = rocketBinderConfigurationProperties; - this.rocketMQProperties = rocketMQProperties; - this.instrumentationManager = instrumentationManager; + this.binderConfigurationProperties = binderConfigurationProperties; } @Override protected MessageHandler createProducerMessageHandler(ProducerDestination destination, - ExtendedProducerProperties producerProperties, + ExtendedProducerProperties extendedProducerProperties, MessageChannel channel, MessageChannel errorChannel) throws Exception { - if (producerProperties.getExtension().getEnabled()) { - - // if producerGroup is empty, using destination - String extendedProducerGroup = producerProperties.getExtension().getGroup(); - String producerGroup = StringUtils.hasLength(extendedProducerGroup) - ? extendedProducerGroup : destination.getName(); - - RocketMQBinderConfigurationProperties mergedProperties = RocketMQBinderUtils - .mergeProperties(rocketBinderConfigurationProperties, - rocketMQProperties); - - RocketMQTemplate rocketMQTemplate; - if (producerProperties.getExtension().getTransactional()) { - Map rocketMQTemplates = getBeanFactory() - .getBeansOfType(RocketMQTemplate.class); - if (rocketMQTemplates.size() == 0) { - throw new IllegalStateException( - "there is no RocketMQTemplate in Spring BeanFactory"); - } - else if (rocketMQTemplates.size() > 1) { - throw new IllegalStateException( - "there is more than 1 RocketMQTemplates in Spring BeanFactory"); - } - rocketMQTemplate = rocketMQTemplates.values().iterator().next(); - } - else { - rocketMQTemplate = new RocketMQTemplate(); - rocketMQTemplate.setObjectMapper(this.getApplicationContext() - .getBeansOfType(ObjectMapper.class).values().iterator().next()); - DefaultMQProducer producer; - String ak = mergedProperties.getAccessKey(); - String sk = mergedProperties.getSecretKey(); - if (StringUtils.hasLength(ak) && StringUtils.hasLength(sk)) { - RPCHook rpcHook = new AclClientRPCHook( - new SessionCredentials(ak, sk)); - producer = new DefaultMQProducer(producerGroup, rpcHook, - mergedProperties.isEnableMsgTrace(), - mergedProperties.getCustomizedTraceTopic()); - producer.setVipChannelEnabled(false); - producer.setInstanceName(RocketMQUtil.getInstanceName(rpcHook, - destination.getName() + "|" + UtilAll.getPid())); - } - else { - producer = new DefaultMQProducer(producerGroup); - producer.setVipChannelEnabled( - producerProperties.getExtension().getVipChannelEnabled()); - } - producer.setNamesrvAddr(RocketMQBinderUtils - .getNameServerStr(mergedProperties.getNameServer())); - producer.setSendMsgTimeout( - producerProperties.getExtension().getSendMessageTimeout()); - producer.setRetryTimesWhenSendFailed( - producerProperties.getExtension().getRetryTimesWhenSendFailed()); - producer.setRetryTimesWhenSendAsyncFailed(producerProperties - .getExtension().getRetryTimesWhenSendAsyncFailed()); - producer.setCompressMsgBodyOverHowmuch(producerProperties.getExtension() - .getCompressMessageBodyThreshold()); - producer.setRetryAnotherBrokerWhenNotStoreOK( - producerProperties.getExtension().isRetryNextServer()); - producer.setMaxMessageSize( - producerProperties.getExtension().getMaxMessageSize()); - rocketMQTemplate.setProducer(producer); - if (producerProperties.isPartitioned()) { - rocketMQTemplate - .setMessageQueueSelector(new PartitionMessageQueueSelector()); - } - } - - RocketMQMessageHandler messageHandler = new RocketMQMessageHandler( - rocketMQTemplate, destination.getName(), producerGroup, - producerProperties.getExtension().getTransactional(), - instrumentationManager, producerProperties, - ((AbstractMessageChannel) channel).getInterceptors().stream().filter( - channelInterceptor -> channelInterceptor instanceof MessageConverterConfigurer.PartitioningInterceptor) - .map(channelInterceptor -> ((MessageConverterConfigurer.PartitioningInterceptor) channelInterceptor)) - .findFirst().orElse(null)); - messageHandler.setBeanFactory(this.getApplicationContext().getBeanFactory()); - messageHandler.setSync(producerProperties.getExtension().getSync()); - messageHandler.setHeaderMapper(createHeaderMapper(producerProperties)); - if (errorChannel != null) { - messageHandler.setSendFailureChannel(errorChannel); - } - return messageHandler; - } - else { + if (!extendedProducerProperties.getExtension().getEnabled()) { throw new RuntimeException("Binding for channel " + destination.getName() + " has been disabled, message can't be delivered"); } + RocketMQProducerProperties mqProducerProperties = RocketMQUtils + .mergeRocketMQProperties(binderConfigurationProperties, + extendedProducerProperties.getExtension()); + RocketMQProducerMessageHandler messageHandler = new RocketMQProducerMessageHandler( + destination, extendedProducerProperties, mqProducerProperties); + messageHandler.setApplicationContext(this.getApplicationContext()); + if (errorChannel != null) { + messageHandler.setSendFailureChannel(errorChannel); + } + MessageConverterConfigurer.PartitioningInterceptor partitioningInterceptor = ((AbstractMessageChannel) channel) + .getInterceptors().stream() + .filter(channelInterceptor -> channelInterceptor instanceof MessageConverterConfigurer.PartitioningInterceptor) + .map(channelInterceptor -> ((MessageConverterConfigurer.PartitioningInterceptor) channelInterceptor)) + .findFirst().orElse(null); + messageHandler.setPartitioningInterceptor(partitioningInterceptor); + messageHandler.setBeanFactory(this.getApplicationContext().getBeanFactory()); + messageHandler.setErrorMessageStrategy(this.getErrorMessageStrategy()); + return messageHandler; } @Override @@ -199,56 +111,43 @@ public class RocketMQMessageChannelBinder extends @Override protected MessageProducer createConsumerEndpoint(ConsumerDestination destination, String group, - ExtendedConsumerProperties consumerProperties) + ExtendedConsumerProperties extendedConsumerProperties) throws Exception { - if (group == null || "".equals(group)) { + // todo support anymous consumer + if (!StringUtils.hasLength(group)) { throw new RuntimeException( "'group must be configured for channel " + destination.getName()); } + RocketMQUtils.mergeRocketMQProperties(binderConfigurationProperties, + extendedConsumerProperties.getExtension()); + extendedConsumerProperties.getExtension().setGroup(group); - RocketMQListenerBindingContainer listenerContainer = new RocketMQListenerBindingContainer( - consumerProperties, rocketBinderConfigurationProperties, this); - listenerContainer.setConsumerGroup(group); - listenerContainer.setTopic(destination.getName()); - listenerContainer.setConsumeThreadMax(consumerProperties.getConcurrency()); - listenerContainer.setSuspendCurrentQueueTimeMillis( - consumerProperties.getExtension().getSuspendCurrentQueueTimeMillis()); - listenerContainer.setDelayLevelWhenNextConsume( - consumerProperties.getExtension().getDelayLevelWhenNextConsume()); - listenerContainer - .setNameServer(rocketBinderConfigurationProperties.getNameServer()); - listenerContainer.setHeaderMapper(createHeaderMapper(consumerProperties)); - - RocketMQInboundChannelAdapter rocketInboundChannelAdapter = new RocketMQInboundChannelAdapter( - listenerContainer, consumerProperties, instrumentationManager); - - topicInUse.put(destination.getName(), group); - + RocketMQInboundChannelAdapter inboundChannelAdapter = new RocketMQInboundChannelAdapter( + destination.getName(), extendedConsumerProperties); ErrorInfrastructure errorInfrastructure = registerErrorInfrastructure(destination, - group, consumerProperties); - if (consumerProperties.getMaxAttempts() > 1) { - rocketInboundChannelAdapter - .setRetryTemplate(buildRetryTemplate(consumerProperties)); - rocketInboundChannelAdapter - .setRecoveryCallback(errorInfrastructure.getRecoverer()); + group, extendedConsumerProperties); + if (extendedConsumerProperties.getMaxAttempts() > 1) { + inboundChannelAdapter + .setRetryTemplate(buildRetryTemplate(extendedConsumerProperties)); + inboundChannelAdapter.setRecoveryCallback(errorInfrastructure.getRecoverer()); } else { - rocketInboundChannelAdapter - .setErrorChannel(errorInfrastructure.getErrorChannel()); + inboundChannelAdapter.setErrorChannel(errorInfrastructure.getErrorChannel()); } - - return rocketInboundChannelAdapter; + return inboundChannelAdapter; } @Override protected PolledConsumerResources createPolledConsumerResources(String name, String group, ConsumerDestination destination, - ExtendedConsumerProperties consumerProperties) { - RocketMQMessageSource rocketMQMessageSource = new RocketMQMessageSource( - rocketBinderConfigurationProperties, consumerProperties, name, group); - return new PolledConsumerResources(rocketMQMessageSource, - registerErrorInfrastructure(destination, group, consumerProperties, - true)); + ExtendedConsumerProperties extendedConsumerProperties) { + RocketMQUtils.mergeRocketMQProperties(binderConfigurationProperties, + extendedConsumerProperties.getExtension()); + extendedConsumerProperties.getExtension().setGroup(group); + RocketMQMessageSource messageSource = new RocketMQMessageSource(name, + extendedConsumerProperties); + return new PolledConsumerResources(messageSource, registerErrorInfrastructure( + destination, group, extendedConsumerProperties, true)); } @Override @@ -262,67 +161,47 @@ public class RocketMQMessageChannelBinder extends ((MessagingException) message.getPayload()) .getFailedMessage()); if (ack != null) { - if (properties.getExtension().shouldRequeue()) { - ack.acknowledge(Status.REQUEUE); - } - else { - ack.acknowledge(Status.REJECT); - } + ErrorAcknowledgeHandler handler = RocketMQBeanContainerCache.getBean( + properties.getExtension().getPull().getErrAcknowledge(), + ErrorAcknowledgeHandler.class, + new DefaultErrorAcknowledgeHandler()); + ack.acknowledge( + handler.handler(((MessagingException) message.getPayload()) + .getFailedMessage())); } } }; } + /** + * Binders can return an {@link ErrorMessageStrategy} for building error messages; + * binder implementations typically might add extra headers to the error message. + * @return the implementation - may be null. + */ @Override - public RocketMQConsumerProperties getExtendedConsumerProperties(String channelName) { - return extendedBindingProperties.getExtendedConsumerProperties(channelName); + protected ErrorMessageStrategy getErrorMessageStrategy() { + // It can be extended to custom if necessary. + return new DefaultErrorMessageStrategy(); } @Override - public RocketMQProducerProperties getExtendedProducerProperties(String channelName) { - return extendedBindingProperties.getExtendedProducerProperties(channelName); + public RocketMQConsumerProperties getExtendedConsumerProperties(String channelName) { + return this.extendedBindingProperties.getExtendedConsumerProperties(channelName); } - public Map getTopicInUse() { - return topicInUse; + @Override + public RocketMQProducerProperties getExtendedProducerProperties(String channelName) { + return this.extendedBindingProperties.getExtendedProducerProperties(channelName); } @Override public String getDefaultsPrefix() { - return extendedBindingProperties.getDefaultsPrefix(); + return this.extendedBindingProperties.getDefaultsPrefix(); } @Override public Class getExtendedPropertiesEntryClass() { - return extendedBindingProperties.getExtendedPropertiesEntryClass(); - } - - public void setExtendedBindingProperties( - RocketMQExtendedBindingProperties extendedBindingProperties) { - this.extendedBindingProperties = extendedBindingProperties; - } - - private RocketMQHeaderMapper createHeaderMapper( - final ExtendedConsumerProperties extendedConsumerProperties) { - Set trustedPackages = extendedConsumerProperties.getExtension() - .getTrustedPackages(); - return createHeaderMapper(trustedPackages); - } - - private RocketMQHeaderMapper createHeaderMapper( - final ExtendedProducerProperties producerProperties) { - return createHeaderMapper(Collections.emptyList()); - } - - private RocketMQHeaderMapper createHeaderMapper(Collection trustedPackages) { - ObjectMapper objectMapper = this.getApplicationContext() - .getBeansOfType(ObjectMapper.class).values().iterator().next(); - JacksonRocketMQHeaderMapper headerMapper = new JacksonRocketMQHeaderMapper( - objectMapper); - if (!CollectionUtils.isEmpty(trustedPackages)) { - headerMapper.addTrustedPackages(trustedPackages); - } - return headerMapper; + return this.extendedBindingProperties.getExtendedPropertiesEntryClass(); } } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/actuator/RocketMQBinderHealthIndicator.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/actuator/RocketMQBinderHealthIndicator.java index 1c49359ea..6e704250a 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/actuator/RocketMQBinderHealthIndicator.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/actuator/RocketMQBinderHealthIndicator.java @@ -19,7 +19,6 @@ package com.alibaba.cloud.stream.binder.rocketmq.actuator; import com.alibaba.cloud.stream.binder.rocketmq.metrics.Instrumentation; import com.alibaba.cloud.stream.binder.rocketmq.metrics.InstrumentationManager; -import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.actuate.health.AbstractHealthIndicator; import org.springframework.boot.actuate.health.Health; @@ -29,23 +28,20 @@ import org.springframework.boot.actuate.health.Health; */ public class RocketMQBinderHealthIndicator extends AbstractHealthIndicator { - @Autowired - private InstrumentationManager instrumentationManager; - @Override protected void doHealthCheck(Health.Builder builder) throws Exception { - if (instrumentationManager.getHealthInstrumentations().stream() + if (InstrumentationManager.getHealthInstrumentations().stream() .allMatch(Instrumentation::isUp)) { builder.up(); return; } - if (instrumentationManager.getHealthInstrumentations().stream() + if (InstrumentationManager.getHealthInstrumentations().stream() .allMatch(Instrumentation::isOutOfService)) { builder.outOfService(); return; } builder.down(); - instrumentationManager.getHealthInstrumentations().stream() + InstrumentationManager.getHealthInstrumentations().stream() .filter(instrumentation -> !instrumentation.isStarted()) .forEach(instrumentation1 -> builder .withException(instrumentation1.getStartException())); diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/autoconfigurate/ExtendedBindingHandlerMappingsProviderConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/autoconfigurate/ExtendedBindingHandlerMappingsProviderConfiguration.java new file mode 100644 index 000000000..df5d47b6b --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/autoconfigurate/ExtendedBindingHandlerMappingsProviderConfiguration.java @@ -0,0 +1,65 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.autoconfigurate; + +import java.util.HashMap; +import java.util.Map; + +import com.alibaba.cloud.stream.binder.rocketmq.convert.RocketMQMessageConverter; +import com.alibaba.cloud.stream.binder.rocketmq.custom.RocketMQConfigBeanPostProcessor; + +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.boot.context.properties.source.ConfigurationPropertyName; +import org.springframework.cloud.stream.config.BindingHandlerAdvise.MappingsProvider; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.messaging.converter.CompositeMessageConverter; + +@Configuration +public class ExtendedBindingHandlerMappingsProviderConfiguration { + + @Bean + public MappingsProvider rocketExtendedPropertiesDefaultMappingsProvider() { + return () -> { + Map mappings = new HashMap<>(); + mappings.put( + ConfigurationPropertyName.of("spring.cloud.stream.rocketmq.bindings"), + ConfigurationPropertyName.of("spring.cloud.stream.rocketmq.default")); + mappings.put( + ConfigurationPropertyName.of("spring.cloud.stream.rocketmq.streams"), + ConfigurationPropertyName + .of("spring.cloud.stream.rocketmq.streams.default")); + return mappings; + }; + } + + @Bean + public RocketMQConfigBeanPostProcessor rocketMQConfigBeanPostProcessor() { + return new RocketMQConfigBeanPostProcessor(); + } + + + /** + * if you want to customize a bean, please use this BeanName {@code RocketMQMessageConverter.DEFAULT_NAME}. + */ + @Bean(RocketMQMessageConverter.DEFAULT_NAME) + @ConditionalOnMissingBean(name = { RocketMQMessageConverter.DEFAULT_NAME }) + public CompositeMessageConverter rocketMQMessageConverter() { + return new RocketMQMessageConverter().getMessageConverter(); + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/autoconfigurate/RocketMQBinderAutoConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/autoconfigurate/RocketMQBinderAutoConfiguration.java new file mode 100644 index 000000000..b9b85db24 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/autoconfigurate/RocketMQBinderAutoConfiguration.java @@ -0,0 +1,68 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.autoconfigurate; + +import com.alibaba.cloud.stream.binder.rocketmq.RocketMQMessageChannelBinder; +import com.alibaba.cloud.stream.binder.rocketmq.actuator.RocketMQBinderHealthIndicator; +import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQBinderConfigurationProperties; +import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQExtendedBindingProperties; +import com.alibaba.cloud.stream.binder.rocketmq.provisioning.RocketMQTopicProvisioner; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.actuate.autoconfigure.health.ConditionalOnEnabledHealthIndicator; +import org.springframework.boot.autoconfigure.condition.ConditionalOnClass; +import org.springframework.boot.context.properties.EnableConfigurationProperties; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * issue:https://github.com/alibaba/spring-cloud-alibaba/issues/1681 . + * + * @author Timur Valiev + * @author Jim + */ +@Configuration(proxyBeanMethods = false) +@EnableConfigurationProperties({ RocketMQExtendedBindingProperties.class, + RocketMQBinderConfigurationProperties.class }) +public class RocketMQBinderAutoConfiguration { + + @Autowired + private RocketMQExtendedBindingProperties extendedBindingProperties; + + @Autowired + private RocketMQBinderConfigurationProperties rocketBinderConfigurationProperties; + + @Bean + @ConditionalOnEnabledHealthIndicator("rocketmq") + @ConditionalOnClass(name = "org.springframework.boot.actuate.health.HealthIndicator") + public RocketMQBinderHealthIndicator rocketMQBinderHealthIndicator() { + return new RocketMQBinderHealthIndicator(); + } + + @Bean + public RocketMQTopicProvisioner rocketMQTopicProvisioner() { + return new RocketMQTopicProvisioner(); + } + + @Bean + public RocketMQMessageChannelBinder rocketMQMessageChannelBinder( + RocketMQTopicProvisioner provisioningProvider) { + return new RocketMQMessageChannelBinder(rocketBinderConfigurationProperties, + extendedBindingProperties, provisioningProvider); + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/config/RocketMQBinderAutoConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/config/RocketMQBinderAutoConfiguration.java deleted file mode 100644 index 3031c2656..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/config/RocketMQBinderAutoConfiguration.java +++ /dev/null @@ -1,81 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq.config; - -import com.alibaba.cloud.stream.binder.rocketmq.RocketMQMessageChannelBinder; -import com.alibaba.cloud.stream.binder.rocketmq.metrics.InstrumentationManager; -import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQBinderConfigurationProperties; -import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQExtendedBindingProperties; -import com.alibaba.cloud.stream.binder.rocketmq.provisioning.RocketMQTopicProvisioner; -import org.apache.rocketmq.spring.autoconfigure.RocketMQAutoConfiguration; -import org.apache.rocketmq.spring.autoconfigure.RocketMQProperties; - -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; -import org.springframework.context.annotation.Import; - -/** - * @author Timur Valiev - * @author Jim - */ -@Configuration(proxyBeanMethods = false) -@Import({ RocketMQAutoConfiguration.class, - RocketMQBinderHealthIndicatorAutoConfiguration.class }) -@EnableConfigurationProperties({ RocketMQBinderConfigurationProperties.class, - RocketMQExtendedBindingProperties.class }) -public class RocketMQBinderAutoConfiguration { - - private final RocketMQExtendedBindingProperties extendedBindingProperties; - - private final RocketMQBinderConfigurationProperties rocketBinderConfigurationProperties; - - @Autowired(required = false) - private RocketMQProperties rocketMQProperties = new RocketMQProperties(); - - @Autowired - public RocketMQBinderAutoConfiguration( - RocketMQExtendedBindingProperties extendedBindingProperties, - RocketMQBinderConfigurationProperties rocketBinderConfigurationProperties) { - this.extendedBindingProperties = extendedBindingProperties; - this.rocketBinderConfigurationProperties = rocketBinderConfigurationProperties; - } - - @Bean - public RocketMQTopicProvisioner provisioningProvider() { - return new RocketMQTopicProvisioner(); - } - - @Bean - public RocketMQMessageChannelBinder rocketMessageChannelBinder( - RocketMQTopicProvisioner provisioningProvider, - InstrumentationManager instrumentationManager) { - RocketMQMessageChannelBinder binder = new RocketMQMessageChannelBinder( - provisioningProvider, extendedBindingProperties, - rocketBinderConfigurationProperties, rocketMQProperties, - instrumentationManager); - binder.setExtendedBindingProperties(extendedBindingProperties); - return binder; - } - - @Bean - public InstrumentationManager instrumentationManager() { - return new InstrumentationManager(); - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/config/RocketMQBinderHealthIndicatorAutoConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/config/RocketMQBinderHealthIndicatorAutoConfiguration.java deleted file mode 100644 index 1c3b5dc0d..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/config/RocketMQBinderHealthIndicatorAutoConfiguration.java +++ /dev/null @@ -1,40 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq.config; - -import com.alibaba.cloud.stream.binder.rocketmq.actuator.RocketMQBinderHealthIndicator; - -import org.springframework.boot.actuate.autoconfigure.health.ConditionalOnEnabledHealthIndicator; -import org.springframework.boot.actuate.endpoint.annotation.Endpoint; -import org.springframework.boot.autoconfigure.condition.ConditionalOnClass; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; - -/** - * @author Jim - */ -@Configuration(proxyBeanMethods = false) -@ConditionalOnClass(Endpoint.class) -public class RocketMQBinderHealthIndicatorAutoConfiguration { - - @Bean - @ConditionalOnEnabledHealthIndicator("rocketmq") - public RocketMQBinderHealthIndicator rocketBinderHealthIndicator() { - return new RocketMQBinderHealthIndicator(); - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/config/RocketMQComponent4BinderAutoConfiguration.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/config/RocketMQComponent4BinderAutoConfiguration.java deleted file mode 100644 index 624ecde64..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/config/RocketMQComponent4BinderAutoConfiguration.java +++ /dev/null @@ -1,102 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq.config; - -import com.alibaba.cloud.stream.binder.rocketmq.RocketMQBinderConstants; -import com.fasterxml.jackson.databind.ObjectMapper; -import org.apache.rocketmq.acl.common.AclClientRPCHook; -import org.apache.rocketmq.acl.common.SessionCredentials; -import org.apache.rocketmq.client.producer.DefaultMQProducer; -import org.apache.rocketmq.spring.autoconfigure.RocketMQAutoConfiguration; -import org.apache.rocketmq.spring.config.RocketMQConfigUtils; -import org.apache.rocketmq.spring.config.RocketMQTransactionAnnotationProcessor; -import org.apache.rocketmq.spring.config.TransactionHandlerRegistry; -import org.apache.rocketmq.spring.core.RocketMQTemplate; - -import org.springframework.boot.autoconfigure.AutoConfigureAfter; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; -import org.springframework.core.env.Environment; -import org.springframework.util.StringUtils; - -/** - * @author Jim - */ -@Configuration(proxyBeanMethods = false) -@AutoConfigureAfter(RocketMQAutoConfiguration.class) -@ConditionalOnMissingBean(DefaultMQProducer.class) -public class RocketMQComponent4BinderAutoConfiguration { - - private final Environment environment; - - public RocketMQComponent4BinderAutoConfiguration(Environment environment) { - this.environment = environment; - } - - @Bean - @ConditionalOnMissingBean(DefaultMQProducer.class) - public DefaultMQProducer defaultMQProducer() { - DefaultMQProducer producer; - String configNameServer = environment.resolveRequiredPlaceholders( - "${spring.cloud.stream.rocketmq.binder.name-server:${rocketmq.producer.name-server:}}"); - String ak = environment.resolveRequiredPlaceholders( - "${spring.cloud.stream.rocketmq.binder.access-key:${rocketmq.producer.access-key:}}"); - String sk = environment.resolveRequiredPlaceholders( - "${spring.cloud.stream.rocketmq.binder.secret-key:${rocketmq.producer.secret-key:}}"); - if (StringUtils.hasLength(ak) && StringUtils.hasLength(sk)) { - producer = new DefaultMQProducer(RocketMQBinderConstants.DEFAULT_GROUP, - new AclClientRPCHook(new SessionCredentials(ak, sk))); - producer.setVipChannelEnabled(false); - } - else { - producer = new DefaultMQProducer(RocketMQBinderConstants.DEFAULT_GROUP); - } - if (!StringUtils.hasLength(configNameServer)) { - configNameServer = RocketMQBinderConstants.DEFAULT_NAME_SERVER; - } - producer.setNamesrvAddr(configNameServer); - return producer; - } - - @Bean(destroyMethod = "destroy") - @ConditionalOnMissingBean - public RocketMQTemplate rocketMQTemplate(DefaultMQProducer mqProducer, - ObjectMapper objectMapper) { - RocketMQTemplate rocketMQTemplate = new RocketMQTemplate(); - rocketMQTemplate.setProducer(mqProducer); - rocketMQTemplate.setObjectMapper(objectMapper); - return rocketMQTemplate; - } - - @Bean - @ConditionalOnBean(RocketMQTemplate.class) - @ConditionalOnMissingBean(TransactionHandlerRegistry.class) - public TransactionHandlerRegistry transactionHandlerRegistry( - RocketMQTemplate template) { - return new TransactionHandlerRegistry(template); - } - - @Bean(name = RocketMQConfigUtils.ROCKETMQ_TRANSACTION_ANNOTATION_PROCESSOR_BEAN_NAME) - @ConditionalOnBean(TransactionHandlerRegistry.class) - public static RocketMQTransactionAnnotationProcessor transactionAnnotationProcessor( - TransactionHandlerRegistry transactionHandlerRegistry) { - return new RocketMQTransactionAnnotationProcessor(transactionHandlerRegistry); - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/constant/RocketMQConst.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/constant/RocketMQConst.java new file mode 100644 index 000000000..d0a3b88e0 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/constant/RocketMQConst.java @@ -0,0 +1,100 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.constant; + +import org.apache.rocketmq.common.message.MessageConst; + +/** + * @author zkzlx + */ +public class RocketMQConst extends MessageConst { + + /** + * Default NameServer value. + */ + public static final String DEFAULT_NAME_SERVER = "127.0.0.1:9876"; + + /** + * Default group for SCS RocketMQ Binder. + */ + public static final String DEFAULT_GROUP = "binder_default_group_name"; + + /** + * user args for SCS RocketMQ Binder. + */ + public static final String USER_TRANSACTIONAL_ARGS = "TRANSACTIONAL_ARGS"; + + /** + * It is mainly provided for conversion between rocketMq-message and Spring-message, + * and parameters are passed through HEADERS. + */ + public static class Headers { + + /** + * keys for SCS RocketMQ Headers. + */ + public static final String KEYS = MessageConst.PROPERTY_KEYS; + + /** + * tags for SCS RocketMQ Headers. + */ + public static final String TAGS = MessageConst.PROPERTY_TAGS; + + /** + * topic for SCS RocketMQ Headers. + */ + public static final String TOPIC = "MQ_TOPIC"; + + /** + * The ID of the message. + */ + public static final String MESSAGE_ID = "MQ_MESSAGE_ID"; + + /** + * The timestamp that the message producer invokes the message sending API. + */ + public static final String BORN_TIMESTAMP = "MQ_BORN_TIMESTAMP"; + + /** + * The IP and port number of the message producer. + */ + public static final String BORN_HOST = "MQ_BORN_HOST"; + + /** + * Message flag, MQ is not processed and is available for use by applications. + */ + public static final String FLAG = "MQ_FLAG"; + + /** + * Message consumption queue ID. + */ + public static final String QUEUE_ID = "MQ_QUEUE_ID"; + + /** + * Message system Flag, such as whether or not to compress, whether or not to + * transactional messages. + */ + public static final String SYS_FLAG = "MQ_SYS_FLAG"; + + /** + * The transaction ID of the transaction message. + */ + public static final String TRANSACTION_ID = "MQ_TRANSACTION_ID"; + + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/consuming/RocketMQListenerBindingContainer.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/consuming/RocketMQListenerBindingContainer.java deleted file mode 100644 index 92c65b65e..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/consuming/RocketMQListenerBindingContainer.java +++ /dev/null @@ -1,465 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq.consuming; - -import java.util.List; -import java.util.Objects; - -import com.alibaba.cloud.stream.binder.rocketmq.RocketMQBinderUtils; -import com.alibaba.cloud.stream.binder.rocketmq.RocketMQMessageChannelBinder; -import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQBinderConfigurationProperties; -import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties; -import com.alibaba.cloud.stream.binder.rocketmq.support.RocketMQHeaderMapper; -import org.apache.rocketmq.acl.common.AclClientRPCHook; -import org.apache.rocketmq.acl.common.SessionCredentials; -import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer; -import org.apache.rocketmq.client.consumer.MessageSelector; -import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext; -import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus; -import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyContext; -import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyStatus; -import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently; -import org.apache.rocketmq.client.consumer.listener.MessageListenerOrderly; -import org.apache.rocketmq.client.consumer.rebalance.AllocateMessageQueueAveragely; -import org.apache.rocketmq.client.exception.MQClientException; -import org.apache.rocketmq.common.UtilAll; -import org.apache.rocketmq.common.message.MessageExt; -import org.apache.rocketmq.remoting.RPCHook; -import org.apache.rocketmq.spring.annotation.ConsumeMode; -import org.apache.rocketmq.spring.annotation.MessageModel; -import org.apache.rocketmq.spring.annotation.SelectorType; -import org.apache.rocketmq.spring.core.RocketMQListener; -import org.apache.rocketmq.spring.core.RocketMQPushConsumerLifecycleListener; -import org.apache.rocketmq.spring.support.RocketMQListenerContainer; -import org.apache.rocketmq.spring.support.RocketMQUtil; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import org.springframework.beans.factory.InitializingBean; -import org.springframework.cloud.stream.binder.ExtendedConsumerProperties; -import org.springframework.context.SmartLifecycle; -import org.springframework.integration.support.MessageBuilder; -import org.springframework.messaging.Message; -import org.springframework.util.Assert; -import org.springframework.util.StringUtils; - -import static com.alibaba.cloud.stream.binder.rocketmq.RocketMQBinderConstants.ROCKETMQ_RECONSUME_TIMES; - -/** - * A class that Listen on rocketmq message. - *

- * this class will delegate {@link RocketMQListener} to handle message - * - * @author Jim - * @author Xiejiashuai - * @see RocketMQListener - */ -public class RocketMQListenerBindingContainer - implements InitializingBean, RocketMQListenerContainer, SmartLifecycle { - - private final static Logger log = LoggerFactory - .getLogger(RocketMQListenerBindingContainer.class); - - private long suspendCurrentQueueTimeMillis = 1000; - - /** - * Message consume retry strategy
- * -1,no retry,put into DLQ directly
- * 0,broker control retry frequency
- * >0,client control retry frequency. - */ - private int delayLevelWhenNextConsume = 0; - - private List nameServer; - - private String consumerGroup; - - private String topic; - - private int consumeThreadMax = 64; - - private String charset = "UTF-8"; - - private RocketMQListener rocketMQListener; - - private RocketMQHeaderMapper headerMapper; - - private DefaultMQPushConsumer consumer; - - private boolean running; - - private final ExtendedConsumerProperties rocketMQConsumerProperties; - - private final RocketMQMessageChannelBinder rocketMQMessageChannelBinder; - - private final RocketMQBinderConfigurationProperties rocketBinderConfigurationProperties; - - // The following properties came from RocketMQConsumerProperties. - private ConsumeMode consumeMode; - - private SelectorType selectorType; - - private String selectorExpression; - - private MessageModel messageModel; - - public RocketMQListenerBindingContainer( - ExtendedConsumerProperties rocketMQConsumerProperties, - RocketMQBinderConfigurationProperties rocketBinderConfigurationProperties, - RocketMQMessageChannelBinder rocketMQMessageChannelBinder) { - this.rocketMQConsumerProperties = rocketMQConsumerProperties; - this.rocketBinderConfigurationProperties = rocketBinderConfigurationProperties; - this.rocketMQMessageChannelBinder = rocketMQMessageChannelBinder; - this.consumeMode = rocketMQConsumerProperties.getExtension().getOrderly() - ? ConsumeMode.ORDERLY : ConsumeMode.CONCURRENTLY; - if (!StringUtils.hasLength(rocketMQConsumerProperties.getExtension().getSql())) { - this.selectorType = SelectorType.TAG; - this.selectorExpression = rocketMQConsumerProperties.getExtension().getTags(); - } - else { - this.selectorType = SelectorType.SQL92; - this.selectorExpression = rocketMQConsumerProperties.getExtension().getSql(); - } - this.messageModel = rocketMQConsumerProperties.getExtension().getBroadcasting() - ? MessageModel.BROADCASTING : MessageModel.CLUSTERING; - } - - @Override - public void setupMessageListener(RocketMQListener rocketMQListener) { - this.rocketMQListener = rocketMQListener; - } - - @Override - public void destroy() throws Exception { - this.setRunning(false); - if (Objects.nonNull(consumer)) { - consumer.shutdown(); - } - log.info("container destroyed, {}", this.toString()); - } - - @Override - public void afterPropertiesSet() throws Exception { - initRocketMQPushConsumer(); - } - - @Override - public boolean isAutoStartup() { - return true; - } - - @Override - public void stop(Runnable callback) { - stop(); - callback.run(); - } - - @Override - public void start() { - if (this.isRunning()) { - throw new IllegalStateException( - "container already running. " + this.toString()); - } - - try { - consumer.start(); - } - catch (MQClientException e) { - throw new IllegalStateException("Failed to start RocketMQ push consumer", e); - } - this.setRunning(true); - - log.info("running container: {}", this.toString()); - } - - @Override - public void stop() { - if (this.isRunning()) { - if (Objects.nonNull(consumer)) { - consumer.shutdown(); - } - setRunning(false); - } - } - - @Override - public boolean isRunning() { - return running; - } - - private void setRunning(boolean running) { - this.running = running; - } - - @Override - public int getPhase() { - return Integer.MAX_VALUE; - } - - private void initRocketMQPushConsumer() throws MQClientException { - Assert.notNull(rocketMQListener, "Property 'rocketMQListener' is required"); - Assert.notNull(consumerGroup, "Property 'consumerGroup' is required"); - Assert.notNull(nameServer, "Property 'nameServer' is required"); - Assert.notNull(topic, "Property 'topic' is required"); - - String ak = rocketBinderConfigurationProperties.getAccessKey(); - String sk = rocketBinderConfigurationProperties.getSecretKey(); - if (StringUtils.hasLength(ak) && StringUtils.hasLength(sk)) { - RPCHook rpcHook = new AclClientRPCHook(new SessionCredentials(ak, sk)); - consumer = new DefaultMQPushConsumer(consumerGroup, rpcHook, - new AllocateMessageQueueAveragely(), - rocketBinderConfigurationProperties.isEnableMsgTrace(), - rocketBinderConfigurationProperties.getCustomizedTraceTopic()); - consumer.setInstanceName(RocketMQUtil.getInstanceName(rpcHook, - topic + "|" + UtilAll.getPid())); - consumer.setVipChannelEnabled(false); - } - else { - consumer = new DefaultMQPushConsumer(consumerGroup, - rocketBinderConfigurationProperties.isEnableMsgTrace(), - rocketBinderConfigurationProperties.getCustomizedTraceTopic()); - } - - consumer.setNamesrvAddr(RocketMQBinderUtils.getNameServerStr(nameServer)); - consumer.setConsumeThreadMax(rocketMQConsumerProperties.getConcurrency()); - consumer.setConsumeThreadMin(rocketMQConsumerProperties.getConcurrency()); - - switch (messageModel) { - case BROADCASTING: - consumer.setMessageModel( - org.apache.rocketmq.common.protocol.heartbeat.MessageModel.BROADCASTING); - break; - case CLUSTERING: - consumer.setMessageModel( - org.apache.rocketmq.common.protocol.heartbeat.MessageModel.CLUSTERING); - break; - default: - throw new IllegalArgumentException("Property 'messageModel' was wrong."); - } - - switch (selectorType) { - case TAG: - consumer.subscribe(topic, selectorExpression); - break; - case SQL92: - consumer.subscribe(topic, MessageSelector.bySql(selectorExpression)); - break; - default: - throw new IllegalArgumentException("Property 'selectorType' was wrong."); - } - - switch (consumeMode) { - case ORDERLY: - consumer.setMessageListener(new DefaultMessageListenerOrderly()); - break; - case CONCURRENTLY: - consumer.setMessageListener(new DefaultMessageListenerConcurrently()); - break; - default: - throw new IllegalArgumentException("Property 'consumeMode' was wrong."); - } - - if (rocketMQListener instanceof RocketMQPushConsumerLifecycleListener) { - ((RocketMQPushConsumerLifecycleListener) rocketMQListener) - .prepareStart(consumer); - } - - } - - @Override - public String toString() { - return "RocketMQListenerBindingContainer{" + "consumerGroup='" + consumerGroup - + '\'' + ", nameServer='" + nameServer + '\'' + ", topic='" + topic + '\'' - + ", consumeMode=" + consumeMode + ", selectorType=" + selectorType - + ", selectorExpression='" + selectorExpression + '\'' + ", messageModel=" - + messageModel + '}'; - } - - public long getSuspendCurrentQueueTimeMillis() { - return suspendCurrentQueueTimeMillis; - } - - public void setSuspendCurrentQueueTimeMillis(long suspendCurrentQueueTimeMillis) { - this.suspendCurrentQueueTimeMillis = suspendCurrentQueueTimeMillis; - } - - public int getDelayLevelWhenNextConsume() { - return delayLevelWhenNextConsume; - } - - public void setDelayLevelWhenNextConsume(int delayLevelWhenNextConsume) { - this.delayLevelWhenNextConsume = delayLevelWhenNextConsume; - } - - public List getNameServer() { - return nameServer; - } - - public void setNameServer(List nameServer) { - this.nameServer = nameServer; - } - - public String getConsumerGroup() { - return consumerGroup; - } - - public void setConsumerGroup(String consumerGroup) { - this.consumerGroup = consumerGroup; - } - - public String getTopic() { - return topic; - } - - public void setTopic(String topic) { - this.topic = topic; - } - - public int getConsumeThreadMax() { - return consumeThreadMax; - } - - public void setConsumeThreadMax(int consumeThreadMax) { - this.consumeThreadMax = consumeThreadMax; - } - - public String getCharset() { - return charset; - } - - public void setCharset(String charset) { - this.charset = charset; - } - - public RocketMQListener getRocketMQListener() { - return rocketMQListener; - } - - public void setRocketMQListener(RocketMQListener rocketMQListener) { - this.rocketMQListener = rocketMQListener; - } - - public DefaultMQPushConsumer getConsumer() { - return consumer; - } - - public void setConsumer(DefaultMQPushConsumer consumer) { - this.consumer = consumer; - } - - public ExtendedConsumerProperties getRocketMQConsumerProperties() { - return rocketMQConsumerProperties; - } - - public ConsumeMode getConsumeMode() { - return consumeMode; - } - - public SelectorType getSelectorType() { - return selectorType; - } - - public String getSelectorExpression() { - return selectorExpression; - } - - public MessageModel getMessageModel() { - return messageModel; - } - - public RocketMQHeaderMapper getHeaderMapper() { - return headerMapper; - } - - public void setHeaderMapper(RocketMQHeaderMapper headerMapper) { - this.headerMapper = headerMapper; - } - - /** - * Convert rocketmq {@link MessageExt} to Spring {@link Message}. - * @param messageExt the rocketmq message - * @return the converted Spring {@link Message} - */ - @SuppressWarnings("unchecked") - private Message convertToSpringMessage(MessageExt messageExt) { - - // add reconsume-times header to messageExt - int reconsumeTimes = messageExt.getReconsumeTimes(); - messageExt.putUserProperty(ROCKETMQ_RECONSUME_TIMES, - String.valueOf(reconsumeTimes)); - Message message = RocketMQUtil.convertToSpringMessage(messageExt); - return MessageBuilder.fromMessage(message) - .copyHeaders(headerMapper.toHeaders(messageExt.getProperties())).build(); - } - - public class DefaultMessageListenerConcurrently - implements MessageListenerConcurrently { - - @SuppressWarnings({ "unchecked", "Duplicates" }) - @Override - public ConsumeConcurrentlyStatus consumeMessage(List msgs, - ConsumeConcurrentlyContext context) { - for (MessageExt messageExt : msgs) { - log.debug("received msg: {}", messageExt); - try { - long now = System.currentTimeMillis(); - rocketMQListener.onMessage(convertToSpringMessage(messageExt)); - long costTime = System.currentTimeMillis() - now; - log.debug("consume {} message key:[{}] cost: {} ms", - messageExt.getMsgId(), messageExt.getKeys(), costTime); - } - catch (Exception e) { - log.warn("consume message failed. messageExt:{}", messageExt, e); - context.setDelayLevelWhenNextConsume(delayLevelWhenNextConsume); - return ConsumeConcurrentlyStatus.RECONSUME_LATER; - } - } - - return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; - } - - } - - public class DefaultMessageListenerOrderly implements MessageListenerOrderly { - - @SuppressWarnings({ "unchecked", "Duplicates" }) - @Override - public ConsumeOrderlyStatus consumeMessage(List msgs, - ConsumeOrderlyContext context) { - for (MessageExt messageExt : msgs) { - log.debug("received msg: {}", messageExt); - try { - long now = System.currentTimeMillis(); - rocketMQListener.onMessage(convertToSpringMessage(messageExt)); - long costTime = System.currentTimeMillis() - now; - log.info("consume {} message key:[{}] cost: {} ms", - messageExt.getMsgId(), messageExt.getKeys(), costTime); - } - catch (Exception e) { - log.warn("consume message failed. messageExt:{}", messageExt, e); - context.setSuspendCurrentQueueTimeMillis( - suspendCurrentQueueTimeMillis); - return ConsumeOrderlyStatus.SUSPEND_CURRENT_QUEUE_A_MOMENT; - } - } - - return ConsumeOrderlyStatus.SUCCESS; - } - - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/consuming/RocketMQMessageQueueChooser.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/consuming/RocketMQMessageQueueChooser.java deleted file mode 100644 index 948555924..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/consuming/RocketMQMessageQueueChooser.java +++ /dev/null @@ -1,62 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq.consuming; - -import java.util.ArrayList; -import java.util.List; -import java.util.Set; - -import org.apache.rocketmq.common.message.MessageQueue; - -/** - * @author Jim - */ -public class RocketMQMessageQueueChooser { - - private volatile int queueIndex = 0; - - private volatile List messageQueues; - - public MessageQueue choose() { - return messageQueues.get(queueIndex); - } - - public int requeue() { - if (queueIndex - 1 < 0) { - this.queueIndex = messageQueues.size() - 1; - } - else { - this.queueIndex = this.queueIndex - 1; - } - return this.queueIndex; - } - - public void increment() { - this.queueIndex = (this.queueIndex + 1) % messageQueues.size(); - } - - public void reset(Set queueSet) { - this.messageQueues = null; - this.messageQueues = new ArrayList<>(queueSet); - this.queueIndex = 0; - } - - public List getMessageQueues() { - return messageQueues; - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/convert/RocketMQMessageConverter.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/convert/RocketMQMessageConverter.java new file mode 100644 index 000000000..f69290997 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/convert/RocketMQMessageConverter.java @@ -0,0 +1,90 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.convert; + +import java.util.ArrayList; +import java.util.List; + +import org.springframework.messaging.converter.ByteArrayMessageConverter; +import org.springframework.messaging.converter.CompositeMessageConverter; +import org.springframework.messaging.converter.MappingJackson2MessageConverter; +import org.springframework.messaging.converter.MessageConverter; +import org.springframework.messaging.converter.StringMessageConverter; +import org.springframework.util.ClassUtils; + +/** + * The default message converter of rocketMq,its bean name is {@link #DEFAULT_NAME} . + * + * @author zkzlx + */ +public class RocketMQMessageConverter { + + /** + * if you want to customize a bean, please use the BeanName. + */ + public static final String DEFAULT_NAME = "com.alibaba.cloud.stream.binder.rocketmq.convert.RocketMQMessageConverter"; + + private static final boolean JACKSON_PRESENT; + + private static final boolean FASTJSON_PRESENT; + + static { + ClassLoader classLoader = RocketMQMessageConverter.class.getClassLoader(); + JACKSON_PRESENT = ClassUtils + .isPresent("com.fasterxml.jackson.databind.ObjectMapper", classLoader) + && ClassUtils.isPresent("com.fasterxml.jackson.core.JsonGenerator", + classLoader); + FASTJSON_PRESENT = ClassUtils.isPresent("com.alibaba.fastjson.JSON", classLoader) + && ClassUtils.isPresent( + "com.alibaba.fastjson.support.config.FastJsonConfig", + classLoader); + } + + private CompositeMessageConverter messageConverter; + + public RocketMQMessageConverter() { + List messageConverters = new ArrayList<>(); + ByteArrayMessageConverter byteArrayMessageConverter = new ByteArrayMessageConverter(); + byteArrayMessageConverter.setContentTypeResolver(null); + messageConverters.add(byteArrayMessageConverter); + messageConverters.add(new StringMessageConverter()); + if (JACKSON_PRESENT) { + messageConverters.add(new MappingJackson2MessageConverter()); + } + if (FASTJSON_PRESENT) { + try { + messageConverters.add((MessageConverter) ClassUtils.forName( + "com.alibaba.fastjson.support.spring.messaging.MappingFastJsonMessageConverter", + ClassUtils.getDefaultClassLoader()).newInstance()); + } + catch (ClassNotFoundException | IllegalAccessException + | InstantiationException ignored) { + // ignore this exception + } + } + messageConverter = new CompositeMessageConverter(messageConverters); + } + + public CompositeMessageConverter getMessageConverter() { + return messageConverter; + } + + public void setMessageConverter(CompositeMessageConverter messageConverter) { + this.messageConverter = messageConverter; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/custom/RocketMQBeanContainerCache.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/custom/RocketMQBeanContainerCache.java new file mode 100644 index 000000000..a90e1f21a --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/custom/RocketMQBeanContainerCache.java @@ -0,0 +1,78 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.custom; + +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; + +import com.alibaba.cloud.stream.binder.rocketmq.extend.ErrorAcknowledgeHandler; +import org.apache.rocketmq.client.consumer.AllocateMessageQueueStrategy; +import org.apache.rocketmq.client.consumer.listener.MessageListener; +import org.apache.rocketmq.client.hook.CheckForbiddenHook; +import org.apache.rocketmq.client.hook.SendMessageHook; +import org.apache.rocketmq.client.producer.MessageQueueSelector; +import org.apache.rocketmq.client.producer.SendCallback; +import org.apache.rocketmq.client.producer.TransactionListener; + +import org.springframework.messaging.converter.CompositeMessageConverter; +import org.springframework.util.StringUtils; + +/** + * Gets the beans configured in the configuration file. + * + * @author junboXiang + */ +public final class RocketMQBeanContainerCache { + + private RocketMQBeanContainerCache() { + } + + private static final Class[] CLASSES = new Class[] { + CompositeMessageConverter.class, AllocateMessageQueueStrategy.class, + MessageQueueSelector.class, MessageListener.class, TransactionListener.class, + SendCallback.class, CheckForbiddenHook.class, SendMessageHook.class, + ErrorAcknowledgeHandler.class }; + + private static final Map BEANS_CACHE = new ConcurrentHashMap<>(); + + static void putBean(String beanName, Object beanObj) { + BEANS_CACHE.put(beanName, beanObj); + } + + static Class[] getClassAry() { + return CLASSES; + } + + public static T getBean(String beanName, Class clazz) { + return getBean(beanName, clazz, null); + } + + public static T getBean(String beanName, Class clazz, T defaultObj) { + if (!StringUtils.hasLength(beanName)) { + return defaultObj; + } + Object obj = BEANS_CACHE.get(beanName); + if (null == obj) { + return defaultObj; + } + if (clazz.isAssignableFrom(obj.getClass())) { + return (T) obj; + } + return defaultObj; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/custom/RocketMQConfigBeanPostProcessor.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/custom/RocketMQConfigBeanPostProcessor.java new file mode 100644 index 000000000..a83fdbe08 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/custom/RocketMQConfigBeanPostProcessor.java @@ -0,0 +1,43 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.custom; + +import java.util.stream.Stream; + +import org.springframework.beans.BeansException; +import org.springframework.beans.factory.config.BeanPostProcessor; + +/** + * find RocketMQ bean by annotations. + * + * @author junboXiang + * + */ +public class RocketMQConfigBeanPostProcessor implements BeanPostProcessor { + + @Override + public Object postProcessAfterInitialization(Object bean, String beanName) + throws BeansException { + Stream.of(RocketMQBeanContainerCache.getClassAry()).forEach(clazz -> { + if (clazz.isAssignableFrom(bean.getClass())) { + RocketMQBeanContainerCache.putBean(beanName, bean); + } + }); + return bean; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/extend/ErrorAcknowledgeHandler.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/extend/ErrorAcknowledgeHandler.java new file mode 100644 index 000000000..dfe65fd94 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/extend/ErrorAcknowledgeHandler.java @@ -0,0 +1,35 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.extend; + +import org.springframework.integration.acks.AcknowledgmentCallback.Status; +import org.springframework.messaging.Message; + +/** + * @author zkzlx + */ +public interface ErrorAcknowledgeHandler { + + /** + * Ack state handling, including receive, reject, and retry, when a consumption + * exception occurs. + * @param message message + * @return see {@link Status} + */ + Status handler(Message message); + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/RocketMQInboundChannelAdapter.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/RocketMQInboundChannelAdapter.java deleted file mode 100644 index a3b680419..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/RocketMQInboundChannelAdapter.java +++ /dev/null @@ -1,176 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq.integration; - -import com.alibaba.cloud.stream.binder.rocketmq.consuming.RocketMQListenerBindingContainer; -import com.alibaba.cloud.stream.binder.rocketmq.metrics.Instrumentation; -import com.alibaba.cloud.stream.binder.rocketmq.metrics.InstrumentationManager; -import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties; -import org.apache.rocketmq.spring.core.RocketMQListener; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import org.springframework.cloud.stream.binder.ExtendedConsumerProperties; -import org.springframework.integration.endpoint.MessageProducerSupport; -import org.springframework.integration.support.MessageBuilder; -import org.springframework.messaging.Message; -import org.springframework.messaging.MessagingException; -import org.springframework.retry.RecoveryCallback; -import org.springframework.retry.RetryCallback; -import org.springframework.retry.RetryContext; -import org.springframework.retry.RetryListener; -import org.springframework.retry.support.RetryTemplate; -import org.springframework.util.Assert; - -/** - * @author Jim - */ -public class RocketMQInboundChannelAdapter extends MessageProducerSupport { - - private static final Logger log = LoggerFactory - .getLogger(RocketMQInboundChannelAdapter.class); - - private RetryTemplate retryTemplate; - - private RecoveryCallback recoveryCallback; - - private RocketMQListenerBindingContainer rocketMQListenerContainer; - - private final ExtendedConsumerProperties consumerProperties; - - private final InstrumentationManager instrumentationManager; - - public RocketMQInboundChannelAdapter( - RocketMQListenerBindingContainer rocketMQListenerContainer, - ExtendedConsumerProperties consumerProperties, - InstrumentationManager instrumentationManager) { - this.rocketMQListenerContainer = rocketMQListenerContainer; - this.consumerProperties = consumerProperties; - this.instrumentationManager = instrumentationManager; - } - - @Override - protected void onInit() { - if (consumerProperties == null - || !consumerProperties.getExtension().getEnabled()) { - return; - } - super.onInit(); - if (this.retryTemplate != null) { - Assert.state(getErrorChannel() == null, - "Cannot have an 'errorChannel' property when a 'RetryTemplate' is " - + "provided; use an 'ErrorMessageSendingRecoverer' in the 'recoveryCallback' property to " - + "send an error message when retries are exhausted"); - } - - BindingRocketMQListener listener = new BindingRocketMQListener(); - rocketMQListenerContainer.setRocketMQListener(listener); - - if (retryTemplate != null) { - this.retryTemplate.registerListener(listener); - } - - try { - rocketMQListenerContainer.afterPropertiesSet(); - - } - catch (Exception e) { - log.error("rocketMQListenerContainer init error: " + e.getMessage(), e); - throw new IllegalArgumentException( - "rocketMQListenerContainer init error: " + e.getMessage(), e); - } - - instrumentationManager.addHealthInstrumentation( - new Instrumentation(rocketMQListenerContainer.getTopic() - + rocketMQListenerContainer.getConsumerGroup())); - } - - @Override - protected void doStart() { - if (consumerProperties == null - || !consumerProperties.getExtension().getEnabled()) { - return; - } - try { - rocketMQListenerContainer.start(); - instrumentationManager - .getHealthInstrumentation(rocketMQListenerContainer.getTopic() - + rocketMQListenerContainer.getConsumerGroup()) - .markStartedSuccessfully(); - } - catch (Exception e) { - instrumentationManager - .getHealthInstrumentation(rocketMQListenerContainer.getTopic() - + rocketMQListenerContainer.getConsumerGroup()) - .markStartFailed(e); - log.error("RocketMQTemplate startup failed, Caused by " + e.getMessage()); - throw new MessagingException(MessageBuilder.withPayload( - "RocketMQTemplate startup failed, Caused by " + e.getMessage()) - .build(), e); - } - } - - @Override - protected void doStop() { - rocketMQListenerContainer.stop(); - } - - public void setRetryTemplate(RetryTemplate retryTemplate) { - this.retryTemplate = retryTemplate; - } - - public void setRecoveryCallback(RecoveryCallback recoveryCallback) { - this.recoveryCallback = recoveryCallback; - } - - protected class BindingRocketMQListener - implements RocketMQListener, RetryListener { - - @Override - public void onMessage(Message message) { - boolean enableRetry = RocketMQInboundChannelAdapter.this.retryTemplate != null; - if (enableRetry) { - RocketMQInboundChannelAdapter.this.retryTemplate.execute(context -> { - RocketMQInboundChannelAdapter.this.sendMessage(message); - return null; - }, (RecoveryCallback) RocketMQInboundChannelAdapter.this.recoveryCallback); - } - else { - RocketMQInboundChannelAdapter.this.sendMessage(message); - } - } - - @Override - public boolean open(RetryContext context, - RetryCallback callback) { - return true; - } - - @Override - public void close(RetryContext context, - RetryCallback callback, Throwable throwable) { - } - - @Override - public void onError(RetryContext context, - RetryCallback callback, Throwable throwable) { - - } - - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/RocketMQMessageHandler.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/RocketMQMessageHandler.java deleted file mode 100644 index 18e4f316d..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/RocketMQMessageHandler.java +++ /dev/null @@ -1,302 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq.integration; - -import java.util.List; -import java.util.Map; -import java.util.Optional; - -import com.alibaba.cloud.stream.binder.rocketmq.RocketMQBinderConstants; -import com.alibaba.cloud.stream.binder.rocketmq.metrics.Instrumentation; -import com.alibaba.cloud.stream.binder.rocketmq.metrics.InstrumentationManager; -import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties; -import com.alibaba.cloud.stream.binder.rocketmq.support.RocketMQHeaderMapper; -import org.apache.rocketmq.client.exception.MQClientException; -import org.apache.rocketmq.client.producer.SendCallback; -import org.apache.rocketmq.client.producer.SendResult; -import org.apache.rocketmq.client.producer.SendStatus; -import org.apache.rocketmq.common.message.MessageConst; -import org.apache.rocketmq.common.message.MessageQueue; -import org.apache.rocketmq.spring.core.RocketMQTemplate; -import org.apache.rocketmq.spring.support.RocketMQHeaders; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import org.springframework.cloud.stream.binder.BinderHeaders; -import org.springframework.cloud.stream.binder.ExtendedProducerProperties; -import org.springframework.cloud.stream.binding.MessageConverterConfigurer; -import org.springframework.context.Lifecycle; -import org.springframework.integration.handler.AbstractMessageHandler; -import org.springframework.integration.support.DefaultErrorMessageStrategy; -import org.springframework.integration.support.ErrorMessageStrategy; -import org.springframework.integration.support.MessageBuilder; -import org.springframework.messaging.Message; -import org.springframework.messaging.MessageChannel; -import org.springframework.messaging.MessagingException; -import org.springframework.messaging.support.ErrorMessage; -import org.springframework.util.Assert; -import org.springframework.util.StringUtils; - -/** - * @author Jim - */ -public class RocketMQMessageHandler extends AbstractMessageHandler implements Lifecycle { - - private final static Logger log = LoggerFactory - .getLogger(RocketMQMessageHandler.class); - - private ErrorMessageStrategy errorMessageStrategy = new DefaultErrorMessageStrategy(); - - private MessageChannel sendFailureChannel; - - private final RocketMQTemplate rocketMQTemplate; - - private RocketMQHeaderMapper headerMapper; - - private final Boolean transactional; - - private final String destination; - - private final String groupName; - - private final InstrumentationManager instrumentationManager; - - private boolean sync = false; - - private volatile boolean running = false; - - private ExtendedProducerProperties producerProperties; - - private MessageConverterConfigurer.PartitioningInterceptor partitioningInterceptor; - - public RocketMQMessageHandler(RocketMQTemplate rocketMQTemplate, String destination, - String groupName, Boolean transactional, - InstrumentationManager instrumentationManager, - ExtendedProducerProperties producerProperties, - MessageConverterConfigurer.PartitioningInterceptor partitioningInterceptor) { - this.rocketMQTemplate = rocketMQTemplate; - this.destination = destination; - this.groupName = groupName; - this.transactional = transactional; - this.instrumentationManager = instrumentationManager; - this.producerProperties = producerProperties; - this.partitioningInterceptor = partitioningInterceptor; - } - - @Override - public void start() { - if (!transactional) { - instrumentationManager - .addHealthInstrumentation(new Instrumentation(destination)); - try { - rocketMQTemplate.afterPropertiesSet(); - instrumentationManager.getHealthInstrumentation(destination) - .markStartedSuccessfully(); - } - catch (Exception e) { - instrumentationManager.getHealthInstrumentation(destination) - .markStartFailed(e); - log.error("RocketMQTemplate startup failed, Caused by " + e.getMessage()); - throw new MessagingException(MessageBuilder.withPayload( - "RocketMQTemplate startup failed, Caused by " + e.getMessage()) - .build(), e); - } - } - if (producerProperties.isPartitioned()) { - try { - List messageQueues = rocketMQTemplate.getProducer() - .fetchPublishMessageQueues(destination); - if (producerProperties.getPartitionCount() != messageQueues.size()) { - logger.info(String.format( - "The partition count of topic '%s' will change from '%s' to '%s'", - destination, producerProperties.getPartitionCount(), - messageQueues.size())); - producerProperties.setPartitionCount(messageQueues.size()); - partitioningInterceptor - .setPartitionCount(producerProperties.getPartitionCount()); - } - } - catch (MQClientException e) { - logger.error(e, "fetch publish message queues fail"); - } - } - running = true; - } - - @Override - public void stop() { - if (!transactional) { - rocketMQTemplate.destroy(); - } - running = false; - } - - @Override - public boolean isRunning() { - return running; - } - - @Override - protected void handleMessageInternal( - org.springframework.messaging.Message message) { - try { - // issue 737 fix - Map jsonHeaders = headerMapper - .fromHeaders(message.getHeaders()); - message = org.springframework.messaging.support.MessageBuilder - .fromMessage(message).copyHeaders(jsonHeaders).build(); - - final StringBuilder topicWithTags = new StringBuilder(destination); - String tags = Optional - .ofNullable(message.getHeaders().get(RocketMQHeaders.TAGS)).orElse("") - .toString(); - if (StringUtils.hasLength(tags)) { - topicWithTags.append(":").append(tags); - } - - SendResult sendRes = null; - if (transactional) { - sendRes = rocketMQTemplate.sendMessageInTransaction(groupName, - topicWithTags.toString(), message, message.getHeaders() - .get(RocketMQBinderConstants.ROCKET_TRANSACTIONAL_ARG)); - log.debug("transactional send to topic " + topicWithTags + " " + sendRes); - } - else { - int delayLevel = 0; - try { - Object delayLevelObj = message.getHeaders() - .getOrDefault(MessageConst.PROPERTY_DELAY_TIME_LEVEL, 0); - if (delayLevelObj instanceof Number) { - delayLevel = ((Number) delayLevelObj).intValue(); - } - else if (delayLevelObj instanceof String) { - delayLevel = Integer.parseInt((String) delayLevelObj); - } - } - catch (Exception e) { - // ignore - } - boolean needSelectQueue = message.getHeaders() - .containsKey(BinderHeaders.PARTITION_HEADER); - if (sync) { - if (needSelectQueue) { - sendRes = rocketMQTemplate.syncSendOrderly( - topicWithTags.toString(), message, "", - rocketMQTemplate.getProducer().getSendMsgTimeout()); - } - else { - sendRes = rocketMQTemplate.syncSend(topicWithTags.toString(), - message, - rocketMQTemplate.getProducer().getSendMsgTimeout(), - delayLevel); - } - log.debug("sync send to topic " + topicWithTags + " " + sendRes); - } - else { - Message finalMessage = message; - SendCallback sendCallback = new SendCallback() { - @Override - public void onSuccess(SendResult sendResult) { - log.debug("async send to topic " + topicWithTags + " " - + sendResult); - } - - @Override - public void onException(Throwable e) { - log.error("RocketMQ Message hasn't been sent. Caused by " - + e.getMessage()); - if (getSendFailureChannel() != null) { - getSendFailureChannel().send( - RocketMQMessageHandler.this.errorMessageStrategy - .buildErrorMessage(new MessagingException( - finalMessage, e), null)); - } - } - }; - if (needSelectQueue) { - rocketMQTemplate.asyncSendOrderly(topicWithTags.toString(), - message, "", sendCallback, - rocketMQTemplate.getProducer().getSendMsgTimeout()); - } - else { - rocketMQTemplate.asyncSend(topicWithTags.toString(), message, - sendCallback); - } - } - } - if (sendRes != null && !sendRes.getSendStatus().equals(SendStatus.SEND_OK)) { - if (getSendFailureChannel() != null) { - this.getSendFailureChannel().send(message); - } - else { - throw new MessagingException(message, - new MQClientException("message hasn't been sent", null)); - } - } - } - catch (Exception e) { - log.error("RocketMQ Message hasn't been sent. Caused by " + e.getMessage()); - if (getSendFailureChannel() != null) { - getSendFailureChannel().send(this.errorMessageStrategy - .buildErrorMessage(new MessagingException(message, e), null)); - } - else { - throw new MessagingException(message, e); - } - } - - } - - /** - * Set the failure channel. After a send failure, an {@link ErrorMessage} will be sent - * to this channel with a payload of a {@link MessagingException} with the failed - * message and cause. - * @param sendFailureChannel the failure channel. - * @since 0.2.2 - */ - public void setSendFailureChannel(MessageChannel sendFailureChannel) { - this.sendFailureChannel = sendFailureChannel; - } - - /** - * Set the error message strategy implementation to use when sending error messages - * after send failures. Cannot be null. - * @param errorMessageStrategy the implementation. - * @since 0.2.2 - */ - public void setErrorMessageStrategy(ErrorMessageStrategy errorMessageStrategy) { - Assert.notNull(errorMessageStrategy, "'errorMessageStrategy' cannot be null"); - this.errorMessageStrategy = errorMessageStrategy; - } - - public MessageChannel getSendFailureChannel() { - return sendFailureChannel; - } - - public void setSync(boolean sync) { - this.sync = sync; - } - - public RocketMQHeaderMapper getHeaderMapper() { - return headerMapper; - } - - public void setHeaderMapper(RocketMQHeaderMapper headerMapper) { - this.headerMapper = headerMapper; - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/RocketMQMessageSource.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/RocketMQMessageSource.java deleted file mode 100644 index fa74a5178..000000000 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/RocketMQMessageSource.java +++ /dev/null @@ -1,382 +0,0 @@ -/* - * Copyright 2013-2018 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.cloud.stream.binder.rocketmq.integration; - -import java.util.List; -import java.util.Set; - -import com.alibaba.cloud.stream.binder.rocketmq.RocketMQBinderUtils; -import com.alibaba.cloud.stream.binder.rocketmq.consuming.RocketMQMessageQueueChooser; -import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQBinderConfigurationProperties; -import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties; -import org.apache.rocketmq.client.consumer.DefaultMQPullConsumer; -import org.apache.rocketmq.client.consumer.MessageQueueListener; -import org.apache.rocketmq.client.consumer.MessageSelector; -import org.apache.rocketmq.client.consumer.PullResult; -import org.apache.rocketmq.client.consumer.PullStatus; -import org.apache.rocketmq.client.exception.MQClientException; -import org.apache.rocketmq.common.message.MessageExt; -import org.apache.rocketmq.common.message.MessageQueue; -import org.apache.rocketmq.common.protocol.heartbeat.MessageModel; -import org.apache.rocketmq.spring.support.RocketMQUtil; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import org.springframework.beans.factory.DisposableBean; -import org.springframework.cloud.stream.binder.ExtendedConsumerProperties; -import org.springframework.context.Lifecycle; -import org.springframework.integration.IntegrationMessageHeaderAccessor; -import org.springframework.integration.acks.AcknowledgmentCallback; -import org.springframework.integration.acks.AcknowledgmentCallbackFactory; -import org.springframework.integration.endpoint.AbstractMessageSource; -import org.springframework.integration.support.MessageBuilder; -import org.springframework.messaging.Message; -import org.springframework.util.Assert; -import org.springframework.util.StringUtils; - -/** - * @author Jim - */ -public class RocketMQMessageSource extends AbstractMessageSource - implements DisposableBean, Lifecycle { - - private final static Logger log = LoggerFactory - .getLogger(RocketMQMessageSource.class); - - private final RocketMQCallbackFactory ackCallbackFactory; - - private final RocketMQBinderConfigurationProperties rocketMQBinderConfigurationProperties; - - private final ExtendedConsumerProperties rocketMQConsumerProperties; - - private final String topic; - - private final String group; - - private final Object consumerMonitor = new Object(); - - private DefaultMQPullConsumer consumer; - - private boolean running; - - private MessageSelector messageSelector; - - private RocketMQMessageQueueChooser messageQueueChooser = new RocketMQMessageQueueChooser(); - - public RocketMQMessageSource( - RocketMQBinderConfigurationProperties rocketMQBinderConfigurationProperties, - ExtendedConsumerProperties rocketMQConsumerProperties, - String topic, String group) { - this(new RocketMQCallbackFactory(), rocketMQBinderConfigurationProperties, - rocketMQConsumerProperties, topic, group); - } - - public RocketMQMessageSource(RocketMQCallbackFactory ackCallbackFactory, - RocketMQBinderConfigurationProperties rocketMQBinderConfigurationProperties, - ExtendedConsumerProperties rocketMQConsumerProperties, - String topic, String group) { - this.ackCallbackFactory = ackCallbackFactory; - this.rocketMQBinderConfigurationProperties = rocketMQBinderConfigurationProperties; - this.rocketMQConsumerProperties = rocketMQConsumerProperties; - this.topic = topic; - this.group = group; - } - - @Override - public synchronized void start() { - if (this.isRunning()) { - throw new IllegalStateException( - "pull consumer already running. " + this.toString()); - } - try { - consumer = new DefaultMQPullConsumer(group); - consumer.setNamesrvAddr(RocketMQBinderUtils.getNameServerStr( - rocketMQBinderConfigurationProperties.getNameServer())); - consumer.setConsumerPullTimeoutMillis( - rocketMQConsumerProperties.getExtension().getPullTimeout()); - consumer.setMessageModel(MessageModel.CLUSTERING); - - String tags = rocketMQConsumerProperties.getExtension().getTags(); - String sql = rocketMQConsumerProperties.getExtension().getSql(); - - if (StringUtils.hasLength(tags) && StringUtils.hasLength(sql)) { - messageSelector = MessageSelector.byTag(tags); - } - else if (StringUtils.hasLength(tags)) { - messageSelector = MessageSelector.byTag(tags); - } - else if (StringUtils.hasLength(sql)) { - messageSelector = MessageSelector.bySql(sql); - } - - consumer.registerMessageQueueListener(topic, new MessageQueueListener() { - @Override - public void messageQueueChanged(String topic, Set mqAll, - Set mqDivided) { - log.info( - "messageQueueChanged, topic='{}', mqAll=`{}`, mqDivided=`{}`", - topic, mqAll, mqDivided); - switch (consumer.getMessageModel()) { - case BROADCASTING: - RocketMQMessageSource.this.resetMessageQueues(mqAll); - break; - case CLUSTERING: - RocketMQMessageSource.this.resetMessageQueues(mqDivided); - break; - default: - break; - } - } - }); - consumer.start(); - } - catch (MQClientException e) { - log.error("DefaultMQPullConsumer startup error: " + e.getMessage(), e); - } - this.setRunning(true); - } - - @Override - public synchronized void stop() { - if (this.isRunning()) { - this.setRunning(false); - consumer.shutdown(); - } - } - - @Override - public synchronized boolean isRunning() { - return running; - } - - @Override - protected synchronized Object doReceive() { - if (messageQueueChooser.getMessageQueues() == null - || messageQueueChooser.getMessageQueues().size() == 0) { - return null; - } - try { - int count = 0; - while (count < messageQueueChooser.getMessageQueues().size()) { - MessageQueue messageQueue; - synchronized (this.consumerMonitor) { - messageQueue = messageQueueChooser.choose(); - messageQueueChooser.increment(); - } - - long offset = consumer.fetchConsumeOffset(messageQueue, - rocketMQConsumerProperties.getExtension().isFromStore()); - - log.debug("topic='{}', group='{}', messageQueue='{}', offset now='{}'", - this.topic, this.group, messageQueue, offset); - - PullResult pullResult; - if (messageSelector != null) { - pullResult = consumer.pull(messageQueue, messageSelector, offset, 1); - } - else { - pullResult = consumer.pull(messageQueue, (String) null, offset, 1); - } - - if (pullResult.getPullStatus() == PullStatus.FOUND) { - List messageExtList = pullResult.getMsgFoundList(); - - Message message = RocketMQUtil - .convertToSpringMessage(messageExtList.get(0)); - - AcknowledgmentCallback ackCallback = this.ackCallbackFactory - .createCallback(new RocketMQAckInfo(messageQueue, pullResult, - consumer, offset)); - - Message messageResult = MessageBuilder.fromMessage(message).setHeader( - IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK, - ackCallback).build(); - return messageResult; - } - else { - log.debug("messageQueue='{}' PullResult='{}' with topic `{}`", - messageQueueChooser.getMessageQueues(), - pullResult.getPullStatus(), topic); - } - count++; - } - } - catch (Exception e) { - log.error("Consumer pull error: " + e.getMessage(), e); - } - return null; - } - - @Override - public String getComponentType() { - return "rocketmq:message-source"; - } - - public synchronized void setRunning(boolean running) { - this.running = running; - } - - public synchronized void resetMessageQueues(Set queueSet) { - log.info("resetMessageQueues, topic='{}', messageQueue=`{}`", topic, queueSet); - synchronized (this.consumerMonitor) { - this.messageQueueChooser.reset(queueSet); - } - } - - public static class RocketMQCallbackFactory - implements AcknowledgmentCallbackFactory { - - @Override - public AcknowledgmentCallback createCallback(RocketMQAckInfo info) { - return new RocketMQAckCallback(info); - } - - } - - public static class RocketMQAckCallback implements AcknowledgmentCallback { - - private final RocketMQAckInfo ackInfo; - - private boolean acknowledged; - - private boolean autoAckEnabled = true; - - public RocketMQAckCallback(RocketMQAckInfo ackInfo) { - this.ackInfo = ackInfo; - } - - protected void setAcknowledged(boolean acknowledged) { - this.acknowledged = acknowledged; - } - - @Override - public boolean isAcknowledged() { - return this.acknowledged; - } - - @Override - public void noAutoAck() { - this.autoAckEnabled = false; - } - - @Override - public boolean isAutoAck() { - return this.autoAckEnabled; - } - - @Override - public void acknowledge(Status status) { - Assert.notNull(status, "'status' cannot be null"); - if (this.acknowledged) { - throw new IllegalStateException("Already acknowledged"); - } - log.debug("acknowledge(" + status.name() + ") for " + this); - synchronized (this.ackInfo.getConsumerMonitor()) { - try { - switch (status) { - case ACCEPT: - case REJECT: - ackInfo.getConsumer().updateConsumeOffset( - ackInfo.getMessageQueue(), - ackInfo.getPullResult().getNextBeginOffset()); - log.debug("messageQueue='{}' offset update to `{}`", - ackInfo.getMessageQueue(), String.valueOf( - ackInfo.getPullResult().getNextBeginOffset())); - break; - case REQUEUE: - // decrease index and update offset of messageQueue of ackInfo - int oldIndex = ackInfo.getMessageQueueChooser().requeue(); - ackInfo.getConsumer().updateConsumeOffset( - ackInfo.getMessageQueue(), ackInfo.getOldOffset()); - log.debug( - "messageQueue='{}' offset requeue to index:`{}`, oldOffset:'{}'", - ackInfo.getMessageQueue(), oldIndex, - ackInfo.getOldOffset()); - break; - default: - break; - } - } - catch (MQClientException e) { - log.error("acknowledge error: " + e.getErrorMessage(), e); - } - finally { - this.acknowledged = true; - } - } - } - - @Override - public String toString() { - return "RocketMQAckCallback{" + "ackInfo=" + ackInfo + ", acknowledged=" - + acknowledged + ", autoAckEnabled=" + autoAckEnabled + '}'; - } - - } - - public class RocketMQAckInfo { - - private final MessageQueue messageQueue; - - private final PullResult pullResult; - - private final DefaultMQPullConsumer consumer; - - private final long oldOffset; - - public RocketMQAckInfo(MessageQueue messageQueue, PullResult pullResult, - DefaultMQPullConsumer consumer, long oldOffset) { - this.messageQueue = messageQueue; - this.pullResult = pullResult; - this.consumer = consumer; - this.oldOffset = oldOffset; - } - - public MessageQueue getMessageQueue() { - return messageQueue; - } - - public PullResult getPullResult() { - return pullResult; - } - - public DefaultMQPullConsumer getConsumer() { - return consumer; - } - - public RocketMQMessageQueueChooser getMessageQueueChooser() { - return RocketMQMessageSource.this.messageQueueChooser; - } - - public long getOldOffset() { - return oldOffset; - } - - public Object getConsumerMonitor() { - return RocketMQMessageSource.this.consumerMonitor; - } - - @Override - public String toString() { - return "RocketMQAckInfo{" + "messageQueue=" + messageQueue + ", pullResult=" - + pullResult + ", consumer=" + consumer + ", oldOffset=" + oldOffset - + '}'; - } - - } - -} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQConsumerFactory.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQConsumerFactory.java new file mode 100644 index 000000000..b4e5b1529 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQConsumerFactory.java @@ -0,0 +1,163 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.integration.inbound; + +import com.alibaba.cloud.stream.binder.rocketmq.custom.RocketMQBeanContainerCache; +import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties; +import com.alibaba.cloud.stream.binder.rocketmq.utils.RocketMQUtils; +import org.apache.rocketmq.acl.common.AclClientRPCHook; +import org.apache.rocketmq.acl.common.SessionCredentials; +import org.apache.rocketmq.client.consumer.AllocateMessageQueueStrategy; +import org.apache.rocketmq.client.consumer.DefaultLitePullConsumer; +import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer; +import org.apache.rocketmq.client.consumer.rebalance.AllocateMessageQueueAveragely; +import org.apache.rocketmq.common.protocol.heartbeat.MessageModel; +import org.apache.rocketmq.remoting.RPCHook; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.cloud.stream.binder.ExtendedConsumerProperties; +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * Extended function related to producer . eg:initial + * + * @author zkzlx + */ +public final class RocketMQConsumerFactory { + + private RocketMQConsumerFactory() { + } + + private final static Logger log = LoggerFactory + .getLogger(RocketMQConsumerFactory.class); + + public static DefaultMQPushConsumer initPushConsumer( + ExtendedConsumerProperties extendedConsumerProperties) { + RocketMQConsumerProperties consumerProperties = extendedConsumerProperties + .getExtension(); + Assert.notNull(consumerProperties.getGroup(), + "Property 'group' is required - consumerGroup"); + Assert.notNull(consumerProperties.getNameServer(), + "Property 'nameServer' is required"); + AllocateMessageQueueStrategy allocateMessageQueueStrategy = RocketMQBeanContainerCache + .getBean(consumerProperties.getAllocateMessageQueueStrategy(), + AllocateMessageQueueStrategy.class, + new AllocateMessageQueueAveragely()); + RPCHook rpcHook = null; + if (StringUtils.hasLength(consumerProperties.getAccessKey()) + && StringUtils.hasLength(consumerProperties.getSecretKey())) { + rpcHook = new AclClientRPCHook( + new SessionCredentials(consumerProperties.getAccessKey(), + consumerProperties.getSecretKey())); + } + DefaultMQPushConsumer consumer = new DefaultMQPushConsumer( + consumerProperties.getGroup(), rpcHook, allocateMessageQueueStrategy, + consumerProperties.getEnableMsgTrace(), + consumerProperties.getCustomizedTraceTopic()); + consumer.setVipChannelEnabled( + null == rpcHook && consumerProperties.getVipChannelEnabled()); + consumer.setInstanceName( + RocketMQUtils.getInstanceName(rpcHook, consumerProperties.getGroup())); + consumer.setNamespace(consumerProperties.getNamespace()); + consumer.setNamesrvAddr(consumerProperties.getNameServer()); + consumer.setMessageModel(getMessageModel(consumerProperties.getMessageModel())); + consumer.setUseTLS(consumerProperties.getUseTLS()); + consumer.setPullTimeDelayMillsWhenException( + consumerProperties.getPullTimeDelayMillsWhenException()); + consumer.setPullBatchSize(consumerProperties.getPullBatchSize()); + consumer.setConsumeFromWhere(consumerProperties.getConsumeFromWhere()); + consumer.setHeartbeatBrokerInterval( + consumerProperties.getHeartbeatBrokerInterval()); + consumer.setPersistConsumerOffsetInterval( + consumerProperties.getPersistConsumerOffsetInterval()); + consumer.setPullInterval(consumerProperties.getPush().getPullInterval()); + consumer.setConsumeThreadMin(extendedConsumerProperties.getConcurrency()); + consumer.setConsumeThreadMax(extendedConsumerProperties.getConcurrency()); + consumer.setUnitName(consumerProperties.getUnitName()); + return consumer; + } + + /** + * todo Compatible with versions less than 4.6 ? + * @param extendedConsumerProperties extendedConsumerProperties + * @return DefaultLitePullConsumer + */ + public static DefaultLitePullConsumer initPullConsumer( + ExtendedConsumerProperties extendedConsumerProperties) { + RocketMQConsumerProperties consumerProperties = extendedConsumerProperties + .getExtension(); + Assert.notNull(consumerProperties.getGroup(), + "Property 'group' is required - consumerGroup"); + Assert.notNull(consumerProperties.getNameServer(), + "Property 'nameServer' is required"); + AllocateMessageQueueStrategy allocateMessageQueueStrategy = RocketMQBeanContainerCache + .getBean(consumerProperties.getAllocateMessageQueueStrategy(), + AllocateMessageQueueStrategy.class); + + RPCHook rpcHook = null; + if (StringUtils.hasLength(consumerProperties.getAccessKey()) + && StringUtils.hasLength(consumerProperties.getSecretKey())) { + rpcHook = new AclClientRPCHook( + new SessionCredentials(consumerProperties.getAccessKey(), + consumerProperties.getSecretKey())); + } + + DefaultLitePullConsumer consumer = new DefaultLitePullConsumer( + consumerProperties.getNamespace(), consumerProperties.getGroup(), + rpcHook); + consumer.setVipChannelEnabled( + null == rpcHook && consumerProperties.getVipChannelEnabled()); + consumer.setInstanceName( + RocketMQUtils.getInstanceName(rpcHook, consumerProperties.getGroup())); + if (null != allocateMessageQueueStrategy) { + consumer.setAllocateMessageQueueStrategy(allocateMessageQueueStrategy); + } + consumer.setNamesrvAddr(consumerProperties.getNameServer()); + consumer.setMessageModel(getMessageModel(consumerProperties.getMessageModel())); + consumer.setUseTLS(consumerProperties.getUseTLS()); + consumer.setPullTimeDelayMillsWhenException( + consumerProperties.getPullTimeDelayMillsWhenException()); + consumer.setConsumerTimeoutMillisWhenSuspend( + consumerProperties.getPull().getConsumerTimeoutMillisWhenSuspend()); + consumer.setPullBatchSize(consumerProperties.getPullBatchSize()); + consumer.setConsumeFromWhere(consumerProperties.getConsumeFromWhere()); + consumer.setHeartbeatBrokerInterval( + consumerProperties.getHeartbeatBrokerInterval()); + consumer.setPersistConsumerOffsetInterval( + consumerProperties.getPersistConsumerOffsetInterval()); + consumer.setPollTimeoutMillis( + consumerProperties.getPull().getPollTimeoutMillis()); + consumer.setPullThreadNums(extendedConsumerProperties.getConcurrency()); + // The internal queues are cached by a maximum of 1000 + consumer.setPullThresholdForAll(extendedConsumerProperties.getExtension() + .getPull().getPullThresholdForAll()); + consumer.setUnitName(consumerProperties.getUnitName()); + return consumer; + } + + private static MessageModel getMessageModel(String messageModel) { + for (MessageModel model : MessageModel.values()) { + if (model.getModeCN().equalsIgnoreCase(messageModel)) { + return model; + } + } + return MessageModel.CLUSTERING; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQInboundChannelAdapter.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQInboundChannelAdapter.java new file mode 100644 index 000000000..1941e8cdd --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQInboundChannelAdapter.java @@ -0,0 +1,230 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.integration.inbound; + +import java.util.List; +import java.util.function.Supplier; + +import com.alibaba.cloud.stream.binder.rocketmq.metrics.Instrumentation; +import com.alibaba.cloud.stream.binder.rocketmq.metrics.InstrumentationManager; +import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties; +import com.alibaba.cloud.stream.binder.rocketmq.support.RocketMQMessageConverterSupport; +import com.alibaba.cloud.stream.binder.rocketmq.utils.RocketMQUtils; +import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer; +import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus; +import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyStatus; +import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently; +import org.apache.rocketmq.client.consumer.listener.MessageListenerOrderly; +import org.apache.rocketmq.common.message.MessageExt; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.cloud.stream.binder.ExtendedConsumerProperties; +import org.springframework.integration.context.OrderlyShutdownCapable; +import org.springframework.integration.endpoint.MessageProducerSupport; +import org.springframework.integration.support.MessageBuilder; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessagingException; +import org.springframework.retry.RecoveryCallback; +import org.springframework.retry.RetryCallback; +import org.springframework.retry.RetryContext; +import org.springframework.retry.RetryListener; +import org.springframework.retry.support.RetryTemplate; +import org.springframework.util.Assert; +import org.springframework.util.CollectionUtils; + +/** + * @author Jim + */ +public class RocketMQInboundChannelAdapter extends MessageProducerSupport + implements OrderlyShutdownCapable { + + private static final Logger log = LoggerFactory + .getLogger(RocketMQInboundChannelAdapter.class); + + private RetryTemplate retryTemplate; + + private RecoveryCallback recoveryCallback; + + private DefaultMQPushConsumer pushConsumer; + + private final String topic; + + private final ExtendedConsumerProperties extendedConsumerProperties; + + public RocketMQInboundChannelAdapter(String topic, + ExtendedConsumerProperties extendedConsumerProperties) { + this.topic = topic; + this.extendedConsumerProperties = extendedConsumerProperties; + } + + @Override + protected void onInit() { + if (extendedConsumerProperties.getExtension() == null + || !extendedConsumerProperties.getExtension().getEnabled()) { + return; + } + try { + super.onInit(); + if (this.retryTemplate != null) { + Assert.state(getErrorChannel() == null, + "Cannot have an 'errorChannel' property when a 'RetryTemplate' is " + + "provided; use an 'ErrorMessageSendingRecoverer' in the 'recoveryCallback' property to " + + "send an error message when retries are exhausted"); + this.retryTemplate.registerListener(new RetryListener() { + @Override + public boolean open(RetryContext context, + RetryCallback callback) { + return true; + } + + @Override + public void close(RetryContext context, + RetryCallback callback, Throwable throwable) { + } + + @Override + public void onError(RetryContext context, + RetryCallback callback, Throwable throwable) { + } + }); + } + pushConsumer = RocketMQConsumerFactory + .initPushConsumer(extendedConsumerProperties); + // prepare register consumer message listener,the next step is to be + // compatible with a custom MessageListener. + if (extendedConsumerProperties.getExtension().getPush().getOrderly()) { + pushConsumer.registerMessageListener((MessageListenerOrderly) (msgs, + context) -> RocketMQInboundChannelAdapter.this + .consumeMessage(msgs, () -> { + context.setSuspendCurrentQueueTimeMillis( + extendedConsumerProperties.getExtension() + .getPush() + .getSuspendCurrentQueueTimeMillis()); + return ConsumeOrderlyStatus.SUSPEND_CURRENT_QUEUE_A_MOMENT; + }, () -> ConsumeOrderlyStatus.SUCCESS)); + } + else { + pushConsumer.registerMessageListener((MessageListenerConcurrently) (msgs, + context) -> RocketMQInboundChannelAdapter.this + .consumeMessage(msgs, () -> { + context.setDelayLevelWhenNextConsume( + extendedConsumerProperties.getExtension() + .getPush() + .getDelayLevelWhenNextConsume()); + return ConsumeConcurrentlyStatus.RECONSUME_LATER; + }, () -> ConsumeConcurrentlyStatus.CONSUME_SUCCESS)); + } + } + catch (Exception e) { + log.error("DefaultMQPushConsumer init failed, Caused by " + e.getMessage()); + throw new MessagingException(MessageBuilder.withPayload( + "DefaultMQPushConsumer init failed, Caused by " + e.getMessage()) + .build(), e); + } + } + + /** + * The actual execution of a user-defined input consumption service method. + * @param messageExtList rocket mq message list + * @param failSupplier {@link ConsumeConcurrentlyStatus} or + * {@link ConsumeOrderlyStatus} + * @param sucSupplier {@link ConsumeConcurrentlyStatus} or + * {@link ConsumeOrderlyStatus} + * @param object + * @return R + */ + private R consumeMessage(List messageExtList, + Supplier failSupplier, Supplier sucSupplier) { + if (CollectionUtils.isEmpty(messageExtList)) { + throw new MessagingException( + "DefaultMQPushConsumer consuming failed, Caused by messageExtList is empty"); + } + for (MessageExt messageExt : messageExtList) { + try { + Message message = RocketMQMessageConverterSupport + .convertMessage2Spring(messageExt); + if (this.retryTemplate != null) { + this.retryTemplate.execute(context -> { + this.sendMessage(message); + return message; + }, this.recoveryCallback); + } + else { + this.sendMessage(message); + } + } + catch (Exception e) { + log.warn("consume message failed. messageExt:{}", messageExt, e); + return failSupplier.get(); + } + } + return sucSupplier.get(); + } + + @Override + protected void doStart() { + if (extendedConsumerProperties.getExtension() == null + || !extendedConsumerProperties.getExtension().getEnabled()) { + return; + } + Instrumentation instrumentation = new Instrumentation(topic, this); + try { + pushConsumer.subscribe(topic, RocketMQUtils.getMessageSelector( + extendedConsumerProperties.getExtension().getSubscription())); + pushConsumer.start(); + instrumentation.markStartedSuccessfully(); + } + catch (Exception e) { + instrumentation.markStartFailed(e); + log.error("DefaultMQPushConsumer init failed, Caused by " + e.getMessage()); + throw new MessagingException(MessageBuilder.withPayload( + "DefaultMQPushConsumer init failed, Caused by " + e.getMessage()) + .build(), e); + } + finally { + InstrumentationManager.addHealthInstrumentation(instrumentation); + } + } + + @Override + protected void doStop() { + if (pushConsumer != null) { + pushConsumer.shutdown(); + } + } + + public void setRetryTemplate(RetryTemplate retryTemplate) { + this.retryTemplate = retryTemplate; + } + + public void setRecoveryCallback(RecoveryCallback recoveryCallback) { + this.recoveryCallback = recoveryCallback; + } + + @Override + public int beforeShutdown() { + this.stop(); + return 0; + } + + @Override + public int afterShutdown() { + return 0; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/pull/DefaultErrorAcknowledgeHandler.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/pull/DefaultErrorAcknowledgeHandler.java new file mode 100644 index 000000000..458324741 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/pull/DefaultErrorAcknowledgeHandler.java @@ -0,0 +1,44 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.integration.inbound.pull; + +import com.alibaba.cloud.stream.binder.rocketmq.extend.ErrorAcknowledgeHandler; + +import org.springframework.integration.acks.AcknowledgmentCallback.Status; +import org.springframework.messaging.Message; + +/** + * By default, if consumption fails, the corresponding MessageQueue will always be + * retried, that is, the consumption of other messages in the MessageQueue will be + * blocked. + * + * @author zkzlx + */ +public class DefaultErrorAcknowledgeHandler implements ErrorAcknowledgeHandler { + + /** + * Ack state handling, including receive, reject, and retry, when a consumption + * exception occurs. + * @param message message + * @return see {@link Status} + */ + @Override + public Status handler(Message message) { + return Status.REQUEUE; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/pull/RocketMQAckCallback.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/pull/RocketMQAckCallback.java new file mode 100644 index 000000000..12540af59 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/pull/RocketMQAckCallback.java @@ -0,0 +1,100 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.integration.inbound.pull; + +import org.apache.rocketmq.client.consumer.DefaultLitePullConsumer; +import org.apache.rocketmq.client.exception.MQClientException; +import org.apache.rocketmq.common.message.MessageExt; +import org.apache.rocketmq.common.message.MessageQueue; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.integration.acks.AcknowledgmentCallback; +import org.springframework.util.Assert; + +/** + * A pollable {@link org.springframework.integration.core.MessageSource} for RocketMQ. + * + * @author zkzlx + */ +public class RocketMQAckCallback implements AcknowledgmentCallback { + + private final static Logger log = LoggerFactory.getLogger(RocketMQAckCallback.class); + + private boolean acknowledged; + + private boolean autoAckEnabled = true; + + private MessageExt messageExt; + + private DefaultLitePullConsumer consumer; + + private final MessageQueue messageQueue; + + public RocketMQAckCallback(DefaultLitePullConsumer consumer, + MessageQueue messageQueue, MessageExt messageExt) { + this.messageExt = messageExt; + this.consumer = consumer; + this.messageQueue = messageQueue; + } + + @Override + public boolean isAcknowledged() { + return this.acknowledged; + } + + @Override + public void noAutoAck() { + this.autoAckEnabled = false; + } + + @Override + public boolean isAutoAck() { + return this.autoAckEnabled; + } + + @Override + public void acknowledge(Status status) { + Assert.notNull(status, "'status' cannot be null"); + if (this.acknowledged) { + throw new IllegalStateException("Already acknowledged"); + } + synchronized (messageQueue) { + try { + long offset = messageExt.getQueueOffset(); + switch (status) { + case REJECT: + case ACCEPT: + consumer.committed(messageQueue); + break; + case REQUEUE: + consumer.seek(messageQueue, offset); + break; + default: + break; + } + } + catch (MQClientException e) { + throw new IllegalStateException(e); + } + finally { + this.acknowledged = true; + } + } + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/pull/RocketMQMessageSource.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/pull/RocketMQMessageSource.java new file mode 100644 index 000000000..d0f6c6234 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/pull/RocketMQMessageSource.java @@ -0,0 +1,174 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.integration.inbound.pull; + +import java.util.Collection; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; + +import com.alibaba.cloud.stream.binder.rocketmq.integration.inbound.RocketMQConsumerFactory; +import com.alibaba.cloud.stream.binder.rocketmq.metrics.Instrumentation; +import com.alibaba.cloud.stream.binder.rocketmq.metrics.InstrumentationManager; +import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties; +import com.alibaba.cloud.stream.binder.rocketmq.support.RocketMQMessageConverterSupport; +import com.alibaba.cloud.stream.binder.rocketmq.utils.RocketMQUtils; +import org.apache.rocketmq.client.consumer.DefaultLitePullConsumer; +import org.apache.rocketmq.client.consumer.MessageSelector; +import org.apache.rocketmq.client.exception.MQClientException; +import org.apache.rocketmq.common.message.MessageExt; +import org.apache.rocketmq.common.message.MessageQueue; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.beans.factory.DisposableBean; +import org.springframework.cloud.stream.binder.ExtendedConsumerProperties; +import org.springframework.context.Lifecycle; +import org.springframework.integration.IntegrationMessageHeaderAccessor; +import org.springframework.integration.endpoint.AbstractMessageSource; +import org.springframework.integration.support.MessageBuilder; +import org.springframework.messaging.Message; +import org.springframework.util.CollectionUtils; + +/** + * @author Jim + */ +public class RocketMQMessageSource extends AbstractMessageSource + implements DisposableBean, Lifecycle { + + private final static Logger log = LoggerFactory + .getLogger(RocketMQMessageSource.class); + + private DefaultLitePullConsumer consumer; + + private final Map> messageQueuesForTopic = new ConcurrentHashMap<>(); + + private volatile boolean running; + + private final String topic; + + private final MessageSelector messageSelector; + + private final ExtendedConsumerProperties extendedConsumerProperties; + + private volatile Iterator messageExtIterator = null; + + public RocketMQMessageSource(String name, + ExtendedConsumerProperties extendedConsumerProperties) { + this.topic = name; + this.messageSelector = RocketMQUtils.getMessageSelector( + extendedConsumerProperties.getExtension().getSubscription()); + this.extendedConsumerProperties = extendedConsumerProperties; + + } + + @Override + public synchronized void start() { + Instrumentation instrumentation = new Instrumentation(topic, this); + try { + if (this.isRunning()) { + throw new IllegalStateException( + "pull consumer already running. " + this.toString()); + } + this.consumer = RocketMQConsumerFactory + .initPullConsumer(extendedConsumerProperties); + // This parameter must be 1, otherwise doReceive cannot be handled singly. + // this.consumer.setPullBatchSize(1); + this.consumer.subscribe(topic, messageSelector); + this.consumer.setAutoCommit(false); + // register TopicMessageQueueChangeListener for messageQueuesForTopic + consumer.registerTopicMessageQueueChangeListener(topic, + messageQueuesForTopic::put); + this.consumer.start(); + // Initialize messageQueuesForTopic immediately + messageQueuesForTopic.put(topic, consumer.fetchMessageQueues(topic)); + instrumentation.markStartedSuccessfully(); + } + catch (MQClientException e) { + instrumentation.markStartFailed(e); + log.error("DefaultMQPullConsumer startup error: " + e.getMessage(), e); + } + finally { + InstrumentationManager.addHealthInstrumentation(instrumentation); + } + this.running = true; + } + + private MessageQueue acquireCurrentMessageQueue(String topic, int queueId) { + Collection messageQueueSet = messageQueuesForTopic.get(topic); + if (CollectionUtils.isEmpty(messageQueueSet)) { + return null; + } + for (MessageQueue messageQueue : messageQueueSet) { + if (messageQueue.getQueueId() == queueId) { + return messageQueue; + } + } + return null; + } + + @Override + public synchronized void stop() { + if (this.isRunning() && null != consumer) { + consumer.unsubscribe(topic); + consumer.shutdown(); + this.running = false; + } + } + + @Override + public synchronized boolean isRunning() { + return running; + } + + @Override + protected synchronized Object doReceive() { + if (messageExtIterator == null) { + List messageExtList = consumer.poll(); + if (CollectionUtils.isEmpty(messageExtList)) { + return null; + } + messageExtIterator = messageExtList.iterator(); + } + MessageExt messageExt = messageExtIterator.next(); + if (!messageExtIterator.hasNext()) { + messageExtIterator = null; + } + if (null == messageExt) { + return null; + } + MessageQueue messageQueue = this.acquireCurrentMessageQueue(messageExt.getTopic(), + messageExt.getQueueId()); + if (messageQueue == null) { + throw new IllegalArgumentException( + "The message queue is not in assigned list"); + } + Message message = RocketMQMessageConverterSupport + .convertMessage2Spring(messageExt); + return MessageBuilder.fromMessage(message) + .setHeader(IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK, + new RocketMQAckCallback(this.consumer, messageQueue, messageExt)) + .build(); + } + + @Override + public String getComponentType() { + return "rocketmq:message-source"; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProduceFactory.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProduceFactory.java new file mode 100644 index 000000000..d50ddf52d --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProduceFactory.java @@ -0,0 +1,137 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.integration.outbound; + +import java.lang.reflect.Field; + +import com.alibaba.cloud.stream.binder.rocketmq.custom.RocketMQBeanContainerCache; +import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties; +import com.alibaba.cloud.stream.binder.rocketmq.utils.RocketMQUtils; +import org.apache.rocketmq.acl.common.AclClientRPCHook; +import org.apache.rocketmq.acl.common.SessionCredentials; +import org.apache.rocketmq.client.hook.CheckForbiddenHook; +import org.apache.rocketmq.client.hook.SendMessageHook; +import org.apache.rocketmq.client.producer.DefaultMQProducer; +import org.apache.rocketmq.client.producer.TransactionMQProducer; +import org.apache.rocketmq.client.trace.AsyncTraceDispatcher; +import org.apache.rocketmq.client.trace.TraceDispatcher; +import org.apache.rocketmq.client.trace.hook.SendMessageTraceHookImpl; +import org.apache.rocketmq.common.UtilAll; +import org.apache.rocketmq.remoting.RPCHook; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.util.Assert; +import org.springframework.util.StringUtils; + +/** + * Extended function related to producer . eg:initial . + * + * @author zkzlx + */ +public final class RocketMQProduceFactory { + + private RocketMQProduceFactory() { + } + + private final static Logger log = LoggerFactory + .getLogger(RocketMQProduceFactory.class); + + /** + * init for the producer,including convert producer params. + * @param topic topic + * @param producerProperties producerProperties + * @return DefaultMQProducer + */ + public static DefaultMQProducer initRocketMQProducer(String topic, + RocketMQProducerProperties producerProperties) { + Assert.notNull(producerProperties.getGroup(), + "Property 'group' is required - producerGroup"); + Assert.notNull(producerProperties.getNameServer(), + "Property 'nameServer' is required"); + + RPCHook rpcHook = null; + if (StringUtils.hasLength(producerProperties.getAccessKey()) + && StringUtils.hasLength(producerProperties.getSecretKey())) { + rpcHook = new AclClientRPCHook( + new SessionCredentials(producerProperties.getAccessKey(), + producerProperties.getSecretKey())); + } + DefaultMQProducer producer; + if (RocketMQProducerProperties.ProducerType.Trans + .equalsName(producerProperties.getProducerType())) { + producer = new TransactionMQProducer(producerProperties.getNamespace(), + producerProperties.getGroup(), rpcHook); + if (producerProperties.getEnableMsgTrace()) { + try { + AsyncTraceDispatcher dispatcher = new AsyncTraceDispatcher( + producerProperties.getGroup(), TraceDispatcher.Type.PRODUCE, + producerProperties.getCustomizedTraceTopic(), rpcHook); + dispatcher.setHostProducer(producer.getDefaultMQProducerImpl()); + Field field = DefaultMQProducer.class + .getDeclaredField("traceDispatcher"); + field.setAccessible(true); + field.set(producer, dispatcher); + producer.getDefaultMQProducerImpl().registerSendMessageHook( + new SendMessageTraceHookImpl(dispatcher)); + } + catch (Throwable e) { + log.error( + "system mq-trace hook init failed ,maybe can't send msg trace data"); + } + } + } + else { + producer = new DefaultMQProducer(producerProperties.getNamespace(), + producerProperties.getGroup(), rpcHook, + producerProperties.getEnableMsgTrace(), + producerProperties.getCustomizedTraceTopic()); + } + + producer.setVipChannelEnabled( + null == rpcHook && producerProperties.getVipChannelEnabled()); + producer.setInstanceName( + RocketMQUtils.getInstanceName(rpcHook, topic + "|" + UtilAll.getPid())); + producer.setNamesrvAddr(producerProperties.getNameServer()); + producer.setSendMsgTimeout(producerProperties.getSendMsgTimeout()); + producer.setRetryTimesWhenSendFailed( + producerProperties.getRetryTimesWhenSendFailed()); + producer.setRetryTimesWhenSendAsyncFailed( + producerProperties.getRetryTimesWhenSendAsyncFailed()); + producer.setCompressMsgBodyOverHowmuch( + producerProperties.getCompressMsgBodyThreshold()); + producer.setRetryAnotherBrokerWhenNotStoreOK( + producerProperties.getRetryAnotherBroker()); + producer.setMaxMessageSize(producerProperties.getMaxMessageSize()); + producer.setUseTLS(producerProperties.getUseTLS()); + producer.setUnitName(producerProperties.getUnitName()); + CheckForbiddenHook checkForbiddenHook = RocketMQBeanContainerCache.getBean( + producerProperties.getCheckForbiddenHook(), CheckForbiddenHook.class); + if (null != checkForbiddenHook) { + producer.getDefaultMQProducerImpl() + .registerCheckForbiddenHook(checkForbiddenHook); + } + SendMessageHook sendMessageHook = RocketMQBeanContainerCache + .getBean(producerProperties.getSendMessageHook(), SendMessageHook.class); + if (null != sendMessageHook) { + producer.getDefaultMQProducerImpl().registerSendMessageHook(sendMessageHook); + } + + return producer; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProducerMessageHandler.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProducerMessageHandler.java new file mode 100644 index 000000000..35450cb1c --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProducerMessageHandler.java @@ -0,0 +1,301 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.integration.outbound; + +import java.util.List; + +import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst; +import com.alibaba.cloud.stream.binder.rocketmq.custom.RocketMQBeanContainerCache; +import com.alibaba.cloud.stream.binder.rocketmq.metrics.Instrumentation; +import com.alibaba.cloud.stream.binder.rocketmq.metrics.InstrumentationManager; +import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties; +import com.alibaba.cloud.stream.binder.rocketmq.provisioning.selector.PartitionMessageQueueSelector; +import com.alibaba.cloud.stream.binder.rocketmq.support.RocketMQMessageConverterSupport; +import org.apache.rocketmq.client.exception.MQBrokerException; +import org.apache.rocketmq.client.exception.MQClientException; +import org.apache.rocketmq.client.producer.DefaultMQProducer; +import org.apache.rocketmq.client.producer.MessageQueueSelector; +import org.apache.rocketmq.client.producer.SendCallback; +import org.apache.rocketmq.client.producer.SendResult; +import org.apache.rocketmq.client.producer.SendStatus; +import org.apache.rocketmq.client.producer.TransactionListener; +import org.apache.rocketmq.client.producer.TransactionMQProducer; +import org.apache.rocketmq.common.message.MessageQueue; +import org.apache.rocketmq.remoting.exception.RemotingException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.cloud.stream.binder.ExtendedProducerProperties; +import org.springframework.cloud.stream.binding.MessageConverterConfigurer; +import org.springframework.cloud.stream.binding.MessageConverterConfigurer.PartitioningInterceptor; +import org.springframework.cloud.stream.provisioning.ProducerDestination; +import org.springframework.context.Lifecycle; +import org.springframework.integration.handler.AbstractMessageHandler; +import org.springframework.integration.support.ErrorMessageStrategy; +import org.springframework.integration.support.ErrorMessageUtils; +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageChannel; +import org.springframework.messaging.MessagingException; + +/** + * @author Jim + */ +public class RocketMQProducerMessageHandler extends AbstractMessageHandler + implements Lifecycle { + + private final static Logger log = LoggerFactory + .getLogger(RocketMQProducerMessageHandler.class); + + private volatile boolean running = false; + + private volatile boolean isTrans = false; + + private ErrorMessageStrategy errorMessageStrategy; + + private MessageChannel sendFailureChannel; + + private MessageConverterConfigurer.PartitioningInterceptor partitioningInterceptor; + + private DefaultMQProducer defaultMQProducer; + + private MessageQueueSelector messageQueueSelector; + + private final ProducerDestination destination; + + private final ExtendedProducerProperties extendedProducerProperties; + + private final RocketMQProducerProperties mqProducerProperties; + + public RocketMQProducerMessageHandler(ProducerDestination destination, + ExtendedProducerProperties extendedProducerProperties, + RocketMQProducerProperties mqProducerProperties) { + this.destination = destination; + this.extendedProducerProperties = extendedProducerProperties; + this.mqProducerProperties = mqProducerProperties; + } + + @Override + protected void onInit() { + if (null == mqProducerProperties || !mqProducerProperties.getEnabled()) { + return; + } + super.onInit(); + this.defaultMQProducer = RocketMQProduceFactory + .initRocketMQProducer(destination.getName(), mqProducerProperties); + this.isTrans = defaultMQProducer instanceof TransactionMQProducer; + // Use the default if the partition is on and no customization is available. + this.messageQueueSelector = RocketMQBeanContainerCache.getBean( + mqProducerProperties.getMessageQueueSelector(), + MessageQueueSelector.class, + extendedProducerProperties.isPartitioned() + ? new PartitionMessageQueueSelector() + : null); + } + + @Override + public void start() { + Instrumentation instrumentation = new Instrumentation(destination.getName(), + this); + try { + defaultMQProducer.start(); + // TransactionMQProducer does not currently support custom + // MessageQueueSelector. + if (!isTrans && extendedProducerProperties.isPartitioned()) { + List messageQueues = defaultMQProducer + .fetchPublishMessageQueues(destination.getName()); + if (extendedProducerProperties.getPartitionCount() != messageQueues + .size()) { + logger.info(String.format( + "The partition count of topic '%s' will change from '%s' to '%s'", + destination.getName(), + extendedProducerProperties.getPartitionCount(), + messageQueues.size())); + extendedProducerProperties.setPartitionCount(messageQueues.size()); + // may be npe! + partitioningInterceptor.setPartitionCount( + extendedProducerProperties.getPartitionCount()); + } + } + running = true; + instrumentation.markStartedSuccessfully(); + } + catch (MQClientException | NullPointerException e) { + instrumentation.markStartFailed(e); + log.error("The defaultMQProducer startup failure !!!", e); + } + finally { + InstrumentationManager.addHealthInstrumentation(instrumentation); + } + } + + @Override + public void stop() { + if (running && null != defaultMQProducer) { + defaultMQProducer.shutdown(); + } + running = false; + } + + @Override + public boolean isRunning() { + return running; + } + + @Override + protected void handleMessageInternal(Message message) { + try { + org.apache.rocketmq.common.message.Message mqMessage = RocketMQMessageConverterSupport + .convertMessage2MQ(destination.getName(), message); + SendResult sendResult; + if (defaultMQProducer instanceof TransactionMQProducer) { + TransactionListener transactionListener = RocketMQBeanContainerCache + .getBean(mqProducerProperties.getTransactionListener(), + TransactionListener.class); + if (transactionListener == null) { + throw new MessagingException( + "TransactionMQProducer must have a TransactionListener !!! "); + } + ((TransactionMQProducer) defaultMQProducer) + .setTransactionListener(transactionListener); + if (log.isDebugEnabled()) { + log.debug("send transaction message ->{}", mqMessage); + } + sendResult = defaultMQProducer.sendMessageInTransaction(mqMessage, + message.getHeaders().get(RocketMQConst.USER_TRANSACTIONAL_ARGS)); + } + else { + if (log.isDebugEnabled()) { + log.debug("send message ->{}", mqMessage); + } + sendResult = this.send(mqMessage, this.messageQueueSelector, + message.getHeaders(), message); + } + log.info("the message has sent,message={},sendResult={}", mqMessage, + sendResult); + if (sendResult == null + || !SendStatus.SEND_OK.equals(sendResult.getSendStatus())) { + log.error("message send fail.SendStatus is not OK.the message={}", + mqMessage); + this.doFail(message, new MessagingException( + "message send fail.SendStatus is not OK.")); + } + } + catch (Exception e) { + log.error("RocketMQ Message hasn't been sent. Caused by " + e.getMessage(), + e); + this.doFail(message, e); + } + } + + private SendResult send(org.apache.rocketmq.common.message.Message mqMessage, + MessageQueueSelector selector, Object args, Message message) + throws RemotingException, MQClientException, InterruptedException, + MQBrokerException { + SendResult sendResult = new SendResult(); + sendResult.setSendStatus(SendStatus.SEND_OK); + if (RocketMQProducerProperties.SendType.OneWay + .equalsName(mqProducerProperties.getSendType())) { + if (null != selector) { + defaultMQProducer.sendOneway(mqMessage, selector, args); + } + else { + defaultMQProducer.sendOneway(mqMessage); + } + return sendResult; + } + if (RocketMQProducerProperties.SendType.Sync + .equalsName(mqProducerProperties.getSendType())) { + if (null != selector) { + return defaultMQProducer.send(mqMessage, selector, args); + } + return defaultMQProducer.send(mqMessage); + } + if (RocketMQProducerProperties.SendType.Async + .equalsName(mqProducerProperties.getSendType())) { + if (null != selector) { + defaultMQProducer.send(mqMessage, selector, args, + this.getSendCallback(message)); + } + else { + defaultMQProducer.send(mqMessage, this.getSendCallback(message)); + } + return sendResult; + } + throw new MessagingException( + "message hasn't been sent,cause by : the SendType must be in this values[OneWay, Async, Sync]"); + } + + /** + * https://github.com/alibaba/spring-cloud-alibaba/issues/1408 . + * @param message message + * @return SendCallback + */ + private SendCallback getSendCallback(Message message) { + SendCallback sendCallback = RocketMQBeanContainerCache + .getBean(mqProducerProperties.getSendCallBack(), SendCallback.class); + if (null == sendCallback) { + sendCallback = new SendCallback() { + @Override + public void onSuccess(SendResult sendResult) { + } + + @Override + public void onException(Throwable e) { + RocketMQProducerMessageHandler.this.doFail(message, e); + } + }; + } + return sendCallback; + } + + private void doFail(Message message, Throwable e) { + if (getSendFailureChannel() != null) { + getSendFailureChannel().send(getErrorMessageStrategy().buildErrorMessage(e, + ErrorMessageUtils.getAttributeAccessor(message, message))); + } + else { + throw new MessagingException(message, e); + } + } + + public MessageChannel getSendFailureChannel() { + return sendFailureChannel; + } + + public void setSendFailureChannel(MessageChannel sendFailureChannel) { + this.sendFailureChannel = sendFailureChannel; + } + + public ErrorMessageStrategy getErrorMessageStrategy() { + return errorMessageStrategy; + } + + public void setErrorMessageStrategy(ErrorMessageStrategy errorMessageStrategy) { + this.errorMessageStrategy = errorMessageStrategy; + } + + public PartitioningInterceptor getPartitioningInterceptor() { + return partitioningInterceptor; + } + + public RocketMQProducerMessageHandler setPartitioningInterceptor( + PartitioningInterceptor partitioningInterceptor) { + this.partitioningInterceptor = partitioningInterceptor; + return this; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/metrics/Instrumentation.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/metrics/Instrumentation.java index 080bebf90..e26482855 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/metrics/Instrumentation.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/metrics/Instrumentation.java @@ -16,8 +16,11 @@ package com.alibaba.cloud.stream.binder.rocketmq.metrics; +import java.util.Objects; import java.util.concurrent.atomic.AtomicBoolean; +import org.springframework.context.Lifecycle; + /** * @author Timur Valiev * @author Jim @@ -26,6 +29,8 @@ public class Instrumentation { private final String name; + private Lifecycle actuator; + protected final AtomicBoolean started = new AtomicBoolean(false); protected Exception startException = null; @@ -34,6 +39,19 @@ public class Instrumentation { this.name = name; } + public Instrumentation(String name, Lifecycle actuator) { + this.name = name; + this.actuator = actuator; + } + + public Lifecycle getActuator() { + return actuator; + } + + public void setActuator(Lifecycle actuator) { + this.actuator = actuator; + } + public boolean isDown() { return startException != null; } @@ -67,4 +85,21 @@ public class Instrumentation { return startException; } + @Override + public int hashCode() { + return Objects.hash(getName(), getActuator()); + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + Instrumentation that = (Instrumentation) o; + return name.equals(that.name) && actuator.equals(that.actuator); + } + } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/metrics/InstrumentationManager.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/metrics/InstrumentationManager.java index 8b4939585..ad7958e44 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/metrics/InstrumentationManager.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/metrics/InstrumentationManager.java @@ -16,37 +16,37 @@ package com.alibaba.cloud.stream.binder.rocketmq.metrics; +import java.util.Collection; import java.util.HashMap; import java.util.Map; -import java.util.Set; -import java.util.concurrent.ConcurrentHashMap; -import java.util.stream.Collectors; /** * @author Timur Valiev * @author Jim */ -public class InstrumentationManager { +public final class InstrumentationManager { - private final Map runtime = new ConcurrentHashMap<>(); - - private final Map healthInstrumentations = new HashMap<>(); - - public Set getHealthInstrumentations() { - return healthInstrumentations.entrySet().stream().map(Map.Entry::getValue) - .collect(Collectors.toSet()); + private InstrumentationManager() { } - public void addHealthInstrumentation(Instrumentation instrumentation) { - healthInstrumentations.put(instrumentation.getName(), instrumentation); - } + private static final Map HEALTH_INSTRUMENTATIONS = new HashMap<>(); - public Instrumentation getHealthInstrumentation(String key) { - return healthInstrumentations.get(key); + public static Collection getHealthInstrumentations() { + return HEALTH_INSTRUMENTATIONS.values(); } - public Map getRuntime() { - return runtime; + public static void addHealthInstrumentation(Instrumentation instrumentation) { + if (null != instrumentation) { + HEALTH_INSTRUMENTATIONS.computeIfPresent(instrumentation.hashCode(), + (k, v) -> { + if (instrumentation.getActuator() != null) { + instrumentation.getActuator().stop(); + } + throw new IllegalArgumentException( + "The current actuator exists, please confirm if there is a repeat operation!!!"); + }); + } + } } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQBinderConfigurationProperties.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQBinderConfigurationProperties.java index 3a9dcb403..3da04ea18 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQBinderConfigurationProperties.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQBinderConfigurationProperties.java @@ -16,86 +16,14 @@ package com.alibaba.cloud.stream.binder.rocketmq.properties; -import java.util.Arrays; -import java.util.List; - -import com.alibaba.cloud.stream.binder.rocketmq.RocketMQBinderConstants; -import org.apache.rocketmq.common.MixAll; - import org.springframework.boot.context.properties.ConfigurationProperties; /** - * @author Timur Valiev + * binding rocketMq properties. + * * @author Jim */ @ConfigurationProperties(prefix = "spring.cloud.stream.rocketmq.binder") -public class RocketMQBinderConfigurationProperties { - - /** - * The name server list for rocketMQ. - */ - private List nameServer = Arrays - .asList(RocketMQBinderConstants.DEFAULT_NAME_SERVER); - - /** - * The property of "access-key". - */ - private String accessKey; - - /** - * The property of "secret-key". - */ - private String secretKey; - - /** - * Switch flag instance for message trace. - */ - private boolean enableMsgTrace = true; - - /** - * The name value of message trace topic.If you don't config,you can use the default - * trace topic name. - */ - private String customizedTraceTopic = MixAll.RMQ_SYS_TRACE_TOPIC; - - public List getNameServer() { - return nameServer; - } - - public void setNameServer(List nameServer) { - this.nameServer = nameServer; - } - - public String getAccessKey() { - return accessKey; - } - - public void setAccessKey(String accessKey) { - this.accessKey = accessKey; - } - - public String getSecretKey() { - return secretKey; - } - - public void setSecretKey(String secretKey) { - this.secretKey = secretKey; - } - - public boolean isEnableMsgTrace() { - return enableMsgTrace; - } - - public void setEnableMsgTrace(boolean enableMsgTrace) { - this.enableMsgTrace = enableMsgTrace; - } - - public String getCustomizedTraceTopic() { - return customizedTraceTopic; - } - - public void setCustomizedTraceTopic(String customizedTraceTopic) { - this.customizedTraceTopic = customizedTraceTopic; - } +public class RocketMQBinderConfigurationProperties extends RocketMQCommonProperties { } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQCommonProperties.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQCommonProperties.java new file mode 100644 index 000000000..4973dd759 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQCommonProperties.java @@ -0,0 +1,214 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.properties; + +import java.io.Serializable; + +import org.apache.rocketmq.client.AccessChannel; +import org.apache.rocketmq.client.impl.factory.MQClientInstance; +import org.apache.rocketmq.remoting.netty.TlsSystemConfig; + +/** + * @author zkzlx + */ +public class RocketMQCommonProperties implements Serializable { + + private static final long serialVersionUID = -6724870154343284715L; + + private boolean enabled = true; + + private String nameServer; + + /** + * The property of "access-key". + */ + private String accessKey; + + /** + * The property of "secret-key". + */ + private String secretKey; + + /** + * Consumers of the same role is required to have exactly same subscriptions and + * consumerGroup to correctly achieve load balance. It's required and needs to be + * globally unique. Producer group conceptually aggregates all producer instances of + * exactly same role, which is particularly important when transactional messages are + * involved. For non-transactional messages, it does not matter as long as it's unique + * per process. See here + * for further discussion. + */ + private String group; + + private String namespace; + + /** + * The property of "unitName". + */ + private String unitName; + + private String accessChannel = AccessChannel.LOCAL.name(); + + /** + * Pulling topic information interval from the named server. + * see{@link MQClientInstance#startScheduledTask()},eg:ScheduledTask + * updateTopicRouteInfoFromNameServer. + */ + private int pollNameServerInterval = 1000 * 30; + + /** + * Heartbeat interval in microseconds with message broker. + * see{@link MQClientInstance#startScheduledTask()},eg:ScheduledTask + * sendHeartbeatToAllBroker . + */ + private int heartbeatBrokerInterval = 1000 * 30; + + /** + * Offset persistent interval for consumer. + * see{@link MQClientInstance#startScheduledTask()},eg:ScheduledTask + * sendHeartbeatToAllBroker . + */ + private int persistConsumerOffsetInterval = 1000 * 5; + + private boolean vipChannelEnabled = false; + + private boolean useTLS = TlsSystemConfig.tlsEnable; + + private boolean enableMsgTrace = true; + + private String customizedTraceTopic; + + public boolean getEnabled() { + return enabled; + } + + public void setEnabled(boolean enabled) { + this.enabled = enabled; + } + + public String getNameServer() { + return nameServer; + } + + public void setNameServer(String nameServer) { + this.nameServer = nameServer; + } + + public String getAccessKey() { + return accessKey; + } + + public void setAccessKey(String accessKey) { + this.accessKey = accessKey; + } + + public String getSecretKey() { + return secretKey; + } + + public void setSecretKey(String secretKey) { + this.secretKey = secretKey; + } + + public String getGroup() { + return group; + } + + public void setGroup(String group) { + this.group = group; + } + + public String getNamespace() { + return namespace; + } + + public void setNamespace(String namespace) { + this.namespace = namespace; + } + + public String getAccessChannel() { + return accessChannel; + } + + public void setAccessChannel(String accessChannel) { + this.accessChannel = accessChannel; + } + + public int getPollNameServerInterval() { + return pollNameServerInterval; + } + + public void setPollNameServerInterval(int pollNameServerInterval) { + this.pollNameServerInterval = pollNameServerInterval; + } + + public int getHeartbeatBrokerInterval() { + return heartbeatBrokerInterval; + } + + public void setHeartbeatBrokerInterval(int heartbeatBrokerInterval) { + this.heartbeatBrokerInterval = heartbeatBrokerInterval; + } + + public int getPersistConsumerOffsetInterval() { + return persistConsumerOffsetInterval; + } + + public void setPersistConsumerOffsetInterval(int persistConsumerOffsetInterval) { + this.persistConsumerOffsetInterval = persistConsumerOffsetInterval; + } + + public boolean getVipChannelEnabled() { + return vipChannelEnabled; + } + + public void setVipChannelEnabled(boolean vipChannelEnabled) { + this.vipChannelEnabled = vipChannelEnabled; + } + + public boolean getUseTLS() { + return useTLS; + } + + public void setUseTLS(boolean useTLS) { + this.useTLS = useTLS; + } + + public boolean getEnableMsgTrace() { + return enableMsgTrace; + } + + public void setEnableMsgTrace(boolean enableMsgTrace) { + this.enableMsgTrace = enableMsgTrace; + } + + public String getCustomizedTraceTopic() { + return customizedTraceTopic; + } + + public void setCustomizedTraceTopic(String customizedTraceTopic) { + this.customizedTraceTopic = customizedTraceTopic; + } + + public String getUnitName() { + return unitName; + } + + public void setUnitName(String unitName) { + this.unitName = unitName; + } +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQConsumerProperties.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQConsumerProperties.java index 9c059ec56..ebe2c1c43 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQConsumerProperties.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQConsumerProperties.java @@ -16,153 +16,441 @@ package com.alibaba.cloud.stream.binder.rocketmq.properties; -import java.util.Set; +import java.io.Serializable; -import com.alibaba.cloud.stream.binder.rocketmq.support.JacksonRocketMQHeaderMapper; -import org.apache.rocketmq.client.consumer.MQPushConsumer; -import org.apache.rocketmq.client.consumer.MessageSelector; import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext; +import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyContext; import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently; import org.apache.rocketmq.client.consumer.listener.MessageListenerOrderly; +import org.apache.rocketmq.client.impl.consumer.ConsumeMessageConcurrentlyService; +import org.apache.rocketmq.client.impl.consumer.ConsumeMessageOrderlyService; +import org.apache.rocketmq.common.UtilAll; +import org.apache.rocketmq.common.consumer.ConsumeFromWhere; import org.apache.rocketmq.common.protocol.heartbeat.MessageModel; /** - * @author Timur Valiev + * Extended consumer properties for RocketMQ binder. + * * @author Jim */ -public class RocketMQConsumerProperties { +public class RocketMQConsumerProperties extends RocketMQCommonProperties { /** - * using '||' to split tag {@link MQPushConsumer#subscribe(String, String)}. + * Message model defines the way how messages are delivered to each consumer clients. + * This field defaults to clustering. */ - private String tags; + private String messageModel = MessageModel.CLUSTERING.getModeCN(); /** - * {@link MQPushConsumer#subscribe(String, MessageSelector)} - * {@link MessageSelector#bySql(String)}. + * Queue allocation algorithm specifying how message queues are allocated to each + * consumer clients. */ - private String sql; + private String allocateMessageQueueStrategy; /** - * {@link MessageModel#BROADCASTING}. + * The expressions include tags or SQL,as follow: + *

+ * tag: {@code tag1||tag2||tag3 }; sql: {@code 'color'='blue' AND 'price'>100 } . + *

+ * Determines whether there are specific characters "{@code ||}" in the expression to + * determine how the message is filtered,tags or SQL. */ - private Boolean broadcasting = false; + private String subscription; /** - * if orderly is true, using {@link MessageListenerOrderly} else if orderly if false, - * using {@link MessageListenerConcurrently}. + * Delay some time when exception occur . */ - private Boolean orderly = false; + private long pullTimeDelayMillsWhenException = 1000; /** - * for concurrently listener. message consume retry strategy. see - * {@link ConsumeConcurrentlyContext#delayLevelWhenNextConsume}. -1 means dlq(or - * discard, see {@link this#shouldRequeue}), others means requeue. + * Consuming point on consumer booting. + * + * There are three consuming points: + *

    + *
  • CONSUME_FROM_LAST_OFFSET: consumer clients pick up where it + * stopped previously. If it were a newly booting up consumer client, according aging + * of the consumer group, there are two cases: + *
      + *
    1. if the consumer group is created so recently that the earliest message being + * subscribed has yet expired, which means the consumer group represents a lately + * launched business, consuming will start from the very beginning;
    2. + *
    3. if the earliest message being subscribed has expired, consuming will start from + * the latest messages, meaning messages born prior to the booting timestamp would be + * ignored.
    4. + *
    + *
  • + *
  • CONSUME_FROM_FIRST_OFFSET: Consumer client will start from + * earliest messages available.
  • + *
  • CONSUME_FROM_TIMESTAMP: Consumer client will start from specified + * timestamp, which means messages born prior to {@link #consumeTimestamp} will be + * ignored
  • + *
*/ - private int delayLevelWhenNextConsume = 0; + private ConsumeFromWhere consumeFromWhere = ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET; /** - * for orderly listener. next retry delay time. + * Backtracking consumption time with second precision. Time format is + * 20131223171201
+ * Implying Seventeen twelve and 01 seconds on December 23, 2013 year
+ * Default backtracking consumption time Half an hour ago. */ - private long suspendCurrentQueueTimeMillis = 1000; + private String consumeTimestamp = UtilAll + .timeMillisToHumanString3(System.currentTimeMillis() - (1000 * 60 * 30)); - private Boolean enabled = true; + /** + * Flow control threshold on queue level, each message queue will cache at most 1000 + * messages by default, Consider the {@link #pullBatchSize}, the instantaneous value + * may exceed the limit . + */ + private int pullThresholdForQueue = 1000; + + /** + * Limit the cached message size on queue level, each message queue will cache at most + * 100 MiB messages by default, Consider the {@link #pullBatchSize}, the instantaneous + * value may exceed the limit . + * + *

+ * The size of a message only measured by message body, so it's not accurate + */ + private int pullThresholdSizeForQueue = 100; /** - * {@link JacksonRocketMQHeaderMapper#addTrustedPackages(String...)}. + * Maximum number of messages pulled each time. */ - private Set trustedPackages; + private int pullBatchSize = 10; - // ------------ For Pull Consumer ------------ + /** + * Consume max span offset.it has no effect on sequential consumption. + */ + private int consumeMaxSpan = 2000; - private long pullTimeout = 10 * 1000; + private Push push = new Push(); - private boolean fromStore; + private Pull pull = new Pull(); - // ------------ For Pull Consumer ------------ + public String getMessageModel() { + return messageModel; + } - public String getTags() { - return tags; + public RocketMQConsumerProperties setMessageModel(String messageModel) { + this.messageModel = messageModel; + return this; } - public void setTags(String tags) { - this.tags = tags; + public String getAllocateMessageQueueStrategy() { + return allocateMessageQueueStrategy; } - public String getSql() { - return sql; + public void setAllocateMessageQueueStrategy(String allocateMessageQueueStrategy) { + this.allocateMessageQueueStrategy = allocateMessageQueueStrategy; } - public void setSql(String sql) { - this.sql = sql; + public String getSubscription() { + return subscription; } - public Boolean getOrderly() { - return orderly; + public void setSubscription(String subscription) { + this.subscription = subscription; } - public void setOrderly(Boolean orderly) { - this.orderly = orderly; + public Push getPush() { + return push; } - public Boolean getEnabled() { - return enabled; + public void setPush(Push push) { + this.push = push; } - public void setEnabled(Boolean enabled) { - this.enabled = enabled; + public long getPullTimeDelayMillsWhenException() { + return pullTimeDelayMillsWhenException; } - public Boolean getBroadcasting() { - return broadcasting; + public RocketMQConsumerProperties setPullTimeDelayMillsWhenException( + long pullTimeDelayMillsWhenException) { + this.pullTimeDelayMillsWhenException = pullTimeDelayMillsWhenException; + return this; } - public void setBroadcasting(Boolean broadcasting) { - this.broadcasting = broadcasting; + public ConsumeFromWhere getConsumeFromWhere() { + return consumeFromWhere; } - public int getDelayLevelWhenNextConsume() { - return delayLevelWhenNextConsume; + public RocketMQConsumerProperties setConsumeFromWhere( + ConsumeFromWhere consumeFromWhere) { + this.consumeFromWhere = consumeFromWhere; + return this; } - public void setDelayLevelWhenNextConsume(int delayLevelWhenNextConsume) { - this.delayLevelWhenNextConsume = delayLevelWhenNextConsume; + public String getConsumeTimestamp() { + return consumeTimestamp; } - public long getSuspendCurrentQueueTimeMillis() { - return suspendCurrentQueueTimeMillis; + public RocketMQConsumerProperties setConsumeTimestamp(String consumeTimestamp) { + this.consumeTimestamp = consumeTimestamp; + return this; } - public void setSuspendCurrentQueueTimeMillis(long suspendCurrentQueueTimeMillis) { - this.suspendCurrentQueueTimeMillis = suspendCurrentQueueTimeMillis; + public int getPullThresholdForQueue() { + return pullThresholdForQueue; } - public long getPullTimeout() { - return pullTimeout; + public RocketMQConsumerProperties setPullThresholdForQueue( + int pullThresholdForQueue) { + this.pullThresholdForQueue = pullThresholdForQueue; + return this; } - public void setPullTimeout(long pullTimeout) { - this.pullTimeout = pullTimeout; + public int getPullThresholdSizeForQueue() { + return pullThresholdSizeForQueue; } - public boolean isFromStore() { - return fromStore; + public RocketMQConsumerProperties setPullThresholdSizeForQueue( + int pullThresholdSizeForQueue) { + this.pullThresholdSizeForQueue = pullThresholdSizeForQueue; + return this; } - public void setFromStore(boolean fromStore) { - this.fromStore = fromStore; + public int getPullBatchSize() { + return pullBatchSize; } - public boolean shouldRequeue() { - return delayLevelWhenNextConsume != -1; + public RocketMQConsumerProperties setPullBatchSize(int pullBatchSize) { + this.pullBatchSize = pullBatchSize; + return this; } - public Set getTrustedPackages() { - return trustedPackages; + public Pull getPull() { + return pull; } - public void setTrustedPackages(Set trustedPackages) { - this.trustedPackages = trustedPackages; + public RocketMQConsumerProperties setPull(Pull pull) { + this.pull = pull; + return this; + } + + public int getConsumeMaxSpan() { + return consumeMaxSpan; + } + + public RocketMQConsumerProperties setConsumeMaxSpan(int consumeMaxSpan) { + this.consumeMaxSpan = consumeMaxSpan; + return this; + } + + public static class Push implements Serializable { + + private static final long serialVersionUID = -7398468554978817630L; + + /** + * if orderly is true, using {@link MessageListenerOrderly} else if orderly if + * false, using {@link MessageListenerConcurrently}. + */ + private boolean orderly = false; + + /** + * Suspending pulling time for cases requiring slow pulling like flow-control + * scenario. see{@link ConsumeMessageOrderlyService#processConsumeResult}. + * see{@link ConsumeOrderlyContext#getSuspendCurrentQueueTimeMillis}. + */ + private int suspendCurrentQueueTimeMillis = 1000; + + /** + * https://github.com/alibaba/spring-cloud-alibaba/issues/1866 Max re-consume + * times. -1 means 16 times. If messages are re-consumed more than + * {@link #maxReconsumeTimes} before success, it's be directed to a deletion queue + * waiting. + */ + private int maxReconsumeTimes; + + /** + * for concurrently listener. message consume retry strategy. -1 means dlq(or + * discard. see {@link ConsumeMessageConcurrentlyService#processConsumeResult}. + * see {@link ConsumeConcurrentlyContext#getDelayLevelWhenNextConsume}. + */ + private int delayLevelWhenNextConsume = 0; + + /** + * Flow control threshold on topic level, default value is -1(Unlimited) + *

+ * The value of {@code pullThresholdForQueue} will be overwrote and calculated + * based on {@code pullThresholdForTopic} if it is't unlimited + *

+ * For example, if the value of pullThresholdForTopic is 1000 and 10 message + * queues are assigned to this consumer, then pullThresholdForQueue will be set to + * 100. + */ + private int pullThresholdForTopic = -1; + + /** + * Limit the cached message size on topic level, default value is -1 + * MiB(Unlimited) + *

+ * The value of {@code pullThresholdSizeForQueue} will be overwrote and calculated + * based on {@code pullThresholdSizeForTopic} if it is't unlimited . + *

+ * For example, if the value of pullThresholdSizeForTopic is 1000 MiB and 10 + * message queues are assigned to this consumer, then pullThresholdSizeForQueue + * will be set to 100 MiB . + */ + private int pullThresholdSizeForTopic = -1; + + /** + * Message pull Interval. + */ + private long pullInterval = 0; + + /** + * Batch consumption size. + */ + private int consumeMessageBatchMaxSize = 1; + + public boolean getOrderly() { + return orderly; + } + + public void setOrderly(boolean orderly) { + this.orderly = orderly; + } + + public int getSuspendCurrentQueueTimeMillis() { + return suspendCurrentQueueTimeMillis; + } + + public void setSuspendCurrentQueueTimeMillis(int suspendCurrentQueueTimeMillis) { + this.suspendCurrentQueueTimeMillis = suspendCurrentQueueTimeMillis; + } + + public int getMaxReconsumeTimes() { + return maxReconsumeTimes; + } + + public void setMaxReconsumeTimes(int maxReconsumeTimes) { + this.maxReconsumeTimes = maxReconsumeTimes; + } + + public int getDelayLevelWhenNextConsume() { + return delayLevelWhenNextConsume; + } + + public void setDelayLevelWhenNextConsume(int delayLevelWhenNextConsume) { + this.delayLevelWhenNextConsume = delayLevelWhenNextConsume; + } + + public int getPullThresholdForTopic() { + return pullThresholdForTopic; + } + + public void setPullThresholdForTopic(int pullThresholdForTopic) { + this.pullThresholdForTopic = pullThresholdForTopic; + } + + public int getPullThresholdSizeForTopic() { + return pullThresholdSizeForTopic; + } + + public void setPullThresholdSizeForTopic(int pullThresholdSizeForTopic) { + this.pullThresholdSizeForTopic = pullThresholdSizeForTopic; + } + + public long getPullInterval() { + return pullInterval; + } + + public void setPullInterval(long pullInterval) { + this.pullInterval = pullInterval; + } + + public int getConsumeMessageBatchMaxSize() { + return consumeMessageBatchMaxSize; + } + + public void setConsumeMessageBatchMaxSize(int consumeMessageBatchMaxSize) { + this.consumeMessageBatchMaxSize = consumeMessageBatchMaxSize; + } + + } + + public static class Pull implements Serializable { + + /** + * The poll timeout in milliseconds. + */ + private long pollTimeoutMillis = 1000 * 5; + + /** + * Pull thread number. + */ + private int pullThreadNums = 20; + + /** + * Interval time in in milliseconds for checking changes in topic metadata. + */ + private long topicMetadataCheckIntervalMillis = 30 * 1000; + + /** + * Long polling mode, the Consumer connection timeout(must greater than + * brokerSuspendMaxTimeMillis), it is not recommended to modify. + */ + private long consumerTimeoutMillisWhenSuspend = 1000 * 30; + + /** + * Ack state handling, including receive, reject, and retry, when a consumption + * exception occurs. + */ + private String errAcknowledge; + + private long pullThresholdForAll = 1000L; + + public long getPollTimeoutMillis() { + return pollTimeoutMillis; + } + + public void setPollTimeoutMillis(long pollTimeoutMillis) { + this.pollTimeoutMillis = pollTimeoutMillis; + } + + public int getPullThreadNums() { + return pullThreadNums; + } + + public void setPullThreadNums(int pullThreadNums) { + this.pullThreadNums = pullThreadNums; + } + + public long getTopicMetadataCheckIntervalMillis() { + return topicMetadataCheckIntervalMillis; + } + + public void setTopicMetadataCheckIntervalMillis( + long topicMetadataCheckIntervalMillis) { + this.topicMetadataCheckIntervalMillis = topicMetadataCheckIntervalMillis; + } + + public long getConsumerTimeoutMillisWhenSuspend() { + return consumerTimeoutMillisWhenSuspend; + } + + public void setConsumerTimeoutMillisWhenSuspend( + long consumerTimeoutMillisWhenSuspend) { + this.consumerTimeoutMillisWhenSuspend = consumerTimeoutMillisWhenSuspend; + } + + public String getErrAcknowledge() { + return errAcknowledge; + } + + public void setErrAcknowledge(String errAcknowledge) { + this.errAcknowledge = errAcknowledge; + } + + public long getPullThresholdForAll() { + return pullThresholdForAll; + } + + public void setPullThresholdForAll(long pullThresholdForAll) { + this.pullThresholdForAll = pullThresholdForAll; + } + } } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQExtendedBindingProperties.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQExtendedBindingProperties.java index 890d22500..6c4af119d 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQExtendedBindingProperties.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQExtendedBindingProperties.java @@ -21,12 +21,14 @@ import org.springframework.cloud.stream.binder.AbstractExtendedBindingProperties import org.springframework.cloud.stream.binder.BinderSpecificPropertiesProvider; /** - * @author Timur Valiev + * rocketMQ specific extended binding properties class that extends from + * {@link AbstractExtendedBindingProperties}. + * * @author Jim */ @ConfigurationProperties("spring.cloud.stream.rocketmq") public class RocketMQExtendedBindingProperties extends - AbstractExtendedBindingProperties { + AbstractExtendedBindingProperties { private static final String DEFAULTS_PREFIX = "spring.cloud.stream.rocketmq.default"; @@ -37,7 +39,7 @@ public class RocketMQExtendedBindingProperties extends @Override public Class getExtendedPropertiesEntryClass() { - return RocketMQBindingProperties.class; + return RocketMQSpecificPropertiesProvider.class; } } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQProducerProperties.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQProducerProperties.java index ab2a92a3a..4035f1483 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQProducerProperties.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQProducerProperties.java @@ -16,55 +16,37 @@ package com.alibaba.cloud.stream.binder.rocketmq.properties; -import org.apache.rocketmq.client.producer.DefaultMQProducer; - /** - * @author Timur Valiev + * Extended producer properties for RocketMQ binder. + * * @author Jim */ -public class RocketMQProducerProperties { - - private Boolean enabled = true; +public class RocketMQProducerProperties extends RocketMQCommonProperties { /** - * Name of producer. + * Timeout for sending messages. */ - private String group; - - /** - * Maximum allowed message size in bytes {@link DefaultMQProducer#maxMessageSize}. - */ - private Integer maxMessageSize = 1024 * 1024 * 4; - - private Boolean transactional = false; - - private Boolean sync = false; - - private Boolean vipChannelEnabled = true; - - /** - * Millis of send message timeout. - */ - private int sendMessageTimeout = 3000; + private int sendMsgTimeout = 3000; /** * Compress message body threshold, namely, message body larger than 4k will be * compressed on default. */ - private int compressMessageBodyThreshold = 1024 * 4; + private int compressMsgBodyThreshold = 1024 * 4; /** * Maximum number of retry to perform internally before claiming sending failure in - * synchronous mode. This may potentially cause message duplication which is up to - * application developers to resolve. + * synchronous mode. + * + * This may potentially cause message duplication which is up to application + * developers to resolve. */ private int retryTimesWhenSendFailed = 2; /** - *

* Maximum number of retry to perform internally before claiming sending failure in * asynchronous mode. - *

+ * * This may potentially cause message duplication which is up to application * developers to resolve. */ @@ -73,94 +55,187 @@ public class RocketMQProducerProperties { /** * Indicate whether to retry another broker on sending failure internally. */ - private boolean retryNextServer = false; + private boolean retryAnotherBroker = false; + + /** + * Maximum allowed message size in bytes. + */ + private int maxMessageSize = 1024 * 1024 * 4; + + private String producerType = ProducerType.Normal.name(); + + private String sendType = SendType.Sync.name(); + + private String sendCallBack; + + private String transactionListener; + + private String messageQueueSelector; + + private String errorMessageStrategy; + + private String sendFailureChannel; + + private String checkForbiddenHook; + + private String sendMessageHook; + + public int getSendMsgTimeout() { + return sendMsgTimeout; + } + + public void setSendMsgTimeout(int sendMsgTimeout) { + this.sendMsgTimeout = sendMsgTimeout; + } + + public int getCompressMsgBodyThreshold() { + return compressMsgBodyThreshold; + } + + public void setCompressMsgBodyThreshold(int compressMsgBodyThreshold) { + this.compressMsgBodyThreshold = compressMsgBodyThreshold; + } - public String getGroup() { - return group; + public int getRetryTimesWhenSendFailed() { + return retryTimesWhenSendFailed; } - public void setGroup(String group) { - this.group = group; + public void setRetryTimesWhenSendFailed(int retryTimesWhenSendFailed) { + this.retryTimesWhenSendFailed = retryTimesWhenSendFailed; } - public Boolean getEnabled() { - return enabled; + public int getRetryTimesWhenSendAsyncFailed() { + return retryTimesWhenSendAsyncFailed; } - public void setEnabled(Boolean enabled) { - this.enabled = enabled; + public void setRetryTimesWhenSendAsyncFailed(int retryTimesWhenSendAsyncFailed) { + this.retryTimesWhenSendAsyncFailed = retryTimesWhenSendAsyncFailed; } - public Integer getMaxMessageSize() { + public boolean getRetryAnotherBroker() { + return retryAnotherBroker; + } + + public void setRetryAnotherBroker(boolean retryAnotherBroker) { + this.retryAnotherBroker = retryAnotherBroker; + } + + public int getMaxMessageSize() { return maxMessageSize; } - public void setMaxMessageSize(Integer maxMessageSize) { + public void setMaxMessageSize(int maxMessageSize) { this.maxMessageSize = maxMessageSize; } - public Boolean getTransactional() { - return transactional; + public String getProducerType() { + return producerType; } - public void setTransactional(Boolean transactional) { - this.transactional = transactional; + public void setProducerType(String producerType) { + this.producerType = producerType; } - public Boolean getSync() { - return sync; + public String getSendType() { + return sendType; } - public void setSync(Boolean sync) { - this.sync = sync; + public void setSendType(String sendType) { + this.sendType = sendType; } - public Boolean getVipChannelEnabled() { - return vipChannelEnabled; + public String getSendCallBack() { + return sendCallBack; } - public void setVipChannelEnabled(Boolean vipChannelEnabled) { - this.vipChannelEnabled = vipChannelEnabled; + public void setSendCallBack(String sendCallBack) { + this.sendCallBack = sendCallBack; } - public int getSendMessageTimeout() { - return sendMessageTimeout; + public String getTransactionListener() { + return transactionListener; } - public void setSendMessageTimeout(int sendMessageTimeout) { - this.sendMessageTimeout = sendMessageTimeout; + public void setTransactionListener(String transactionListener) { + this.transactionListener = transactionListener; } - public int getCompressMessageBodyThreshold() { - return compressMessageBodyThreshold; + public String getMessageQueueSelector() { + return messageQueueSelector; } - public void setCompressMessageBodyThreshold(int compressMessageBodyThreshold) { - this.compressMessageBodyThreshold = compressMessageBodyThreshold; + public void setMessageQueueSelector(String messageQueueSelector) { + this.messageQueueSelector = messageQueueSelector; } - public int getRetryTimesWhenSendFailed() { - return retryTimesWhenSendFailed; + public String getErrorMessageStrategy() { + return errorMessageStrategy; } - public void setRetryTimesWhenSendFailed(int retryTimesWhenSendFailed) { - this.retryTimesWhenSendFailed = retryTimesWhenSendFailed; + public void setErrorMessageStrategy(String errorMessageStrategy) { + this.errorMessageStrategy = errorMessageStrategy; } - public int getRetryTimesWhenSendAsyncFailed() { - return retryTimesWhenSendAsyncFailed; + public String getSendFailureChannel() { + return sendFailureChannel; } - public void setRetryTimesWhenSendAsyncFailed(int retryTimesWhenSendAsyncFailed) { - this.retryTimesWhenSendAsyncFailed = retryTimesWhenSendAsyncFailed; + public void setSendFailureChannel(String sendFailureChannel) { + this.sendFailureChannel = sendFailureChannel; + } + + public String getCheckForbiddenHook() { + return checkForbiddenHook; + } + + public void setCheckForbiddenHook(String checkForbiddenHook) { + this.checkForbiddenHook = checkForbiddenHook; + } + + public String getSendMessageHook() { + return sendMessageHook; } - public boolean isRetryNextServer() { - return retryNextServer; + public void setSendMessageHook(String sendMessageHook) { + this.sendMessageHook = sendMessageHook; } - public void setRetryNextServer(boolean retryNextServer) { - this.retryNextServer = retryNextServer; + public enum ProducerType { + + /** + * Is not a transaction. + */ + Normal, + /** + * a transaction. + */ + Trans; + + public boolean equalsName(String name) { + return this.name().equalsIgnoreCase(name); + } + + } + + public enum SendType { + + /** + * one way. + */ + OneWay, + /** + * Asynchronization Model. + */ + Async, + /** + * synchronization. + */ + Sync,; + + public boolean equalsName(String name) { + return this.name().equalsIgnoreCase(name); + } + } } diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQBindingProperties.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQSpecificPropertiesProvider.java similarity index 65% rename from spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQBindingProperties.java rename to spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQSpecificPropertiesProvider.java index 814ae1018..990a211ce 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQBindingProperties.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQSpecificPropertiesProvider.java @@ -19,27 +19,44 @@ package com.alibaba.cloud.stream.binder.rocketmq.properties; import org.springframework.cloud.stream.binder.BinderSpecificPropertiesProvider; /** - * @author Timur Valiev + * Container object for RocketMQ specific extended producer and consumer binding + * properties. + * * @author Jim */ -public class RocketMQBindingProperties implements BinderSpecificPropertiesProvider { +public class RocketMQSpecificPropertiesProvider + implements BinderSpecificPropertiesProvider { + /** + * Consumer specific binding properties. @see {@link RocketMQConsumerProperties}. + */ private RocketMQConsumerProperties consumer = new RocketMQConsumerProperties(); + /** + * Producer specific binding properties. @see {@link RocketMQProducerProperties}. + */ private RocketMQProducerProperties producer = new RocketMQProducerProperties(); + /** + * @return {@link RocketMQConsumerProperties} Consumer specific binding + * properties. @see {@link RocketMQConsumerProperties}. + */ @Override public RocketMQConsumerProperties getConsumer() { - return consumer; + return this.consumer; } public void setConsumer(RocketMQConsumerProperties consumer) { this.consumer = consumer; } + /** + * @return {@link RocketMQProducerProperties} Producer specific binding + * properties. @see {@link RocketMQProducerProperties}. + */ @Override public RocketMQProducerProperties getProducer() { - return producer; + return this.producer; } public void setProducer(RocketMQProducerProperties producer) { diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/provisioning/selector/PartitionMessageQueueSelector.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/provisioning/selector/PartitionMessageQueueSelector.java index 694dcdc96..0580fe859 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/provisioning/selector/PartitionMessageQueueSelector.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/provisioning/selector/PartitionMessageQueueSelector.java @@ -36,7 +36,7 @@ public class PartitionMessageQueueSelector implements MessageQueueSelector { @Override public MessageQueue select(List mqs, Message msg, Object arg) { - Integer partition = 0; + int partition = 0; try { partition = Math.abs( Integer.parseInt(msg.getProperty(BinderHeaders.PARTITION_HEADER))); diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/support/RocketMQMessageConverterSupport.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/support/RocketMQMessageConverterSupport.java new file mode 100644 index 000000000..c6e0a29b1 --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/support/RocketMQMessageConverterSupport.java @@ -0,0 +1,187 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.support; + +import java.nio.charset.Charset; +import java.util.Map; +import java.util.Objects; + +import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst; +import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst.Headers; +import com.alibaba.cloud.stream.binder.rocketmq.convert.RocketMQMessageConverter; +import com.alibaba.cloud.stream.binder.rocketmq.custom.RocketMQBeanContainerCache; +import org.apache.rocketmq.common.message.MessageConst; +import org.apache.rocketmq.common.message.MessageExt; + +import org.springframework.messaging.Message; +import org.springframework.messaging.MessageHeaders; +import org.springframework.messaging.converter.CompositeMessageConverter; +import org.springframework.messaging.support.MessageBuilder; +import org.springframework.util.CollectionUtils; +import org.springframework.util.MimeTypeUtils; +import org.springframework.util.StringUtils; + +/** + * @author zkzlx + */ +public final class RocketMQMessageConverterSupport { + + private RocketMQMessageConverterSupport() { + } + + private static final CompositeMessageConverter MESSAGE_CONVERTER = RocketMQBeanContainerCache + .getBean(RocketMQMessageConverter.DEFAULT_NAME, + CompositeMessageConverter.class, + new RocketMQMessageConverter().getMessageConverter()); + + public static Message convertMessage2Spring(MessageExt message) { + MessageBuilder messageBuilder = MessageBuilder.withPayload(message.getBody()) + .setHeader(toRocketHeaderKey(Headers.KEYS), message.getKeys()) + .setHeader(toRocketHeaderKey(Headers.TAGS), message.getTags()) + .setHeader(toRocketHeaderKey(Headers.TOPIC), message.getTopic()) + .setHeader(toRocketHeaderKey(Headers.MESSAGE_ID), message.getMsgId()) + .setHeader(toRocketHeaderKey(Headers.BORN_TIMESTAMP), + message.getBornTimestamp()) + .setHeader(toRocketHeaderKey(Headers.BORN_HOST), + message.getBornHostString()) + .setHeader(toRocketHeaderKey(Headers.FLAG), message.getFlag()) + .setHeader(toRocketHeaderKey(Headers.QUEUE_ID), message.getQueueId()) + .setHeader(toRocketHeaderKey(Headers.SYS_FLAG), message.getSysFlag()) + .setHeader(toRocketHeaderKey(Headers.TRANSACTION_ID), + message.getTransactionId()); + addUserProperties(message.getProperties(), messageBuilder); + return messageBuilder.build(); + } + + public static String toRocketHeaderKey(String rawKey) { + return "ROCKET_" + rawKey; + } + + private static void addUserProperties(Map properties, + MessageBuilder messageBuilder) { + if (!CollectionUtils.isEmpty(properties)) { + properties.forEach((key, val) -> { + if (!MessageConst.STRING_HASH_SET.contains(key) + && !MessageHeaders.ID.equals(key) + && !MessageHeaders.TIMESTAMP.equals(key)) { + messageBuilder.setHeader(key, val); + } + }); + } + } + + public static org.apache.rocketmq.common.message.Message convertMessage2MQ( + String destination, Message source) { + Message message = MESSAGE_CONVERTER.toMessage(source.getPayload(), + source.getHeaders()); + assert message != null; + MessageBuilder builder = MessageBuilder.fromMessage(message); + builder.setHeaderIfAbsent(MessageHeaders.CONTENT_TYPE, MimeTypeUtils.TEXT_PLAIN); + message = builder.build(); + return doConvert(destination, message); + } + + private static org.apache.rocketmq.common.message.Message doConvert(String topic, + Message message) { + Charset charset = Charset.defaultCharset(); + Object payloadObj = message.getPayload(); + byte[] payloads; + try { + if (payloadObj instanceof String) { + payloads = ((String) payloadObj).getBytes(charset); + } + else if (payloadObj instanceof byte[]) { + payloads = (byte[]) message.getPayload(); + } + else { + String jsonObj = (String) MESSAGE_CONVERTER.fromMessage(message, + payloadObj.getClass()); + if (null == jsonObj) { + throw new RuntimeException(String.format( + "empty after conversion [messageConverter:%s,payloadClass:%s,payloadObj:%s]", + MESSAGE_CONVERTER.getClass(), payloadObj.getClass(), + payloadObj)); + } + payloads = jsonObj.getBytes(charset); + } + } + catch (Exception e) { + throw new RuntimeException("convert to RocketMQ message failed.", e); + } + return getAndWrapMessage(topic, message.getHeaders(), payloads); + } + + private static org.apache.rocketmq.common.message.Message getAndWrapMessage( + String topic, MessageHeaders headers, byte[] payloads) { + if (topic == null || topic.length() < 1) { + return null; + } + if (payloads == null || payloads.length < 1) { + return null; + } + org.apache.rocketmq.common.message.Message rocketMsg = new org.apache.rocketmq.common.message.Message( + topic, payloads); + if (Objects.nonNull(headers) && !headers.isEmpty()) { + Object tag = headers.getOrDefault(Headers.TAGS, + headers.get(toRocketHeaderKey(Headers.TAGS))); + if (StringUtils.hasLength(tag.toString())) { + rocketMsg.setTags(String.valueOf(tag)); + } + + Object keys = headers.getOrDefault(Headers.KEYS, + headers.get(toRocketHeaderKey(Headers.KEYS))); + if (StringUtils.hasLength(keys.toString())) { + rocketMsg.setKeys(keys.toString()); + } + Object flagObj = headers.getOrDefault(Headers.FLAG, + headers.get(toRocketHeaderKey(Headers.FLAG))); + int flag = 0; + int delayLevel = 0; + try { + flagObj = flagObj == null ? 0 : flagObj; + Object delayLevelObj = headers.getOrDefault( + RocketMQConst.PROPERTY_DELAY_TIME_LEVEL, + headers.get(toRocketHeaderKey( + RocketMQConst.PROPERTY_DELAY_TIME_LEVEL))); + delayLevelObj = delayLevelObj == null ? 0 : delayLevelObj; + delayLevel = Integer.parseInt(String.valueOf(delayLevelObj)); + flag = Integer.parseInt(String.valueOf(flagObj)); + } + catch (Exception ignored) { + } + if (delayLevel > 0) { + rocketMsg.setDelayTimeLevel(delayLevel); + } + rocketMsg.setFlag(flag); + Object waitStoreMsgOkObj = headers + .getOrDefault(RocketMQConst.PROPERTY_WAIT_STORE_MSG_OK, "true"); + rocketMsg.setWaitStoreMsgOK( + Boolean.parseBoolean(String.valueOf(waitStoreMsgOkObj))); + headers.entrySet().stream() + .filter(entry -> !Objects.equals(entry.getKey(), Headers.FLAG)) + .forEach(entry -> { + if (!MessageConst.STRING_HASH_SET.contains(entry.getKey())) { + rocketMsg.putUserProperty(entry.getKey(), + String.valueOf(entry.getValue())); + } + }); + + } + return rocketMsg; + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/utils/RocketMQUtils.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/utils/RocketMQUtils.java new file mode 100644 index 000000000..4ab1411da --- /dev/null +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/utils/RocketMQUtils.java @@ -0,0 +1,103 @@ +/* + * Copyright 2013-2018 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.cloud.stream.binder.rocketmq.utils; + +import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst; +import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQBinderConfigurationProperties; +import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQCommonProperties; +import org.apache.commons.lang3.StringUtils; +import org.apache.rocketmq.acl.common.AclClientRPCHook; +import org.apache.rocketmq.acl.common.SessionCredentials; +import org.apache.rocketmq.client.consumer.MessageSelector; +import org.apache.rocketmq.common.UtilAll; +import org.apache.rocketmq.remoting.RPCHook; + + +/** + * @author Jim + */ +public final class RocketMQUtils { + + private RocketMQUtils() { + } + + public static T mergeRocketMQProperties( + RocketMQBinderConfigurationProperties binderConfigurationProperties, + T mqProperties) { + if (null == binderConfigurationProperties || mqProperties == null) { + return mqProperties; + } + if (StringUtils.isEmpty(mqProperties.getNameServer())) { + mqProperties.setNameServer(binderConfigurationProperties.getNameServer()); + } + if (StringUtils.isEmpty(mqProperties.getSecretKey())) { + mqProperties.setSecretKey(binderConfigurationProperties.getSecretKey()); + } + if (StringUtils.isEmpty(mqProperties.getAccessKey())) { + mqProperties.setAccessKey(binderConfigurationProperties.getAccessKey()); + } + if (StringUtils.isEmpty(mqProperties.getAccessChannel())) { + mqProperties + .setAccessChannel(binderConfigurationProperties.getAccessChannel()); + } + if (StringUtils.isEmpty(mqProperties.getNamespace())) { + mqProperties.setNamespace(binderConfigurationProperties.getNamespace()); + } + if (StringUtils.isEmpty(mqProperties.getGroup())) { + mqProperties.setGroup(binderConfigurationProperties.getGroup()); + } + if (StringUtils.isEmpty(mqProperties.getCustomizedTraceTopic())) { + mqProperties.setCustomizedTraceTopic( + binderConfigurationProperties.getCustomizedTraceTopic()); + } + if (StringUtils.isEmpty(mqProperties.getUnitName())) { + mqProperties.setUnitName(binderConfigurationProperties.getUnitName()); + } + mqProperties.setNameServer(getNameServerStr(mqProperties.getNameServer())); + return mqProperties; + } + + public static String getInstanceName(RPCHook rpcHook, String identify) { + String separator = "|"; + StringBuilder instanceName = new StringBuilder(); + if (null != rpcHook) { + SessionCredentials sessionCredentials = ((AclClientRPCHook) rpcHook) + .getSessionCredentials(); + instanceName.append(sessionCredentials.getAccessKey()).append(separator) + .append(sessionCredentials.getSecretKey()).append(separator); + } + instanceName.append(identify).append(separator).append(UtilAll.getPid()); + return instanceName.toString(); + } + + public static String getNameServerStr(String nameServer) { + if (StringUtils.isEmpty(nameServer)) { + return RocketMQConst.DEFAULT_NAME_SERVER; + } + return nameServer.replaceAll(",", ";"); + } + + private static final String SQL = "sql:"; + + public static MessageSelector getMessageSelector(String expression) { + if (StringUtils.isNotBlank(expression) && expression.startsWith(SQL)) { + return MessageSelector.bySql(expression.replaceFirst(SQL, "")); + } + return MessageSelector.byTag(expression); + } + +} diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/resources/META-INF/spring.binders b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/resources/META-INF/spring.binders index 2e5b99538..b232e1f10 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/resources/META-INF/spring.binders +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/resources/META-INF/spring.binders @@ -1 +1 @@ -rocketmq:com.alibaba.cloud.stream.binder.rocketmq.config.RocketMQBinderAutoConfiguration \ No newline at end of file +rocketmq:com.alibaba.cloud.stream.binder.rocketmq.autoconfigurate.RocketMQBinderAutoConfiguration \ No newline at end of file diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/resources/META-INF/spring.factories b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/resources/META-INF/spring.factories index 82d344e00..b67a42530 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/resources/META-INF/spring.factories +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/resources/META-INF/spring.factories @@ -1,2 +1,2 @@ org.springframework.boot.autoconfigure.EnableAutoConfiguration=\ -com.alibaba.cloud.stream.binder.rocketmq.config.RocketMQComponent4BinderAutoConfiguration +com.alibaba.cloud.stream.binder.rocketmq.autoconfigurate.ExtendedBindingHandlerMappingsProviderConfiguration \ No newline at end of file diff --git a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/test/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQAutoConfigurationTests.java b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/test/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQAutoConfigurationTests.java index 8207f9893..f850afb4e 100644 --- a/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/test/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQAutoConfigurationTests.java +++ b/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/test/java/com/alibaba/cloud/stream/binder/rocketmq/RocketMQAutoConfigurationTests.java @@ -16,9 +16,7 @@ package com.alibaba.cloud.stream.binder.rocketmq; -import java.util.Arrays; - -import com.alibaba.cloud.stream.binder.rocketmq.config.RocketMQBinderAutoConfiguration; +import com.alibaba.cloud.stream.binder.rocketmq.autoconfigurate.RocketMQBinderAutoConfiguration; import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQBinderConfigurationProperties; import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQExtendedBindingProperties; import org.junit.Test; @@ -37,20 +35,20 @@ public class RocketMQAutoConfigurationTests { .withConfiguration( AutoConfigurations.of(RocketMQBinderAutoConfiguration.class)) .withPropertyValues( - "spring.cloud.stream.rocketmq.binder.name-server[0]=127.0.0.1:9876", - "spring.cloud.stream.rocketmq.binder.name-server[1]=127.0.0.1:9877", + "spring.cloud.stream.rocketmq.binder.name-server=127.0.0.1:9876,127.0.0.1:9877", "spring.cloud.stream.bindings.output.destination=TopicOrderTest", "spring.cloud.stream.bindings.output.content-type=application/json", + "spring.cloud.stream.bindings.input1.destination=TopicOrderTest", "spring.cloud.stream.bindings.input1.content-type=application/json", "spring.cloud.stream.bindings.input1.group=test-group1", - "spring.cloud.stream.rocketmq.bindings.input1.consumer.orderly=true", + "spring.cloud.stream.rocketmq.bindings.input1.consumer.push.orderly=true", "spring.cloud.stream.bindings.input1.consumer.maxAttempts=1", "spring.cloud.stream.bindings.input2.destination=TopicOrderTest", "spring.cloud.stream.bindings.input2.content-type=application/json", "spring.cloud.stream.bindings.input2.group=test-group2", - "spring.cloud.stream.rocketmq.bindings.input2.consumer.orderly=false", - "spring.cloud.stream.rocketmq.bindings.input2.consumer.tags=tag1"); + "spring.cloud.stream.rocketmq.bindings.input2.consumer.push.orderly=false", + "spring.cloud.stream.rocketmq.bindings.input2.consumer.subscription=tag1"); @Test public void testProperties() { @@ -58,15 +56,14 @@ public class RocketMQAutoConfigurationTests { RocketMQBinderConfigurationProperties binderConfigurationProperties = context .getBean(RocketMQBinderConfigurationProperties.class); assertThat(binderConfigurationProperties.getNameServer()) - .isEqualTo(Arrays.asList("127.0.0.1:9876", "127.0.0.1:9877")); + .isEqualTo("127.0.0.1:9876,127.0.0.1:9877"); RocketMQExtendedBindingProperties bindingProperties = context .getBean(RocketMQExtendedBindingProperties.class); - assertThat( - bindingProperties.getExtendedConsumerProperties("input2").getTags()) - .isEqualTo("tag1"); assertThat(bindingProperties.getExtendedConsumerProperties("input2") + .getSubscription()).isEqualTo("tag1"); + assertThat(bindingProperties.getExtendedConsumerProperties("input2").getPush() .getOrderly()).isFalse(); - assertThat(bindingProperties.getExtendedConsumerProperties("input1") + assertThat(bindingProperties.getExtendedConsumerProperties("input1").getPush() .getOrderly()).isTrue(); }); }