Merge pull request #2217 from zkzlx/rocketmq

Append other module updates and optimize parts of the MQ code
pull/2302/head
zkzlx 4 years ago committed by GitHub
commit f9c2c3a531
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -48,15 +48,16 @@ Spring Cloud Alibaba 致力于提供微服务开发的一站式解决方案。
更多组件请参考 [Roadmap](https://github.com/alibaba/spring-cloud-alibaba/blob/master/Roadmap-zh.md)。
## 如何构建
* master 分支对应的是 Spring Cloud Greenwich最低支持 JDK 1.8。
* 2020.0 分支对应的是 Spring Cloud 2020最低支持 JDK 1.8。
* master 分支对应的是 Spring Cloud Hoxton最低支持 JDK 1.8。
* greenwich 分支对应的是 Spring Cloud Greenwich最低支持 JDK 1.8。
* finchley 分支对应的是 Spring Cloud Finchley最低支持 JDK 1.8。
* 1.x 分支对应的是 Spring Cloud Edgware最低支持 JDK 1.7。
Spring Cloud 使用 Maven 来构建,最快的使用方式是将本项目 clone 到本地,然后执行以下命令:
```bash
./mvnw install
```
执行完毕后,项目将被安装到本地 Maven 仓库。
## 如何使用
@ -64,7 +65,7 @@ Spring Cloud 使用 Maven 来构建,最快的使用方式是将本项目 clone
### 如何引入依赖
如果需要使用已发布的版本,在 `dependencyManagement` 中添加如下配置。
```xml
<dependencyManagement>
<dependencies>
<dependency>
@ -76,7 +77,7 @@ Spring Cloud 使用 Maven 来构建,最快的使用方式是将本项目 clone
</dependency>
</dependencies>
</dependencyManagement>
```
然后在 `dependencies` 中添加自己所需使用的依赖即可使用。
## 演示 Demo
@ -111,7 +112,7 @@ Example 列表:
* 2.0.x 版本适用于 Spring Boot 2.0.x
* 2.1.x 版本适用于 Spring Boot 2.1.x
* 2.2.x 版本适用于 Spring Boot 2.2.x
* 2021.x 版本适用于 Spring Boot 2.4.x
## 社区交流

@ -51,22 +51,23 @@ For more features, please refer to [Roadmap](https://github.com/alibaba/spring-c
For more features please refer to [Roadmap](https://github.com/alibaba/spring-cloud-alibaba/blob/master/Roadmap.md).
## How to build
* **master branch**: Corresponds to Spring Cloud Greenwich & Spring Boot 2.x. JDK 1.8 or later versions are supported.
* **finchley branch**: Corresponds to Spring Cloud Finchley & Spring Boot 2.x. JDK 1.8 or later versions are supported.
* **2020.0 branch**: Corresponds to Spring Cloud 2020 & Spring Boot 2.4.x. JDK 1.8 or later versions are supported.
* **master branch**: Corresponds to Spring Cloud Hoxton & Spring Boot 2.2.x. JDK 1.8 or later versions are supported.
* **greenwich branch**: Corresponds to Spring Cloud Greenwich & Spring Boot 2.1.x. JDK 1.8 or later versions are supported.
* **finchley branch**: Corresponds to Spring Cloud Finchley & Spring Boot 2.0.x. JDK 1.8 or later versions are supported.
* **1.x branch**: Corresponds to Spring Cloud Edgware & Spring Boot 1.x, JDK 1.7 or later versions are supported.
Spring Cloud uses Maven for most build-related activities, and you should be able to get off the ground quite quickly by cloning the project you are interested in and typing:
```bash
./mvnw install
```
## How to Use
### Add maven dependency
These artifacts are available from Maven Central and Spring Release repository via BOM:
```xml
<dependencyManagement>
<dependencies>
<dependency>
@ -78,7 +79,7 @@ These artifacts are available from Maven Central and Spring Release repository v
</dependency>
</dependencies>
</dependencyManagement>
```
add the module in `dependencies`.
@ -118,6 +119,7 @@ As the interfaces and annotations of Spring Boot 1 and Spring Boot 2 have been c
* 2.0.x for Spring Boot 2.0.x
* 2.1.x for Spring Boot 2.1.x
* 2.2.x for Spring Boot 2.2.x
* 2020.x for Spring Boot 2.4.x
## Code of Conduct
This project is a sub-project of Spring Cloud, it adheres to the Contributor Covenant [code of conduct](https://github.com/spring-cloud/spring-cloud-build/blob/master/docs/src/main/asciidoc/code-of-conduct.adoc). By participating, you are expected to uphold this code. Please report unacceptable behavior to spring-code-of-conduct@pivotal.io.

@ -21,7 +21,7 @@
<revision>2.2.6-SNAPSHOT</revision>
<sentinel.version>1.8.1</sentinel.version>
<seata.version>1.3.0</seata.version>
<nacos.client.version>1.4.1</nacos.client.version>
<nacos.client.version>1.4.2</nacos.client.version>
<nacos.config.version>0.8.0</nacos.config.version>
<spring.context.support.version>1.0.10</spring.context.support.version>

@ -260,7 +260,7 @@ Nacos 内部有 https://nacos.io/zh-cn/docs/concepts.html[Namespace 的概念]:
[quote]
用于进行租户粒度的配置隔离。不同的命名空间下,可以存在相同的 Group 或 Data ID 的配置。Namespace 的常用场景之一是不同环境的配置的区分隔离,例如开发测试环境和生产环境的资源(如配置、服务)隔离等。
在没有明确指定 `${spring.cloud.nacos.config.namespace}` 配置的情况下, 默认使用的是 Nacos 上 Public 这个namespae。如果需要使用自定义的命名空间可以通过以下配置来实现
在没有明确指定 `${spring.cloud.nacos.config.namespace}` 配置的情况下, 默认使用的是 Nacos 上 Public 这个namespace。如果需要使用自定义的命名空间可以通过以下配置来实现
[source,properties]
----
spring.cloud.nacos.config.namespace=b3404bc0-d7dc-4855-b519-570ed34b62d7
@ -348,7 +348,13 @@ Nacos Config 目前提供了三种配置能力从 Nacos 拉取相关的配置
* B: 通过 `spring.cloud.nacos.config.ext-config[n].data-id` 的方式支持多个扩展 Data Id 的配置
* C: 通过内部相关规则(应用名、应用名+ Profile )自动生成相关的 Data Id 配置
当三种方式共同使用时,他们的一个优先级关系是:A < B < C
当三种方式共同使用时,他们的一个优先级关系是:
->A为优先级最高的
->B的优先级低于A
->C的优先级是最低的
=== Nacos Config 对外暴露的 Endpoint

@ -291,6 +291,17 @@ Endpoint 暴露的 json 中包含了两种属性:
}
----
=== 如何开启权重路由
==== Ribbon
.application.properties
[source,properties]
----
[service_name].ribbon.NFLoadBalancerRuleClassName=com.alibaba.cloud.nacos.ribbon.NacosRule
----
=== 关于 Nacos Discovery Starter 更多的配置项信息
更多关于 Nacos Discovery Starter 的配置项如下所示:

@ -0,0 +1,378 @@
== Spring Cloud Alibaba RocketMQ Binder (NEW)
=== RocketMQ 介绍
https://rocketmq.apache.org[RocketMQ] 是一款开源的分布式消息系统,基于高可用分布式集群技术,提供低延时的、高可靠的消息发布与订阅服务。同时,广泛应用于多个领域,包括异步通信解耦、企业解决方案、金融支付、电信、电子商务、快递物流、广告营销、社交、即时通信、移动应用、手游、视频、物联网、车联网等。
具有以下特点:
* 能够保证严格的消息顺序
* 提供丰富的消息拉取模式
* 高效的订阅者水平扩展能力
* 实时的消息订阅机制
* 亿级消息堆积能力
=== RocketMQ 基本使用
* 下载 RocketMQ
下载 https://www.apache.org/dyn/closer.cgi?path=rocketmq/4.3.2/rocketmq-all-4.3.2-bin-release.zip[RocketMQ最新的二进制文件],并解压
解压后的目录结构如下:
```
apache-rocketmq
├── LICENSE
├── NOTICE
├── README.md
├── benchmark
├── bin
├── conf
└── lib
```
* 启动 NameServer
```bash
nohup sh bin/mqnamesrv &
tail -f ~/logs/rocketmqlogs/namesrv.log
```
* 启动 Broker
```bash
nohup sh bin/mqbroker -n localhost:9876 &
tail -f ~/logs/rocketmqlogs/broker.log
```
* 发送、接收消息
发送消息:
```bash
sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer
```
发送成功后显示:`SendResult [sendStatus=SEND_OK, msgId= ...`
接收消息:
```bash
sh bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer
```
接收成功后显示:`ConsumeMessageThread_%d Receive New Messages: [MessageExt...`
* 关闭 Server
```bash
sh bin/mqshutdown broker
sh bin/mqshutdown namesrv
```
=== Spring Cloud Stream 介绍
Spring Cloud Stream 是一个用于构建基于消息的微服务应用框架。它基于 SpringBoot 来创建具有生产级别的单机 Spring 应用,并且使用 `Spring Integration` 与 Broker 进行连接。
Spring Cloud Stream 提供了消息中间件配置的统一抽象,推出了 publish-subscribe、consumer groups、partition 这些统一的概念。
Spring Cloud Stream 内部有两个概念Binder 和 Binding。
* Binder: 跟外部消息中间件集成的组件,用来创建 Binding各消息中间件都有自己的 Binder 实现。
比如 `Kafka` 的实现 `KafkaMessageChannelBinder``RabbitMQ` 的实现 `RabbitMessageChannelBinder` 以及 `RocketMQ` 的实现 `RocketMQMessageChannelBinder`。
* Binding: 包括 Input Binding 和 Output Binding。
Binding 在消息中间件与应用程序提供的 Provider 和 Consumer 之间提供了一个桥梁,实现了开发者只需使用应用程序的 Provider 或 Consumer 生产或消费数据即可,屏蔽了开发者与底层消息中间件的接触。
.Spring Cloud Stream
image::https://docs.spring.io/spring-cloud-stream/docs/current/reference/htmlsingle/images/SCSt-overview.png[]
使用 Spring Cloud Stream 完成一段简单的消息发送和消息接收代码:
```java
MessageChannel messageChannel = new DirectChannel();
// 消息订阅
((SubscribableChannel) messageChannel).subscribe(new MessageHandler() {
@Override
public void handleMessage(Message<?> message) throws MessagingException {
System.out.println("receive msg: " + message.getPayload());
}
});
// 消息发送
messageChannel.send(MessageBuilder.withPayload("simple msg").build());
```
这段代码所有的消息类都是 `spring-messaging` 模块里提供的。屏蔽具体消息中间件的底层实现,如果想用更换消息中间件,在配置文件里配置相关消息中间件信息以及修改 binder 依赖即可。
**Spring Cloud Stream 底层基于这段代码去做了各种抽象。**
=== 如何使用 Spring Cloud Alibaba RocketMQ Binder
如果要在您的项目中引入 RocketMQ Binder需要引入如下 maven 依赖:
```xml
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-stream-binder-rocketmq</artifactId>
</dependency>
```
或者可以使用 Spring Cloud Stream RocketMQ Starter
```xml
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-stream-rocketmq</artifactId>
</dependency>
```
=== Spring Cloud Alibaba RocketMQ Binder 实现
这是 Spring Cloud Stream RocketMQ Binder 的实现架构:
.SCS RocketMQ Binder
image::https://img.alicdn.com/tfs/TB1v8rcbUY1gK0jSZFCXXcwqXXa-1236-773.png[]
RocketMQ Binder 的重构优化去除了对 https://github.com/apache/rocketmq-spring[RocketMQ-Spring]框架的依赖 。
RocketMQ Binder 核心类 `RocketMQMessageChannelBinder` 实现了 Spring Cloud Stream 规范,内部会构建 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQInboundChannelAdapter.java[RocketMQInboundChannelAdapter] 和 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProducerMessageHandler.java[RocketMQProducerMessageHandler]。
`RocketMQProducerMessageHandler` 会基于 Binding 配置通过 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/outbound/RocketMQProduceFactory.java[RocketMQProduceFactory]构造 RocketMQ Producer其内部会把 `spring-messaging` 模块内 `org.springframework.messaging.Message` 消息类转换成 RocketMQ 的消息类 `org.apache.rocketmq.common.message.Message`,然后发送出去。
`RocketMQInboundChannelAdapter` 也会基于 Binding 配置通过 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/integration/inbound/RocketMQConsumerFactory.java[RocketMQConsumerFactory]构造 DefaultMQPushConsumer其内部会启动 RocketMQ Consumer 接收消息。
NOTE: 与 https://github.com/apache/rocketmq-spring[RocketMQ-Spring] 框架的兼容需要手动处理
目前 Binder 支持在 `Header` 中设置相关的 key 来进行 RocketMQ Message 消息的特性设置。
比如 `TAGS`、`KEYS`、`TRANSACTIONAL_ARGS` 等 RocketMQ 消息对应的标签,详情见 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/contants/RocketMQConst.java[com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst]
```java
MessageBuilder builder = MessageBuilder.withPayload(msg)
.setHeader(RocketMQHeaders.TAGS, "binder")
.setHeader(RocketMQHeaders.KEYS, "my-key");
Message message = builder.build();
output().send(message);
```
NOTE: 更多使用请参考样例: https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-examples/rocketmq-example/rocketmq-produce-example/src/main/java/com/alibaba/cloud/examples/SenderService.java[com.alibaba.cloud.examples.SenderService]
=== MessageSource 支持
SCS RocketMQ Binder 支持 `MessageSource`,可以进行消息的拉取,例子如下:
```java
@SpringBootApplication
@EnableBinding(MQApplication.PolledProcessor.class)
public class MQApplication {
private final Logger logger =
LoggerFactory.getLogger(MQApplication.class);
public static void main(String[] args) {
SpringApplication.run(MQApplication.class, args);
}
@Bean
public ApplicationRunner runner(PollableMessageSource source,
MessageChannel dest) {
return args -> {
while (true) {
boolean result = source.poll(m -> {
String payload = (String) m.getPayload();
logger.info("Received: " + payload);
dest.send(MessageBuilder.withPayload(payload.toUpperCase())
.copyHeaders(m.getHeaders())
.build());
}, new ParameterizedTypeReference<String>() { });
if (result) {
logger.info("Processed a message");
}
else {
logger.info("Nothing to do");
}
Thread.sleep(5_000);
}
};
}
public static interface PolledProcessor {
@Input
PollableMessageSource source();
@Output
MessageChannel dest();
}
}
```
=== 配置选项
==== RocketMQ Binder Properties
spring.cloud.stream.rocketmq.binder.name-server::
RocketMQ NameServer 地址(老版本使用 namesrv-addr 配置项)。
+
Default: `127.0.0.1:9876`.
spring.cloud.stream.rocketmq.binder.access-key::
阿里云账号 AccessKey。
+
Default: null.
spring.cloud.stream.rocketmq.binder.secret-key::
阿里云账号 SecretKey。
+
Default: null.
spring.cloud.stream.rocketmq.binder.enable-msg-trace::
是否为 Producer 和 Consumer 开启消息轨迹功能
+
Default: `true`.
spring.cloud.stream.rocketmq.binder.customized-trace-topic::
消息轨迹开启后存储的 topic 名称。
+
Default: `RMQ_SYS_TRACE_TOPIC`.
==== RocketMQ Consumer Properties
下面的这些配置是以 `spring.cloud.stream.rocketmq.bindings.<channelName>.consumer.` 为前缀的 RocketMQ Consumer 相关的配置。
更多见 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQConsumerProperties.java[com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties]。
enable::
是否启用 Consumer。
+
默认值: `true`.
subscription::
Consumer 基于 TAGS 订阅,多个 tag 以 `||` 分割。更多见 `com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties.subscription`
+
默认值: empty.
messageModel::
Consumer 消费模式。如果想让每一个的订阅者都能接收到消息,可以使用广播模式。更多见 `org.apache.rocketmq.common.protocol.heartbeat.MessageModel`
+
默认值: `CLUSTERING`.
consumeFromWhere::
Consumer 从哪里开始消费。更多见 `org.apache.rocketmq.common.consumer.ConsumeFromWhere`
+
默认值: `CONSUME_FROM_LAST_OFFSET`.
#下面的这些配置是 Consumer Push 模式相关的配置。#
`spring.cloud.stream.rocketmq.bindings.<channelName>.consumer.push.`
orderly::
是否同步消费消息模式
+
默认值: `false`.
delayLevelWhenNextConsume::
异步消费消息模式下消费失败重试策略:
* -1,不重复,直接放入死信队列
* 0,broker 控制重试策略
* >0,client 控制重试策略
+
默认值: `0`.
suspendCurrentQueueTimeMillis::
同步消费消息模式下消费失败后再次消费的时间间隔。
+
默认值: `1000`.
其他更多参数见 `com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties.Push`
#下面的这些配置是 Consumer Pull 模式相关的配置。#
`spring.cloud.stream.rocketmq.bindings.<channelName>.consumer.pull.`
pullThreadNums::
消费时拉取的线程数
+
默认值: `20`.
pollTimeoutMillis::
拉取时的超时毫秒数
+
默认值: `1000 * 5`.
其他更多参数见 `com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties.Pull`.
NOTE: 更多参数见 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQConsumerProperties.java[com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQConsumerProperties]
==== RocketMQ Provider Properties
下面的这些配置是以 `spring.cloud.stream.rocketmq.bindings.<channelName>.producer.` 为前缀的 RocketMQ Producer 相关的配置。更多见 https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQProducerProperties.java[com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties]
enable::
是否启用 Producer。
+
默认值: `true`.
group::
Producer group name。
+
默认值: empty.
maxMessageSize::
消息发送的最大字节数。
+
默认值: `8249344`.
producerType::
消息生产者类型,普通或者事务。更多见 `com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties.ProducerType`.
+
默认值: `Normal`.
transactionListener::
事务消息监听器的beanName在 `producerType=Trans` 时才有效;必须是实现 `org.apache.rocketmq.client.producer.TransactionListener` 接口的Spring Bean。
sendType::
消息发送类型(同步、异步、单向)。更多见`com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties.SendType`.
+
默认值: `Sync`.
sendCallBack::
消息发送后回调函数的beanName在 `sendType=Async` 时才有效;必须是实现 `org.apache.rocketmq.client.producer.SendCallback` 接口的Spring Bean。
vipChannelEnabled::
是否在 Vip Channel 上发送消息。
+
默认值: `true`.
sendMessageTimeout::
发送消息的超时时间(毫秒)。
+
默认值: `3000`.
compressMessageBodyThreshold::
消息体压缩阀值(当消息体超过 4k 的时候会被压缩)。
+
默认值: `4096`.
retryTimesWhenSendFailed::
在同步发送消息的模式下,消息发送失败的重试次数。
+
默认值: `2`.
retryTimesWhenSendAsyncFailed::
在异步发送消息的模式下,消息发送失败的重试次数。
+
默认值: `2`.
retryAnotherBroker::
消息发送失败的情况下是否重试其它的 broker。
+
默认值: `false`.
NOTE: 生产者其他更多参数请见:
https://github.com/alibaba/spring-cloud-alibaba/blob/rocketmq/spring-cloud-alibaba-starters/spring-cloud-starter-stream-rocketmq/src/main/java/com/alibaba/cloud/stream/binder/rocketmq/properties/RocketMQProducerProperties.java[com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQProducerProperties]
=== 阿里云 MQ 服务
使用阿里云 MQ 服务需要配置 AccessKey、SecretKey 以及云上的 NameServer 地址。
NOTE: 0.1.2 & 0.2.2 & 0.9.0 才支持该功能
```properties
spring.cloud.stream.rocketmq.binder.access-key=YourAccessKey
spring.cloud.stream.rocketmq.binder.secret-key=YourSecretKey
spring.cloud.stream.rocketmq.binder.name-server=NameServerInMQ
```
NOTE: topic 和 group 请以 实例id% 为前缀进行配置。比如 topic 为 "test",需要配置成 "实例id%test"
.NameServer 的获取(配置中请去掉 http:// 前缀)
image::https://spring-cloud-alibaba.oss-cn-beijing.aliyuncs.com/MQ.png[]

@ -19,6 +19,8 @@ include::sentinel.adoc[]
include::dubbo.adoc[]
include::rocketmq-new.adoc[]
include::rocketmq.adoc[]
include::ans.adoc[]

@ -294,6 +294,16 @@ The followings shows how a service instance accesses the Endpoint:
}
----
=== Weight Route
==== Ribbon
.application.properties
[source,properties]
----
[service_name].ribbon.NFLoadBalancerRuleClassName=com.alibaba.cloud.nacos.ribbon.NacosRule
----
=== More Information about Nacos Discovery Starter Configurations
The following shows the other configurations of the starter of Nacos Discovery:

@ -57,7 +57,7 @@ Send messages:
sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer
```
Output when the message is successfuly sent: `SendResult [sendStatus=SEND_OK, msgId= ...`
Output when the message is successfully sent: `SendResult [sendStatus=SEND_OK, msgId= ...`
Receive messages:

@ -82,7 +82,7 @@ Job Description Empty
Custom Parameters Empty
----
The job above is a “Simple Single-Server Job”, and speficied a Cron expression of "0 * * * * ?" . This means that the job will be executed once and once only in every minute.
The job above is a “Simple Single-Server Job”, and specified a Cron expression of "0 * * * * ?" . This means that the job will be executed once and once only in every minute.
For more job types, refer to https://help.aliyun.com/document_detail/43136.html[SchedulerX Documentation].

@ -18,18 +18,18 @@ spring.cloud.nacos.password=nacos
#spring.cloud.nacos.config.shared-data-ids=common.properties,base-common.properties
## recommended.
spring.cloud.nacos.config.shared-configs[0].data-id= test2.yaml
spring.cloud.nacos.config.shared-configs[0].data-id=test2.yaml
spring.cloud.nacos.config.shared-configs[0].refresh=true
## the default value is 'DEFAULT_GROUP' , if not specified.
spring.cloud.nacos.config.shared-configs[0].group= GROUP_APP1
spring.cloud.nacos.config.shared-configs[0].group=GROUP_APP1
## not recommended.
#spring.cloud.nacos.config.ext-config[0]=ext.properties
## recommended.
spring.cloud.nacos.config.extension-configs[0].data-id= extension1.properties
spring.cloud.nacos.config.extension-configs[0].data-id=extension1.properties
spring.cloud.nacos.config.extension-configs[0].refresh=true
spring.cloud.nacos.config.extension-configs[1].data-id= test1.yml
spring.cloud.nacos.config.extension-configs[1].refresh= true
spring.cloud.nacos.config.extension-configs[1].data-id=test1.yml
spring.cloud.nacos.config.extension-configs[1].refresh=true

@ -2,6 +2,7 @@ spring.application.name=service-consumer
server.port=18083
management.endpoints.web.exposure.include=*
spring.cloud.nacos.discovery.server-addr=127.0.0.1:8848
spring.cloud.nacos.discovery.fail-fast=true
spring.cloud.nacos.username=nacos
spring.cloud.nacos.password=nacos

@ -20,7 +20,7 @@ import java.util.stream.Collectors;
import java.util.stream.Stream;
import com.alibaba.cloud.examples.RocketMQProduceApplication.MySource;
import com.alibaba.cloud.stream.binder.rocketmq.contants.RocketMQConst;
import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst;
import org.apache.rocketmq.common.message.MessageConst;
import org.springframework.beans.factory.annotation.Autowired;

@ -73,7 +73,7 @@ Provider端在application.properties文件中定义dubbo相关的配置比如
定义具体的服务:
@Service(
@DubboService(
version = "${foo.service.version}",
application = "${dubbo.application.id}",
protocol = "${dubbo.protocol.id}",
@ -111,7 +111,7 @@ Consumer端在服务调用之前先定义限流规则。
根据Provider端中发布的定义使用Dubbo的@Reference注解注入服务对应的Bean
@Reference(version = "${foo.service.version}", application = "${dubbo.application.id}",
@DubboReference(version = "${foo.service.version}", application = "${dubbo.application.id}",
path = "dubbo://localhost:12345", timeout = 30000)
private FooService fooService;

@ -16,14 +16,14 @@
package com.alibaba.cloud.examples;
import org.apache.dubbo.config.annotation.Reference;
import org.apache.dubbo.config.annotation.DubboReference;
/**
* @author fangjian
*/
public class FooServiceConsumer {
@Reference(version = "${foo.service.version}",
@DubboReference(version = "${foo.service.version}",
application = "${dubbo.application.id}",
url = "dubbo://localhost:12345?version=1.0.0", timeout = 30000)
private FooService fooService;

@ -16,12 +16,12 @@
package com.alibaba.cloud.examples;
import org.apache.dubbo.config.annotation.Service;
import org.apache.dubbo.config.annotation.DubboService;
/**
* @author fangjian
*/
@Service(version = "${foo.service.version}", application = "${dubbo.application.id}",
@DubboService(version = "${foo.service.version}", application = "${dubbo.application.id}",
protocol = "${dubbo.protocol.id}", registry = "${dubbo.registry.id}")
public class FooServiceImpl implements FooService {

@ -114,6 +114,8 @@ public class EchoController {
- 启动nacos 注册中心
- 启动sentinel
- 启动服务提供方:
1. IDE直接启动找到主类 `ProviderApplication`,执行 main 方法启动应用。
@ -123,3 +125,5 @@ public class EchoController {
1. IDE直接启动找到主类 `ConsumerApplication`,执行 main 方法启动应用。
2. 打包编译后启动:首先执行 `mvn clean package` 将工程编译打包,然后执行 `java -jar sentinel-feign-consumer-example.jar`启动应用。
- 启动之后Sentinel Dashboard可能看不见service-consumer服务的详细信息多请求几次接口即可。

@ -8,6 +8,9 @@ spring:
nacos:
discovery:
server-addr: 127.0.0.1:8848
sentinel:
transport:
dashboard: 127.0.0.1:8081
feign:
sentinel:

@ -10,7 +10,6 @@
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-dubbo-client-sample</artifactId>
<name>Spring Cloud Dubbo Client Sample</name>

@ -12,7 +12,11 @@ spring:
allow-bean-definition-overriding: true
cloud:
nacos:
username: nacos
password: nacos
discovery:
server-addr: 127.0.0.1:8848
username: nacos
password: nacos
server-addr: 127.0.0.1:8848
namespace: public
server:
port: 8080

@ -1,4 +1,6 @@
dubbo:
cloud:
subscribed-services: ${spring.application.name}
scan:
base-packages: com.alibaba.cloud.dubbo.bootstrap
protocol:

@ -132,7 +132,6 @@ public class DataSourcePropertiesConfiguration {
if (!ObjectUtils.isEmpty(field.get(this))) {
return field.getName();
}
return null;
}
catch (IllegalAccessException e) {
// won't happen

@ -86,9 +86,8 @@ public abstract class SentinelConverter<T extends Object>
});
for (Object obj : sourceArray) {
String item = null;
try {
item = objectMapper.writeValueAsString(obj);
String item = objectMapper.writeValueAsString(obj);
Optional.ofNullable(convertRule(item))
.ifPresent(convertRule -> ruleCollection.add(convertRule));
}

@ -60,9 +60,14 @@ public class NacosDataSourceFactoryBean implements FactoryBean<NacosDataSource>
properties.setProperty(PropertyKeyConst.SERVER_ADDR, this.serverAddr);
}
else {
properties.setProperty(PropertyKeyConst.ENDPOINT, this.endpoint);
}
if (!StringUtils.isEmpty(this.accessKey)) {
properties.setProperty(PropertyKeyConst.ACCESS_KEY, this.accessKey);
}
if (!StringUtils.isEmpty(this.secretKey)) {
properties.setProperty(PropertyKeyConst.SECRET_KEY, this.secretKey);
properties.setProperty(PropertyKeyConst.ENDPOINT, this.endpoint);
}
if (!StringUtils.isEmpty(this.namespace)) {
properties.setProperty(PropertyKeyConst.NAMESPACE, this.namespace);

@ -32,6 +32,16 @@ public class NacosConfigHealthIndicator extends AbstractHealthIndicator {
private final ConfigService configService;
/**
* status up .
*/
private final String STATUS_UP = "UP";
/**
* status down .
*/
private final String STATUS_DOWN = "DOWN";
public NacosConfigHealthIndicator(ConfigService configService) {
this.configService = configService;
}
@ -43,10 +53,10 @@ public class NacosConfigHealthIndicator extends AbstractHealthIndicator {
// Set the status to Builder
builder.status(status);
switch (status) {
case "UP":
case STATUS_UP:
builder.up();
break;
case "DOWN":
case STATUS_DOWN:
builder.down();
break;
default:

@ -207,6 +207,12 @@ public class NacosDiscoveryProperties {
*/
private boolean ephemeral = true;
/**
* Throw exceptions during service registration if true, otherwise, log error
* (defaults to true).
*/
private boolean failFast = true;
@Autowired
private InetUtils inetUtils;
@ -486,6 +492,14 @@ public class NacosDiscoveryProperties {
this.ephemeral = ephemeral;
}
public boolean isFailFast() {
return failFast;
}
public void setFailFast(boolean failFast) {
this.failFast = failFast;
}
@Override
public boolean equals(Object o) {
if (this == o) {
@ -510,6 +524,7 @@ public class NacosDiscoveryProperties {
&& Objects.equals(secretKey, that.secretKey)
&& Objects.equals(heartBeatInterval, that.heartBeatInterval)
&& Objects.equals(heartBeatTimeout, that.heartBeatTimeout)
&& Objects.equals(failFast, that.failFast)
&& Objects.equals(ipDeleteTimeout, that.ipDeleteTimeout);
}
@ -519,7 +534,7 @@ public class NacosDiscoveryProperties {
watchDelay, logName, service, weight, clusterName, group,
namingLoadCacheAtStart, registerEnabled, ip, networkInterface, port,
secure, accessKey, secretKey, heartBeatInterval, heartBeatTimeout,
ipDeleteTimeout, instanceEnabled, ephemeral);
ipDeleteTimeout, instanceEnabled, ephemeral, failFast);
}
@Override
@ -535,7 +550,7 @@ public class NacosDiscoveryProperties {
+ ", port=" + port + ", secure=" + secure + ", accessKey='" + accessKey
+ '\'' + ", secretKey='" + secretKey + '\'' + ", heartBeatInterval="
+ heartBeatInterval + ", heartBeatTimeout=" + heartBeatTimeout
+ ", ipDeleteTimeout=" + ipDeleteTimeout + '}';
+ ", ipDeleteTimeout=" + ipDeleteTimeout + ", failFast=" + failFast + '}';
}
public void overrideFromEnv(Environment env) {

@ -112,9 +112,14 @@ public class NacosServiceManager {
}
public void nacosServiceShutDown() throws NacosException {
this.namingService.shutDown();
namingService = null;
namingMaintainService = null;
if (Objects.nonNull(this.namingService)) {
this.namingService.shutDown();
this.namingService = null;
}
if (Objects.nonNull(this.namingMaintainService)) {
this.namingMaintainService.shutDown();
this.namingMaintainService = null;
}
}
@EventListener

@ -20,7 +20,6 @@ import com.alibaba.cloud.nacos.ConditionalOnNacosDiscoveryEnabled;
import com.alibaba.cloud.nacos.NacosDiscoveryProperties;
import com.alibaba.cloud.nacos.NacosServiceManager;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.AutoConfigureBefore;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
@ -32,7 +31,6 @@ import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.client.discovery.simple.SimpleDiscoveryClientAutoConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
/**
* @author xiaojing
@ -58,10 +56,8 @@ public class NacosDiscoveryClientConfiguration {
@ConditionalOnProperty(value = "spring.cloud.nacos.discovery.watch.enabled",
matchIfMissing = true)
public NacosWatch nacosWatch(NacosServiceManager nacosServiceManager,
NacosDiscoveryProperties nacosDiscoveryProperties,
ObjectProvider<ThreadPoolTaskScheduler> taskExecutorObjectProvider) {
return new NacosWatch(nacosServiceManager, nacosDiscoveryProperties,
taskExecutorObjectProvider);
NacosDiscoveryProperties nacosDiscoveryProperties) {
return new NacosWatch(nacosServiceManager, nacosDiscoveryProperties);
}
}

@ -66,6 +66,14 @@ public class NacosWatch implements ApplicationEventPublisherAware, SmartLifecycl
private final ThreadPoolTaskScheduler taskScheduler;
public NacosWatch(NacosServiceManager nacosServiceManager,
NacosDiscoveryProperties properties) {
this.nacosServiceManager = nacosServiceManager;
this.properties = properties;
this.taskScheduler = getTaskScheduler();
}
@Deprecated
public NacosWatch(NacosServiceManager nacosServiceManager,
NacosDiscoveryProperties properties,
ObjectProvider<ThreadPoolTaskScheduler> taskScheduler) {

@ -31,6 +31,16 @@ import org.springframework.boot.actuate.health.HealthIndicator;
*/
public class NacosDiscoveryHealthIndicator extends AbstractHealthIndicator {
/**
* status up.
*/
private static final String STATUS_UP = "UP";
/**
* status down.
*/
private static final String STATUS_DOWN = "DOWN";
private final NamingService namingService;
public NacosDiscoveryHealthIndicator(NamingService namingService) {
@ -44,10 +54,10 @@ public class NacosDiscoveryHealthIndicator extends AbstractHealthIndicator {
// Set the status to Builder
builder.status(status);
switch (status) {
case "UP":
case STATUS_UP:
builder.up();
break;
case "DOWN":
case STATUS_DOWN:
builder.down();
break;
default:

@ -76,11 +76,15 @@ public class NacosServiceRegistry implements ServiceRegistry<Registration> {
instance.getIp(), instance.getPort());
}
catch (Exception e) {
log.error("nacos registry, {} register failed...{},", serviceId,
registration.toString(), e);
// rethrow a RuntimeException if the registration is failed.
// issue : https://github.com/alibaba/spring-cloud-alibaba/issues/1132
rethrowRuntimeException(e);
if (nacosDiscoveryProperties.isFailFast()) {
log.error("nacos registry, {} register failed...{},", serviceId,
registration.toString(), e);
rethrowRuntimeException(e);
}
else {
log.warn("Failfast is false. {} register failed...{},", serviceId,
registration.toString(), e);
}
}
}
@ -155,8 +159,10 @@ public class NacosServiceRegistry implements ServiceRegistry<Registration> {
public Object getStatus(Registration registration) {
String serviceName = registration.getServiceId();
String group = nacosDiscoveryProperties.getGroup();
try {
List<Instance> instances = namingService().getAllInstances(serviceName);
List<Instance> instances = namingService().getAllInstances(serviceName,
group);
for (Instance instance : instances) {
if (instance.getIp().equalsIgnoreCase(nacosDiscoveryProperties.getIp())
&& instance.getPort() == nacosDiscoveryProperties.getPort()) {

@ -86,13 +86,23 @@ public class SentinelProtectInterceptor implements ClientHttpRequestInterceptor
Tracer.trace(
new IllegalStateException("RestTemplate ErrorHandler has error"));
}
return response;
}
catch (Throwable e) {
if (!BlockException.isBlockException(e)) {
Tracer.trace(e);
if (BlockException.isBlockException(e)) {
return handleBlockException(request, body, execution, (BlockException) e);
}
else {
return handleBlockException(request, body, execution, (BlockException) e);
Tracer.traceEntry(e, hostEntry);
if (e instanceof IOException) {
throw (IOException) e;
}
else if (e instanceof RuntimeException) {
throw (RuntimeException) e;
}
else {
throw new IOException(e);
}
}
}
finally {
@ -103,7 +113,6 @@ public class SentinelProtectInterceptor implements ClientHttpRequestInterceptor
hostEntry.exit();
}
}
return response;
}
private ClientHttpResponse handleBlockException(HttpRequest request, byte[] body,

@ -80,34 +80,37 @@ public final class SentinelFeign {
// using reflect get fallback and fallbackFactory properties from
// FeignClientFactoryBean because FeignClientFactoryBean is a package
// level class, we can not use it in our package
Object feignClientFactoryBean = Builder.this.applicationContext
.getBean("&" + target.type().getName());
Class fallback = (Class) getFieldValue(feignClientFactoryBean,
"fallback");
Class fallbackFactory = (Class) getFieldValue(feignClientFactoryBean,
"fallbackFactory");
String beanName = (String) getFieldValue(feignClientFactoryBean,
"contextId");
if (!StringUtils.hasText(beanName)) {
beanName = (String) getFieldValue(feignClientFactoryBean, "name");
}
Object fallbackInstance;
FallbackFactory fallbackFactoryInstance;
// check fallback and fallbackFactory properties
if (void.class != fallback) {
fallbackInstance = getFromContext(beanName, "fallback", fallback,
target.type());
return new SentinelInvocationHandler(target, dispatch,
new FallbackFactory.Default(fallbackInstance));
}
if (void.class != fallbackFactory) {
fallbackFactoryInstance = (FallbackFactory) getFromContext(
beanName, "fallbackFactory", fallbackFactory,
FallbackFactory.class);
return new SentinelInvocationHandler(target, dispatch,
fallbackFactoryInstance);
Object feignClientFactoryBean = SentinelTargeterAspect
.getFeignClientFactoryBean();
if (feignClientFactoryBean != null) {
Class fallback = (Class) getFieldValue(feignClientFactoryBean,
"fallback");
Class fallbackFactory = (Class) getFieldValue(
feignClientFactoryBean, "fallbackFactory");
String beanName = (String) getFieldValue(feignClientFactoryBean,
"contextId");
if (!StringUtils.hasText(beanName)) {
beanName = (String) getFieldValue(feignClientFactoryBean,
"name");
}
Object fallbackInstance;
FallbackFactory fallbackFactoryInstance;
// check fallback and fallbackFactory properties
if (void.class != fallback) {
fallbackInstance = getFromContext(beanName, "fallback",
fallback, target.type());
return new SentinelInvocationHandler(target, dispatch,
new FallbackFactory.Default(fallbackInstance));
}
if (void.class != fallbackFactory) {
fallbackFactoryInstance = (FallbackFactory) getFromContext(
beanName, "fallbackFactory", fallbackFactory,
FallbackFactory.class);
return new SentinelInvocationHandler(target, dispatch,
fallbackFactoryInstance);
}
}
return new SentinelInvocationHandler(target, dispatch);
}

@ -41,4 +41,11 @@ public class SentinelFeignAutoConfiguration {
return SentinelFeign.builder();
}
@Bean
@ConditionalOnProperty(name = "feign.sentinel.enabled")
@ConditionalOnClass(name = "org.springframework.cloud.openfeign.Targeter")
public SentinelTargeterAspect sentinelTargeterAspect() {
return new SentinelTargeterAspect();
}
}

@ -109,7 +109,7 @@ public class SentinelInvocationHandler implements InvocationHandler {
catch (Throwable ex) {
// fallback handle
if (!BlockException.isBlockException(ex)) {
Tracer.trace(ex);
Tracer.traceEntry(ex, entry);
}
if (fallbackFactory != null) {
try {

@ -0,0 +1,51 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.cloud.sentinel.feign;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
/**
* Record FeignClientFactoryBean to threadlocal, so that SentinelFeign can get it when
* creating SentinelInvocationHandler.
*
* @see com.alibaba.cloud.sentinel.feign.SentinelFeign.Builder
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
*/
@Aspect
public class SentinelTargeterAspect {
private static final ThreadLocal<Object> FEIGN_CLIENT_FACTORY_BEAN = new ThreadLocal<>();
public static Object getFeignClientFactoryBean() {
return FEIGN_CLIENT_FACTORY_BEAN.get();
}
@Around("execution(* org.springframework.cloud.openfeign.Targeter.target(..))")
public Object process(ProceedingJoinPoint pjp) throws Throwable {
Object factory = pjp.getArgs()[0];
try {
FEIGN_CLIENT_FACTORY_BEAN.set(factory);
return pjp.proceed();
}
finally {
FEIGN_CLIENT_FACTORY_BEAN.remove();
}
}
}

@ -17,6 +17,11 @@
<dependencies>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-alibaba-commons</artifactId>
</dependency>
<!-- Spring Boot dependencies -->
<dependency>
<groupId>org.springframework.boot</groupId>

@ -18,6 +18,7 @@ package com.alibaba.cloud.dubbo.autoconfigure;
import java.util.Collection;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
@ -50,6 +51,7 @@ import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.serviceregistry.Registration;
import org.springframework.cloud.client.serviceregistry.ServiceRegistry;
import org.springframework.cloud.consul.discovery.ConsulDiscoveryProperties;
import org.springframework.cloud.consul.serviceregistry.ConsulRegistration;
import org.springframework.cloud.netflix.eureka.EurekaInstanceConfigBean;
import org.springframework.cloud.netflix.eureka.serviceregistry.EurekaAutoServiceRegistration;
@ -61,6 +63,7 @@ import org.springframework.context.annotation.Conditional;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
import org.springframework.context.event.EventListener;
import org.springframework.util.StringUtils;
import static com.alibaba.cloud.dubbo.autoconfigure.DubboServiceRegistrationAutoConfiguration.CONSUL_AUTO_SERVICE_AUTO_CONFIGURATION_CLASS_NAME;
import static com.alibaba.cloud.dubbo.autoconfigure.DubboServiceRegistrationAutoConfiguration.EUREKA_CLIENT_AUTO_CONFIGURATION_CLASS_NAME;
@ -260,6 +263,9 @@ public class DubboServiceRegistrationAutoConfiguration {
@AutoConfigureOrder
class ConsulConfiguration {
@Autowired
private ConsulDiscoveryProperties consulDiscoveryProperties;
@EventListener(DubboBootstrapStartedEvent.class)
public void attachURLsIntoMetadataBeforeReRegist(
DubboBootstrapStartedEvent event) {
@ -284,16 +290,40 @@ public class DubboServiceRegistrationAutoConfiguration {
}));
}
@EventListener(ServiceInstancePreRegisteredEvent.class)
public void onServiceInstancePreRegistered(
ServiceInstancePreRegisteredEvent event) {
Registration registration = event.getSource();
attachURLsIntoMetadata((ConsulRegistration) registration);
}
private void attachURLsIntoMetadata(ConsulRegistration consulRegistration) {
NewService newService = consulRegistration.getService();
Map<String, String> serviceMetadata = dubboServiceMetadataRepository
.getDubboMetadataServiceMetadata();
if (!isEmpty(serviceMetadata)) {
List<String> tags = newService.getTags();
if (isEmpty(serviceMetadata)) {
return;
}
NewService newService = consulRegistration.getService();
if (consulDiscoveryProperties.isTagsAsMetadata()) {
for (Map.Entry<String, String> entry : serviceMetadata.entrySet()) {
tags.add(entry.getKey() + "=" + entry.getValue());
attAsTag(newService.getTags(), entry.getKey(), entry.getValue());
}
}
else {
newService.getMeta().putAll(serviceMetadata);
}
}
private void attAsTag(List<String> tags, String key, String value) {
Iterator<String> iter = tags.iterator();
while (iter.hasNext()) {
String tag = iter.next();
String[] tmp = tag.split("=");
if (StringUtils.pathEquals(tmp[0], key)) {
iter.remove();
}
}
tags.add(key + "=" + value);
}
}

@ -51,6 +51,10 @@ public class DubboCloudProperties {
private String registryType = DUBBO_CLOUD_REGISTRY_PROPERTY_VALUE;
private int maxReSubscribeMetadataTimes = 1000;
private int reSubscribeMetadataIntervial = 5;
public String getSubscribedServices() {
return subscribedServices;
}
@ -91,4 +95,20 @@ public class DubboCloudProperties {
this.registryType = registryType;
}
public int getMaxReSubscribeMetadataTimes() {
return maxReSubscribeMetadataTimes;
}
public void setMaxReSubscribeMetadataTimes(int maxReSubscribeMetadataTimes) {
this.maxReSubscribeMetadataTimes = maxReSubscribeMetadataTimes;
}
public int getReSubscribeMetadataIntervial() {
return reSubscribeMetadataIntervial;
}
public void setReSubscribeMetadataIntervial(int reSubscribeMetadataIntervial) {
this.reSubscribeMetadataIntervial = reSubscribeMetadataIntervial;
}
}

@ -0,0 +1,93 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.cloud.dubbo.metadata;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Map;
import org.apache.dubbo.common.logger.Logger;
import org.apache.dubbo.common.logger.LoggerFactory;
import org.springframework.cloud.client.ServiceInstance;
import static com.alibaba.cloud.dubbo.metadata.repository.DubboServiceMetadataRepository.EXPORTED_SERVICES_REVISION_PROPERTY_NAME;
import static java.nio.charset.StandardCharsets.UTF_8;
/**
* Copy from org.apache.dubbo.metadata.RevisionResolver.
*
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
*/
public final class RevisionResolver {
/**
* The param key in url.
*/
public static final String SCA_REVSION_KEY = "sca_revision";
private static final String EMPTY_REVISION = "0";
private static final Logger logger = LoggerFactory.getLogger(RevisionResolver.class);
private static final char[] hexDigits = { '0', '1', '2', '3', '4', '5', '6', '7', '8',
'9', 'A', 'B', 'C', 'D', 'E', 'F' };
private static MessageDigest mdInst;
static {
try {
mdInst = MessageDigest.getInstance("MD5");
}
catch (NoSuchAlgorithmException e) {
logger.error("Failed to calculate metadata revision", e);
}
}
private RevisionResolver() {
}
public static String getEmptyRevision() {
return EMPTY_REVISION;
}
public static String calRevision(String metadata) {
mdInst.update(metadata.getBytes(UTF_8));
byte[] md5 = mdInst.digest();
int j = md5.length;
char[] str = new char[j * 2];
int k = 0;
for (byte byte0 : md5) {
str[k++] = hexDigits[byte0 >>> 4 & 0xf];
str[k++] = hexDigits[byte0 & 0xf];
}
return new String(str);
}
public static String getRevision(ServiceInstance instance) {
Map<String, String> metadata = instance.getMetadata();
String revision = metadata.get(EXPORTED_SERVICES_REVISION_PROPERTY_NAME);
if (revision == null) {
revision = RevisionResolver.getEmptyRevision();
}
return revision;
}
}

@ -0,0 +1,232 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.cloud.dubbo.metadata;
import java.io.Serializable;
import java.lang.reflect.Method;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.SortedSet;
import java.util.TreeMap;
import java.util.TreeSet;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.common.compiler.support.ClassUtils;
import org.apache.dubbo.common.utils.StringUtils;
import static org.apache.dubbo.common.constants.CommonConstants.GROUP_CHAR_SEPARATOR;
import static org.apache.dubbo.common.constants.CommonConstants.GROUP_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.INTERFACE_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.METHODS_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.PID_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.TIMESTAMP_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY;
/**
* Copy from org.apache.dubbo.metadata.MetadataInfo.ServiceInfo.
*
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
*/
public class ServiceInfo implements Serializable {
private static final long serialVersionUID = -258557978718735302L;
private String name;
private String group;
private String version;
private String protocol;
private String path; // most of the time, path is the same with the interface name.
private Map<String, String> params;
// params configured on consumer side,
private transient Map<String, String> consumerParams;
// service + group + version
private transient String serviceKey;
// service + group + version + protocol
private transient String matchKey;
private transient URL url;
private static final Set<String> IGNORE_KEYS = new HashSet<>();
static {
IGNORE_KEYS.add(TIMESTAMP_KEY);
IGNORE_KEYS.add(PID_KEY);
IGNORE_KEYS.add(INTERFACE_KEY);
IGNORE_KEYS.add(METHODS_KEY);
}
public ServiceInfo(URL url) {
this(url.getServiceInterface(), url.getParameter(GROUP_KEY),
url.getParameter(VERSION_KEY), url.getProtocol(), url.getPath(), null);
this.url = url;
Map<String, String> params = new TreeMap<>();
url.getParameters().forEach((k, v) -> {
if (IGNORE_KEYS.contains(k)) {
return;
}
params.put(k, v);
});
this.params = params;
}
public ServiceInfo(String name, String group, String version, String protocol,
String path, Map<String, String> params) {
this.name = name;
this.group = group;
this.version = version;
this.protocol = protocol;
this.path = path;
this.params = params == null ? new HashMap<>() : params;
this.serviceKey = URL.buildKey(name, group, version);
this.matchKey = buildMatchKey();
}
public String getMatchKey() {
if (matchKey != null) {
return matchKey;
}
buildMatchKey();
return matchKey;
}
private String buildMatchKey() {
matchKey = getServiceKey();
if (StringUtils.isNotEmpty(protocol)) {
matchKey = getServiceKey() + GROUP_CHAR_SEPARATOR + protocol;
}
return matchKey;
}
public String getServiceKey() {
if (serviceKey != null) {
return serviceKey;
}
this.serviceKey = URL.buildKey(name, group, version);
return serviceKey;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getGroup() {
return group;
}
public void setGroup(String group) {
this.group = group;
}
public String getVersion() {
return version;
}
public void setVersion(String version) {
this.version = version;
}
public String getPath() {
return path;
}
public void setPath(String path) {
this.path = path;
}
public Map<String, String> getParams() {
if (params == null) {
return Collections.emptyMap();
}
return params;
}
public void setParams(Map<String, String> params) {
this.params = params;
}
public String getParameter(String key) {
if (consumerParams != null) {
String value = consumerParams.get(key);
if (value != null) {
return value;
}
}
return params.get(key);
}
public String toDescString() {
return this.getMatchKey() + getMethodSignaturesString() + getParams();
}
private String getMethodSignaturesString() {
SortedSet<String> methodStrings = new TreeSet<>();
Method[] methods = ClassUtils.forName(name).getMethods();
for (Method method : methods) {
methodStrings.add(method.toString());
}
return methodStrings.toString();
}
public URL getUrl() {
return url;
}
@Override
public boolean equals(Object obj) {
if (obj == null) {
return false;
}
if (!(obj instanceof ServiceInfo)) {
return false;
}
ServiceInfo serviceInfo = (ServiceInfo) obj;
return this.getMatchKey().equals(serviceInfo.getMatchKey())
&& this.getParams().equals(serviceInfo.getParams());
}
@Override
public int hashCode() {
return Objects.hash(getMatchKey(), getParams());
}
@Override
public String toString() {
return "service{" + "name='" + name + "'," + "group='" + group + "',"
+ "version='" + version + "'," + "protocol='" + protocol + "',"
+ "params=" + params + "," + "consumerParams=" + consumerParams + "}";
}
}

@ -16,9 +16,11 @@
package com.alibaba.cloud.dubbo.metadata.repository;
import java.util.ArrayList;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Objects;
@ -31,6 +33,8 @@ import com.alibaba.cloud.dubbo.env.DubboCloudProperties;
import com.alibaba.cloud.dubbo.http.matcher.RequestMetadataMatcher;
import com.alibaba.cloud.dubbo.metadata.DubboRestServiceMetadata;
import com.alibaba.cloud.dubbo.metadata.RequestMetadata;
import com.alibaba.cloud.dubbo.metadata.RevisionResolver;
import com.alibaba.cloud.dubbo.metadata.ServiceInfo;
import com.alibaba.cloud.dubbo.metadata.ServiceRestMetadata;
import com.alibaba.cloud.dubbo.registry.event.SubscribedServicesChangedEvent;
import com.alibaba.cloud.dubbo.service.DubboMetadataService;
@ -41,6 +45,8 @@ import com.alibaba.cloud.dubbo.util.JSONUtils;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.type.TypeFactory;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.common.constants.CommonConstants;
import org.apache.dubbo.common.utils.CollectionUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -95,9 +101,10 @@ public class DubboServiceMetadataRepository
public static final String DUBBO_METADATA_SERVICE_URLS_PROPERTY_NAME = METADATA_SERVICE_URLS_PROPERTY_NAME;
/**
* The {@link String#format(String, Object...) pattern} of dubbo protocols port.
* The key of dubbo metadata revision. copyed from
* ServiceInstanceMetadataUtils.EXPORTED_SERVICES_REVISION_PROPERTY_NAME.
*/
public static final String DUBBO_PROTOCOLS_PORT_PROPERTY_NAME_PATTERN = "dubbo.protocols.%s.port";
public static String EXPORTED_SERVICES_REVISION_PROPERTY_NAME = "dubbo.metadata.revision";
private final Logger logger = LoggerFactory.getLogger(getClass());
@ -115,14 +122,11 @@ public class DubboServiceMetadataRepository
*/
private final MultiValueMap<String, URL> allExportedURLs = new LinkedMultiValueMap<>();
// =================================== Registration
// =================================== //
// ======================== Registration ======================== //
// ====================================================================================
// //
// ============================================================== //
// =================================== Subscription
// =================================== //
// ======================== Subscription ======================== //
/**
* A Map to store REST metadata temporary, its' key is the special service name for a
* Dubbo service, the value is a JSON content of JAX-RS or Spring MVC REST metadata
@ -132,24 +136,20 @@ public class DubboServiceMetadataRepository
private ApplicationEventPublisher applicationEventPublisher;
// ====================================================================================
// //
// =============================================================== //
// =================================== REST Metadata
// ================================== //
// ======================== REST Metadata ======================== //
private volatile Set<String> subscribedServices = emptySet();
/**
* Key is application name Value is Map&lt;RequestMetadata,
* DubboRestServiceMetadata&gt;.
*/
private Map<String, Map<RequestMetadataMatcher, DubboRestServiceMetadata>> dubboRestServiceMetadataRepository = newHashMap();
private final Map<String, Map<RequestMetadataMatcher, DubboRestServiceMetadata>> dubboRestServiceMetadataRepository = newHashMap();
// ====================================================================================
// //
// =============================================================== //
// =================================== Dependencies
// =================================== //
// ======================== Dependencies ========================= //
@Autowired
private DubboCloudProperties dubboCloudProperties;
@ -178,8 +178,7 @@ public class DubboServiceMetadataRepository
@Autowired
private DubboMetadataServiceExporter dubboMetadataServiceExporter;
// ====================================================================================
// //
// =============================================================== //
private static <K, V> Map<K, V> getMap(Map<String, Map<K, V>> repository,
String key) {
@ -187,12 +186,7 @@ public class DubboServiceMetadataRepository
}
private static <K, V> V getOrDefault(Map<K, V> source, K key, V defaultValue) {
V value = source.get(key);
if (value == null) {
value = defaultValue;
source.put(key, value);
}
return value;
return source.computeIfAbsent(key, k -> defaultValue);
}
private static <K, V> Map<K, V> newHashMap() {
@ -237,9 +231,6 @@ public class DubboServiceMetadataRepository
dispatchEvent(new SubscribedServicesChangedEvent(this, oldSubscribedServices,
newSubscribedServices));
// clear old one, help GC
oldSubscribedServices.clear();
return newSubscribedServices.stream();
}
@ -249,13 +240,14 @@ public class DubboServiceMetadataRepository
@Override
public void afterSingletonsInstantiated() {
initializeMetadata();
// inited by DubboCloudRegistry.preInit() @theonefx
// initializeMetadata();
}
/**
* Initialize the metadata.
*/
private void initializeMetadata() {
public void initializeMetadata() {
doGetSubscribedServices().forEach(this::initializeMetadata);
if (logger.isInfoEnabled()) {
logger.info("The metadata of Dubbo services has been initialized");
@ -302,12 +294,36 @@ public class DubboServiceMetadataRepository
addDubboMetadataServiceURLsMetadata(metadata, dubboMetadataServiceURLs);
addDubboProtocolsPortMetadata(metadata);
addRevision(metadata);
return Collections.unmodifiableMap(metadata);
}
private void addRevision(Map<String, String> metadata) {
metadata.put(EXPORTED_SERVICES_REVISION_PROPERTY_NAME, calAndGetRevision());
}
public String calAndGetRevision() {
if (CollectionUtils.isEmptyMap(allExportedURLs)) {
return RevisionResolver.getEmptyRevision();
}
else {
List<String> descs = new ArrayList<>(allExportedURLs.size());
for (Map.Entry<String, List<URL>> entry : allExportedURLs.entrySet()) {
entry.getValue().stream().map(ServiceInfo::new)
.map(ServiceInfo::toDescString).forEach(descs::add);
}
descs.sort(String::compareTo);
return RevisionResolver.calRevision(descs.toString());
}
}
private void removeDubboMetadataServiceURLs(List<URL> dubboMetadataServiceURLs) {
dubboMetadataServiceURLs.forEach(this::unexportURL);
dubboMetadataServiceURLs.stream().map(URL::getServiceKey).distinct()
.forEach(allExportedURLs::remove);
// dubboMetadataServiceURLs.forEach(this::unexportURL);
}
private void addDubboMetadataServiceURLsMetadata(Map<String, String> metadata,
@ -407,8 +423,34 @@ public class DubboServiceMetadataRepository
public List<URL> getExportedURLs(String serviceInterface, String group,
String version) {
String serviceKey = URL.buildKey(serviceInterface, group, version);
return allExportedURLs.getOrDefault(serviceKey, Collections.emptyList());
if (group != null) {
List<URL> urls = new LinkedList<>();
if (CommonConstants.ANY_VALUE.equals(group)) {
String serviceKey = URL.buildKey(serviceInterface, group, version);
String expectKey = serviceKey.substring(2);
for (String key : allExportedURLs.keySet()) {
if (key.endsWith(expectKey)) {
urls.addAll(allExportedURLs.get(key));
}
}
}
else {
String[] groups = group.split(CommonConstants.COMMA_SEPARATOR);
for (String expectKey : groups) {
String serviceKey = URL.buildKey(serviceInterface, expectKey,
version);
List<URL> urlList = allExportedURLs.get(serviceKey);
if (urlList != null) {
urls.addAll(urlList);
}
}
}
return urls;
}
else {
String serviceKey = URL.buildKey(serviceInterface, null, version);
return allExportedURLs.getOrDefault(serviceKey, Collections.emptyList());
}
}
/**

@ -0,0 +1,90 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.cloud.dubbo.registry;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.atomic.AtomicBoolean;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.registry.NotifyListener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import static org.apache.dubbo.common.URLBuilder.from;
import static org.apache.dubbo.common.constants.RegistryConstants.CATEGORY_KEY;
import static org.apache.dubbo.common.constants.RegistryConstants.EMPTY_PROTOCOL;
import static org.apache.dubbo.common.utils.CollectionUtils.isEmpty;
/**
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
*/
public abstract class AbstractServiceSubscribeHandler {
protected final Logger logger = LoggerFactory.getLogger(getClass());
protected final URL url;
protected final NotifyListener listener;
protected final DubboCloudRegistry registry;
public AbstractServiceSubscribeHandler(URL url, NotifyListener listener,
DubboCloudRegistry registry) {
this.url = url;
this.listener = listener;
this.registry = registry;
}
protected void notifyAllSubscribedURLs(URL url, List<URL> subscribedURLs,
NotifyListener listener) {
if (isEmpty(subscribedURLs)) {
// Add the EMPTY_PROTOCOL URL
listener.notify(Collections.singletonList(emptyURL(url)));
// if (isDubboMetadataServiceURL(url)) {
// if meta service change, and serviceInstances is zero, will clean up
// information about this client
// String serviceName = url.getParameter(GROUP_KEY);
// repository.removeMetadataAndInitializedService(serviceName, url);
// }
}
else {
// Notify all
listener.notify(subscribedURLs);
}
}
private URL emptyURL(URL url) {
// issue : When the last service provider is closed, the client still periodically
// connects to the last provider.n
// fix https://github.com/alibaba/spring-cloud-alibaba/issues/1259
return from(url).setProtocol(EMPTY_PROTOCOL).removeParameter(CATEGORY_KEY)
.build();
}
private final AtomicBoolean inited = new AtomicBoolean(false);
public void init() {
if (inited.compareAndSet(false, true)) {
doInit();
}
}
protected abstract void doInit();
}

@ -16,25 +16,27 @@
package com.alibaba.cloud.dubbo.registry;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.function.Supplier;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.stream.Collectors;
import com.alibaba.cloud.commons.lang.StringUtils;
import com.alibaba.cloud.dubbo.metadata.RevisionResolver;
import com.alibaba.cloud.dubbo.metadata.repository.DubboServiceMetadataRepository;
import com.alibaba.cloud.dubbo.registry.event.ServiceInstancesChangedEvent;
import com.alibaba.cloud.dubbo.service.DubboGenericServiceFactory;
import com.alibaba.cloud.dubbo.service.DubboMetadataService;
import com.alibaba.cloud.dubbo.service.DubboMetadataServiceProxy;
import com.alibaba.cloud.dubbo.util.DubboMetadataUtils;
import com.alibaba.cloud.dubbo.util.JSONUtils;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.common.URLBuilder;
import org.apache.dubbo.registry.NotifyListener;
import org.apache.dubbo.registry.support.FailbackRegistry;
import org.slf4j.Logger;
@ -44,25 +46,14 @@ import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.context.ApplicationListener;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.util.CollectionUtils;
import static java.lang.String.format;
import static java.util.Collections.emptyList;
import static java.util.stream.StreamSupport.stream;
import static org.apache.dubbo.common.URLBuilder.from;
import static org.apache.dubbo.common.constants.CommonConstants.CONSUMER;
import static org.apache.dubbo.common.constants.CommonConstants.GROUP_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.PID_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.PROTOCOL_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.PROVIDER;
import static org.apache.dubbo.common.constants.CommonConstants.PROVIDER_SIDE;
import static org.apache.dubbo.common.constants.CommonConstants.SIDE_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.TIMESTAMP_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY;
import static org.apache.dubbo.common.constants.RegistryConstants.CATEGORY_KEY;
import static org.apache.dubbo.common.constants.RegistryConstants.EMPTY_PROTOCOL;
import static org.apache.dubbo.common.utils.CollectionUtils.isEmpty;
import static org.apache.dubbo.registry.Constants.ADMIN_PROTOCOL;
import static org.apache.dubbo.registry.client.metadata.ServiceInstanceMetadataUtils.METADATA_SERVICE_URLS_PROPERTY_NAME;
import static org.springframework.util.StringUtils.hasText;
@ -71,22 +62,14 @@ import static org.springframework.util.StringUtils.hasText;
* Dubbo Cloud {@link FailbackRegistry} is based on Spring Cloud {@link DiscoveryClient}.
*
* @author <a href="mailto:mercyblitz@gmail.com">Mercy</a>
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
*/
public class DubboCloudRegistry extends FailbackRegistry {
/**
* The parameter name of {@link #servicesLookupInterval}.
*/
public static final String SERVICES_LOOKUP_INTERVAL_PARAM_NAME = "dubbo.services.lookup.interval";
public class DubboCloudRegistry extends FailbackRegistry
implements ApplicationListener<ServiceInstancesChangedEvent> {
protected static final String DUBBO_METADATA_SERVICE_CLASS_NAME = DubboMetadataService.class
.getName();
/**
* Caches the IDs of {@link ApplicationListener}.
*/
private static final Set<String> REGISTER_LISTENERS = new HashSet<>();
protected final Logger logger = LoggerFactory.getLogger(getClass());
private final DiscoveryClient discoveryClient;
@ -97,275 +80,362 @@ public class DubboCloudRegistry extends FailbackRegistry {
private final JSONUtils jsonUtils;
private final DubboGenericServiceFactory dubboGenericServiceFactory;
private final DubboMetadataUtils dubboMetadataUtils;
private final ConfigurableApplicationContext applicationContext;
private final ReSubscribeManager reSubscribeManager;
private final AtomicBoolean inited = new AtomicBoolean(false);
/**
* The interval in second of lookup service names(only for Dubbo-OPS).
* {subscribedURL : ServiceSubscribeHandler}.
*/
private final long servicesLookupInterval;
private final Map<URL, GenearalServiceSubscribeHandler> urlSubscribeHandlerMap = new ConcurrentHashMap<>();
private final ConfigurableApplicationContext applicationContext;
/**
* {appName: MetadataServiceSubscribeHandler}.
*/
private final Map<String, MetadataServiceSubscribeHandler> metadataSubscribeHandlerMap = new ConcurrentHashMap<>();
private final String currentApplicationName;
/**
* {appName : {revision: [instances]}}.
*/
private final Map<String, Map<String, List<ServiceInstance>>> serviceRevisionInstanceMap = new ConcurrentHashMap<>();
public DubboCloudRegistry(URL url, DiscoveryClient discoveryClient,
DubboServiceMetadataRepository repository,
DubboMetadataServiceProxy dubboMetadataConfigServiceProxy,
JSONUtils jsonUtils, DubboGenericServiceFactory dubboGenericServiceFactory,
ConfigurableApplicationContext applicationContext) {
JSONUtils jsonUtils, ConfigurableApplicationContext applicationContext) {
super(url);
this.servicesLookupInterval = url
.getParameter(SERVICES_LOOKUP_INTERVAL_PARAM_NAME, 60L);
this.discoveryClient = discoveryClient;
this.repository = repository;
this.dubboMetadataConfigServiceProxy = dubboMetadataConfigServiceProxy;
this.jsonUtils = jsonUtils;
this.dubboGenericServiceFactory = dubboGenericServiceFactory;
this.applicationContext = applicationContext;
this.dubboMetadataUtils = getBean(DubboMetadataUtils.class);
this.currentApplicationName = dubboMetadataUtils.getCurrentApplicationName();
this.reSubscribeManager = new ReSubscribeManager(this);
}
private void preInit() {
if (inited.compareAndSet(false, true)) {
Set<String> subscribeApps = getServices(null);
for (String appName : subscribeApps) {
List<ServiceInstance> instances = discoveryClient.getInstances(appName);
Map<String, List<ServiceInstance>> map = serviceRevisionInstanceMap
.computeIfAbsent(appName, k -> new HashMap<>());
for (ServiceInstance instance : instances) {
String revision = RevisionResolver.getRevision(instance);
List<ServiceInstance> list = map.computeIfAbsent(revision,
k -> new ArrayList<>());
list.add(instance);
}
if (map.size() == 0) {
logger.debug("APP {} preInited, instance siez is zero!!", appName);
}
else {
map.forEach((revision, list) -> logger.debug(
"APP {} revision {} preInited, instance size = {}", appName,
revision, list.size()));
}
}
metadataSubscribeHandlerMap.forEach((url, handler) -> handler.init());
urlSubscribeHandlerMap.forEach((url, handler) -> handler.init());
repository.initializeMetadata();
// meke sure everything prepared, then can listening
// ServiceInstanceChangeEvent
applicationContext.addApplicationListener(this);
logger.info("DubboCloudRegistry preInit Done.");
}
}
private <T> T getBean(Class<T> beanClass) {
protected <T> T getBean(Class<T> beanClass) {
return this.applicationContext.getBean(beanClass);
}
protected boolean shouldRegister(URL url) {
protected boolean shouldNotRegister(URL url) {
String side = url.getParameter(SIDE_KEY);
boolean should = PROVIDER_SIDE.equals(side); // Only register the Provider.
if (!should) {
if (logger.isDebugEnabled()) {
logger.debug("The URL[{}] should not be registered.", url.toString());
if (logger.isDebugEnabled()) {
if (!should) {
logger.debug("The URL should NOT!! be registered & unregistered [{}] .",
url);
}
else {
logger.debug("The URL should be registered & unregistered [{}] .", url);
}
}
return should;
return !should;
}
@Override
public final void doRegister(URL url) {
if (!shouldRegister(url)) {
return;
synchronized (this) {
preInit();
if (shouldNotRegister(url)) {
return;
}
repository.exportURL(url);
}
repository.exportURL(url);
}
@Override
public final void doUnregister(URL url) {
if (!shouldRegister(url)) {
return;
synchronized (this) {
preInit();
if (shouldNotRegister(url)) {
return;
}
repository.unexportURL(url);
}
repository.unexportURL(url);
}
@Override
public final void doSubscribe(URL url, NotifyListener listener) {
if (isAdminURL(url)) {
// TODO in future
if (logger.isWarnEnabled()) {
logger.warn("This feature about admin will be supported in the future.");
synchronized (this) {
preInit();
if (isAdminURL(url)) {
// TODO in future
if (logger.isWarnEnabled()) {
logger.warn(
"This feature about admin will be supported in the future.");
}
}
else if (isDubboMetadataServiceURL(url) && containsProviderCategory(url)) {
// for DubboMetadataService
String appName = getServiceName(url);
MetadataServiceSubscribeHandler handler = new MetadataServiceSubscribeHandler(
appName, url, listener, this, dubboMetadataUtils);
if (inited.get()) {
handler.init();
}
metadataSubscribeHandlerMap.put(appName, handler);
}
else if (isConsumerServiceURL(url)) {
// for general Dubbo Services
GenearalServiceSubscribeHandler handler = new GenearalServiceSubscribeHandler(
url, listener, this, repository, jsonUtils,
dubboMetadataConfigServiceProxy);
if (inited.get()) {
handler.init();
}
urlSubscribeHandlerMap.put(url, handler);
}
}
else if (isDubboMetadataServiceURL(url)) { // for DubboMetadataService
subscribeDubboMetadataServiceURLs(url, listener);
}
else { // for general Dubbo Services
subscribeURLs(url, listener);
}
}
private void subscribeURLs(URL url, NotifyListener listener) {
// Sync subscription
subscribeURLs(url, getServices(url), listener);
// Async subscription
registerServiceInstancesChangedListener(url,
new ApplicationListener<ServiceInstancesChangedEvent>() {
/**
* Process ServiceInstanceChangedEvent, refresh dubbo reference and metadata info.
*/
@Override
public void onApplicationEvent(ServiceInstancesChangedEvent event) {
private final URL url2subscribe = url;
String appName = event.getServiceName();
@Override
@Order
public void onApplicationEvent(ServiceInstancesChangedEvent event) {
Set<String> serviceNames = getServices(url);
List<ServiceInstance> instances = filter(event.getServiceInstances() != null
? event.getServiceInstances() : Collections.emptyList());
String serviceName = event.getServiceName();
Set<String> subscribedServiceNames = getServices(null);
if (serviceNames.contains(serviceName)) {
subscribeURLs(url, serviceNames, listener);
}
}
if (!subscribedServiceNames.contains(appName)) {
return;
}
@Override
public String toString() {
return "ServiceInstancesChangedEventListener:"
+ url.getServiceKey();
}
});
}
if (instances.size() == 0) {
logger.warn("APP {} instance changed, size changed zero!!!", appName);
}
else {
logger.info("APP {} instance changed, size changed to {}", appName,
instances.size());
}
// group by revision
Map<String, List<ServiceInstance>> newGroup = instances.stream()
.collect(Collectors.groupingBy(RevisionResolver::getRevision));
private void subscribeURLs(URL url, Set<String> serviceNames,
NotifyListener listener) {
synchronized (this) {
List<URL> subscribedURLs = new LinkedList<>();
Map<String, List<ServiceInstance>> oldGroup = serviceRevisionInstanceMap
.computeIfAbsent(appName, k -> new HashMap<>());
serviceNames.forEach(serviceName -> {
if (serviceInstanceNotChanged(oldGroup, newGroup)) {
logger.debug("APP {} instance changed, but nothing different", appName);
return;
}
subscribeURLs(url, subscribedURLs, serviceName,
() -> getServiceInstances(serviceName));
try {
});
// ensure that the service metadata is correct
refreshServiceMetadataInfo(appName, instances);
// Notify all
notifyAllSubscribedURLs(url, subscribedURLs, listener);
}
// then , refresh general service associated with current application
refreshGeneralServiceInfo(appName, oldGroup, newGroup);
private void registerServiceInstancesChangedListener(URL url,
ApplicationListener<ServiceInstancesChangedEvent> listener) {
String listenerId = generateId(url);
if (REGISTER_LISTENERS.add(listenerId)) {
applicationContext.addApplicationListener(listener);
// mark process successful
reSubscribeManager.onRefreshSuccess(event);
}
catch (Exception e) {
logger.error(String.format(
"APP %s instance changed, handler faild, try resubscribe",
appName), e);
reSubscribeManager.onRefreshFail(event);
}
}
}
private void subscribeURLs(URL subscribedURL, List<URL> subscribedURLs,
String serviceName,
Supplier<List<ServiceInstance>> serviceInstancesSupplier) {
List<ServiceInstance> serviceInstances = serviceInstancesSupplier.get();
subscribeURLs(subscribedURL, subscribedURLs, serviceName, serviceInstances);
}
private void refreshGeneralServiceInfo(String appName,
Map<String, List<ServiceInstance>> oldGroup,
Map<String, List<ServiceInstance>> newGroup) {
Set<URL> urls2refresh = new HashSet<>();
private void subscribeURLs(URL subscribedURL, List<URL> subscribedURLs,
String serviceName, List<ServiceInstance> serviceInstances) {
// compare with local
for (String revision : oldGroup.keySet()) {
if (CollectionUtils.isEmpty(serviceInstances)) {
if (logger.isWarnEnabled()) {
logger.warn(format("There is no instance in service[name : %s]",
serviceName));
if (!newGroup.containsKey(revision)) {
// all instances of this list with revision has losted
urlSubscribeHandlerMap.forEach((url, handler) -> {
if (handler.relatedWith(appName, revision)) {
handler.removeAppNameWithRevision(appName, revision);
urls2refresh.add(url);
}
});
logger.debug("Subscription app {} revision {} has all losted", appName,
revision);
}
}
List<URL> exportedURLs = getExportedURLs(subscribedURL, serviceName,
serviceInstances);
for (Map.Entry<String, List<ServiceInstance>> entry : newGroup.entrySet()) {
String revision = entry.getKey();
List<ServiceInstance> instanceList = entry.getValue();
/**
* Add the exported URLs from {@link MetadataService}
*/
subscribedURLs.addAll(exportedURLs);
}
private List<URL> getExportedURLs(URL subscribedURL, String serviceName,
List<ServiceInstance> serviceInstances) {
if (!oldGroup.containsKey(revision)) {
// this instance list of revision not exists
// should acquire urls
urlSubscribeHandlerMap.forEach(
(url, handler) -> handler.init(appName, revision, instanceList));
}
List<ServiceInstance> validServiceInstances = filter(serviceInstances);
urlSubscribeHandlerMap.forEach((url, handler) -> {
if (handler.relatedWith(appName, revision)) {
urls2refresh.add(url);
}
});
// If there is no valid ServiceInstance, return empty result
if (isEmpty(validServiceInstances)) {
if (logger.isWarnEnabled()) {
logger.warn(
"There is no instance from service[name : {}], and then Dubbo Service[key : {}] will not be "
+ "available , please make sure the further impact",
serviceName, subscribedURL.getServiceKey());
if (logger.isDebugEnabled()) {
logger.debug("Subscription app {} revision {} changed, instance list {}",
appName, revision,
instanceList.stream().map(
instance -> instance.getHost() + ":" + instance.getPort())
.collect(Collectors.toList()));
}
return emptyList();
}
List<URL> subscribedURLs = cloneExportedURLs(subscribedURL, serviceInstances);
// clear local service instances, help GC
validServiceInstances.clear();
serviceRevisionInstanceMap.put(appName, newGroup);
return subscribedURLs;
if (urls2refresh.size() == 0) {
logger.debug("Subscription app {}, no urls will be refreshed", appName);
}
else {
logger.debug("Subscription app {}, the following url will be refresh:{}",
appName, urls2refresh.stream().map(URL::getServiceKey)
.collect(Collectors.toList()));
for (URL url : urls2refresh) {
GenearalServiceSubscribeHandler handler = urlSubscribeHandlerMap.get(url);
if (handler == null) {
logger.warn("Subscription app {}, can't find handler for service {}",
appName, url.getServiceKey());
continue;
}
handler.refresh();
}
}
}
/**
* Clone the subscribed URLs based on the template URLs.
* @param subscribedURL the URL to be subscribed
* @param serviceInstances the list of {@link ServiceInstance service instances}
* @return non-null
*/
private List<URL> cloneExportedURLs(URL subscribedURL,
private void refreshServiceMetadataInfo(String serviceName,
List<ServiceInstance> serviceInstances) {
MetadataServiceSubscribeHandler handler = metadataSubscribeHandlerMap
.get(serviceName);
List<URL> clonedExportedURLs = new LinkedList<>();
serviceInstances.forEach(serviceInstance -> {
String host = serviceInstance.getHost();
getTemplateExportedURLs(subscribedURL, serviceInstances).stream()
.map(templateURL -> templateURL.removeParameter(TIMESTAMP_KEY))
.map(templateURL -> templateURL.removeParameter(PID_KEY))
.map(templateURL -> {
String protocol = templateURL.getProtocol();
Integer port = repository.getDubboProtocolPort(serviceInstance,
protocol);
if (Objects.equals(templateURL.getHost(), host)
&& Objects.equals(templateURL.getPort(), port)) { // use
// templateURL
// if
// equals
return templateURL;
}
if (port == null) {
if (logger.isWarnEnabled()) {
logger.warn(
"The protocol[{}] port of Dubbo service instance[host : {}] "
+ "can't be resolved",
protocol, host);
}
return null;
}
else {
URLBuilder clonedURLBuilder = from(templateURL) // remove the
// parameters from
// the template
// URL
.setHost(host) // reset the host
.setPort(port); // reset the port
return clonedURLBuilder.build();
}
}).filter(Objects::nonNull).forEach(clonedExportedURLs::add);
});
return clonedExportedURLs;
if (handler == null) {
logger.warn("Subscription app {}, can't find metadata handler", serviceName);
return;
}
handler.refresh(serviceInstances);
}
private List<URL> getTemplateExportedURLs(URL subscribedURL,
List<ServiceInstance> serviceInstances) {
private boolean serviceInstanceNotChanged(Map<String, List<ServiceInstance>> oldGroup,
Map<String, List<ServiceInstance>> newGroup) {
if (newGroup.size() != oldGroup.size()) {
return false;
}
DubboMetadataService dubboMetadataService = getProxy(serviceInstances);
for (Map.Entry<String, List<ServiceInstance>> entry : newGroup.entrySet()) {
String appName = entry.getKey();
List<ServiceInstance> newInstances = entry.getValue();
List<URL> templateExportedURLs = emptyList();
if (!oldGroup.containsKey(appName)) {
return false;
}
if (dubboMetadataService != null) {
templateExportedURLs = getExportedURLs(dubboMetadataService, subscribedURL);
}
else {
if (logger.isWarnEnabled()) {
logger.warn(
"The metadata of Dubbo service[key : {}] still can't be found, it could effect the further "
+ "Dubbo service invocation",
subscribedURL.getServiceKey());
List<ServiceInstance> oldInstances = oldGroup.get(appName);
if (newInstances.size() != oldInstances.size()) {
return false;
}
boolean matched = newInstances.stream().allMatch(newInstance -> {
for (ServiceInstance oldInstance : oldInstances) {
if (instanceSame(newInstance, oldInstance)) {
return true;
}
}
return false;
});
if (!matched) {
return false;
}
}
return templateExportedURLs;
return true;
}
private DubboMetadataService getProxy(List<ServiceInstance> serviceInstances) {
return dubboMetadataConfigServiceProxy.getProxy(serviceInstances);
private boolean instanceSame(ServiceInstance newInstance,
ServiceInstance oldInstance) {
if (!StringUtils.equals(newInstance.getInstanceId(),
oldInstance.getInstanceId())) {
return false;
}
if (!StringUtils.equals(newInstance.getHost(), oldInstance.getHost())) {
return false;
}
if (!StringUtils.equals(newInstance.getServiceId(), oldInstance.getServiceId())) {
return false;
}
if (!StringUtils.equals(newInstance.getScheme(), oldInstance.getScheme())) {
return false;
}
if (oldInstance.getPort() != newInstance.getPort()) {
return false;
}
if (!oldInstance.getMetadata().equals(newInstance.getMetadata())) {
return false;
}
return true;
}
private List<ServiceInstance> filter(Collection<ServiceInstance> serviceInstances) {
@ -380,40 +450,14 @@ public class DubboCloudRegistry extends FailbackRegistry {
private Set<String> getServices(URL url) {
Set<String> subscribedServices = repository.getSubscribedServices();
if (subscribedServices.contains("*")) {
subscribedServices = new HashSet<>(discoveryClient.getServices());
}
// TODO Add the filter feature
return subscribedServices;
}
private void notifyAllSubscribedURLs(URL url, List<URL> subscribedURLs,
NotifyListener listener) {
if (isEmpty(subscribedURLs)) {
// Add the EMPTY_PROTOCOL URL
subscribedURLs.add(emptyURL(url));
// if (isDubboMetadataServiceURL(url)) {
// if meta service change, and serviceInstances is zero, will clean up
// information about this client
// String serviceName = url.getParameter(GROUP_KEY);
// repository.removeMetadataAndInitializedService(serviceName, url);
// }
}
if (logger.isDebugEnabled()) {
logger.debug("The subscribed URL[{}] will notify all URLs : {}", url,
subscribedURLs);
}
// Notify all
listener.notify(subscribedURLs);
}
private List<ServiceInstance> getServiceInstances(Iterable<String> serviceNames) {
return stream(serviceNames.spliterator(), false).map(this::getServiceInstances)
.flatMap(Collection::stream).collect(Collectors.toList());
}
private List<ServiceInstance> getServiceInstances(String serviceName) {
List<ServiceInstance> getServiceInstances(String serviceName) {
return hasText(serviceName) ? doGetServiceInstances(serviceName) : emptyList();
}
@ -430,106 +474,14 @@ public class DubboCloudRegistry extends FailbackRegistry {
return serviceInstances;
}
private String generateId(URL url) {
return url.toString();
}
private URL emptyURL(URL url) {
// issue : When the last service provider is closed, the client still periodically
// connects to the last provider.n
// fix https://github.com/alibaba/spring-cloud-alibaba/issues/1259
return from(url).setProtocol(EMPTY_PROTOCOL).removeParameter(CATEGORY_KEY)
.build();
}
private List<URL> getExportedURLs(DubboMetadataService dubboMetadataService,
URL subscribedURL) {
String serviceInterface = subscribedURL.getServiceInterface();
String group = subscribedURL.getParameter(GROUP_KEY);
String version = subscribedURL.getParameter(VERSION_KEY);
// The subscribed protocol may be null
String subscribedProtocol = subscribedURL.getParameter(PROTOCOL_KEY);
String exportedURLsJSON = dubboMetadataService.getExportedURLs(serviceInterface,
group, version);
return jsonUtils.toURLs(exportedURLsJSON).stream()
.filter(exportedURL -> subscribedProtocol == null
|| subscribedProtocol.equalsIgnoreCase(exportedURL.getProtocol()))
.collect(Collectors.toList());
}
private void subscribeDubboMetadataServiceURLs(URL subscribedURL,
NotifyListener listener) {
// Sync subscription
subscribeDubboMetadataServiceURLs(subscribedURL, listener,
getServiceName(subscribedURL));
// Sync subscription
if (containsProviderCategory(subscribedURL)) {
registerServiceInstancesChangedListener(subscribedURL,
new ApplicationListener<ServiceInstancesChangedEvent>() {
private final URL url2subscribe = subscribedURL;
@Override
@Order(Ordered.LOWEST_PRECEDENCE - 1)
public void onApplicationEvent(
ServiceInstancesChangedEvent event) {
String sourceServiceName = event.getServiceName();
String serviceName = getServiceName(subscribedURL);
if (Objects.equals(sourceServiceName, serviceName)) {
subscribeDubboMetadataServiceURLs(subscribedURL, listener,
sourceServiceName);
}
}
@Override
public String toString() {
return "ServiceInstancesChangedEventListener:"
+ subscribedURL.getServiceKey();
}
});
}
}
// the group of DubboMetadataService is current application name
private String getServiceName(URL subscribedURL) {
return subscribedURL.getParameter(GROUP_KEY);
}
private void subscribeDubboMetadataServiceURLs(URL subscribedURL,
NotifyListener listener, String serviceName) {
String serviceInterface = subscribedURL.getServiceInterface();
String version = subscribedURL.getParameter(VERSION_KEY);
String protocol = subscribedURL.getParameter(PROTOCOL_KEY);
List<ServiceInstance> serviceInstances = getServiceInstances(serviceName);
List<URL> urls = dubboMetadataUtils.getDubboMetadataServiceURLs(serviceInstances,
serviceInterface, version, protocol);
notifyAllSubscribedURLs(subscribedURL, urls, listener);
}
// private void subscribeDubboMetadataServiceURLs(URL subscribedURL,
// NotifyListener listener, Set<String> serviceNames) {
//
// String serviceInterface = subscribedURL.getServiceInterface();
// String version = subscribedURL.getParameter(VERSION_KEY);
// String protocol = subscribedURL.getParameter(PROTOCOL_KEY);
//
// List<ServiceInstance> serviceInstances = getServiceInstances(serviceNames);
//
// List<URL> urls = dubboMetadataUtils.getDubboMetadataServiceURLs(serviceInstances,
// serviceInterface, version, protocol);
//
// notifyAllSubscribedURLs(subscribedURL, urls, listener);
// }
private boolean containsProviderCategory(URL subscribedURL) {
String category = subscribedURL.getParameter(CATEGORY_KEY);
return category == null ? false : category.contains(PROVIDER);
return category != null && category.contains(PROVIDER);
}
@Override
@ -550,4 +502,32 @@ public class DubboCloudRegistry extends FailbackRegistry {
return DUBBO_METADATA_SERVICE_CLASS_NAME.equals(url.getServiceInterface());
}
protected boolean isConsumerServiceURL(URL url) {
return CONSUMER.equals(url.getProtocol());
}
public List<ServiceInstance> getServiceInstances(Map<String, Set<String>> providers) {
List<ServiceInstance> instances = new ArrayList<>();
providers.forEach((appName, revisions) -> {
Map<String, List<ServiceInstance>> revisionMap = serviceRevisionInstanceMap
.get(appName);
if (revisionMap == null) {
return;
}
for (String revision : revisions) {
List<ServiceInstance> list = revisionMap.get(revision);
if (list != null) {
instances.addAll(list);
}
}
});
return instances;
}
public Map<String, Map<String, List<ServiceInstance>>> getServiceRevisionInstanceMap() {
return serviceRevisionInstanceMap;
}
}

@ -0,0 +1,280 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.cloud.dubbo.registry;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import com.alibaba.cloud.dubbo.metadata.RevisionResolver;
import com.alibaba.cloud.dubbo.metadata.repository.DubboServiceMetadataRepository;
import com.alibaba.cloud.dubbo.service.DubboMetadataService;
import com.alibaba.cloud.dubbo.service.DubboMetadataServiceProxy;
import com.alibaba.cloud.dubbo.util.JSONUtils;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.common.URLBuilder;
import org.apache.dubbo.registry.NotifyListener;
import org.apache.dubbo.rpc.RpcContext;
import org.springframework.cloud.client.ServiceInstance;
import static com.alibaba.cloud.dubbo.metadata.RevisionResolver.SCA_REVSION_KEY;
import static java.util.Collections.emptyList;
import static org.apache.dubbo.common.URLBuilder.from;
import static org.apache.dubbo.common.constants.CommonConstants.GROUP_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.PID_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.PROTOCOL_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.TIMESTAMP_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY;
/**
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
*/
public class GenearalServiceSubscribeHandler extends AbstractServiceSubscribeHandler {
/**
* the provider which can provide service of the url. {appName, [revisions]}
*/
private final Map<String, Set<String>> providers = new HashMap<>();
private final Map<String, URL> urlTemplateMap = new HashMap<>();
private final JSONUtils jsonUtils;
private final DubboServiceMetadataRepository repository;
private final DubboMetadataServiceProxy dubboMetadataConfigServiceProxy;
public GenearalServiceSubscribeHandler(URL url, NotifyListener listener,
DubboCloudRegistry registry, DubboServiceMetadataRepository repository,
JSONUtils jsonUtils,
DubboMetadataServiceProxy dubboMetadataConfigServiceProxy) {
super(url, listener, registry);
this.repository = repository;
this.jsonUtils = jsonUtils;
this.dubboMetadataConfigServiceProxy = dubboMetadataConfigServiceProxy;
}
public boolean relatedWith(String appName, String revision) {
Set<String> list = providers.get(appName);
if (list != null && list.size() > 0) {
if (list.contains(revision)) {
return true;
}
}
return false;
}
public void removeAppNameWithRevision(String appName, String revision) {
Set<String> list = providers.get(appName);
if (list != null) {
list.remove(revision);
if (list.size() == 0) {
providers.remove(appName);
}
}
}
public void addAppNameWithRevision(String appName, String revision) {
Set<String> set = providers.computeIfAbsent(appName, k -> new HashSet<>());
set.add(revision);
}
public synchronized void doInit() {
logger.debug("Subscription interface {}, GenearalServiceSubscribeHandler init",
url.getServiceKey());
Map<String, Map<String, List<ServiceInstance>>> map = registry
.getServiceRevisionInstanceMap();
for (Map.Entry<String, Map<String, List<ServiceInstance>>> entry : map
.entrySet()) {
String appName = entry.getKey();
Map<String, List<ServiceInstance>> revisionMap = entry.getValue();
for (Map.Entry<String, List<ServiceInstance>> revisionEntity : revisionMap
.entrySet()) {
String revision = revisionEntity.getKey();
List<ServiceInstance> instances = revisionEntity.getValue();
init(appName, revision, instances);
}
}
refresh();
}
public void init(String appName, String revision,
List<ServiceInstance> instanceList) {
List<URL> urls = getTemplateExportedURLs(url, revision, instanceList);
if (urls != null && urls.size() > 0) {
addAppNameWithRevision(appName, revision);
setUrlTemplate(appName, revision, urls);
}
}
public synchronized void refresh() {
List<URL> urls = getProviderURLs();
notifyAllSubscribedURLs(url, urls, listener);
}
private List<URL> getProviderURLs() {
List<ServiceInstance> instances = registry.getServiceInstances(providers);
logger.debug("Subscription interfece {}, providers {}, total {}",
url.getServiceKey(), providers, instances.size());
if (instances.size() == 0) {
return Collections.emptyList();
}
return cloneExportedURLs(instances);
}
void setUrlTemplate(String appName, String revision, List<URL> urls) {
if (urls == null || urls.size() == 0) {
return;
}
String key = getAppRevisionKey(appName, revision);
if (urlTemplateMap.containsKey(key)) {
return;
}
urlTemplateMap.put(key, urls.get(0));
}
private String getAppRevisionKey(String appName, String revision) {
return appName + "@" + revision;
}
/**
* Clone the subscribed URLs based on the template URLs.
* @param serviceInstances the list of
* {@link org.springframework.cloud.client.ServiceInstance service instances}
* @return
*/
List<URL> cloneExportedURLs(List<ServiceInstance> serviceInstances) {
List<URL> urlsCloneTo = new ArrayList<>();
serviceInstances.forEach(serviceInstance -> {
String host = serviceInstance.getHost();
String appName = serviceInstance.getServiceId();
String revision = RevisionResolver.getRevision(serviceInstance);
URL template = urlTemplateMap.get(getAppRevisionKey(appName, revision));
Stream.of(template)
.map(templateURL -> templateURL.removeParameter(TIMESTAMP_KEY))
.map(templateURL -> templateURL.removeParameter(PID_KEY))
.map(templateURL -> {
String protocol = templateURL.getProtocol();
Integer port = repository.getDubboProtocolPort(serviceInstance,
protocol);
// reserve tag
String tag = null;
List<URL> urls = jsonUtils.toURLs(serviceInstance.getMetadata()
.get("dubbo.metadata-service.urls"));
if (urls != null && urls.size() > 0) {
Map<String, String> parameters = urls.get(0).getParameters();
tag = parameters.get("dubbo.tag");
}
if (Objects.equals(templateURL.getHost(), host)
&& Objects.equals(templateURL.getPort(), port)) { // use
// templateURL
// if
// equals
return templateURL;
}
if (port == null) {
if (logger.isWarnEnabled()) {
logger.warn(
"The protocol[{}] port of Dubbo service instance[host : {}] "
+ "can't be resolved",
protocol, host);
}
return null;
}
else {
URLBuilder clonedURLBuilder = from(templateURL) // remove the
// parameters from
// the template
// URL
.setHost(host) // reset the host
.setPort(port) // reset the port
.addParameter("dubbo.tag", tag); // reset the tag
return clonedURLBuilder.build();
}
}).filter(Objects::nonNull).forEach(urlsCloneTo::add);
});
return urlsCloneTo;
}
private List<URL> getTemplateExportedURLs(URL subscribedURL, String revision,
List<ServiceInstance> serviceInstances) {
DubboMetadataService dubboMetadataService = getProxy(serviceInstances);
List<URL> templateExportedURLs = emptyList();
if (dubboMetadataService != null) {
templateExportedURLs = getExportedURLs(dubboMetadataService, revision,
subscribedURL);
}
else {
if (logger.isWarnEnabled()) {
logger.warn(
"The metadata of Dubbo service[key : {}] still can't be found, it could effect the further "
+ "Dubbo service invocation",
subscribedURL.getServiceKey());
}
}
return templateExportedURLs;
}
private DubboMetadataService getProxy(List<ServiceInstance> serviceInstances) {
return dubboMetadataConfigServiceProxy.getProxy(serviceInstances);
}
private List<URL> getExportedURLs(DubboMetadataService dubboMetadataService,
String revision, URL subscribedURL) {
String serviceInterface = subscribedURL.getServiceInterface();
String group = subscribedURL.getParameter(GROUP_KEY);
String version = subscribedURL.getParameter(VERSION_KEY);
RpcContext.getContext().setAttachment(SCA_REVSION_KEY, revision);
String exportedURLsJSON = dubboMetadataService.getExportedURLs(serviceInterface,
group, version);
// The subscribed protocol may be null
String subscribedProtocol = subscribedURL.getParameter(PROTOCOL_KEY);
return jsonUtils.toURLs(exportedURLsJSON).stream()
.filter(exportedURL -> subscribedProtocol == null
|| subscribedProtocol.equalsIgnoreCase(exportedURL.getProtocol()))
.collect(Collectors.toList());
}
}

@ -0,0 +1,76 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.cloud.dubbo.registry;
import java.util.List;
import com.alibaba.cloud.dubbo.util.DubboMetadataUtils;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.registry.NotifyListener;
import org.springframework.cloud.client.ServiceInstance;
import static org.apache.dubbo.common.constants.CommonConstants.PROTOCOL_KEY;
import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY;
/**
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
*/
public class MetadataServiceSubscribeHandler extends AbstractServiceSubscribeHandler {
private final String appName;
private final DubboMetadataUtils dubboMetadataUtils;
public MetadataServiceSubscribeHandler(String appName, URL url,
NotifyListener listener, DubboCloudRegistry registry,
DubboMetadataUtils dubboMetadataUtils) {
super(url, listener, registry);
this.appName = appName;
this.dubboMetadataUtils = dubboMetadataUtils;
}
@Override
public void doInit() {
logger.debug("Subscription app {} MetadataService handler init", appName);
List<ServiceInstance> serviceInstances = registry.getServiceInstances(appName);
subscribeDubboMetadataServiceURLs(url, listener, serviceInstances);
}
public void refresh(List<ServiceInstance> serviceInstances) {
logger.debug("Subscription app {}, instance changed, new size = {}", appName,
serviceInstances.size());
subscribeDubboMetadataServiceURLs(url, listener, serviceInstances);
}
private void subscribeDubboMetadataServiceURLs(URL subscribedURL,
NotifyListener listener, List<ServiceInstance> serviceInstances) {
logger.debug("Subscription app {}, service instance changed to size {}", appName,
serviceInstances.size());
String serviceInterface = subscribedURL.getServiceInterface();
String version = subscribedURL.getParameter(VERSION_KEY);
String protocol = subscribedURL.getParameter(PROTOCOL_KEY);
List<URL> urls = dubboMetadataUtils.getDubboMetadataServiceURLs(serviceInstances,
serviceInterface, version, protocol);
notifyAllSubscribedURLs(subscribedURL, urls, listener);
}
}

@ -0,0 +1,125 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.cloud.dubbo.registry;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import com.alibaba.cloud.dubbo.env.DubboCloudProperties;
import com.alibaba.cloud.dubbo.registry.event.ServiceInstancesChangedEvent;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cloud.client.ServiceInstance;
/**
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
*/
public class ReSubscribeManager {
private final Logger logger = LoggerFactory.getLogger(ReSubscribeManager.class);
private final Map<String, ReSubscribeMetadataJob> reConnectJobMap = new ConcurrentHashMap<>();
private final ScheduledThreadPoolExecutor reConnectPool = new ScheduledThreadPoolExecutor(
5);
private final DubboCloudRegistry registry;
private final DubboCloudProperties properties;
public ReSubscribeManager(DubboCloudRegistry registry) {
this.registry = registry;
this.properties = registry.getBean(DubboCloudProperties.class);
reConnectPool.setKeepAliveTime(10, TimeUnit.MINUTES);
reConnectPool.allowCoreThreadTimeOut(true);
}
public void onRefreshSuccess(ServiceInstancesChangedEvent event) {
reConnectJobMap.remove(event.getServiceName());
}
public void onRefreshFail(ServiceInstancesChangedEvent event) {
String serviceName = event.getServiceName();
int count = 1;
if (event instanceof FakeServiceInstancesChangedEvent) {
count = ((FakeServiceInstancesChangedEvent) event).getCount() + 1;
}
if (count >= properties.getMaxReSubscribeMetadataTimes()) {
logger.error(
"reSubscribe failed too many times, serviceName = {}, count = {}",
serviceName, count);
return;
}
ReSubscribeMetadataJob job = new ReSubscribeMetadataJob(serviceName, count);
reConnectPool.schedule(job, properties.getReSubscribeMetadataIntervial(),
TimeUnit.SECONDS);
}
private final class ReSubscribeMetadataJob implements Runnable {
private final String serviceName;
private final int errorCounts;
private ReSubscribeMetadataJob(String serviceName, int errorCounts) {
this.errorCounts = errorCounts;
this.serviceName = serviceName;
}
@Override
public void run() {
if (!reConnectJobMap.containsKey(serviceName)
|| reConnectJobMap.get(serviceName) != this) {
return;
}
List<ServiceInstance> list = registry.getServiceInstances(serviceName);
FakeServiceInstancesChangedEvent event = new FakeServiceInstancesChangedEvent(
serviceName, list, errorCounts);
registry.onApplicationEvent(event);
}
}
private static final class FakeServiceInstancesChangedEvent
extends ServiceInstancesChangedEvent {
private static final long serialVersionUID = -2832478604601472915L;
private final int count;
private FakeServiceInstancesChangedEvent(String serviceName,
List<ServiceInstance> serviceInstances, int count) {
super(serviceName, serviceInstances);
this.count = count;
}
public int getCount() {
return count;
}
}
}

@ -0,0 +1,35 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.cloud.dubbo.registry;
import com.alibaba.cloud.dubbo.registry.event.ServiceInstancesChangedEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.core.Ordered;
/**
* The interface of ServiceInstanceChange event Listener.
*
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
* @see ServiceInstancesChangedEvent
* @see Ordered
* @see ApplicationListener
*/
public interface ServiceInstanceChangeListener
extends ApplicationListener<ServiceInstancesChangedEvent>, Ordered {
}

@ -89,7 +89,7 @@ public class SpringCloudRegistryFactory extends AbstractRegistryFactory {
DubboCloudProperties dubboCloudProperties = applicationContext
.getBean(DubboCloudProperties.class);
Registry registry = null;
Registry registry;
switch (dubboCloudProperties.getRegistryType()) {
case SPRING_CLOUD_REGISTRY_PROPERTY_VALUE:
@ -100,7 +100,7 @@ public class SpringCloudRegistryFactory extends AbstractRegistryFactory {
default:
registry = new DubboCloudRegistry(url, discoveryClient,
dubboServiceMetadataRepository, dubboMetadataConfigServiceProxy,
jsonUtils, dubboGenericServiceFactory, applicationContext);
jsonUtils, applicationContext);
break;
}

@ -78,6 +78,9 @@ public class DubboGenericServiceFactory {
String interfaceName = serviceClass.getName();
ReferenceBean<GenericService> referenceBean = build(interfaceName, version,
serviceName, emptyMap());
if (DubboMetadataService.class == serviceClass) {
referenceBean.setRouter("-default,revisionRouter");
}
return referenceBean.get();
}

@ -125,6 +125,7 @@ public class DubboMetadataServiceProxy implements BeanClassLoaderAware, Disposab
private DubboMetadataService createProxyIfAbsent(URL dubboMetadataServiceURL) {
String serviceName = dubboMetadataServiceURL.getParameter(APPLICATION_KEY);
String version = dubboMetadataServiceURL.getParameter(VERSION_KEY);
// Initialize DubboMetadataService with right version
return createProxyIfAbsent(serviceName, version);
}

@ -0,0 +1,75 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.cloud.dubbo.service;
import java.util.ArrayList;
import java.util.List;
import com.alibaba.cloud.commons.lang.StringUtils;
import org.apache.dubbo.common.URL;
import org.apache.dubbo.rpc.Invocation;
import org.apache.dubbo.rpc.Invoker;
import org.apache.dubbo.rpc.RpcException;
import org.apache.dubbo.rpc.cluster.Router;
import org.apache.dubbo.rpc.cluster.RouterFactory;
import org.apache.dubbo.rpc.cluster.router.AbstractRouter;
import org.springframework.util.CollectionUtils;
import static com.alibaba.cloud.dubbo.metadata.RevisionResolver.SCA_REVSION_KEY;
/**
* @author <a href="mailto:chenxilzx1@gmail.com">theonefx</a>
*/
public class MetadataServiceRevisionRouterFactory implements RouterFactory {
@Override
public Router getRouter(URL url) {
return new AbstractRouter() {
@Override
public <T> List<Invoker<T>> route(List<Invoker<T>> invokers, URL url,
Invocation invocation) throws RpcException {
if (CollectionUtils.isEmpty(invokers)) {
return invokers;
}
if (!DubboMetadataService.class.getName()
.equalsIgnoreCase(url.getServiceInterface())) {
return invokers;
}
String revision = invocation.getAttachment(SCA_REVSION_KEY);
if (StringUtils.isEmpty(revision)) {
return invokers;
}
List<Invoker<T>> list = new ArrayList<>(invokers.size());
for (Invoker<T> invoker : invokers) {
if (StringUtils.equals(revision,
invoker.getUrl().getParameter(SCA_REVSION_KEY))) {
list.add(invoker);
}
}
return list;
}
};
}
}

@ -21,12 +21,14 @@ import java.util.Map;
import java.util.Objects;
import java.util.stream.Collectors;
import com.alibaba.cloud.dubbo.metadata.RevisionResolver;
import com.alibaba.cloud.dubbo.service.DubboMetadataService;
import org.apache.dubbo.common.URL;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.core.env.Environment;
import static com.alibaba.cloud.dubbo.metadata.RevisionResolver.SCA_REVSION_KEY;
import static java.lang.String.format;
import static org.apache.dubbo.common.constants.CommonConstants.VERSION_KEY;
import static org.apache.dubbo.registry.client.metadata.ServiceInstanceMetadataUtils.METADATA_SERVICE_URLS_PROPERTY_NAME;
@ -74,7 +76,11 @@ public class DubboMetadataUtils {
public List<URL> getDubboMetadataServiceURLs(ServiceInstance serviceInstance) {
Map<String, String> metadata = serviceInstance.getMetadata();
String dubboURLsJSON = metadata.get(METADATA_SERVICE_URLS_PROPERTY_NAME);
return jsonUtils.toURLs(dubboURLsJSON);
List<URL> urls = jsonUtils.toURLs(dubboURLsJSON);
String revision = RevisionResolver.getRevision(serviceInstance);
urls = urls.stream().map(url -> url.addParameter(SCA_REVSION_KEY, revision))
.collect(Collectors.toList());
return urls;
}
/**

@ -14,7 +14,7 @@
* limitations under the License.
*/
package com.alibaba.cloud.stream.binder.rocketmq.contants;
package com.alibaba.cloud.stream.binder.rocketmq.constant;
import org.apache.rocketmq.common.message.MessageConst;

@ -18,7 +18,7 @@ package com.alibaba.cloud.stream.binder.rocketmq.integration.outbound;
import java.util.List;
import com.alibaba.cloud.stream.binder.rocketmq.contants.RocketMQConst;
import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst;
import com.alibaba.cloud.stream.binder.rocketmq.custom.RocketMQBeanContainerCache;
import com.alibaba.cloud.stream.binder.rocketmq.metrics.Instrumentation;
import com.alibaba.cloud.stream.binder.rocketmq.metrics.InstrumentationManager;

@ -20,8 +20,8 @@ import java.nio.charset.Charset;
import java.util.Map;
import java.util.Objects;
import com.alibaba.cloud.stream.binder.rocketmq.contants.RocketMQConst;
import com.alibaba.cloud.stream.binder.rocketmq.contants.RocketMQConst.Headers;
import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst;
import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst.Headers;
import com.alibaba.cloud.stream.binder.rocketmq.convert.RocketMQMessageConverter;
import com.alibaba.cloud.stream.binder.rocketmq.custom.RocketMQBeanContainerCache;
import org.apache.rocketmq.common.message.MessageConst;

@ -16,7 +16,7 @@
package com.alibaba.cloud.stream.binder.rocketmq.utils;
import com.alibaba.cloud.stream.binder.rocketmq.contants.RocketMQConst;
import com.alibaba.cloud.stream.binder.rocketmq.constant.RocketMQConst;
import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQBinderConfigurationProperties;
import com.alibaba.cloud.stream.binder.rocketmq.properties.RocketMQCommonProperties;
import org.apache.rocketmq.acl.common.AclClientRPCHook;

Loading…
Cancel
Save