Java 使用异步拉取连续接收来自Google PubSub的消息
借助于此,我成功地创建了一个小型Java应用程序,它可以在一分钟内提取已发布的消息。我的实现看起来像这样Java 使用异步拉取连续接收来自Google PubSub的消息,java,multithreading,asynchronous,google-cloud-pubsub,Java,Multithreading,Asynchronous,Google Cloud Pubsub,借助于此,我成功地创建了一个小型Java应用程序,它可以在一分钟内提取已发布的消息。我的实现看起来像这样 public static void eventListener() throws InterruptedException { MessageReceiver receiver = new MessageReceiver() { @Override public void receiveMessage(PubsubMessage message,
public static void eventListener() throws InterruptedException {
MessageReceiver receiver = new MessageReceiver() {
@Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
System.out.println("Received message: " + message.getData().toStringUtf8());
consumer.ack();
}
};
//Subscriber subscriber = null;
try {
subscriber = Subscriber.newBuilder(subscription, receiver)
.setCredentialsProvider(FixedCredentialsProvider.create(creds)).build();
subscriber.addListener(new Subscriber.Listener() {
@Override
public void failed(Subscriber.State from, Throwable failure) {
// Handle failure. This is called when the Subscriber encountered a fatal error
// and is
// shutting down.
System.err.println(failure);
}
}, MoreExecutors.directExecutor());
subscriber.startAsync().awaitRunning();
// In this example, we will pull messages for one minute (60,000ms) then stop.
// In a real application, this sleep-then-stop is not necessary.
// Simply call stopAsync().awaitTerminated() when the server is shutting down,
// etc.
Thread.sleep(60000);
} finally {
if (subscriber != null) {
subscriber.stopAsync().awaitTerminated();
}
}
}
当我在main
public static void main(String[] args) throws InterruptedException {
eventListener();
}
然后将一个对象上传到我的Google云存储中,程序会打印一条由发布者发送的消息,如下所示
Received message: {
"kind": "storage#object",
"id": "roshanbucket/stones.jpg/1553765105996166",
"selfLink": "https://www.googleapis.com/storage/v1/b/roshanbucket/o/stones.jpg",
"name": "stones.jpg",
"bucket": "roshanbucket",
"generation": "1553765105996166",
"metageneration": "1",
"contentType": "image/jpeg",
"timeCreated": "2019-03-28T09:25:05.995Z",
"updated": "2019-03-28T09:25:05.995Z",
"storageClass": "STANDARD",
"timeStorageClassUpdated": "2019-03-28T09:25:05.995Z",
"size": "137256",
"md5Hash": "1GmpUnGeiW+/KU+0U8c8Wg==",
"mediaLink": "https://www.googleapis.com/download/storage/v1/b/roshanbucket/o/stones.jpg?generation=1553765105996166&alt=media",
"crc32c": "FMaEGg==",
"etag": "CIaj1InCpOECEAE="
}
public static void main(String[] args) throws InterruptedException {
while(true) {
eventListener();
}
}
在程序执行后的一分钟内,它打印对象上载帐户上收到的所有消息,然后停止。要在一分钟后接收事件消息,我需要重新启动应用程序。现在,我要做的是连续运行侦听器,因此,我尝试在主方法的无限循环中运行方法eventListener()
,如下所示
Received message: {
"kind": "storage#object",
"id": "roshanbucket/stones.jpg/1553765105996166",
"selfLink": "https://www.googleapis.com/storage/v1/b/roshanbucket/o/stones.jpg",
"name": "stones.jpg",
"bucket": "roshanbucket",
"generation": "1553765105996166",
"metageneration": "1",
"contentType": "image/jpeg",
"timeCreated": "2019-03-28T09:25:05.995Z",
"updated": "2019-03-28T09:25:05.995Z",
"storageClass": "STANDARD",
"timeStorageClassUpdated": "2019-03-28T09:25:05.995Z",
"size": "137256",
"md5Hash": "1GmpUnGeiW+/KU+0U8c8Wg==",
"mediaLink": "https://www.googleapis.com/download/storage/v1/b/roshanbucket/o/stones.jpg?generation=1553765105996166&alt=media",
"crc32c": "FMaEGg==",
"etag": "CIaj1InCpOECEAE="
}
public static void main(String[] args) throws InterruptedException {
while(true) {
eventListener();
}
}
有了它,我似乎能够在每次上传之后立即接收事件消息,而不管何时上传对象。但是,每隔一段时间,它就会抛出这个堆栈跟踪
Mar 28, 2019 12:56:34 PM io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=6, target=pubsub.googleapis.com:443} was not shutdown properly!!! ~*~*~*
Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
java.lang.RuntimeException: ManagedChannel allocation site
at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init>(ManagedChannelOrphanWrapper.java:103)
at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:53)
at io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:44)
at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:440)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:223)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:164)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:156)
at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:157)
at com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:260)
at com.google.cloud.pubsub.v1.Subscriber.doStart(Subscriber.java:268)
at com.google.api.core.AbstractApiService$InnerService.doStart(AbstractApiService.java:148)
at com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:225)
at com.google.api.core.AbstractApiService.startAsync(AbstractApiService.java:120)
at com.google.cloud.pubsub.v1.Subscriber.startAsync(Subscriber.java:260)
at listener.AsynchronousPull.eventListener(AsynchronousPull.java:57)
at listener.AsynchronousPull.main(AsynchronousPull.java:74)
2019年3月28日12:56:34 PM io.grpc.internal.managedChannel孤儿包装器$managedChannel引用清理队列
严重:~*~*~Channel ManagedChannel Impl{logId=6,target=pubsub.googleapis.com:443}未正确关闭~*~*~*
确保调用shutdown()/shutdownNow()并等待waittermination()返回true。
java.lang.RuntimeException:ManagedChannel分配站点
位于io.grpc.internal.managedChannel孤儿包装器$managedChannel参考。(managedChannel孤儿包装器.java:103)
位于io.grpc.internal.managedChannel孤儿包装器。(managedChannel孤儿包装器.java:53)
在io.grpc.internal.managedChannel孤儿包装器上。(managedChannel孤儿包装器.java:44)
位于io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:440)
位于com.google.api.gax.grpc.instantinggrpcchannelprovider.createSingleChannel(instantinggrpcchannelprovider.java:223)
位于com.google.api.gax.grpc.instantinggrpcchannelprovider.createChannel(instantinggrpcchannelprovider.java:164)
位于com.google.api.gax.grpc.instantinggrpcchannelprovider.getTransportChannel(instantinggrpcchannelprovider.java:156)
在com.google.api.gax.rpc.ClientContext.create(ClientContext.java:157)上
在com.google.cloud.pubsub.v1.stub.GrpcSubscriberStub.create(GrpcSubscriberStub.java:260)上
位于com.google.cloud.pubsub.v1.Subscriber.doStart(Subscriber.java:268)
位于com.google.api.core.AbstractApiService$InnerService.doStart(AbstractApiService.java:148)
位于com.google.common.util.concurrent.AbstractService.startAsync(AbstractService.java:225)
位于com.google.api.core.AbstractApiService.startAsync(AbstractApiService.java:120)
位于com.google.cloud.pubsub.v1.Subscriber.startAsync(Subscriber.java:260)
在listener.AsynchronousPull.eventListener上(AsynchronousPull.java:57)
位于listener.AsynchronousPull.main(AsynchronousPull.java:74)
但是,它仍然在每次上传后打印消息,同时不时抛出堆栈跟踪。我对
线程没有太多经验,我非常感谢您能帮我解决这个问题。在紧循环中调用eventListener()
。这将创建许多新的订阅者实例,这些订阅者接收消息,每个消息持续60秒。您希望创建的订阅服务器的单个实例在您希望关闭它之前一直处于活动状态。通常,您可以通过创建订阅服务器并通过awaiterminated()
等待其终止来完成此操作
上面的代码将更改为如下所示:
public static void eventListener() throws InterruptedException {
MessageReceiver receiver = new MessageReceiver() {
@Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
System.out.println("Received message: " + message.getData().toStringUtf8());
consumer.ack();
}
};
Subscriber subscriber = null;
try {
subscriber = Subscriber.newBuilder(subscription, receiver)
.setCredentialsProvider(FixedCredentialsProvider.create(creds)).build();
subscriber.addListener(new Subscriber.Listener() {
@Override
public void failed(Subscriber.State from, Throwable failure) {
// Handle failure. This is called when the Subscriber encountered a fatal error
// and is
// shutting down.
System.err.println(failure);
}
}, MoreExecutors.directExecutor());
subscriber.startAsync().awaitRunning();
subscriber.awaitTerminated();
} finally {
if (subscriber != null) {
subscriber.stopAsync().awaitTerminated();
}
}
}
public static void main(String[] args) throws InterruptedException {
eventListener();
}
如果您只希望订阅服务器在应用程序终止时停止,而不想进行任何额外的清理,那么上面的代码将起作用,允许订阅服务器运行并接收消息,直到出现错误或应用程序关闭。如果您希望在应用程序的干净终止时执行一些清理,例如,您希望确保receiveMessage
已处理的任何消息运行到完成,那么您可以捕获此类终止(尽管它不会在所有情况下运行)。在这个钩子中,您将调用stopAsync()
。例如,可以在try
块之前插入以下内容:
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
subscriber.stopAsync().awaitTerminated();
}
});
非常感谢。如何在node中实现这一点?我试着在这里问: