C# Elasticsearch.NET7&；Nest 7-无效的批量更新请求_C#_<img Src="//i.stack.imgur.com/RUiNP.png" Height="16" Width="18" Alt="" Class="sponsor Tag Img">elasticsearch_Nest

C# Elasticsearch.NET7&；Nest 7-无效的批量更新请求

c# nest

C# Elasticsearch.NET7&；Nest 7-无效的批量更新请求,c#,elasticsearch,nest,C#,elasticsearch,Nest,我正在尝试创建一个.Net Core（netcoreapp3.1）应用程序，将我的数据从Azure Eventhub发送到ES 7 我正在使用以下软件包： ElasticSearch.Net 7.8.1 Nest 7.8.1 我从Eventhub检索的数据是从IElasticBaseEntity继承的两种类型。我想批量更新这些对象的列表，可以是包含所有信息的对象，也可以是更新已编制索引的人员的一个字段的对象。 ES中要匹配/搜索的字段是id字段为了简化我的示例，我将使用以下虚拟类： publ

我正在尝试创建一个.Net Core（netcoreapp3.1）应用程序，将我的数据从Azure Eventhub发送到ES 7

我正在使用以下软件包：
ElasticSearch.Net 7.8.1
Nest 7.8.1

我从Eventhub检索的数据是从IElasticBaseEntity继承的两种类型。
我想批量更新这些对象的列表，可以是包含所有信息的对象，也可以是更新已编制索引的人员的一个字段的对象。
ES中要匹配/搜索的字段是id字段

为了简化我的示例，我将使用以下虚拟类：

public interface IElasticBaseEntity
{
   string Id { get; set; }
   DateTime ServerTimestamp { get; set; }
}

person类是包含所有信息的类

public abstract class Person: IElasticBaseEntity
{
   public string Id { get; set; }
   public string Firstname {get; set;}
   public string Name {get; set;}   
   public decimal? Score { get; set; }
}

Score类是我想要基于Id对索引的人进行的更新

public abstract class Score : IElasticBaseEntity
{
   public string Id {get; set;}
   public decimal? Score { get; set; }
}

我使用此方法来建立与ES的连接

public static IElasticClient CreateInstance(ElasticsearchConfig config)
{
   IConnectionPool pool;    
   var connection = new HttpConnection();

   pool = new SniffingConnectionPool(new[] { new Uri(config.Url) }) { SniffedOnStartup = true };

   var connectionSettings = new ConnectionSettings(pool, connection);
   connectionSettings.BasicAuthentication(config.Username, config.Password);
   connectionSettings.ServerCertificateValidationCallback(ServerCertificateValidationCallback);
   connectionSettings.RequestTimeout(TimeSpan.FromMinutes(2)).EnableHttpCompression();
   var client = new ElasticClient(connectionSettings);
    
   return client;
}

所以我从ElasticClient上的批量命令开始。在过去，我可以使用descriptor.Index添加对象，但当然，我需要的是更新，而不是插入/创建所有内容

所以我想到了这一点，但出于某种原因，我一直在Visual Studio 2019中收到一个错误，即“无效/_批量请求”，而没有任何其他信息

IEnumerable<IElasticBaseEntity> list = RetrievedData();

var descriptor = new BulkDescriptor();
            
foreach (var eachDoc in list)
{
   var doc = eachDoc;
   descriptor.Update<IElasticBaseEntity>(i => i
      .Id(doc.Id)
      .Doc(doc)
      .DocAsUpsert(true));   
}

var response = await _client.BulkAsync(descriptor);

// Try to debug & see what was send
if (response.ApiCall.RequestBodyInBytes != null)
{
   var jsonOutput = System.Text.Encoding.UTF8.GetString(response.ApiCall.RequestBodyInBytes);
}

IEnumerable list=RetrievedData（）；
var descriptor=新的BulkDescriptor（）；
foreach（列表中的var eachDoc）
{
var doc=eachDoc；
descriptor.Update（i=>i
.Id（doc.Id）
.Doc（Doc）
.DocAsUpsert（真））；
}
var response=wait_client.BulkAsync（描述符）；
//尝试调试并查看发送的内容
if（response.ApiCall.RequestBodyInBytes！=null）
{
var jsonOutput=System.Text.Encoding.UTF8.GetString（response.ApiCall.RequestBodyInBytes）；
}

我收到的错误如下（从

响应中检索。DebugInformation

）：

从POST:/persons-20200717/\u bulk上不成功的（）低级调用生成的嵌套响应无效

无效的批量项目：此API调用的审核跟踪：

[1] PingFailure:节点：https://myconnectiontoES:9243/ 异常：PipelineException:00:00:01.2859155
[2] 嗅探失败：拍摄时间：00:00:02.1577638
[3] 嗅探失败：节点：https://myconnectiontoES:9243/ 异常：PipelineException:00:00:02.0840985

OriginalException:Elasticsearch.Net.ElasticsearchClientException:嗅探群集状态失败..>呼叫：未知资源 --->Elasticsearch.Net.PipelineException:无法嗅探群集状态。 --->Elasticsearch.Net.PipelineException:尝试从>指定节点读取响应时出错。在Elasticsearch.Net.RequestPipeline.SniffAsync（CancellationToken CancellationToken） ---内部异常堆栈跟踪的结束--- 在Elasticsearch.Net.RequestPipeline.SniffAsync（CancellationToken CancellationToken）在Elasticsearch.Net.RequestPipeline.SniffOnConnectionFailureAsync（CancellationToken>CancellationToken）在Elasticsearch.Net.Transport

1.PingAsync（IRequestPipeline管道、节点节点、CancellationToken>CancellationToken）在Elasticsearch.Net.Transport

1.RequestAsync[TreResponse]（HttpMethod方法、字符串路径、>CancellationToken CancellationToken、PostData、IRequestParameters requestParameters） ---内部异常堆栈跟踪的结束---

步骤1中的审核异常PingFailure： Elasticsearch.Net.PipelineException:尝试从指定的>节点读取响应时出错。在Elasticsearch.Net.RequestPipeline.PingAsync（节点节点，CancellationToken CancellationToken）

第3步中的审核异常失败： Elasticsearch.Net.PipelineException:尝试从指定的>节点读取响应时出错。在Elasticsearch.Net.RequestPipeline.SniffAsync（CancellationToken CancellationToken）

请求：在连接设置上强制在响应上设置它。>

答复：所以我的

/\u bulk

请求似乎有问题

我所尝试的：

用fiddler捕获
```
/\u bulk
```
请求该请求无效&未发送

尝试设置

connectionSettings.DisableDirectStreaming（true）

打印请求>此值始终为

null

因此，如果有人能指出我在构建

/\u bulk

请求时犯的错误，或者能为我指明调试的方向并检索更多信息，我将不胜感激。目前我正在转圈，重新阅读文档，谷歌搜索，但没有任何结果

感谢

尝试读取对的响应时发生故障。似乎发生了嗅探，因为ping集群失败了，所以值得检查一下，您是否可以使用客户端配置使用的凭据向Elasticsearch集群的基址发出

HEAD

请求

基于Elasticsearch连接中使用的端口9243，我怀疑Elasticsearch集群正在运行<代码>嗅探连接池不得与运行在弹性云中的Elasticsearch群集一起使用，因为嗅探响应中返回的到达Elasticsearch节点的地址是客户端无法访问的内部地址。相反，应该使用

CloudConnectionPool

，在创建

ElasticClient

var客户端=新的ElasticClient(
"", 
新的基本身份验证凭据（config.Username、config.Password））；

CloudConnectionPool

默认情况下将使用http压缩，但如果您需要对请求超时等其他配置进行更多控制，则可以使用

var-pool=新的CloudConnectionPool(
"", 
新的基本身份验证凭据（config.Username、config.Password））；
var设置=新连接设置（池）
.RequestTimeout（TimeSpan.FromMinutes（2））；
var客户端=新的ElasticClient（设置）；

当

IEnumerable<IElasticBaseEntity> list = RetrievedData();

var descriptor = new BulkDescriptor();
        
foreach (var eachDoc in list)
{
   var doc = eachDoc;
   descriptor.Update<IElasticBaseEntity>(i => i
      .Id(doc.Id)
      .Doc(doc)
      .DocAsUpsert(true));   
}

var response = await _client.BulkAsync(descriptor);

2020-08-05 11:22:01.5553| INFO| 001 ES_indexName processed documents 08/05/2020 11:22:01 1 in 202.3803 ms
2020-08-05 11:29:28.9633| INFO| 001 ES_indexName processed documents 08/05/2020 11:29:28 1 in 179.7982 ms
2020-08-05 11:40:08.4666| INFO| 001 ES_indexName processed documents 08/05/2020 11:40:07 1 in 291.5695 ms
2020-08-05 11:47:26.2924|ERROR| failed Invalid NEST response built from a unsuccessful () low level call on POST: /ES_indexName/_bulk
# Invalid Bulk items:
# Audit trail of this API call:
 - [1] PingFailure: Node: https://ES_node1:port/ Exception: PipelineException Took: 00:01:40.0193513
 - [2] SniffOnFail: Took: 00:05:00.0132241
 - [3] SniffFailure: Node: https://ES_node2:port/ Exception: PipelineException Took: 00:01:40.0036955
 - [4] SniffFailure: Node: https://ES_node3:port/ Exception: PipelineException Took: 00:01:40.0019296
 - [5] SniffFailure: Node: https://ES_node1:port/ Exception: PipelineException Took: 00:01:40.0005639
 - [6] MaxTimeoutReached:
# OriginalException: Elasticsearch.Net.ElasticsearchClientException: Maximum timeout reached while retrying request. Call: unknown resource
 ---> Elasticsearch.Net.PipelineException: Failed sniffing cluster state.
 ---> System.AggregateException: One or more errors occurred. (An error occurred trying to write the request data to the specified node.) (An error occurred trying to write the request data to the specified node.) (An error occurred trying to write the request data to the specified node.)
 ---> Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.SniffAsync(CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
 ---> (Inner Exception #1) Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.SniffAsync(CancellationToken cancellationToken)<---

 ---> (Inner Exception #2) Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.SniffAsync(CancellationToken cancellationToken)<---

   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.SniffAsync(CancellationToken cancellationToken)
   at Elasticsearch.Net.RequestPipeline.SniffOnConnectionFailureAsync(CancellationToken cancellationToken)
   at Elasticsearch.Net.Transport`1.PingAsync(IRequestPipeline pipeline, Node node, CancellationToken cancellationToken)
   at Elasticsearch.Net.Transport`1.RequestAsync[TResponse](HttpMethod method, String path, CancellationToken cancellationToken, PostData data, IRequestParameters requestParameters)
   --- End of inner exception stack trace ---
# Audit exception in step 1 PingFailure:
Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.PingAsync(Node node, CancellationToken cancellationToken)
# Audit exception in step 3 SniffFailure:
Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.SniffAsync(CancellationToken cancellationToken)
# Audit exception in step 4 SniffFailure:
Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.SniffAsync(CancellationToken cancellationToken)
# Audit exception in step 5 SniffFailure:
Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
   at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.RequestPipeline.SniffAsync(CancellationToken cancellationToken)
# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
# Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
 400039.9975 ms MUST RETRY```