如何调试/排除Azure Cosmos DB(表API)上的元数据DTU限制?

如何调试/排除Azure Cosmos DB(表API)上的元数据DTU限制?,azure,azure-cosmosdb,throttling,http-status-code-429,azure-cosmosdb-tables,Azure,Azure Cosmosdb,Throttling,Http Status Code 429,Azure Cosmosdb Tables,我们正在使用Azure cosmos DB保存作业处理管道的状态信息。为此,我们使用表API和相应的SDK。最近,我们注意到系统经常运行到429–请求速率太大错误。我们的事务性DTU利用率远低于表中配置的最大值,但我们注意到在metrics选项卡下,系统DTU被枚举表等操作使用。。已经精疲力竭了,所以429 我们最初修复了删除“CreateIfNotExists”方法调用的问题,帮助修复了一段时间,但最近我们又开始遇到这个问题(尽管不像以前那样频繁)。调试/故障排除很困难,因为在那里我找不到足够

我们正在使用Azure cosmos DB保存作业处理管道的状态信息。为此,我们使用表API和相应的SDK。最近,我们注意到系统经常运行到429–请求速率太大错误。我们的事务性DTU利用率远低于表中配置的最大值,但我们注意到在metrics选项卡下,系统DTU被枚举表等操作使用。。已经精疲力竭了,所以429

我们最初修复了删除“CreateIfNotExists”方法调用的问题,帮助修复了一段时间,但最近我们又开始遇到这个问题(尽管不像以前那样频繁)。调试/故障排除很困难,因为在那里我找不到足够的文档说明哪些SDK方法调用耗尽了这个不可伸缩的资源。我已经在我们的CosmosDB实例上启用了日志记录,但我不确定我在日志中寻找什么来解决这个问题

下面是我们用于与Azure Cosmos DB接口的singleton类

public class CosmosDbTableFacade : ICosmosDbTableFacade
{
        /// <summary>
        /// Initializes a new instance of the <see cref="CosmosDbTableFacade"/> class.
        /// </summary>
        /// <param name="connectionString">
        /// The connection string.
        /// </param>
        /// <param name="tableName">
        /// The table name.
        /// </param>
        public CosmosDbTableFacade(string connectionString)
        {
            var storageAccount = CloudStorageAccount.Parse(connectionString);
            this.CosmosTableClient = storageAccount.CreateCloudTableClient();
        }

        /// <summary>
        /// Gets or sets the cosmos table.
        /// </summary>
        public CloudTableClient CosmosTableClient { get; set; }

        /// <summary>
        /// The execute async.
        /// </summary>
        /// <param name="tableName">
        /// The table Name.
        /// </param>
        /// <param name="operation">
        /// The operation.
        /// </param>
        /// <returns>
        /// The <see cref="Task"/>.
        /// </returns>
        public Task<TableResult> ExecuteAsync(string tableName, TableOperation operation)
        {
            var table = this.CosmosTableClient.GetTableReference(tableName);
            return table.ExecuteAsync(operation);
        }

        /// <summary>
        /// The execute query segmented async.
        /// </summary>
        /// <param name="tableName">
        /// The table name.
        /// </param>
        /// <param name="query">
        /// The query.
        /// </param>
        /// <param name="continuationToken">
        /// The continuation token.
        /// </param>
        /// <returns>
        /// The <see cref="Task"/> which returns the list of entities.
        /// </returns>
        public Task<TableQuerySegment<DynamicTableEntity>> ExecuteQuerySegmentedAsync(string tableName, TableQuery query, TableContinuationToken continuationToken)
        {
            var table = this.CosmosTableClient.GetTableReference(tableName);
            return table.ExecuteQuerySegmentedAsync(query, continuationToken);
        }
}
公共类CosmosDbTableFacade:IComosDBTableFacade
{
/// 
///初始化类的新实例。
/// 
/// 
///连接字符串。
/// 
/// 
///表名。
/// 
公共CosmosDbTableFacade(字符串连接字符串)
{
var storageAccount=CloudStorageAccount.Parse(connectionString);
this.CosmosTableClient=storageAccount.CreateCloudTableClient();
}
/// 
///获取或设置cosmos表。
/// 
public CloudTableClient CosmosTableClient{get;set;}
/// 
///执行异步。
/// 
/// 
///表名。
/// 
/// 
///手术。
/// 
/// 
///这个。
/// 
公共任务ExecuteAsync(字符串表名、表操作)
{
var table=this.CosmosTableClient.GetTableReference(tableName);
返回表.ExecuteAsync(操作);
}
/// 
///执行查询是分段异步的。
/// 
/// 
///表名。
/// 
/// 
///查询。
/// 
/// 
///继续标记。
/// 
/// 
///返回实体列表的。
/// 
公共任务ExecuteQuerySegmentedAsync(字符串tableName、TableQuery查询、TableContinuationToken continuationToken)
{
var table=this.CosmosTableClient.GetTableReference(tableName);
返回表.ExecuteQuerySegmentedAsync(查询,continuationToken);
}
}
下面的代码段列出了我们正在使用的不同查询-

public async Task InsertOrMergeEntityAsync<T>(string tableName, T entity)
            where T : TableEntity
{
            var insertOrMergeOperation = TableOperation.InsertOrMerge(entity);
            var result = await this.CosmosDbTableFacade.ExecuteAsync(tableName, insertOrMergeOperation).ConfigureAwait(false);
            ValidateCosmosTableResult(result, "Failed to write to Cosmos Table");
}

public async Task<T> GetEntityAsync<T>(string tableName, string partitionKey, string rowKey)
            where T : TableEntity
{
            var retrieveOperation = TableOperation.Retrieve<T>(partitionKey, rowKey);
            TableResult result = await this.CosmosDbTableFacade.ExecuteAsync(tableName, retrieveOperation).ConfigureAwait(false);
            ValidateCosmosTableResult(result, "Failed to read from Cosmos Table");
            return result.Result as T;
}

public async Task<IEnumerable<T>> GetEntitiesAsync<T>(string tableName, string filterCondition)
            where T : TableEntity
{
            var query = new TableQuery().Where(filterCondition);
            var continuationToken = default(TableContinuationToken);
            var results = new List<T>();
            do
            {
                var currentQueryResults = await this.CosmosDbTableFacade.ExecuteQuerySegmentedAsync(tableName, query, continuationToken).ConfigureAwait(false);
                results.AddRange(currentQueryResults.Select(currentQueryResult =>
                    {
                        var currentEntity = TableEntity.ConvertBack<T>(currentQueryResult.Properties, null);
                        currentEntity.RowKey = currentQueryResult.RowKey;
                        currentEntity.PartitionKey = currentQueryResult.PartitionKey;
                        currentEntity.Timestamp = currentQueryResult.Timestamp;
                        currentEntity.ETag = currentQueryResult.ETag;
                        return currentEntity;
                    }));
                continuationToken = currentQueryResults.ContinuationToken;
            }
            while (continuationToken != null);

            return results;
}
public async Task insertormergentityasync(string tableName,T entity)
其中T:TableEntity
{
var insertOrMergeOperation=TableOperation.InsertOrMerge(实体);
var result=wait this.CosmosDbTableFacade.ExecuteAsync(表名,insertOrMergeOperation).configurewait(false);
ValidateCosmoStablerResult(结果,“未能写入Cosmos表”);
}
公共异步任务GetEntityAsync(字符串表名、字符串分区键、字符串行键)
其中T:TableEntity
{
var retrieveOperation=TableOperation.Retrieve(partitionKey,rowKey);
TableResult result=Wait this.CosmosDbTableFacade.ExecuteAsync(tableName,retrieveOperation).ConfigureWait(false);
ValidateCosmoStablerResult(结果,“未能从Cosmos表中读取”);
返回结果,结果为T;
}
公共异步任务GetEntityAsync(字符串表名、字符串筛选器条件)
其中T:TableEntity
{
var query=new TableQuery()。其中(filterCondition);
var continuationToken=默认值(TableContinuationToken);
var results=新列表();
做
{
var currentQueryResults=wait this.CosmosDbTableFacade.ExecuteQuerySegmentedAsync(表名、查询、continuationToken)。ConfigureWait(false);
results.AddRange(currentQueryResults.Select)(currentQueryResult=>
{
var currentEntity=TableEntity.ConvertBack(currentQueryResult.Properties,null);
currentEntity.RowKey=currentQueryResult.RowKey;
currentEntity.PartitionKey=currentQueryResult.PartitionKey;
currentEntity.Timestamp=currentQueryResult.Timestamp;
currentEntity.ETag=currentQueryResult.ETag;
返回当前实体;
}));
continuationToken=currentQueryResults.continuationToken;
}
while(continuationToken!=null);
返回结果;
}

下面最后一个方法中的筛选器包含一个分区键和一个自定义列

对于遇到类似问题的任何人,元数据DTU节流的根本原因是:GetTableReference(tableName)方法(通过部署更改并将该行移到启动代码并监视DTU利用率来找到)。
我这样做是为了能够在运行时动态地指向要读/写的表,但由于这消耗了元数据DTU,我将代码改为使用单例作为表引用。

您可以共享您正在执行的查询吗?查询是否包括
分区键
?添加了有关查询模式的信息。是的,我们扩展了在我们几乎所有的查询中都会使用分区键