C# 由于在大型列表中的位置而导致EF超时异常。Contains()
当我向以LINQ样式编写的查询添加条件筛选器(或C# 由于在大型列表中的位置而导致EF超时异常。Contains(),c#,entity-framework-6,timeoutexception,C#,Entity Framework 6,Timeoutexception,当我向以LINQ样式编写的查询添加条件筛选器(或WHERE)时,我遇到了一个异常 我得到一个例外: 问题过滤器: if (filterCriteria.ContainsKey(isExcludeKiosk) && filterCriteria.TryGetValue(isExcludeKiosk, out var actualExcludeKiosk)) { if ((bool)actualExcludeKiosk) { var kioskRec
WHERE
)时,我遇到了一个异常
我得到一个例外:
问题过滤器:
if (filterCriteria.ContainsKey(isExcludeKiosk) && filterCriteria.TryGetValue(isExcludeKiosk, out var actualExcludeKiosk))
{
if ((bool)actualExcludeKiosk)
{
var kioskRecordsList = await context.View_BillInfoDetailWithKiosks.Where(w => w.KioskMadeId != null).Select(s => s.KioskMadeId ?? -1).ToListAsync();
result = result.Where(where => !kioskRecordsList.Contains(where.REFERENCE_ID));
}
}
此文件管理器将取出由信息亭生成的某些账单详细信息
。没有直接关系。我在数据库中生成了一个视图,该视图使用一个SQL链接服务器生成一个可能的ID列表,这些ID由kiosk
生成。问题是名单上大约有12000条记录,账单明细也超过500000条。我这样做是为了得到一个SQL查询EF executes的转储,该查询在Management Studio中执行大约需要十分钟:
SELECT
[Project1].[PAY_MODE_ID] AS [PAY_MODE_ID],
[Project1].[BILL_NUMBER] AS [BILL_NUMBER],
[Project1].[CASHIER_ID] AS [CASHIER_ID],
[Project1].[SERVICE_CODE] AS [SERVICE_CODE],
[Project1].[REFERENCE_ID] AS [REFERENCE_ID],
[Project1].[ACCOUNT_CODE] AS [ACCOUNT_CODE],
[Project1].[SERVICE_FEE] AS [SERVICE_FEE],
[Project1].[REQUESTED_QUANTITY] AS [REQUESTED_QUANTITY],
[Project1].[CURRENT_QUANTITY] AS [CURRENT_QUANTITY],
[Project1].[VAT_AMOUNT] AS [VAT_AMOUNT],
[Project1].[KNOWLEDGE_FEE] AS [KNOWLEDGE_FEE],
[Project1].[KNOWLEDGE_FEE_TOTAL] AS [KNOWLEDGE_FEE_TOTAL],
[Project1].[INNOVATION_FEE] AS [INNOVATION_FEE],
[Project1].[INNOVATION_FEE_TOTAL] AS [INNOVATION_FEE_TOTAL],
[Project1].[ITEM_TOTAL] AS [ITEM_TOTAL],
[Project1].[LAST_MODIFIED_BY] AS [LAST_MODIFIED_BY],
[Project1].[INPUT_STATUS] AS [INPUT_STATUS],
[Project1].[STAMP_DATE] AS [STAMP_DATE],
[Project1].[BILL_NUMBER1] AS [BILL_NUMBER1],
[Project1].[PAY_MODE_ID1] AS [PAY_MODE_ID1],
[Project1].[CASHIER_ID1] AS [CASHIER_ID1],
[Project1].[FOCUS_BILL_NUMBER] AS [FOCUS_BILL_NUMBER],
[Project1].[BILL_DATE] AS [BILL_DATE],
[Project1].[CUSTOMER_CODE] AS [CUSTOMER_CODE],
[Project1].[CUSTOMER_NAME] AS [CUSTOMER_NAME],
[Project1].[CUSTOMER_KIND_ID] AS [CUSTOMER_KIND_ID],
[Project1].[BILL_AMOUNT] AS [BILL_AMOUNT],
[Project1].[BANK_ID] AS [BANK_ID],
[Project1].[BANK_BRANCH_ID] AS [BANK_BRANCH_ID],
[Project1].[BANK_TRANSFER_TRANSACTION_NUMBER] AS [BANK_TRANSFER_TRANSACTION_NUMBER],
[Project1].[CHEQUE_NUMBER] AS [CHEQUE_NUMBER],
[Project1].[CHEQUE_DUE_DATE] AS [CHEQUE_DUE_DATE],
[Project1].[REMARKS] AS [REMARKS],
[Project1].[DEPOSIT_ID] AS [DEPOSIT_ID],
[Project1].[GRP_IS_SUBMITTED] AS [GRP_IS_SUBMITTED],
[Project1].[GRP_BATCH_FULL_ID] AS [GRP_BATCH_FULL_ID],
[Project1].[GRP_BATCH_ID] AS [GRP_BATCH_ID],
[Project1].[GRP_BATCH_DATE] AS [GRP_BATCH_DATE],
[Project1].[GRP_INVOICE_NUMBER] AS [GRP_INVOICE_NUMBER],
[Project1].[GRP_RECEIPT_NUMBER] AS [GRP_RECEIPT_NUMBER],
[Project1].[GRP_DOCUMENT_NUMBER] AS [GRP_DOCUMENT_NUMBER],
[Project1].[REFUND_ID] AS [REFUND_ID],
[Project1].[BILL_TYPE_ID] AS [BILL_TYPE_ID],
[Project1].[REASON_ID] AS [REASON_ID],
[Project1].[ACCOUNT_GROUP] AS [ACCOUNT_GROUP],
[Project1].[ACCOUNT_CODE1] AS [ACCOUNT_CODE1],
[Project1].[REFERENCE_NUMBER] AS [REFERENCE_NUMBER],
[Project1].[CREDIT_CARD_TYPE_ID] AS [CREDIT_CARD_TYPE_ID],
[Project1].[CREDIT_CARD_CHARGES] AS [CREDIT_CARD_CHARGES],
[Project1].[CREDIT_CARD_AUTH_CODE] AS [CREDIT_CARD_AUTH_CODE],
[Project1].[EDIRHAM_CARD_TYPE_ID] AS [EDIRHAM_CARD_TYPE_ID],
[Project1].[EDIRHAM_AUTH_CODE] AS [EDIRHAM_AUTH_CODE],
[Project1].[PAID_CURRENCY] AS [PAID_CURRENCY],
[Project1].[AMOUNT_IN_WORD] AS [AMOUNT_IN_WORD],
[Project1].[TRANSACTION_LOCATION_ID] AS [TRANSACTION_LOCATION_ID],
[Project1].[DA_TRN] AS [DA_TRN],
[Project1].[CUSTOMER_TRN] AS [CUSTOMER_TRN],
[Project1].[LAST_MODIFIED_BY1] AS [LAST_MODIFIED_BY1],
[Project1].[INPUT_STATUS1] AS [INPUT_STATUS1],
[Project1].[STAMP_DATE1] AS [STAMP_DATE1],
[Project1].[ID] AS [ID],
[Project1].[NAME] AS [NAME],
[Project1].[REPORT_NAME] AS [REPORT_NAME],
[Project1].[INPUT_STATUS2] AS [INPUT_STATUS2],
[Project1].[SERVICE_CODE1] AS [SERVICE_CODE1],
[Project1].[NAME1] AS [NAME1],
[Project1].[FEE] AS [FEE],
[Project1].[ACCOUNT_CODE2] AS [ACCOUNT_CODE2],
[Project1].[TERMINAL] AS [TERMINAL],
[Project1].[DURATION] AS [DURATION],
[Project1].[KNOWLEDGE_FEE1] AS [KNOWLEDGE_FEE1],
[Project1].[INNOVATION_FEE1] AS [INNOVATION_FEE1],
[Project1].[IS_VAT] AS [IS_VAT],
[Project1].[VAT_VALUE] AS [VAT_VALUE],
[Project1].[VAT_TYPE] AS [VAT_TYPE],
[Project1].[INPUT_STATUS3] AS [INPUT_STATUS3],
[Project1].[STAMP_DATE2] AS [STAMP_DATE2],
[Project1].[GRP_MEMO_LINE] AS [GRP_MEMO_LINE],
[Project1].[PROCESS_TYPE_ID] AS [PROCESS_TYPE_ID]
FROM ( SELECT
[Filter1].[BILL_NUMBER1] AS [BILL_NUMBER],
[Filter1].[PAY_MODE_ID1] AS [PAY_MODE_ID],
[Filter1].[CASHIER_ID1] AS [CASHIER_ID],
[Filter1].[SERVICE_CODE] AS [SERVICE_CODE],
[Filter1].[REFERENCE_ID] AS [REFERENCE_ID],
[Filter1].[ACCOUNT_CODE1] AS [ACCOUNT_CODE],
[Filter1].[SERVICE_FEE] AS [SERVICE_FEE],
[Filter1].[REQUESTED_QUANTITY] AS [REQUESTED_QUANTITY],
[Filter1].[CURRENT_QUANTITY] AS [CURRENT_QUANTITY],
[Filter1].[VAT_AMOUNT] AS [VAT_AMOUNT],
[Filter1].[KNOWLEDGE_FEE] AS [KNOWLEDGE_FEE],
[Filter1].[KNOWLEDGE_FEE_TOTAL] AS [KNOWLEDGE_FEE_TOTAL],
[Filter1].[INNOVATION_FEE] AS [INNOVATION_FEE],
[Filter1].[INNOVATION_FEE_TOTAL] AS [INNOVATION_FEE_TOTAL],
[Filter1].[ITEM_TOTAL] AS [ITEM_TOTAL],
[Filter1].[LAST_MODIFIED_BY1] AS [LAST_MODIFIED_BY],
[Filter1].[INPUT_STATUS1] AS [INPUT_STATUS],
[Filter1].[STAMP_DATE1] AS [STAMP_DATE],
[Filter1].[BILL_NUMBER2] AS [BILL_NUMBER1],
[Filter1].[PAY_MODE_ID2] AS [PAY_MODE_ID1],
[Filter1].[CASHIER_ID2] AS [CASHIER_ID1],
[Filter1].[FOCUS_BILL_NUMBER] AS [FOCUS_BILL_NUMBER],
[Filter1].[BILL_DATE] AS [BILL_DATE],
[Filter1].[CUSTOMER_CODE] AS [CUSTOMER_CODE],
[Filter1].[CUSTOMER_NAME] AS [CUSTOMER_NAME],
[Filter1].[CUSTOMER_KIND_ID] AS [CUSTOMER_KIND_ID],
[Filter1].[BILL_AMOUNT] AS [BILL_AMOUNT],
[Filter1].[BANK_ID] AS [BANK_ID],
[Filter1].[BANK_BRANCH_ID] AS [BANK_BRANCH_ID],
[Filter1].[BANK_TRANSFER_TRANSACTION_NUMBER] AS [BANK_TRANSFER_TRANSACTION_NUMBER],
[Filter1].[CHEQUE_NUMBER] AS [CHEQUE_NUMBER],
[Filter1].[CHEQUE_DUE_DATE] AS [CHEQUE_DUE_DATE],
[Filter1].[REMARKS] AS [REMARKS],
[Filter1].[DEPOSIT_ID] AS [DEPOSIT_ID],
[Filter1].[GRP_IS_SUBMITTED] AS [GRP_IS_SUBMITTED],
[Filter1].[GRP_BATCH_FULL_ID] AS [GRP_BATCH_FULL_ID],
[Filter1].[GRP_BATCH_ID] AS [GRP_BATCH_ID],
[Filter1].[GRP_BATCH_DATE] AS [GRP_BATCH_DATE],
[Filter1].[GRP_INVOICE_NUMBER] AS [GRP_INVOICE_NUMBER],
[Filter1].[GRP_RECEIPT_NUMBER] AS [GRP_RECEIPT_NUMBER],
[Filter1].[GRP_DOCUMENT_NUMBER] AS [GRP_DOCUMENT_NUMBER],
[Filter1].[REFUND_ID] AS [REFUND_ID],
[Filter1].[BILL_TYPE_ID] AS [BILL_TYPE_ID],
[Filter1].[REASON_ID] AS [REASON_ID],
[Filter1].[ACCOUNT_GROUP] AS [ACCOUNT_GROUP],
[Filter1].[ACCOUNT_CODE2] AS [ACCOUNT_CODE1],
[Filter1].[REFERENCE_NUMBER] AS [REFERENCE_NUMBER],
[Filter1].[CREDIT_CARD_TYPE_ID] AS [CREDIT_CARD_TYPE_ID],
[Filter1].[CREDIT_CARD_CHARGES] AS [CREDIT_CARD_CHARGES],
[Filter1].[CREDIT_CARD_AUTH_CODE] AS [CREDIT_CARD_AUTH_CODE],
[Filter1].[EDIRHAM_CARD_TYPE_ID] AS [EDIRHAM_CARD_TYPE_ID],
[Filter1].[EDIRHAM_AUTH_CODE] AS [EDIRHAM_AUTH_CODE],
[Filter1].[PAID_CURRENCY] AS [PAID_CURRENCY],
[Filter1].[AMOUNT_IN_WORD] AS [AMOUNT_IN_WORD],
[Filter1].[TRANSACTION_LOCATION_ID] AS [TRANSACTION_LOCATION_ID],
[Filter1].[DA_TRN] AS [DA_TRN],
[Filter1].[CUSTOMER_TRN] AS [CUSTOMER_TRN],
[Filter1].[LAST_MODIFIED_BY2] AS [LAST_MODIFIED_BY1],
[Filter1].[INPUT_STATUS2] AS [INPUT_STATUS1],
[Filter1].[STAMP_DATE2] AS [STAMP_DATE1],
[Filter1].[ID] AS [ID],
[Filter1].[NAME] AS [NAME],
[Filter1].[REPORT_NAME] AS [REPORT_NAME],
[Filter1].[INPUT_STATUS3] AS [INPUT_STATUS2],
[Extent4].[SERVICE_CODE] AS [SERVICE_CODE1],
[Extent4].[NAME] AS [NAME1],
[Extent4].[FEE] AS [FEE],
[Extent4].[PROCESS_TYPE_ID] AS [PROCESS_TYPE_ID],
[Extent4].[ACCOUNT_CODE] AS [ACCOUNT_CODE2],
[Extent4].[TERMINAL] AS [TERMINAL],
[Extent4].[DURATION] AS [DURATION],
[Extent4].[KNOWLEDGE_FEE] AS [KNOWLEDGE_FEE1],
[Extent4].[INNOVATION_FEE] AS [INNOVATION_FEE1],
[Extent4].[IS_VAT] AS [IS_VAT],
[Extent4].[VAT_VALUE] AS [VAT_VALUE],
[Extent4].[VAT_TYPE] AS [VAT_TYPE],
[Extent4].[GRP_MEMO_LINE] AS [GRP_MEMO_LINE],
[Extent4].[INPUT_STATUS] AS [INPUT_STATUS3],
[Extent4].[STAMP_DATE] AS [STAMP_DATE2]
FROM (SELECT [Extent1].[BILL_NUMBER] AS [BILL_NUMBER1], [Extent1].[PAY_MODE_ID] AS [PAY_MODE_ID1], [Extent1].[CASHIER_ID] AS [CASHIER_ID1], [Extent1].[SERVICE_CODE] AS [SERVICE_CODE], [Extent1].[REFERENCE_ID] AS [REFERENCE_ID], [Extent1].[ACCOUNT_CODE] AS [ACCOUNT_CODE1], [Extent1].[SERVICE_FEE] AS [SERVICE_FEE], [Extent1].[REQUESTED_QUANTITY] AS [REQUESTED_QUANTITY], [Extent1].[CURRENT_QUANTITY] AS [CURRENT_QUANTITY], [Extent1].[VAT_AMOUNT] AS [VAT_AMOUNT], [Extent1].[KNOWLEDGE_FEE] AS [KNOWLEDGE_FEE], [Extent1].[KNOWLEDGE_FEE_TOTAL] AS [KNOWLEDGE_FEE_TOTAL], [Extent1].[INNOVATION_FEE] AS [INNOVATION_FEE], [Extent1].[INNOVATION_FEE_TOTAL] AS [INNOVATION_FEE_TOTAL], [Extent1].[ITEM_TOTAL] AS [ITEM_TOTAL], [Extent1].[LAST_MODIFIED_BY] AS [LAST_MODIFIED_BY1], [Extent1].[INPUT_STATUS] AS [INPUT_STATUS1], [Extent1].[STAMP_DATE] AS [STAMP_DATE1], [Extent2].[BILL_NUMBER] AS [BILL_NUMBER2], [Extent2].[PAY_MODE_ID] AS [PAY_MODE_ID2], [Extent2].[CASHIER_ID] AS [CASHIER_ID2], [Extent2].[FOCUS_BILL_NUMBER] AS [FOCUS_BILL_NUMBER], [Extent2].[BILL_DATE] AS [BILL_DATE], [Extent2].[CUSTOMER_CODE] AS [CUSTOMER_CODE], [Extent2].[CUSTOMER_NAME] AS [CUSTOMER_NAME], [Extent2].[CUSTOMER_KIND_ID] AS [CUSTOMER_KIND_ID], [Extent2].[BILL_AMOUNT] AS [BILL_AMOUNT], [Extent2].[BANK_ID] AS [BANK_ID], [Extent2].[BANK_BRANCH_ID] AS [BANK_BRANCH_ID], [Extent2].[BANK_TRANSFER_TRANSACTION_NUMBER] AS [BANK_TRANSFER_TRANSACTION_NUMBER], [Extent2].[CHEQUE_NUMBER] AS [CHEQUE_NUMBER], [Extent2].[CHEQUE_DUE_DATE] AS [CHEQUE_DUE_DATE], [Extent2].[REMARKS] AS [REMARKS], [Extent2].[DEPOSIT_ID] AS [DEPOSIT_ID], [Extent2].[GRP_IS_SUBMITTED] AS [GRP_IS_SUBMITTED], [Extent2].[GRP_BATCH_FULL_ID] AS [GRP_BATCH_FULL_ID], [Extent2].[GRP_BATCH_ID] AS [GRP_BATCH_ID], [Extent2].[GRP_BATCH_DATE] AS [GRP_BATCH_DATE], [Extent2].[GRP_INVOICE_NUMBER] AS [GRP_INVOICE_NUMBER], [Extent2].[GRP_RECEIPT_NUMBER] AS [GRP_RECEIPT_NUMBER], [Extent2].[GRP_DOCUMENT_NUMBER] AS [GRP_DOCUMENT_NUMBER], [Extent2].[REFUND_ID] AS [REFUND_ID], [Extent2].[BILL_TYPE_ID] AS [BILL_TYPE_ID], [Extent2].[REASON_ID] AS [REASON_ID], [Extent2].[ACCOUNT_GROUP] AS [ACCOUNT_GROUP], [Extent2].[ACCOUNT_CODE] AS [ACCOUNT_CODE2], [Extent2].[REFERENCE_NUMBER] AS [REFERENCE_NUMBER], [Extent2].[CREDIT_CARD_TYPE_ID] AS [CREDIT_CARD_TYPE_ID], [Extent2].[CREDIT_CARD_CHARGES] AS [CREDIT_CARD_CHARGES], [Extent2].[CREDIT_CARD_AUTH_CODE] AS [CREDIT_CARD_AUTH_CODE], [Extent2].[EDIRHAM_CARD_TYPE_ID] AS [EDIRHAM_CARD_TYPE_ID], [Extent2].[EDIRHAM_AUTH_CODE] AS [EDIRHAM_AUTH_CODE], [Extent2].[PAID_CURRENCY] AS [PAID_CURRENCY], [Extent2].[AMOUNT_IN_WORD] AS [AMOUNT_IN_WORD], [Extent2].[TRANSACTION_LOCATION_ID] AS [TRANSACTION_LOCATION_ID], [Extent2].[DA_TRN] AS [DA_TRN], [Extent2].[CUSTOMER_TRN] AS [CUSTOMER_TRN], [Extent2].[LAST_MODIFIED_BY] AS [LAST_MODIFIED_BY2], [Extent2].[INPUT_STATUS] AS [INPUT_STATUS2], [Extent2].[STAMP_DATE] AS [STAMP_DATE2], [Extent3].[ID] AS [ID], [Extent3].[NAME] AS [NAME], [Extent3].[REPORT_NAME] AS [REPORT_NAME], [Extent3].[INPUT_STATUS] AS [INPUT_STATUS3]
FROM [dbo].[BILL_INFO_DETAIL] AS [Extent1]
INNER JOIN [dbo].[BILL_INFO] AS [Extent2] ON ([Extent1].[BILL_NUMBER] = [Extent2].[BILL_NUMBER]) AND ([Extent1].[PAY_MODE_ID] = [Extent2].[PAY_MODE_ID]) AND ([Extent1].[CASHIER_ID] = [Extent2].[CASHIER_ID])
INNER JOIN [dbo].[MASTER_PAY_MODE] AS [Extent3] ON [Extent2].[PAY_MODE_ID] = [Extent3].[ID]
WHERE ([Extent1].[PAY_MODE_ID] IN (1, 2, 5, 10))
AND ( NOT ([Extent1].[REFERENCE_ID] IN (328645, 325892, 325892, 325892, ... ~12,000 Ids))) AND ([Extent2].[DEPOSIT_ID] IS NOT NULL) AND ([Extent2].[DEPOSIT_ID] > 0) ) AS [Filter1]
LEFT OUTER JOIN [dbo].[SERVICE_INFO] AS [Extent4] ON [Filter1].[SERVICE_CODE] = [Extent4].[SERVICE_CODE]
WHERE ([Filter1].[INPUT_STATUS2] = 1) AND (([Filter1].[GRP_IS_SUBMITTED] = 0) OR (1 = 0))
) AS [Project1]
ORDER BY [Project1].[DEPOSIT_ID] DESC
-- p__linq__0: 'True' (Type = Boolean, IsNullable = false)
-- p__linq__1: 'False' (Type = Boolean)
我执行了一个执行计划,注意到最后一个排序占了80%,所以我删除了它。我把时间缩短到1.11分钟。但这还需要一段时间
计划:
还有优化的余地吗?@Hassan,
首先,您不应该从数据库中将Kiosk列表提取到内存中,然后将其传递给linq查询。这会导致查询未优化。
尝试以使所有命令都能在数据库中运行的方式编写查询。
这样,您将使用一个将已存在的linq查询,而不是IN运算符
var conditionSubQuery=context.View_BillInfoDetailWithKiosks.Where(w => w.KioskMadeId != null);
result = await result.Where(r => !conditionSubQuery.Any(c=>r.REFERENCE_ID==c.KioskMadeId)).ToListAsync();
问候,,
Massoud您的数据库是否有索引?如果是,它们是碎片化的吗?只是通过避免内存转储,查询执行下降到24秒。@哈桑,在优化LINQ查询之后,您应该考虑通过任何合适的索引优化表以使查询运行得更快。一旦主要瓶颈消失,执行计划开始给出索引建议,我看到了所有这些建议。现在是1.11秒。