Oracle10g 由于采用静态优化器策略,Oracle SQL策略用于缓慢的参数化/过滤查询

Oracle10g 由于采用静态优化器策略,Oracle SQL策略用于缓慢的参数化/过滤查询,oracle10g,sql-execution-plan,union-all,optimization,optimizer-hints,Oracle10g,Sql Execution Plan,Union All,Optimization,Optimizer Hints,更简单地说,如果一个查询参数有一个或多个,例如x_id,(或报表/表格功能参数)对性能至关重要(例如,可以使用一些主键索引),并且可能是(取决于应用的用例/报表过滤器,…) null 精确匹配(例如某个唯一id) 类似的表达方式 甚至是一个regexp表达式 然后,如果所有这些可能性都编码在一个查询中,我只看到并知道优化器将 生成独立于实际参数运行时值的唯一静态计划 因此,不能假设在x_id上使用某个索引,尽管它可能是某个精确匹配 除了 让一些PL/SQL代码从n预定义和用例优化的查询/

更简单地说,如果一个查询参数有一个或多个,例如
x_id
,(或报表/表格功能参数)对性能至关重要(例如,可以使用一些主键索引),并且可能是(取决于应用的用例/报表过滤器,…)

  • null
  • 精确匹配(例如某个唯一id)
  • 类似的表达方式
  • 甚至是一个regexp表达式
然后,如果所有这些可能性都编码在一个查询中,我只看到并知道优化器将

  • 生成独立于实际参数运行时值的唯一静态计划
  • 因此,不能假设在
    x_id
    上使用某个索引,尽管它可能是某个精确匹配
除了

  • 让一些PL/SQL代码从
    n
    预定义和用例优化的查询/视图中进行选择?
    • 这可能是相当大的一个更灵活的参数有一个
  • 或者一些手动构造的字符串和动态编译的查询?

  • 基本上,我有两个稍微不同的用例/问题,如下所示:

    A-
    从tf_sel中选择*

    B-
    从数据联合中选择*

    这可能通过SQL提示或使用其他技巧来解决

    为了加快这些查询的速度,我目前正在将“合并查询”分离到某个实现级别(表函数),这相当麻烦,而且更难维护,但由于查询的执行计划更好,因此确保查询运行得相当快

    如我所见,“强”>主要问题似乎是优化器SQL计划的静态性质,如果它考虑一些“查询时间常数”过滤器参数,那么它总是更有效的。

    with
        -- Question A: What would be a good strategy to make tf_sel with tf_params nearly as fast as query_use_case_1_eq
        --             which actually provides the same result?
        --
        -- - a complex query should be used in various reports with filters
        -- - we want to keep as much as possible filter functionality on the db side (not the report engine side)
        --   to be able to utilize the fast and efficient db engine and for loosely coupled software design
    
    
        complex_query as (  -- just some imaginable complex query with a lot of table/view joins, aggregation/analytical functions etc.
            select 1 as id, 'ab12' as indexed_val, 'asdfasdf' x from dual
            union all select 2, 'ab34', 'a uiop345' from dual
            union all select 3, 'xy34', 'asdf  0u0duaf' from dual
            union all select 4, 'xy55', ' asdja´sf asd' from dual
        )
    
    
    -- <<< comment the following lines in to test it with the above
    
    --  , query_use_case_1_eq as (  -- quite fast and maybe the 95% use case
    --      select * from complex_query where indexed_val = 'ab12'
    --  )
    --select * from query_use_case_1_eq 
    
    -- >>>
    
    -- ID INDEXED_VAL X
    -- -- ----------- --------
    --  1 ab12        asdfasdf
    
    
    -- <<< comment the following lines in to test it with the above
    
    --  , query_use_case_2_all as (  -- significantly slower due to a lot of underlying calculations
    --      select * from complex_query
    --  )
    --select * from query_use_case_2_all
    
    -- >>>
    
    -- ID INDEXED_VAL X
    -- -- ----------- -------------
    --  1 ab12        asdfasdf
    --  2 ab34        a uiop345
    --  3 xy34        asdf  0u0duaf
    --  4 xy55         asdja´sf asd
    
    
    -- <<< comment the following lines in to test it with the above
    
    --  , query_use_case_3_like as (
    --      select * from complex_query where indexed_val like 'ab%'
    --  )
    --select * from query_use_case_3_like
    
    -- >>>
    
    -- ID INDEXED_VAL X
    -- -- ----------- ---------
    --  1 ab12        asdfasdf
    --  2 ab34        a uiop345
    
    
    -- <<< comment the following lines to simulate the table function
    
        , tf_params as (  -- table function params: imagine we have a table function where these are passed depending on the report
            select  'ab12' p_indexed_val,  'eq' p_filter_type  from dual
        )
        , tf_sel as (  -- table function select: nicely integrating all query possiblities, but beeing veeery slow :-(
            select q.* 
            from 
                tf_params p  -- just here so this example works without the need for the actual function
                join complex_query q on (1=1)
            where
                    p_filter_type = 'all'
                or (p_filter_type = 'eq' and indexed_val = p_indexed_val)
                or (p_filter_type = 'like' and indexed_val like p_indexed_val)
                or (p_filter_type = 'regexp' and regexp_like(indexed_val, p_indexed_val))
        )
    
    -- actually we would pass the tf_params above if it were a real table function
    select * from tf_sel
    
    -- >>>
    
    -- ID INDEXED_VAL X
    -- -- ----------- --------
    --  1 ab12        asdfasdf
    
    
    
    -- Question B: How can we speed up data_union with dg_filter to be as fast as the data_group1 query which
    --             actually provides the same result? 
    -- 
    -- A very similar approach is considered in other scenarios where we like to join the results of 
    -- different queries (>5) returning joinable data and beeing filtered based on the same parameters.
    
    -- <<< comment the following lines to simulate the union problem
    
    --  , data_group1 as (  -- may run quite fast
    --      select 'dg1' dg_id, q.* from complex_query q where x < 'a'  -- just an example returning some special rows that should be filtered later on!
    --  )
    --  
    --  , data_group2 as (  -- may run quite fast
    --      select 'dg2' dg_id, q.* from complex_query q where instr(x,'p') >= 0  -- just an example returning some special rows that should be filtered later on!
    --  )   
    --  
    --  
    --  , dg_filter as (  -- may be set by a report or indirectly by user filters
    --      select  'dg1' dg_id  from dual
    --  )
    --  
    --  , data_union as (  -- runs much slower due to another execution plan
    --      select * from (
    --          select * from data_group1 
    --          union all select * from data_group2
    --      )
    --      where dg_id in (select dg_id from dg_filter)
    --  )
    --
    --select * from data_union
    
    -- >>>
    
    -- DG_ID ID INDEXED_VAL X
    -- ----- -- ----------- -------------
    -- dg1    4 xy55         asdja´sf asd
    
    用例A(按列查询类型筛选) 根据你的样本表,这更像是我想要做的。 正如您可以看到的那样,仅使用fast
    查询,其中p.ft_id='eq'和x.indexed_val=p.val
    显示索引使用情况,但是在
    where
    子句中使用所有不同的筛选选项将导致计划开关始终使用完整表扫描:-/
    (即使我在SQL中的任何地方使用
    :p_filter_type
    :p_index_val_filter
    ,而不仅仅是在我放置它的一个位置,它也不会改变。)


    几个特性使优化器能够生成动态计划。最常见的特性是
    FILTER
    操作,不应将其与FILTER谓词混淆。
    FILTER
    操作允许Oracle在运行时基于动态值启用或禁用部分计划。此功能通常用于绑定变量,其他类型的动态查询可能不使用它

    示例模式

    create table sample_table
    (
        indexed_val        varchar2(100),
        some_other_column1 varchar2(100),
        some_other_column2 varchar2(100),
        some_other_column3 varchar2(100)
    );
    
    insert into sample_table
    select level, level, level, level
    from dual
    connect by level <= 100000;
    
    create index sample_table_idx1 on sample_table(indexed_val);
    
    begin
        dbms_stats.gather_table_stats(user, 'sample_table');
    end;
    /
    
    样本计划

    这说明了根据输入使用的计划大不相同。单个
    =
    将使用
    索引范围扫描
    ,没有谓词将使用
    表访问完整
    。这个 正则表达式还使用全表扫描,因为无法索引正则表达式。尽管取决于表达式的确切类型,它可能是 可以通过基于函数的索引或Oracle文本索引启用有用的索引

    Plan hash value: 100704550
    
    ------------------------------------------------------------------------------
    | Id  | Operation                             | Name              | Time     |
    ------------------------------------------------------------------------------
    |   0 | SELECT STATEMENT                      |                   | 00:00:01 |
    |   1 |  UNION-ALL                            |                   |          |
    |*  2 |   FILTER                              |                   |          |
    |   3 |    TABLE ACCESS FULL                  | SAMPLE_TABLE      | 00:00:01 |
    |*  4 |   FILTER                              |                   |          |
    |   5 |    TABLE ACCESS BY INDEX ROWID BATCHED| SAMPLE_TABLE      | 00:00:01 |
    |*  6 |     INDEX RANGE SCAN                  | SAMPLE_TABLE_IDX1 | 00:00:01 |
    |*  7 |   FILTER                              |                   |          |
    |   8 |    TABLE ACCESS BY INDEX ROWID BATCHED| SAMPLE_TABLE      | 00:00:01 |
    |*  9 |     INDEX RANGE SCAN                  | SAMPLE_TABLE_IDX1 | 00:00:01 |
    |* 10 |   FILTER                              |                   |          |
    |* 11 |    TABLE ACCESS FULL                  | SAMPLE_TABLE      | 00:00:01 |
    ------------------------------------------------------------------------------
    
    Predicate Information (identified by operation id):
    ---------------------------------------------------
    
       2 - filter(:P_FILTER_TYPE='all')
       4 - filter(:P_FILTER_TYPE='eq')
       6 - access("INDEXED_VAL"=:P_INDEXED_VAL)
       7 - filter(:P_FILTER_TYPE='like')
       9 - access("INDEXED_VAL" LIKE :P_INDEXED_VAL)
           filter("INDEXED_VAL" LIKE :P_INDEXED_VAL)
      10 - filter(:P_FILTER_TYPE='regexp')
      11 - filter( REGEXP_LIKE ("INDEXED_VAL",:P_INDEXED_VAL))
    
    (更多用于情况A)但也适用于B)以这种方式……)

    我现在正在使用一些混合方法(我问题中的1.点和2.点的组合),实际上非常喜欢它,因为它还提供了良好的调试和封装可能性
    优化器根本不需要在更大的查询中基于基本逻辑分离的查询来寻找最佳策略,例如根据内部的
    过滤器
    规则,哪一种可能是好的,或者最坏的情况下是难以置信的更低效:

  • 在报告中使用此选项

    select * 
    from table(my_report_data_func_sql(
      :val1,
      :val1_filter_type,
      :val2
    ))
    
  • 其中table函数的定义如下

    create or replace function my_report_data_func_sql(
      p_val1              integer   default 1234,
      p_val1_filter_type  varchar2  default 'eq',
      p_val2              varchar2  default null
    ) return varchar2 is
    
      query varchar2(4000) := ' 
        with params as (  -- *: default param
            select
                ''||p_val1||'' p_val1,                            -- eq*
                '''||p_val1_filter_type||''' p_val1_filter_type,  -- [eq, all*, like, regexp]
                '''||p_val2||''' p_val2                           -- null*
            from dual
        )
        select x.*
        from
            params p  -- workaround for standalone-sql-debugging using "with" statement above
            join my_report_data_base_view x on (1=1)
        where 1=1  -- ease of filter expression adding below
        '          
        -- #### FILTER CRITERIAS are appended here ####
    
        -- val1-filter
        ||case p_val1_filter_type
            when 'eq' then '
            and val1 = p_val1
        '   when 'like' then '
            and val1 like p_val1
        '   when 'regexp' then '
            and regexp_like(val1,  p_val1)
        '   else '' end  -- all
    
      ;
    begin
      return query;
    end;
    ;
    
  • 并将通过示例产生以下结果:

    select * 
    from table(my_report_data_func_sql(
      1234,
      'eq',
      'someval2'
    ))
    
    /*
            with params as (  -- *: default param
                select
                    1          p_val1,              -- eq*
                    'eq'       p_val1_filter_type,  -- [eq, all*, like, regexp]
                    'someval2' p_val2               -- null*
                from dual
            )
            select x.*
            from
                params p  -- workaround for standalone-sql-debugging using "with" statement above
                join my_report_data_base_view x on (1=1)
            where 1=1  -- ease of filter expression adding below
                and val1 = p_val1
    */
    

  • 不幸的是,代码高亮显示似乎不起作用,但我不知道如何修复它。。。最好在一些SQL编辑器中复制/粘贴它,以便更容易地发现代码。在这里复制:也在顶部添加了一些更简单的问题描述,无需查看示例。对于这个好答案,它更适合Thx。对于场景B来说听起来很酷,所以我将看看为什么在我的情况下,联盟会如此缓慢。我想这对场景A也有同样的效果,我会向大家汇报。我在底部更新了我的问题,回答并澄清了我想做的事情。我再次拒绝了这个答案,虽然它真的很有帮助,但还没有得到证实,它并不能真正解决A的问题(正如我所看到的),但它表明,它至少可以解决B的问题(我必须用真正的SQL测试它)。另一个问题是,如果我想将这样的逻辑封装在一个表函数中(与直接在最高查询层(=报表)中使用这样的SQL参数相比),这些优化器规则是否仍然有效,或者我是否需要做一些技巧(提示等)?
    select * 
    from table(my_report_data_func_sql(
      :val1,
      :val1_filter_type,
      :val2
    ))
    
    create or replace function my_report_data_func_sql(
      p_val1              integer   default 1234,
      p_val1_filter_type  varchar2  default 'eq',
      p_val2              varchar2  default null
    ) return varchar2 is
    
      query varchar2(4000) := ' 
        with params as (  -- *: default param
            select
                ''||p_val1||'' p_val1,                            -- eq*
                '''||p_val1_filter_type||''' p_val1_filter_type,  -- [eq, all*, like, regexp]
                '''||p_val2||''' p_val2                           -- null*
            from dual
        )
        select x.*
        from
            params p  -- workaround for standalone-sql-debugging using "with" statement above
            join my_report_data_base_view x on (1=1)
        where 1=1  -- ease of filter expression adding below
        '          
        -- #### FILTER CRITERIAS are appended here ####
    
        -- val1-filter
        ||case p_val1_filter_type
            when 'eq' then '
            and val1 = p_val1
        '   when 'like' then '
            and val1 like p_val1
        '   when 'regexp' then '
            and regexp_like(val1,  p_val1)
        '   else '' end  -- all
    
      ;
    begin
      return query;
    end;
    ;
    
    select * 
    from table(my_report_data_func_sql(
      1234,
      'eq',
      'someval2'
    ))
    
    /*
            with params as (  -- *: default param
                select
                    1          p_val1,              -- eq*
                    'eq'       p_val1_filter_type,  -- [eq, all*, like, regexp]
                    'someval2' p_val2               -- null*
                from dual
            )
            select x.*
            from
                params p  -- workaround for standalone-sql-debugging using "with" statement above
                join my_report_data_base_view x on (1=1)
            where 1=1  -- ease of filter expression adding below
                and val1 = p_val1
    */