Dynamic Pivot in Oracle's SQL

Question

... pivot (sum(A) for B in (X))

Now B is of datatype varchar2 and X is a string of varchar2 values separated by commas.
Values for X are select distinct values from a column(say CL) of same table. This way pivot query was working.

But the problem is that whenever there is a new value in column CL I have to manually add that to the string X.

I tried replacing X with select distinct values from CL. But query is not running.
The reason I felt was due to the fact that for replacing X we need values separated by commas.
Then i created a function to return exact output to match with string X. But query still doesn't run.
The error messages shown are like "missing righr parantheses", "end of file communication channel" etc etc.
I tried pivot xml instead of just pivot, the query runs but gives vlaues like oraxxx etc which are no values at all.

Maybe I am not using it properly.
Can you tell me some method to create a pivot with dynamic values?

Answer 1

You cannot put a dynamic statement in the PIVOT's IN statement without using PIVOT XML, which outputs some less than desirable output. However, you can create an IN string and input it into your statement.

First, here is my sample table;

  myNumber    myValue myLetter
---------- ---------- --------
         1          2 A        
         1          4 B        
         2          6 C        
         2          8 A        
         2         10 B        
         3         12 C        
         3         14 A      

First setup the string to use in your IN statement. Here you are putting the string into "str_in_statement". We are using COLUMN NEW_VALUE and LISTAGG to setup the string.

clear columns
COLUMN temp_in_statement new_value str_in_statement
SELECT DISTINCT 
    LISTAGG('''' || myLetter || ''' AS ' || myLetter,',')
        WITHIN GROUP (ORDER BY myLetter) AS temp_in_statement 
    FROM (SELECT DISTINCT myLetter FROM myTable);

Your string will look like:

'A' AS A,'B' AS B,'C' AS C

Now use the String statement in your PIVOT query.

SELECT * FROM 
    (SELECT myNumber, myLetter, myValue FROM myTable)
    PIVOT (Sum(myValue) AS val FOR myLetter IN (&str_in_statement));

Here is the Output:

  MYNUMBER      A_VAL      B_VAL      C_VAL
---------- ---------- ---------- ----------
         1          2          4            
         2          8         10          6 
         3         14                    12 

There are limitations though. You can only concatenate a string up to 4000 bytes.

Answer 2

You can't put a non constant string in the IN clause of the pivot clause.
You can use Pivot XML for that.

From documentation:

subquery A subquery is used only in conjunction with the XML keyword. When you specify a subquery, all values found by the subquery are used for pivoting

It should look like this:

select xmlserialize(content t.B_XML) from t_aa
pivot xml(
sum(A) for B in(any)
) t;

You can also have a subquery instead of the ANY keyword:

select xmlserialize(content t.B_XML) from t_aa
pivot xml(
sum(A) for B in (select cl from t_bb)
) t;

Here is a sqlfiddle demo

Answer 3

For later readers, here is another solution https://technology.amis.nl/2006/05/24/dynamic-sql-pivoting-stealing-antons-thunder/

allowing a query like

select * from table( pivot(  'select deptno,  job, count(*) c from scott.emp group by deptno,job' ) )

Answer 4

I am not exactly going to give answer for the question OP has asked, instead I will be just describing how dynamic pivot can be done.

Here we have to use dynamic sql, by initially retrieving the column values into a variable and passing the variable inside dynamic sql.

EXAMPLE

Consider we have a table like below.

If we need to show the values in the column YR as column names and the values in those columns from QTY, then we can use the below code.

declare
  sqlqry clob;
  cols clob;
begin
  select listagg('''' || YR || ''' as "' || YR || '"', ',') within group (order by YR)
  into   cols
  from   (select distinct YR from EMPLOYEE);


  sqlqry :=
  '      
  select * from
  (
      select *
      from EMPLOYEE
  )
  pivot
  (
    MIN(QTY) for YR in (' || cols  || ')
  )';

  execute immediate sqlqry;
end;
/

RESULT

If required, you can also create a temp table and do a select query in that temp table to see the results. Its simple, just add the CREATE TABLE TABLENAME AS in the above code.

sqlqry :=
'    
  CREATE TABLE TABLENAME AS
  select * from

Answer 5

USE DYNAMIC QUERY

Test code is below

---

--  DDL for Table TMP_TEST
--------------------------------------------------------

  CREATE TABLE "TMP_TEST" 
   (    "NAME" VARCHAR2(20), 
    "APP" VARCHAR2(20)
   );
/
SET DEFINE OFF;
Insert into TMP_TEST (NAME,APP) values ('suhaib','2');
Insert into TMP_TEST (NAME,APP) values ('suhaib','1');
Insert into TMP_TEST (NAME,APP) values ('shahzad','3');
Insert into TMP_TEST (NAME,APP) values ('shahzad','2');
Insert into TMP_TEST (NAME,APP) values ('shahzad','5');
Insert into TMP_TEST (NAME,APP) values ('tariq','1');
Insert into TMP_TEST (NAME,APP) values ('tariq','2');
Insert into TMP_TEST (NAME,APP) values ('tariq','6');
Insert into TMP_TEST (NAME,APP) values ('tariq','4');
/
  CREATE TABLE "TMP_TESTAPP" 
   (    "APP" VARCHAR2(20)
   );

SET DEFINE OFF;
Insert into TMP_TESTAPP (APP) values ('1');
Insert into TMP_TESTAPP (APP) values ('2');
Insert into TMP_TESTAPP (APP) values ('3');
Insert into TMP_TESTAPP (APP) values ('4');
Insert into TMP_TESTAPP (APP) values ('5');
Insert into TMP_TESTAPP (APP) values ('6');
/
create or replace PROCEDURE temp_test(
  pcursor out sys_refcursor,
    PRESULT                   OUT VARCHAR2
    )
AS
V_VALUES VARCHAR2(4000);
V_QUERY VARCHAR2(4000);
BEGIN
 PRESULT := 'Nothing';

-- concating activities name using comma, replace "'" with "''" because we will use it in dynamic query so "'" can effect query.
  SELECT DISTINCT 
         LISTAGG('''' || REPLACE(APP,'''','''''') || '''',',')
         WITHIN GROUP (ORDER BY APP) AS temp_in_statement 
    INTO V_VALUES
    FROM (SELECT DISTINCT APP 
            FROM TMP_TESTAPP);

-- designing dynamic query  

  V_QUERY := 'select * 
                from (  select NAME,APP 
                          from TMP_TEST   )   
               pivot (count(*) for APP in 
                     (' ||V_VALUES|| '))  
           order  by NAME' ;

    OPEN PCURSOR
     FOR V_QUERY;


 PRESULT := 'Success';

Exception
WHEN OTHERS THEN
 PRESULT := SQLcode || ' - ' || SQLERRM;
END temp_test;

Answer 6

I used the above method (Anton PL/SQL custom function pivot()) and it done the job! As I am not a professional Oracle developer, these are simple steps I've done:

1) Download the zip package to find pivotFun.sql in there. 2) Run once the pivotFun.sql to create a new function 3) Use the function in normal SQL.

Just be careful with dynamic columns names. In my environment I found that column name is limited with 30 characters and cannot contain a single quote in it. So, my query is now something like this:

SELECT 
  *
FROM   
  table( 
        pivot('
                SELECT DISTINCT
                    P.proj_id,
                    REPLACE(substr(T.UDF_TYPE_LABEL, 1, 30), '''''''','','') as Attribute,
                    CASE
                      WHEN V.udf_text is null     and V.udf_date is null and      V.udf_number is NOT null  THEN to_char(V.udf_number)
                      WHEN V.udf_text is null     and V.udf_date is NOT null and  V.udf_number is null      THEN to_char(V.udf_date)
                      WHEN V.udf_text is NOT null and V.udf_date is null and      V.udf_number is null      THEN V.udf_text
                      ELSE NULL END
                    AS VALUE
                FROM
                    project   P
                LEFT JOIN UDFVALUE V ON P.proj_id     = V.proj_id 
                LEFT JOIN UDFTYPE  T ON V.UDF_TYPE_ID = T.UDF_TYPE_ID
                WHERE 
                    P.delete_session_id  IS NULL AND
                    T.TABLE_NAME = ''PROJECT''
    ')
)

Works well with up to 1m records.

Answer 7

Looks like it became possible without extra development effort since Oracle 19c with introduction of SQL_MACRO (and possibly Polymorphic Table Functions, which I haven't use yet).

create table t as
select
  trunc(level/5) as id
  , chr(65+mod(level, 5)) as code
  , level as val
from dual
connect by level < 10

create function f_pivot
return varchar2 SQL_MACRO(TABLE)
is
  l_codes varchar2(1000);
begin
  select listagg(
    distinct '''' || code
    || ''' as ' || code, ',')
    into l_codes
  from t;
  
  return
    'select *
    from t
    pivot (
      max(val) for code in (
      ' || l_codes || '))';
end;
/

select *
from f_pivot()

ID |  B |  C |  D |  E |    A
-: | -: | -: | -: | -: | ---:
 0 |  1 |  2 |  3 |  4 | null
 1 |  6 |  7 |  8 |  9 |    5

The only issue (in case of SQL_MACRO approach) is that result set doen't change its structure during one session:

insert into t
values(1, 'Q', 100);

commit;

select *
from f_pivot()

ID |  B |  C |  D |  E |    A
-: | -: | -: | -: | -: | ---:
 0 |  1 |  2 |  3 |  4 | null
 1 |  6 |  7 |  8 |  9 |    5

But in separate session it works fine:

select dbms_xmlgen.getxml('select * from f_pivot()') as v
from dual

V
<?xml version="1.0"?> <ROWSET> <ROW> <ID>0</ID> <B>1</B> <C>2</C> <D>3</D> <E>4</E> </ROW> <ROW> <ID>1</ID> <B>6</B> <C>7</C> <D>8</D> <E>9</E> <A>5</A> <Q>100</Q> </ROW> </ROWSET>

Using with function feature dynamic pivot may be used in-place without predefined function:

with function f_pivot1
return varchar2 SQL_MACRO(TABLE)
is
  l_codes varchar2(1000);
begin
  select listagg(distinct '''' || code || ''' as ' || code, ',')
    into l_codes
  from t;
  
  return
    'select *
    from t
    pivot (
      max(val) for code in (
      ' || l_codes || '))';
end;

select *
from f_pivot1()

ID |  B |  C |  D |  E |    A |    Q
-: | -: | -: | -: | -: | ---: | ---:
 0 |  1 |  2 |  3 |  4 | null | null
 1 |  6 |  7 |  8 |  9 |    5 |  100

db<>fiddle here

Answer 8

You cannot put a dynamic statement in the PIVOT's IN statement without using PIVOT XML, but you can use small Technic to use dynamic statement in PIVOT. In PL/SQL, within a string value, two apostrophe is equal to one apostrophes.

declare
  sqlqry clob;   
  search_ids  varchar(256) := '''2016'',''2017'',''2018'',''2019''';
begin
  search_ids := concat( search_ids,'''2020''' ); -- you can append new search id dynamically as you wanted
  sqlqry :=
  '      
  select * from
  (
      select *
      from EMPLOYEE
  )
  pivot
  (
    MIN(QTY) for YR in (' || search_ids   || ')
  )';

  execute immediate sqlqry;
end;

Answer 9

There’s no straightforward method for dynamic pivoting in Oracle’s SQL, unless it returns XML type results. For the non-XML results PL/SQL might be used through creating functions of SYS_REFCURSOR return type

• With Conditional Aggregation

  CREATE OR REPLACE FUNCTION Get_Jobs_ByYear RETURN SYS_REFCURSOR IS
    v_recordset SYS_REFCURSOR;
    v_sql       VARCHAR2(32767);
    v_cols      VARCHAR2(32767); 
  BEGIN
    SELECT LISTAGG( 'SUM(  CASE  WHEN job_title =  '''||job_title||'''  THEN  1  ELSE  0  END )  AS  "'||job_title||'"' , ',' )
                   WITHIN GROUP ( ORDER BY job_title )
      INTO v_cols
      FROM ( SELECT DISTINCT job_title
               FROM jobs j );
  
    v_sql :=
    'SELECT "HIRE YEAR",'|| v_cols ||
    '  FROM
       (
        SELECT TO_NUMBER(TO_CHAR(hire_date,''YYYY'')) AS "HIRE YEAR", job_title
          FROM employees e
          JOIN jobs j
            ON j.job_id = e.job_id
       )
      GROUP BY "HIRE YEAR"
      ORDER BY "HIRE YEAR"';
  
    OPEN v_recordset FOR v_sql;
    DBMS_OUTPUT.PUT_LINE(v_sql);
    RETURN v_recordset;
  END;
  /
  

• With PIVOT Clause

  CREATE OR REPLACE FUNCTION Get_Jobs_ByYear RETURN SYS_REFCURSOR IS
    v_recordset SYS_REFCURSOR;
    v_sql       VARCHAR2(32767);
    v_cols      VARCHAR2(32767);
  BEGIN
    SELECT LISTAGG( ''''||job_title||''' AS "'||job_title||'"' , ',' )
                   WITHIN GROUP ( ORDER BY job_title )
      INTO v_cols
      FROM ( SELECT DISTINCT job_title
               FROM jobs j  );
  
    v_sql :=
    'SELECT *
       FROM
       (
        SELECT TO_NUMBER(TO_CHAR(hire_date,''YYYY'')) AS "HIRE YEAR", job_title
          FROM employees e
          JOIN jobs j
            ON j.job_id = e.job_id  
       )
      PIVOT
      (
       COUNT(*) FOR job_title IN ( '|| v_cols ||' )
      )
      ORDER BY "HIRE YEAR"';
  
    OPEN v_recordset FOR v_sql;
    DBMS_OUTPUT.PUT_LINE(v_sql);
    RETURN v_recordset;
  END;
  /
  

But there's a drawback with LISTAGG() that's coded ORA-01489: result of string concatenation is too long raises whenever the concatenated string within the first argument exceeds the length of 4000 characters. In this case, the query returning the value of v_cols variable might be replaced with the XMLELEMENT() function nested within XMLAGG() such as

CREATE OR REPLACE FUNCTION Get_Jobs_ByYear RETURN SYS_REFCURSOR IS
  v_recordset SYS_REFCURSOR;
  v_sql       VARCHAR2(32767);
  v_cols      VARCHAR2(32767); 
BEGIN
  SELECT RTRIM(DBMS_XMLGEN.CONVERT(
         XMLAGG(
                XMLELEMENT(e, 'SUM(  CASE  WHEN job_title =  '''||job_title||
                            '''  THEN  1  ELSE  0  END )  AS  "'||job_title||'",')
               ).EXTRACT('//text()').GETCLOBVAL() ,1),',') AS "v_cols"
    FROM ( SELECT DISTINCT job_title
               FROM jobs j);
           
  v_sql :=
  'SELECT "HIRE YEAR",'|| v_cols ||
  '  FROM
     (
      SELECT TO_NUMBER(TO_CHAR(hire_date,''YYYY'')) AS "HIRE YEAR", job_title
        FROM employees e
        JOIN jobs j
          ON j.job_id = e.job_id
     )
    GROUP BY "HIRE YEAR"
    ORDER BY "HIRE YEAR"';
  DBMS_OUTPUT.put_line(LENGTH(v_sql));
  OPEN v_recordset FOR v_sql;
  RETURN v_recordset;
END;
/

unless the upper limit 32767 for VARCHAR2 type is exceeded. This last method might also be applied for the database with version prior to Oracle 11g Release 2 as they don't contain LISTAGG() function.

Btw, yet LISTAGG() function can be used during the checkout of the v_cols even for very long concatenated string generated without getting ORA-01489 error while the trailing part of the string is truncated through use of ON OVERFLOW TRUNCATE clause if the version for the database is 12.2+ such as

LISTAGG( <concatenated string>,',' ON OVERFLOW TRUNCATE 'THE REST IS TRUNCATED' WITHOUT COUNT )

The function can be invoked as

VAR rc REFCURSOR
EXEC :rc := Get_Jobs_ByYear;
PRINT rc

from SQL Developer's command line

or

BEGIN
  :result := Get_Jobs_ByYear;
END;

from Test window of PL/SQL Developer in order to get the result set.

Demo for generated queries

Answer 10

You can dynamically pivot data in a single SQL statement with the open source program Method4.Pivot.

After installing the package, call the function and pass in a SQL statement as a string. The last column of your SQL statement defines the values, and the second-to-last column defines the column names. The default aggregation function is MAX, which works well for common entity-attribute-value queries like this one:

select * from table(method4.pivot(
    q'[
        select 'A' name, 1 value from dual union all
        select 'B' name, 2 value from dual union all
        select 'C' name, 3 value from dual
    ]'
));

A   B   C
-   -   -
1   2   3

The program also supports different aggregation functions through the parameter P_AGGREGATE_FUNCTION, and allows for a custom column name order if you add a column named PIVOT_COLUMN_ID.

The package uses an Oracle Data Cartridge approach similar to Anton's pivot, but Method4.Pivot has several important advantages:

1. Regular open source program with a repo, installation instructions, license, unit tests, documentation, and comments - not just a Zip file on a blog.
2. Handles unusual column names.
3. Handles unusual data types, like floats.
4. Handles up to 1000 columns.
5. Provides meaningful error messages for common mistakes.
6. Handles NULL column names.
7. Handles 128-character column names.
8. Prevents misleading implicit conversion.
9. Hard-parses statements each time to catch underlying table changes.

But most users are still better off creating a dynamic pivot at the application layer or with the pivot XML option.

Answer 11

There are a few ways to work around this issue. One way is to use the PIVOT XML statement. The PIVOT XML statement allows you to use a dynamic value in the IN clause. However, the output of the PIVOT XML statement is not as readable as the output of the PIVOT statement.

Here is an example of how to use the PIVOT XML statement to pivot the values in the A column by the values in the CL column:

SELECT
  XMLTYPE(pivot_xml(sum(A) FOR B IN (CL))) AS pivot_xml
FROM
  my_table

The output of this query would be an XML document that contains the pivoted data. The XML document would be in a format that is not very readable, but it can be parsed by other applications.

Another way to work around this issue is to use dbForge Studio for Oracle, which helps to create a pivot with dynamic values by allowing you to use dynamic SQL in the pivot table layout. Dynamic SQL is a type of SQL that is not fully defined until runtime. This means that you can use dynamic SQL to create a pivot table that uses values that are not known until the pivot table is created.

For example, you could use dynamic SQL to create a pivot table that uses the current year as the column values. The following is an example of how to do this:

SELECT
  year,
  SUM(salary) AS total_salary
FROM
  employees
WHERE
  year = :year
PIVOT
  (
    SUM(salary)
    FOR year IN (:years)
  )

In this example, the :year and :years parameters are dynamic values. The :year parameter is the current year, and the :years parameter is a list of years that the pivot table should use.

When the pivot table is created, the :year and :years parameters will be replaced with the actual values. This will allow the pivot table to use the current year as the column values, and to use the list of years that you specified as the possible values for the column values.

This tool makes it easy to use dynamic SQL in pivot tables. You can simply drag and drop the dynamic SQL into the pivot table layout, and dbForge Studio for Oracle will take care of the rest.