AnySQL.net English

Hint USE_CONCAT does not work sometime on 10g database

2007-08-12T12:14:40Z

I have introduced a fast way to access X$KSLEI kernel table. But after lots of test on different versions of Oracle database, I found that sometime this hint does not work. I hit this problem on 10g version. For example:

SQL> SELECT /*+ USE_CONCAT */ * FROM X$KSLEI WHERE INDX IN (1,2);

----------------------------------------------------
| Id  | Operation        | Name    | Rows  | Bytes |
----------------------------------------------------
|   0 | SELECT STATEMENT |         |    20 |  1680 |
|*  1 |  FIXED TABLE FULL| X$KSLEI |    20 |  1680 |
----------------------------------------------------

Finally I have to get it done by complex way, rewrite the SQL with UNION ALL, then Oracle have no way except the right way.

SQL> SELECT * FROM X$KSLEI WHERE INDX = 1 UNION ALL
  2  SELECT * FROM X$KSLEI WHERE INDX = 2
  3  /

----------------------------------------------------
| Id  | Operation                | Name            |
----------------------------------------------------
|   0 | SELECT STATEMENT         |                 |
|   1 |  UNION-ALL               |                 |
|*  2 |   FIXED TABLE FIXED INDEX| X$KSLEI (ind:2) |
|*  3 |   FIXED TABLE FIXED INDEX| X$KSLEI (ind:2) |
----------------------------------------------------

I do think of direct SGA access, but with well tuned SQLs, we still can get the job done well.

How to access Oracle kernel table quickly?

2007-08-07T10:54:58Z

There are special indexes on Oracle kernel tables (Usually the INDX column), if your query can use it, it can run more quickly, else it can be very slow. The simplest way is to equal access. Let's choose the base table of V$SYSTEM_EVENT : X$KSLEI for example.

SQL> SELECT INDX, KSLESWTS,KSLESTIM FROM X$KSLEI WHERE INDX=1;

INDX KSLESWTS KSLESTIM
---------- ---------- ----------
1 0 0

Elapsed: 00:00:00.01

Replace it with IN? It becomes slowly because index is not used.

SQL> SELECT INDX, KSLESWTS,KSLESTIM FROM X$KSLEI WHERE INDX IN (100,200);

      INDX   KSLESWTS   KSLESTIM
---------- ---------- ----------
       100          0          0
       200          6      57927

Elapsed: 00:00:02.11

I cannot believe it! Let's add a hint into the SQL.

SQL> SELECT /*+ USE_CONCAT */ INDX, KSLESWTS,KSLESTIM
   2 FROM X$KSLEI WHERE INDX IN (100,200);

      INDX   KSLESWTS   KSLESTIM
---------- ---------- ----------
       200          6      57927
       100          0          0

Elapsed: 00:00:00.03

In one of my program I use -1 to represent a non-existent wait event, the result is very bad performance!

SQL> SELECT /*+ USE_CONCAT */ INDX, KSLESWTS,KSLESTIM
   2 FROM X$KSLEI WHERE INDX IN (100,200,-1);

      INDX   KSLESWTS   KSLESTIM
---------- ---------- ----------
       200          6      57927
       100          0          0

Elapsed: 00:00:02.12

One of my friend suggest to replace -1 with a big positive number.

SQL> SELECT /*+ USE_CONCAT */ INDX, KSLESWTS,KSLESTIM
   2 FROM X$KSLEI WHERE INDX IN (100,200,100000);

      INDX   KSLESWTS   KSLESTIM
---------- ---------- ----------
       200          6      57927
       100          0          0

Elapsed: 00:00:00.02

The kerenl table indexes are not the same as normal indexes, it's just a algorith based on hash method.

How to get the current session id in Oracle?

2007-08-07T10:43:27Z

Usually I get it from the first row of V$MYSTAT, this is a good way.

SQL> SELECT SID FROM V$MYSTAT WHERE ROWNUM =1;

SID
----------
19949

In Oracle 10g or above, we can use a more simple way with the USERENV function

SQL> SELECT USERENV('SID') FROM DUAL;

USERENV('SID')
--------------
19949

Sometime I also use the following method, but in Oracle 10g, it does work but return 2 rows for you, so actually not work for you in Oracle 10g or above. We should not use it any more.

SQL> SELECT SID FROM V$SESSION WHERE AUDSID=USERENV('SESSIONID');

       SID
----------
     19949
     19829

Are there any other ways?.

I have two questions, any suggestion?

2007-07-28T13:47:32Z

Oracle will release 11g soon, from the white paper, we can see that Oracle made some big changes to the compress tables, it will support for OLTP system, and help you to save about 50% of the storage. Because of CPU are faster and faster, but IO are really slowly improved compare to the CPU technology. So I have to enable the compress table support for my data recovery tool -- AUL/MyDUL. This is really a big challenge for me.

I spent some time to continue the oracle log format research, and now I can basically extract the SQL from the redo log file (Sample). Because Oracle 11g will support open-recovery standby database, so this research have less and less value to me to convert it to a useful disaster recovery tools. I have very less resource and cannot get it done by personal. So I decide to open source it, anybody have experience of the open source? I want somebody to join the development, or anybody have fun?

Is open source good for a people who are busy in making the live.

Learning from MySQL, make AnySQL more powerful.

2007-07-28T13:33:13Z

After some research of MySQL, Yong Huang gave me a excellent advice to add a new feature in AnySQL, to display in the form mode other than grid mode. This feature is very useful when display few rows with lots of columns. Let's get some objects for testing.

ASQL> select * from tab where rownum < 5;

TNAME      TABTYPE CLUSTERID
---------- ------- ---------
BONUS      TABLE
CLU_A      CLUSTER
CR_5043802 TABLE
DEPT       TABLE

4 rows returned.

Then we are required to display the object information from table "USER_OBJECTS", by adding one of patterns ("/g","/G","\G","\g") to the standard select statement, the row will be displayed in form mode. As following.

ASQL> select * from user_objects where object_name='BONUS'/g;

OBJECT_NAME                   : BONUS
SUBOBJECT_NAME                : null
OBJECT_ID                     : 637961
DATA_OBJECT_ID                : 637961
OBJECT_TYPE                   : TABLE
CREATED                       : 2007-05-16 22:03:27.0
LAST_DDL_TIME                 : 2007-05-16 22:03:27.0
TIMESTAMP                     : 2007-05-16:22:03:27
STATUS                        : VALID
TEMPORARY                     : N
GENERATED                     : N
SECONDARY                     : N

1 rows returned.

Download patch if you want to enjoy it!

Delete from DBA_TABLESPACES where tablespace_name = ...

2007-06-28T14:59:50Z

I saw somebody execute the delete statement on DBA_TABLESPACES directly without using drop tablespace statement by mistake. Non of us hit this issue before, so it's very hard to give him an answer. Today I test it on my personal database (10g version).

SQL> delete from dba_tablespaces where tablespace_name='USERS';

1 row deleted.

SQL> commit;

Commit complete.

SQL> alter system checkpoint;

System altered.

I decide to restart the database because I am worrying about whether I can open the database. But lucky I open the database without any error, but get ORA-600 error when I try to query the table in that tablespace.

SQL> select * from anysql.emp;
select * from anysql.emp
*
ERROR at line 1:
ORA-00600: internal error code, arguments: [5119], [], [], [], [], [], [], []

I find out the deleted row with AUL software, and then load it into sys.ts$ table, and restart the database successfully. Everything is fine now, no errors when I query that table again. But of cause, you should not make this kind of mistake.

Test Oracle 11g data file recovery with AUL utility

2007-06-20T12:25:38Z

I asked someone who have Oracle 11g beta version to send me a data file of Oracle 11g, and I got it this morning, great thanks to this unknown friend. Then I use AUL to test it, extract the zip file and create a AUL configuration file named "db.txt" as following.

1 1 users01.dbf

Then run the AUL and open it, check the data file header block with ORADUMP command. "ver=0x0b100000" means the data file is from Oracle 11g, "fmt=0xa2" means the block format is the same with oracle 10g's.

AUL> open db.txt
* ts# fno rfn ver bsize blocks filename
- ---- ---- ---- --- ----- ---------- -----------------------
Y 4 4 4 a2 8192 640 users01.dbf
AUL> oradump file 4 block 1
RDBA=0x01000001(4/1),type=0x0b,fmt=0xa2, seq=0x01, flag=0x04
DBID=0x44814ad1=1149323985, db=ORCL, ts#=4, ts=USERS, file#=4, blksiz=8192, blks=640, ver=0x0b100000, fzy=--O-

There is no system information of this data file, so try the method of non-system recovery. Run the "SCAN TABLE to scan_11g.txt" command, and then find the following command lines from the generated text file.

CMD:UNLOAD OBJECT 68415 CLUSTER 0 COLUMN  NUMBER VARCHAR VARCHAR TO OBJD0000068415C000.txt;
CMD:UNLOAD OBJECT 68417 CLUSTER 0 COLUMN  NUMBER VARCHAR VARCHAR NUMBER DATE NUMBER NUMBER NUMBER TO OBJD0000068417C000.txt;
CMD:UNLOAD OBJECT 68420 CLUSTER 0 COLUMN  NUMBER NUMBER NUMBER TO OBJD0000068420C000.txt;
CMD:UNLOAD OBJECT 70052 CLUSTER 0 COLUMN  NUMBER VARCHAR VARCHAR NUMBER DATE NUMBER NUMBER NUMBER TO OBJD0000070052C000.txt;

By carefully check, the guessed column type is completely correct, just run these command, you will see rows are recovered from Oracle 11g data file. But still need more test on Oracle 11g, then I will announce that AUL formally support Oracle 11g.

aul4b is not beta version, The "B" means LOB support

2007-06-19T23:06:19Z

When you download the AUL binary and extract it, you may find that the execution file name on Windows is aul4b.exe, Linux is aul4b_linux, and Solaris is aul4b_solaris. However it's not a beta version, the "4b" means version 4 with LOB support. Since release 4.0.2, it's a release version after two formal recoveries of LOB data.

If you see "B" is added to the suffix of the version number as following, then it's beta version.

Register Code: TRBR-CCPF-F6KJ-ALTT-MNGQ
AUL : AnySQL UnLoader(MyDUL) for Oracle 8/8i/9i/10g, release 4.0.1B

(C) Copyright Lou Fangxin 2005-2007 (AnySQL.net), all rights reserved.

AUL> exit

Just notice.

The speed of LOB recovery with AUL utility, fast or slow?

2007-06-19T13:06:29Z

Some one used AUL utility to recover millions of LOB values, and told me the recovery was very slow, about 10MB generated per minute. I don't think so, and I want to tell him it's impossible slow. They may choose the wrong method to estimate the recovery speed. There are two storage modes of the LOB values. First mode is inline storage, store the LOB value with other columns value, the second mode is file storage, store each LOB value in a distinct file.

Inline is the default method (SET LOB_STORAGE 0), so all the LOB value is stored in one file, the file increase should be very fast. When I try to unload the SYS.SOURCE$ table, the speed is usually 6-8MB per second on my notebook. So this is impossible.

Under file storage mode (SET LOB_STORAGE 1), each LOB value will be stored in distinct file (LOB_xxxxxxxx_xxxx.dat), and the relative LOB column will record the LOB file name. In this mode, the file increase speed maybe slow for the text file which stores the table data, and 10MB per minute is reasonable. When you estimate the recovery speed, you should take the LOB files into account. Or you just count the rows of the text file contains the table data (BLOB Recovery Example).

If AUL give you such a bad impression, it's really unlucky, for me not a big issue, but for you, it's critical because you are not choosing the right recovery methods, and spend a lot of time in doing nothing. On a Linux platform based Intel x86 CPUs, it will run very fast.

How much files can be stored in one directory?

2007-06-19T13:04:16Z

How many files can be stored in on directory? I don't know the answer. But when one directory have a lot of files, it will be very hard to operate, and may slow down the file access speed.

I performed two times of LOB recovery, all recovered in TXT mode. At this mode, each LOB values is stored as a single file in the directory where you run the AUL binary. For this two times, there are no so many LOB rows, the first time has about 12000 LOB values, the second time has about 15000 LOB values, so there is no problem for all of them are stored in one directory.

But how about there are millions of LOB values? I add a new option to AUL to distribute the LOB files among different subdirectories. The option name is "MAXLOBDIR" with default value 500 (The maximum value you can give is 2000). Then the LOB files will be stored in different directories, which can support millions of LOB values recovery, the subdirectory name will look like "LOBxxxx" (xxxx is four digit). For example :

AUL> set MAXLOBDIR 1000
Current MAXLOBDIR is : 1000
AUL>

Now perform the LOB recovery, and check the current directory.

C:\MYDUL\LOBREC>ls
AULCOL.TXT      LOB0245       LOB0394
AULOBJ.TXT      LOB0246       LOB0395
AULTAB.TXT      LOB0247       LOB0396
AULUSR.TXT      LOB0248       LOB0397
LOB0212         LOB0377       LOB0398
LOB0213         LOB0378       LOB0399
LOB0214         LOB0379       LOB0400
LOB0215         LOB0380       LOB0402
LOB0216         LOB0381       LOB0403
LOB0233         LOB0382       LOB0404
LOB0234         LOB0383       LOB0405
LOB0235         LOB0384       LOB0406
LOB0236         LOB0386       LOB0407
LOB0237         LOB0387       LOB0408
LOB0238         LOB0388       LOB0413
LOB0239         LOB0389       T_LOBTEST_sqlldr.ctl
LOB0240         LOB0390       T_LOBTEST_syntax.sql
LOB0242         LOB0391       aul4b.exe
LOB0243         LOB0392       lobtest.txt
LOB0244         LOB0393       t_lobtest.txt

I unload about 5000 LOB values in the example, it looks much better than store all of them in one directory.

Don't abuse the resetlogs option when open database

2007-06-14T13:20:40Z

Someone required a database recovery after a sudden server crash due to the CPU fans. He asked to use AUL software to recovery the table structure, procedure & package source code etc. AUL is not good at recovering them, it's good at recover the data. And a server crash should not cause such a big problem usually, you should be able to open it in normal way in most cases, even the database is running in noarchivelog mode.

Let me tell you the correct steps you should take :

1, Mount the database. If you lost the control file, recreate it.

2, Get the active online log file and current online log file.

3, Try to open the database with "alter database open" command.

4, If failed to open the database, then issue the "recover database" command, input the absolute path of the active online log file, then current online log file when prompted for the archive log file.

Usually you should be able to open the database after a server crash. Don't issue the "ALTER DATABASE OPEN RESETLOGS" first, always backup all the files before you open it with resetlogs option, so when anything wrong, you can restore to the original state.

For today's case, the DBA try to open the database with resetlogs option first without backup, so I have use some hidden parameters to skip the consistent check, and offline some rollback segments to skip the transaction recovery. The problem is he have to create a new database and export the data from old database, and import them to the new database.

The data volume is 210GB, is it easy and quick to perform export and import?

Create a SQL file to create the table for faster recovery

2007-06-14T08:21:45Z

When we cannot open the database for access, we can try AUL software. With system available we can use the DMP format for recovery, however TXT format is more robust according to the previous experience. And sometime DBA cannot find out the script to recreate the tables in new database, so generate a script to recreate the tables is useful during recovery. I made this change, when you recover database with TXT format, a new SQL file will be created.

For example, describe the table structure in AUL.

AUL> desc anysql.emp

Storage(OBJ#=10560 OBJD=10560 TS=4 FILE=4 BLOCK=627 CLUSTER=0)
No. SEQ INT Column Name                   Type
--- --- --- ----------------------------- ----------------
  1   1   1 EMPNO                         NUMBER(4) NOT NULL
  2   2   2 ENAME                         VARCHAR2(30)
  3   3   3 JOB                           VARCHAR2(9)
  4   4   4 MGR                           NUMBER(4)
  5   5   5 HIREDATE                      DATE
  6   6   6 SAL                           NUMBER(7,2)
  7   7   7 COMM                          NUMBER(7,2)
  8   8   8 DEPTNO                        NUMBER(2)

Recover the table by TXT format.

AUL> unload table anysql.emp to emp.txt;
2007-05-31 21:43:35
Unload OBJD=10560 FILE=4 BLOCK=627 CLUSTER=0 ...
2007-05-31 21:43:35
AUL>

You will find that a new SQL file (tablename_syntax.sql) will be created under the directory where you run the AUL, for example (EMP_syntax.sql) :

CREATE TABLE "EMP" ( "EMPNO" NUMBER(4) NOT NULL , "ENAME" VARCHAR2(30) , "JOB" VARCHAR2(9) , "MGR" NUMBER(4) , "HIREDATE" DATE , "SAL" NUMBER(7,2) , "COMM" NUMBER(7,2) , "DEPTNO" NUMBER(2) );

Hope this improvement can fasten the database recovery.

Filter table? Filter index? The OR where clause.

2007-06-06T11:23:57Z

I created a composite index on DEPTNO and ENAME columns for EMP table under SCOTT schema. Then run a SQL with OR in the where clause, check the execute plan:

SQL> SELECT /*+ first_rows no_expand */ * FROM EMP
   2 WHERE deptNO=10 AND (ENAME IS NULL OR ENAME > 'A');

---------------------------------------------------------------
| Id  | Operation                   | Name    | Rows  | Bytes |
---------------------------------------------------------------
|   0 | SELECT STATEMENT            |         |     3 |   330 |
|*  1 |  TABLE ACCESS BY INDEX ROWID| EMP     |     3 |   330 |
|*  2 |   INDEX RANGE SCAN          | IDX_EMP |     3 |       |
---------------------------------------------------------------

   1 - filter("ENAME" IS NULL OR "ENAME">'A')
   2 - access("DEPTNO"=10)

The filter operation performed on the table access. Then I rewrite the SQL with a NVL function, and check execution plan again :

SQL> SELECT /*+ first_rows no_expand */ * FROM EMP
   2 WHERE deptNO=10 AND NVL(ENAME,'B') > 'A';

---------------------------------------------------------------
| Id  | Operation                   | Name    | Rows  | Bytes |
---------------------------------------------------------------
|   0 | SELECT STATEMENT            |         |     3 |   330 |
|   1 |  TABLE ACCESS BY INDEX ROWID| EMP     |     3 |   330 |
|*  2 |   INDEX RANGE SCAN          | IDX_EMP |     1 |       |
---------------------------------------------------------------

   2 - access("DEPTNO"=10)
       filter(NVL("ENAME",'B')>'A')

In a real case of our database, the first SQL need more than 10k consistent gets, while the second just need 150 consistent gets.

Don't use 128 for multiple block read count

2007-06-04T09:43:42Z

We have a very old database which upgraded from Oracle 7, when I was creating an index on this database, the database run out of memory. Before I started to create the index, I did check the free memory of the host, it's 8GB free memory there.

I run the following statement to tuning the index creation.

alter session set sort_area_size=100000000;
alter session set sort_area_retained_size=100000000;
alter session set db_file_multiblock_read_count=128;

We found that each parallel process is allocated about 5G memory as PGA, this is abnormal. Did anyone else hit this problem before? We filed a tar, but Oracle did not give us a solution, just told us that do not specify 128 as the multiple block read count.

After we change the value to 64, it works fine.

An Offline Oracle Log Miner, 8i/9i Version.

2007-06-04T05:18:40Z

After announcing the test version of offline oracle log miner for 10g, I spent a few day's free time in testing and bug fixing. Also I try to modify the code to support 8i/19 version oracle log files. Fortunately it's not so difficult thing to make the code work for 8i/9i, only few code need to be changed. Now you can download the MyLOG binary for oracle 8i/9i.

No new command added, all the same as MyLOG 10g version.

I also provide the binary file for RedHat (x86) and Solaris (Sparc64), for other platforms, I haven't the compile environments. But you can open log file from any platform by setting the BYTE_ORDER option. For example, I open an Solaris oracle log file on windows computer.

LOG> set BYTE_ORDER BIG
BYTE_ORDER = BIG
LOG> open c:\mydul\utility\gbcust1_24112.arc
DBID = 0x9d671cf9 = 2640780537
GROUP      = 8, SEQUENCE   = 24112
File Type  = 2, Next Block = 16059
Start SCN  = 0x031f.05c00824 = 3431775340580
Start Time = 2006-08-23 19:08:21
End SCN    = 0x031f.05c00990 = 3431775340944
End Time   = 2006-08-23 19:08:58
LOG>

Do not use this script for any data recovery, it's just for testing, and no replace of Oracle's log miner work.