Skip to content
Keep Learning Keep Living
Keep Learning Keep Living
Keep Learning Keep Living

STOPPED Job While Migrating Non-CDB to PDB via Autoupgrade

Mustafa, 2022-03-202022-03-20

Hi,

Lately, I started to learn about autoupgrade which is the new upgrade tool for almost anything. Many thanks to Mike Dietrich for amazing blog posts ( https://mikedietrichde.com ). His blog is a real treasure.

I am about to migrate some non-cdbs to pdbs and before doing that I wanted to learn my options about how I can achieve that. I will write a blog post about three ways of migrating non-cdb to pdb soon. one of them, of course, autoupgrade and I almost trying to do it for last 10 days! All my efforts has ended with a STOPPED autoupgrade job at “compatibility check” step and it drove me crazy. You must know that for 10 days I tries tens of different combinations to achieve that and every each of them failed! Finally (just a mistake) I realized what happened. Let me explain:

I don’t want to go into details about autoupgrade non-cdb to pdb operation in this post, so I will just share my config file and command to run autoupgrade.

Here is the test case:

Oracle Database EE 19.14 on Oracle Linux 8 (virtual machine on Windows 11 host).

Latest autopgrade.jar is downloaded from Oracle Support.

Oracle Home is the same for both cdb (sid: cdb) and non-cdb (sid: orcl) databases.

here is my autoupgrade config file:

Oracle PL/SQL
1
2
3
4
5
6
7
8
9
10
11
12
global.autoupg_log_dir=/home/oracle/autoupgrade
 
upg1.log_dir=/home/oracle/autoupgrade
upg1.sid=orcl
upg1.source_home=/u01/app/oracle/product/19c/dbhome_1
upg1.target_cdb=cdb
upg1.target_home=/u01/app/oracle/product/19c/dbhome_1
upg1.target_pdb_name=my_pdb
upg1.start_time=now
upg1.upgrade_node=localhost
upg1.run_utlrp=yes
upg1.timezone_upg=yes

then run the autoupgrade to migrade orcl db into a CDB as a pdb:

Shell
1
$ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config /home/oracle/upg2.cfg -mode deploy

I have read so many documents and blog posts about it and every time I run this command, it started Command Line Interface (CLI) of autoupgrade and when I query my job with lsj command, I got this result:

Oracle PL/SQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
oracle@Linux8_19c ~ $ $ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config /home/oracle/upg2.cfg -mode deploy
AutoUpgrade 22.1.220311 launched with default internal options
Processing config file ...
+--------------------------------+
| Starting AutoUpgrade execution |
+--------------------------------+
1 Non-CDB(s) will be processed
Type 'help' to list console commands
upg> lsj
+----+-------+-----------+---------+-------+----------+-------+-------------------+
|Job#|DB_NAME|      STAGE|OPERATION| STATUS|START_TIME|UPDATED|            MESSAGE|
+----+-------+-----------+---------+-------+----------+-------+-------------------+
| 100|   orcl|NONCDBTOPDB|  STOPPED|RUNNING|  22:02:12|19s ago|Compatibility check|
+----+-------+-----------+---------+-------+----------+-------+-------------------+
Total jobs 1
 
upg>

This is instant. I type lsj immediately when I reach to cli and I saw the upgrade job is STOPPED (status is running but operation is stopped and job is not working). So I check the log files of course and here is the error:

Java
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
2022-03-20 22:02:14.970 WARNING Execution of the JobStageAction [oracle.upgrade.autoupgrade.dispatcher.facade.subsystems.ExecuteNonCDBToPDB] failed due to [java.lang.ArrayIndexOutOfBoundsException] - JobStage.executeStage
java.lang.ArrayIndexOutOfBoundsException: 1
        at oracle.upgrade.autoupgrade.noncdbtopdb.converttopdb.AbstractNonCDBToPDB.getPDBPlugInViolations(AbstractNonCDBToPDB.java:208)
        at oracle.upgrade.autoupgrade.noncdbtopdb.converttopdb.CheckPluginCompatibility.getReportableViolations(CheckPluginCompatibility.java:132)
        at oracle.upgrade.autoupgrade.noncdbtopdb.converttopdb.CheckPluginCompatibility.execCheckPlugIn(CheckPluginCompatibility.java:103)
        at oracle.upgrade.autoupgrade.noncdbtopdb.converttopdb.CheckPluginCompatibility.action(CheckPluginCompatibility.java:88)
        at oracle.upgrade.autoupgrade.noncdbtopdb.converttopdb.QuickPluginCompatibility.action(QuickPluginCompatibility.java:66)
        at oracle.upgrade.autoupgrade.noncdbtopdb.NonCDBToPDB.executeNonCDBToPDB(NonCDBToPDB.java:284)
        at oracle.upgrade.autoupgrade.dispatcher.facade.subsystems.ExecuteNonCDBToPDB.executeStage(ExecuteNonCDBToPDB.java:85)
        at oracle.upgrade.autoupgrade.dispatcher.v2.JobStage.executeStage(JobStage.java:71)
        at oracle.upgrade.autoupgrade.dispatcher.v2.RunJobDefinition.runTest(RunJobDefinition.java:129)
        at oracle.upgrade.autoupgrade.dispatcher.v2.def.RunAutoUpgradeJob.executeDispatcher(RunAutoUpgradeJob.java:54)
        at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.run(AutoUpgDispatcher.java:233)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

so an ArrayIndexOutOfBoundsException is thrown. Also, in noncdb_to_pdb_orcl.log file I got this:

Oracle PL/SQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
2022-03-20 22:02:14.756 INFO [688D4E] Executing SQL [/* 688D4E */select name || ':_:' ||
       cause || ':_:' ||
       type || ':_:' ||
       error_number || ':_:' ||
       line || ':_:' ||
       message || ':_:' ||
       status || ':_:' ||
       action || ':_:' ||
       con_id || ':_:'
from PDB_PLUG_IN_VIOLATIONS
where type = 'ERROR' and
status = 'PENDING' and
upper(name) = upper('MY_PDB')
order by time;] in [cdb, container:null] - SQLClient.run
2022-03-20 22:02:14.968 INFO [688D4E] End  - ExecuteSql.quickSQL
2022-03-20 22:02:14.969 INFO Total number of rows found in PDB_PLUG_IN_VIOLATIONS was [1] - AbstractNonCDBToPDB.getPDBPlugInViolations

This makes me think that there might be a compatibility problem or a violation but actually there were none! I spend days to understand this 🙂

After tens of autoupgrade tests, reading hundereds of log file results, with some luck and coincidence (and my hard work of course)  something got my attention. in log files I saw same familiar codes of mine! Where did I use those codes and why are they in autoupgrade logs? They are my “glogin.sql” file codes. for those who are not familiar to glogin.sql it is a initial script file for sqlplus tool in $ORACLE_HOME/sqlplus/admin directory. I use glogin.sql to set some default values like assigning db name in prompt or set SID as terminal header so I can see my current sqlplus SID all the time etc. but non of them is causing this but the guilty one is:

Oracle PL/SQL
1
set timing on

yes! I got “set timing on” in my glogin.sql file so I always get execution time of my sql scripts and somehow autoupgrade does not run a “set timing off” command and this cause some unexpected results. as you know, set timing on adds execution time to end of of your commands outout. probably, some scripts are assume it as a result of a query.  I remove it and re-run the autoupgrade tool. everything work smooth and perfect. database is migrated as a pdb.

So,  I spent 10 days because of “set timing on” but at least I learned a lot and solved a problem of mine, I am proud of it 🙂 I hope this helps whoever uses glogin.sql file and set timing on in it.

Wish you healthy days.

Peace at the Home, Peace in the World (Mustafa Kemal ATATÜRK) .

19c Administration Multi-tenant AbstractNonCDBToPDB.getPDBPlugInViolationsArrayIndexOutOfBoundsExceptionautoupgradeautoupgrade noncdb migrationglogin.sqlnon-cdb to pdbPDB_PLUG_IN_VIOLATIONSSTOPPED job

Post navigation

Previous post
Next post

Comments (4)

  1. Mike Dietrich says:
    2022-03-22 at 11:13

    Hi Mustafa,

    can you please give us access to the logs?

    java -jar autoupgrade.jar -config yourconfig.cfg -zip

    Let us know in case you uploaded it to an SR already. Then we can fetch them from the SR.

    Thanks,
    Mike

    Reply
    1. Mustafa says:
      2022-03-22 at 11:29

      Hi Mike,

      Thanks for your reply. I didn’t create an SR since this was just a test case on a virtual machine (I haven’t do it on my clients system yet). I can send an email to you all logs and config if it’s ok.

      Reply
  2. Joseph Errede says:
    2022-03-24 at 18:18

    Hi Mustafa,

    I am glad you were able to figure the problem out but I am sad it took you 10 days to do it. We have fixed the problem and it will be in the next kit due out soon!

    – Joe –

    Reply
    1. Mustafa says:
      2022-03-24 at 19:00

      Hi Joseph,

      thanks for your reply. I am always happy when I figure out things by myself so 10 days is not a problem 🙂 thank you very much for your interest and glad to hear it will be fixed.

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recent Posts

  • How to call HTTPS Url Without SSL Wallet in 19c
  • Is Table Unnecessary for Pipelined Function
  • Password Rollover Time in Seconds
  • PDB Syncronization Issue
  • How to limit DB Link Connection Timeout
  • Cloud Base Database Service
  • 29th of February and interval
  • How to Copy Local Files to Oracle Object Storage

Recent Comments

  • Mustafa on How to call HTTPS Url Without SSL Wallet in 19c
  • Накрутка авито on How to call HTTPS Url Without SSL Wallet in 19c
  • Mustafa on Cloud Base Database Service
  • Raja on Cloud Base Database Service
  • Mustafa on Refreshable PDB and ORA-17627: ORA-12578: TNS:wallet open failed

Categories

  • 11g
  • 12c
  • 18c
  • 19c
  • 21c
  • 23ai
  • Administration
  • Cloud
  • Compression
  • Development
  • Materialized View
  • Multi-tenant
  • Performance
  • Security
  • SQL / PLSQL
  • Uncategorized
  • Undocumented
  • Useful Scripts

Archives

  • April 2025
  • November 2024
  • July 2024
  • April 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • October 2021
  • September 2021
  • August 2021
  • April 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • July 2019
  • June 2019
  • May 2019
  • March 2019
  • February 2019
  • June 2018

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org

RSS Follow This Blog

  • How to call HTTPS Url Without SSL Wallet in 19c
  • Is Table Unnecessary for Pipelined Function
  • Password Rollover Time in Seconds
  • PDB Syncronization Issue
  • How to limit DB Link Connection Timeout
  • Cloud Base Database Service
  • 29th of February and interval
  • How to Copy Local Files to Oracle Object Storage
  • Guid vs Sequences
  • Refreshable PDB and ORA-17627: ORA-12578: TNS:wallet open failed

Archives

  • April 2025
  • November 2024
  • July 2024
  • April 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • October 2021
  • September 2021
  • August 2021
  • April 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • July 2019
  • June 2019
  • May 2019
  • March 2019
  • February 2019
  • June 2018

RSS Follow This Blog

  • How to call HTTPS Url Without SSL Wallet in 19c
  • Is Table Unnecessary for Pipelined Function
  • Password Rollover Time in Seconds
  • PDB Syncronization Issue
  • How to limit DB Link Connection Timeout
  • Cloud Base Database Service
  • 29th of February and interval
  • How to Copy Local Files to Oracle Object Storage
  • Guid vs Sequences
  • Refreshable PDB and ORA-17627: ORA-12578: TNS:wallet open failed
RSS Error: A feed could not be found at `http://www.mywebsite.com/feed/`; the status code is `200` and content-type is `text/html; charset=UTF-8`
©2025 Keep Learning Keep Living | WordPress Theme by SuperbThemes