Hello,
We are running Netbackup 7.5.0.6 on windows server 2008 R2 SP1.
All our policies backup to SLP. All the SLP in our environment is built up like the following exmaple:
First Operation: Backup - "Backup the DB/FS/Vmware.. to OST disks"
Second Operaion: Duplication - "run a duplication job that copy the backups to LTO"
Third Operation: Duplication - "run a duplication job that make 3rd copy to another OST disk in a different controller"
Most of the SLPs finish with status 0 but everyday we got some that failed with error code 191.
This is the Job status of the failed job:
12/22/2013 5:58:52 PM - requesting resource LCM_Robot2
12/22/2013 6:00:41 PM - granted resource LCM_Robot2
12/22/2013 6:00:42 PM - started process RUNCMD (8708)
12/22/2013 6:00:42 PM - ended process 0 (8708)
12/22/2013 6:00:42 PM - begin Duplicate
12/22/2013 6:00:47 PM - requesting resource Robot2
12/22/2013 6:00:47 PM - requesting resource @aaab2
12/22/2013 6:00:47 PM - reserving resource @aaab2
12/22/2013 6:05:45 PM - awaiting resource Robot2 Reason: Drives are in use, Media Server: Media7,
Robot Number: 3, Robot Type: TLD, Media ID: N/A, Drive Name: N/A,
Volume Pool: Duplicate, Storage Unit: Media7-Robot2, Drive Scan Host: N/A
12/22/2013 6:34:01 PM - awaiting resource @aaab2 Reason: Disk pool is unavailable, Media Server: N/A,
Robot Number: NONE, Robot Type: NONE, Media ID: @aaab2, Drive Name: N/A,
Volume Pool: N/A, Storage Unit: N/A, Drive Scan Host: N/A
12/22/2013 6:35:35 PM - awaiting resource Robot2 Reason: Drives are in use, Media Server: Media7,
Robot Number: 3, Robot Type: TLD, Media ID: N/A, Drive Name: N/A,
Volume Pool: Duplicate, Storage Unit: Media7-Robot2, Drive Scan Host: N/A
12/23/2013 12:04:08 AM - awaiting resource Robot2 - No drives are available
12/23/2013 12:14:20 AM - awaiting resource Robot2 Reason: Drives are in use, Media Server: Media7,
Robot Number: 3, Robot Type: TLD, Media ID: N/A, Drive Name: N/A,
Volume Pool: Duplicate, Storage Unit: Media7-Robot2, Drive Scan Host: N/A
12/23/2013 12:17:11 AM - awaiting resource Robot2 - No drives are available
12/23/2013 12:24:05 AM - awaiting resource Robot2 Reason: Drives are in use, Media Server: Media7,
Robot Number: 3, Robot Type: TLD, Media ID: N/A, Drive Name: N/A,
Volume Pool: Duplicate, Storage Unit: Media7-Robot2, Drive Scan Host: N/A
12/23/2013 12:26:55 AM - awaiting resource Robot2 - No drives are available
12/23/2013 12:40:49 AM - awaiting resource Robot2 Reason: Drives are in use, Media Server: Media7,
Robot Number: 3, Robot Type: TLD, Media ID: N/A, Drive Name: N/A,
Volume Pool: Duplicate, Storage Unit: Media7-Robot2, Drive Scan Host: N/A
12/23/2013 12:42:46 AM - awaiting resource Robot2 - No drives are available
12/23/2013 1:17:04 AM - awaiting resource Robot2 Reason: Drives are in use, Media Server: Media7,
Robot Number: 3, Robot Type: TLD, Media ID: N/A, Drive Name: N/A,
Volume Pool: Duplicate, Storage Unit: Media7-Robot2, Drive Scan Host: N/A
12/23/2013 3:06:43 AM - reserved resource @aaab2
12/23/2013 3:06:43 AM - granted resource B01602
12/23/2013 3:06:43 AM - granted resource QNT-0303
12/23/2013 3:06:43 AM - granted resource Media1-Robot2
12/23/2013 3:06:43 AM - granted resource MediaID=@aaab2;DiskVolume=DB2_Store_Backup;DiskPool=StoreOnce_DB2_Backups;Path=DB2_Store_Backup;StorageServer=B6200Node2;MediaServer=Media1
12/23/2013 3:06:46 AM - Info bptm(pid=9104) start
12/23/2013 3:06:46 AM - started process bptm (9104)
12/23/2013 3:06:49 AM - Info bptm(pid=9104) start backup
12/23/2013 3:06:51 AM - Info bpdm(pid=13748) started
12/23/2013 3:06:51 AM - started process bpdm (13748)
12/23/2013 3:06:52 AM - Info bpdm(pid=13748) reading backup image
12/23/2013 3:06:53 AM - Info bpdm(pid=13748) using 256 data buffers
12/23/2013 3:06:53 AM - Info bpdm(pid=13748) requesting nbjm for media
12/23/2013 3:06:54 AM - Info bptm(pid=9104) media id B01602 mounted on drive index 35, drivepath {1,0,43,0}, drivename QNT-0303, copy 2
12/23/2013 3:06:54 AM - Info bptm(pid=9104) INF - Waiting for positioning of media id B01602 on server Media1 for writing.
12/23/2013 3:06:58 AM - begin reading
12/23/2013 3:06:58 AM - Critical bpdm(pid=13748) sts_read_image failed: error 2060022 software error
12/23/2013 3:06:58 AM - Critical bpdm(pid=13748) image read failed: error 2060022: software error
12/23/2013 3:06:58 AM - Error bpdm(pid=13748) cannot read image from disk, Invalid argument
12/23/2013 3:07:11 AM - Error bpduplicate(pid=8708) host Media1 backup id db2server1_1387527793 read failed, media read error (85).
12/23/2013 3:07:11 AM - Error bpduplicate(pid=8708) host Media1 backupid db2server1_1387527793 write failed, termination requested by administrator (150).
12/23/2013 3:07:11 AM - Error bpduplicate(pid=8708) db_IMAGE() failed: images are in process (1519)
12/23/2013 3:07:11 AM - Error bpduplicate(pid=8708) Duplicate of backupid db2server1_1387527793 failed, termination requested by administrator (150).
12/23/2013 3:07:11 AM - requesting resource @aaab2
12/23/2013 3:07:16 AM - granted resource MediaID=@aaab2;DiskVolume=DB2_Store_Backup;DiskPool=StoreOnce_DB2_Backups;Path=DB2_Store_Backup;StorageServer=B6200Node2;MediaServer=Media1
12/23/2013 3:07:17 AM - Info bptm(pid=12560) start
12/23/2013 3:07:18 AM - started process bptm (12560)
12/23/2013 3:07:18 AM - requesting resource Media1-Robot2
12/23/2013 3:07:19 AM - granted resource B01602
12/23/2013 3:07:19 AM - granted resource QNT-0303
12/23/2013 3:07:19 AM - granted resource Media1-Robot2
12/23/2013 3:07:22 AM - Info bptm(pid=12560) start backup
12/23/2013 3:07:24 AM - Info bpdm(pid=14620) started
12/23/2013 3:07:24 AM - started process bpdm (14620)
12/23/2013 3:07:26 AM - Info bpdm(pid=14620) reading backup image
12/23/2013 3:07:26 AM - Info bpdm(pid=14620) using 256 data buffers
12/23/2013 3:07:26 AM - Info bpdm(pid=14620) requesting nbjm for media
12/23/2013 3:07:26 AM - Info bptm(pid=12560) media id B01602 mounted on drive index 35, drivepath {1,0,43,0}, drivename QNT-0303, copy 2
12/23/2013 3:07:27 AM - Info bptm(pid=12560) INF - Waiting for positioning of media id B01602 on server Media1 for writing.
12/23/2013 3:07:29 AM - begin reading
12/23/2013 3:07:29 AM - Critical bpdm(pid=14620) sts_read_image failed: error 2060022 software error
12/23/2013 3:07:30 AM - Critical bpdm(pid=14620) image read failed: error 2060022: software error
12/23/2013 3:07:30 AM - Error bpdm(pid=14620) cannot read image from disk, Invalid argument
12/23/2013 3:07:43 AM - Error bpduplicate(pid=8708) host Media1 backup id db2server1_1387528330 read failed, media read error (85).
12/23/2013 3:07:43 AM - Error bpduplicate(pid=8708) host Media1 backupid db2server1_1387528330 write failed, termination requested by administrator (150).
12/23/2013 3:07:44 AM - Error bpduplicate(pid=8708) db_IMAGE() failed: images are in process (1519)
12/23/2013 3:07:44 AM - Error bpduplicate(pid=8708) Duplicate of backupid db2server1_1387528330 failed, termination requested by administrator (150).
12/23/2013 3:07:44 AM - requesting resource @aaab2
12/23/2013 3:07:46 AM - granted resource MediaID=@aaab2;DiskVolume=DB2_Store_Backup;DiskPool=StoreOnce_DB2_Backups;Path=DB2_Store_Backup;StorageServer=B6200Node2;MediaServer=Media1
12/23/2013 3:07:48 AM - Info bptm(pid=13584) start
12/23/2013 3:07:48 AM - started process bptm (13584)
12/23/2013 3:07:49 AM - requesting resource Media1-Robot2
12/23/2013 3:07:50 AM - granted resource B01602
12/23/2013 3:07:50 AM - granted resource QNT-0303
12/23/2013 3:07:50 AM - granted resource Media1-Robot2
12/23/2013 3:07:53 AM - Info bptm(pid=13584) start backup
12/23/2013 3:07:55 AM - Info bpdm(pid=18304) started
12/23/2013 3:07:55 AM - started process bpdm (18304)
12/23/2013 3:07:56 AM - Info bpdm(pid=18304) reading backup image
12/23/2013 3:07:56 AM - Info bpdm(pid=18304) using 256 data buffers
12/23/2013 3:07:56 AM - Info bpdm(pid=18304) requesting nbjm for media
12/23/2013 3:07:57 AM - Info bptm(pid=13584) media id B01602 mounted on drive index 35, drivepath {1,0,43,0}, drivename QNT-0303, copy 2
12/23/2013 3:07:57 AM - Info bptm(pid=13584) INF - Waiting for positioning of media id B01602 on server Media1 for writing.
12/23/2013 3:07:59 AM - begin reading
12/23/2013 3:08:00 AM - Critical bpdm(pid=18304) sts_read_image failed: error 2060022 software error
12/23/2013 3:08:00 AM - Critical bpdm(pid=18304) image read failed: error 2060022: software error
12/23/2013 3:08:00 AM - Error bpdm(pid=18304) cannot read image from disk, Invalid argument
12/23/2013 3:08:15 AM - Error bpduplicate(pid=8708) host Media1 backup id db2server1_1387529027 read failed, media read error (85).
12/23/2013 3:08:15 AM - Error bpduplicate(pid=8708) host Media1 backupid db2server1_1387529027 write failed, termination requested by administrator (150).
12/23/2013 3:08:15 AM - Error bpduplicate(pid=8708) db_IMAGE() failed: images are in process (1519)
12/23/2013 3:08:15 AM - Error bpduplicate(pid=8708) Duplicate of backupid db2server1_1387529027 failed, termination requested by administrator (150).
12/23/2013 3:08:15 AM - requesting resource @aaab2
12/23/2013 3:08:21 AM - granted resource MediaID=@aaab2;DiskVolume=DB2_Store_Backup;DiskPool=StoreOnce_DB2_Backups;Path=DB2_Store_Backup;StorageServer=B6200Node2;MediaServer=Media1
12/23/2013 3:08:23 AM - Info bptm(pid=18400) start
12/23/2013 3:08:23 AM - started process bptm (18400)
12/23/2013 3:08:23 AM - requesting resource Media1-Robot2
12/23/2013 3:08:25 AM - granted resource B01602
12/23/2013 3:08:25 AM - granted resource QNT-0303
12/23/2013 3:08:25 AM - granted resource Media1-Robot2
12/23/2013 3:08:29 AM - Info bptm(pid=18400) start backup
12/23/2013 3:08:30 AM - Info bpdm(pid=15492) started
12/23/2013 3:08:30 AM - started process bpdm (15492)
12/23/2013 3:08:31 AM - Info bpdm(pid=15492) reading backup image
12/23/2013 3:08:32 AM - Info bpdm(pid=15492) using 256 data buffers
12/23/2013 3:08:32 AM - Info bpdm(pid=15492) requesting nbjm for media
12/23/2013 3:08:33 AM - Info bptm(pid=18400) media id B01602 mounted on drive index 35, drivepath {1,0,43,0}, drivename QNT-0303, copy 2
12/23/2013 3:08:33 AM - Info bptm(pid=18400) INF - Waiting for positioning of media id B01602 on server Media1 for writing.
12/23/2013 3:08:34 AM - begin reading
12/23/2013 3:08:34 AM - Critical bpdm(pid=15492) sts_read_image failed: error 2060022 software error
12/23/2013 3:08:35 AM - Critical bpdm(pid=15492) image read failed: error 2060022: software error
12/23/2013 3:08:36 AM - Error bpdm(pid=15492) cannot read image from disk, Invalid argument
12/23/2013 3:08:51 AM - Error bpduplicate(pid=8708) host Media1 backup id db2server2_1387529660 read failed, media read error (85).
12/23/2013 3:08:51 AM - Error bpduplicate(pid=8708) host Media1 backupid db2server2_1387529660 write failed, termination requested by administrator (150).
12/23/2013 3:08:51 AM - Error bpduplicate(pid=8708) db_IMAGE() failed: images are in process (1519)
12/23/2013 3:08:51 AM - Error bpduplicate(pid=8708) Duplicate of backupid db2server2_1387529660 failed, termination requested by administrator (150).
12/23/2013 3:08:51 AM - Error bpduplicate(pid=8708) Status = no images were successfully processed.
no images were successfully processed(191)
12/23/2013 3:08:51 AM - end Duplicate; elapsed time: 09:08:09