Recovery from LED 552, 554, or 556 in AIX V4


Contents

About this document
Causes of an LED 552, 554, or 556
Recovery procedure

About this document

This document discusses the known causes of LED 552, 554, and 556. Included is a procedure for recovery from these errors. This document applies to AIX Version 4.


Causes of an LED 552, 554, or 556

An LED code of 552, 554, or 556 during a standard disk based boot indicates a failure occurred during the varyon of the rootvg volume group.

Some known causes of an LED 552, 554, or 556 are:


Recovery procedure

To diagnose and fix the problem, boot to a Service mode shell and run the fsck command (file system check) on each file system. If the file system check fails, you may need to perform other steps.

WARNING: Do not use this document if the system is a /usr client, diskless client, or dataless client.

  1. Boot your system into a limited function maintenance shell (Service or Maintenance mode) from bootable AIX media to use this recovery procedure.

    Please refer to your system user's or installation and service guide for specific IPL procedures related to your type and model of hardware. You can also refer to the document titled "Booting in Service Mode", available at http:// techsupport.services.ibm.com/rs6k/techbrowse for more information.

  2. With bootable media of the same version and level as the system, boot the system into Service mode. The bootable media can be any ONE of the following:

    Follow the screen prompts to the Welcome to Base OS menu.

  3. Choose Start Maintenance Mode for System Recovery (Option 3). The next screen displays prompts for the Maintenance menu.

    If you receive errors from the preceding option, do not continue with the rest of this procedure. Correct the problem causing the error. If you need assistance correcting the problem causing the error, contact one of the following:

  4. Run the following commands to check and repair file systems.
       fsck /dev/hd4 
       fsck /dev/hd2 
       fsck /dev/hd9var 
       fsck /dev/hd3
       fsck /dev/hd1 
    

    NOTE: The -y option gives the fsck command permission to repair file system corruption when necessary. This flag can be used to avoid having to manually answer multiple confirmation prompts, however, use of this flag can cause permanent data loss in some situations.

    If any of the following conditions occur, proceed accordingly.

  5. The easiest way to fix an unrecoverable file system is to recreate it. This involves deleting it from the system and restoring it from a very current system backup. Note that hd2, hd9var and hd3 can be recreated but hd4 cannot be recreated. If hd4 is unrecoverable, AIX must be reinstalled. For assistance with unrecoverable file systems, contact your local branch office, point of sale, or AIX support center. Do not follow the rest of the steps in this document.

  6. Attempt to repair the file system with this command:
       fsck -p /dev/[hd#]
    

    Replace [hd#] with the appropriate file system.

    Now skip to step 8.

  7. A corruption of the JFS log logical volume has been detected. Use the logform command to reformat it.
       /usr/sbin/logform /dev/hd8
    

    Answer yes when asked if you want to destroy the log.

  8. Repeat step 4 for all file systems that did not successfully complete fsck the first time. If step 4 fails a second time, the file system is almost always unrecoverable. See step 5 for an explanation of the options at this point. In most cases, step 4 will be successful. If step 4 is successful, continue to step 9.

  9. With the key in Normal position, run the following commands to reboot the system:
       exit
       sync;sync;sync;reboot
    

    As you reboot in Normal mode, notice how many times LED 551 appears. If LED 551 appears twice, fsck is probably failing because of a bad fshelper file. If this is the case and you are running AFS, see step 12.

The majority of instances of LED 552, 554, and 556 will be resolved at this point. If you still have an LED 552, 554, or 556, you may try the following steps.

NOTE: The following procedure will overwrite your Object Data Manager (ODM) database files with a very primitive, minimal ODM database. Due to the potential loss of user configuration data caused by this procedure, it should only be used as a last resort effort to gain access to your system to attempt to back up any data that you can. It is NOT recommended to use the following procedure in lieu of restoring from a system backup.

  1. Repeat step 1 through step 3.

  2. Run the following commands, which remove much of the system's configuration and save it to a backup directory.
       mount /dev/hd4 /mnt
       mount /dev/hd2 /mnt/usr
       mkdir /mnt/etc/objrepos/bak
       cp /mnt/etc/objrepos/Cu* /mnt/etc/objrepos/bak
       cp /etc/objrepos/Cu* /mnt/etc/objrepos
       umount /dev/hd2
       umount /dev/hd4
       exit
    

    Determine which disk is the boot disk with the lslv command. The boot disk will be shown in the PV1 column of the lslv output.

       lslv -m hd5
    

    Save the clean ODM database to the boot logical volume. (# is the number of the fixed disk, determined with the previous command.)

       savebase -d /dev/[hdisk#]
    

    If you are running AFS, go to step 12; otherwise, go to step 13.

  3. If you are running the Andrew File System (AFS), use the following commands to find out whether you have more than one version of the v3fshelper file.
       cd /sbin/helpers
       ls -l v3fshelper*
    

    If you have only one version of the v3fshelper file (for example, v3fshelper), proceed to step 13.

    If there is a version of v3fshelper marked as original (for example, v3fshelper.orig), run the following commands:

       cp v3fshelper v3fshelper.afs
       cp v3fshelper.orig v3fshelper
    
  4. WARNING: Do not proceed further if the system is a /usr client, diskless client, or dataless client.

    Recreate the boot image (hdisk# is the fixed disk determined in step 11):

       bosboot -a -d /dev/[hdisk#]
    
  5. If you copied files in step 12, copy the AFS file system helper back to v3fshelper:
       cp v3fshelper.afs v3fshelper
    
  6. Turn the key to the Normal position and run
       shutdown -Fr
    

If after following all of the preceding steps, the system still stops at an LED 552, 554, or 556 during a reboot in Normal mode, reinstalling AIX from a recent system backup is recommended. The time and effort involved attempting to isolate and correct the cause of the failure would generally be extensive at this point. This may not be cost-effective in your operating environment.

If you wish to pursue further system recovery, you may be able to obtain assistence from one of the following:




[ Doc Ref: 90605189214746     Publish Date: Dec. 22, 2000     4FAX Ref: 4188 ]