Troubleshooting for UNIX

How to:

Reference:

To troubleshoot an installation problem, identify your problem in the following list, and follow the link to a description of the solution.

If you cannot find your problem described in the list, and cannot resolve it yourself, contact Customer Support Services as described in Information You Should Have and Customer Support.

If you have a troubleshooting suggestion that is not described in the list, and you think others will find it helpful, we invite you to send it to us, as described in How to Add Your Problem to the Troubleshooting Guide. We will consider including your problem in a future release of this manual.

Problems:


Top of page

x
Reference: Problem: The Server Starts in Safe Mode

Problem: The server starts in safe mode. The Web Console home page displays a message stating that the server is in safe mode and describing what triggered it.

Cause: A common cause for the server starting in safe mode is a problem with the server administrator ID password. For example, the password may have been updated on the operating system but not on the server, so the encrypted copy of the password stored by the server is out of synchronization with the password on the operating system.

Solution: The server administrator can click the fix hyperlink, which is displayed under the problem description, to display the relevant pane and resolve the problem.

For example, if the problem is that the server administrator password is out of synchronization:

  1. Click the fix hyperlink displayed under the problem description.
  2. In the left pane, open the Users folder, then the Server Administrator folder.
  3. Click your user ID and select Properties from the pop-up menu.

    The Access Control pane is displayed on the right.

  4. Type the correct operating system password in the Password field, and type it again in the Confirm Password field.
  5. Click Save and Restart.

    The Security Mode pane opens on the right.

  6. Click the Home icon in the menu bar to return to the Web Console home page.

Top of page

x
Reference: Problem: Java Listener Fails to Start With JVM not found Messages Written to the Log

Problem: The listener start request fails with JVM not found messages written to the edaprint.log file.

Cause: If the server cannot find the Java Virtual Machine (JVM), the JSCOM Listener will not be able to start, and messages will be written to the server log stating that the JVM cannot be found.

The server log may be found in the following location:

On Windows:

drive:\ibi\srv77\server_type\edaprint.log

On UNIX:

ibi/srv77/server_type/edaprint.log

On IBM i:

/home/iadmin/ibi/srv77/server_type/edaprint.log

where:

drive

Is the hard drive on which the directory resides (on Windows).

server_type

Designates the type of server. The default values are:

FFS

for a Full-Function Server

DM

for a DataMigrator Server

WFS

for a WebFOCUS Reporting Server

WFM

for a Shared Application Server for WebFOCUS Maintain

Solution: Set up the JVM as described in JVM Requirements for Java Services.


Top of page

x
Reference: Problem: Setting ulimit to Allow Core Dumps

The ulimit value of a process controls how large (in blocks) a core can grow. If the value is set to zero, no dump is produced, and the dump information is not read.

To check the current value, issue:

ulimit -c 

To set a ulimit so that dump information can be produced, stop the server, set a value, and restart:

bin/edastart -stop
ulimit -c 99999
bin/edastart -start 

The actual size value is in blocks and will vary by need, Since the need is unpredictable, select a number and then check the dump information. If the information is incomplete, increase the value.


Top of page

x
Reference: Problem: Forcing Core Dump Information on Solaris

Solaris uses the coreadm command to control the ability to produce core files.

To see the current value, issue:

coreadm

For secured servers, before the server starts, issue:

coreadm -e proc-setid

No reboot of OS or service daemons is required, but core files must have a non-zero ulimit –c value.


Top of page

x
Reference: Problem: Forcing Core Dump Information on HP-UX

As of HP-UX 11.31, the coreadm command controls the ability to produce core files. Sites with an operating system prior to HP-UX 11.31 can only use the unsecure server method to produce a complete save diagnostic for a crash.

To see the current settings or values for machines that support the coreadm command, issue:

coreadm

For secured server purposes (on HP-UX 11.31 or higher), before the server starts, issue:

coreadm -e proc-setid

No reboot of OS or service daemons is required, but core files must have a non-zero ulimit –c value.


Top of page

x
Reference: Problem: Forcing Core Dump Information on AIX

AIX uses the chdev command to control the ability to produce core files. This command is on by default, so it only needs to be adjusted if it has been turned off.

To see the current value, issue:

lsattr -El sys0 -a fullcore

For secured server purposes, before the server starts, issue:

chdev -l sys0 -a fullcore=true

Top of page

x
Reference: Problem: Forcing Core Dump Information on Linux

While Linux has options to activate core dumps, none currently work in the context of the server. Linux sites can only use the unsecure server method to produce a complete save diagnostic for a crash.


Top of page

x
Reference: Problem: Process Core Dumps, Core Produced, No Span, and Debugger Missing Error

When a process crashes, the operating system generally produces a core dump. The server software is designed to detect this event and use the system debugger to extract the state of the crashed process from the core and produce what is known as a snap. While the specific debugger command may vary by vendor, the standard debugger for the vendor must be installed or a Not Found condition on running the debugger can occur (effectively a core, but no snap information). Some vendors install the debugger in /bin or /usr/bin, which are normally on $PATH, but some vendors use locations not normally on $PATH. This can result in a secondary reason for a debugger Not Found condition. Once the debugger is installed and/or on the $PATH, reproducing the crash condition will then produce the snap information.


Top of page

x
Reference: Problem: Server Fails to Start With Cannot Create Shared Memory Message Written to edaprint

The full message indicates the need to review edapth traces for r1shmop* entries with errors. If the server was not started with traces, start it with traces, and then view the edapth trace.

One of the r1shmop* entries in the edapth trace will show a specific error, but a common error is size is greater than system shared memory limit. This particular message indicates that the system kernel value for shared memory needs to be increased. The actual required value is generally a multiple of machine page size (typically 4K, but it can vary). The number of agents a server runs, and other installed software can also be a factor, and the required value may vary (slightly) from release to release.

There are tools, such as size and ps that will allow an experienced administrator to narrow down the precise shared memory size requirements, considering all of the software in use. However, a good rule of thumb is to increase memory in 10% increments until a working value is found.

Error messages other than shared memory size can occur, in which case, the system message is displayed. These other messages may provide an administrator with enough information to determine the appropriate action. If not, call Customer Support Services for a review. Actual kernel change commands or steps vary by vendor, so they are not explicitly outlined here.


Top of page

x
Reference: Problem: Secured Server Starts Unsecured or Does not Start after Upgrade

A server will implicitly attempt to start unsecured if proper authorization steps have not been completed. Starting the server normally clears edatemp. If prior edatemp files exist (and authorization has not been done), start up will fail due to an inability to clear the directory. However, if an edastart -cleardir command was issued just before the upgrade, there is nothing to clear, no error occurs, and the server starts. If the server starts and is not inspected after the initial start up, the server being in the wrong mode may go unnoticed.

The proper solution is to add proper authorizations after an upgrade, as described in How to Satisfy Security Provider OPSYS Requirements, and restart the server. A new safety measure has also been added. If the environment variable EDAEXTSEC is set to OPSYS explicitly, and a server lacks authorization, it will not start (see Preventing Unsecured Server Starts After Upgrades for details).


Top of page

x
Procedure: How to Add Your Problem to the Troubleshooting Guide

If you have troubleshooting suggestions that you think others will find helpful, we invite you to send them to us so that we can consider including them in a future release. You can:


Top of page

x
Procedure: How to Install and Activate the Debuggable Version of the Server

In core dump (crash) situations, the stack information may or may not provide enough information for a problem to be resolved. Debuggable versions of the software will generally provide that information, but would not normally be installed nor used due to the disk overhead they take and they are not optimized for performance.

If a diagnostic is determined to not have enough information and use of debuggables is warranted, Customer Support Services will inform you to install and activate the debuggable version of the server and re-run the reproduction to capture a new diagnostic with the detailed stack to help troubleshoot the problem.

Caution: Do not activate the debuggable version unless explicitly requested to by Customer Support Services.

To activate the debuggable version of the server:

  1. Log on with the server administrator ID (often referred to as iadmin).
  2. Download the iserverd archive file (for example, .tar, .zip, or .bck) from the FTP site to a local directory. Debuggables for UNIX environments are not normally shipped on the original CD media, but can be made available on CD by special request to Customer Support Services and requires a lead time of approximately one week. If CD media is being used, mount the media.
  3. Run the isetup installation program located in the EDAHOME bin (if download was used) or in the root directory of the CD media.
  4. At the main menu, select option 4, Install Debuggables to the Installation Directory and follow its steps supplying information similar to when the original install was performed.
  5. After completion of isetup the server may be run in debug mode with the following steps.
    edastart -stop
    edastart -dbgon
    edastart -start (run until repro is completed)
    edastart -stop
    edastart -dbgoff
    edastart -start
  6. If the debug version is no longer needed, the debuggables may be removed. If a service pack is being installed, the debuggables must be removed to prevent mismatches with the new release. To remove the debuggables, change the directory to the home directory of EDAHOME and issue rm -f dbg.

Customer Support Services will provide you with additional instructions as your situation requires.


iWay Software