User guide

C-117
Cisco Media Gateway Manager 5.0 User Guide
OL-5461-02
Appendix C Troubleshooting
Miscellaneous Problems
This section will give a list of steps to try to determine more precisely the cause of the inconsistency.
These steps should be performed in order.
1. Verify Node's Sync States
2. Isolate via Database Query
3. Isolate via Message Flow
Step 1 One common reason for inconsistency is nodes in a non-ok state after startup. This will result in
connection counts being incorrect or the presence of incomplete connections.
a. Determine if sync-up has been completed.
b. If system has declared sync-up, verify that the nodes which you see inconsistency on are OK (in
mode = 3). This can be done using the Cisco MGM CLI command "selnd" The second to last column
identifies the mode for a given node.
Step 2 The information displayed on the connection manager GUI is taken directly from the user_connection
table. The connection manager is rarely the problem. This section uses database access queries to help
isolate the problem. If the problem is found in the user_connection table and the segment tables are
incorrect, the problem is usually in EM. If the user_connection table is wrong but the segment tables are
correct, we will need to continue with the investigation before we can determine a root cause.
Note All ports in the database and the messages are 0 based and all ports on the switch cli are 1 based.
So if the switch said port 3, then you should query for port 2.
a. Query the user_connection table for one of the connections in question. For example, if the
connection has a local endpoint of node=n, slot=s, logical_port= p, vpi/dlci = s1, vci = s2 then the
query would look like this: "SELECT * from user_connection where l_node_id = n, l_slot = s,
l_logical_port = p, l_subchnl_1 = s1, l_subchnl_2 = s2" If you are unsure which side is local and
which is remote, try a query with both l_node_id = n and r_node_id = n (and slot, port etc.). If this
row in the database is inconsistent with the switch, more investigation is required.
b. Verify that the databroker has received all traps from the EMs. To do this, view the columns
inseg_tbl_2, inseg_tbl_2 and inseg_tbl_3. If the flag is set to '1' then all traps for the given segment
have been received. These flags tell if the databroker received an add message for the given
segment. If they are set to 0 it means the segment trap has not been processed by databroker. If the
flag is set to 2 or 3, that means a only one end of the segment has been processed. (note that if the
connection doesn't have 3 segments the non-used segments will be defaulted to '1')
c. We should now check the EM connection tables for the connections in question. <see EM section>
d. If we find that the segment tables are correct and the user_connection table is not, or if the
inseg_tbl_x flags are not all = '1', we should continue with the testing from the next section. If we
see that the segment tables are also incorrect, we should begin looking at the EM for debugging
information.
Step 3 In some cases a more detailed review of the message flow between EM and DMD and sDbroker is
necessary to determine the source of the error:
a. Verify that DMD received the message from EM. Search message log for the connection. This
search is on the DMD first, followed by the sdbroker and xdbroker (xpvc only). The search is ether
by connection object ID or node/slot/port/vpi/vci The DMD logs are checked, and if no messages
are found then the OOEMC or other EM processes is checked. If the message is found, or the logs
are incomplete, then we need to look deeper into the Databroker processes. If the DMD or the EM
logs are complete and the message wasn't sent to DMD, then it is most likely an EM issue.