Search Ali's Blog

Friday, July 30, 2010

objdump parser

If you've been programming in Linux kernel, you must had the chance to debug kernel oops using objdump.
Normally, you invoke objdump with the name of the problematic driver, and then parse the output based on the information printed in the call trace of the kernel oops. Here is a nice example.

Among many hexadecimal numbers and function names that are printed in the call trace of the kernel oops, the most important piece of information is the function name and the offset: for example:

I wrote this script that receives the function name and offset as a pointer, and prints the source file name and line number that caused the oops.

For example:
# vnic_login_create_2+0xd1 drivers/net/mlx4_vnic/mlx4_vnic.ko

vnic_login_create_2+0xd1 = 0x6e3 => drivers/net/mlx4_vnic/vnic_data_main.c:210

Wednesday, July 21, 2010

"always on top" option for Windows

Always On Top Maker is a tiny interface less tool to make any window "always on top", or, make a topmost windows not always on top. How to use: Run AlwaysOnTopMaker.exe, then you can use 2 hot-keys, one is CTRL-ALT-T, is to switch the foreground application "always on top"/"not always on top", the other hotkey is CTRL-ALT-Q, is to exit Always On Top Maker.


Saturday, July 10, 2010

netconsole for redhat

Net-Console module logs kernel printk messages over UDP allowing debugging of problem where disk logging fails and serial consoles are impractical. It can be used either built-in or as a module. As a built-in, netconsole initializes immediately after NIC cards and will bring up the specified interface as soon as possible. While this doesn't allow capture of early kernel panics, it does capture most of the boot process. it's very useful for kernel developers to dump kernel oops callstack (source).

The configuration in Red Hat is very simple:

On the monitor machine:
1. Edit /etc/sysconfig/syslog and include the line SYSLOGD_OPTIONS="-m 0 -r -x"
2. Run dmesg -n 8
3. Restart syslog service: /etc/init.d/syslog restart

On the development machine (that may have kernel oops):

1. Edit /etc/sysconfig/netconsole and include the line SYSLOGADDR=(monitor-machine-IP)
2. Restart netconsole service: /etc/init.d/netconsole restart

Now you can work on the development machine and watch its kernel messages on the monitor machine, I watch the development machine by running this command on the monitoring machine:

#tail -f /var/log/messages | grep (devel-machine-ip)

Below some useful bash scripts for your convenience:

If you're using remote console switch, you can check the console output by invoking the command:
# echo `date`: Hello from `whoami` > /dev/ttyS0

To test netconsole, you just need to trigger any kernel module that prints messages into /var/log/messages, for example:
# modprobe -qr tun; modprobe tun

If your running in Dom0 of Xen/KVM, the netconsole script get confused by the interface bridging, to fix this, you need to update /etc/init.d/netconsole:
First, find the corresponding peth interface of the netdev interface you want to use  for netconsole, normally if you're using eth0, then the corresponding inetrafce in Xen is peth0, then find the MAC address of the remote logger IP, for example if the remote sys logger is, find its MAC by running:
# ping -c 1 ; arp -n | grep
And finally, include the following two lines in /etc/init.d/netconsole:
 DEV=peth0 #your peth name here
 SYSLOGMACADDR=00:30:48:7D:DE:E4 #your MAC here
At this point you can use the scripts above to enable netconsole.

Thursday, July 08, 2010

SMS notification for Linux commands

Sometimes you need to run a Linux command that may take time (such as kernel compilation, virus scan, long test run..) instead of waiting in front of the monitor, you can do other stuff (e.g., good coffee) and you'll be notified when the command had finished with the results.

I have ATT cell phone, they support SMS2EMAIL service; simply send a message (plain text, no longer than 160 characters) to the address:
For example:

The Email address varies based on your carrier, check this website for worldwide cellular network list.

Now, we just need to run the command that takes time, and send a short text message with the report to notify us.. from a Linux machine with Email access, run this command:
(msg-body) | mail -s (msg-subject) (sms2email-address)

The message body can be the command output, or the command returned code, or anything else you want.

a) To report the returned code of an application, run:
gcc windows.c; echo "rc=$?" | mail -s "gcc report"
This will send you the returned code (rc) of the compilation command when its done, usually zero (rc=0) indicates a successful run.

b) To compile a Linux kernel and send an SMS when done with the output tail (last 120 bytes for example), run:
make 2>&1 | tail -c120 | mail -s "Kernel Compilation Status"

- You may need to adjust 'tail -c' flag to fit into 160 characters, see 'man tail' for more info.
- To redirect command error messages, use the suitable bash redirection methods.
- ATT sucks.

Wednesday, July 07, 2010

Fix Slow SSH Login Time

If your login times are really high, it may be that reverse DNS is not working correctly.

(a) Disable DNS in SSH daemon: go to /etc/ssh/sshd_config and include the line UseDNS no, then restart ssh service: /etc/init.d/sshd restart
(b) Or, fix your DNS configuration: in redhat, go to setup -> Network Configuration -> Edit DNS configuration -> and set the primary DNS and the search domain