Demonstrations of alimutexsnoop.

This profile tool monitors the mutex that is held by the process longer than
a specified time threshold, such as 100us. Once alimutexsnoop finds such mutexes,
it prints the pid of the process that held the mutex and the address of the
mutex. In addition, alimutexsnoop also prints the user and kernel stack of the
process.


Usage:
alimutexsnoop.py [-h] [-t THRESHOLD] [-p PID]

Trace all mutexes that have been held for a long time

optional arguments:
  -h, --help            show this help message and exit
  -t THRESHOLD, --threshold THRESHOLD
                        The mutexes held longer than this time threshold will
                        be traced(us)
  -p PID, --pid PID     trace this PID only

examples:
    alimutexsnoop           # trace all mutexes that are held for a long time
    alimutexsnoop -t 50     # set the time threshold (us), mutexes are held longer
                              than this time threshold will be printed
    alimutexsnoop -p 123    # only trace the specified process


Output example:

python alimutexsnoop.py -t 20 # monitor the mutexes that are held by processes longer than 20 us

PCOMM            PID     MUTEX_ADDR

AliYunDun        4283    ffffa30fd1b4ea48
Kernel Stack:    ffffffffa5800088  entry_SYSCALL_64_after_hwframe
Kernel Stack:    ffffffffa500201b  do_syscall_64
Kernel Stack:    ffffffffa52316fc  ksys_read
Kernel Stack:    ffffffffa57cfd21  mutex_unlock
USER Stack:      7f1532c4e6fd      read
=================================

AliYunDun        4283    ffffa310d7e43c40
Kernel Stack:    ffffffffa5800088  entry_SYSCALL_64_after_hwframe
Kernel Stack:    ffffffffa500201b  do_syscall_64
Kernel Stack:    ffffffffa52316ba  ksys_read
Kernel Stack:    ffffffffa5231149  vfs_read
Kernel Stack:    ffffffffa5230f86  __vfs_read
Kernel Stack:    ffffffffa5254b30  seq_read
Kernel Stack:    ffffffffa57cfd21  mutex_unlock
USER Stack:      7f1532c4e6fd      read
=================================

AliYunDun        4283    ffffa30fd1b4e348
Kernel Stack:    ffffffffa5800088  entry_SYSCALL_64_after_hwframe
Kernel Stack:    ffffffffa500201b  do_syscall_64
Kernel Stack:    ffffffffa52316fc  ksys_read
Kernel Stack:    ffffffffa57cfd21  mutex_unlock
USER Stack:      7f1532c4e6fd      read
=================================

AliYunDunUpdate  4256    ffffa310d1146a40
Kernel Stack:    ffffffffa5800088  entry_SYSCALL_64_after_hwframe
Kernel Stack:    ffffffffa500201b  do_syscall_64
Kernel Stack:    ffffffffa52316ba  ksys_read
Kernel Stack:    ffffffffa5231149  vfs_read
Kernel Stack:    ffffffffa5230f86  __vfs_read
Kernel Stack:    ffffffffa5254b30  seq_read
Kernel Stack:    ffffffffa57cfd21  mutex_unlock
USER Stack:      7fbce69966fd      read
=================================

AliYunDunUpdate  4256    ffffa310d3b48a48
Kernel Stack:    ffffffffa5800088  entry_SYSCALL_64_after_hwframe
Kernel Stack:    ffffffffa500201b  do_syscall_64
Kernel Stack:    ffffffffa52316fc  ksys_read
Kernel Stack:    ffffffffa57cfd21  mutex_unlock
USER Stack:      7fbce69966fd      read
=================================
