I use strace all the time doing ops at Crittercism. Some of the random things it's helped with/taught me:
- allowed exploring forking behavior of daemons, in particular the nitty-gritty of gunicorn's prefork behavior, and understanding the rationale behind single- and double-fork daemons generally (very important to understand for job control e.g. writing upstart/init.d jobs)
- isolated hot reads to memcache in situ, by identifying the socket associated with the memcache connection, and finding which key was read the most by a process (we built better logging after the fact, but sometimes there's no substitute for instrumenting prod during tough perf/stress problems)
- let me explore the behavior of node.js's several threads, and find one of them sending "X" over a socket to the other (still not quite sure what this is, some kind of heartbeat/clock tick?)
- helped understanding "primordial processes" and the exact details of how forking/reparenting work on linux
It's a great tool and one that every ops/infrastructure engineer should be familiar with.
let me explore the behavior of node.js's several threads, and find one of them sending "X" over a socket to the other (still not quite sure what this is, some kind of heartbeat/clock tick?)
I don't know about node.js specifically, but this is a common pattern to wake another thread that uses a select()-style event loop.
- allowed exploring forking behavior of daemons, in particular the nitty-gritty of gunicorn's prefork behavior, and understanding the rationale behind single- and double-fork daemons generally (very important to understand for job control e.g. writing upstart/init.d jobs)
- isolated hot reads to memcache in situ, by identifying the socket associated with the memcache connection, and finding which key was read the most by a process (we built better logging after the fact, but sometimes there's no substitute for instrumenting prod during tough perf/stress problems)
- let me explore the behavior of node.js's several threads, and find one of them sending "X" over a socket to the other (still not quite sure what this is, some kind of heartbeat/clock tick?)
- helped understanding "primordial processes" and the exact details of how forking/reparenting work on linux
It's a great tool and one that every ops/infrastructure engineer should be familiar with.