Commit Graph

32 Commits

Author SHA1 Message Date
Changbin Du
129f70bd60 perf: ftrace: add graph tracer options args/retval/retval-hex/retaddr
This change adds support for new funcgraph tracer options funcgraph-args,
funcgraph-retval, funcgraph-retval-hex and funcgraph-retaddr.

The new added options are:
  - args       : Show function arguments.
  - retval     : Show function return value.
  - retval-hex : Show function return value in hexadecimal format.
  - retaddr    : Show function return address.

 # ./perf ftrace -G vfs_write --graph-opts retval,retaddr
 # tracer: function_graph
 #
 # CPU  DURATION                  FUNCTION CALLS
 # |     |   |                     |   |   |   |
 5)               |  mutex_unlock() { /* <-rb_simple_write+0xda/0x150 */
 5)   0.188 us    |    local_clock(); /* <-lock_release+0x2ad/0x440 ret=0x3bf2a3cf90e */
 5)               |    rt_mutex_slowunlock() { /* <-rb_simple_write+0xda/0x150 */
 5)               |      _raw_spin_lock_irqsave() { /* <-rt_mutex_slowunlock+0x4f/0x200 */
 5)   0.123 us    |        preempt_count_add(); /* <-_raw_spin_lock_irqsave+0x23/0x90 ret=0x0 */
 5)   0.128 us    |        local_clock(); /* <-__lock_acquire.isra.0+0x17a/0x740 ret=0x3bf2a3cfc8b */
 5)   0.086 us    |        do_raw_spin_trylock(); /* <-_raw_spin_lock_irqsave+0x4a/0x90 ret=0x1 */
 5)   0.845 us    |      } /* _raw_spin_lock_irqsave ret=0x292 */
 5)               |      _raw_spin_unlock_irqrestore() { /* <-rt_mutex_slowunlock+0x191/0x200 */
 5)   0.097 us    |        local_clock(); /* <-lock_release+0x2ad/0x440 ret=0x3bf2a3cff1f */
 5)   0.086 us    |        do_raw_spin_unlock(); /* <-_raw_spin_unlock_irqrestore+0x23/0x60 ret=0x1 */
 5)   0.104 us    |        preempt_count_sub(); /* <-_raw_spin_unlock_irqrestore+0x35/0x60 ret=0x0 */
 5)   0.726 us    |      } /* _raw_spin_unlock_irqrestore ret=0x80000000 */
 5)   1.881 us    |    } /* rt_mutex_slowunlock ret=0x0 */
 5)   2.931 us    |  } /* mutex_unlock ret=0x0 */

Signed-off-by: Changbin Du <changbin.du@huawei.com>
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250613114048.132336-1-changbin.du@huawei.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
2025-07-22 17:47:22 -07:00
Namhyung Kim
8db1d77248 perf ftrace latency: Add -e option to measure time between two events
In addition to the function latency, it can measure events latencies.
Some kernel tracepoints are paired and it's menningful to measure how
long it takes between the two events.  The latency is tracked for the
same thread.

Currently it only uses BPF to do the work but it can be lifted later.
Instead of having separate a BPF program for each tracepoint, it only
uses generic 'event_begin' and 'event_end' programs to attach to any
(raw) tracepoints.

  $ sudo perf ftrace latency -a -b --hide-empty \
    -e i915_request_wait_begin,i915_request_wait_end -- sleep 1
  #   DURATION     |      COUNT | GRAPH                                |
     256 -  512 us |          4 | ######                               |
       2 -    4 ms |          2 | ###                                  |
       4 -    8 ms |         12 | ###################                  |
       8 -   16 ms |         10 | ################                     |

  # statistics  (in usec)
    total time:               194915
      avg time:                 6961
      max time:                12855
      min time:                  373
         count:                   28

Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250714052143.342851-1-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
2025-07-14 22:51:58 -07:00
Namhyung Kim
e5f2024cb9 perf ftrace profile: Add --graph-opts option
Like trace subcommand, it should be able to pass some options to control
the tracing behavior for the function graph tracer.

But some options are limited in order to maintain the internal behavior.

For example, it can limit the function call depth like below:

  # perf ftrace profile --graph-opts depth=5 -- myprog

Committer testing:

  root@number:~# perf ftrace profile --graph-opts thresh=1000 -- sleep 1
  # Total (us)   Avg (us)   Max (us)      Count   Function
   1001419.301 500709.650 1000032.000          2   x64_sys_call
   1000032.000 1000032.000 1000032.000          1   __x64_sys_clock_nanosleep
   1000032.000 1000032.000 1000032.000          1   common_nsleep
   1000031.000 1000031.000 1000031.000          1   do_nanosleep
   1000031.000 1000031.000 1000031.000          1   hrtimer_nanosleep
   1000024.000 1000024.000 1000024.000          1   schedule
      1387.208   1387.208   1387.208          1   __x64_sys_execve
      1386.691   1386.691   1386.691          1   do_execveat_common.isra.0
      1334.170   1334.170   1334.170          1   bprm_execve
      1258.413   1258.413   1258.413          1   load_elf_binary
      1123.068   1123.068   1123.068          1   begin_new_exec
      1113.550   1113.550   1113.550          1   mmput
      1109.237   1109.237   1109.237          1   exit_mmap
  root@number:~# perf ftrace profile --graph-opts thresh=1200 -- sleep 1
  # Total (us)   Avg (us)   Max (us)      Count   Function
   1001448.204 500724.102 1000018.000          2   x64_sys_call
   1000017.000 1000017.000 1000017.000          1   __x64_sys_clock_nanosleep
   1000017.000 1000017.000 1000017.000          1   common_nsleep
   1000017.000 1000017.000 1000017.000          1   hrtimer_nanosleep
   1000016.000 1000016.000 1000016.000          1   do_nanosleep
   1000012.000 1000012.000 1000012.000          1   schedule
      1430.112   1430.112   1430.112          1   __x64_sys_execve
      1429.581   1429.581   1429.581          1   do_execveat_common.isra.0
      1376.289   1376.289   1376.289          1   bprm_execve
      1301.743   1301.743   1301.743          1   load_elf_binary
  root@number:~#

Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20250107224352.1128669-2-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2025-01-08 17:20:42 -03:00
Gabriele Monaco
690a052a6d perf ftrace latency: Add --max-latency option
This patch adds a max-latency option as discussed, in case the number of
buckets is more than 22, we don't observe the setting (for now, let's
say).

By default or if 0 is passed, the value is automatically determined
based on the number of buckets, range and minimum, so that we fill all
available buffers (equivalent to the behaviour before this patch).

We now get something like this:

  # perf ftrace latency --bucket-range=20 \
			--min-latency 10 \
			--max-latency=100 \
			-T switch_mm_irqs_off -a sleep 2
  #   DURATION     |      COUNT | GRAPH             |
       0 -   10 us |       1731 | ################  |
      10 -   30 us |          1 |                   |
      30 -   50 us |          0 |                   |
      50 -   70 us |          0 |                   |
      70 -   90 us |          0 |                   |
      90 -  100 us |          0 |                   |
     100 -  ... us |          0 |                   |

Note the maximum is observed also if it doesn't cover completely a full
range (the second to last range is 10us long to let the last start at
100 sharp), this looks to me more sensible and eases the computations,
since we don't need to account for the range while filling the buckets.

Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Clark Williams <williams@redhat.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20241112181214.1171244-5-acme@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2024-12-10 15:16:40 -03:00
Arnaldo Carvalho de Melo
08b875b6bf perf ftrace latency: Introduce --min-latency to narrow down into a latency range
Things below and over will be in the first and last, outlier, buckets.

Without it:

  # perf ftrace latency --use-nsec --use-bpf \
			--bucket-range=200 \
			-T switch_mm_irqs_off -a sleep 2
  #   DURATION     |      COUNT | GRAPH                                   |
       0 -  200 ns |          0 |                                         |
     200 -  400 ns |         44 |                                         |
     400 -  600 ns |        291 | #                                       |
     600 -  800 ns |        506 | ##                                      |
     800 - 1000 ns |        148 |                                         |
    1.00 - 1.20 us |        581 | ##                                      |
    1.20 - 1.40 us |       2199 | ##########                              |
    1.40 - 1.60 us |       1048 | ####                                    |
    1.60 - 1.80 us |       1448 | ######                                  |
    1.80 - 2.00 us |       1091 | #####                                   |
    2.00 - 2.20 us |        517 | ##                                      |
    2.20 - 2.40 us |        318 | #                                       |
    2.40 - 2.60 us |        370 | #                                       |
    2.60 - 2.80 us |        271 | #                                       |
    2.80 - 3.00 us |        150 |                                         |
    3.00 - 3.20 us |         85 |                                         |
    3.20 - 3.40 us |         48 |                                         |
    3.40 - 3.60 us |         40 |                                         |
    3.60 - 3.80 us |         22 |                                         |
    3.80 - 4.00 us |         13 |                                         |
    4.00 - 4.20 us |         14 |                                         |
    4.20 - ...  us |        626 | ##                                      |
  #
  # perf ftrace latency --use-nsec --use-bpf \
			--bucket-range=20 --min-latency=1200 \
			-T switch_mm_irqs_off -a sleep 2
  #   DURATION     |      COUNT | GRAPH                                   |
       0 - 1200 ns |       1243 | #####                                   |
    1.20 - 1.22 us |        141 |                                         |
    1.22 - 1.24 us |        202 |                                         |
    1.24 - 1.26 us |        209 |                                         |
    1.26 - 1.28 us |        219 |                                         |
    1.28 - 1.30 us |        208 |                                         |
    1.30 - 1.32 us |        245 | #                                       |
    1.32 - 1.34 us |        246 | #                                       |
    1.34 - 1.36 us |        224 | #                                       |
    1.36 - 1.38 us |        219 |                                         |
    1.38 - 1.40 us |        206 |                                         |
    1.40 - 1.42 us |        190 |                                         |
    1.42 - 1.44 us |        190 |                                         |
    1.44 - 1.46 us |        146 |                                         |
    1.46 - 1.48 us |        140 |                                         |
    1.48 - 1.50 us |        125 |                                         |
    1.50 - 1.52 us |        115 |                                         |
    1.52 - 1.54 us |        102 |                                         |
    1.54 - 1.56 us |         87 |                                         |
    1.56 - 1.58 us |         90 |                                         |
    1.58 - 1.60 us |         85 |                                         |
    1.60 - ...  us |       5487 | ########################                |
  #

Now we want focus on the latencies starting at 1.2us, with a finer
grained range of 20ns:

This is all on a live system, so statistically interesting, but not
narrowing down on the same numbers, so a 'perf ftrace latency record'
seems interesting to then use all on the same snapshot of latencies.

A --max-latency counterpart should come next, at first limiting the
max-latency to 20 * bucket-size, as we have a fixed buckets array with
20 + 2 entries (+ for the outliers) and thus would need to make it
larger for higher latencies.

We also may need a way to ask for not considering the out of range
values (first and last buckets) when drawing the buckets bars.

Co-developed-by: Gabriele Monaco <gmonaco@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Clark Williams <williams@redhat.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20241112181214.1171244-4-acme@kernel.org
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2024-12-10 15:16:27 -03:00
Arnaldo Carvalho de Melo
e8536dd47a perf ftrace latency: Introduce --bucket-range to ask for linear bucketing
In addition to showing it exponentially, using log2() to figure out the
histogram index, allow for showing it linearly:

The preexisting more, the default:

  # perf ftrace latency --use-nsec --use-bpf \
  			-T switch_mm_irqs_off -a sleep 2
  #   DURATION     |      COUNT | GRAPH                                   |
       0 -    1 ns |          0 |                                         |
       1 -    2 ns |          0 |                                         |
       2 -    4 ns |          0 |                                         |
       4 -    8 ns |          0 |                                         |
       8 -   16 ns |          0 |                                         |
      16 -   32 ns |          0 |                                         |
      32 -   64 ns |          0 |                                         |
      64 -  128 ns |        238 | #                                       |
     128 -  256 ns |       1704 | ##########                              |
     256 -  512 ns |        672 | ###                                     |
     512 - 1024 ns |       4458 | ##########################              |
       1 -    2 us |        677 | ####                                    |
       2 -    4 us |          5 |                                         |
       4 -    8 us |          0 |                                         |
       8 -   16 us |          0 |                                         |
      16 -   32 us |          0 |                                         |
      32 -   64 us |          0 |                                         |
      64 -  128 us |          0 |                                         |
     128 -  256 us |          0 |                                         |
     256 -  512 us |          0 |                                         |
     512 - 1024 us |          0 |                                         |
       1 - ...  ms |          0 |                                         |
  #

The new histogram mode:

  # perf ftrace latency --bucket-range=150 --use-nsec --use-bpf \
  			-T switch_mm_irqs_off -a sleep 2
  #   DURATION     |      COUNT | GRAPH                                   |
       0 -    1 ns |          0 |                                         |
       1 -  151 ns |        265 | #                                       |
     151 -  301 ns |       1797 | ###########                             |
     301 -  451 ns |        258 | #                                       |
     451 -  601 ns |        289 | #                                       |
     601 -  751 ns |       2049 | #############                           |
     751 -  901 ns |        967 | ######                                  |
     901 - 1051 ns |        513 | ###                                     |
    1.05 - 1.20 us |        114 |                                         |
    1.20 - 1.35 us |        559 | ###                                     |
    1.35 - 1.50 us |        189 | #                                       |
    1.50 - 1.65 us |        137 |                                         |
    1.65 - 1.80 us |         32 |                                         |
    1.80 - 1.95 us |          2 |                                         |
    1.95 - 2.10 us |          0 |                                         |
    2.10 - 2.25 us |          1 |                                         |
    2.25 - 2.40 us |          1 |                                         |
    2.40 - 2.55 us |          0 |                                         |
    2.55 - 2.70 us |          0 |                                         |
    2.70 - 2.85 us |          0 |                                         |
    2.85 - 3.00 us |          1 |                                         |
    3.00 - ...  us |          4 |                                         |
  #

Co-developed-by: Gabriele Monaco <gmonaco@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Clark Williams <williams@redhat.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20241112181214.1171244-3-acme@kernel.org
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2024-12-10 15:16:01 -03:00
Namhyung Kim
74ae366c37 perf ftrace profile: Add -s/--sort option
The -s/--sort option is to sort the output by given column.

  $ sudo perf ftrace profile -s max sync | head
  # Total (us)   Avg (us)   Max (us)      Count   Function
      6301.811   6301.811   6301.811          1   __do_sys_sync
      6301.328   6301.328   6301.328          1   ksys_sync
      5320.300   1773.433   2858.819          3   iterate_supers
      2755.875     17.012   2610.633        162   sync_fs_one_sb
      2728.351    682.088   2610.413          4   ext4_sync_fs [ext4]
      2603.654   2603.654   2603.654          1   jbd2_log_wait_commit [jbd2]
      4750.615    593.827   2597.427          8   schedule
      2164.986     26.728   2115.673         81   sync_inodes_one_sb
      2143.842     26.467   2115.438         81   sync_inodes_sb

Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Changbin Du <changbin.du@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: https://lore.kernel.org/lkml/20240729004127.238611-5-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2024-07-31 16:58:18 -03:00
Namhyung Kim
0f223813ed perf ftrace: Add 'profile' command
The 'perf ftrace profile' command is to get function execution profiles
using function-graph tracer so that users can see the total, average,
max execution time as well as the number of invocations easily.

The following is a profile for the perf_event_open syscall.

  $ sudo perf ftrace profile -G __x64_sys_perf_event_open -- \
    perf stat -e cycles -C1 true 2> /dev/null | head
  # Total (us)   Avg (us)   Max (us)      Count   Function
        65.611     65.611     65.611          1   __x64_sys_perf_event_open
        30.527     30.527     30.527          1   anon_inode_getfile
        30.260     30.260     30.260          1   __anon_inode_getfile
        29.700     29.700     29.700          1   alloc_file_pseudo
        17.578     17.578     17.578          1   d_alloc_pseudo
        17.382     17.382     17.382          1   __d_alloc
        16.738     16.738     16.738          1   kmem_cache_alloc_lru
        15.686     15.686     15.686          1   perf_event_alloc
        14.012      7.006     11.264          2   obj_cgroup_charge
  #

Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Changbin Du <changbin.du@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: https://lore.kernel.org/lkml/20240729004127.238611-4-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2024-07-31 16:58:18 -03:00
Namhyung Kim
c77800894b perf ftrace: Add 'tail' option to --graph-opts
The 'graph-tail' option is to print function name as a comment at the end.
This is useful when a large function is mixed with other functions
(possibly from different CPUs).

For example,

  $ sudo perf ftrace -- perf stat true
  ...
   1)               |    get_unused_fd_flags() {
   1)               |      alloc_fd() {
   1)   0.178 us    |        _raw_spin_lock();
   1)   0.187 us    |        expand_files();
   1)   0.169 us    |        _raw_spin_unlock();
   1)   1.211 us    |      }
   1)   1.503 us    |    }

  $ sudo perf ftrace --graph-opts tail -- perf stat true
  ...
   1)               |    get_unused_fd_flags() {
   1)               |      alloc_fd() {
   1)   0.099 us    |        _raw_spin_lock();
   1)   0.083 us    |        expand_files();
   1)   0.081 us    |        _raw_spin_unlock();
   1)   0.601 us    |      } /* alloc_fd */
   1)   0.751 us    |    } /* get_unused_fd_flags */

Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Changbin Du <changbin.du@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: https://lore.kernel.org/lkml/20240729004127.238611-2-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2024-07-31 16:58:18 -03:00
Changbin Du
a1ef3aaf6a perf docs: Fix format of unordered lists
Fix the format of unordered lists so the can wrap properly.

Signed-off-by: Changbin Du <changbin.du@huawei.com>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20230718085242.3090797-1-changbin.du@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2023-08-16 08:37:49 -03:00
Namhyung Kim
feff08395b perf ftrace latency: Update documentation
Add description of 'perf ftrace latency' subcommand.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Changbin Du <changbin.du@gmail.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: https://lore.kernel.org/r/20220321234609.90455-2-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2022-03-22 17:45:39 -03:00
Ian Rogers
b55ff1d145 perf tools: Fix documentation of verbose options
Option doesn't take a value, make sure the man pages agree. For example:

  $ perf evlist --verbose=1
   Error: option `verbose' takes no value

Signed-off-by: Ian Rogers <irogers@google.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lore.kernel.org/lkml/20210226183145.1878782-1-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-03-06 16:54:26 -03:00
Changbin Du
2ae05fe0a9 perf: ftrace: Add filter support for option -F/--funcs
Same as 'perf probe -F', this patch adds filter support for the ftrace
subcommand option '-F, --funcs <[FILTER]>'.

Here is an example that only lists functions which start with 'vfs_':

  $ sudo perf ftrace -F vfs_*
  vfs_fadvise
  vfs_fallocate
  vfs_truncate
  vfs_open
  vfs_setpos
  vfs_llseek
  vfs_readf
  vfs_writef
  ...

Suggested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Changbin Du <changbin.du@gmail.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lore.kernel.org/lkml/20200904152357.6053-1-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-09-04 16:11:16 -03:00
Changbin Du
42145d71dd perf ftrace: Add option --tid to filter by thread id
This allows us to trace single thread instead of the whole process.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-17-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 09:34:52 -03:00
Changbin Du
6555c2f6db perf ftrace: Add option -D/--delay to delay tracing
This adds an option '-D/--delay' to allow us to start tracing some times
later after workload is launched.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-16-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 09:34:05 -03:00
Changbin Du
a8f87a5cb4 perf: ftrace: Allow set graph depth by '--graph-opts'
This is to have a consistent view of all graph tracer options.
The original option '--graph-depth' is marked as deprecated.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-15-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 09:33:38 -03:00
Changbin Du
00c85d5f45 perf ftrace: Add support for trace option tracing_thresh
This adds an option '--graph-opts thresh' to setup trace duration
threshold for funcgraph tracer.

  $ sudo ./perf ftrace -G '*' --graph-opts thresh=100
   3) ! 184.060 us  |    } /* schedule */
   3) ! 185.600 us  |  } /* exit_to_usermode_loop */
   2) ! 225.989 us  |    } /* schedule_idle */
   2) # 4140.051 us |  } /* do_idle */

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-14-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 09:29:35 -03:00
Changbin Du
59486fb0c8 perf ftrace: Add option 'verbose' to show more info for graph tracer
Sometimes we want ftrace display more and longer information about the
trace.

  $ sudo perf ftrace -G '*'
   2)   0.979 us    |  mutex_unlock();
   2)   1.540 us    |  __fsnotify_parent();
   2)   0.433 us    |  fsnotify();

  $ sudo perf ftrace -G '*' --graph-opts verbose
  14160.770883 |   0)  <...>-47814   |  .... |   1.289 us    |  mutex_unlock();
  14160.770886 |   0)  <...>-47814   |  .... |   1.624 us    |  __fsnotify_parent();
  14160.770887 |   0)  <...>-47814   |  .... |   0.636 us    |  fsnotify();
  14160.770888 |   0)  <...>-47814   |  .... |   0.328 us    |  __sb_end_write();
  14160.770888 |   0)  <...>-47814   |  d... |   0.430 us    |  fpregs_assert_state_consistent();
  14160.770889 |   0)  <...>-47814   |  d... |               |  do_syscall_64() {
  14160.770889 |   0)  <...>-47814   |  .... |               |    __x64_sys_close() {

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-13-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 09:28:17 -03:00
Changbin Du
c81fc34e31 perf ftrace: Add support for tracing option 'irq-info'
This adds support to display irq context info for function tracer. To do
this, just specify a '--func-opts irq-info' option.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-12-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 09:27:43 -03:00
Changbin Du
d1bcf17cda perf ftrace: Add support for trace option funcgraph-irqs
This adds an option '--graph-opts noirqs' to filter out functions executed
in irq context.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-11-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 09:26:56 -03:00
Changbin Du
38988f2e7e perf ftrace: Add support for trace option sleep-time
This adds an option '--graph-opts nosleep-time' which allow us only to
measure on-CPU time. This option is function_graph tracer only.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-10-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 09:25:17 -03:00
Changbin Du
b1d84af6f5 perf ftrace: Add support for tracing option 'func_stack_trace'
This adds support to display call trace for function tracer. To do this,
just specify a '--func-opts call-graph' option.

Example:

  $ sudo perf ftrace -T vfs_read --func-opts call-graph
   iio-sensor-prox-855   [003]   6168.369657: vfs_read <-ksys_read
   iio-sensor-prox-855   [003]   6168.369677: <stack trace>
   => vfs_read
   => ksys_read
   => __x64_sys_read
   => do_syscall_64
   => entry_SYSCALL_64_after_hwframe
   ...

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-9-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 09:16:43 -03:00
Changbin Du
5b34747238 perf ftrace: Add option '--inherit' to trace children processes
This adds an option '--inherit' to allow us trace children
processes spawned by our target.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-7-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 08:59:16 -03:00
Changbin Du
846e193980 perf ftrace: Add option '-m/--buffer-size' to set per-cpu buffer size
This adds an option '-m/--buffer-size' to allow us set the size of per-cpu
tracing buffer.

Committer testing:

Before running with this option:

  # find /sys/kernel/tracing/ -name buffer_size_kb | xargs cat
  1408
  1408
  1408
  1408
  1408
  1408
  1408
  1408
  1408
  #

Then, run:

  # perf ftrace -m 2048K | head -10
   2)               |  mutex_unlock() {
   2)   ==========> |
   2)               |    smp_irq_work_interrupt() {
   2)               |      irq_enter() {
   2)   0.121 us    |        rcu_irq_enter();
   2)   0.128 us    |        irqtime_account_irq();
   2)   0.719 us    |      }
   2)               |      __wake_up() {
   2)               |        __wake_up_common_lock() {
   2)   0.105 us    |          _raw_spin_lock_irqsave();
  #

Now look at those tracefs knobs:

  # find /sys/kernel/tracing/ -name buffer_size_kb | xargs cat
  2048
  2048
  2048
  2048
  2048
  2048
  2048
  2048
  2048
  #

This should be similar to the -m option in the other perf tools, such as
'perf record', 'perf trace', etc.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-5-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 08:37:46 -03:00
Changbin Du
d6d81bfe42 perf ftrace: Add option '-F/--funcs' to list available functions
This adds an option '-F/--funcs' to list all available functions to
trace, which is read from tracing file 'available_filter_functions'.

  $ sudo ./perf ftrace -F | head
  trace_initcall_finish_cb
  initcall_blacklisted
  do_one_initcall
  do_one_initcall
  trace_initcall_start_cb
  run_init_process
  try_to_run_init_process
  match_dev_by_label
  match_dev_by_uuid
  rootfs_init_fs_context
  $

Committer notes:

This is the same command line option and for the same purpose as in
'perf probe'.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-3-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 08:36:38 -03:00
Changbin Du
eb6d31ae22 perf ftrace: Select function/function_graph tracer automatically
The '-g/-G' options have already implied function_graph tracer should be
used instead of function tracer. So we don't need extra option
'--tracer' in this case.

This patch changes the behavior as below:

  - If '-g' or '-G' option is on, then function_graph tracer is used.
  - If '-T' or '-N' option is on, then function tracer is used.
  - The function_graph has priority over function tracer.
  - The option '--tracer' only take effect if neither -g/-G nor -T/-N
    is specified.

Here are some examples.

This will start tracing all functions using default tracer:

  $ sudo perf ftrace

This will trace all functions using function graph tracer:

  $ sudo perf ftrace -G '*'

This will trace function vfs_read using function graph tracer:

  $ sudo perf ftrace -G vfs_read

This will trace function vfs_read using function tracer:

  $ sudo perf ftrace -T vfs_read

Committer notes:

Using '-h -G' will tell what that option is about, so to further clarify
the above examples:

  # perf ftrace -h -G

    -G, --graph-funcs <func>	Set graph filter on given functions

  # perf ftrace -h -g

    -g, --nograph-funcs <func>	Set nograph filter on given functions

  # perf ftrace -h -T

    -T, --trace-funcs <func>	trace given functions only

  # perf ftrace -h -N

    -N, --notrace-funcs <func>	do not trace given functions

  #

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: http://lore.kernel.org/lkml/20200808023141.14227-2-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-08-14 08:36:31 -03:00
Takashi Iwai
ea66536ab2 perf tools: Correct title markers for asciidoctor
I've tested to process the perf man pages with asciidoctor that is
picker than asciidoc, and it revealed minor syntax errors in some
documents.  Namely, the title markers aren't aligned with the previous
line, hence asciidoctor didn't recognize as titles.

This patch corrects these markers to be processed properly.

Signed-off-by: Takashi Iwai <tiwai@suse.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20180307105441.28512-1-tiwai@suse.de
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-03-07 10:26:32 -03:00
Namhyung Kim
1096c35aa8 perf ftrace: Add -D option for depth filter
The -D/--graph-depth option is to set max graph depth.  The following
example traces max 2-depth of page fault handler.

  $ sudo perf ftrace -G __do_page_fault -D 2 -- hello
   ...
   0)               |  __do_page_fault() {
   0)   0.063 us    |    down_read_trylock();
   0)   0.251 us    |    find_vma();
   0)   5.374 us    |    handle_mm_fault();
   0)   0.054 us    |    up_read();
   0)   7.463 us    |  }
   ...

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170618142302.25390-4-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-19 22:05:54 -03:00
Namhyung Kim
78b83e8b12 perf ftrace: Add option for function filtering
The -T/--trace-funcs and -N/--notrace-funcs options are to specify
functions to enable/disable tracing dynamically.

The -G/--graph-funcs and -g/--nograph-funcs options are to set filters
for function graph tracer.

For example, to trace fault handling functions only:

  $ sudo perf ftrace -T *fault hello
   0)               |  __do_page_fault() {
   0)               |    handle_mm_fault() {
   0)   2.117 us    |      __handle_mm_fault();
   0)   3.627 us    |    }
   0)   7.811 us    |  }
   0)               |  __do_page_fault() {
   0)               |    handle_mm_fault() {
   0)   2.014 us    |      __handle_mm_fault();
   0)   2.424 us    |    }
   0)   2.951 us    |  }
   ...

To trace all functions executed in __do_page_fault:

  $ sudo perf ftrace -G __do_page_fault hello
   2)               |  __do_page_fault() {
   3)   0.060 us    |    down_read_trylock();
   3)               |    find_vma() {
   3)   0.075 us    |      vmacache_find();
   3)   0.053 us    |      vmacache_update();
   3)   1.246 us    |    }
   3)               |    handle_mm_fault() {
   3)   0.063 us    |      __rcu_read_lock();
   3)   0.056 us    |      mem_cgroup_from_task();
   3)   0.057 us    |      __rcu_read_unlock();
   3)               |      __handle_mm_fault() {
   3)               |        filemap_map_pages() {
   3)   0.058 us    |          __rcu_read_lock();
   3)               |          alloc_set_pte() {
   ...

But don't want to show details in handle_mm_fault:

  $ sudo perf ftrace -G __do_page_fault -g handle_mm_fault hello
   3)               |  __do_page_fault() {
   3)   0.049 us    |    down_read_trylock();
   3)               |    find_vma() {
   3)   0.048 us    |      vmacache_find();
   3)   0.041 us    |      vmacache_update();
   3)   0.680 us    |    }
   3)   0.036 us    |    up_read();
   3)   4.547 us    |  } /* __do_page_fault */
   ...

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170618142302.25390-3-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-19 22:05:53 -03:00
Namhyung Kim
dc23103278 perf ftrace: Add support for -a and -C option
The -a/--all-cpus and -C/--cpu option is for controlling tracing cpus.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170224011251.14946-3-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-03 19:07:17 -03:00
Namhyung Kim
a9af6be5bc perf ftrace: Add support for --pid option
The -p (--pid) option enables to trace existing process by its pid.

Committer notes:

Testing it:

Using the function_graph tracer on a process that is just waiting for user
input and thus will make 'perf ftrace' sit there waiting for that, then press
any key on that mutt session and see what happens:

  # perf ftrace -t function_graph -p `pidof mutt` | head -40
  2)   1.038 us    |  switch_mm_irqs_off();
  ------------------------------------------
  2)    <idle>-0    =>   mutt-3595
  ------------------------------------------

  2)               |              finish_task_switch() {
  2)               |                smp_irq_work_interrupt() {
  2)               |                  irq_enter() {
  2)   0.180 us    |                    rcu_irq_enter();
  2)   1.248 us    |                  }
  2)               |                  __wake_up() {
  2)   0.126 us    |                    _raw_spin_lock_irqsave();
  2)               |                    __wake_up_common() {
  2)               |                      pollwake() {
  2)               |                        default_wake_function() {
  2)               |                          try_to_wake_up() {
  2)   0.662 us    |                            _raw_spin_lock_irqsave();
  2)               |                            select_task_rq_fair() {
  2)   1.719 us    |                              effective_load.isra.41();
  2)   1.343 us    |                              effective_load.isra.41();
  2)               |                              select_idle_sibling() {
  2)   0.331 us    |                                idle_cpu();
  2)   1.458 us    |                              }
  2)   8.350 us    |                            }
  2)   0.200 us    |                            _raw_spin_lock();
  2)               |                            ttwu_do_activate() {
  2)               |                              activate_task() {
  2)   0.136 us    |                                update_rq_clock.part.77();
  2)               |                                enqueue_task_fair() {
  2)               |                                  enqueue_entity() {
  2)   0.146 us    |                                    update_curr();
  2)   0.330 us    |                                    account_entity_enqueue();
  2)   0.280 us    |                                    update_cfs_shares();
  2)   0.321 us    |                                    place_entity();
  2)   0.206 us    |                                    __enqueue_entity();
  2)   6.926 us    |                                  }
  2)               |                                  enqueue_entity() {
  2)   0.105 us    |                                    update_curr();
  2)   0.175 us    |                                    account_entity_enqueue();
  2)   0.531 us    |                                    update_cfs_shares();
 #

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170224011251.14946-1-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-03-03 19:07:16 -03:00
Namhyung Kim
d01f4e8db2 perf ftrace: Introduce new 'ftrace' tool
The 'perf ftrace' command is a simple wrapper of kernel's ftrace
functionality.  It only supports single thread tracing currently and
just reads trace_pipe in text and then write it to stdout.

Committer notes:

Testing it:

  # perf ftrace -f function_graph usleep 123456
  <SNIP>
  2)               |  SyS_nanosleep() {
  2)               |    _copy_from_user() {
  <SNIP>
  2)   0.900 us    |      }
  2)   1.354 us    |    }
  2)               |    hrtimer_nanosleep() {
  2)   0.062 us    |      __hrtimer_init();
  2)               |      do_nanosleep() {
  2)               |        hrtimer_start_range_ns() {
  <SNIP>
  2)   5.025 us    |        }
  2)               |        schedule() {
  2)   0.125 us    |          rcu_note_context_switch();
  2)   0.057 us    |          _raw_spin_lock();
  2)               |          deactivate_task() {
  2)   0.369 us    |            update_rq_clock.part.77();
  2)               |            dequeue_task_fair() {
  <SNIP>
  2) + 22.453 us   |            }
  2) + 23.736 us   |          }
  2)               |          pick_next_task_fair() {
  <SNIP>
  2) + 47.167 us   |          }
  2)               |          pick_next_task_idle() {
  <SNIP>
  2)   4.462 us    |          }
  ------------------------------------------
  2)  usleep-20387  =>    <idle>-0
  ------------------------------------------

  2)   0.806 us    |  switch_mm_irqs_off();
  ------------------------------------------
  2)    <idle>-0    =>  usleep-20387
  ------------------------------------------

  2)   0.151 us    |          finish_task_switch();
  2) @ 123597.2 us |        }
  2)   0.037 us    |        _cond_resched();
  2)               |        hrtimer_try_to_cancel() {
  2)   0.064 us    |          hrtimer_active();
  2)   0.353 us    |        }
  2) @ 123605.3 us |      }
  2) @ 123606.2 us |    }
  2) @ 123608.3 us |  } /* SyS_nanosleep */
  2)               |  __do_page_fault() {
 <SNIP>

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jeremy Eder <jeder@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>,
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/n/tip-r1hgmsj4dxny8arn3o9mw512@git.kernel.org
[ Various foward port fixes, add man page ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-01-26 11:43:01 -03:00