[mm/damon] 2fcb93629a: WARNING:at_kernel/tracepoint.c:#tracepoint_add_func
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with clang-14):
commit: 2fcb93629ad8911c846cdc44521c746e53cc4e6d ("mm/damon: add a tracepoint")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: trinity
version: trinity-static-i386-x86_64-f93256fb_2019-08-28
with following parameters:
runtime: 300s
group: group-01
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 4G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-----------------------------------------------------+------------+------------+
| | 3f49584b26 | 2fcb93629a |
+-----------------------------------------------------+------------+------------+
| WARNING:at_kernel/tracepoint.c:#tracepoint_add_func | 0 | 15 |
| EIP:tracepoint_add_func | 0 | 15 |
+-----------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang(a)intel.com>
[ 125.457949][ T3574] WARNING: CPU: 1 PID: 3574 at kernel/tracepoint.c:188 tracepoint_add_func (kernel/tracepoint.c:336 kernel/tracepoint.c:338)
[ 125.459938][ T3574] Modules linked in: serio_raw evdev input_leds tiny_power_button button
[ 125.461589][ T3574] CPU: 1 PID: 3574 Comm: trinity-main Not tainted 5.14.0-00035-g2fcb93629ad8 #1
[ 125.463374][ T3574] EIP: tracepoint_add_func (kernel/tracepoint.c:336 kernel/tracepoint.c:338)
[ 125.464470][ T3574] Code: 75 ab c2 00 ff ff c6 05 ff ff c6 05 01 b8 c1 aa 01 b8 c1 aa 01 00 00 b9 01 00 00 b9 e8 51 e6 f7 e8 51 e6 f7 ff ff 0f 0b ff ff <0f> 0b c2 89 da 31 c2 89 da 31 fa 81 02 00 fa 81 02 00 5d f0 bf ea
All code
========
0: 75 ab jne 0xffffffffffffffad
2: c2 00 ff retq $0xff00
5: ff c6 inc %esi
7: 05 ff ff c6 05 add $0x5c6ffff,%eax
c: 01 b8 c1 aa 01 b8 add %edi,-0x47fe553f(%rax)
12: c1 aa 01 00 00 b9 01 shrl $0x1,-0x46ffffff(%rdx)
19: 00 00 add %al,(%rax)
1b: b9 e8 51 e6 f7 mov $0xf7e651e8,%ecx
20: e8 51 e6 f7 ff callq 0xfffffffffff7e676
25: ff 0f decl (%rdi)
27: 0b ff or %edi,%edi
29: ff 0f decl (%rdi)
2b:* 0b c2 or %edx,%eax <-- trapping instruction
2d: 89 da mov %ebx,%edx
2f: 31 c2 xor %eax,%edx
31: 89 da mov %ebx,%edx
33: 31 fa xor %edi,%edx
35: 81 02 00 fa 81 02 addl $0x281fa00,(%rdx)
3b: 00 5d f0 add %bl,-0x10(%rbp)
3e: bf .byte 0xbf
3f: ea (bad)
Code starting with the faulting instruction
===========================================
0: 0f 0b ud2
2: c2 89 da retq $0xda89
5: 31 c2 xor %eax,%edx
7: 89 da mov %ebx,%edx
9: 31 fa xor %edi,%edx
b: 81 02 00 fa 81 02 addl $0x281fa00,(%rdx)
11: 00 5d f0 add %bl,-0x10(%rbp)
14: bf .byte 0xbf
15: ea (bad)
[ 125.478613][ T3574] EAX: 00000001 EBX: 00000001 ECX: 00000000 EDX: ffffffff
[ 125.479986][ T3574] ESI: 00000000 EDI: 00000000 EBP: f5f7fdd8 ESP: f5f7fdbc
[ 125.481344][ T3574] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 EFLAGS: 00010246
[ 125.482847][ T3574] CR0: 80050033 CR2: 08d8100c CR3: 35f10000 CR4: 000406f0
[ 125.484190][ T3574] Call Trace:
[ 125.484844][ T3574] ? mutex_lock_nested (kernel/locking/mutex.c:1104 kernel/locking/mutex.c:1119)
[ 125.485778][ T3574] tracepoint_probe_register (kernel/tracepoint.c:511 kernel/tracepoint.c:531)
[ 125.486892][ T3574] trace_event_reg (kernel/trace/trace_events.c:512)
[ 125.487828][ T3574] ? trace_event_buffer_reserve (kernel/trace/trace_events.c:495)
[ 125.488922][ T3574] perf_trace_event_init (kernel/trace/trace_event_perf.c:129 kernel/trace/trace_event_perf.c:204)
[ 125.489946][ T3574] perf_trace_init (kernel/trace/trace_event_perf.c:229)
[ 125.500175][ T3574] perf_tp_event_init (kernel/events/core.c:9725)
[ 125.501858][ T3574] perf_try_init_event (kernel/events/core.c:11192)
[ 125.503727][ T3574] perf_event_alloc (kernel/events/core.c:11256 kernel/events/core.c:11547)
[ 125.505324][ T3574] __ia32_sys_perf_event_open (kernel/events/core.c:12068 kernel/events/core.c:11960 kernel/events/core.c:11960)
[ 125.507235][ T3574] ? syscall_enter_from_user_mode (arch/x86/include/asm/irqflags.h:45 arch/x86/include/asm/irqflags.h:80 kernel/entry/common.c:107)
[ 125.509227][ T3574] ? syscall_enter_from_user_mode (arch/x86/include/asm/irqflags.h:45 arch/x86/include/asm/irqflags.h:80 kernel/entry/common.c:107)
[ 125.511394][ T3574] do_int80_syscall_32 (arch/x86/entry/common.c:112 arch/x86/entry/common.c:132)
[ 125.512627][ T3574] ? exc_page_fault (arch/x86/mm/fault.c:1535)
[ 125.513600][ T3574] entry_INT80_32 (arch/x86/entry/entry_32.S:981)
[ 125.515381][ T3574] EIP: 0x8097522
[ 125.516815][ T3574] Code: 89 c8 c3 90 8d 74 26 00 85 c0 c7 01 01 00 00 00 75 d8 a1 cc 3c ad 08 eb d1 66 90 66 90 66 90 66 90 66 90 66 90 66 90 90 cd 80 <c3> 8d b6 00 00 00 00 8d bc 27 00 00 00 00 8b 10 a3 f4 3c ad 08 85
All code
========
0: 89 c8 mov %ecx,%eax
2: c3 retq
3: 90 nop
4: 8d 74 26 00 lea 0x0(%rsi,%riz,1),%esi
8: 85 c0 test %eax,%eax
a: c7 01 01 00 00 00 movl $0x1,(%rcx)
10: 75 d8 jne 0xffffffffffffffea
12: a1 cc 3c ad 08 eb d1 movabs 0x9066d1eb08ad3ccc,%eax
19: 66 90
1b: 66 90 xchg %ax,%ax
1d: 66 90 xchg %ax,%ax
1f: 66 90 xchg %ax,%ax
21: 66 90 xchg %ax,%ax
23: 66 90 xchg %ax,%ax
25: 66 90 xchg %ax,%ax
27: 90 nop
28: cd 80 int $0x80
2a:* c3 retq <-- trapping instruction
2b: 8d b6 00 00 00 00 lea 0x0(%rsi),%esi
31: 8d bc 27 00 00 00 00 lea 0x0(%rdi,%riz,1),%edi
38: 8b 10 mov (%rax),%edx
3a: a3 .byte 0xa3
3b: f4 hlt
3c: 3c ad cmp $0xad,%al
3e: 08 .byte 0x8
3f: 85 .byte 0x85
Code starting with the faulting instruction
===========================================
0: c3 retq
1: 8d b6 00 00 00 00 lea 0x0(%rsi),%esi
7: 8d bc 27 00 00 00 00 lea 0x0(%rdi,%riz,1),%edi
e: 8b 10 mov (%rax),%edx
10: a3 .byte 0xa3
11: f4 hlt
12: 3c ad cmp $0xad,%al
14: 08 .byte 0x8
15: 85 .byte 0x85
[ 125.524040][ T3574] EAX: ffffffda EBX: 08d81718 ECX: 00000df6 EDX: ffffffff
[ 125.526956][ T3574] ESI: 00000111 EDI: 00000009 EBP: 00000005 ESP: bfade408
[ 125.529387][ T3574] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000282
[ 125.532073][ T3574] irq event stamp: 1392883
[ 125.533918][ T3574] hardirqs last enabled at (1392891): console_unlock (arch/x86/include/asm/irqflags.h:22 arch/x86/include/asm/irqflags.h:70 arch/x86/include/asm/irqflags.h:132 kernel/printk/printk.c:2668)
[ 125.537178][ T3574] hardirqs last disabled at (1392906): console_unlock (kernel/printk/printk.c:2589)
[ 125.540379][ T3574] softirqs last enabled at (1392904): __do_softirq (arch/x86/include/asm/preempt.h:27 kernel/softirq.c:402 kernel/softirq.c:587)
[ 125.543618][ T3574] softirqs last disabled at (1392899): do_softirq_own_stack (arch/x86/kernel/irq_32.c:60 arch/x86/kernel/irq_32.c:149)
[ 125.547068][ T3574] ---[ end trace 3cf4e488c277e941 ]---
[ 125.549053][ T3574] ------------[ cut here ]------------
To reproduce:
# build kernel
cd linux
cp config-5.14.0-00035-g2fcb93629ad8 .config
make HOSTCC=clang-14 CC=clang-14 ARCH=i386 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation
Thanks,
Oliver Sang
9 months, 2 weeks
- 1
- 0
29b51054ec: stress-ng.vm-rw.ops_per_sec 81.4% improvement
by kernel test robot
Greeting,
FYI, we noticed a 81.4% improvement of stress-ng.vm-rw.ops_per_sec due to commit:
commit: 29b51054eca8c8c58122830b8a747df1a4605045 ("SIS_DEPTH")
https://git.kernel.org/cgit/linux/kernel/git/mel/linux.git sched-sisdepth-v1r3
in testcase: stress-ng
on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory
with following parameters:
nr_threads: 100%
testtime: 60s
class: memory
test: vm-rw
cpufreq_governor: performance
ucode: 0x5003006
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
=========================================================================================
class/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode:
memory/gcc-9/performance/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp5/vm-rw/stress-ng/60s/0x5003006
commit:
b1168d8cf9 ("sched/fair: Enforce proportional scan limits when scanning for an idle core")
29b51054ec ("SIS_DEPTH")
b1168d8cf909432b 29b51054eca8c8c58122830b8a7
---------------- ---------------------------
%stddev %change %stddev
\ | \
5538231 ± 48% +1989.3% 1.157e+08 ± 3% stress-ng.time.involuntary_context_switches
137981 ± 2% -26.8% 100941 stress-ng.time.minor_page_faults
7899 +15.8% 9145 stress-ng.time.percent_of_cpu_this_job_got
4670 +15.0% 5371 ± 2% stress-ng.time.system_time
64073704 ± 3% +81.4% 1.162e+08 ± 3% stress-ng.vm-rw.ops
1067893 ± 3% +81.4% 1936842 ± 3% stress-ng.vm-rw.ops_per_sec
4131 ± 2% -20.2% 3297 ± 4% uptime.idle
9.786e+08 ± 2% -66.0% 3.323e+08 ± 4% cpuidle..time
84664815 ± 2% -98.3% 1450183 ± 98% cpuidle..usage
19.00 ± 3% -62.6% 7.11 ± 4% vmstat.cpu.id
3261026 ± 2% +10.1% 3589451 ± 3% vmstat.system.cs
236959 -11.6% 209414 vmstat.system.in
17.02 ± 4% -12.3 4.76 ± 3% mpstat.cpu.all.idle%
1.43 ± 2% -0.2 1.19 mpstat.cpu.all.irq%
0.22 ± 8% -0.2 0.03 ± 42% mpstat.cpu.all.soft%
77.46 +11.8 89.29 ± 2% mpstat.cpu.all.sys%
1.301e+09 ± 4% +77.3% 2.307e+09 ± 3% numa-vmstat.node0.nr_foll_pin_acquired
1.301e+09 ± 4% +77.3% 2.307e+09 ± 3% numa-vmstat.node0.nr_foll_pin_released
1.304e+09 ± 9% +80.4% 2.352e+09 ± 4% numa-vmstat.node1.nr_foll_pin_acquired
1.304e+09 ± 9% +80.4% 2.352e+09 ± 4% numa-vmstat.node1.nr_foll_pin_released
85.37 +8.9 94.24 turbostat.Busy%
2721 -4.7% 2594 turbostat.Bzy_MHz
81295809 ± 2% -99.1% 749426 ±168% turbostat.C1
10.22 ± 4% -10.0 0.19 ±103% turbostat.C1%
14.28 ± 4% -60.8% 5.59 ± 4% turbostat.CPU%c1
15433339 -12.0% 13578750 turbostat.IRQ
2.601e+09 ± 4% +78.5% 4.641e+09 ± 3% proc-vmstat.nr_foll_pin_acquired
2.601e+09 ± 4% +78.5% 4.641e+09 ± 3% proc-vmstat.nr_foll_pin_released
20787 -1.5% 20468 proc-vmstat.nr_kernel_stack
114737 ± 3% -32.5% 77413 ± 4% proc-vmstat.numa_hint_faults
109876 ± 2% -30.3% 76571 ± 3% proc-vmstat.numa_hint_faults_local
453394 -2.1% 444065 proc-vmstat.numa_hit
365310 -2.2% 357435 proc-vmstat.numa_local
128802 ± 13% -33.1% 86189 ± 3% proc-vmstat.numa_pte_updates
454878 -1.7% 447022 proc-vmstat.pgalloc_normal
475917 -9.0% 433297 proc-vmstat.pgfault
338872 -6.0% 318435 ± 2% proc-vmstat.pgfree
16630 +4.9% 17446 proc-vmstat.pgreuse
33.94 ± 7% -84.4% 5.30 ± 10% perf-stat.i.MPKI
2.014e+10 ± 3% +64.2% 3.307e+10 ± 3% perf-stat.i.branch-instructions
0.60 -0.2 0.42 ± 5% perf-stat.i.branch-miss-rate%
0.78 ± 12% +0.2 0.97 ± 10% perf-stat.i.cache-miss-rate%
8734810 ± 10% -57.4% 3717736 ± 12% perf-stat.i.cache-misses
3.524e+09 ± 4% -74.6% 8.963e+08 ± 11% perf-stat.i.cache-references
3390246 ± 2% +9.4% 3710252 ± 3% perf-stat.i.context-switches
2.20 -36.6% 1.39 ± 4% perf-stat.i.cpi
2.269e+11 +3.8% 2.356e+11 perf-stat.i.cpu-cycles
1124709 ± 2% -98.9% 12887 ±197% perf-stat.i.cpu-migrations
27017 ± 10% +146.1% 66498 ± 11% perf-stat.i.cycles-between-cache-misses
0.24 ± 8% -0.2 0.01 ±116% perf-stat.i.dTLB-load-miss-rate%
60478377 ± 5% -95.6% 2662359 ±132% perf-stat.i.dTLB-load-misses
2.492e+10 ± 3% +62.7% 4.056e+10 ± 3% perf-stat.i.dTLB-loads
0.09 ± 5% -0.1 0.00 ± 66% perf-stat.i.dTLB-store-miss-rate%
12990615 ± 7% -98.6% 183426 ±121% perf-stat.i.dTLB-store-misses
1.378e+10 ± 3% +62.1% 2.233e+10 ± 3% perf-stat.i.dTLB-stores
51.42 +44.6 95.99 perf-stat.i.iTLB-load-miss-rate%
35796355 ± 2% +19.4% 42740986 ± 17% perf-stat.i.iTLB-load-misses
33650607 ± 2% -97.3% 917813 ± 81% perf-stat.i.iTLB-loads
1.02e+11 ± 3% +64.7% 1.68e+11 ± 3% perf-stat.i.instructions
0.47 +54.7% 0.72 ± 3% perf-stat.i.ipc
2.36 +4.0% 2.45 perf-stat.i.metric.GHz
595.87 ± 2% -86.8% 78.44 ± 15% perf-stat.i.metric.K/sec
648.74 ± 2% +55.5% 1008 ± 3% perf-stat.i.metric.M/sec
5598 -8.5% 5123 perf-stat.i.minor-faults
2441321 ± 7% -83.7% 397862 ± 29% perf-stat.i.node-load-misses
313570 ± 8% -75.6% 76449 ± 9% perf-stat.i.node-loads
89.23 ± 2% +7.7 96.90 perf-stat.i.node-store-miss-rate%
2152556 ± 4% -35.2% 1395004 ± 5% perf-stat.i.node-store-misses
225240 ± 28% -87.3% 28705 ± 7% perf-stat.i.node-stores
5610 -8.5% 5135 perf-stat.i.page-faults
34.64 ± 7% -84.6% 5.34 ± 11% perf-stat.overall.MPKI
0.49 -0.2 0.30 ± 8% perf-stat.overall.branch-miss-rate%
0.25 ± 6% +0.2 0.42 ± 16% perf-stat.overall.cache-miss-rate%
2.23 ± 2% -36.9% 1.40 ± 4% perf-stat.overall.cpi
26253 ± 10% +144.4% 64164 ± 12% perf-stat.overall.cycles-between-cache-misses
0.24 ± 8% -0.2 0.01 ±134% perf-stat.overall.dTLB-load-miss-rate%
0.09 ± 6% -0.1 0.00 ±119% perf-stat.overall.dTLB-store-miss-rate%
51.54 +46.2 97.78 perf-stat.overall.iTLB-load-miss-rate%
0.45 ± 2% +58.7% 0.71 ± 4% perf-stat.overall.ipc
90.58 ± 2% +7.4 97.97 perf-stat.overall.node-store-miss-rate%
1.98e+10 ± 3% +64.4% 3.256e+10 ± 3% perf-stat.ps.branch-instructions
8595790 ± 10% -57.3% 3671433 ± 12% perf-stat.ps.cache-misses
3.465e+09 ± 4% -74.5% 8.826e+08 ± 11% perf-stat.ps.cache-references
3332969 ± 2% +9.6% 3653064 ± 3% perf-stat.ps.context-switches
2.231e+11 +4.0% 2.32e+11 perf-stat.ps.cpu-cycles
1105944 ± 2% -98.9% 12538 ±196% perf-stat.ps.cpu-migrations
59497423 ± 5% -95.6% 2643257 ±131% perf-stat.ps.dTLB-load-misses
2.45e+10 ± 3% +63.0% 3.993e+10 ± 3% perf-stat.ps.dTLB-loads
12774486 ± 7% -98.6% 181288 ±119% perf-stat.ps.dTLB-store-misses
1.354e+10 ± 3% +62.3% 2.199e+10 ± 3% perf-stat.ps.dTLB-stores
35191808 ± 2% +19.6% 42083022 ± 17% perf-stat.ps.iTLB-load-misses
33083605 ± 2% -97.3% 898866 ± 81% perf-stat.ps.iTLB-loads
1.003e+11 ± 3% +65.0% 1.655e+11 ± 3% perf-stat.ps.instructions
5472 -7.9% 5038 perf-stat.ps.minor-faults
2399916 ± 7% -83.6% 392567 ± 30% perf-stat.ps.node-load-misses
314494 ± 8% -74.7% 79525 ± 9% perf-stat.ps.node-loads
2116241 ± 4% -35.1% 1373832 ± 5% perf-stat.ps.node-store-misses
221598 ± 28% -87.2% 28337 ± 7% perf-stat.ps.node-stores
5484 -7.9% 5050 perf-stat.ps.page-faults
6.306e+12 ± 3% +65.4% 1.043e+13 ± 3% perf-stat.total.instructions
29297 ± 3% -72.8% 7981 ± 25% softirqs.CPU0.SCHED
25131 ± 2% -85.3% 3686 ± 47% softirqs.CPU1.SCHED
24180 ± 4% -88.0% 2898 ± 57% softirqs.CPU10.SCHED
24342 ± 3% -88.9% 2691 ± 59% softirqs.CPU11.SCHED
25129 ± 3% -89.3% 2690 ± 59% softirqs.CPU12.SCHED
24625 ± 2% -89.0% 2700 ± 61% softirqs.CPU13.SCHED
24721 -86.5% 3332 ± 63% softirqs.CPU14.SCHED
24606 ± 2% -88.5% 2822 ± 57% softirqs.CPU15.SCHED
24846 ± 4% -88.8% 2772 ± 60% softirqs.CPU16.SCHED
24666 ± 2% -87.6% 3064 ± 57% softirqs.CPU17.SCHED
24798 ± 3% -87.7% 3056 ± 56% softirqs.CPU18.SCHED
24594 ± 2% -88.7% 2767 ± 63% softirqs.CPU19.SCHED
24932 ± 2% -86.1% 3458 ± 51% softirqs.CPU2.SCHED
24510 ± 2% -88.4% 2849 ± 58% softirqs.CPU20.SCHED
24882 ± 4% -88.4% 2896 ± 57% softirqs.CPU21.SCHED
24765 ± 2% -88.7% 2799 ± 61% softirqs.CPU22.SCHED
24679 ± 2% -88.3% 2886 ± 63% softirqs.CPU23.SCHED
24056 ± 5% -90.5% 2278 ± 25% softirqs.CPU24.SCHED
24304 ± 5% -91.9% 1970 ± 18% softirqs.CPU25.SCHED
23775 ± 3% -91.6% 2008 ± 20% softirqs.CPU26.SCHED
23925 ± 4% -91.4% 2055 ± 25% softirqs.CPU27.SCHED
23681 ± 4% -91.7% 1968 ± 17% softirqs.CPU28.SCHED
23918 ± 2% -91.2% 2103 ± 25% softirqs.CPU29.SCHED
24509 ± 3% -86.7% 3259 ± 53% softirqs.CPU3.SCHED
23882 ± 3% -91.8% 1960 ± 18% softirqs.CPU30.SCHED
23819 ± 4% -91.2% 2097 ± 22% softirqs.CPU31.SCHED
23933 ± 4% -91.2% 2114 ± 23% softirqs.CPU32.SCHED
23808 ± 4% -91.4% 2041 ± 20% softirqs.CPU33.SCHED
23861 ± 3% -90.9% 2173 ± 31% softirqs.CPU34.SCHED
23867 ± 4% -91.8% 1949 ± 19% softirqs.CPU35.SCHED
23869 ± 3% -91.1% 2115 ± 22% softirqs.CPU36.SCHED
24142 ± 4% -91.9% 1948 ± 19% softirqs.CPU37.SCHED
23705 ± 4% -91.8% 1936 ± 19% softirqs.CPU38.SCHED
23784 ± 3% -91.5% 2016 ± 23% softirqs.CPU39.SCHED
24562 -88.0% 2950 ± 74% softirqs.CPU4.SCHED
23989 ± 5% -90.6% 2250 ± 35% softirqs.CPU40.SCHED
23928 ± 4% -91.8% 1953 ± 21% softirqs.CPU41.SCHED
23864 ± 3% -92.4% 1811 ± 19% softirqs.CPU42.SCHED
23933 ± 4% -91.8% 1961 ± 21% softirqs.CPU43.SCHED
23720 ± 3% -91.4% 2045 ± 24% softirqs.CPU44.SCHED
23807 ± 3% -91.7% 1964 ± 19% softirqs.CPU45.SCHED
24364 ± 7% -91.2% 2140 ± 27% softirqs.CPU46.SCHED
23064 ± 3% -91.5% 1956 ± 21% softirqs.CPU47.SCHED
24399 ± 2% -88.4% 2832 ± 61% softirqs.CPU48.SCHED
24378 ± 2% -89.0% 2680 ± 66% softirqs.CPU49.SCHED
24769 -88.2% 2923 ± 56% softirqs.CPU5.SCHED
24491 ± 2% -88.8% 2730 ± 62% softirqs.CPU50.SCHED
24401 ± 2% -88.7% 2764 ± 60% softirqs.CPU51.SCHED
24487 -87.7% 3023 ± 52% softirqs.CPU52.SCHED
24392 ± 2% -88.4% 2820 ± 61% softirqs.CPU53.SCHED
24457 ± 2% -87.5% 3049 ± 55% softirqs.CPU54.SCHED
24455 ± 2% -88.5% 2822 ± 57% softirqs.CPU55.SCHED
24469 ± 3% -88.9% 2717 ± 62% softirqs.CPU56.SCHED
24321 ± 2% -88.5% 2800 ± 58% softirqs.CPU57.SCHED
24356 ± 3% -88.3% 2857 ± 55% softirqs.CPU58.SCHED
24629 ± 5% -88.6% 2798 ± 59% softirqs.CPU59.SCHED
24798 -87.0% 3212 ± 56% softirqs.CPU6.SCHED
24530 ± 2% -88.3% 2868 ± 57% softirqs.CPU60.SCHED
24623 ± 3% -88.1% 2942 ± 57% softirqs.CPU61.SCHED
24587 ± 2% -88.3% 2865 ± 61% softirqs.CPU62.SCHED
24432 ± 2% -88.4% 2841 ± 55% softirqs.CPU63.SCHED
24445 ± 2% -88.9% 2713 ± 59% softirqs.CPU64.SCHED
24474 ± 2% -88.7% 2766 ± 63% softirqs.CPU65.SCHED
24595 ± 2% -88.6% 2807 ± 58% softirqs.CPU66.SCHED
24395 ± 2% -88.4% 2834 ± 61% softirqs.CPU67.SCHED
24329 -88.8% 2719 ± 59% softirqs.CPU68.SCHED
24357 ± 2% -88.5% 2799 ± 59% softirqs.CPU69.SCHED
24623 ± 2% -86.9% 3234 ± 62% softirqs.CPU7.SCHED
24405 -88.1% 2893 ± 61% softirqs.CPU70.SCHED
24582 ± 2% -88.0% 2958 ± 56% softirqs.CPU71.SCHED
23827 ± 4% -91.9% 1933 ± 20% softirqs.CPU72.SCHED
23767 ± 4% -91.6% 1995 ± 21% softirqs.CPU73.SCHED
23700 ± 4% -91.7% 1976 ± 20% softirqs.CPU74.SCHED
23641 ± 4% -91.6% 1991 ± 20% softirqs.CPU75.SCHED
24948 ± 8% -91.9% 2029 ± 19% softirqs.CPU76.SCHED
23750 ± 3% -91.8% 1957 ± 20% softirqs.CPU77.SCHED
23848 ± 4% -90.2% 2347 ± 38% softirqs.CPU78.SCHED
23829 ± 3% -91.7% 1981 ± 20% softirqs.CPU79.SCHED
24566 ± 2% -88.3% 2866 ± 58% softirqs.CPU8.SCHED
23671 ± 4% -91.7% 1965 ± 21% softirqs.CPU80.SCHED
23701 ± 4% -91.8% 1950 ± 23% softirqs.CPU81.SCHED
23746 ± 4% -91.2% 2095 ± 17% softirqs.CPU82.SCHED
23740 ± 4% -91.1% 2112 ± 22% softirqs.CPU83.SCHED
23798 ± 5% -91.4% 2058 ± 18% softirqs.CPU84.SCHED
23582 ± 4% -91.6% 1969 ± 20% softirqs.CPU85.SCHED
23603 ± 4% -91.3% 2056 ± 29% softirqs.CPU86.SCHED
23635 ± 4% -91.7% 1954 ± 19% softirqs.CPU87.SCHED
23693 ± 3% -91.6% 1999 ± 23% softirqs.CPU88.SCHED
23693 ± 4% -91.0% 2122 ± 23% softirqs.CPU89.SCHED
24933 ± 3% -88.7% 2811 ± 65% softirqs.CPU9.SCHED
23636 ± 4% -91.6% 1994 ± 21% softirqs.CPU90.SCHED
23734 ± 3% -91.4% 2041 ± 19% softirqs.CPU91.SCHED
23561 ± 4% -91.6% 1978 ± 18% softirqs.CPU92.SCHED
23653 ± 4% -91.5% 2000 ± 18% softirqs.CPU93.SCHED
23609 ± 3% -91.4% 2021 ± 20% softirqs.CPU94.SCHED
22989 ± 3% -91.9% 1867 ± 16% softirqs.CPU95.SCHED
2326871 ± 2% -89.6% 241771 ± 33% softirqs.SCHED
544023 ± 29% -87.3% 69185 ± 46% interrupts.CAL:Function_call_interrupts
6016 ± 31% -85.2% 888.33 ± 73% interrupts.CPU0.CAL:Function_call_interrupts
15138 ± 8% -95.4% 694.89 ±143% interrupts.CPU0.RES:Rescheduling_interrupts
6013 ± 28% -85.3% 884.44 ± 72% interrupts.CPU1.CAL:Function_call_interrupts
15445 ± 8% -96.6% 525.89 ±185% interrupts.CPU1.RES:Rescheduling_interrupts
6730 ± 34% -87.6% 837.56 ± 78% interrupts.CPU10.CAL:Function_call_interrupts
15083 ± 9% -96.8% 489.67 ±200% interrupts.CPU10.RES:Rescheduling_interrupts
7121 ± 31% -87.9% 864.56 ± 92% interrupts.CPU11.CAL:Function_call_interrupts
14687 ± 9% -96.5% 518.78 ±188% interrupts.CPU11.RES:Rescheduling_interrupts
7023 ± 32% -88.6% 798.00 ± 78% interrupts.CPU12.CAL:Function_call_interrupts
14542 ± 8% -96.4% 519.00 ±189% interrupts.CPU12.RES:Rescheduling_interrupts
6532 ± 29% -86.3% 893.33 ± 99% interrupts.CPU13.CAL:Function_call_interrupts
15164 ± 8% -96.4% 542.11 ±185% interrupts.CPU13.RES:Rescheduling_interrupts
7408 ± 32% -87.2% 951.78 ±102% interrupts.CPU14.CAL:Function_call_interrupts
15750 ± 9% -96.0% 629.78 ±152% interrupts.CPU14.RES:Rescheduling_interrupts
6190 ± 32% -85.8% 878.22 ± 95% interrupts.CPU15.CAL:Function_call_interrupts
15334 ± 8% -96.6% 527.44 ±192% interrupts.CPU15.RES:Rescheduling_interrupts
6580 ± 31% -86.8% 869.00 ± 91% interrupts.CPU16.CAL:Function_call_interrupts
15162 ± 8% -96.5% 525.44 ±181% interrupts.CPU16.RES:Rescheduling_interrupts
6587 ± 31% -87.5% 825.89 ± 81% interrupts.CPU17.CAL:Function_call_interrupts
14689 ± 8% -96.4% 533.44 ±183% interrupts.CPU17.RES:Rescheduling_interrupts
7509 ± 32% -88.9% 829.89 ± 85% interrupts.CPU18.CAL:Function_call_interrupts
14901 ± 8% -96.4% 535.33 ±186% interrupts.CPU18.RES:Rescheduling_interrupts
7320 ± 32% -87.4% 920.78 ± 96% interrupts.CPU19.CAL:Function_call_interrupts
15334 ± 9% -96.3% 570.56 ±177% interrupts.CPU19.RES:Rescheduling_interrupts
6799 ± 33% -82.2% 1208 ± 52% interrupts.CPU2.CAL:Function_call_interrupts
15463 ± 8% -96.7% 510.11 ±188% interrupts.CPU2.RES:Rescheduling_interrupts
6130 ± 30% -85.6% 882.33 ± 94% interrupts.CPU20.CAL:Function_call_interrupts
15426 ± 6% -96.4% 559.22 ±188% interrupts.CPU20.RES:Rescheduling_interrupts
5990 ± 30% -85.9% 842.78 ± 83% interrupts.CPU21.CAL:Function_call_interrupts
14819 ± 8% -96.3% 550.78 ±189% interrupts.CPU21.RES:Rescheduling_interrupts
6227 ± 31% -87.0% 809.67 ± 79% interrupts.CPU22.CAL:Function_call_interrupts
14439 ± 8% -96.2% 547.33 ±173% interrupts.CPU22.RES:Rescheduling_interrupts
6301 ± 31% -87.1% 814.00 ± 81% interrupts.CPU23.CAL:Function_call_interrupts
14559 ± 9% -96.3% 532.11 ±187% interrupts.CPU23.RES:Rescheduling_interrupts
4778 ± 28% -87.9% 578.56 ± 4% interrupts.CPU24.CAL:Function_call_interrupts
14525 ± 8% -98.7% 189.67 ± 31% interrupts.CPU24.RES:Rescheduling_interrupts
5328 ± 30% -88.8% 597.11 ± 9% interrupts.CPU25.CAL:Function_call_interrupts
14957 ± 7% -98.8% 183.67 ± 19% interrupts.CPU25.RES:Rescheduling_interrupts
4517 ± 27% -87.2% 578.56 ± 6% interrupts.CPU26.CAL:Function_call_interrupts
14744 ± 7% -98.6% 201.33 ± 66% interrupts.CPU26.RES:Rescheduling_interrupts
4958 ± 28% -88.1% 592.22 ± 7% interrupts.CPU27.CAL:Function_call_interrupts
14886 ± 7% -98.9% 157.11 ± 26% interrupts.CPU27.RES:Rescheduling_interrupts
5072 ± 28% -88.6% 579.67 ± 7% interrupts.CPU28.CAL:Function_call_interrupts
14596 ± 7% -98.7% 193.22 ± 40% interrupts.CPU28.RES:Rescheduling_interrupts
5107 ± 29% -88.7% 575.11 ± 5% interrupts.CPU29.CAL:Function_call_interrupts
14220 ± 8% -99.0% 145.89 ± 26% interrupts.CPU29.RES:Rescheduling_interrupts
6327 ± 31% -83.9% 1017 ± 81% interrupts.CPU3.CAL:Function_call_interrupts
15349 ± 11% -96.6% 515.44 ±194% interrupts.CPU3.RES:Rescheduling_interrupts
5516 ± 29% -89.1% 600.22 ± 5% interrupts.CPU30.CAL:Function_call_interrupts
14302 ± 6% -98.7% 182.78 ± 35% interrupts.CPU30.RES:Rescheduling_interrupts
5361 ± 30% -89.4% 569.22 ± 14% interrupts.CPU31.CAL:Function_call_interrupts
14961 ± 7% -98.9% 160.33 ± 22% interrupts.CPU31.RES:Rescheduling_interrupts
4260 ± 28% -87.5% 531.56 ± 15% interrupts.CPU32.CAL:Function_call_interrupts
14387 ± 7% -99.1% 134.89 ± 32% interrupts.CPU32.RES:Rescheduling_interrupts
4675 ± 26% -87.6% 580.22 ± 5% interrupts.CPU33.CAL:Function_call_interrupts
14563 ± 7% -98.9% 155.67 ± 27% interrupts.CPU33.RES:Rescheduling_interrupts
4964 ± 28% -88.4% 577.22 ± 5% interrupts.CPU34.CAL:Function_call_interrupts
14414 ± 7% -98.8% 176.56 ± 56% interrupts.CPU34.RES:Rescheduling_interrupts
4898 ± 26% -88.3% 571.00 ± 6% interrupts.CPU35.CAL:Function_call_interrupts
13928 ± 6% -98.4% 220.56 ± 82% interrupts.CPU35.RES:Rescheduling_interrupts
4938 ± 28% -87.6% 611.67 ± 13% interrupts.CPU36.CAL:Function_call_interrupts
14398 ± 6% -98.5% 216.67 ± 70% interrupts.CPU36.RES:Rescheduling_interrupts
4512 ± 27% -87.5% 562.89 ± 12% interrupts.CPU37.CAL:Function_call_interrupts
14616 ± 7% -98.8% 173.00 ± 54% interrupts.CPU37.RES:Rescheduling_interrupts
4588 ± 28% -87.7% 565.33 ± 11% interrupts.CPU38.CAL:Function_call_interrupts
14783 ± 6% -98.7% 191.11 ± 56% interrupts.CPU38.RES:Rescheduling_interrupts
4522 ± 29% -87.6% 562.89 ± 14% interrupts.CPU39.CAL:Function_call_interrupts
14340 ± 7% -99.0% 145.56 ± 30% interrupts.CPU39.RES:Rescheduling_interrupts
6541 ± 30% -88.6% 748.00 ± 76% interrupts.CPU4.CAL:Function_call_interrupts
15200 ± 9% -96.7% 506.44 ±180% interrupts.CPU4.RES:Rescheduling_interrupts
4650 ± 26% -87.5% 580.89 ± 9% interrupts.CPU40.CAL:Function_call_interrupts
14143 ± 7% -98.9% 162.33 ± 33% interrupts.CPU40.RES:Rescheduling_interrupts
5445 ± 28% -89.5% 570.11 ± 6% interrupts.CPU41.CAL:Function_call_interrupts
14144 ± 6% -98.9% 154.33 ± 25% interrupts.CPU41.RES:Rescheduling_interrupts
4727 ± 26% -87.6% 586.33 ± 4% interrupts.CPU42.CAL:Function_call_interrupts
14803 ± 8% -99.0% 148.44 ± 26% interrupts.CPU42.RES:Rescheduling_interrupts
5452 ± 29% -89.4% 579.22 ± 5% interrupts.CPU43.CAL:Function_call_interrupts
14677 ± 6% -98.7% 184.11 ± 59% interrupts.CPU43.RES:Rescheduling_interrupts
4711 ± 29% -87.0% 611.89 ± 13% interrupts.CPU44.CAL:Function_call_interrupts
14722 ± 8% -98.9% 169.00 ± 36% interrupts.CPU44.RES:Rescheduling_interrupts
4287 ± 27% -86.3% 585.78 ± 6% interrupts.CPU45.CAL:Function_call_interrupts
14491 ± 7% -99.0% 147.89 ± 30% interrupts.CPU45.RES:Rescheduling_interrupts
4489 ± 28% -87.0% 584.56 ± 7% interrupts.CPU46.CAL:Function_call_interrupts
14126 ± 8% -98.7% 184.78 ± 51% interrupts.CPU46.RES:Rescheduling_interrupts
4635 ± 25% -76.7% 1078 ± 3% interrupts.CPU47.CAL:Function_call_interrupts
13933 ± 8% -97.2% 395.33 ± 10% interrupts.CPU47.RES:Rescheduling_interrupts
6475 ± 30% -86.9% 848.56 ± 75% interrupts.CPU48.CAL:Function_call_interrupts
15160 ± 8% -96.3% 554.56 ±179% interrupts.CPU48.RES:Rescheduling_interrupts
6544 ± 31% -87.2% 836.89 ± 89% interrupts.CPU49.CAL:Function_call_interrupts
15462 ± 8% -96.6% 526.44 ±182% interrupts.CPU49.RES:Rescheduling_interrupts
6533 ± 32% -85.5% 949.44 ± 74% interrupts.CPU5.CAL:Function_call_interrupts
14789 ± 9% -96.5% 520.44 ±189% interrupts.CPU5.RES:Rescheduling_interrupts
7679 ± 31% -88.8% 861.33 ± 87% interrupts.CPU50.CAL:Function_call_interrupts
15730 ± 9% -96.7% 523.00 ±185% interrupts.CPU50.RES:Rescheduling_interrupts
6869 ± 30% -88.2% 812.33 ± 84% interrupts.CPU51.CAL:Function_call_interrupts
15446 ± 9% -96.7% 513.78 ±187% interrupts.CPU51.RES:Rescheduling_interrupts
6832 ± 31% -88.5% 788.89 ± 74% interrupts.CPU52.CAL:Function_call_interrupts
15223 ± 8% -96.4% 541.00 ±186% interrupts.CPU52.RES:Rescheduling_interrupts
6973 ± 31% -88.0% 835.00 ± 87% interrupts.CPU53.CAL:Function_call_interrupts
14721 ± 8% -96.4% 527.33 ±187% interrupts.CPU53.RES:Rescheduling_interrupts
7026 ± 32% -88.8% 788.00 ± 79% interrupts.CPU54.CAL:Function_call_interrupts
14677 ± 9% -96.2% 556.44 ±174% interrupts.CPU54.RES:Rescheduling_interrupts
7634 ± 32% -89.0% 842.78 ± 92% interrupts.CPU55.CAL:Function_call_interrupts
15696 ± 9% -96.8% 509.89 ±190% interrupts.CPU55.RES:Rescheduling_interrupts
6284 ± 31% -87.3% 799.33 ± 85% interrupts.CPU56.CAL:Function_call_interrupts
15414 ± 9% -96.6% 518.56 ±184% interrupts.CPU56.RES:Rescheduling_interrupts
6621 ± 28% -86.0% 925.33 ± 93% interrupts.CPU57.CAL:Function_call_interrupts
15789 ± 8% -96.7% 516.22 ±180% interrupts.CPU57.RES:Rescheduling_interrupts
6493 ± 32% -86.5% 879.22 ± 93% interrupts.CPU58.CAL:Function_call_interrupts
15448 ± 9% -96.4% 560.22 ±171% interrupts.CPU58.RES:Rescheduling_interrupts
6836 ± 31% -88.0% 821.44 ± 81% interrupts.CPU59.CAL:Function_call_interrupts
15137 ± 9% -96.6% 510.00 ±184% interrupts.CPU59.RES:Rescheduling_interrupts
6730 ± 31% -86.8% 890.67 ± 71% interrupts.CPU6.CAL:Function_call_interrupts
14824 ± 8% -96.5% 521.78 ±187% interrupts.CPU6.RES:Rescheduling_interrupts
6688 ± 29% -88.3% 785.11 ± 87% interrupts.CPU60.CAL:Function_call_interrupts
14965 ± 9% -96.7% 501.22 ±189% interrupts.CPU60.RES:Rescheduling_interrupts
6277 ± 28% -86.8% 828.33 ± 82% interrupts.CPU61.CAL:Function_call_interrupts
15174 ± 8% -96.6% 522.11 ±184% interrupts.CPU61.RES:Rescheduling_interrupts
7223 ± 32% -88.0% 868.89 ± 89% interrupts.CPU62.CAL:Function_call_interrupts
15871 ± 8% -96.5% 555.33 ±183% interrupts.CPU62.RES:Rescheduling_interrupts
6144 ± 31% -85.5% 892.56 ± 99% interrupts.CPU63.CAL:Function_call_interrupts
15534 ± 9% -96.7% 510.67 ±191% interrupts.CPU63.RES:Rescheduling_interrupts
5792 ± 30% -85.4% 845.78 ± 88% interrupts.CPU64.CAL:Function_call_interrupts
15091 ± 8% -96.6% 520.00 ±186% interrupts.CPU64.RES:Rescheduling_interrupts
6073 ± 30% -87.1% 780.44 ± 73% interrupts.CPU65.CAL:Function_call_interrupts
14621 ± 8% -96.5% 509.33 ±187% interrupts.CPU65.RES:Rescheduling_interrupts
6771 ± 32% -89.1% 741.11 ± 67% interrupts.CPU66.CAL:Function_call_interrupts
14985 ± 8% -96.6% 506.67 ±182% interrupts.CPU66.RES:Rescheduling_interrupts
6382 ± 31% -88.6% 726.33 ± 53% interrupts.CPU67.CAL:Function_call_interrupts
15625 ± 9% -96.5% 548.11 ±171% interrupts.CPU67.RES:Rescheduling_interrupts
5234 ± 29% -86.3% 716.00 ± 61% interrupts.CPU68.CAL:Function_call_interrupts
15195 ± 8% -96.7% 506.67 ±186% interrupts.CPU68.RES:Rescheduling_interrupts
5280 ± 29% -85.9% 743.33 ± 67% interrupts.CPU69.CAL:Function_call_interrupts
14968 ± 8% -96.7% 491.67 ±184% interrupts.CPU69.RES:Rescheduling_interrupts
6773 ± 30% -87.2% 866.33 ± 64% interrupts.CPU7.CAL:Function_call_interrupts
15574 ± 8% -96.5% 546.22 ±178% interrupts.CPU7.RES:Rescheduling_interrupts
5430 ± 30% -87.1% 698.33 ± 55% interrupts.CPU70.CAL:Function_call_interrupts
14736 ± 8% -96.6% 502.67 ±188% interrupts.CPU70.RES:Rescheduling_interrupts
5586 ± 30% -86.9% 733.33 ± 66% interrupts.CPU71.CAL:Function_call_interrupts
14381 ± 8% -96.4% 515.00 ±183% interrupts.CPU71.RES:Rescheduling_interrupts
5484 ± 28% -89.6% 572.89 ± 4% interrupts.CPU72.CAL:Function_call_interrupts
14585 ± 7% -98.3% 243.89 ±104% interrupts.CPU72.RES:Rescheduling_interrupts
5000 ± 28% -88.3% 587.44 ± 7% interrupts.CPU73.CAL:Function_call_interrupts
15114 ± 7% -98.5% 222.00 ± 81% interrupts.CPU73.RES:Rescheduling_interrupts
4267 ± 25% -86.2% 589.22 ± 6% interrupts.CPU74.CAL:Function_call_interrupts
14751 ± 7% -98.9% 165.11 ± 31% interrupts.CPU74.RES:Rescheduling_interrupts
4886 ± 27% -87.9% 590.00 ± 6% interrupts.CPU75.CAL:Function_call_interrupts
14714 ± 7% -98.5% 216.89 ± 56% interrupts.CPU75.RES:Rescheduling_interrupts
5130 ± 27% -88.4% 594.22 ± 7% interrupts.CPU76.CAL:Function_call_interrupts
14728 ± 7% -98.6% 200.44 ± 79% interrupts.CPU76.RES:Rescheduling_interrupts
4930 ± 26% -88.3% 578.00 ± 6% interrupts.CPU77.CAL:Function_call_interrupts
14546 ± 8% -99.0% 145.78 ± 26% interrupts.CPU77.RES:Rescheduling_interrupts
5099 ± 28% -88.3% 596.11 ± 6% interrupts.CPU78.CAL:Function_call_interrupts
14284 ± 6% -98.8% 176.33 ± 27% interrupts.CPU78.RES:Rescheduling_interrupts
4989 ± 28% -88.3% 585.00 ± 6% interrupts.CPU79.CAL:Function_call_interrupts
15208 ± 6% -99.0% 153.78 ± 27% interrupts.CPU79.RES:Rescheduling_interrupts
6482 ± 30% -87.0% 845.22 ± 87% interrupts.CPU8.CAL:Function_call_interrupts
15611 ± 8% -96.5% 540.89 ±189% interrupts.CPU8.RES:Rescheduling_interrupts
3949 ± 26% -85.2% 584.44 ± 6% interrupts.CPU80.CAL:Function_call_interrupts
14489 ± 6% -98.9% 152.67 ± 26% interrupts.CPU80.RES:Rescheduling_interrupts
4696 ± 26% -87.7% 577.11 ± 5% interrupts.CPU81.CAL:Function_call_interrupts
14708 ± 7% -99.0% 140.89 ± 29% interrupts.CPU81.RES:Rescheduling_interrupts
4789 ± 28% -87.7% 586.78 ± 7% interrupts.CPU82.CAL:Function_call_interrupts
14419 ± 7% -98.9% 154.44 ± 34% interrupts.CPU82.RES:Rescheduling_interrupts
4675 ± 26% -87.3% 592.00 ± 7% interrupts.CPU83.CAL:Function_call_interrupts
13695 ± 6% -98.9% 152.11 ± 30% interrupts.CPU83.RES:Rescheduling_interrupts
4887 ± 27% -87.5% 609.44 ± 9% interrupts.CPU84.CAL:Function_call_interrupts
14664 ± 7% -98.5% 215.00 ± 75% interrupts.CPU84.RES:Rescheduling_interrupts
4378 ± 25% -86.4% 594.44 ± 9% interrupts.CPU85.CAL:Function_call_interrupts
14611 ± 7% -98.9% 159.56 ± 41% interrupts.CPU85.RES:Rescheduling_interrupts
4550 ± 25% -86.9% 596.67 ± 11% interrupts.CPU86.CAL:Function_call_interrupts
14946 ± 7% -98.8% 173.22 ± 43% interrupts.CPU86.RES:Rescheduling_interrupts
4597 ± 28% -87.1% 592.11 ± 6% interrupts.CPU87.CAL:Function_call_interrupts
14568 ± 7% -99.0% 143.11 ± 23% interrupts.CPU87.RES:Rescheduling_interrupts
4751 ± 28% -87.9% 575.00 ± 4% interrupts.CPU88.CAL:Function_call_interrupts
14181 ± 7% -98.9% 153.56 ± 39% interrupts.CPU88.RES:Rescheduling_interrupts
5186 ± 29% -88.8% 579.00 ± 6% interrupts.CPU89.CAL:Function_call_interrupts
14221 ± 7% -98.8% 167.56 ± 25% interrupts.CPU89.RES:Rescheduling_interrupts
6726 ± 31% -85.9% 948.78 ± 89% interrupts.CPU9.CAL:Function_call_interrupts
15702 ± 8% -96.7% 514.89 ±196% interrupts.CPU9.RES:Rescheduling_interrupts
4819 ± 26% -87.9% 582.11 ± 6% interrupts.CPU90.CAL:Function_call_interrupts
14872 ± 7% -98.9% 167.44 ± 51% interrupts.CPU90.RES:Rescheduling_interrupts
5325 ± 29% -88.8% 597.22 ± 5% interrupts.CPU91.CAL:Function_call_interrupts
14976 ± 7% -98.7% 199.56 ± 43% interrupts.CPU91.RES:Rescheduling_interrupts
4699 ± 29% -87.7% 579.67 ± 5% interrupts.CPU92.CAL:Function_call_interrupts
14981 ± 6% -99.1% 138.78 ± 23% interrupts.CPU92.RES:Rescheduling_interrupts
4193 ± 25% -86.0% 585.22 ± 7% interrupts.CPU93.CAL:Function_call_interrupts
14511 ± 6% -98.9% 161.11 ± 31% interrupts.CPU93.RES:Rescheduling_interrupts
4308 ± 26% -86.5% 582.22 ± 5% interrupts.CPU94.CAL:Function_call_interrupts
14032 ± 7% -98.4% 219.56 ± 87% interrupts.CPU94.RES:Rescheduling_interrupts
4275 ± 27% -85.8% 605.44 ± 6% interrupts.CPU95.CAL:Function_call_interrupts
13456 ± 7% -98.8% 162.33 ± 26% interrupts.CPU95.RES:Rescheduling_interrupts
1424994 ± 7% -97.6% 34084 ±140% interrupts.RES:Rescheduling_interrupts
628.86 ± 13% -67.5% 204.33 ± 23% interrupts.TLB:TLB_shootdowns
14.97 ± 3% -15.0 0.00 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
14.80 ± 3% -14.8 0.00 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
14.79 ± 3% -14.8 0.00 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
14.77 ± 3% -14.8 0.00 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
10.48 ± 4% -10.5 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
10.46 ± 4% -10.5 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
9.82 ± 4% -9.8 0.00 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
9.73 ± 4% -4.7 5.03 ± 2% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
9.83 ± 4% -4.7 5.15 ± 2% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.45 ± 3% -4.6 5.81 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
10.16 ± 3% -4.6 5.54 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
8.47 ± 4% -4.5 3.92 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
8.61 ± 4% -4.5 4.08 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
8.24 ± 4% -4.4 3.86 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
8.12 ± 4% -4.3 3.82 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
7.20 ± 3% -3.9 3.32 ± 2% perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
7.36 ± 3% -3.9 3.48 ± 2% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
7.64 ± 3% -3.8 3.83 ± 2% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
7.81 ± 3% -3.8 4.03 ± 2% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
9.27 ± 2% -3.7 5.52 ± 6% perf-profile.calltrace.cycles-pp.read
8.43 ± 3% -3.7 4.75 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
8.33 ± 3% -3.7 4.66 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
4.90 ± 4% -3.1 1.77 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.new_sync_read.vfs_read
4.93 ± 4% -3.1 1.81 ± 3% perf-profile.calltrace.cycles-pp.schedule.pipe_read.new_sync_read.vfs_read.ksys_read
10.61 ± 3% -3.0 7.66 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
10.66 ± 3% -2.9 7.76 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
19.90 -2.4 17.47 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.process_vm_rw_core.process_vm_rw
10.89 ± 3% -2.3 8.55 ± 3% perf-profile.calltrace.cycles-pp.write
20.13 -2.3 17.86 ± 2% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv
2.69 ± 4% -1.8 0.89 ± 3% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.new_sync_read
23.16 -1.4 21.78 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv.do_syscall_64
2.11 ± 5% -1.4 0.74 ± 3% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
2.83 ± 5% -1.3 1.56 ± 3% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
2.80 ± 5% -1.3 1.55 ± 3% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
1.56 ± 46% -1.0 0.57 ± 3% perf-profile.calltrace.cycles-pp.stress_vm_child
1.93 ± 2% -0.8 1.12 perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.30 ± 4% -0.7 0.58 ± 6% perf-profile.calltrace.cycles-pp.__might_sleep.copy_page_to_iter.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv
1.60 ± 2% -0.7 0.95 perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
1.22 ± 3% -0.5 0.72 perf-profile.calltrace.cycles-pp.select_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
0.56 +0.5 1.04 ± 2% perf-profile.calltrace.cycles-pp.__might_fault.copy_page_to_iter.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv
0.00 +0.5 0.54 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode
0.46 ± 41% +0.8 1.22 ± 2% perf-profile.calltrace.cycles-pp.put_compound_head.unpin_user_pages.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv
0.56 ± 5% +0.8 1.36 ± 2% perf-profile.calltrace.cycles-pp.put_compound_head.unpin_user_pages_dirty_lock.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_writev
0.00 +0.8 0.82 ± 2% perf-profile.calltrace.cycles-pp.mod_node_page_state.put_compound_head.unpin_user_pages.process_vm_rw_core.process_vm_rw
0.07 ±244% +0.9 0.96 ± 2% perf-profile.calltrace.cycles-pp.__might_fault.copy_page_from_iter.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_writev
0.00 +0.9 0.90 ± 2% perf-profile.calltrace.cycles-pp.mod_node_page_state.put_compound_head.unpin_user_pages_dirty_lock.process_vm_rw_core.process_vm_rw
0.97 ± 4% +1.0 1.98 ± 4% perf-profile.calltrace.cycles-pp.put_page_refs.unpin_user_pages_dirty_lock.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_writev
0.96 ± 4% +1.0 1.99 ± 5% perf-profile.calltrace.cycles-pp.put_page_refs.unpin_user_pages.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv
0.00 +1.1 1.15 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.00 +1.2 1.19 ± 2% perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.20 ± 4% +1.2 2.43 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.follow_page_pte.__get_user_pages.__get_user_pages_remote.process_vm_rw_core
0.00 +1.3 1.32 ± 5% perf-profile.calltrace.cycles-pp.mark_page_accessed.follow_page_pte.__get_user_pages.__get_user_pages_remote.process_vm_rw_core
0.00 +1.4 1.36 ± 3% perf-profile.calltrace.cycles-pp.mod_node_page_state.try_grab_compound_head.follow_page_pte.__get_user_pages.__get_user_pages_remote
0.00 +1.4 1.44 ± 2% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
0.00 +1.4 1.44 ± 5% perf-profile.calltrace.cycles-pp.___might_sleep.__get_user_pages.__get_user_pages_remote.process_vm_rw_core.process_vm_rw
0.00 +1.7 1.70 ± 23% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
1.30 ± 6% +1.8 3.12 ± 3% perf-profile.calltrace.cycles-pp.follow_pmd_mask.__get_user_pages.__get_user_pages_remote.process_vm_rw_core.process_vm_rw
1.75 ± 4% +1.9 3.63 ± 4% perf-profile.calltrace.cycles-pp.unpin_user_pages.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv.do_syscall_64
15.35 +2.0 17.30 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.process_vm_rw_core.process_vm_rw
1.84 ± 4% +2.0 3.86 ± 3% perf-profile.calltrace.cycles-pp.unpin_user_pages_dirty_lock.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_writev.do_syscall_64
4.92 +2.1 7.04 ± 3% perf-profile.calltrace.cycles-pp.try_grab_compound_head.follow_page_pte.__get_user_pages.__get_user_pages_remote.process_vm_rw_core
1.46 ± 6% +2.2 3.61 ± 3% perf-profile.calltrace.cycles-pp.follow_page_mask.__get_user_pages.__get_user_pages_remote.process_vm_rw_core.process_vm_rw
15.56 +2.2 17.74 perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_writev
17.70 +3.4 21.06 perf-profile.calltrace.cycles-pp.copy_page_from_iter.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_writev.do_syscall_64
7.38 ± 2% +5.6 12.94 ± 2% perf-profile.calltrace.cycles-pp.__get_user_pages.__get_user_pages_remote.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv
7.47 ± 2% +5.7 13.16 ± 2% perf-profile.calltrace.cycles-pp.__get_user_pages_remote.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv.do_syscall_64
8.23 ± 2% +6.0 14.18 ± 3% perf-profile.calltrace.cycles-pp.follow_page_pte.__get_user_pages.__get_user_pages_remote.process_vm_rw_core.process_vm_rw
33.77 +6.6 40.40 ± 2% perf-profile.calltrace.cycles-pp.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_readv.do_syscall_64.entry_SYSCALL_64_after_hwframe
34.14 +6.9 41.08 ± 2% perf-profile.calltrace.cycles-pp.process_vm_rw.__x64_sys_process_vm_readv.do_syscall_64.entry_SYSCALL_64_after_hwframe.process_vm_readv
34.17 +7.0 41.15 ± 2% perf-profile.calltrace.cycles-pp.__x64_sys_process_vm_readv.do_syscall_64.entry_SYSCALL_64_after_hwframe.process_vm_readv
5.66 ± 5% +7.0 12.69 ± 3% perf-profile.calltrace.cycles-pp.__get_user_pages.__get_user_pages_remote.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_writev
5.75 ± 5% +7.1 12.90 ± 3% perf-profile.calltrace.cycles-pp.__get_user_pages_remote.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_writev.do_syscall_64
34.30 +7.4 41.68 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.process_vm_readv
34.38 +7.5 41.84 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.process_vm_readv
34.72 +7.7 42.39 perf-profile.calltrace.cycles-pp.process_vm_readv
26.43 ± 2% +13.2 39.67 ± 2% perf-profile.calltrace.cycles-pp.process_vm_rw_core.process_vm_rw.__x64_sys_process_vm_writev.do_syscall_64.entry_SYSCALL_64_after_hwframe
26.81 ± 2% +13.7 40.46 ± 2% perf-profile.calltrace.cycles-pp.process_vm_rw.__x64_sys_process_vm_writev.do_syscall_64.entry_SYSCALL_64_after_hwframe.process_vm_writev
26.84 ± 2% +13.7 40.54 ± 2% perf-profile.calltrace.cycles-pp.__x64_sys_process_vm_writev.do_syscall_64.entry_SYSCALL_64_after_hwframe.process_vm_writev
26.91 ± 2% +14.1 41.05 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.process_vm_writev
26.95 ± 2% +14.2 41.13 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.process_vm_writev
27.23 ± 2% +14.4 41.66 perf-profile.calltrace.cycles-pp.process_vm_writev
14.97 ± 3% -14.9 0.05 ±282% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
14.97 ± 3% -14.9 0.05 ±282% perf-profile.children.cycles-pp.cpu_startup_entry
14.95 ± 3% -14.9 0.05 ±282% perf-profile.children.cycles-pp.do_idle
14.80 ± 3% -14.7 0.05 ±282% perf-profile.children.cycles-pp.start_secondary
10.60 ± 4% -10.6 0.04 ±282% perf-profile.children.cycles-pp.cpuidle_enter
10.59 ± 4% -10.6 0.04 ±282% perf-profile.children.cycles-pp.cpuidle_enter_state
9.93 ± 4% -9.9 0.03 ±282% perf-profile.children.cycles-pp.intel_idle
7.99 ± 4% -5.0 2.94 ± 2% perf-profile.children.cycles-pp.__schedule
9.74 ± 4% -4.7 5.04 ± 2% perf-profile.children.cycles-pp.pipe_write
9.84 ± 4% -4.7 5.16 ± 2% perf-profile.children.cycles-pp.new_sync_write
10.46 ± 3% -4.6 5.82 perf-profile.children.cycles-pp.ksys_write
10.17 ± 3% -4.6 5.55 ± 2% perf-profile.children.cycles-pp.vfs_write
8.47 ± 4% -4.5 3.93 ± 2% perf-profile.children.cycles-pp.__wake_up_common
8.61 ± 4% -4.5 4.09 ± 2% perf-profile.children.cycles-pp.__wake_up_common_lock
8.25 ± 4% -4.4 3.86 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function
8.13 ± 4% -4.3 3.82 ± 2% perf-profile.children.cycles-pp.try_to_wake_up
7.36 ± 3% -3.9 3.49 ± 2% perf-profile.children.cycles-pp.new_sync_read
7.21 ± 3% -3.9 3.35 ± 2% perf-profile.children.cycles-pp.pipe_read
7.69 ± 3% -3.8 3.84 ± 2% perf-profile.children.cycles-pp.vfs_read
7.81 ± 3% -3.8 4.03 ± 2% perf-profile.children.cycles-pp.ksys_read
9.37 ± 2% -3.8 5.62 ± 6% perf-profile.children.cycles-pp.read
20.45 -2.4 18.05 ± 2% perf-profile.children.cycles-pp.copyout
10.94 ± 3% -2.3 8.64 ± 3% perf-profile.children.cycles-pp.write
4.97 ± 3% -2.0 3.01 ± 2% perf-profile.children.cycles-pp.schedule
2.67 ± 3% -1.8 0.84 ± 4% perf-profile.children.cycles-pp.update_load_avg
2.70 ± 4% -1.8 0.89 ± 3% perf-profile.children.cycles-pp.dequeue_task_fair
2.17 ± 5% -1.8 0.39 ± 4% perf-profile.children.cycles-pp.dequeue_entity
23.61 -1.4 22.23 ± 2% perf-profile.children.cycles-pp.copy_page_to_iter
2.13 ± 4% -1.4 0.77 ± 3% perf-profile.children.cycles-pp.enqueue_entity
2.84 ± 5% -1.3 1.56 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate
2.81 ± 5% -1.3 1.55 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair
1.44 ± 8% -1.2 0.23 ± 9% perf-profile.children.cycles-pp.update_cfs_group
1.57 ± 47% -1.0 0.57 ± 3% perf-profile.children.cycles-pp.stress_vm_child
1.93 ± 2% -0.8 1.12 perf-profile.children.cycles-pp.select_task_rq_fair
3.59 ± 4% -0.7 2.88 ± 3% perf-profile.children.cycles-pp._raw_spin_lock
1.61 ± 2% -0.6 0.96 perf-profile.children.cycles-pp.select_idle_sibling
0.85 ± 3% -0.5 0.32 ± 4% perf-profile.children.cycles-pp.set_next_entity
1.23 ± 3% -0.5 0.73 perf-profile.children.cycles-pp.select_idle_cpu
0.76 ± 3% -0.5 0.27 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.45 ± 3% -0.4 0.04 ± 55% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.62 ± 3% -0.4 0.22 ± 5% perf-profile.children.cycles-pp.prepare_to_wait_event
0.82 ± 3% -0.4 0.42 perf-profile.children.cycles-pp.available_idle_cpu
1.22 ± 3% -0.4 0.87 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair
1.17 ± 3% -0.3 0.85 ± 5% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
1.07 ± 3% -0.3 0.78 ± 5% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.39 ± 6% -0.3 0.10 ± 8% perf-profile.children.cycles-pp.finish_task_switch
2.21 -0.3 1.94 ± 2% perf-profile.children.cycles-pp.__might_sleep
0.42 ± 5% -0.2 0.21 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.43 ± 3% -0.2 0.22 ± 5% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.36 ± 7% -0.2 0.16 ± 8% perf-profile.children.cycles-pp.scheduler_tick
0.49 ± 6% -0.2 0.29 ± 6% perf-profile.children.cycles-pp.tick_sched_timer
0.47 ± 6% -0.2 0.27 ± 6% perf-profile.children.cycles-pp.tick_sched_handle
0.58 ± 5% -0.2 0.38 ± 5% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.47 ± 6% -0.2 0.27 ± 6% perf-profile.children.cycles-pp.update_process_times
0.30 ± 9% -0.2 0.10 ± 7% perf-profile.children.cycles-pp.task_tick_fair
0.47 ± 2% -0.2 0.28 ± 3% perf-profile.children.cycles-pp.__switch_to_asm
0.49 ± 4% -0.2 0.30 ± 2% perf-profile.children.cycles-pp.update_rq_clock
0.43 ± 3% -0.1 0.28 perf-profile.children.cycles-pp.__switch_to
0.50 -0.1 0.37 ± 2% perf-profile.children.cycles-pp.copy_user_generic_unrolled
0.33 ± 2% -0.1 0.21 ± 4% perf-profile.children.cycles-pp.___perf_sw_event
0.17 ± 4% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.__list_del_entry_valid
0.15 ± 4% -0.1 0.06 ± 5% perf-profile.children.cycles-pp.cpumask_next
0.18 ± 4% -0.1 0.09 ± 6% perf-profile.children.cycles-pp.resched_curr
0.21 ± 3% -0.1 0.12 ± 3% perf-profile.children.cycles-pp.perf_tp_event
0.19 ± 3% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.sched_clock_cpu
0.17 ± 4% -0.1 0.09 ± 4% perf-profile.children.cycles-pp._find_next_bit
0.25 ± 5% -0.1 0.18 ± 2% perf-profile.children.cycles-pp.__update_load_avg_se
0.22 ± 3% -0.1 0.15 ± 4% perf-profile.children.cycles-pp.mutex_unlock
0.16 ± 3% -0.1 0.10 ± 5% perf-profile.children.cycles-pp.native_sched_clock
0.11 ± 5% -0.1 0.05 perf-profile.children.cycles-pp.perf_trace_buf_update
0.18 ± 3% -0.1 0.12 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.13 ± 8% -0.1 0.07 ± 6% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.15 ± 4% -0.0 0.10 ± 8% perf-profile.children.cycles-pp.anon_pipe_buf_release
0.12 ± 8% -0.0 0.09 ± 6% perf-profile.children.cycles-pp.update_min_vruntime
0.32 ± 2% -0.0 0.30 ± 2% perf-profile.children.cycles-pp.__restore_fpregs_from_fpstate
0.11 ± 3% -0.0 0.09 perf-profile.children.cycles-pp.aa_file_perm
0.06 ± 5% +0.0 0.07 ± 4% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.07 ± 6% +0.0 0.09 ± 7% perf-profile.children.cycles-pp.down_read
0.09 ± 5% +0.0 0.11 ± 3% perf-profile.children.cycles-pp.get_task_mm
0.13 ± 5% +0.0 0.16 ± 3% perf-profile.children.cycles-pp.mmput
0.07 ± 6% +0.0 0.11 ± 4% perf-profile.children.cycles-pp.cpumask_next_wrap
0.05 ± 6% +0.0 0.08 ± 8% perf-profile.children.cycles-pp.kmalloc_slab
0.10 ± 7% +0.0 0.13 ± 6% perf-profile.children.cycles-pp.pick_next_entity
0.06 ± 5% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.up_read
0.09 ± 5% +0.0 0.13 ± 5% perf-profile.children.cycles-pp.current_time
0.11 ± 9% +0.0 0.14 ± 8% perf-profile.children.cycles-pp.find_extend_vma
0.04 ± 63% +0.0 0.08 ± 5% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
0.22 ± 3% +0.0 0.26 ± 2% perf-profile.children.cycles-pp.touch_atime
0.00 +0.1 0.05 perf-profile.children.cycles-pp.timestamp_truncate
0.16 ± 5% +0.1 0.21 ± 6% perf-profile.children.cycles-pp.ktime_get
0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
0.06 ± 8% +0.1 0.12 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.00 +0.1 0.06 ± 8% perf-profile.children.cycles-pp.__cgroup_account_cputime
0.04 ± 41% +0.1 0.10 ± 7% perf-profile.children.cycles-pp.__calc_delta
0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.rb_erase
0.15 ± 4% +0.1 0.21 ± 2% perf-profile.children.cycles-pp.atime_needs_update
0.36 ± 2% +0.1 0.42 ± 2% perf-profile.children.cycles-pp.security_file_permission
0.00 +0.1 0.06 ± 10% perf-profile.children.cycles-pp.check_vma_flags
0.25 ± 4% +0.1 0.31 ± 2% perf-profile.children.cycles-pp.mm_access
0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.perf_trace_sched_switch
0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.map_id_range_down
0.00 +0.1 0.07 ± 8% perf-profile.children.cycles-pp.check_stack_object
0.09 ± 5% +0.1 0.18 ± 2% perf-profile.children.cycles-pp.__check_object_size
0.00 +0.1 0.10 ± 6% perf-profile.children.cycles-pp.set_next_buddy
0.12 ± 5% +0.1 0.24 ± 3% perf-profile.children.cycles-pp.kfree
0.20 ± 4% +0.1 0.32 ± 3% perf-profile.children.cycles-pp.__radix_tree_lookup
0.11 ± 8% +0.1 0.22 ± 3% perf-profile.children.cycles-pp.save_fpregs_to_fpstate
0.10 ± 6% +0.1 0.22 ± 4% perf-profile.children.cycles-pp.pud_huge
0.53 ± 2% +0.1 0.65 ± 2% perf-profile.children.cycles-pp.find_get_task_by_vpid
0.09 ± 6% +0.1 0.23 ± 6% perf-profile.children.cycles-pp.clockevents_program_event
0.11 ± 4% +0.1 0.24 ± 3% perf-profile.children.cycles-pp.follow_huge_addr
0.37 ± 2% +0.2 0.53 ± 2% perf-profile.children.cycles-pp.find_task_by_vpid
0.03 ± 87% +0.2 0.20 ± 4% perf-profile.children.cycles-pp.put_prev_entity
0.14 ± 5% +0.2 0.34 ± 4% perf-profile.children.cycles-pp.pmd_huge
0.26 ± 2% +0.2 0.50 ± 2% perf-profile.children.cycles-pp.__kmalloc
0.20 ± 6% +0.2 0.44 ± 2% perf-profile.children.cycles-pp.reweight_entity
0.00 +0.2 0.25 ± 6% perf-profile.children.cycles-pp.check_preempt_wakeup
0.20 ± 7% +0.3 0.46 ± 5% perf-profile.children.cycles-pp.vm_normal_page
0.34 ± 3% +0.3 0.66 ± 2% perf-profile.children.cycles-pp.__import_iovec
0.33 ± 3% +0.3 0.67 ± 3% perf-profile.children.cycles-pp.rcu_all_qs
0.35 ± 3% +0.3 0.70 ± 2% perf-profile.children.cycles-pp.import_iovec
0.35 ± 4% +0.4 0.71 ± 2% perf-profile.children.cycles-pp._copy_from_user
0.35 ± 5% +0.4 0.72 perf-profile.children.cycles-pp.syscall_return_via_sysret
0.35 ± 4% +0.4 0.74 perf-profile.children.cycles-pp.__entry_text_start
0.85 ± 2% +0.4 1.29 ± 3% perf-profile.children.cycles-pp.__cond_resched
0.52 ± 3% +0.5 1.04 ± 2% perf-profile.children.cycles-pp.iovec_from_user
0.54 ± 6% +0.9 1.39 ± 5% perf-profile.children.cycles-pp.mark_page_accessed
1.28 ± 3% +1.2 2.49 perf-profile.children.cycles-pp.__might_fault
0.52 ± 10% +1.3 1.79 perf-profile.children.cycles-pp.exit_to_user_mode_prepare
1.11 ± 5% +1.5 2.62 ± 2% perf-profile.children.cycles-pp.put_compound_head
2.23 ± 3% +1.8 3.98 ± 3% perf-profile.children.cycles-pp.___might_sleep
1.35 ± 5% +1.8 3.15 ± 2% perf-profile.children.cycles-pp.mod_node_page_state
1.33 ± 6% +1.8 3.18 ± 3% perf-profile.children.cycles-pp.follow_pmd_mask
1.77 ± 4% +1.9 3.66 ± 4% perf-profile.children.cycles-pp.unpin_user_pages
1.93 ± 5% +2.0 3.97 ± 4% perf-profile.children.cycles-pp.put_page_refs
1.85 ± 4% +2.0 3.89 ± 3% perf-profile.children.cycles-pp.unpin_user_pages_dirty_lock
4.96 +2.1 7.11 ± 3% perf-profile.children.cycles-pp.try_grab_compound_head
15.76 +2.2 17.93 perf-profile.children.cycles-pp.copyin
1.51 ± 5% +2.2 3.72 ± 3% perf-profile.children.cycles-pp.follow_page_mask
0.60 ± 9% +2.5 3.11 ± 69% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
18.31 +3.2 21.50 perf-profile.children.cycles-pp.copy_page_from_iter
8.38 ± 2% +6.2 14.58 ± 3% perf-profile.children.cycles-pp.follow_page_pte
34.18 +7.0 41.15 ± 2% perf-profile.children.cycles-pp.__x64_sys_process_vm_readv
34.79 +7.7 42.53 perf-profile.children.cycles-pp.process_vm_readv
13.18 ± 3% +12.8 25.97 ± 3% perf-profile.children.cycles-pp.__get_user_pages
13.22 ± 3% +12.8 26.06 ± 3% perf-profile.children.cycles-pp.__get_user_pages_remote
26.85 ± 2% +13.7 40.55 ± 2% perf-profile.children.cycles-pp.__x64_sys_process_vm_writev
27.29 ± 2% +14.5 41.80 perf-profile.children.cycles-pp.process_vm_writev
80.22 +14.9 95.13 perf-profile.children.cycles-pp.do_syscall_64
80.48 +15.1 95.56 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
60.23 +19.9 80.12 ± 2% perf-profile.children.cycles-pp.process_vm_rw_core
60.96 +20.6 81.56 ± 2% perf-profile.children.cycles-pp.process_vm_rw
9.93 ± 4% -9.9 0.03 ±282% perf-profile.self.cycles-pp.intel_idle
1.81 ± 4% -1.4 0.44 ± 5% perf-profile.self.cycles-pp.update_load_avg
1.44 ± 8% -1.2 0.23 ± 9% perf-profile.self.cycles-pp.update_cfs_group
1.51 ± 47% -1.0 0.56 ± 3% perf-profile.self.cycles-pp.stress_vm_child
1.29 ± 3% -0.9 0.43 ± 2% perf-profile.self.cycles-pp.__schedule
0.75 ± 2% -0.5 0.26 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.81 ± 3% -0.4 0.41 ± 2% perf-profile.self.cycles-pp.available_idle_cpu
2.10 -0.3 1.78 ± 2% perf-profile.self.cycles-pp.__might_sleep
0.33 ± 4% -0.3 0.06 ± 8% perf-profile.self.cycles-pp.set_next_entity
0.26 ± 4% -0.2 0.04 ± 71% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.41 ± 5% -0.2 0.20 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.42 ± 3% -0.2 0.22 ± 4% perf-profile.self.cycles-pp.pipe_read
0.39 ± 6% -0.2 0.19 ± 3% perf-profile.self.cycles-pp.update_rq_clock
0.47 ± 2% -0.2 0.28 ± 3% perf-profile.self.cycles-pp.__switch_to_asm
0.24 ± 5% -0.2 0.07 ± 7% perf-profile.self.cycles-pp.finish_task_switch
0.23 ± 5% -0.2 0.06 ± 7% perf-profile.self.cycles-pp.__wake_up_common
0.41 ± 2% -0.2 0.24 ± 5% perf-profile.self.cycles-pp.select_idle_cpu
0.30 ± 4% -0.2 0.14 ± 5% perf-profile.self.cycles-pp.try_to_wake_up
0.35 ± 5% -0.2 0.20 ± 6% perf-profile.self.cycles-pp.enqueue_entity
0.42 ± 3% -0.1 0.27 ± 2% perf-profile.self.cycles-pp.__switch_to
0.49 ± 2% -0.1 0.35 perf-profile.self.cycles-pp.copy_user_generic_unrolled
0.25 ± 3% -0.1 0.12 ± 6% perf-profile.self.cycles-pp.prepare_to_wait_event
0.16 ± 6% -0.1 0.03 ± 71% perf-profile.self.cycles-pp.__list_del_entry_valid
0.23 ± 14% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.read
0.29 ± 2% -0.1 0.19 ± 4% perf-profile.self.cycles-pp.___perf_sw_event
0.18 ± 4% -0.1 0.09 ± 6% perf-profile.self.cycles-pp.resched_curr
0.52 ± 5% -0.1 0.44 ± 3% perf-profile.self.cycles-pp.update_curr
0.16 ± 4% -0.1 0.08 ± 5% perf-profile.self.cycles-pp._find_next_bit
0.22 ± 4% -0.1 0.14 ± 4% perf-profile.self.cycles-pp.mutex_unlock
0.25 ± 4% -0.1 0.17 ± 4% perf-profile.self.cycles-pp.__update_load_avg_se
0.29 ± 4% -0.1 0.22 ± 5% perf-profile.self.cycles-pp.mutex_lock
0.18 ± 3% -0.1 0.12 perf-profile.self.cycles-pp._raw_spin_lock_irq
0.15 ± 3% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.native_sched_clock
0.14 ± 5% -0.1 0.09 ± 6% perf-profile.self.cycles-pp.anon_pipe_buf_release
0.14 ± 3% -0.1 0.09 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair
0.12 ± 6% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.dequeue_entity
0.15 ± 4% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.find_get_task_by_vpid
0.12 ± 7% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.update_min_vruntime
0.20 ± 2% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.select_task_rq_fair
0.10 ± 5% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.atime_needs_update
0.32 ± 2% -0.0 0.29 ± 2% perf-profile.self.cycles-pp.__restore_fpregs_from_fpstate
0.10 ± 6% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.perf_tp_event
0.10 ± 6% -0.0 0.08 ± 3% perf-profile.self.cycles-pp.aa_file_perm
0.17 ± 2% -0.0 0.15 ± 4% perf-profile.self.cycles-pp.vfs_read
0.06 ± 5% +0.0 0.07 ± 4% perf-profile.self.cycles-pp.__wrgsbase_inactive
0.13 ± 3% +0.0 0.15 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair
0.10 ± 5% +0.0 0.12 ± 4% perf-profile.self.cycles-pp.switch_fpu_return
0.09 ± 5% +0.0 0.11 ± 5% perf-profile.self.cycles-pp.new_sync_write
0.14 ± 3% +0.0 0.17 ± 2% perf-profile.self.cycles-pp.find_task_by_vpid
0.04 ± 63% +0.0 0.07 ± 10% perf-profile.self.cycles-pp.ksys_read
0.05 +0.0 0.08 ± 9% perf-profile.self.cycles-pp.kmalloc_slab
0.13 ± 5% +0.0 0.16 ± 5% perf-profile.self.cycles-pp.reweight_entity
0.06 ± 14% +0.0 0.10 ± 3% perf-profile.self.cycles-pp.syscall_enter_from_user_mode
0.06 ± 5% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.up_read
0.11 ± 4% +0.0 0.15 ± 3% perf-profile.self.cycles-pp.write
0.12 ± 2% +0.0 0.16 ± 3% perf-profile.self.cycles-pp.vfs_write
0.20 ± 5% +0.0 0.24 ± 3% perf-profile.self.cycles-pp.common_file_perm
0.05 ± 6% +0.1 0.10 ± 4% perf-profile.self.cycles-pp.__check_object_size
0.06 ± 8% +0.1 0.11 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
0.04 ± 41% +0.1 0.10 ± 4% perf-profile.self.cycles-pp.__calc_delta
0.00 +0.1 0.05 ± 9% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.00 +0.1 0.06 ± 12% perf-profile.self.cycles-pp.file_update_time
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.rb_erase
0.01 ±244% +0.1 0.07 ± 7% perf-profile.self.cycles-pp.cpumask_next_wrap
0.00 +0.1 0.06 ± 5% perf-profile.self.cycles-pp.check_stack_object
0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.__x64_sys_process_vm_readv
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.put_prev_entity
0.01 ±244% +0.1 0.07 ± 6% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare
0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.06 +0.1 0.13 ± 4% perf-profile.self.cycles-pp.__import_iovec
0.00 +0.1 0.07 ± 8% perf-profile.self.cycles-pp.security_file_permission
0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.map_id_range_down
0.00 +0.1 0.08 ± 6% perf-profile.self.cycles-pp.schedule
0.02 ±115% +0.1 0.10 ± 4% perf-profile.self.cycles-pp._copy_from_user
0.16 ± 3% +0.1 0.24 perf-profile.self.cycles-pp.process_vm_readv
0.00 +0.1 0.08 ± 5% perf-profile.self.cycles-pp.__x64_sys_process_vm_writev
0.01 ±244% +0.1 0.09 ± 8% perf-profile.self.cycles-pp.__get_user_pages_remote
0.00 +0.1 0.09 ± 9% perf-profile.self.cycles-pp.check_preempt_wakeup
0.12 ± 5% +0.1 0.22 ± 5% perf-profile.self.cycles-pp.process_vm_rw
0.08 ± 6% +0.1 0.17 ± 5% perf-profile.self.cycles-pp.iovec_from_user
0.00 +0.1 0.10 ± 6% perf-profile.self.cycles-pp.set_next_buddy
0.07 ± 9% +0.1 0.17 ± 3% perf-profile.self.cycles-pp.follow_huge_addr
0.12 ± 7% +0.1 0.23 ± 3% perf-profile.self.cycles-pp.process_vm_writev
0.09 ± 11% +0.1 0.20 ± 8% perf-profile.self.cycles-pp.ktime_get
0.12 ± 5% +0.1 0.23 ± 4% perf-profile.self.cycles-pp.kfree
0.20 ± 3% +0.1 0.31 ± 3% perf-profile.self.cycles-pp.__radix_tree_lookup
0.11 ± 6% +0.1 0.22 ± 3% perf-profile.self.cycles-pp.save_fpregs_to_fpstate
0.11 ± 6% +0.1 0.22 ± 7% perf-profile.self.cycles-pp.copyout
0.09 ± 5% +0.1 0.21 ± 4% perf-profile.self.cycles-pp.pud_huge
0.06 ± 16% +0.1 0.18 ± 8% perf-profile.self.cycles-pp.exit_to_user_mode_prepare
0.12 ± 8% +0.1 0.26 ± 7% perf-profile.self.cycles-pp.copyin
0.15 ± 4% +0.1 0.29 ± 2% perf-profile.self.cycles-pp.__kmalloc
0.28 ± 7% +0.2 0.45 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.11 ± 7% +0.2 0.29 ± 5% perf-profile.self.cycles-pp.pmd_huge
0.28 ± 4% +0.2 0.46 ± 7% perf-profile.self.cycles-pp.unpin_user_pages
0.15 ± 4% +0.2 0.34 ± 4% perf-profile.self.cycles-pp.__entry_text_start
0.31 ± 4% +0.2 0.54 ± 4% perf-profile.self.cycles-pp.unpin_user_pages_dirty_lock
0.20 ± 7% +0.3 0.45 ± 5% perf-profile.self.cycles-pp.vm_normal_page
0.23 ± 4% +0.3 0.49 ± 3% perf-profile.self.cycles-pp.rcu_all_qs
0.26 ± 5% +0.3 0.52 ± 3% perf-profile.self.cycles-pp.__might_fault
0.30 ± 4% +0.3 0.61 ± 3% perf-profile.self.cycles-pp.__cond_resched
0.34 ± 4% +0.4 0.71 perf-profile.self.cycles-pp.syscall_return_via_sysret
0.90 ± 2% +0.5 1.36 perf-profile.self.cycles-pp.copy_page_from_iter
0.35 ± 5% +0.5 0.86 ± 2% perf-profile.self.cycles-pp.put_compound_head
0.85 ± 2% +0.6 1.43 perf-profile.self.cycles-pp.process_vm_rw_core
0.49 ± 6% +0.8 1.25 ± 4% perf-profile.self.cycles-pp.mark_page_accessed
0.90 ± 3% +0.8 1.72 ± 2% perf-profile.self.cycles-pp.copy_page_to_iter
1.78 ± 3% +1.0 2.76 ± 3% perf-profile.self.cycles-pp._raw_spin_lock
0.00 +1.2 1.24 ±175% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
4.31 +1.3 5.64 ± 3% perf-profile.self.cycles-pp.try_grab_compound_head
1.10 ± 5% +1.5 2.61 ± 3% perf-profile.self.cycles-pp.__get_user_pages
2.11 ± 3% +1.6 3.73 ± 3% perf-profile.self.cycles-pp.___might_sleep
1.19 ± 5% +1.6 2.84 ± 3% perf-profile.self.cycles-pp.follow_pmd_mask
1.28 ± 6% +1.7 3.03 ± 2% perf-profile.self.cycles-pp.mod_node_page_state
1.55 ± 4% +1.8 3.39 ± 3% perf-profile.self.cycles-pp.follow_page_pte
1.32 ± 5% +1.9 3.21 ± 3% perf-profile.self.cycles-pp.follow_page_mask
1.91 ± 5% +2.0 3.93 ± 4% perf-profile.self.cycles-pp.put_page_refs
stress-ng.time.percent_of_cpu_this_job_got
10000 +-------------------------------------------------------------------+
9000 |-OO O OO O OO O OO O OO O OO O OO OO O OO O OO O OO |
| |
8000 |.++.+.++ ++ ++.+.++.+.++.+.++.++.+.++.+.++.+.++.+.++.+.++.+.++.|
7000 |-+ : :: : |
| : :: : |
6000 |-+ : : : : |
5000 |-+ : : : : |
4000 |-+ : : : : |
| : : : : |
3000 |-+ : : : : |
2000 |-+ : : : : |
| : : |
1000 |-+ : : |
0 +-------------------------------------------------------------------+
stress-ng.time.minor_page_faults
160000 +------------------------------------------------------------------+
| + +. +. |
140000 |-+ .+.++ +.+ :+ .++.+. .++. .+ .+.+ ++.+.++.+.++.+.+ ++.+.++.|
120000 |+++ : : : : + ++ + + |
| + : : : : |
100000 |-OO O OO:O: O:O: O OO O OO OO O OO O OO OO O OO O O |
| : : : : |
80000 |-+ : : : : |
| : : :: |
60000 |-+ :: :: |
40000 |-+ :: :: |
| : : |
20000 |-+ : : |
| : : |
0 +------------------------------------------------------------------+
stress-ng.time.involuntary_context_switches
1.4e+08 +-----------------------------------------------------------------+
| |
1.2e+08 |-+O OO O OO O O O OO OO O O O OO O O O |
| O O O O O O O O O O |
1e+08 |-+ |
| |
8e+07 |-+ |
| |
6e+07 |-+ |
| |
4e+07 |-+ |
| + |
2e+07 |-+ :: |
|.++. : : +.++. .+ +.+. .+ .+ .+ +.+ |
0 +-----------------------------------------------------------------+
stress-ng.vm-rw.ops
1.4e+08 +-----------------------------------------------------------------+
| |
1.2e+08 |-+O OO O OO OO O OO O OO OO O O O O OO OO O O |
| O O O O O |
1e+08 |-+ |
| |
8e+07 |-+ |
|.+ .+ .+ .+ .+ .+ +.+ |
6e+07 |-++.++.+ +.+ + +.+.++.++.+ +.++.+.++ +.+ +.+.++ +.+.+ +.|
| : : : : |
4e+07 |-+ : : : : |
| : : :: |
2e+07 |-+ :: :: |
| : : |
0 +-----------------------------------------------------------------+
stress-ng.vm-rw.ops_per_sec
2.5e+06 +-----------------------------------------------------------------+
| |
| |
2e+06 |-+O OO O OO OO O OO O OO OO O O O O OO OO O O |
| O O O O O |
| |
1.5e+06 |-+ |
| .+ |
1e+06 |.++.++.+ +.+ + +.+.++.++.+.+ .++.+.++.++.+.++.+.++.++.+.++.++.|
| : : : : + |
| : : : : |
500000 |-+ : : :: |
| :: :: |
| : : |
0 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation
Thanks,
Oliver Sang
9 months, 2 weeks
- 1
- 0
[watchdog] 34db6459c6: stress-ng.loop.ops_per_sec -36.4% regression
by kernel test robot
Greeting,
FYI, we noticed a -36.4% regression of stress-ng.loop.ops_per_sec due to commit:
commit: 34db6459c6ff08d993fa1b9f0ef75aca2b2017ee ("watchdog: iTCO: Drop vendor support")
https://git.kernel.org/cgit/linux/kernel/git/groeck/linux-staging.git watchdog-next
in testcase: stress-ng
on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory
with following parameters:
nr_threads: 100%
disk: 1HDD
testtime: 60s
class: device
test: loop
cpufreq_governor: performance
ucode: 0x5003006
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
=========================================================================================
class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode:
device/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp7/loop/stress-ng/60s/0x5003006
commit:
9545ed1a62 ("watchdog: iTCO_wdt: No need to stop the timer in probe")
34db6459c6 ("watchdog: iTCO: Drop vendor support")
9545ed1a6217fef3 34db6459c6ff08d993fa1b9f0ef
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:10 100% 10:10 kmsg.iTCO_wdt_iTCO_wdt:failed_to_reset_NO_REBOOT_flag,reboot_disabled_by_hardware/BIOS
%stddev %change %stddev
\ | \
3642 -34.7% 2380 ± 5% stress-ng.loop.ops
59.11 -36.4% 37.60 ± 4% stress-ng.loop.ops_per_sec
5478 ± 2% -23.9% 4169 ± 5% stress-ng.time.involuntary_context_switches
206.80 ± 4% -43.2% 117.40 ± 19% stress-ng.time.percent_of_cpu_this_job_got
134.28 ± 5% -41.0% 79.29 ± 20% stress-ng.time.system_time
236439 ± 8% -28.0% 170186 ± 11% stress-ng.time.voluntary_context_switches
0.43 ± 3% -0.1 0.31 ± 7% mpstat.cpu.all.usr%
3841 ± 3% -28.7% 2737 ± 9% vmstat.io.bi
22294 ± 3% -27.5% 16156 ± 5% vmstat.system.cs
32034 ± 3% -25.9% 23739 ± 10% softirqs.BLOCK
269102 ± 6% -16.7% 224266 ± 5% softirqs.RCU
31077 ± 3% -14.8% 26463 ± 7% softirqs.TIMER
718284 -12.6% 627767 ± 2% proc-vmstat.numa_hit
627378 -14.4% 536904 ± 2% proc-vmstat.numa_local
27597 ± 2% -23.8% 21038 ± 6% proc-vmstat.pgactivate
762921 -13.0% 664122 ± 2% proc-vmstat.pgalloc_normal
631716 -16.4% 528351 ± 3% proc-vmstat.pgfree
260814 ± 3% -25.9% 193161 ± 11% proc-vmstat.pgpgin
32689 ± 4% -24.2% 24771 ± 10% proc-vmstat.unevictable_pgs_culled
0.00 ± 13% -44.4% 0.00 perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork
10.33 ± 25% +1109.1% 124.91 ±156% perf-sched.total_wait_and_delay.max.ms
9.66 ± 26% +912.2% 97.80 ±148% perf-sched.total_wait_time.max.ms
22.10 ± 21% +328.5% 94.70 ±139% perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
18.80 ± 40% +201.6% 56.70 ± 20% perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
19.50 ± 20% +384.1% 94.40 ±149% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork
9.29 ± 28% +1244.9% 124.91 ±156% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork
2.17 ± 31% +53.1% 3.33 ± 17% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
9.28 ± 28% +953.5% 97.80 ±148% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork
2406 ± 6% +32.4% 3186 ± 20% perf-stat.i.cycles-between-cache-misses
2.52 ± 6% +14.4% 2.89 ± 6% perf-stat.overall.cpi
2296 ± 5% +17.7% 2703 ± 7% perf-stat.overall.cycles-between-cache-misses
72.20 +1.3 73.52 perf-stat.overall.node-load-miss-rate%
9.606e+08 ± 3% -22.2% 7.475e+08 ± 7% perf-stat.ps.branch-instructions
4806075 ± 7% -23.2% 3689732 ± 10% perf-stat.ps.cache-misses
22083 ± 3% -28.8% 15728 ± 6% perf-stat.ps.context-switches
432.07 ± 3% -28.0% 311.23 ± 9% perf-stat.ps.cpu-migrations
1.055e+09 ± 2% -21.4% 8.299e+08 ± 6% perf-stat.ps.dTLB-loads
4.494e+08 ± 2% -27.6% 3.254e+08 ± 6% perf-stat.ps.dTLB-stores
1266757 ± 4% -22.6% 980124 ± 7% perf-stat.ps.iTLB-load-misses
4.366e+09 ± 2% -21.2% 3.442e+09 ± 6% perf-stat.ps.instructions
4190 ± 2% -9.3% 3799 ± 6% perf-stat.ps.minor-faults
1056859 ± 2% -24.8% 795272 ± 4% perf-stat.ps.node-load-misses
406905 -29.7% 286074 ± 3% perf-stat.ps.node-loads
772814 -23.9% 588285 ± 3% perf-stat.ps.node-store-misses
405848 ± 2% -28.5% 290312 ± 2% perf-stat.ps.node-stores
4287 ± 2% -9.7% 3872 ± 6% perf-stat.ps.page-faults
2.834e+11 ± 2% -18.4% 2.312e+11 ± 8% perf-stat.total.instructions
49289 ± 2% -24.3% 37318 ± 5% slabinfo.filp.active_objs
1544 ± 2% -24.0% 1174 ± 5% slabinfo.filp.active_slabs
49432 ± 2% -23.9% 37595 ± 5% slabinfo.filp.num_objs
1544 ± 2% -24.0% 1174 ± 5% slabinfo.filp.num_slabs
7311 ± 2% +77.8% 13002 ± 3% slabinfo.kmalloc-1k.active_objs
256.60 ± 2% +59.2% 408.50 ± 3% slabinfo.kmalloc-1k.active_slabs
8225 ± 2% +59.1% 13084 ± 3% slabinfo.kmalloc-1k.num_objs
256.60 ± 2% +59.2% 408.50 ± 3% slabinfo.kmalloc-1k.num_slabs
4109 ± 6% -16.7% 3424 ± 6% slabinfo.kmalloc-cg-1k.active_objs
4261 ± 6% -19.6% 3425 ± 6% slabinfo.kmalloc-cg-1k.num_objs
1108 ± 11% -47.7% 579.30 ± 7% slabinfo.kmalloc-cg-2k.active_objs
1178 ± 10% -50.8% 580.20 ± 7% slabinfo.kmalloc-cg-2k.num_objs
17607 ± 12% -34.7% 11492 ± 5% slabinfo.kmalloc-rcl-64.active_objs
17612 ± 12% -34.7% 11492 ± 5% slabinfo.kmalloc-rcl-64.num_objs
4265 ± 11% -27.6% 3089 ± 9% slabinfo.kmalloc-rcl-96.active_objs
4265 ± 11% -27.6% 3089 ± 9% slabinfo.kmalloc-rcl-96.num_objs
6313 ± 7% +62.7% 10273 ± 3% slabinfo.skbuff_head_cache.active_objs
200.60 ± 7% +60.0% 321.00 ± 3% slabinfo.skbuff_head_cache.active_slabs
6432 ± 7% +60.0% 10290 ± 3% slabinfo.skbuff_head_cache.num_objs
200.60 ± 7% +60.0% 321.00 ± 3% slabinfo.skbuff_head_cache.num_slabs
134085 ± 3% +38.6% 185862 ± 5% interrupts.CAL:Function_call_interrupts
1538 ± 24% +48.9% 2290 ± 10% interrupts.CPU0.CAL:Function_call_interrupts
1360 ± 11% +38.5% 1884 ± 10% interrupts.CPU10.CAL:Function_call_interrupts
1411 ± 8% +41.6% 1998 ± 8% interrupts.CPU11.CAL:Function_call_interrupts
1420 ± 6% +35.4% 1923 ± 9% interrupts.CPU13.CAL:Function_call_interrupts
1352 ± 11% +39.9% 1891 ± 10% interrupts.CPU14.CAL:Function_call_interrupts
1354 ± 9% +37.1% 1856 ± 10% interrupts.CPU15.CAL:Function_call_interrupts
1397 ± 7% +46.6% 2048 ± 10% interrupts.CPU16.CAL:Function_call_interrupts
1392 ± 7% +42.6% 1985 ± 7% interrupts.CPU17.CAL:Function_call_interrupts
1307 ± 11% +50.5% 1967 ± 6% interrupts.CPU18.CAL:Function_call_interrupts
1366 ± 4% +41.6% 1934 ± 6% interrupts.CPU19.CAL:Function_call_interrupts
1420 ± 10% +36.0% 1930 ± 11% interrupts.CPU20.CAL:Function_call_interrupts
1388 ± 6% +34.8% 1870 ± 6% interrupts.CPU21.CAL:Function_call_interrupts
1358 ± 6% +37.4% 1866 ± 7% interrupts.CPU23.CAL:Function_call_interrupts
1312 ± 8% +46.1% 1916 ± 6% interrupts.CPU24.CAL:Function_call_interrupts
1311 ± 9% +57.8% 2069 ± 17% interrupts.CPU25.CAL:Function_call_interrupts
1330 ± 5% +47.6% 1964 ± 8% interrupts.CPU26.CAL:Function_call_interrupts
1340 ± 9% +41.7% 1899 ± 8% interrupts.CPU27.CAL:Function_call_interrupts
1469 ± 10% +32.9% 1952 ± 4% interrupts.CPU28.CAL:Function_call_interrupts
1364 ± 5% +42.9% 1950 ± 9% interrupts.CPU29.CAL:Function_call_interrupts
1451 ± 9% +29.6% 1881 ± 6% interrupts.CPU32.CAL:Function_call_interrupts
1381 ± 6% +36.9% 1890 ± 6% interrupts.CPU33.CAL:Function_call_interrupts
1310 ± 5% +43.4% 1878 ± 5% interrupts.CPU34.CAL:Function_call_interrupts
1397 ± 9% +38.4% 1933 ± 6% interrupts.CPU35.CAL:Function_call_interrupts
1317 ± 7% +49.4% 1968 ± 6% interrupts.CPU36.CAL:Function_call_interrupts
1380 ± 8% +38.4% 1911 ± 8% interrupts.CPU37.CAL:Function_call_interrupts
1298 ± 8% +47.5% 1915 ± 6% interrupts.CPU38.CAL:Function_call_interrupts
1347 ± 8% +42.5% 1920 ± 4% interrupts.CPU39.CAL:Function_call_interrupts
1367 ± 7% +47.6% 2017 ± 9% interrupts.CPU4.CAL:Function_call_interrupts
1380 ± 10% +39.1% 1921 ± 7% interrupts.CPU40.CAL:Function_call_interrupts
1366 ± 6% +38.7% 1895 ± 6% interrupts.CPU41.CAL:Function_call_interrupts
1390 ± 5% +37.1% 1906 ± 8% interrupts.CPU42.CAL:Function_call_interrupts
1344 ± 12% +42.6% 1917 ± 7% interrupts.CPU43.CAL:Function_call_interrupts
1349 ± 10% +41.1% 1903 ± 8% interrupts.CPU44.CAL:Function_call_interrupts
1376 ± 8% +35.1% 1860 ± 9% interrupts.CPU45.CAL:Function_call_interrupts
1368 ± 9% +39.4% 1906 ± 5% interrupts.CPU46.CAL:Function_call_interrupts
1429 ± 10% +38.1% 1974 ± 8% interrupts.CPU47.CAL:Function_call_interrupts
1471 ± 10% +28.1% 1883 ± 9% interrupts.CPU48.CAL:Function_call_interrupts
1403 ± 8% +35.6% 1902 ± 10% interrupts.CPU49.CAL:Function_call_interrupts
1376 ± 10% +39.8% 1923 ± 9% interrupts.CPU50.CAL:Function_call_interrupts
1463 ± 12% +33.1% 1948 ± 11% interrupts.CPU51.CAL:Function_call_interrupts
1446 ± 14% +28.0% 1851 ± 6% interrupts.CPU52.CAL:Function_call_interrupts
1311 ± 9% +54.0% 2020 ± 11% interrupts.CPU53.CAL:Function_call_interrupts
1373 ± 8% +40.2% 1925 ± 8% interrupts.CPU54.CAL:Function_call_interrupts
1373 ± 9% +42.1% 1950 ± 8% interrupts.CPU55.CAL:Function_call_interrupts
1313 ± 9% +43.6% 1885 ± 6% interrupts.CPU56.CAL:Function_call_interrupts
1370 ± 8% +40.3% 1922 ± 9% interrupts.CPU57.CAL:Function_call_interrupts
1361 ± 6% +39.5% 1898 ± 8% interrupts.CPU58.CAL:Function_call_interrupts
1380 ± 8% +39.2% 1921 ± 9% interrupts.CPU59.CAL:Function_call_interrupts
1440 ± 11% +48.6% 2141 ± 21% interrupts.CPU6.CAL:Function_call_interrupts
1365 ± 11% +41.0% 1924 ± 7% interrupts.CPU60.CAL:Function_call_interrupts
1394 ± 8% +41.1% 1968 ± 9% interrupts.CPU62.CAL:Function_call_interrupts
1388 ± 8% +39.7% 1939 ± 5% interrupts.CPU63.CAL:Function_call_interrupts
1400 ± 10% +36.9% 1917 ± 8% interrupts.CPU64.CAL:Function_call_interrupts
1432 ± 9% +40.4% 2010 ± 12% interrupts.CPU66.CAL:Function_call_interrupts
1392 ± 7% +39.0% 1936 ± 7% interrupts.CPU67.CAL:Function_call_interrupts
1413 ± 8% +37.3% 1940 ± 8% interrupts.CPU68.CAL:Function_call_interrupts
1420 ± 11% +45.7% 2069 ± 11% interrupts.CPU69.CAL:Function_call_interrupts
1397 ± 7% +49.3% 2086 ± 13% interrupts.CPU7.CAL:Function_call_interrupts
1396 ± 10% +39.4% 1947 ± 9% interrupts.CPU71.CAL:Function_call_interrupts
1387 ± 6% +35.8% 1883 ± 5% interrupts.CPU72.CAL:Function_call_interrupts
1373 ± 9% +34.2% 1843 ± 7% interrupts.CPU73.CAL:Function_call_interrupts
1390 ± 9% +38.7% 1928 ± 7% interrupts.CPU74.CAL:Function_call_interrupts
1368 ± 7% +43.0% 1957 ± 11% interrupts.CPU75.CAL:Function_call_interrupts
1361 ± 8% +39.7% 1902 ± 6% interrupts.CPU76.CAL:Function_call_interrupts
1263 ± 12% +52.0% 1919 ± 10% interrupts.CPU78.CAL:Function_call_interrupts
1288 ± 13% +49.5% 1926 ± 9% interrupts.CPU79.CAL:Function_call_interrupts
1474 ± 12% +32.8% 1958 ± 9% interrupts.CPU8.CAL:Function_call_interrupts
1425 ± 6% +34.2% 1912 ± 8% interrupts.CPU80.CAL:Function_call_interrupts
1454 ± 7% +34.1% 1950 ± 7% interrupts.CPU81.CAL:Function_call_interrupts
1344 ± 5% +42.8% 1921 ± 10% interrupts.CPU82.CAL:Function_call_interrupts
1375 ± 8% +45.9% 2007 ± 6% interrupts.CPU83.CAL:Function_call_interrupts
1442 ± 5% +41.4% 2039 ± 9% interrupts.CPU84.CAL:Function_call_interrupts
1374 ± 9% +40.9% 1937 ± 9% interrupts.CPU85.CAL:Function_call_interrupts
1393 ± 11% +38.8% 1934 ± 7% interrupts.CPU86.CAL:Function_call_interrupts
1391 ± 10% +36.2% 1894 ± 6% interrupts.CPU87.CAL:Function_call_interrupts
1303 ± 5% +48.6% 1937 ± 4% interrupts.CPU89.CAL:Function_call_interrupts
1437 ± 9% +37.7% 1979 ± 6% interrupts.CPU9.CAL:Function_call_interrupts
1344 ± 7% +42.9% 1922 ± 8% interrupts.CPU90.CAL:Function_call_interrupts
1385 ± 10% +45.4% 2014 ± 8% interrupts.CPU91.CAL:Function_call_interrupts
1296 ± 13% +43.7% 1861 ± 6% interrupts.CPU92.CAL:Function_call_interrupts
1380 ± 10% +37.8% 1903 ± 11% interrupts.CPU93.CAL:Function_call_interrupts
1354 ± 7% +40.6% 1905 ± 4% interrupts.CPU94.CAL:Function_call_interrupts
42.50 ± 7% -24.6 17.87 ± 39% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.ioctl
42.50 ± 7% -24.6 17.87 ± 39% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.ioctl
42.51 ± 7% -24.6 17.88 ± 39% perf-profile.calltrace.cycles-pp.ioctl
42.50 ± 7% -24.6 17.87 ± 39% perf-profile.calltrace.cycles-pp.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe.ioctl
19.49 ± 9% -18.9 0.55 ± 57% perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit._printk
15.38 ± 38% -13.7 1.67 ± 48% perf-profile.calltrace.cycles-pp.block_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe.ioctl
15.38 ± 38% -13.7 1.67 ± 48% perf-profile.calltrace.cycles-pp.blkdev_ioctl.block_ioctl.__x64_sys_ioctl.do_syscall_64.entry_SYSCALL_64_after_hwframe
15.32 ± 38% -13.7 1.62 ± 48% perf-profile.calltrace.cycles-pp.lo_ioctl.blkdev_ioctl.block_ioctl.__x64_sys_ioctl.do_syscall_64
13.84 ± 43% -13.3 0.51 ± 72% perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit._printk.set_capacity_and_notify.cold.loop_set_size
13.83 ± 43% -13.3 0.51 ± 72% perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit._printk.set_capacity_and_notify.cold
12.88 ± 43% -12.7 0.19 ±152% perf-profile.calltrace.cycles-pp.loop_configure.lo_ioctl.blkdev_ioctl.block_ioctl.__x64_sys_ioctl
12.82 ± 43% -12.7 0.17 ±152% perf-profile.calltrace.cycles-pp.vprintk_emit._printk.set_capacity_and_notify.cold.loop_set_size.loop_configure
12.82 ± 43% -12.7 0.17 ±152% perf-profile.calltrace.cycles-pp._printk.set_capacity_and_notify.cold.loop_set_size.loop_configure.lo_ioctl
12.82 ± 43% -12.7 0.17 ±152% perf-profile.calltrace.cycles-pp.loop_set_size.loop_configure.lo_ioctl.blkdev_ioctl.block_ioctl
12.82 ± 43% -12.7 0.17 ±152% perf-profile.calltrace.cycles-pp.set_capacity_and_notify.cold.loop_set_size.loop_configure.lo_ioctl.blkdev_ioctl
11.60 ± 13% -4.6 7.02 ± 40% perf-profile.calltrace.cycles-pp.mntput_no_expire.simple_recursive_removal.debugfs_remove.blk_mq_debugfs_unregister_hctx.blk_mq_exit_queue
10.63 ± 14% -4.3 6.30 ± 40% perf-profile.calltrace.cycles-pp._raw_spin_lock.mntput_no_expire.simple_recursive_removal.debugfs_remove.blk_mq_debugfs_unregister_hctx
10.50 ± 14% -4.3 6.21 ± 40% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.mntput_no_expire.simple_recursive_removal.debugfs_remove
2.58 ± 42% -2.2 0.42 ±100% perf-profile.calltrace.cycles-pp.blkdev_open.do_dentry_open.do_open.path_openat.do_filp_open
2.58 ± 42% -2.2 0.42 ±100% perf-profile.calltrace.cycles-pp.blkdev_get_by_dev.blkdev_open.do_dentry_open.do_open.path_openat
0.00 +4.8 4.76 ± 37% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.misc_open.chrdev_open.do_dentry_open
0.00 +7.4 7.44 ± 85% perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.misc_open.chrdev_open.do_dentry_open
0.00 +12.2 12.20 ± 54% perf-profile.calltrace.cycles-pp.__mutex_lock.misc_open.chrdev_open.do_dentry_open.do_open
0.00 +17.6 17.65 ± 24% perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.dev_vprintk_emit
0.00 +18.3 18.30 ± 24% perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.dev_vprintk_emit.dev_printk_emit
0.00 +18.3 18.31 ± 24% perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.dev_vprintk_emit.dev_printk_emit._dev_err
0.00 +19.2 19.19 ± 22% perf-profile.calltrace.cycles-pp.vprintk_emit.dev_vprintk_emit.dev_printk_emit._dev_err.iTCO_wdt_start
0.00 +19.2 19.19 ± 22% perf-profile.calltrace.cycles-pp._dev_err.iTCO_wdt_start.watchdog_start.watchdog_open.misc_open
0.00 +19.2 19.19 ± 22% perf-profile.calltrace.cycles-pp.dev_printk_emit._dev_err.iTCO_wdt_start.watchdog_start.watchdog_open
0.00 +19.2 19.19 ± 22% perf-profile.calltrace.cycles-pp.dev_vprintk_emit.dev_printk_emit._dev_err.iTCO_wdt_start.watchdog_start
0.00 +19.2 19.19 ± 22% perf-profile.calltrace.cycles-pp.watchdog_open.misc_open.chrdev_open.do_dentry_open.do_open
0.00 +19.2 19.19 ± 22% perf-profile.calltrace.cycles-pp.watchdog_start.watchdog_open.misc_open.chrdev_open.do_dentry_open
0.00 +19.2 19.19 ± 22% perf-profile.calltrace.cycles-pp.iTCO_wdt_start.watchdog_start.watchdog_open.misc_open.chrdev_open
3.86 ± 34% +28.6 32.48 ± 20% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.86 ± 34% +28.6 32.48 ± 20% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64
3.28 ± 34% +29.2 32.52 ± 20% perf-profile.calltrace.cycles-pp.open64
3.27 ± 34% +29.2 32.51 ± 20% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.open64
3.27 ± 34% +29.2 32.51 ± 20% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
3.25 ± 34% +29.2 32.50 ± 20% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
3.24 ± 34% +29.3 32.50 ± 20% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.open64
2.62 ± 41% +29.4 32.05 ± 21% perf-profile.calltrace.cycles-pp.do_open.path_openat.do_filp_open.do_sys_openat2.do_sys_open
2.60 ± 42% +29.4 32.03 ± 21% perf-profile.calltrace.cycles-pp.do_dentry_open.do_open.path_openat.do_filp_open.do_sys_openat2
0.00 +31.4 31.40 ± 22% perf-profile.calltrace.cycles-pp.chrdev_open.do_dentry_open.do_open.path_openat.do_filp_open
0.00 +31.4 31.40 ± 22% perf-profile.calltrace.cycles-pp.misc_open.chrdev_open.do_dentry_open.do_open.path_openat
42.51 ± 7% -24.6 17.92 ± 39% perf-profile.children.cycles-pp.ioctl
42.50 ± 7% -24.6 17.91 ± 39% perf-profile.children.cycles-pp.__x64_sys_ioctl
22.52 ± 10% -20.5 2.01 ± 36% perf-profile.children.cycles-pp._printk
15.38 ± 38% -13.7 1.67 ± 48% perf-profile.children.cycles-pp.block_ioctl
15.38 ± 38% -13.7 1.67 ± 48% perf-profile.children.cycles-pp.blkdev_ioctl
15.32 ± 38% -13.7 1.62 ± 48% perf-profile.children.cycles-pp.lo_ioctl
15.02 ± 39% -13.7 1.32 ± 51% perf-profile.children.cycles-pp.loop_set_size
15.02 ± 39% -13.7 1.32 ± 51% perf-profile.children.cycles-pp.set_capacity_and_notify.cold
12.88 ± 43% -12.5 0.36 ± 65% perf-profile.children.cycles-pp.loop_configure
7.25 ± 87% -6.8 0.40 ± 69% perf-profile.children.cycles-pp.loop_control_ioctl.cold
11.88 ± 14% -4.7 7.21 ± 39% perf-profile.children.cycles-pp.mntput_no_expire
2.58 ± 42% -2.0 0.61 ± 43% perf-profile.children.cycles-pp.blkdev_open
2.58 ± 42% -2.0 0.61 ± 43% perf-profile.children.cycles-pp.blkdev_get_by_dev
1.54 ± 53% -1.2 0.37 ± 37% perf-profile.children.cycles-pp.blkdev_get_whole
1.54 ± 53% -1.2 0.37 ± 37% perf-profile.children.cycles-pp.lo_open
1.22 ± 32% -1.1 0.10 ± 78% perf-profile.children.cycles-pp.unit_file_find_dropin_paths
1.13 ± 28% -0.9 0.25 ± 37% perf-profile.children.cycles-pp.openat64
1.43 ± 21% -0.8 0.64 ± 36% perf-profile.children.cycles-pp.path_lookupat
0.67 ± 24% -0.4 0.30 ± 39% perf-profile.children.cycles-pp.path_init
0.75 ± 15% -0.3 0.43 ± 38% perf-profile.children.cycles-pp.walk_component
0.47 ± 23% -0.3 0.18 ± 38% perf-profile.children.cycles-pp.__legitimize_mnt
0.33 ± 24% -0.2 0.11 ± 37% perf-profile.children.cycles-pp.try_to_unlazy
0.33 ± 24% -0.2 0.11 ± 38% perf-profile.children.cycles-pp.__legitimize_path
0.30 ± 31% -0.2 0.10 ± 48% perf-profile.children.cycles-pp.alloc_empty_file
0.36 ± 24% -0.2 0.17 ± 43% perf-profile.children.cycles-pp.__close
0.29 ± 31% -0.2 0.10 ± 48% perf-profile.children.cycles-pp.__alloc_file
0.45 ± 16% -0.2 0.28 ± 37% perf-profile.children.cycles-pp.kmem_cache_alloc
0.35 ± 25% -0.2 0.18 ± 44% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.50 ± 18% -0.2 0.33 ± 41% perf-profile.children.cycles-pp.rcu_do_batch
0.40 ± 18% -0.2 0.24 ± 39% perf-profile.children.cycles-pp.step_into
0.21 ± 26% -0.2 0.05 ± 36% perf-profile.children.cycles-pp.complete_walk
0.35 ± 17% -0.1 0.22 ± 42% perf-profile.children.cycles-pp.kmem_cache_free
0.29 ± 19% -0.1 0.17 ± 43% perf-profile.children.cycles-pp.__traverse_mounts
0.28 ± 19% -0.1 0.16 ± 43% perf-profile.children.cycles-pp.lookup_mnt
0.28 ± 12% -0.1 0.17 ± 40% perf-profile.children.cycles-pp.lookup_fast
0.17 ± 20% -0.1 0.06 ± 39% perf-profile.children.cycles-pp.__fxstat64
0.17 ± 23% -0.1 0.09 ± 52% perf-profile.children.cycles-pp.legitimize_mnt
0.13 ± 18% -0.1 0.04 ± 82% perf-profile.children.cycles-pp.____sys_sendmsg
0.13 ± 19% -0.1 0.05 ± 68% perf-profile.children.cycles-pp.__sys_sendmsg
0.12 ± 22% -0.1 0.04 ± 67% perf-profile.children.cycles-pp.__do_sys_newfstat
0.13 ± 20% -0.1 0.05 ± 68% perf-profile.children.cycles-pp.___sys_sendmsg
0.13 ± 14% -0.1 0.07 ± 43% perf-profile.children.cycles-pp.__wake_up_common_lock
0.11 ± 20% -0.1 0.05 ± 52% perf-profile.children.cycles-pp.__wake_up_common
0.09 ± 19% -0.1 0.04 ± 65% perf-profile.children.cycles-pp.vfs_fstat
0.08 ± 16% -0.0 0.04 ±102% perf-profile.children.cycles-pp.obj_cgroup_uncharge_pages
2.86 ± 12% +5.0 7.82 ± 28% perf-profile.children.cycles-pp.osq_lock
2.93 ± 39% +10.2 13.12 ± 49% perf-profile.children.cycles-pp.__mutex_lock
0.00 +19.2 19.19 ± 22% perf-profile.children.cycles-pp._dev_err
0.00 +19.2 19.19 ± 22% perf-profile.children.cycles-pp.dev_printk_emit
0.00 +19.2 19.19 ± 22% perf-profile.children.cycles-pp.dev_vprintk_emit
0.00 +19.2 19.19 ± 22% perf-profile.children.cycles-pp.watchdog_open
0.00 +19.2 19.19 ± 22% perf-profile.children.cycles-pp.watchdog_start
0.00 +19.2 19.19 ± 22% perf-profile.children.cycles-pp.iTCO_wdt_start
4.50 ± 29% +28.3 32.77 ± 20% perf-profile.children.cycles-pp.do_sys_open
4.50 ± 29% +28.3 32.77 ± 20% perf-profile.children.cycles-pp.do_sys_openat2
4.34 ± 29% +28.4 32.74 ± 20% perf-profile.children.cycles-pp.do_filp_open
4.33 ± 29% +28.4 32.73 ± 20% perf-profile.children.cycles-pp.path_openat
3.48 ± 33% +29.1 32.54 ± 20% perf-profile.children.cycles-pp.open64
2.63 ± 41% +29.4 32.05 ± 21% perf-profile.children.cycles-pp.do_open
2.62 ± 41% +29.4 32.04 ± 21% perf-profile.children.cycles-pp.do_dentry_open
0.00 +31.4 31.40 ± 22% perf-profile.children.cycles-pp.chrdev_open
0.00 +31.4 31.40 ± 22% perf-profile.children.cycles-pp.misc_open
0.60 ± 24% -0.3 0.28 ± 39% perf-profile.self.cycles-pp.path_init
0.13 ± 26% -0.1 0.06 ± 53% perf-profile.self.cycles-pp.kmem_cache_alloc
2.84 ± 12% +4.9 7.76 ± 28% perf-profile.self.cycles-pp.osq_lock
stress-ng.loop.ops_per_sec
70 +----------------------------------------------------------------------+
| +. |
60 |-+ +.+ .++.+++.+ ++.++.+ .+ +.++. .++.+|
|+.+ + + ++.++ + +.+ ++ ++ + + + ++ |
50 |-+ : : : : : : : :: :: : |
| : : : : : : : :: :: : |
40 |-+ : :: : : :: : : : :O: O: : O |
|O OO O :O:: :O O :O::O: :O:O: :O :O: O |
30 |-+ O: ::O: : :: : : : : : : : |
| :: :: :: :: :: :: :: |
20 |-+ :: :: :: :: :: :: :: |
| :: :: :: :: :: :: :: |
10 |-+ : : : : : : : |
| : : : : : : : |
0 +----------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation
Thanks,
Oliver Sang
9 months, 2 weeks
- 1
- 0
Cash App - The easiest way to send/receive, spend, banking, and invest in 2021
by dianaphilip064＠gmail.com
You can find a comprehensive detail about Cash App, that allows peer-to-peer payments directly from your mobile device. You must have a valid bank account associated with your Cash App to transfer funds. Cash app users can get an optional Visa debit card that allows them to use funds in the cash app account and even withdraw cash from ATMs. For more details, please visit site. https://appsapknew.com
9 months, 2 weeks
- 1
- 0
9 months, 3 weeks
- 1
- 0
[PM] bfcc1e67ff: kernel-selftests.breakpoints.step_after_suspend_test.fail
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: bfcc1e67ff1e4aa8bfe2ca57f99390fc284c799d ("PM: sleep: Do not assume that "mem" is always present")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: kernel-selftests
version: kernel-selftests-x86_64-c8c9111a-1_20210929
with following parameters:
group: group-00
ucode: 0x11
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: 288 threads 2 sockets Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang(a)intel.com>
TAP version 13
1..2
# selftests: breakpoints: step_after_suspend_test
# TAP version 13
# Bail out! Failed to enter Suspend state
# # Totals: pass:0 fail:0 xfail:0 xpass:0 skip:0 error:0
not ok 1 selftests: breakpoints: step_after_suspend_test # exit=1
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation
Thanks,
Oliver Sang
9 months, 3 weeks
- 4
- 14
[vmlinux.lds.h] d4c6399900: BUG:unable_to_handle_page_fault_for_address
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: d4c6399900364facd84c9e35ce1540b6046c345f ("vmlinux.lds.h: Avoid orphan section with !SMP")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: trinity
version: trinity-x86_64-03f10b67-1_20210401
with following parameters:
runtime: 300s
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang(a)intel.com>
[ 103.254262] BUG: unable to handle page fault for address: ffffffffbb443040
[ 103.255486] #PF: supervisor write access in kernel mode
[ 103.256427] #PF: error_code(0x0002) - not-present page
[ 103.257362] PGD 2cec37067 P4D 2cec37067 PUD 2cec38063 PMD 100235063 PTE 800ffffd2f9bc062
[ 103.258757] Oops: 0002 [#1] KASAN PTI
[ 103.259355] CPU: 0 PID: 1 Comm: swapper Not tainted 5.13.0-rc2+ #1
[ 103.260390] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 103.261811] RIP: 0010:kvm_guest_apic_eoi_write+0x12/0x90
[ 103.262740] Code: 00 48 c7 c7 28 8d b1 ba e8 2b b5 60 00 eb cc 66 0f 1f 84 00 00 00 00 00 53 be 08 00 00 00 48 c7 c7 40 30 44 bb e8 ee b8 60 00 <48> 0f ba 35 95 d0 59 05 00 72 4e 48 c7 c0 80 fc 7f b9 48 ba 00 00
[ 103.265736] RSP: 0018:ffffc90000007fc8 EFLAGS: 00010046
[ 103.266640] RAX: 0000000000000001 RBX: ffffffffb97ffa40 RCX: ffffffffb5ea5fa2
[ 103.267869] RDX: 0000000000000001 RSI: 0000000000000008 RDI: ffffffffbb443040
[ 103.269087] RBP: 0000000000000000 R08: 0000000000000001 R09: fffffbfff7688609
[ 103.270329] R10: ffffffffbb443047 R11: fffffbfff7688608 R12: 0000000000000000
[ 103.271490] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 103.272646] FS: 0000000000000000(0000) GS:ffffffffb9a7f000(0000) knlGS:0000000000000000
[ 103.273993] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 103.274963] CR2: ffffffffbb443040 CR3: 00000002cec34000 CR4: 00000000000406b0
[ 103.276150] Call Trace:
[ 103.276564] <IRQ>
[ 103.276913] __sysvec_apic_timer_interrupt+0x62/0x370
[ 103.277781] sysvec_apic_timer_interrupt+0x62/0x80
[ 103.278602] </IRQ>
[ 103.278984] asm_sysvec_apic_timer_interrupt+0x12/0x20
[ 103.279890] RIP: 0010:call_rcu+0xc/0x150
[ 103.280559] Code: c7 c7 e0 db 05 ba e8 d3 23 33 02 85 c0 75 cc eb 9f 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 b8 00 00 00 00 00 fc ff df 55 53 <48> 89 fb 48 83 c7 08 48 89 fa 48 c1 ea 03 48 83 ec 08 80 3c 02 00
[ 103.283542] RSP: 0018:ffffc9000001fdb8 EFLAGS: 00000246
[ 103.284404] RAX: dffffc0000000000 RBX: fffff52000003fc6 RCX: 1ffffffff75653d5
[ 103.285532] RDX: 1ffff92000003fdd RSI: ffffffffb60b1810 RDI: ffffc9000001fe80
[ 103.286712] RBP: ffffc9000001fe80 R08: ffffc9000001fe60 R09: 0000000000000000
[ 103.287888] R10: 0000000000000001 R11: ffffc9000001fe90 R12: ffffc9000001fe60
[ 103.289058] R13: 0000000000000000 R14: dffffc0000000000 R15: 0000000000000000
[ 103.290290] ? rcu_tasks_pregp_step+0x10/0x10
[ 103.291029] __wait_rcu_gp+0x160/0x440
[ 103.291650] rcu_barrier+0x83/0xc0
[ 103.292234] ? poll_state_synchronize_rcu+0x10/0x10
[ 103.293040] ? synchronize_rcu+0x80/0x80
[ 103.293720] ? lockdep_hardirqs_on_prepare+0x26b/0x3e0
[ 103.294554] ? trace_hardirqs_on+0x3d/0x1d0
[ 103.295279] ? _vdso_data+0xf80/0xf80
[ 103.295919] ? _vdso_data+0xf80/0xf80
[ 103.296525] ? free_kernel_image_pages+0xd/0x30
[ 103.297307] ? rest_init+0x18e/0x18e
[ 103.297937] kernel_init+0x20/0x112
[ 103.298537] ret_from_fork+0x22/0x30
[ 103.299197] Modules linked in:
[ 103.299740] CR2: ffffffffbb443040
[ 103.300304] ---[ end trace 733607da50d3f759 ]---
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
---
0DAY/LKP+ Test Infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation
Thanks,
Oliver Sang
9 months, 3 weeks
- 3
- 2
