July 2020 - LKP - Ml01.01.Org

[sched/fair] 3c29e651e1: hackbench.throughput -15.2% regression

by kernel test robot

Greeting, FYI, we noticed a -15.2% regression of hackbench.throughput due to commit: commit: 3c29e651e16dd3b3179cfb2d055ee9538e37515c ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: hackbench on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory with following parameters: nr_threads: 100% mode: threads ipc: pipe cpufreq_governor: performance ucode: 0xca test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler. test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sc... In addition to that, the commit also has significant impact on the following tests: +------------------+-----------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput -11.3% regression | | test machine | 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | iterations=18 | | | mode=threads | | | nr_threads=1600% | | | ucode=0x43 | +------------------+-----------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput -10.6% regression | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | mode=process | | | nr_threads=50% | | | ucode=0xb000038 | +------------------+-----------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput -6.8% regression | | test machine | 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | mode=threads | | | nr_threads=100% | | | ucode=0xb8 | +------------------+-----------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-20191114.cgz/lkp-cfl-e1/hackbench/0xca commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- %stddev %change %stddev \ | \ 69492 ± 3% -15.2% 58899 hackbench.throughput 7.727e+08 ± 7% +38.3% 1.069e+09 ± 2% hackbench.time.involuntary_context_switches 131708 ± 4% -15.4% 111382 hackbench.time.minor_page_faults 684.98 ± 2% -2.5% 667.72 hackbench.time.user_time 1.755e+09 ± 2% +11.2% 1.953e+09 hackbench.time.voluntary_context_switches 4.32e+08 ± 4% -15.6% 3.648e+08 hackbench.workload 186.50 ± 12% -16.6% 155.50 ± 11% interrupts.TLB:TLB_shootdowns 4147892 ± 5% +19.6% 4961643 vmstat.system.cs 0.72 ± 10% -0.1 0.58 ± 17% mpstat.cpu.all.idle% 0.00 ±156% +0.0 0.01 ± 62% mpstat.cpu.all.soft% 1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.active_objs 1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.num_objs 40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_hit 40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_local 40724029 ± 10% -34.5% 26661546 proc-vmstat.pgalloc_normal 763906 ± 2% -7.5% 706781 proc-vmstat.pgfault 40701242 ± 10% -34.6% 26637947 proc-vmstat.pgfree 1636058 ± 11% -45.0% 899629 ± 11% turbostat.C1 0.10 ± 7% -0.0 0.07 ± 7% turbostat.C1% 15201 ± 25% +113.3% 32422 ± 8% turbostat.C1E 0.01 ± 57% +0.0 0.03 ± 13% turbostat.C1E% 9093 ± 11% +217.2% 28841 ± 11% turbostat.C3 0.00 ±173% +0.0 0.03 ± 13% turbostat.C3% 16977 ± 2% -92.8% 1222 ± 98% turbostat.C8 0.17 ± 4% -0.2 0.01 ±110% turbostat.C8% 0.13 -100.0% 0.00 turbostat.CPU%c7 9623106 ± 5% -35.1% 6241651 ± 8% cpuidle.C1.time 1636668 ± 11% -45.0% 899926 ± 11% cpuidle.C1.usage 899494 ± 33% +272.6% 3351412 ± 5% cpuidle.C1E.time 15327 ± 25% +112.6% 32592 ± 8% cpuidle.C1E.usage 435674 ± 20% +643.5% 3239351 ± 14% cpuidle.C3.time 9179 ± 11% +215.2% 28933 ± 11% cpuidle.C3.usage 16850924 ± 2% -94.2% 979516 ± 86% cpuidle.C8.time 17150 ± 2% -92.0% 1378 ± 88% cpuidle.C8.usage 6022328 ± 18% -57.1% 2584773 ± 7% cpuidle.POLL.time 5339703 ± 19% -60.1% 2130089 ± 7% cpuidle.POLL.usage 14.06 ± 25% +64.0% 23.07 ± 13% sched_debug.cfs_rq:/.runnable_load_avg.min 25.10 ± 5% -15.8% 21.12 ± 7% sched_debug.cfs_rq:/.runnable_load_avg.stddev 13339 ± 24% +45.7% 19433 ± 7% sched_debug.cfs_rq:/.runnable_weight.min 166.51 ± 24% +48.9% 247.86 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.min 253.56 ± 2% -12.3% 222.30 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.stddev 159602 ± 23% +55.6% 248411 ± 6% sched_debug.cpu.avg_idle.max 51567 ± 24% +49.1% 76905 ± 11% sched_debug.cpu.avg_idle.stddev 11576 ± 5% -9.5% 10478 sched_debug.cpu.curr->pid.avg 9976 ± 8% -16.8% 8305 ± 9% sched_debug.cpu.curr->pid.min 4.81 ± 15% +43.3% 6.89 ± 8% sched_debug.cpu.nr_running.min 6.68 ± 4% -17.0% 5.54 ± 8% sched_debug.cpu.nr_running.stddev 76003928 ± 2% +21.0% 91975904 sched_debug.cpu.nr_switches.avg 78115404 ± 2% +23.4% 96397551 sched_debug.cpu.nr_switches.max 74275755 ± 2% +19.0% 88359903 sched_debug.cpu.nr_switches.min 1043054 ± 15% +83.9% 1917752 ± 10% sched_debug.cpu.nr_switches.stddev -685.26 +64.8% -1129 sched_debug.cpu.nr_uninterruptible.min 15470 ± 9% -31.0% 10674 ± 9% softirqs.CPU0.SCHED 13756 ± 10% -40.8% 8140 ± 4% softirqs.CPU1.SCHED 13645 ± 11% -39.7% 8227 ± 3% softirqs.CPU10.SCHED 14065 ± 12% -38.8% 8609 ± 2% softirqs.CPU11.SCHED 13948 ± 11% -39.9% 8381 ± 6% softirqs.CPU12.SCHED 13823 ± 12% -39.7% 8332 ± 9% softirqs.CPU13.SCHED 73323 +13.4% 83165 ± 11% softirqs.CPU14.RCU 13872 ± 12% -36.5% 8809 ± 7% softirqs.CPU14.SCHED 13726 ± 12% -40.4% 8175 ± 4% softirqs.CPU15.SCHED 13780 ± 11% -34.8% 8987 ± 12% softirqs.CPU2.SCHED 13895 ± 10% -40.5% 8266 ± 6% softirqs.CPU3.SCHED 221879 +13.4% 251589 ± 14% softirqs.CPU3.TIMER 13873 ± 10% -40.4% 8262 ± 6% softirqs.CPU4.SCHED 72625 +12.0% 81355 ± 10% softirqs.CPU5.RCU 13876 ± 13% -41.4% 8137 ± 3% softirqs.CPU5.SCHED 14118 ± 12% -40.8% 8354 ± 3% softirqs.CPU6.SCHED 13503 ± 12% -40.0% 8099 softirqs.CPU7.SCHED 13549 ± 11% -38.2% 8374 ± 8% softirqs.CPU8.SCHED 14273 ± 10% -40.1% 8548 ± 6% softirqs.CPU9.SCHED 223184 ± 11% -38.9% 136383 ± 5% softirqs.SCHED 55.80 +9.9% 61.34 perf-stat.i.MPKI 1.32 +0.1 1.39 perf-stat.i.branch-miss-rate% 1.066e+08 +3.7% 1.105e+08 perf-stat.i.branch-misses 0.15 ± 8% -0.0 0.11 ± 3% perf-stat.i.cache-miss-rate% 2521584 ± 9% -26.5% 1854353 ± 3% perf-stat.i.cache-misses 2.27e+09 +8.0% 2.451e+09 perf-stat.i.cache-references 4162783 ± 4% +19.7% 4981708 perf-stat.i.context-switches 1.52 +2.0% 1.55 perf-stat.i.cpi 227633 ± 14% +75.1% 398694 ± 3% perf-stat.i.cpu-migrations 33931 ± 12% +42.7% 48428 ± 2% perf-stat.i.cycles-between-cache-misses 0.01 -0.0 0.01 ± 4% perf-stat.i.dTLB-load-miss-rate% 1408442 ± 2% -16.4% 1177727 ± 4% perf-stat.i.dTLB-load-misses 1.229e+10 -2.1% 1.203e+10 perf-stat.i.dTLB-loads 0.00 ± 2% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate% 35757 ± 2% -16.0% 30032 ± 3% perf-stat.i.dTLB-store-misses 7.449e+09 -2.3% 7.279e+09 perf-stat.i.dTLB-stores 39246970 -5.5% 37079604 perf-stat.i.iTLB-load-misses 76813 ± 8% -26.2% 56724 ± 6% perf-stat.i.iTLB-loads 4.096e+10 -1.9% 4.02e+10 perf-stat.i.instructions 1062 +4.1% 1105 perf-stat.i.instructions-per-iTLB-miss 0.66 -1.9% 0.65 perf-stat.i.ipc 1210 -7.1% 1124 perf-stat.i.minor-faults 105889 ± 10% -24.8% 79596 ± 5% perf-stat.i.node-loads 174508 ± 7% -24.4% 131915 perf-stat.i.node-stores 1210 -7.1% 1124 perf-stat.i.page-faults 55.42 ± 2% +10.0% 60.98 perf-stat.overall.MPKI 1.31 +0.1 1.38 perf-stat.overall.branch-miss-rate% 0.11 ± 11% -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate% 1.51 +1.9% 1.54 perf-stat.overall.cpi 24740 ± 9% +34.9% 33377 ± 3% perf-stat.overall.cycles-between-cache-misses 0.01 -0.0 0.01 ± 4% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 2% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate% 1043 +3.9% 1084 perf-stat.overall.instructions-per-iTLB-miss 0.66 -1.9% 0.65 perf-stat.overall.ipc 57826 ± 3% +15.6% 66874 perf-stat.overall.path-length 1.064e+08 +3.7% 1.104e+08 perf-stat.ps.branch-misses 2517524 ± 9% -26.5% 1851335 ± 3% perf-stat.ps.cache-misses 2.266e+09 +8.0% 2.447e+09 perf-stat.ps.cache-references 4155898 ± 4% +19.7% 4973458 perf-stat.ps.context-switches 227258 ± 14% +75.1% 398041 ± 3% perf-stat.ps.cpu-migrations 1406119 ± 2% -16.4% 1175777 ± 4% perf-stat.ps.dTLB-load-misses 1.227e+10 -2.1% 1.201e+10 perf-stat.ps.dTLB-loads 35699 ± 2% -16.0% 29983 ± 3% perf-stat.ps.dTLB-store-misses 7.437e+09 -2.3% 7.267e+09 perf-stat.ps.dTLB-stores 39182154 -5.5% 37018206 perf-stat.ps.iTLB-load-misses 76695 ± 8% -26.2% 56638 ± 6% perf-stat.ps.iTLB-loads 4.09e+10 -1.9% 4.013e+10 perf-stat.ps.instructions 1208 -7.1% 1122 perf-stat.ps.minor-faults 105716 ± 10% -24.8% 79465 ± 5% perf-stat.ps.node-loads 174234 ± 7% -24.4% 131703 perf-stat.ps.node-stores 1208 -7.1% 1122 perf-stat.ps.page-faults 29.39 ±104% -29.4 0.00 perf-profile.calltrace.cycles-pp.start_thread 15.86 ±105% -15.9 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread 14.88 ±105% -14.9 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 14.75 ±105% -14.8 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 13.53 ±104% -13.5 0.00 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 12.51 ±104% -12.5 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread 12.45 ±105% -12.4 0.00 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 11.21 ±105% -11.2 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 11.08 ±105% -11.1 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 10.47 ±105% -10.5 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 9.73 ±105% -9.7 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 3.09 ± 12% -0.6 2.50 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 2.45 ± 14% -0.5 1.95 ± 2% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 1.15 ± 13% -0.2 0.95 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 0.99 ± 13% -0.2 0.83 ± 5% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write 0.92 ± 15% +0.2 1.10 perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.84 ± 23% +0.3 1.10 ± 2% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.99 ± 19% +0.3 1.26 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.97 ± 23% +0.3 1.26 perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.29 ±100% +0.3 0.60 ± 5% perf-profile.calltrace.cycles-pp.reschedule_interrupt.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write 0.55 ± 62% +0.3 0.89 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule 0.28 ±100% +0.4 0.63 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop 0.33 ±100% +0.4 0.69 ± 2% perf-profile.calltrace.cycles-pp.prepare_to_wait.pipe_wait.pipe_read.new_sync_read.vfs_read 0.29 ±100% +0.4 0.65 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 1.00 ± 27% +0.4 1.37 ± 2% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function 0.57 ± 63% +0.4 0.95 ± 5% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.46 ± 31% +0.4 1.86 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.72 ± 60% +0.4 1.14 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.29 ±100% +0.4 0.70 ± 6% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.34 ±100% +0.4 0.77 ± 7% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task 1.26 ± 26% +0.5 1.72 perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common 1.13 ± 26% +0.5 1.59 perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.35 ±100% +0.5 0.83 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 1.28 ± 28% +0.5 1.77 ± 3% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 1.40 ± 26% +0.5 1.89 perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.44 ±100% +0.5 0.95 ± 7% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 2.33 ± 17% +0.5 2.88 ± 2% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read 1.33 ± 28% +0.6 1.89 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 1.40 ± 24% +0.6 1.97 ± 4% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 0.83 ± 66% +0.6 1.40 ± 5% perf-profile.calltrace.cycles-pp.native_write_msr 5.01 ± 8% +0.8 5.81 ± 2% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 1.75 ± 39% +0.9 2.61 ± 6% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 1.33 ± 67% +1.0 2.28 ± 6% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.56 ± 12% +1.2 7.78 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read 8.96 ± 9% +1.6 10.56 perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 9.23 ± 10% +1.6 10.87 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 9.17 ± 10% +1.6 10.81 perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 3.72 ± 31% +1.7 5.44 ± 6% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.37 ± 8% +1.9 5.32 ± 4% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 4.37 ± 10% +2.3 6.65 ± 5% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 15.27 ± 10% +2.3 17.59 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read 15.59 ± 10% +2.4 17.96 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read 3.00 ± 71% +2.6 5.60 ± 6% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.10 ± 71% +2.7 5.77 ± 6% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 18.05 ± 10% +2.7 20.73 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read 28.21 ± 5% +3.7 31.87 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 24.29 ± 8% +4.7 28.98 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 23.38 ± 8% +4.7 28.08 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 23.72 ± 8% +4.7 28.43 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 31.18 ± 41% +12.7 43.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 33.44 ± 40% +13.2 46.61 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 64.81 ± 43% +27.5 92.34 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 65.30 ± 43% +27.7 92.99 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 29.39 ±104% -29.4 0.00 perf-profile.children.cycles-pp.start_thread 16.09 ±105% -16.1 0.00 perf-profile.children.cycles-pp.__GI___libc_write 12.73 ±104% -12.7 0.00 perf-profile.children.cycles-pp.__GI___libc_read 5.80 ± 10% -1.2 4.65 ± 3% perf-profile.children.cycles-pp.security_file_permission 4.10 ± 16% -1.0 3.10 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 3.21 ± 9% -0.7 2.55 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter 1.45 ± 32% -0.6 0.83 perf-profile.children.cycles-pp.entry_SYSCALL_64 3.04 ± 8% -0.6 2.44 ± 3% perf-profile.children.cycles-pp.selinux_file_permission 2.56 ± 11% -0.6 1.99 ± 2% perf-profile.children.cycles-pp.copy_page_from_iter 2.80 ± 9% -0.5 2.25 ± 2% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 3.38 ± 7% -0.5 2.87 ± 2% perf-profile.children.cycles-pp.mutex_lock 1.62 ± 12% -0.4 1.23 ± 3% perf-profile.children.cycles-pp.___might_sleep 1.82 ± 9% -0.4 1.43 ± 4% perf-profile.children.cycles-pp.syscall_return_via_sysret 2.09 ± 12% -0.4 1.72 ± 6% perf-profile.children.cycles-pp.file_has_perm 0.90 ± 20% -0.3 0.57 ± 9% perf-profile.children.cycles-pp.__mutex_lock 1.14 ± 12% -0.3 0.82 ± 3% perf-profile.children.cycles-pp.__inode_security_revalidate 1.26 ± 10% -0.3 0.97 ± 4% perf-profile.children.cycles-pp.copyin 1.70 ± 8% -0.3 1.42 ± 3% perf-profile.children.cycles-pp.copyout 1.25 ± 11% -0.3 0.98 ± 3% perf-profile.children.cycles-pp.fsnotify 1.04 ± 9% -0.3 0.79 ± 3% perf-profile.children.cycles-pp._cond_resched 1.72 ± 7% -0.2 1.48 ± 5% perf-profile.children.cycles-pp.fput_many 1.16 ± 10% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.__might_sleep 1.09 ± 8% -0.2 0.85 ± 3% perf-profile.children.cycles-pp.__fsnotify_parent 0.83 ± 13% -0.2 0.63 ± 3% perf-profile.children.cycles-pp.__might_fault 0.69 ± 11% -0.1 0.56 perf-profile.children.cycles-pp.current_time 0.99 ± 4% -0.1 0.89 ± 4% perf-profile.children.cycles-pp.touch_atime 0.65 ± 6% -0.1 0.55 ± 6% perf-profile.children.cycles-pp.atime_needs_update 0.39 ± 4% -0.1 0.29 perf-profile.children.cycles-pp.rcu_all_qs 0.13 ± 39% -0.1 0.04 ± 59% perf-profile.children.cycles-pp.mutex_spin_on_owner 0.34 ± 11% -0.1 0.26 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common 0.49 ± 5% -0.1 0.41 ± 2% perf-profile.children.cycles-pp.wake_up_q 0.24 ± 2% -0.1 0.17 ± 4% perf-profile.children.cycles-pp.fpregs_assert_state_consistent 0.30 ± 11% -0.1 0.23 ± 3% perf-profile.children.cycles-pp.inode_has_perm 0.23 ± 8% -0.1 0.18 ± 6% perf-profile.children.cycles-pp.__sb_end_write 0.21 ± 9% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.__x64_sys_read 0.29 ± 8% -0.0 0.24 ± 2% perf-profile.children.cycles-pp.timespec64_trunc 0.21 ± 8% -0.0 0.17 ± 8% perf-profile.children.cycles-pp.__x64_sys_write 0.22 ± 7% -0.0 0.19 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.08 ± 16% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.bpf_fd_pass 0.10 ± 4% +0.0 0.11 ± 3% perf-profile.children.cycles-pp.perf_exclude_event 0.08 ± 24% +0.0 0.13 ± 5% perf-profile.children.cycles-pp.rcu_note_context_switch 0.17 ± 19% +0.1 0.24 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.23 ± 19% +0.1 0.32 ± 3% perf-profile.children.cycles-pp.resched_curr 0.61 ± 8% +0.1 0.72 ± 6% perf-profile.children.cycles-pp.sched_clock 0.67 ± 8% +0.1 0.78 ± 6% perf-profile.children.cycles-pp.sched_clock_cpu 0.50 ± 9% +0.1 0.62 ± 3% perf-profile.children.cycles-pp.account_entity_enqueue 0.61 ± 10% +0.2 0.79 perf-profile.children.cycles-pp.account_entity_dequeue 0.43 ± 19% +0.2 0.62 ± 7% perf-profile.children.cycles-pp.clear_buddies 1.42 ± 14% +0.2 1.66 ± 4% perf-profile.children.cycles-pp.native_write_msr 1.17 ± 13% +0.3 1.42 ± 2% perf-profile.children.cycles-pp.check_preempt_wakeup 0.85 ± 23% +0.3 1.12 ± 7% perf-profile.children.cycles-pp.put_prev_entity 0.94 ± 14% +0.3 1.22 ± 5% perf-profile.children.cycles-pp.___perf_sw_event 1.45 ± 11% +0.3 1.73 perf-profile.children.cycles-pp.check_preempt_curr 1.45 ± 12% +0.3 1.76 perf-profile.children.cycles-pp.__enqueue_entity 1.60 ± 10% +0.3 1.91 perf-profile.children.cycles-pp.ttwu_do_wakeup 1.12 ± 9% +0.3 1.43 ± 3% perf-profile.children.cycles-pp.__update_load_avg_se 1.64 ± 8% +0.4 2.00 ± 4% perf-profile.children.cycles-pp.available_idle_cpu 0.96 ± 19% +0.4 1.34 ± 7% perf-profile.children.cycles-pp.pick_next_entity 1.85 ± 16% +0.4 2.28 ± 6% perf-profile.children.cycles-pp.switch_fpu_return 1.90 ± 9% +0.6 2.50 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 3.22 ± 13% +0.6 3.87 perf-profile.children.cycles-pp.update_load_avg 4.03 ± 10% +0.6 4.68 ± 2% perf-profile.children.cycles-pp.update_curr 3.01 ± 14% +0.7 3.72 ± 2% perf-profile.children.cycles-pp.reweight_entity 5.10 ± 8% +0.8 5.87 ± 2% perf-profile.children.cycles-pp.enqueue_entity 4.71 ± 15% +1.1 5.77 ± 4% perf-profile.children.cycles-pp.pick_next_task_fair 6.73 ± 11% +1.1 7.87 perf-profile.children.cycles-pp.dequeue_task_fair 9.07 ± 9% +1.5 10.61 perf-profile.children.cycles-pp.enqueue_task_fair 9.32 ± 9% +1.6 10.89 perf-profile.children.cycles-pp.ttwu_do_activate 91.46 +1.6 93.03 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 9.27 ± 9% +1.6 10.84 perf-profile.children.cycles-pp.activate_task 4.16 ± 26% +1.6 5.78 ± 6% perf-profile.children.cycles-pp.exit_to_usermode_loop 90.79 +1.7 92.47 perf-profile.children.cycles-pp.do_syscall_64 3.48 ± 8% +1.9 5.36 ± 4% perf-profile.children.cycles-pp.select_idle_sibling 4.44 ± 10% +2.2 6.67 ± 5% perf-profile.children.cycles-pp.select_task_rq_fair 18.46 ± 9% +2.3 20.81 perf-profile.children.cycles-pp.pipe_wait 19.93 ± 11% +3.5 23.46 ± 2% perf-profile.children.cycles-pp.__schedule 28.82 ± 5% +3.6 32.45 perf-profile.children.cycles-pp.__wake_up_common_lock 19.89 ± 12% +3.7 23.62 ± 2% perf-profile.children.cycles-pp.schedule 23.96 ± 8% +4.5 28.45 perf-profile.children.cycles-pp.try_to_wake_up 24.53 ± 8% +4.5 29.06 perf-profile.children.cycles-pp.__wake_up_common 23.91 ± 8% +4.6 28.46 perf-profile.children.cycles-pp.autoremove_wake_function 2.76 ± 9% -0.5 2.23 ± 2% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.46 ± 18% -0.4 1.06 ± 4% perf-profile.self.cycles-pp.pipe_write 1.81 ± 9% -0.4 1.43 ± 4% perf-profile.self.cycles-pp.syscall_return_via_sysret 1.56 ± 12% -0.4 1.19 ± 3% perf-profile.self.cycles-pp.___might_sleep 2.09 ± 7% -0.3 1.81 ± 2% perf-profile.self.cycles-pp.mutex_lock 1.86 ± 8% -0.3 1.59 ± 3% perf-profile.self.cycles-pp.selinux_file_permission 1.21 ± 11% -0.3 0.96 ± 2% perf-profile.self.cycles-pp.fsnotify 1.70 ± 7% -0.2 1.46 ± 5% perf-profile.self.cycles-pp.fput_many 1.04 ± 11% -0.2 0.82 ± 3% perf-profile.self.cycles-pp.__might_sleep 1.01 ± 8% -0.2 0.80 ± 3% perf-profile.self.cycles-pp.__fsnotify_parent 1.04 ± 10% -0.2 0.83 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.67 ± 10% -0.2 0.51 ± 5% perf-profile.self.cycles-pp.new_sync_write 0.64 ± 12% -0.2 0.48 ± 5% perf-profile.self.cycles-pp.vfs_write 1.67 -0.2 1.51 ± 2% perf-profile.self.cycles-pp.pipe_read 0.58 ± 17% -0.1 0.44 ± 7% perf-profile.self.cycles-pp.file_has_perm 0.44 ± 18% -0.1 0.31 ± 7% perf-profile.self.cycles-pp.__mutex_lock 0.49 ± 13% -0.1 0.35 ± 3% perf-profile.self.cycles-pp.copy_page_from_iter 0.64 ± 8% -0.1 0.52 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter 0.67 ± 7% -0.1 0.55 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.30 ± 5% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.ksys_write 0.40 ± 12% -0.1 0.30 ± 7% perf-profile.self.cycles-pp.security_file_permission 0.13 ± 38% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.36 ± 11% -0.1 0.28 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate 0.76 ± 5% -0.1 0.68 ± 5% perf-profile.self.cycles-pp.do_syscall_64 0.21 ± 2% -0.1 0.14 ± 8% perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.26 ± 12% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.inode_has_perm 0.29 ± 11% -0.1 0.23 ± 6% perf-profile.self.cycles-pp.current_time 0.28 ± 4% -0.1 0.21 ± 2% perf-profile.self.cycles-pp.rcu_all_qs 0.22 ± 7% -0.1 0.17 ± 7% perf-profile.self.cycles-pp.__sb_end_write 0.31 -0.1 0.26 ± 7% perf-profile.self.cycles-pp.ksys_read 0.20 ± 11% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_read 0.26 ± 8% -0.0 0.22 perf-profile.self.cycles-pp.timespec64_trunc 0.20 ± 5% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_write 0.24 ± 9% -0.0 0.21 ± 6% perf-profile.self.cycles-pp.__might_fault 0.09 ± 9% -0.0 0.06 perf-profile.self.cycles-pp.copyin 0.14 ± 5% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.07 ± 10% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.sched_clock 0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.ttwu_do_wakeup 0.20 ± 12% +0.0 0.24 ± 5% perf-profile.self.cycles-pp.__list_add_valid 0.10 ± 26% +0.0 0.14 ± 10% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.15 ± 16% +0.1 0.21 ± 11% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.35 ± 7% +0.1 0.43 ± 4% perf-profile.self.cycles-pp.prepare_to_wait 0.55 ± 2% +0.1 0.63 ± 3% perf-profile.self.cycles-pp.__fget_light 0.22 ± 18% +0.1 0.30 ± 3% perf-profile.self.cycles-pp.resched_curr 0.48 ± 11% +0.1 0.58 ± 3% perf-profile.self.cycles-pp.check_preempt_wakeup 0.39 ± 11% +0.1 0.50 perf-profile.self.cycles-pp.account_entity_enqueue 0.85 ± 7% +0.1 0.97 ± 4% perf-profile.self.cycles-pp.enqueue_entity 0.43 ± 16% +0.1 0.55 ± 7% perf-profile.self.cycles-pp.pick_next_entity 0.86 ± 10% +0.1 0.99 perf-profile.self.cycles-pp.dequeue_task_fair 0.37 ± 21% +0.2 0.55 ± 6% perf-profile.self.cycles-pp.clear_buddies 0.45 ± 14% +0.2 0.65 perf-profile.self.cycles-pp.account_entity_dequeue 1.41 ± 14% +0.2 1.65 ± 4% perf-profile.self.cycles-pp.native_write_msr 0.80 ± 16% +0.3 1.06 ± 4% perf-profile.self.cycles-pp.___perf_sw_event 1.09 ± 9% +0.3 1.39 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se 0.84 ± 16% +0.3 1.14 ± 7% perf-profile.self.cycles-pp.select_task_rq_fair 1.45 ± 12% +0.3 1.75 perf-profile.self.cycles-pp.__enqueue_entity 1.62 ± 8% +0.4 1.99 ± 4% perf-profile.self.cycles-pp.available_idle_cpu 1.83 ± 16% +0.4 2.25 ± 6% perf-profile.self.cycles-pp.switch_fpu_return 2.15 ± 11% +0.5 2.60 ± 5% perf-profile.self.cycles-pp.update_curr 1.88 ± 9% +0.6 2.45 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 1.21 ± 9% +0.7 1.92 ± 3% perf-profile.self.cycles-pp._raw_spin_lock 0.80 ± 11% +1.4 2.17 ± 3% perf-profile.self.cycles-pp.select_idle_sibling hackbench.throughput 74000 +-+-----------------------------------------------------------------+ 72000 +-+.+ .+ | |.+ : .+. .+. + | 70000 +-+ : + + +. : | 68000 +-+ + + : | | :: | 66000 +-+ + | 64000 +-+ | 62000 +-+ | | | 60000 +-+ O O OO O O O O O O O OO | 58000 +-+ O O O O O O O O O O O | O O OO O O | 56000 O-+ O O O O O O | 54000 +-O-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-hsw-ep4: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory ========================================================================================= compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/18/x86_64-rhel-7.6/threads/1600%/debian-x86_64-2019-11-14.cgz/lkp-hsw-ep4/hackbench/0x43 commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 176075 ± 2% -11.3% 156152 hackbench.throughput 891.79 +12.3% 1001 hackbench.time.elapsed_time 891.79 +12.3% 1001 hackbench.time.elapsed_time.max 1.889e+09 ± 9% +33.1% 2.514e+09 ± 3% hackbench.time.involuntary_context_switches 42939 ± 3% +17.0% 50220 hackbench.time.system_time 18978 +3.2% 19592 hackbench.time.user_time 2.558e+09 ± 8% +33.5% 3.415e+09 ± 2% hackbench.time.voluntary_context_switches 43133704 ± 3% +39.2% 60035429 ± 33% cpuidle.C1.time 48661 +17.7% 57274 ± 5% meminfo.Shmem 19043 ± 2% -14.0% 16377 meminfo.max_used_kB 2502 +1.3% 2535 turbostat.Avg_MHz 3.368e+08 ± 5% +30.6% 4.398e+08 turbostat.IRQ 66.25 +4.5% 69.25 vmstat.cpu.sy 29.25 ± 2% -8.5% 26.75 vmstat.cpu.us 5000443 ± 7% +18.6% 5931772 vmstat.system.cs 370173 ± 3% +17.1% 433616 vmstat.system.in 98786923 ± 7% +74.5% 1.724e+08 numa-numastat.node0.local_node 98791606 ± 7% +74.5% 1.724e+08 numa-numastat.node0.numa_hit 4686 ± 99% +297.8% 18642 ± 24% numa-numastat.node0.other_node 1.818e+08 -51.4% 88447082 ± 3% numa-numastat.node1.local_node 1.819e+08 -51.4% 88451833 ± 3% numa-numastat.node1.numa_hit 18694 ± 25% -74.6% 4753 ± 97% numa-numastat.node1.other_node 155316 +1.5% 157632 proc-vmstat.nr_active_anon 99.50 +5.3% 104.75 ± 3% proc-vmstat.nr_dirtied 4440 +4.9% 4656 proc-vmstat.nr_inactive_anon 6283 +1.9% 6399 proc-vmstat.nr_mapped 12248 ± 2% +16.6% 14276 ± 5% proc-vmstat.nr_shmem 98.75 +5.3% 104.00 proc-vmstat.nr_written 155316 +1.5% 157632 proc-vmstat.nr_zone_active_anon 4440 +4.9% 4656 proc-vmstat.nr_zone_inactive_anon 2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_hit 2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_local 2.83e+08 -7.0% 2.632e+08 proc-vmstat.pgalloc_normal 3212744 +2.3% 3286078 proc-vmstat.pgfault 2.828e+08 -7.0% 2.631e+08 proc-vmstat.pgfree 39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_active_anon 31864 ± 44% +211.0% 99111 ± 4% numa-vmstat.node0.nr_anon_pages 11183 ± 3% +4212.8% 482321 ± 2% numa-vmstat.node0.nr_kernel_stack 499.50 ± 23% +66.9% 833.75 ± 8% numa-vmstat.node0.nr_page_table_pages 11417 -63.5% 4168 ± 5% numa-vmstat.node0.nr_shmem 15905 ± 4% -12.5% 13919 ± 2% numa-vmstat.node0.nr_slab_reclaimable 17996 ± 3% +587.4% 123708 ± 2% numa-vmstat.node0.nr_slab_unreclaimable 39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_zone_active_anon 52471566 ± 6% +73.3% 90915199 numa-vmstat.node0.numa_hit 52466577 ± 6% +73.2% 90896366 numa-vmstat.node0.numa_local 4990 ± 92% +277.5% 18836 ± 24% numa-vmstat.node0.numa_other 114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_active_anon 114545 ± 12% -58.2% 47874 ± 9% numa-vmstat.node1.nr_anon_pages 462075 -96.6% 15717 ± 67% numa-vmstat.node1.nr_kernel_stack 871.50 ± 13% -37.9% 541.00 ± 13% numa-vmstat.node1.nr_page_table_pages 691.25 ± 22% +1359.2% 10086 ± 8% numa-vmstat.node1.nr_shmem 11357 ± 5% +18.1% 13414 ± 4% numa-vmstat.node1.nr_slab_reclaimable 118594 -85.5% 17243 ± 16% numa-vmstat.node1.nr_slab_unreclaimable 114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_zone_active_anon 93308245 ± 2% -49.7% 46921467 ± 4% numa-vmstat.node1.numa_hit 93128224 ± 2% -49.8% 46755145 ± 4% numa-vmstat.node1.numa_local 157762 ± 35% +154.4% 401333 ± 4% numa-meminfo.node0.Active 157721 ± 35% +154.4% 401249 ± 4% numa-meminfo.node0.Active(anon) 128232 ± 44% +212.2% 400337 ± 4% numa-meminfo.node0.AnonPages 63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.KReclaimable 11691 ± 3% +4087.3% 489553 numa-meminfo.node0.KernelStack 1170596 ± 4% +104.2% 2390602 numa-meminfo.node0.MemUsed 2003 ± 23% +67.3% 3351 ± 8% numa-meminfo.node0.PageTables 63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.SReclaimable 72301 ± 3% +591.9% 500285 numa-meminfo.node0.SUnreclaim 45749 -63.5% 16679 ± 5% numa-meminfo.node0.Shmem 135789 ± 3% +309.6% 556136 numa-meminfo.node0.Slab 460025 ± 11% -50.2% 228882 ± 7% numa-meminfo.node1.Active 459902 ± 11% -50.2% 228802 ± 7% numa-meminfo.node1.Active(anon) 459237 ± 11% -58.3% 191354 ± 9% numa-meminfo.node1.AnonPages 45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.KReclaimable 468955 -96.7% 15461 ± 67% numa-meminfo.node1.KernelStack 2241450 -51.9% 1078199 ± 3% numa-meminfo.node1.MemUsed 3493 ± 13% -38.0% 2167 ± 13% numa-meminfo.node1.PageTables 45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.SReclaimable 479010 -85.6% 68750 ± 16% numa-meminfo.node1.SUnreclaim 2779 ± 23% +1354.1% 40419 ± 8% numa-meminfo.node1.Shmem 524580 -76.7% 122408 ± 8% numa-meminfo.node1.Slab 250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.avg 1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.stddev 20766 ± 9% -25.3% 15519 ± 10% sched_debug.cfs_rq:/.load.avg 43820 ± 19% -40.6% 26044 ± 36% sched_debug.cfs_rq:/.load.stddev 250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.max_vruntime.avg 1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.max_vruntime.stddev 10990567 ± 9% +22.9% 13509735 ± 7% sched_debug.cfs_rq:/.min_vruntime.stddev 102.40 -33.3% 68.27 ± 57% sched_debug.cfs_rq:/.removed.load_avg.max 4711 -33.4% 3136 ± 57% sched_debug.cfs_rq:/.removed.runnable_sum.max 19930 ± 10% -25.1% 14917 ± 10% sched_debug.cfs_rq:/.runnable_weight.avg -5713498 -349.0% 14226386 ± 8% sched_debug.cfs_rq:/.spread0.avg 8106715 ± 13% +283.1% 31057182 ± 6% sched_debug.cfs_rq:/.spread0.max -18025645 -99.7% -45393 sched_debug.cfs_rq:/.spread0.min 11027552 ± 9% +21.5% 13398318 ± 7% sched_debug.cfs_rq:/.spread0.stddev 570.46 ± 5% +109.2% 1193 ± 21% sched_debug.cfs_rq:/.util_est_enqueued.avg 2898 ± 11% +90.0% 5507 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.max 563.27 ± 13% +149.3% 1404 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.stddev 155578 ± 22% +84.1% 286457 ± 28% sched_debug.cpu.avg_idle.min 1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock.stddev 1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock_task.stddev 50207 -12.3% 44017 ± 7% sched_debug.cpu.curr->pid.avg 82673 -19.5% 66514 ± 5% sched_debug.cpu.curr->pid.max 746.05 ± 11% +554.3% 4881 ± 8% sched_debug.cpu.curr->pid.min 30728 ± 2% -40.8% 18187 ± 10% sched_debug.cpu.curr->pid.stddev 0.00 ± 43% +106.0% 0.00 ± 31% sched_debug.cpu.next_balance.stddev 20.49 ± 6% +149.0% 51.03 ± 23% sched_debug.cpu.nr_running.avg 157.83 ± 9% +77.0% 279.37 ± 8% sched_debug.cpu.nr_running.max 29.95 ± 11% +137.1% 70.99 ± 16% sched_debug.cpu.nr_running.stddev 29803727 ± 9% +29.6% 38623870 ± 3% sched_debug.cpu.nr_switches.avg 51549476 ± 10% +22.5% 63171598 ± 3% sched_debug.cpu.nr_switches.max 12792570 ± 3% +43.5% 18362678 ± 4% sched_debug.cpu.nr_switches.min 2.03 ± 80% +430.9% 10.76 ± 38% sched_debug.cpu.nr_uninterruptible.avg 6.99 ± 4% +48.7% 10.39 ± 5% perf-stat.i.MPKI 7.49 ± 5% -1.2 6.29 ± 5% perf-stat.i.cache-miss-rate% 27202880 ± 2% +46.9% 39966416 ± 2% perf-stat.i.cache-misses 4.662e+08 ± 5% +71.5% 7.998e+08 ± 5% perf-stat.i.cache-references 5989366 ± 4% +12.9% 6760958 ± 5% perf-stat.i.context-switches 197319 ± 12% -19.2% 159504 ± 3% perf-stat.i.cpu-migrations 7415 ± 2% -23.3% 5686 ± 3% perf-stat.i.cycles-between-cache-misses 0.61 ± 35% -0.3 0.35 ± 4% perf-stat.i.dTLB-load-miss-rate% 1.506e+08 ± 38% -44.9% 82967539 ± 5% perf-stat.i.dTLB-load-misses 0.22 ± 4% -0.0 0.20 ± 4% perf-stat.i.dTLB-store-miss-rate% 35974976 ± 5% -13.4% 31146077 ± 6% perf-stat.i.dTLB-store-misses 1.627e+10 -3.7% 1.566e+10 perf-stat.i.dTLB-stores 46665620 ± 3% -14.6% 39838962 ± 5% perf-stat.i.iTLB-load-misses 2191 ± 5% +21.4% 2660 ± 7% perf-stat.i.instructions-per-iTLB-miss 0.47 -2.5% 0.46 perf-stat.i.ipc 6660 ± 8% -19.1% 5391 ± 8% perf-stat.i.minor-faults 62.93 ± 2% +11.1 74.06 perf-stat.i.node-load-miss-rate% 12537290 ± 4% +73.5% 21750457 ± 4% perf-stat.i.node-load-misses 5459308 ± 3% -7.4% 5057404 ± 3% perf-stat.i.node-loads 29.39 ± 2% -4.1 25.29 ± 2% perf-stat.i.node-store-miss-rate% 2863317 +13.6% 3252632 ± 2% perf-stat.i.node-store-misses 6381925 ± 2% +53.5% 9796918 ± 4% perf-stat.i.node-stores 6638 ± 8% -19.4% 5348 ± 8% perf-stat.i.page-faults 4.89 ± 4% +78.4% 8.73 perf-stat.overall.MPKI 1.29 +0.0 1.31 perf-stat.overall.branch-miss-rate% 6.13 ± 4% -1.3 4.85 ± 3% perf-stat.overall.cache-miss-rate% 2.15 +2.0% 2.19 perf-stat.overall.cpi 7187 -27.9% 5182 ± 3% perf-stat.overall.cycles-between-cache-misses 0.63 ± 39% -0.3 0.36 ± 5% perf-stat.overall.dTLB-load-miss-rate% 0.23 ± 5% -0.0 0.21 ± 3% perf-stat.overall.dTLB-store-miss-rate% 75.65 -1.1 74.59 perf-stat.overall.iTLB-load-miss-rate% 1604 ± 3% +15.9% 1859 perf-stat.overall.instructions-per-iTLB-miss 0.47 -2.0% 0.46 perf-stat.overall.ipc 63.14 ± 2% +13.6 76.70 perf-stat.overall.node-load-miss-rate% 30.74 ± 2% -5.1 25.62 ± 2% perf-stat.overall.node-store-miss-rate% 47754 ± 2% +11.7% 53321 perf-stat.overall.path-length 24893507 +40.6% 34993733 ± 2% perf-stat.ps.cache-misses 4.072e+08 ± 5% +77.1% 7.212e+08 perf-stat.ps.cache-references 4968881 ± 7% +19.0% 5911450 perf-stat.ps.context-switches 1.789e+11 +1.3% 1.812e+11 perf-stat.ps.cpu-cycles 1.59e+08 ± 39% -44.1% 88801351 ± 6% perf-stat.ps.dTLB-load-misses 2.502e+10 -2.0% 2.453e+10 perf-stat.ps.dTLB-loads 39138578 ± 5% -12.6% 34190129 ± 3% perf-stat.ps.dTLB-store-misses 1.675e+10 -3.7% 1.613e+10 perf-stat.ps.dTLB-stores 51901139 ± 2% -14.3% 44461197 perf-stat.ps.iTLB-load-misses 16711264 ± 2% -9.3% 15153636 ± 2% perf-stat.ps.iTLB-loads 3573 -9.1% 3247 perf-stat.ps.minor-faults 10427704 ± 3% +73.2% 18055799 ± 3% perf-stat.ps.node-load-misses 6085553 ± 4% -10.0% 5479559 ± 2% perf-stat.ps.node-loads 2566511 +13.4% 2911367 ± 2% perf-stat.ps.node-store-misses 5784179 ± 2% +46.2% 8454210 ± 3% perf-stat.ps.node-stores 3573 -9.1% 3247 perf-stat.ps.page-faults 7.427e+13 ± 2% +11.7% 8.293e+13 perf-stat.total.instructions 127394 ± 5% +19.1% 151665 softirqs.CPU0.RCU 321761 ± 3% +12.8% 363107 ± 3% softirqs.CPU0.TIMER 124526 ± 5% +20.2% 149697 ± 3% softirqs.CPU1.RCU 123768 ± 4% +24.7% 154332 ± 2% softirqs.CPU10.RCU 130315 ± 3% +22.1% 159111 softirqs.CPU11.RCU 126684 ± 4% +26.1% 159755 ± 2% softirqs.CPU12.RCU 124649 ± 9% +27.8% 159298 ± 3% softirqs.CPU13.RCU 124528 ± 4% +23.2% 153372 ± 2% softirqs.CPU14.RCU 326150 ± 2% +9.8% 357991 ± 4% softirqs.CPU14.TIMER 121923 ± 6% +17.4% 143119 softirqs.CPU15.RCU 122114 ± 5% +16.7% 142560 softirqs.CPU16.RCU 121425 ± 6% +14.4% 138861 ± 3% softirqs.CPU17.RCU 330948 ± 2% +22.2% 404570 ± 16% softirqs.CPU17.TIMER 312317 ± 4% +18.2% 369091 ± 2% softirqs.CPU18.TIMER 9626 ± 2% +15.0% 11067 ± 3% softirqs.CPU19.SCHED 322308 ± 3% +21.1% 390454 ± 2% softirqs.CPU19.TIMER 124281 ± 5% +22.8% 152670 ± 2% softirqs.CPU2.RCU 9452 ± 2% +15.6% 10927 ± 3% softirqs.CPU20.SCHED 318575 ± 6% +22.3% 389741 ± 3% softirqs.CPU20.TIMER 9327 ± 3% +16.7% 10886 ± 3% softirqs.CPU21.SCHED 306628 ± 4% +23.3% 377962 ± 3% softirqs.CPU21.TIMER 9424 ± 2% +17.6% 11084 ± 3% softirqs.CPU22.SCHED 302796 ± 4% +24.2% 376126 ± 2% softirqs.CPU22.TIMER 9798 ± 2% +11.4% 10918 ± 4% softirqs.CPU23.SCHED 312009 ± 4% +23.6% 385626 ± 2% softirqs.CPU23.TIMER 9559 ± 3% +13.2% 10822 softirqs.CPU24.SCHED 311772 ± 4% +23.0% 383504 ± 2% softirqs.CPU24.TIMER 9514 ± 4% +13.2% 10766 ± 3% softirqs.CPU25.SCHED 307640 ± 4% +22.4% 376479 ± 2% softirqs.CPU25.TIMER 9573 ± 3% +13.8% 10897 ± 3% softirqs.CPU26.SCHED 314607 ± 9% +21.1% 380918 ± 2% softirqs.CPU26.TIMER 9385 +13.7% 10668 ± 2% softirqs.CPU27.SCHED 301949 ± 4% +25.2% 378140 ± 3% softirqs.CPU27.TIMER 9483 ± 2% +14.7% 10874 ± 2% softirqs.CPU28.SCHED 299462 ± 3% +24.9% 373896 ± 2% softirqs.CPU28.TIMER 9505 ± 3% +13.3% 10769 ± 3% softirqs.CPU29.SCHED 307087 ± 4% +23.8% 380066 ± 3% softirqs.CPU29.TIMER 124992 ± 4% +19.4% 149253 ± 3% softirqs.CPU3.RCU 9630 ± 4% +13.6% 10942 ± 3% softirqs.CPU30.SCHED 306382 ± 4% +25.4% 384355 ± 2% softirqs.CPU30.TIMER 9496 ± 6% +12.6% 10689 ± 2% softirqs.CPU31.SCHED 308689 ± 4% +24.5% 384419 ± 2% softirqs.CPU31.TIMER 9714 +12.6% 10937 ± 4% softirqs.CPU32.SCHED 304137 ± 3% +35.9% 413240 ± 13% softirqs.CPU32.TIMER 9598 ± 2% +13.4% 10882 ± 5% softirqs.CPU33.SCHED 9622 +13.8% 10949 ± 3% softirqs.CPU34.SCHED 305882 ± 4% +27.3% 389315 ± 4% softirqs.CPU34.TIMER 9409 ± 4% +15.3% 10853 ± 4% softirqs.CPU35.SCHED 306215 ± 4% +23.8% 379222 ± 2% softirqs.CPU35.TIMER 321617 ± 4% +12.9% 362966 ± 3% softirqs.CPU36.TIMER 134843 ± 6% +19.4% 161053 ± 2% softirqs.CPU37.RCU 130954 ± 6% +20.4% 157716 softirqs.CPU38.RCU 129884 ± 5% +17.8% 152978 softirqs.CPU39.RCU 124744 ± 4% +20.1% 149790 ± 3% softirqs.CPU4.RCU 130593 ± 5% +17.2% 153092 ± 2% softirqs.CPU40.RCU 134185 ± 5% +19.9% 160871 softirqs.CPU41.RCU 133336 ± 5% +18.1% 157409 softirqs.CPU42.RCU 134486 ± 4% +18.7% 159613 softirqs.CPU43.RCU 129857 ± 6% +18.5% 153887 softirqs.CPU44.RCU 131816 ± 5% +20.0% 158144 softirqs.CPU45.RCU 132691 ± 5% +22.4% 162412 softirqs.CPU46.RCU 138736 ± 3% +18.7% 164668 softirqs.CPU47.RCU 132971 ± 6% +24.2% 165142 softirqs.CPU48.RCU 136142 ± 3% +20.8% 164493 softirqs.CPU49.RCU 125011 ± 4% +23.6% 154462 ± 2% softirqs.CPU5.RCU 133660 ± 4% +19.5% 159753 ± 2% softirqs.CPU50.RCU 140405 ± 4% +19.5% 167772 softirqs.CPU51.RCU 140523 ± 4% +18.5% 166576 softirqs.CPU52.RCU 138849 ± 5% +14.6% 159157 ± 4% softirqs.CPU53.RCU 332171 ± 2% +21.4% 403260 ± 15% softirqs.CPU53.TIMER 133037 ± 4% +14.0% 151688 ± 2% softirqs.CPU54.RCU 9584 +14.3% 10959 ± 2% softirqs.CPU54.SCHED 310603 ± 4% +18.0% 366472 ± 3% softirqs.CPU54.TIMER 9565 ± 2% +13.8% 10889 ± 5% softirqs.CPU55.SCHED 321231 ± 3% +21.3% 389753 ± 2% softirqs.CPU55.TIMER 136633 ± 4% +12.6% 153854 softirqs.CPU56.RCU 9430 ± 2% +15.6% 10898 ± 5% softirqs.CPU56.SCHED 319477 ± 6% +22.1% 390107 ± 2% softirqs.CPU56.TIMER 128672 ± 4% +14.1% 146791 ± 3% softirqs.CPU57.RCU 9493 ± 2% +13.1% 10740 ± 3% softirqs.CPU57.SCHED 305670 ± 4% +23.5% 377597 ± 2% softirqs.CPU57.TIMER 127129 ± 4% +15.6% 146993 ± 2% softirqs.CPU58.RCU 9529 ± 3% +13.6% 10824 ± 4% softirqs.CPU58.SCHED 302317 ± 4% +23.8% 374371 ± 3% softirqs.CPU58.TIMER 9427 ± 2% +14.0% 10743 ± 2% softirqs.CPU59.SCHED 310985 ± 4% +23.5% 384119 ± 3% softirqs.CPU59.TIMER 124937 ± 4% +21.6% 151962 ± 3% softirqs.CPU6.RCU 9435 ± 2% +14.1% 10762 ± 3% softirqs.CPU60.SCHED 311646 ± 4% +22.8% 382731 ± 3% softirqs.CPU60.TIMER 9585 +13.5% 10877 ± 2% softirqs.CPU61.SCHED 306857 ± 4% +22.8% 376944 ± 2% softirqs.CPU61.TIMER 9483 ± 2% +12.5% 10671 ± 5% softirqs.CPU62.SCHED 9620 ± 4% +9.7% 10556 ± 4% softirqs.CPU63.SCHED 301085 ± 3% +24.7% 375397 ± 3% softirqs.CPU63.TIMER 300443 ± 4% +23.7% 371620 ± 2% softirqs.CPU64.TIMER 9667 +11.2% 10751 ± 4% softirqs.CPU65.SCHED 307194 ± 4% +23.4% 379020 ± 3% softirqs.CPU65.TIMER 9493 ± 3% +14.9% 10909 ± 3% softirqs.CPU66.SCHED 306226 ± 4% +25.1% 383140 ± 3% softirqs.CPU66.TIMER 9674 ± 3% +13.8% 11010 ± 3% softirqs.CPU67.SCHED 307870 ± 4% +25.0% 384793 ± 2% softirqs.CPU67.TIMER 9451 ± 2% +14.2% 10792 ± 4% softirqs.CPU68.SCHED 301873 ± 3% +26.1% 380748 softirqs.CPU68.TIMER 9542 +13.8% 10863 ± 3% softirqs.CPU69.SCHED 316206 ± 8% +20.5% 380893 ± 2% softirqs.CPU69.TIMER 128158 ± 4% +20.4% 154294 ± 2% softirqs.CPU7.RCU 9456 ± 3% +13.5% 10734 ± 3% softirqs.CPU70.SCHED 305361 ± 4% +36.4% 416546 ± 15% softirqs.CPU70.TIMER 9520 ± 3% +15.4% 10985 ± 4% softirqs.CPU71.SCHED 305605 ± 4% +24.6% 380653 ± 2% softirqs.CPU71.TIMER 122831 ± 4% +21.6% 149424 ± 2% softirqs.CPU8.RCU 123870 ± 4% +23.0% 152396 ± 2% softirqs.CPU9.RCU 9357957 ± 5% +15.4% 10794953 softirqs.RCU 23209193 ± 3% +15.5% 26806870 ± 3% softirqs.TIMER 4.79 ± 4% -3.1 1.71 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.85 ± 4% -2.4 1.41 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.39 ± 4% -2.0 1.35 ± 16% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 2.93 ± 5% -1.9 0.99 ± 18% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 3.25 ± 4% -1.9 1.31 ± 10% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write 2.14 ± 4% -1.2 0.91 ± 14% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64 2.06 ± 4% -1.2 0.85 ± 13% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64 1.52 ± 3% -1.0 0.49 ± 59% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 1.46 ± 3% -1.0 0.46 ± 59% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read 1.53 ± 8% -0.8 0.68 ± 12% perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.new_sync_read.vfs_read.ksys_read 1.65 ± 4% -0.8 0.81 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.45 ± 5% -0.7 0.74 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.35 ± 5% -0.7 0.68 ± 10% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64 1.28 ± 8% -0.7 0.62 ± 12% perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.new_sync_write.vfs_write.ksys_write 1.29 ± 3% -0.6 0.67 ± 9% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 0.64 ± 11% +0.3 0.96 ± 7% perf-profile.calltrace.cycles-pp.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 0.00 +0.5 0.55 ± 3% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 0.00 +0.6 0.55 ± 5% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.00 +0.6 0.57 ± 6% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.00 +0.6 0.61 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule 0.00 +0.7 0.69 ± 8% perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_wait.pipe_read 0.00 +0.7 0.70 ± 6% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.00 +0.7 0.71 ± 11% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule 0.00 +0.7 0.72 ± 7% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.00 +0.7 0.72 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +0.8 0.76 ± 10% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.00 +0.8 0.77 ± 2% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule 0.00 +0.8 0.77 ± 5% perf-profile.calltrace.cycles-pp.__switch_to 0.00 +0.8 0.77 ± 6% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 0.00 +0.8 0.79 ± 5% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule 0.00 +0.8 0.81 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read 0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read 0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.00 +0.9 0.90 ± 3% perf-profile.calltrace.cycles-pp.find_next_bit.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up 0.00 +0.9 0.91 ± 4% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.00 +0.9 0.91 ± 6% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function 0.00 +0.9 0.92 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr 0.00 +1.1 1.05 ± 5% perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.00 +1.1 1.08 ± 8% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 0.00 +1.1 1.13 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.00 +1.2 1.17 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.00 +1.2 1.18 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm 0.00 +1.2 1.20 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read 0.00 +1.3 1.30 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.00 +1.3 1.34 ± 6% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +1.6 1.62 ± 4% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.9 1.88 ± 5% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read 0.00 +1.9 1.93 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.13 ±173% +2.0 2.11 ± 5% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 0.00 +2.1 2.06 ± 4% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +2.4 2.38 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.00 +2.7 2.65 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.35 ±103% +4.3 4.67 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.22 ±173% +4.6 4.81 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.23 ±173% +4.7 4.97 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.72 ± 23% +5.0 5.72 ± 5% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read 0.69 ± 17% +5.6 6.25 ± 4% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 0.74 ± 17% +5.8 6.50 ± 4% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.74 ± 18% +5.8 6.55 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 15.99 ± 54% +7.5 23.51 ± 6% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 13.13 +7.7 20.86 ± 3% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.49 +8.7 20.19 ± 3% perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 1.31 ± 27% +8.9 10.21 ± 7% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 1.81 ± 22% +11.2 12.98 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read 1.89 ± 22% +11.4 13.28 ± 5% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read 2.18 ± 21% +13.1 15.28 ± 4% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read 2.01 ± 25% +19.1 21.10 ± 7% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 2.38 ± 23% +20.0 22.40 ± 7% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 19.01 ± 3% +23.4 42.38 ± 4% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 21.13 ± 53% +23.8 44.92 ± 5% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 17.85 ± 4% +23.9 41.76 ± 4% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 19.16 ± 53% +24.6 43.74 ± 5% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.79 ± 22% +30.7 34.45 ± 6% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 3.84 ± 23% +30.9 34.69 ± 6% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 4.13 ± 21% +31.0 35.10 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 5.77 ± 14% +31.0 36.76 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 9.20 ± 6% -6.0 3.16 ± 12% perf-profile.children.cycles-pp.syscall_return_via_sysret 8.88 ± 3% -5.8 3.03 ± 12% perf-profile.children.cycles-pp.entry_SYSCALL_64 8.72 ± 4% -5.3 3.39 ± 11% perf-profile.children.cycles-pp.security_file_permission 4.27 ± 3% -2.4 1.89 ± 9% perf-profile.children.cycles-pp.selinux_file_permission 3.15 ± 5% -2.1 1.07 ± 12% perf-profile.children.cycles-pp.file_has_perm 3.48 ± 3% -2.0 1.46 ± 10% perf-profile.children.cycles-pp.copy_page_to_iter 3.80 ± 3% -2.0 1.79 ± 6% perf-profile.children.cycles-pp.mutex_unlock 3.02 ± 5% -1.9 1.11 ± 11% perf-profile.children.cycles-pp.copy_page_from_iter 3.31 ± 4% -1.6 1.75 ± 6% perf-profile.children.cycles-pp.__fdget_pos 2.55 ± 3% -1.5 1.02 ± 10% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 3.17 ± 4% -1.5 1.70 ± 6% perf-profile.children.cycles-pp.__fget_light 2.10 ± 7% -1.4 0.71 ± 12% perf-profile.children.cycles-pp.fsnotify 2.52 ± 6% -1.3 1.23 ± 8% perf-profile.children.cycles-pp.__fget 1.77 ± 6% -1.1 0.66 ± 9% perf-profile.children.cycles-pp.avc_has_perm 1.89 ± 5% -1.1 0.78 ± 10% perf-profile.children.cycles-pp.___might_sleep 1.44 ± 4% -1.0 0.48 ± 12% perf-profile.children.cycles-pp.__inode_security_revalidate 1.33 ± 10% -0.9 0.40 ± 16% perf-profile.children.cycles-pp.current_time 1.56 ± 3% -0.9 0.66 ± 9% perf-profile.children.cycles-pp.copyout 1.56 ± 8% -0.8 0.73 ± 7% perf-profile.children.cycles-pp.touch_atime 1.33 ± 4% -0.8 0.53 ± 10% perf-profile.children.cycles-pp.__might_sleep 1.15 ± 9% -0.8 0.36 ± 12% perf-profile.children.cycles-pp.atime_needs_update 1.17 ± 4% -0.7 0.43 ± 13% perf-profile.children.cycles-pp.copyin 1.23 ± 5% -0.7 0.54 ± 9% perf-profile.children.cycles-pp.__fsnotify_parent 1.32 ± 7% -0.6 0.68 ± 6% perf-profile.children.cycles-pp.file_update_time 1.02 ± 3% -0.6 0.38 ± 12% perf-profile.children.cycles-pp.__might_fault 2.54 ± 2% -0.5 2.03 perf-profile.children.cycles-pp.mutex_lock 0.78 ± 4% -0.4 0.36 ± 10% perf-profile.children.cycles-pp.fput_many 0.53 ± 4% -0.4 0.15 ± 21% perf-profile.children.cycles-pp.inode_has_perm 0.44 ± 12% -0.3 0.11 ± 22% perf-profile.children.cycles-pp.iov_iter_init 0.76 ± 4% -0.3 0.44 ± 8% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.53 ± 3% -0.3 0.21 ± 6% perf-profile.children.cycles-pp.rcu_all_qs 0.41 ± 9% -0.3 0.12 ± 15% perf-profile.children.cycles-pp.timespec64_trunc 0.97 -0.3 0.71 ± 3% perf-profile.children.cycles-pp._cond_resched 0.47 ± 7% -0.3 0.21 ± 11% perf-profile.children.cycles-pp.__x64_sys_read 0.45 ± 7% -0.3 0.20 ± 10% perf-profile.children.cycles-pp.__x64_sys_write 0.37 ± 4% -0.2 0.17 ± 16% perf-profile.children.cycles-pp.generic_pipe_buf_confirm 0.44 ± 12% -0.2 0.24 ± 3% perf-profile.children.cycles-pp.__sb_start_write 0.31 ± 6% -0.2 0.12 ± 13% perf-profile.children.cycles-pp.__sb_end_write 0.36 ± 7% -0.2 0.18 ± 6% perf-profile.children.cycles-pp.rw_verify_area 0.26 ± 14% -0.2 0.09 ± 14% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64 0.29 ± 8% -0.2 0.14 ± 16% perf-profile.children.cycles-pp.fpregs_assert_state_consistent 0.22 ± 39% -0.2 0.06 ± 11% perf-profile.children.cycles-pp.__vfs_read 0.21 ± 19% -0.1 0.11 ± 9% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 0.12 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_write 0.12 ± 17% -0.0 0.08 ± 11% perf-profile.children.cycles-pp.wake_up_q 0.08 ± 10% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.kill_fasync 0.11 ± 7% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.09 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.01 ±173% +0.1 0.06 ± 6% perf-profile.children.cycles-pp.__mnt_want_write 0.07 ± 25% +0.1 0.12 ± 27% perf-profile.children.cycles-pp.__softirqentry_text_start 0.08 ± 30% +0.1 0.14 ± 26% perf-profile.children.cycles-pp.irq_exit 0.06 ± 6% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.__mark_inode_dirty 0.00 +0.1 0.06 perf-profile.children.cycles-pp.preempt_schedule_common 0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.is_cpu_allowed 0.03 ±100% +0.1 0.10 ± 8% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.__x2apic_send_IPI_dest 0.01 ±173% +0.1 0.09 ± 32% perf-profile.children.cycles-pp.migrate_task_rq_fair 0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.update_cfs_rq_h_load 0.05 ± 61% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.native_irq_return_iret 0.00 +0.1 0.08 ± 8% perf-profile.children.cycles-pp.rcu_note_context_switch 0.00 +0.1 0.08 ± 13% perf-profile.children.cycles-pp.smp_reschedule_interrupt 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.native_load_tls 0.03 ±102% +0.1 0.12 ± 30% perf-profile.children.cycles-pp.set_task_cpu 0.00 +0.1 0.09 ± 11% perf-profile.children.cycles-pp.check_cfs_rq_runtime 0.00 +0.1 0.10 ± 5% perf-profile.children.cycles-pp.rb_next 0.10 ± 10% +0.1 0.20 ± 9% perf-profile.children.cycles-pp.generic_update_time 0.00 +0.1 0.10 ± 29% perf-profile.children.cycles-pp.__x64_sys_exit 0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.do_exit 0.00 +0.1 0.11 ± 8% perf-profile.children.cycles-pp.rb_insert_color 0.00 +0.1 0.11 ± 7% perf-profile.children.cycles-pp.__list_add_valid 0.00 +0.1 0.12 ± 7% perf-profile.children.cycles-pp.resched_curr 0.00 +0.1 0.12 ± 6% perf-profile.children.cycles-pp.switch_mm_irqs_off 0.13 ± 10% +0.1 0.26 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid 0.00 +0.1 0.15 ± 4% perf-profile.children.cycles-pp.set_next_buddy 0.00 +0.2 0.16 ± 5% perf-profile.children.cycles-pp.deactivate_task 0.05 ± 9% +0.2 0.22 ± 7% perf-profile.children.cycles-pp.rb_erase 0.06 ± 13% +0.2 0.23 ± 9% perf-profile.children.cycles-pp.finish_wait 0.17 ± 12% +0.2 0.35 ± 2% perf-profile.children.cycles-pp.anon_pipe_buf_release 0.01 ±173% +0.2 0.20 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.00 +0.2 0.21 ± 4% perf-profile.children.cycles-pp.cpumask_next 0.07 ± 22% +0.3 0.40 ± 2% perf-profile.children.cycles-pp.prepare_to_wait 0.03 ±100% +0.4 0.39 ± 5% perf-profile.children.cycles-pp.update_min_vruntime 0.03 ±100% +0.4 0.39 ± 12% perf-profile.children.cycles-pp.cpus_share_cache 0.06 ± 14% +0.4 0.43 perf-profile.children.cycles-pp.clear_buddies 0.11 ± 15% +0.4 0.50 ± 3% perf-profile.children.cycles-pp.cpuacct_charge 0.10 ± 26% +0.4 0.49 ± 4% perf-profile.children.cycles-pp.native_sched_clock 0.32 ± 31% +0.4 0.75 ± 14% perf-profile.children.cycles-pp.sched_ttwu_pending 0.11 ± 26% +0.4 0.56 ± 3% perf-profile.children.cycles-pp.sched_clock 0.34 ± 32% +0.5 0.80 ± 15% perf-profile.children.cycles-pp.scheduler_ipi 0.08 ± 12% +0.5 0.56 ± 7% perf-profile.children.cycles-pp.account_entity_enqueue 0.13 ± 26% +0.5 0.62 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.14 ± 26% +0.6 0.77 ± 4% perf-profile.children.cycles-pp.__calc_delta 0.08 ± 16% +0.7 0.76 ± 3% perf-profile.children.cycles-pp.account_entity_dequeue 0.84 ± 5% +0.7 1.54 ± 5% perf-profile.children.cycles-pp.__lock_text_start 1.61 ± 16% +0.7 2.31 ± 9% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.42 ± 31% +0.7 1.12 ± 13% perf-profile.children.cycles-pp.reschedule_interrupt 0.24 ± 28% +0.9 1.11 perf-profile.children.cycles-pp.native_write_msr 0.14 ± 22% +0.9 1.01 ± 5% perf-profile.children.cycles-pp.check_preempt_wakeup 0.24 ± 32% +0.9 1.12 ± 7% perf-profile.children.cycles-pp.finish_task_switch 0.22 ± 22% +0.9 1.10 ± 3% perf-profile.children.cycles-pp.find_next_bit 0.22 ± 23% +0.9 1.12 ± 3% perf-profile.children.cycles-pp.set_next_entity 0.21 ± 32% +0.9 1.14 ± 3% perf-profile.children.cycles-pp.update_cfs_group 0.16 ± 23% +1.0 1.15 ± 4% perf-profile.children.cycles-pp.check_preempt_curr 0.21 ± 30% +1.0 1.21 ± 2% perf-profile.children.cycles-pp.__switch_to 0.23 ± 21% +1.0 1.24 ± 2% perf-profile.children.cycles-pp.__switch_to_asm 0.16 ± 38% +1.0 1.20 ± 4% perf-profile.children.cycles-pp.___perf_sw_event 0.15 ± 26% +1.0 1.20 ± 3% perf-profile.children.cycles-pp.pick_next_entity 0.15 ± 20% +1.1 1.21 ± 7% perf-profile.children.cycles-pp.put_prev_entity 0.17 ± 22% +1.1 1.23 ± 4% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.19 ± 10% +1.3 1.52 ± 10% perf-profile.children.cycles-pp.__enqueue_entity 0.24 ± 23% +1.3 1.59 ± 6% perf-profile.children.cycles-pp.__update_load_avg_se 0.28 ± 23% +1.4 1.64 ± 3% perf-profile.children.cycles-pp.switch_fpu_return 0.31 ± 29% +1.6 1.88 ± 5% perf-profile.children.cycles-pp.update_rq_clock 0.44 ± 21% +1.8 2.24 ± 4% perf-profile.children.cycles-pp.cpumask_next_wrap 0.33 ± 20% +1.8 2.15 ± 4% perf-profile.children.cycles-pp.dequeue_entity 23.28 ± 2% +2.0 25.27 ± 2% perf-profile.children.cycles-pp.ksys_read 0.46 ± 25% +2.1 2.57 ± 5% perf-profile.children.cycles-pp.update_load_avg 0.20 ± 21% +2.1 2.34 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.91 ± 29% +2.2 3.12 ± 11% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.38 ± 23% +2.4 2.80 ± 4% perf-profile.children.cycles-pp.reweight_entity 0.52 ± 18% +2.4 2.96 ± 3% perf-profile.children.cycles-pp.enqueue_entity 0.49 ± 23% +2.6 3.10 ± 4% perf-profile.children.cycles-pp.update_curr 20.92 ± 2% +3.2 24.12 ± 2% perf-profile.children.cycles-pp.vfs_read 0.21 ± 23% +3.4 3.65 ± 7% perf-profile.children.cycles-pp._raw_spin_lock 0.76 ± 24% +3.9 4.67 ± 3% perf-profile.children.cycles-pp.pick_next_task_fair 0.66 ± 34% +4.4 5.06 ± 2% perf-profile.children.cycles-pp.exit_to_usermode_loop 0.81 ± 22% +5.0 5.83 ± 4% perf-profile.children.cycles-pp.dequeue_task_fair 1.01 ± 21% +5.8 6.86 ± 4% perf-profile.children.cycles-pp.enqueue_task_fair 1.06 ± 21% +6.0 7.10 ± 4% perf-profile.children.cycles-pp.activate_task 1.07 ± 21% +6.1 7.15 ± 4% perf-profile.children.cycles-pp.ttwu_do_activate 13.19 +7.8 20.94 ± 3% perf-profile.children.cycles-pp.new_sync_read 11.61 +8.7 20.29 ± 3% perf-profile.children.cycles-pp.pipe_read 1.45 ± 25% +8.9 10.35 ± 7% perf-profile.children.cycles-pp.available_idle_cpu 78.85 ± 2% +9.8 88.66 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 77.90 ± 2% +10.1 88.00 perf-profile.children.cycles-pp.do_syscall_64 2.42 ± 21% +12.9 15.36 ± 4% perf-profile.children.cycles-pp.pipe_wait 2.71 ± 26% +15.4 18.12 ± 4% perf-profile.children.cycles-pp.__schedule 2.76 ± 25% +15.6 18.37 ± 4% perf-profile.children.cycles-pp.schedule 27.59 ± 2% +18.6 46.17 ± 3% perf-profile.children.cycles-pp.ksys_write 2.26 ± 23% +19.0 21.30 ± 7% perf-profile.children.cycles-pp.select_idle_sibling 24.98 ± 2% +19.9 44.88 ± 3% perf-profile.children.cycles-pp.vfs_write 2.48 ± 23% +20.0 22.46 ± 7% perf-profile.children.cycles-pp.select_task_rq_fair 19.06 ± 3% +23.3 42.41 ± 4% perf-profile.children.cycles-pp.new_sync_write 17.98 ± 4% +23.9 41.93 ± 4% perf-profile.children.cycles-pp.pipe_write 4.03 ± 21% +30.6 34.60 ± 6% perf-profile.children.cycles-pp.try_to_wake_up 3.97 ± 22% +30.8 34.74 ± 6% perf-profile.children.cycles-pp.autoremove_wake_function 4.30 ± 20% +30.9 35.19 ± 6% perf-profile.children.cycles-pp.__wake_up_common 6.06 ± 14% +31.2 37.21 ± 5% perf-profile.children.cycles-pp.__wake_up_common_lock 24.41 ± 6% -15.5 8.96 ± 10% perf-profile.self.cycles-pp.do_syscall_64 9.18 ± 6% -6.0 3.14 ± 12% perf-profile.self.cycles-pp.syscall_return_via_sysret 8.63 ± 5% -5.6 3.00 ± 12% perf-profile.self.cycles-pp.entry_SYSCALL_64 3.71 ± 3% -2.0 1.73 ± 6% perf-profile.self.cycles-pp.mutex_unlock 2.47 ± 3% -1.5 0.96 ± 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 2.76 ± 3% -1.4 1.35 ± 8% perf-profile.self.cycles-pp.selinux_file_permission 2.02 ± 7% -1.3 0.68 ± 12% perf-profile.self.cycles-pp.fsnotify 2.46 ± 6% -1.3 1.20 ± 8% perf-profile.self.cycles-pp.__fget 1.75 ± 6% -1.1 0.64 ± 10% perf-profile.self.cycles-pp.avc_has_perm 1.83 ± 5% -1.1 0.75 ± 10% perf-profile.self.cycles-pp.___might_sleep 1.75 -1.0 0.77 ± 7% perf-profile.self.cycles-pp.pipe_write 1.20 ± 4% -0.7 0.46 ± 10% perf-profile.self.cycles-pp.__might_sleep 1.29 ± 11% -0.7 0.58 ± 10% perf-profile.self.cycles-pp.new_sync_read 1.63 ± 4% -0.7 0.97 ± 5% perf-profile.self.cycles-pp.pipe_read 1.15 ± 5% -0.7 0.49 ± 8% perf-profile.self.cycles-pp.__fsnotify_parent 0.89 ± 8% -0.6 0.27 ± 15% perf-profile.self.cycles-pp.copy_page_from_iter 0.89 ± 4% -0.6 0.29 ± 14% perf-profile.self.cycles-pp.security_file_permission 0.83 ± 5% -0.6 0.24 ± 21% perf-profile.self.cycles-pp.file_has_perm 0.90 ± 2% -0.5 0.41 ± 6% perf-profile.self.cycles-pp.new_sync_write 0.67 ± 10% -0.5 0.20 ± 15% perf-profile.self.cycles-pp.current_time 0.83 ± 4% -0.4 0.41 ± 10% perf-profile.self.cycles-pp.copy_page_to_iter 0.72 ± 7% -0.4 0.31 ± 9% perf-profile.self.cycles-pp.vfs_write 0.75 ± 4% -0.4 0.35 ± 9% perf-profile.self.cycles-pp.fput_many 1.01 ± 4% -0.3 0.69 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.45 ± 4% -0.3 0.14 ± 19% perf-profile.self.cycles-pp.inode_has_perm 0.54 ± 8% -0.3 0.23 ± 9% perf-profile.self.cycles-pp.file_update_time 0.41 ± 10% -0.3 0.11 ± 20% perf-profile.self.cycles-pp.iov_iter_init 0.49 ± 7% -0.3 0.19 ± 8% perf-profile.self.cycles-pp.atime_needs_update 0.44 ± 7% -0.3 0.15 ± 14% perf-profile.self.cycles-pp.ksys_read 0.44 ± 7% -0.3 0.17 ± 9% perf-profile.self.cycles-pp.__inode_security_revalidate 0.41 ± 2% -0.3 0.15 ± 8% perf-profile.self.cycles-pp.rcu_all_qs 0.43 ± 8% -0.3 0.17 ± 11% perf-profile.self.cycles-pp.ksys_write 0.37 ± 9% -0.3 0.11 ± 17% perf-profile.self.cycles-pp.timespec64_trunc 0.42 ± 14% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.__sb_start_write 0.39 ± 7% -0.2 0.17 ± 11% perf-profile.self.cycles-pp.__x64_sys_write 0.40 ± 8% -0.2 0.17 ± 8% perf-profile.self.cycles-pp.__x64_sys_read 0.41 ± 7% -0.2 0.20 ± 11% perf-profile.self.cycles-pp.__wake_up_common_lock 0.60 ± 4% -0.2 0.39 ± 7% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 1.25 -0.2 1.04 ± 2% perf-profile.self.cycles-pp.mutex_lock 0.30 ± 6% -0.2 0.12 ± 15% perf-profile.self.cycles-pp.__sb_end_write 0.35 ± 5% -0.2 0.18 ± 3% perf-profile.self.cycles-pp.rw_verify_area 0.34 ± 7% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.touch_atime 0.32 ± 4% -0.2 0.15 ± 14% perf-profile.self.cycles-pp.generic_pipe_buf_confirm 0.23 ± 16% -0.2 0.08 ± 11% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.74 ± 3% -0.2 0.59 ± 5% perf-profile.self.cycles-pp.vfs_read 0.60 ± 3% -0.2 0.45 ± 4% perf-profile.self.cycles-pp.__fget_light 0.27 ± 8% -0.1 0.12 ± 15% perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.18 ± 45% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.__vfs_read 0.18 ± 5% -0.1 0.06 ± 17% perf-profile.self.cycles-pp.__fdget_pos 0.25 ± 5% -0.1 0.14 ± 11% perf-profile.self.cycles-pp.__might_fault 0.07 ± 7% -0.0 0.04 ± 57% perf-profile.self.cycles-pp.kill_fasync 0.01 ±173% +0.1 0.06 ± 17% perf-profile.self.cycles-pp.sched_ttwu_pending 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.scheduler_ipi 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.rcu_note_context_switch 0.01 ±173% +0.1 0.07 ± 12% perf-profile.self.cycles-pp.generic_update_time 0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.cpumask_next 0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.ttwu_do_activate 0.00 +0.1 0.06 perf-profile.self.cycles-pp.__mnt_want_write 0.06 ± 14% +0.1 0.12 ± 10% perf-profile.self.cycles-pp.__mark_inode_dirty 0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.is_cpu_allowed 0.00 +0.1 0.07 ± 13% perf-profile.self.cycles-pp.sched_clock_cpu 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.sched_clock 0.03 ±100% +0.1 0.10 ± 11% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.00 +0.1 0.08 perf-profile.self.cycles-pp.native_load_tls 0.05 ± 60% +0.1 0.13 ± 5% perf-profile.self.cycles-pp.native_irq_return_iret 0.00 +0.1 0.09 ± 9% perf-profile.self.cycles-pp.ttwu_do_wakeup 0.00 +0.1 0.09 ± 11% perf-profile.self.cycles-pp.rb_next 0.00 +0.1 0.10 ± 8% perf-profile.self.cycles-pp.__list_add_valid 0.00 +0.1 0.10 ± 10% perf-profile.self.cycles-pp.rb_insert_color 0.00 +0.1 0.11 ± 9% perf-profile.self.cycles-pp.resched_curr 0.00 +0.1 0.11 ± 7% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.32 ± 7% +0.1 0.45 ± 5% perf-profile.self.cycles-pp.__wake_up_common 0.13 ± 10% +0.1 0.25 ± 5% perf-profile.self.cycles-pp.__list_del_entry_valid 0.00 +0.1 0.13 ± 5% perf-profile.self.cycles-pp.autoremove_wake_function 0.00 +0.1 0.13 ± 8% perf-profile.self.cycles-pp.check_preempt_curr 0.04 ± 58% +0.1 0.18 ± 10% perf-profile.self.cycles-pp.finish_wait 0.00 +0.1 0.14 ± 5% perf-profile.self.cycles-pp.set_next_buddy 0.00 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.put_prev_entity 0.00 +0.2 0.15 ± 5% perf-profile.self.cycles-pp.deactivate_task 0.05 ± 9% +0.2 0.21 ± 5% perf-profile.self.cycles-pp.rb_erase 0.00 +0.2 0.15 ± 10% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.00 +0.2 0.18 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.15 ± 7% +0.2 0.34 ± 2% perf-profile.self.cycles-pp.anon_pipe_buf_release 0.03 ±100% +0.2 0.21 ± 8% perf-profile.self.cycles-pp.pipe_wait 0.03 ±102% +0.2 0.24 ± 3% perf-profile.self.cycles-pp.activate_task 0.01 ±173% +0.2 0.23 ± 3% perf-profile.self.cycles-pp.prepare_to_wait 0.03 ±100% +0.2 0.25 ± 4% perf-profile.self.cycles-pp.set_next_entity 0.02 ±173% +0.2 0.26 ± 9% perf-profile.self.cycles-pp.dequeue_entity 0.04 ±100% +0.3 0.29 ± 5% perf-profile.self.cycles-pp.enqueue_entity 0.08 ± 31% +0.3 0.34 ± 2% perf-profile.self.cycles-pp.schedule 0.78 ± 3% +0.3 1.09 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.09 ± 24% +0.3 0.42 ± 7% perf-profile.self.cycles-pp.finish_task_switch 0.03 ±100% +0.3 0.37 ± 6% perf-profile.self.cycles-pp.update_min_vruntime 0.03 ±102% +0.4 0.38 perf-profile.self.cycles-pp.clear_buddies 0.03 ±100% +0.4 0.39 ± 12% perf-profile.self.cycles-pp.cpus_share_cache 0.07 ± 22% +0.4 0.44 ± 7% perf-profile.self.cycles-pp.check_preempt_wakeup 0.09 ± 27% +0.4 0.47 ± 4% perf-profile.self.cycles-pp.native_sched_clock 0.07 ± 26% +0.4 0.45 ± 4% perf-profile.self.cycles-pp.pick_next_entity 0.11 ± 15% +0.4 0.50 ± 3% perf-profile.self.cycles-pp.cpuacct_charge 0.07 ± 14% +0.5 0.53 ± 8% perf-profile.self.cycles-pp.account_entity_enqueue 0.10 ± 21% +0.5 0.56 ± 8% perf-profile.self.cycles-pp.try_to_wake_up 0.10 ± 18% +0.5 0.62 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair 0.14 ± 26% +0.6 0.75 ± 4% perf-profile.self.cycles-pp.__calc_delta 0.06 ± 13% +0.6 0.68 ± 4% perf-profile.self.cycles-pp.account_entity_dequeue 0.15 ± 27% +0.6 0.79 ± 6% perf-profile.self.cycles-pp.enqueue_task_fair 0.17 ± 30% +0.7 0.89 ± 4% perf-profile.self.cycles-pp.update_load_avg 0.15 ± 20% +0.7 0.88 ± 4% perf-profile.self.cycles-pp.reweight_entity 0.15 ± 23% +0.8 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair 0.24 ± 30% +0.9 1.11 perf-profile.self.cycles-pp.native_write_msr 0.20 ± 23% +0.9 1.07 ± 5% perf-profile.self.cycles-pp.select_task_rq_fair 0.20 ± 22% +0.9 1.07 ± 3% perf-profile.self.cycles-pp.find_next_bit 0.20 ± 31% +0.9 1.11 ± 3% perf-profile.self.cycles-pp.update_cfs_group 0.20 ± 28% +0.9 1.14 ± 2% perf-profile.self.cycles-pp.__switch_to 0.14 ± 37% +1.0 1.13 ± 5% perf-profile.self.cycles-pp.___perf_sw_event 0.21 ± 24% +1.0 1.22 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 0.24 ± 23% +1.1 1.31 ± 5% perf-profile.self.cycles-pp.cpumask_next_wrap 0.23 ± 32% +1.3 1.49 ± 5% perf-profile.self.cycles-pp.update_rq_clock 0.19 ± 28% +1.3 1.45 ± 6% perf-profile.self.cycles-pp.update_curr 0.23 ± 23% +1.3 1.55 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se 0.19 ± 10% +1.3 1.51 ± 10% perf-profile.self.cycles-pp.__enqueue_entity 0.28 ± 22% +1.3 1.63 ± 3% perf-profile.self.cycles-pp.switch_fpu_return 0.11 ± 19% +1.6 1.72 ± 5% perf-profile.self.cycles-pp._raw_spin_lock 0.42 ± 26% +1.7 2.12 ± 2% perf-profile.self.cycles-pp.__schedule 0.19 ± 20% +2.0 2.20 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.91 ± 29% +2.2 3.11 ± 12% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.32 ± 19% +7.7 8.05 ± 7% perf-profile.self.cycles-pp.select_idle_sibling 1.44 ± 25% +8.8 10.23 ± 7% perf-profile.self.cycles-pp.available_idle_cpu 439.00 +11.7% 490.50 interrupts.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59 436.25 +13.1% 493.25 interrupts.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60 436.25 +12.4% 490.50 interrupts.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61 436.25 +12.4% 490.50 interrupts.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62 509.75 ± 3% +545.1% 3288 ±120% interrupts.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3 474.00 ± 3% +49.7% 709.75 ± 24% interrupts.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7 588.00 ± 35% +196.0% 1740 ± 50% interrupts.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12 436.75 ± 2% +12.8% 492.50 interrupts.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16 438.00 +39.5% 611.00 ± 33% interrupts.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18 436.25 +13.8% 496.25 interrupts.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19 436.25 +14.3% 498.50 interrupts.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20 437.00 +12.2% 490.50 interrupts.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21 436.25 +12.4% 490.50 interrupts.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22 436.25 +12.4% 490.50 interrupts.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23 441.00 +11.8% 493.00 interrupts.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24 437.25 +13.3% 495.25 interrupts.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25 436.25 +12.4% 490.50 interrupts.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26 436.25 +12.4% 490.50 interrupts.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29 436.25 +12.4% 490.50 interrupts.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30 436.25 +12.4% 490.50 interrupts.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32 437.25 ± 2% +12.2% 490.50 interrupts.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33 436.25 +12.6% 491.25 interrupts.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34 440.75 +12.4% 495.25 ± 2% interrupts.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35 436.25 +12.6% 491.00 interrupts.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36 436.75 +13.1% 493.75 interrupts.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37 436.25 +12.8% 492.25 interrupts.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38 436.25 +12.8% 492.25 interrupts.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39 436.25 +12.4% 490.50 interrupts.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40 436.25 +12.4% 490.50 interrupts.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41 436.25 +12.4% 490.50 interrupts.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42 436.25 +14.6% 499.75 ± 2% interrupts.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43 436.25 +12.4% 490.50 interrupts.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44 436.50 +12.4% 490.50 interrupts.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45 436.25 +12.4% 490.50 interrupts.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46 436.25 +12.4% 490.50 interrupts.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47 436.25 +12.6% 491.00 interrupts.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48 441.25 ± 3% +14.1% 503.25 ± 3% interrupts.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49 436.25 +12.4% 490.50 interrupts.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50 436.25 +14.4% 499.25 ± 3% interrupts.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51 439.25 ± 2% +11.7% 490.50 interrupts.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53 437.75 +12.1% 490.50 interrupts.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54 436.50 +12.4% 490.50 interrupts.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55 436.25 +12.4% 490.50 interrupts.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56 439.00 ± 2% +11.7% 490.50 interrupts.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57 442.25 ± 3% +10.9% 490.50 interrupts.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58 1989 +12.1% 2230 interrupts.9:IR-IO-APIC.9-fasteoi.acpi 6596036 ± 10% -21.1% 5206443 ± 6% interrupts.CAL:Function_call_interrupts 86097 ± 13% -18.4% 70229 ± 7% interrupts.CPU0.CAL:Function_call_interrupts 1785369 +12.4% 2006294 interrupts.CPU0.LOC:Local_timer_interrupts 102823 ± 12% -18.3% 84031 ± 5% interrupts.CPU0.TLB:TLB_shootdowns 388425 ± 12% +27.8% 496548 ± 4% interrupts.CPU0.TRM:Thermal_event_interrupts 1989 +12.1% 2230 interrupts.CPU1.9:IR-IO-APIC.9-fasteoi.acpi 90370 ± 8% -21.4% 70995 ± 6% interrupts.CPU1.CAL:Function_call_interrupts 1784834 +12.4% 2006046 interrupts.CPU1.LOC:Local_timer_interrupts 3088391 ± 15% -20.9% 2441882 ± 7% interrupts.CPU1.RES:Rescheduling_interrupts 105391 ± 8% -18.6% 85793 ± 6% interrupts.CPU1.TLB:TLB_shootdowns 388373 ± 12% +27.9% 496558 ± 4% interrupts.CPU1.TRM:Thermal_event_interrupts 88236 ± 11% -19.5% 71001 ± 7% interrupts.CPU10.CAL:Function_call_interrupts 1785471 +12.3% 2004740 interrupts.CPU10.LOC:Local_timer_interrupts 3040774 ± 4% -21.1% 2398396 ± 8% interrupts.CPU10.RES:Rescheduling_interrupts 104483 ± 10% -18.6% 85025 ± 5% interrupts.CPU10.TLB:TLB_shootdowns 388417 ± 12% +27.8% 496560 ± 4% interrupts.CPU10.TRM:Thermal_event_interrupts 1785039 +12.3% 2005274 interrupts.CPU11.LOC:Local_timer_interrupts 925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.NMI:Non-maskable_interrupts 925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.PMI:Performance_monitoring_interrupts 3094193 ± 14% -21.5% 2430228 ± 8% interrupts.CPU11.RES:Rescheduling_interrupts 102398 ± 10% -16.1% 85884 ± 5% interrupts.CPU11.TLB:TLB_shootdowns 388413 ± 12% +27.8% 496474 ± 4% interrupts.CPU11.TRM:Thermal_event_interrupts 588.00 ± 35% +196.0% 1740 ± 50% interrupts.CPU12.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12 87438 ± 9% -18.1% 71580 ± 7% interrupts.CPU12.CAL:Function_call_interrupts 1785376 +12.3% 2005068 interrupts.CPU12.LOC:Local_timer_interrupts 103057 ± 10% -16.4% 86186 ± 6% interrupts.CPU12.TLB:TLB_shootdowns 388416 ± 12% +27.8% 496490 ± 4% interrupts.CPU12.TRM:Thermal_event_interrupts 88735 ± 9% -19.4% 71496 ± 9% interrupts.CPU13.CAL:Function_call_interrupts 1785460 +12.3% 2005445 interrupts.CPU13.LOC:Local_timer_interrupts 104230 ± 9% -17.5% 86038 ± 6% interrupts.CPU13.TLB:TLB_shootdowns 388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU13.TRM:Thermal_event_interrupts 88327 ± 12% -19.0% 71507 ± 7% interrupts.CPU14.CAL:Function_call_interrupts 1785094 +12.3% 2005441 interrupts.CPU14.LOC:Local_timer_interrupts 103317 ± 11% -18.1% 84581 ± 6% interrupts.CPU14.TLB:TLB_shootdowns 388412 ± 12% +27.8% 496564 ± 4% interrupts.CPU14.TRM:Thermal_event_interrupts 86972 ± 10% -17.3% 71898 ± 7% interrupts.CPU15.CAL:Function_call_interrupts 1784668 +12.4% 2005375 interrupts.CPU15.LOC:Local_timer_interrupts 3039799 ± 9% -21.1% 2397505 ± 9% interrupts.CPU15.RES:Rescheduling_interrupts 103367 ± 12% -16.9% 85892 ± 6% interrupts.CPU15.TLB:TLB_shootdowns 388342 ± 12% +27.9% 496559 ± 4% interrupts.CPU15.TRM:Thermal_event_interrupts 436.75 ± 2% +12.8% 492.50 interrupts.CPU16.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16 1784750 +12.4% 2005330 interrupts.CPU16.LOC:Local_timer_interrupts 55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.NMI:Non-maskable_interrupts 55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.PMI:Performance_monitoring_interrupts 2969996 ± 10% -18.7% 2413141 ± 9% interrupts.CPU16.RES:Rescheduling_interrupts 102367 ± 11% -13.7% 88305 ± 3% interrupts.CPU16.TLB:TLB_shootdowns 388423 ± 12% +27.8% 496563 ± 4% interrupts.CPU16.TRM:Thermal_event_interrupts 1785363 +12.3% 2005849 interrupts.CPU17.LOC:Local_timer_interrupts 388418 ± 12% +27.8% 496560 ± 4% interrupts.CPU17.TRM:Thermal_event_interrupts 438.00 +39.5% 611.00 ± 33% interrupts.CPU18.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18 88992 ± 12% -22.3% 69145 ± 6% interrupts.CPU18.CAL:Function_call_interrupts 1785398 +12.1% 2001316 interrupts.CPU18.LOC:Local_timer_interrupts 2199105 ± 7% +115.2% 4731679 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts 108465 ± 8% -23.0% 83467 ± 5% interrupts.CPU18.TLB:TLB_shootdowns 329294 ± 3% +15.4% 380131 ± 2% interrupts.CPU18.TRM:Thermal_event_interrupts 436.25 +13.8% 496.25 interrupts.CPU19.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19 92059 ± 6% -24.2% 69776 ± 5% interrupts.CPU19.CAL:Function_call_interrupts 1784665 +12.2% 2001579 interrupts.CPU19.LOC:Local_timer_interrupts 1284622 ± 7% +225.8% 4185208 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts 108492 ± 7% -23.4% 83059 ± 4% interrupts.CPU19.TLB:TLB_shootdowns 329166 ± 3% +15.5% 380141 ± 2% interrupts.CPU19.TRM:Thermal_event_interrupts 1784512 +12.4% 2005572 interrupts.CPU2.LOC:Local_timer_interrupts 3091132 ± 8% -22.3% 2401342 ± 4% interrupts.CPU2.RES:Rescheduling_interrupts 102562 ± 9% -16.5% 85643 ± 5% interrupts.CPU2.TLB:TLB_shootdowns 388380 ± 12% +27.8% 496544 ± 4% interrupts.CPU2.TRM:Thermal_event_interrupts 436.25 +14.3% 498.50 interrupts.CPU20.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20 91785 ± 9% -23.9% 69853 ± 4% interrupts.CPU20.CAL:Function_call_interrupts 1785284 +12.1% 2001952 interrupts.CPU20.LOC:Local_timer_interrupts 1281621 ± 12% +203.6% 3890516 ± 7% interrupts.CPU20.RES:Rescheduling_interrupts 106297 ± 9% -19.9% 85180 ± 3% interrupts.CPU20.TLB:TLB_shootdowns 329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU20.TRM:Thermal_event_interrupts 437.00 +12.2% 490.50 interrupts.CPU21.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21 92635 ± 7% -23.9% 70512 ± 6% interrupts.CPU21.CAL:Function_call_interrupts 1785480 +12.1% 2001723 interrupts.CPU21.LOC:Local_timer_interrupts 1595447 ± 6% +187.8% 4591004 interrupts.CPU21.RES:Rescheduling_interrupts 107575 ± 7% -21.6% 84296 ± 5% interrupts.CPU21.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380086 ± 2% interrupts.CPU21.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU22.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22 92674 ± 10% -24.4% 70047 ± 6% interrupts.CPU22.CAL:Function_call_interrupts 1784881 +12.1% 2001482 interrupts.CPU22.LOC:Local_timer_interrupts 1605143 ± 6% +179.5% 4486945 ± 5% interrupts.CPU22.RES:Rescheduling_interrupts 108749 ± 10% -23.3% 83409 ± 5% interrupts.CPU22.TLB:TLB_shootdowns 329295 ± 3% +15.4% 380136 ± 2% interrupts.CPU22.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU23.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23 90975 ± 11% -23.6% 69492 ± 5% interrupts.CPU23.CAL:Function_call_interrupts 1785236 +12.1% 2001811 interrupts.CPU23.LOC:Local_timer_interrupts 1600988 ± 2% +180.2% 4486667 ± 5% interrupts.CPU23.RES:Rescheduling_interrupts 106819 ± 10% -22.3% 83007 ± 4% interrupts.CPU23.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380146 ± 2% interrupts.CPU23.TRM:Thermal_event_interrupts 441.00 +11.8% 493.00 interrupts.CPU24.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24 91919 ± 11% -23.1% 70689 ± 5% interrupts.CPU24.CAL:Function_call_interrupts 1784996 +12.1% 2000922 interrupts.CPU24.LOC:Local_timer_interrupts 1612237 ± 5% +188.4% 4649702 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts 108187 ± 8% -22.5% 83831 ± 3% interrupts.CPU24.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380122 ± 2% interrupts.CPU24.TRM:Thermal_event_interrupts 437.25 +13.3% 495.25 interrupts.CPU25.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25 88417 ± 12% -20.8% 70029 ± 4% interrupts.CPU25.CAL:Function_call_interrupts 1785404 +12.0% 1999622 interrupts.CPU25.LOC:Local_timer_interrupts 1612074 ± 2% +182.4% 4551699 interrupts.CPU25.RES:Rescheduling_interrupts 105324 ± 10% -20.1% 84201 ± 4% interrupts.CPU25.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU25.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU26.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26 92632 ± 12% -23.5% 70907 ± 5% interrupts.CPU26.CAL:Function_call_interrupts 1784730 +12.1% 2000264 interrupts.CPU26.LOC:Local_timer_interrupts 1639148 ± 6% +188.0% 4720149 ± 4% interrupts.CPU26.RES:Rescheduling_interrupts 108099 ± 11% -21.6% 84727 ± 4% interrupts.CPU26.TLB:TLB_shootdowns 329271 ± 3% +15.4% 379984 ± 2% interrupts.CPU26.TRM:Thermal_event_interrupts 91312 ± 12% -23.4% 69963 ± 6% interrupts.CPU27.CAL:Function_call_interrupts 1785171 +12.1% 2001743 interrupts.CPU27.LOC:Local_timer_interrupts 1704847 ± 11% +179.4% 4764019 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts 106515 ± 11% -21.6% 83491 ± 5% interrupts.CPU27.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380080 ± 2% interrupts.CPU27.TRM:Thermal_event_interrupts 89178 ± 8% -22.0% 69565 ± 6% interrupts.CPU28.CAL:Function_call_interrupts 1785194 +12.1% 2001836 interrupts.CPU28.LOC:Local_timer_interrupts 1593025 ± 8% +185.7% 4551782 ± 5% interrupts.CPU28.RES:Rescheduling_interrupts 104408 ± 10% -19.2% 84365 ± 6% interrupts.CPU28.TLB:TLB_shootdowns 329294 ± 3% +15.4% 380142 ± 2% interrupts.CPU28.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU29.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29 90850 ± 12% -21.9% 70993 ± 6% interrupts.CPU29.CAL:Function_call_interrupts 1785398 +12.1% 2000545 interrupts.CPU29.LOC:Local_timer_interrupts 1657508 ± 5% +179.4% 4631109 ± 3% interrupts.CPU29.RES:Rescheduling_interrupts 106383 ± 13% -20.8% 84284 ± 4% interrupts.CPU29.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380133 ± 2% interrupts.CPU29.TRM:Thermal_event_interrupts 509.75 ± 3% +545.1% 3288 ±120% interrupts.CPU3.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3 87008 ± 11% -19.9% 69730 ± 5% interrupts.CPU3.CAL:Function_call_interrupts 1784446 +12.4% 2005368 interrupts.CPU3.LOC:Local_timer_interrupts 3169674 ± 9% -20.4% 2521963 ± 8% interrupts.CPU3.RES:Rescheduling_interrupts 103703 ± 12% -17.1% 85958 ± 7% interrupts.CPU3.TLB:TLB_shootdowns 388356 ± 12% +27.9% 496551 ± 4% interrupts.CPU3.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU30.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30 92043 ± 12% -24.2% 69765 ± 5% interrupts.CPU30.CAL:Function_call_interrupts 1785175 +12.1% 2001248 interrupts.CPU30.LOC:Local_timer_interrupts 1637465 ± 6% +191.3% 4769701 ± 3% interrupts.CPU30.RES:Rescheduling_interrupts 106836 ± 11% -22.5% 82810 ± 5% interrupts.CPU30.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU30.TRM:Thermal_event_interrupts 1785301 +12.0% 1999270 interrupts.CPU31.LOC:Local_timer_interrupts 62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.NMI:Non-maskable_interrupts 62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.PMI:Performance_monitoring_interrupts 1589500 ± 5% +184.1% 4515325 interrupts.CPU31.RES:Rescheduling_interrupts 106576 ± 11% -21.5% 83698 ± 5% interrupts.CPU31.TLB:TLB_shootdowns 329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU31.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU32.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32 93348 ± 12% -24.4% 70595 ± 8% interrupts.CPU32.CAL:Function_call_interrupts 1785245 +12.1% 2001145 interrupts.CPU32.LOC:Local_timer_interrupts 1626883 ± 6% +176.3% 4494456 ± 5% interrupts.CPU32.RES:Rescheduling_interrupts 107151 ± 12% -21.4% 84236 ± 5% interrupts.CPU32.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380131 ± 2% interrupts.CPU32.TRM:Thermal_event_interrupts 437.25 ± 2% +12.2% 490.50 interrupts.CPU33.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33 91487 ± 12% -22.7% 70758 ± 3% interrupts.CPU33.CAL:Function_call_interrupts 1785371 +12.0% 1999987 interrupts.CPU33.LOC:Local_timer_interrupts 1578761 ± 6% +201.2% 4754762 ± 6% interrupts.CPU33.RES:Rescheduling_interrupts 106071 ± 12% -21.3% 83430 ± 2% interrupts.CPU33.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU33.TRM:Thermal_event_interrupts 436.25 +12.6% 491.25 interrupts.CPU34.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34 91116 ± 11% -24.3% 68943 ± 7% interrupts.CPU34.CAL:Function_call_interrupts 1785353 +12.1% 2001525 interrupts.CPU34.LOC:Local_timer_interrupts 1553828 ± 4% +186.0% 4444035 ± 3% interrupts.CPU34.RES:Rescheduling_interrupts 105575 ± 10% -20.7% 83762 ± 4% interrupts.CPU34.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380139 ± 2% interrupts.CPU34.TRM:Thermal_event_interrupts 440.75 +12.4% 495.25 ± 2% interrupts.CPU35.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35 91396 ± 11% -25.6% 67986 ± 11% interrupts.CPU35.CAL:Function_call_interrupts 1785237 +12.0% 2000288 interrupts.CPU35.LOC:Local_timer_interrupts 1602616 ± 10% +189.7% 4642115 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts 107669 ± 12% -21.3% 84736 ± 4% interrupts.CPU35.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380145 ± 2% interrupts.CPU35.TRM:Thermal_event_interrupts 436.25 +12.6% 491.00 interrupts.CPU36.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36 93553 ± 8% -19.8% 75044 ± 6% interrupts.CPU36.CAL:Function_call_interrupts 1784636 +12.4% 2005389 interrupts.CPU36.LOC:Local_timer_interrupts 2588984 ± 6% +25.8% 3256348 ± 6% interrupts.CPU36.RES:Rescheduling_interrupts 107053 ± 7% -19.0% 86754 ± 5% interrupts.CPU36.TLB:TLB_shootdowns 388423 ± 12% +27.8% 496560 ± 4% interrupts.CPU36.TRM:Thermal_event_interrupts 436.75 +13.1% 493.75 interrupts.CPU37.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37 89518 ± 10% -18.9% 72619 ± 5% interrupts.CPU37.CAL:Function_call_interrupts 1784648 +12.4% 2005651 interrupts.CPU37.LOC:Local_timer_interrupts 3112476 ± 10% -20.8% 2464103 ± 5% interrupts.CPU37.RES:Rescheduling_interrupts 102576 ± 10% -14.4% 87794 ± 6% interrupts.CPU37.TLB:TLB_shootdowns 388412 ± 12% +27.8% 496563 ± 4% interrupts.CPU37.TRM:Thermal_event_interrupts 436.25 +12.8% 492.25 interrupts.CPU38.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38 92566 ± 10% -20.6% 73466 ± 6% interrupts.CPU38.CAL:Function_call_interrupts 1785132 +12.4% 2005883 interrupts.CPU38.LOC:Local_timer_interrupts 3163520 ± 10% -26.0% 2340820 ± 11% interrupts.CPU38.RES:Rescheduling_interrupts 105823 ± 10% -18.0% 86785 ± 4% interrupts.CPU38.TLB:TLB_shootdowns 388401 ± 12% +27.8% 496554 ± 4% interrupts.CPU38.TRM:Thermal_event_interrupts 436.25 +12.8% 492.25 interrupts.CPU39.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39 93197 ± 11% -21.8% 72854 ± 7% interrupts.CPU39.CAL:Function_call_interrupts 1784624 +12.4% 2005934 interrupts.CPU39.LOC:Local_timer_interrupts 105953 ± 11% -20.8% 83925 ± 6% interrupts.CPU39.TLB:TLB_shootdowns 388418 ± 12% +27.8% 496548 ± 4% interrupts.CPU39.TRM:Thermal_event_interrupts 86883 ± 13% -17.8% 71449 ± 7% interrupts.CPU4.CAL:Function_call_interrupts 1784433 +12.4% 2005462 interrupts.CPU4.LOC:Local_timer_interrupts 3119739 ± 9% -21.1% 2461192 ± 4% interrupts.CPU4.RES:Rescheduling_interrupts 104829 ± 11% -19.1% 84802 ± 5% interrupts.CPU4.TLB:TLB_shootdowns 388417 ± 12% +27.8% 496563 ± 4% interrupts.CPU4.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU40.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40 91881 ± 12% -21.4% 72188 ± 8% interrupts.CPU40.CAL:Function_call_interrupts 1784831 +12.4% 2005991 interrupts.CPU40.LOC:Local_timer_interrupts 3101288 ± 10% -17.2% 2569221 ± 7% interrupts.CPU40.RES:Rescheduling_interrupts 105884 ± 10% -19.0% 85772 ± 5% interrupts.CPU40.TLB:TLB_shootdowns 388415 ± 12% +27.8% 496564 ± 4% interrupts.CPU40.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU41.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41 91739 ± 12% -19.9% 73475 ± 7% interrupts.CPU41.CAL:Function_call_interrupts 1785231 +12.3% 2005398 interrupts.CPU41.LOC:Local_timer_interrupts 3223405 ± 11% -27.9% 2325498 ± 5% interrupts.CPU41.RES:Rescheduling_interrupts 104410 ± 11% -18.7% 84846 ± 6% interrupts.CPU41.TLB:TLB_shootdowns 388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU41.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU42.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42 1785354 +12.4% 2006034 interrupts.CPU42.LOC:Local_timer_interrupts 3068304 ± 9% -22.1% 2390347 ± 6% interrupts.CPU42.RES:Rescheduling_interrupts 107102 ± 9% -20.7% 84890 ± 4% interrupts.CPU42.TLB:TLB_shootdowns 388423 ± 12% +27.8% 496564 ± 4% interrupts.CPU42.TRM:Thermal_event_interrupts 436.25 +14.6% 499.75 ± 2% interrupts.CPU43.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43 92178 ± 9% -18.3% 75306 ± 7% interrupts.CPU43.CAL:Function_call_interrupts 1784759 +12.4% 2005986 interrupts.CPU43.LOC:Local_timer_interrupts 6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.NMI:Non-maskable_interrupts 6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.PMI:Performance_monitoring_interrupts 104773 ± 9% -17.6% 86369 ± 6% interrupts.CPU43.TLB:TLB_shootdowns 388425 ± 12% +27.8% 496556 ± 4% interrupts.CPU43.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU44.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44 92387 ± 10% -21.9% 72174 ± 9% interrupts.CPU44.CAL:Function_call_interrupts 1784603 +12.4% 2006064 interrupts.CPU44.LOC:Local_timer_interrupts 106179 ± 9% -18.6% 86461 ± 6% interrupts.CPU44.TLB:TLB_shootdowns 388426 ± 12% +27.8% 496562 ± 4% interrupts.CPU44.TRM:Thermal_event_interrupts 436.50 +12.4% 490.50 interrupts.CPU45.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45 93641 ± 7% -21.9% 73155 ± 6% interrupts.CPU45.CAL:Function_call_interrupts 1784811 +12.4% 2005654 interrupts.CPU45.LOC:Local_timer_interrupts 3093195 ± 11% -15.6% 2609354 ± 4% interrupts.CPU45.RES:Rescheduling_interrupts 106463 ± 7% -19.6% 85593 ± 5% interrupts.CPU45.TLB:TLB_shootdowns 388408 ± 12% +27.8% 496564 ± 4% interrupts.CPU45.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU46.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46 91739 ± 10% -19.0% 74279 ± 8% interrupts.CPU46.CAL:Function_call_interrupts 1784609 +12.4% 2005522 interrupts.CPU46.LOC:Local_timer_interrupts 2970471 ± 9% -21.0% 2345439 ± 8% interrupts.CPU46.RES:Rescheduling_interrupts 108158 ± 10% -19.8% 86743 ± 4% interrupts.CPU46.TLB:TLB_shootdowns 388368 ± 12% +27.8% 496469 ± 4% interrupts.CPU46.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU47.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47 92856 ± 9% -20.0% 74327 ± 8% interrupts.CPU47.CAL:Function_call_interrupts 1784835 +12.4% 2005700 interrupts.CPU47.LOC:Local_timer_interrupts 3116242 ± 10% -20.4% 2480398 ± 5% interrupts.CPU47.RES:Rescheduling_interrupts 104927 ± 9% -18.6% 85393 ± 6% interrupts.CPU47.TLB:TLB_shootdowns 388403 ± 12% +27.8% 496554 ± 4% interrupts.CPU47.TRM:Thermal_event_interrupts 436.25 +12.6% 491.00 interrupts.CPU48.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48 92889 ± 11% -21.5% 72890 ± 8% interrupts.CPU48.CAL:Function_call_interrupts 1784772 +12.4% 2006205 interrupts.CPU48.LOC:Local_timer_interrupts 3175210 ± 12% -19.7% 2548339 ± 13% interrupts.CPU48.RES:Rescheduling_interrupts 105183 ± 11% -18.6% 85618 ± 6% interrupts.CPU48.TLB:TLB_shootdowns 388426 ± 12% +27.8% 496563 ± 4% interrupts.CPU48.TRM:Thermal_event_interrupts 441.25 ± 3% +14.1% 503.25 ± 3% interrupts.CPU49.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49 90134 ± 10% -18.3% 73603 ± 6% interrupts.CPU49.CAL:Function_call_interrupts 1785145 +12.4% 2005932 interrupts.CPU49.LOC:Local_timer_interrupts 3185609 ± 11% -17.6% 2626502 ± 4% interrupts.CPU49.RES:Rescheduling_interrupts 102737 ± 10% -17.3% 84932 ± 5% interrupts.CPU49.TLB:TLB_shootdowns 388425 ± 12% +27.8% 496560 ± 4% interrupts.CPU49.TRM:Thermal_event_interrupts 92114 ± 9% -23.2% 70721 ± 6% interrupts.CPU5.CAL:Function_call_interrupts 1785135 +12.3% 2005424 interrupts.CPU5.LOC:Local_timer_interrupts 3210646 ± 12% -24.8% 2415935 ± 4% interrupts.CPU5.RES:Rescheduling_interrupts 107349 ± 9% -18.9% 87034 ± 6% interrupts.CPU5.TLB:TLB_shootdowns 388418 ± 12% +27.8% 496564 ± 4% interrupts.CPU5.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU50.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50 92406 ± 10% -20.2% 73771 ± 8% interrupts.CPU50.CAL:Function_call_interrupts 1785200 +12.4% 2006488 interrupts.CPU50.LOC:Local_timer_interrupts 106635 ± 9% -19.9% 85431 ± 6% interrupts.CPU50.TLB:TLB_shootdowns 388419 ± 12% +27.8% 496564 ± 4% interrupts.CPU50.TRM:Thermal_event_interrupts 436.25 +14.4% 499.25 ± 3% interrupts.CPU51.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51 91491 ± 9% -20.8% 72473 ± 6% interrupts.CPU51.CAL:Function_call_interrupts 1784683 +12.4% 2005527 interrupts.CPU51.LOC:Local_timer_interrupts 3051494 ± 8% -24.6% 2300032 ± 10% interrupts.CPU51.RES:Rescheduling_interrupts 104954 ± 10% -20.2% 83775 ± 5% interrupts.CPU51.TLB:TLB_shootdowns 388264 ± 12% +27.9% 496462 ± 4% interrupts.CPU51.TRM:Thermal_event_interrupts 92938 ± 11% -19.7% 74594 ± 7% interrupts.CPU52.CAL:Function_call_interrupts 1785451 +12.4% 2006171 interrupts.CPU52.LOC:Local_timer_interrupts 3025681 ± 11% -17.1% 2506800 ± 10% interrupts.CPU52.RES:Rescheduling_interrupts 107100 ± 11% -19.6% 86156 ± 6% interrupts.CPU52.TLB:TLB_shootdowns 388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU52.TRM:Thermal_event_interrupts 439.25 ± 2% +11.7% 490.50 interrupts.CPU53.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53 93000 ± 12% -20.9% 73549 ± 7% interrupts.CPU53.CAL:Function_call_interrupts 1785290 +12.3% 2005441 interrupts.CPU53.LOC:Local_timer_interrupts 3212623 ± 7% -18.2% 2629478 ± 10% interrupts.CPU53.RES:Rescheduling_interrupts 107303 ± 11% -17.5% 88567 ± 10% interrupts.CPU53.TLB:TLB_shootdowns 388411 ± 12% +27.8% 496557 ± 4% interrupts.CPU53.TRM:Thermal_event_interrupts 437.75 +12.1% 490.50 interrupts.CPU54.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54 1785170 +12.0% 2000090 interrupts.CPU54.LOC:Local_timer_interrupts 5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.NMI:Non-maskable_interrupts 5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.PMI:Performance_monitoring_interrupts 2066106 ± 9% +92.7% 3982033 ± 6% interrupts.CPU54.RES:Rescheduling_interrupts 108150 ± 12% -21.2% 85173 ± 5% interrupts.CPU54.TLB:TLB_shootdowns 329294 ± 3% +15.4% 380146 ± 2% interrupts.CPU54.TRM:Thermal_event_interrupts 436.50 +12.4% 490.50 interrupts.CPU55.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55 91164 ± 12% -19.9% 73014 ± 6% interrupts.CPU55.CAL:Function_call_interrupts 1785472 +12.0% 1999115 interrupts.CPU55.LOC:Local_timer_interrupts 1286654 ± 6% +228.2% 4223034 ± 5% interrupts.CPU55.RES:Rescheduling_interrupts 104943 ± 11% -19.5% 84466 ± 5% interrupts.CPU55.TLB:TLB_shootdowns 329292 ± 3% +15.4% 379918 ± 2% interrupts.CPU55.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU56.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56 96934 ± 13% -23.4% 74215 ± 6% interrupts.CPU56.CAL:Function_call_interrupts 1785010 +12.1% 2000169 interrupts.CPU56.LOC:Local_timer_interrupts 1312758 ± 17% +194.5% 3866522 ± 3% interrupts.CPU56.RES:Rescheduling_interrupts 109307 ± 12% -22.1% 85139 ± 4% interrupts.CPU56.TLB:TLB_shootdowns 329273 ± 3% +15.4% 380131 ± 2% interrupts.CPU56.TRM:Thermal_event_interrupts 439.00 ± 2% +11.7% 490.50 interrupts.CPU57.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57 94960 ± 11% -22.9% 73246 ± 5% interrupts.CPU57.CAL:Function_call_interrupts 1785297 +12.0% 2000019 interrupts.CPU57.LOC:Local_timer_interrupts 1666475 ± 10% +169.6% 4492937 interrupts.CPU57.RES:Rescheduling_interrupts 107534 ± 10% -22.0% 83871 ± 5% interrupts.CPU57.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380136 ± 2% interrupts.CPU57.TRM:Thermal_event_interrupts 442.25 ± 3% +10.9% 490.50 interrupts.CPU58.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58 92818 ± 12% -21.4% 72925 ± 5% interrupts.CPU58.CAL:Function_call_interrupts 1785429 +12.0% 1999528 interrupts.CPU58.LOC:Local_timer_interrupts 1606064 ± 5% +176.8% 4446179 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts 103696 ± 11% -19.5% 83458 ± 4% interrupts.CPU58.TLB:TLB_shootdowns 329293 ± 3% +15.4% 380022 ± 2% interrupts.CPU58.TRM:Thermal_event_interrupts 439.00 +11.7% 490.50 interrupts.CPU59.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59 93876 ± 13% -20.5% 74669 ± 6% interrupts.CPU59.CAL:Function_call_interrupts 1785085 +12.0% 1999992 interrupts.CPU59.LOC:Local_timer_interrupts 1634812 ± 5% +174.5% 4487501 ± 4% interrupts.CPU59.RES:Rescheduling_interrupts 106929 ± 13% -21.0% 84522 ± 5% interrupts.CPU59.TLB:TLB_shootdowns 329291 ± 3% +15.4% 380111 ± 2% interrupts.CPU59.TRM:Thermal_event_interrupts 89930 ± 9% -20.5% 71520 ± 7% interrupts.CPU6.CAL:Function_call_interrupts 1785220 +12.3% 2005468 interrupts.CPU6.LOC:Local_timer_interrupts 106307 ± 7% -19.9% 85162 ± 6% interrupts.CPU6.TLB:TLB_shootdowns 388421 ± 12% +27.8% 496564 ± 4% interrupts.CPU6.TRM:Thermal_event_interrupts 436.25 +13.1% 493.25 interrupts.CPU60.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60 98595 ± 11% -25.6% 73379 ± 5% interrupts.CPU60.CAL:Function_call_interrupts 1785116 +12.1% 2000324 interrupts.CPU60.LOC:Local_timer_interrupts 1604314 ± 8% +186.8% 4601614 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts 110442 ± 10% -24.0% 83932 ± 5% interrupts.CPU60.TLB:TLB_shootdowns 329256 ± 3% +15.5% 380140 ± 2% interrupts.CPU60.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU61.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61 93035 ± 11% -21.4% 73124 ± 5% interrupts.CPU61.CAL:Function_call_interrupts 1785176 +12.1% 2000960 interrupts.CPU61.LOC:Local_timer_interrupts 4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.NMI:Non-maskable_interrupts 4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.PMI:Performance_monitoring_interrupts 1575186 ± 6% +194.9% 4644760 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts 103665 ± 11% -18.7% 84259 ± 5% interrupts.CPU61.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380127 ± 2% interrupts.CPU61.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU62.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62 95188 ± 10% -21.9% 74376 ± 6% interrupts.CPU62.CAL:Function_call_interrupts 1785028 +12.0% 1999262 interrupts.CPU62.LOC:Local_timer_interrupts 1613834 ± 8% +184.7% 4594499 ± 3% interrupts.CPU62.RES:Rescheduling_interrupts 106856 ± 10% -21.4% 83987 ± 5% interrupts.CPU62.TLB:TLB_shootdowns 329270 ± 3% +15.4% 380120 ± 2% interrupts.CPU62.TRM:Thermal_event_interrupts 94411 ± 11% -21.3% 74269 ± 5% interrupts.CPU63.CAL:Function_call_interrupts 1785322 +12.0% 1999237 interrupts.CPU63.LOC:Local_timer_interrupts 1669218 ± 6% +187.2% 4794684 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts 105402 ± 10% -18.3% 86076 ± 8% interrupts.CPU63.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU63.TRM:Thermal_event_interrupts 96060 ± 11% -22.6% 74364 ± 6% interrupts.CPU64.CAL:Function_call_interrupts 1785304 +12.0% 1999571 interrupts.CPU64.LOC:Local_timer_interrupts 1681501 ± 7% +164.0% 4438647 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts 108534 ± 10% -22.8% 83813 ± 5% interrupts.CPU64.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU64.TRM:Thermal_event_interrupts 94178 ± 9% -22.7% 72758 ± 8% interrupts.CPU65.CAL:Function_call_interrupts 1785338 +12.0% 1999408 interrupts.CPU65.LOC:Local_timer_interrupts 1674108 ± 4% +182.2% 4724262 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts 107519 ± 10% -22.6% 83259 ± 5% interrupts.CPU65.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380128 ± 2% interrupts.CPU65.TRM:Thermal_event_interrupts 95135 ± 8% -22.4% 73795 ± 5% interrupts.CPU66.CAL:Function_call_interrupts 1785413 +12.0% 2000352 interrupts.CPU66.LOC:Local_timer_interrupts 616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.NMI:Non-maskable_interrupts 616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.PMI:Performance_monitoring_interrupts 1646166 ± 6% +169.3% 4433267 ± 6% interrupts.CPU66.RES:Rescheduling_interrupts 107190 ± 7% -22.5% 83034 ± 4% interrupts.CPU66.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380118 ± 2% interrupts.CPU66.TRM:Thermal_event_interrupts 95627 ± 10% -22.4% 74249 ± 5% interrupts.CPU67.CAL:Function_call_interrupts 1785314 +12.1% 2001128 interrupts.CPU67.LOC:Local_timer_interrupts 1611146 ± 4% +183.8% 4572204 ± 7% interrupts.CPU67.RES:Rescheduling_interrupts 109283 ± 11% -22.5% 84676 ± 4% interrupts.CPU67.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU67.TRM:Thermal_event_interrupts 95479 ± 11% -20.6% 75791 ± 5% interrupts.CPU68.CAL:Function_call_interrupts 1785192 +12.0% 1999411 interrupts.CPU68.LOC:Local_timer_interrupts 1575394 ± 7% +189.9% 4567277 ± 4% interrupts.CPU68.RES:Rescheduling_interrupts 106972 ± 12% -18.6% 87039 ± 5% interrupts.CPU68.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380129 ± 2% interrupts.CPU68.TRM:Thermal_event_interrupts 93067 ± 14% -19.9% 74538 ± 4% interrupts.CPU69.CAL:Function_call_interrupts 1785257 +12.1% 2000636 interrupts.CPU69.LOC:Local_timer_interrupts 1578663 ± 9% +192.9% 4624518 ± 3% interrupts.CPU69.RES:Rescheduling_interrupts 104271 ± 12% -18.5% 84989 ± 2% interrupts.CPU69.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380066 ± 2% interrupts.CPU69.TRM:Thermal_event_interrupts 474.00 ± 3% +49.7% 709.75 ± 24% interrupts.CPU7.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7 92062 ± 9% -19.7% 73884 ± 5% interrupts.CPU7.CAL:Function_call_interrupts 1784266 +12.4% 2004980 interrupts.CPU7.LOC:Local_timer_interrupts 2505703 ± 11% -19.0% 2029187 ± 5% interrupts.CPU7.RES:Rescheduling_interrupts 107698 ± 8% -18.4% 87903 ± 6% interrupts.CPU7.TLB:TLB_shootdowns 388281 ± 12% +27.8% 496347 ± 4% interrupts.CPU7.TRM:Thermal_event_interrupts 92337 ± 12% -17.5% 76148 ± 2% interrupts.CPU70.CAL:Function_call_interrupts 1785161 +12.0% 1999757 interrupts.CPU70.LOC:Local_timer_interrupts 1562360 ± 4% +170.8% 4231208 ± 9% interrupts.CPU70.RES:Rescheduling_interrupts 103862 ± 10% -18.8% 84371 ± 3% interrupts.CPU70.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU70.TRM:Thermal_event_interrupts 95829 ± 12% -21.2% 75509 ± 5% interrupts.CPU71.CAL:Function_call_interrupts 1784894 +12.1% 2001287 interrupts.CPU71.LOC:Local_timer_interrupts 1533567 ± 7% +196.9% 4553440 interrupts.CPU71.RES:Rescheduling_interrupts 105883 ± 11% -20.4% 84248 ± 6% interrupts.CPU71.TLB:TLB_shootdowns 329247 ± 3% +15.5% 380146 ± 2% interrupts.CPU71.TRM:Thermal_event_interrupts 88372 ± 11% -20.3% 70389 ± 7% interrupts.CPU8.CAL:Function_call_interrupts 1784452 +12.4% 2005305 interrupts.CPU8.LOC:Local_timer_interrupts 103992 ± 10% -17.4% 85887 ± 5% interrupts.CPU8.TLB:TLB_shootdowns 388422 ± 12% +27.8% 496557 ± 4% interrupts.CPU8.TRM:Thermal_event_interrupts 89290 ± 10% -19.1% 72221 ± 6% interrupts.CPU9.CAL:Function_call_interrupts 1784716 +12.4% 2005710 interrupts.CPU9.LOC:Local_timer_interrupts 104340 ± 10% -16.0% 87647 ± 3% interrupts.CPU9.TLB:TLB_shootdowns 388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU9.TRM:Thermal_event_interrupts 1.285e+08 +12.2% 1.442e+08 interrupts.LOC:Local_timer_interrupts 144.00 +50.0% 216.00 interrupts.MCP:Machine_check_polls 1.677e+08 ± 8% +50.4% 2.522e+08 ± 2% interrupts.RES:Rescheduling_interrupts 7624145 ± 10% -19.7% 6125835 ± 5% interrupts.TLB:TLB_shootdowns 25836833 ± 8% +22.2% 31559779 ± 2% interrupts.TRM:Thermal_event_interrupts *************************************************************************************************** lkp-bdw-ep6: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory ========================================================================================= compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/x86_64-rhel-7.6/process/50%/debian-x86_64-2019-11-14.cgz/lkp-bdw-ep6/hackbench/0xb000038 commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 134091 -10.6% 119937 hackbench.throughput 3.352e+09 ± 2% +16.1% 3.892e+09 ± 5% hackbench.time.involuntary_context_switches 535571 ± 2% +10.6% 592420 ± 4% numa-meminfo.node0.FilePages 12614 ± 18% -12.2% 11078 ± 18% numa-meminfo.node1.Mapped 3472 ± 6% +18.0% 4096 slabinfo.skbuff_head_cache.active_objs 3568 ± 6% +15.7% 4128 slabinfo.skbuff_head_cache.num_objs 730128 ± 5% -52.2% 349073 ± 25% turbostat.C3 14.77 +5.9% 15.64 turbostat.RAMWatt 24613783 ± 5% -32.6% 16595193 ± 23% numa-numastat.node0.local_node 24632941 ± 5% -32.6% 16605181 ± 23% numa-numastat.node0.numa_hit 25827961 -29.9% 18097150 ± 19% numa-numastat.node1.local_node 25836948 -29.9% 18115676 ± 19% numa-numastat.node1.numa_hit 50467170 ± 2% -31.2% 34733456 ± 10% proc-vmstat.numa_hit 50439016 ± 2% -31.2% 34704933 ± 10% proc-vmstat.numa_local 50622650 ± 2% -31.1% 34896854 ± 10% proc-vmstat.pgalloc_normal 50548381 ± 2% -31.1% 34834391 ± 10% proc-vmstat.pgfree 79.00 +2.2% 80.75 vmstat.cpu.sy 19.00 -7.9% 17.50 ± 2% vmstat.cpu.us 13450755 +4.6% 14066265 vmstat.system.cs 1122772 -12.1% 986681 vmstat.system.in 133891 ± 2% +10.6% 148112 ± 4% numa-vmstat.node0.nr_file_pages 12301995 ± 4% -30.2% 8590153 ± 22% numa-vmstat.node0.numa_hit 12282910 ± 4% -30.1% 8580139 ± 22% numa-vmstat.node0.numa_local 12964354 ± 5% -31.2% 8914408 ± 18% numa-vmstat.node1.numa_hit 12806275 ± 5% -31.7% 8746928 ± 19% numa-vmstat.node1.numa_local 33816695 ± 10% +17.8% 39824104 ± 7% cpuidle.C1.time 21015951 ± 3% +19.6% 25127666 ± 7% cpuidle.C1E.time 730991 ± 5% -52.1% 350128 ± 25% cpuidle.C3.usage 1.497e+08 ± 28% +41.6% 2.119e+08 ± 15% cpuidle.C6.time 6771631 ± 4% -45.2% 3713804 ± 15% cpuidle.POLL.time 5209842 ± 4% -44.1% 2913245 ± 15% cpuidle.POLL.usage 26965506 +5.5% 28444381 ± 4% sched_debug.cfs_rq:/.min_vruntime.avg 28257967 +6.6% 30126596 ± 4% sched_debug.cfs_rq:/.min_vruntime.max 582203 ± 9% +37.0% 797353 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev 4.65 ± 8% -10.7% 4.16 ± 6% sched_debug.cfs_rq:/.runnable_load_avg.stddev 4620 ± 6% -10.7% 4125 ± 4% sched_debug.cfs_rq:/.runnable_weight.stddev 582831 ± 9% +36.6% 796226 ± 7% sched_debug.cfs_rq:/.spread0.stddev 154.73 ± 2% -14.7% 132.00 ± 3% sched_debug.cfs_rq:/.util_avg.stddev 120.93 ± 25% +43.5% 173.50 ± 19% sched_debug.cfs_rq:/.util_est_enqueued.min 184.93 ± 2% -11.0% 164.54 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.stddev 182135 ± 18% +57.1% 286098 ± 6% sched_debug.cpu.avg_idle.avg 185926 ± 19% +49.0% 276991 ± 9% sched_debug.cpu.avg_idle.stddev 3.32 ± 4% -10.8% 2.96 ± 3% sched_debug.cpu.nr_running.stddev 46710201 +9.0% 50934866 ± 5% sched_debug.cpu.nr_switches.avg 50053735 +16.9% 58491253 ± 7% sched_debug.cpu.nr_switches.max 43751407 +6.7% 46698664 ± 4% sched_debug.cpu.nr_switches.min 1344975 ± 10% +78.9% 2406504 ± 16% sched_debug.cpu.nr_switches.stddev 1.63 ± 3% -32.8% 1.09 ± 11% sched_debug.cpu.nr_uninterruptible.avg 1017 ± 27% -59.5% 411.75 ± 16% interrupts.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6 6547829 -14.4% 5603910 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts 1017 ± 27% -59.5% 411.75 ± 16% interrupts.CPU19.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6 6801598 -14.7% 5801847 ± 5% interrupts.CPU19.RES:Rescheduling_interrupts 6609756 -13.7% 5703084 ± 8% interrupts.CPU20.RES:Rescheduling_interrupts 6446878 ± 2% -11.8% 5689145 ± 7% interrupts.CPU21.RES:Rescheduling_interrupts 6634825 -9.4% 6008203 ± 6% interrupts.CPU24.RES:Rescheduling_interrupts 6801182 -9.8% 6137528 ± 3% interrupts.CPU25.RES:Rescheduling_interrupts 6687252 -12.9% 5821967 ± 6% interrupts.CPU28.RES:Rescheduling_interrupts 6583387 -10.9% 5868659 ± 6% interrupts.CPU29.RES:Rescheduling_interrupts 6585631 -16.6% 5490726 ± 9% interrupts.CPU37.RES:Rescheduling_interrupts 6759973 -13.2% 5867800 ± 7% interrupts.CPU57.RES:Rescheduling_interrupts 6459616 ± 2% -10.5% 5783047 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts 6580818 ± 2% -16.9% 5468945 ± 3% interrupts.CPU60.RES:Rescheduling_interrupts 6723178 ± 2% -12.4% 5887778 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts 6480129 -16.2% 5427168 ± 7% interrupts.CPU64.RES:Rescheduling_interrupts 6439558 ± 2% -11.6% 5691160 ± 5% interrupts.CPU65.RES:Rescheduling_interrupts 6662822 -12.0% 5866390 ± 7% interrupts.CPU68.RES:Rescheduling_interrupts 6602178 -12.0% 5809760 ± 7% interrupts.CPU7.RES:Rescheduling_interrupts 6616172 -14.1% 5685889 ± 7% interrupts.CPU73.RES:Rescheduling_interrupts 6591477 -13.8% 5682655 ± 8% interrupts.CPU81.RES:Rescheduling_interrupts 13.97 +15.8% 16.18 perf-stat.i.MPKI 1.70 -0.1 1.65 perf-stat.i.branch-miss-rate% 3.607e+08 -1.6% 3.549e+08 perf-stat.i.branch-misses 1.22 ± 6% +0.8 2.01 ± 18% perf-stat.i.cache-miss-rate% 17650711 ± 6% +98.0% 34953649 ± 18% perf-stat.i.cache-misses 1.496e+09 +17.3% 1.755e+09 perf-stat.i.cache-references 13503062 +4.6% 14122231 perf-stat.i.context-switches 2946654 ± 2% +18.5% 3491269 ± 3% perf-stat.i.cpu-migrations 29365 ± 8% -30.6% 20387 ± 31% perf-stat.i.cycles-between-cache-misses 0.35 ± 2% +0.0 0.36 ± 2% perf-stat.i.dTLB-store-miss-rate% 2.048e+10 -2.1% 2.006e+10 perf-stat.i.dTLB-stores 55.17 -1.4 53.74 perf-stat.i.iTLB-load-miss-rate% 1.019e+08 +5.5% 1.074e+08 perf-stat.i.iTLB-loads 3068 -3.0% 2975 perf-stat.i.minor-faults 9195204 ± 6% +113.0% 19589023 ± 18% perf-stat.i.node-load-misses 28132 ± 6% +125.3% 63396 ± 13% perf-stat.i.node-loads 58.92 -9.8 49.16 ± 4% perf-stat.i.node-store-miss-rate% 3588196 ± 6% +60.5% 5760265 ± 19% perf-stat.i.node-store-misses 2415953 ± 5% +145.2% 5924042 ± 18% perf-stat.i.node-stores 3068 -3.0% 2975 perf-stat.i.page-faults 13.96 +16.0% 16.19 perf-stat.overall.MPKI 1.70 -0.1 1.65 perf-stat.overall.branch-miss-rate% 1.18 ± 6% +0.8 1.99 ± 18% perf-stat.overall.cache-miss-rate% 13823 ± 6% -47.8% 7221 ± 20% perf-stat.overall.cycles-between-cache-misses 0.35 ± 2% +0.0 0.36 ± 2% perf-stat.overall.dTLB-store-miss-rate% 55.11 -1.4 53.70 perf-stat.overall.iTLB-load-miss-rate% 59.75 -10.5 49.23 perf-stat.overall.node-store-miss-rate% 78192 +13.0% 88370 perf-stat.overall.path-length 3.601e+08 -1.6% 3.543e+08 perf-stat.ps.branch-misses 17622620 ± 6% +98.0% 34899005 ± 18% perf-stat.ps.cache-misses 1.494e+09 +17.3% 1.752e+09 perf-stat.ps.cache-references 13480657 +4.6% 14099389 perf-stat.ps.context-switches 2941749 ± 2% +18.5% 3485631 ± 3% perf-stat.ps.cpu-migrations 2.045e+10 -2.1% 2.003e+10 perf-stat.ps.dTLB-stores 1.017e+08 +5.5% 1.072e+08 perf-stat.ps.iTLB-loads 3064 -3.0% 2972 perf-stat.ps.minor-faults 9180406 ± 6% +113.0% 19558101 ± 18% perf-stat.ps.node-load-misses 28101 ± 6% +125.3% 63314 ± 13% perf-stat.ps.node-loads 3582467 ± 6% +60.5% 5751138 ± 19% perf-stat.ps.node-store-misses 2412086 ± 5% +145.2% 5914636 ± 18% perf-stat.ps.node-stores 3064 -3.0% 2972 perf-stat.ps.page-faults 223485 +9.1% 243918 ± 5% softirqs.CPU0.TIMER 11887 ± 11% -18.2% 9723 ± 6% softirqs.CPU1.SCHED 215411 +9.2% 235267 ± 6% softirqs.CPU1.TIMER 11293 ± 3% -14.3% 9678 ± 10% softirqs.CPU10.SCHED 213916 +9.8% 234908 ± 5% softirqs.CPU10.TIMER 212758 +7.7% 229141 ± 5% softirqs.CPU11.TIMER 11098 ± 2% -15.9% 9337 ± 10% softirqs.CPU12.SCHED 213397 +9.0% 232706 ± 6% softirqs.CPU12.TIMER 212419 +8.3% 230115 ± 6% softirqs.CPU13.TIMER 10777 ± 4% -12.5% 9426 ± 6% softirqs.CPU14.SCHED 11241 ± 3% -11.5% 9950 ± 6% softirqs.CPU15.SCHED 214604 +10.0% 236116 ± 8% softirqs.CPU15.TIMER 11147 -14.9% 9482 ± 8% softirqs.CPU16.SCHED 216343 +8.4% 234558 ± 6% softirqs.CPU16.TIMER 214680 +7.0% 229748 ± 6% softirqs.CPU17.TIMER 212791 +8.6% 230988 ± 6% softirqs.CPU19.TIMER 11211 ± 4% -10.4% 10047 ± 6% softirqs.CPU2.SCHED 219918 +9.8% 241513 ± 6% softirqs.CPU2.TIMER 10825 ± 4% -14.3% 9273 ± 7% softirqs.CPU20.SCHED 215209 +8.8% 234149 ± 6% softirqs.CPU20.TIMER 10869 ± 4% -11.4% 9626 ± 4% softirqs.CPU21.SCHED 216225 +8.7% 235105 ± 5% softirqs.CPU21.TIMER 13013 ± 7% -14.0% 11192 ± 3% softirqs.CPU22.SCHED 220406 +8.9% 239975 ± 4% softirqs.CPU22.TIMER 11925 -14.9% 10145 ± 3% softirqs.CPU23.SCHED 220231 +7.6% 237056 ± 5% softirqs.CPU23.TIMER 11140 ± 2% -11.5% 9861 ± 7% softirqs.CPU24.SCHED 216091 +8.9% 235338 ± 5% softirqs.CPU24.TIMER 11438 ± 3% -16.3% 9578 ± 2% softirqs.CPU25.SCHED 214880 +9.4% 235143 ± 5% softirqs.CPU25.TIMER 214888 +8.8% 233874 ± 5% softirqs.CPU26.TIMER 11410 ± 2% -13.7% 9844 ± 10% softirqs.CPU27.SCHED 215012 +8.7% 233790 ± 4% softirqs.CPU27.TIMER 220618 ± 2% +7.8% 237788 ± 5% softirqs.CPU28.TIMER 11096 ± 5% -14.3% 9513 ± 6% softirqs.CPU29.SCHED 217194 +8.2% 235108 ± 5% softirqs.CPU29.TIMER 11445 ± 2% -16.7% 9536 ± 10% softirqs.CPU3.SCHED 214692 +9.1% 234221 ± 5% softirqs.CPU3.TIMER 11127 ± 5% -10.5% 9954 ± 6% softirqs.CPU30.SCHED 214861 +8.2% 232532 ± 5% softirqs.CPU30.TIMER 217582 +7.8% 234493 ± 4% softirqs.CPU31.TIMER 214606 +8.8% 233473 ± 4% softirqs.CPU32.TIMER 11299 ± 2% -11.7% 9975 ± 6% softirqs.CPU33.SCHED 11503 ± 2% -16.0% 9658 ± 10% softirqs.CPU34.SCHED 213233 +10.9% 236580 ± 6% softirqs.CPU34.TIMER 11162 ± 2% -11.6% 9863 ± 7% softirqs.CPU35.SCHED 11241 -13.9% 9678 ± 6% softirqs.CPU36.SCHED 215681 +9.2% 235446 ± 4% softirqs.CPU36.TIMER 11358 -15.2% 9632 ± 6% softirqs.CPU37.SCHED 219691 ± 2% +18.5% 260407 ± 13% softirqs.CPU37.TIMER 11570 ± 2% -13.1% 10049 ± 6% softirqs.CPU38.SCHED 215559 +8.2% 233189 ± 4% softirqs.CPU38.TIMER 11295 ± 4% -13.1% 9819 ± 10% softirqs.CPU39.SCHED 215781 ± 2% +7.6% 232108 ± 5% softirqs.CPU39.TIMER 11506 ± 7% -13.0% 10006 ± 6% softirqs.CPU4.SCHED 214973 +8.9% 234140 ± 6% softirqs.CPU4.TIMER 11304 ± 2% -13.7% 9759 ± 10% softirqs.CPU40.SCHED 213827 +10.1% 235341 ± 4% softirqs.CPU40.TIMER 11462 ± 4% -13.5% 9913 ± 8% softirqs.CPU41.SCHED 215879 ± 2% +7.2% 231499 ± 5% softirqs.CPU41.TIMER 11418 ± 3% -14.0% 9819 ± 8% softirqs.CPU43.SCHED 214864 +8.3% 232698 ± 4% softirqs.CPU43.TIMER 10706 ± 5% -13.9% 9223 ± 11% softirqs.CPU44.SCHED 217386 +9.4% 237916 ± 5% softirqs.CPU44.TIMER 215191 +9.7% 236017 ± 5% softirqs.CPU45.TIMER 214214 +9.9% 235466 ± 6% softirqs.CPU46.TIMER 214471 +9.9% 235675 ± 5% softirqs.CPU47.TIMER 11251 ± 2% -13.0% 9785 ± 8% softirqs.CPU48.SCHED 212500 +9.9% 233451 ± 6% softirqs.CPU48.TIMER 213609 +9.9% 234657 ± 5% softirqs.CPU49.TIMER 216836 +8.5% 235225 ± 5% softirqs.CPU5.TIMER 215854 +9.0% 235383 ± 5% softirqs.CPU50.TIMER 11468 -14.4% 9820 ± 9% softirqs.CPU51.SCHED 215402 +8.8% 234360 ± 5% softirqs.CPU51.TIMER 214152 +8.8% 233038 ± 5% softirqs.CPU52.TIMER 212843 +8.4% 230716 ± 5% softirqs.CPU53.TIMER 11373 ± 2% -14.9% 9679 ± 7% softirqs.CPU54.SCHED 214008 +9.5% 234380 ± 5% softirqs.CPU54.TIMER 212619 +7.5% 228539 ± 5% softirqs.CPU55.TIMER 11136 ± 3% -16.4% 9312 ± 10% softirqs.CPU56.SCHED 213404 +9.0% 232542 ± 6% softirqs.CPU56.TIMER 11171 ± 4% -11.9% 9837 ± 7% softirqs.CPU57.SCHED 212283 +8.6% 230474 ± 6% softirqs.CPU57.TIMER 11137 ± 4% -14.0% 9578 ± 6% softirqs.CPU58.SCHED 11226 ± 4% -17.2% 9289 ± 4% softirqs.CPU59.SCHED 214193 +23.1% 263668 ± 22% softirqs.CPU59.TIMER 217195 +8.2% 234975 ± 5% softirqs.CPU6.TIMER 11051 ± 4% -14.4% 9460 ± 9% softirqs.CPU60.SCHED 217131 +7.9% 234363 ± 5% softirqs.CPU60.TIMER 11206 ± 3% -12.6% 9793 ± 9% softirqs.CPU61.SCHED 211791 +9.2% 231302 ± 5% softirqs.CPU61.TIMER 214956 ± 2% +8.8% 233770 ± 5% softirqs.CPU62.TIMER 11180 ± 3% -14.8% 9526 ± 8% softirqs.CPU63.SCHED 213413 +8.7% 232008 ± 5% softirqs.CPU63.TIMER 215674 +8.1% 233086 ± 6% softirqs.CPU64.TIMER 11032 ± 3% -15.7% 9302 ± 7% softirqs.CPU65.SCHED 215870 +9.0% 235386 ± 5% softirqs.CPU65.TIMER 11403 ± 3% -14.2% 9786 ± 6% softirqs.CPU66.SCHED 215968 +9.4% 236175 ± 5% softirqs.CPU66.TIMER 11098 ± 3% -7.4% 10281 ± 4% softirqs.CPU67.SCHED 217846 +8.3% 236023 ± 4% softirqs.CPU67.TIMER 11094 -12.2% 9740 ± 7% softirqs.CPU68.SCHED 215512 +9.4% 235796 ± 5% softirqs.CPU68.TIMER 11321 ± 3% -14.1% 9724 ± 3% softirqs.CPU69.SCHED 213564 +9.0% 232867 ± 5% softirqs.CPU69.TIMER 218822 +8.8% 238080 ± 6% softirqs.CPU7.TIMER 11507 ± 2% -12.9% 10026 ± 4% softirqs.CPU70.SCHED 213928 +9.4% 233986 ± 4% softirqs.CPU70.TIMER 213528 +8.5% 231730 ± 4% softirqs.CPU71.TIMER 11262 ± 2% -11.1% 10013 ± 5% softirqs.CPU72.SCHED 11145 ± 3% -14.5% 9533 ± 8% softirqs.CPU73.SCHED 216467 +9.2% 236409 ± 5% softirqs.CPU73.TIMER 11447 ± 4% -13.2% 9935 ± 4% softirqs.CPU74.SCHED 212578 +9.1% 232022 ± 5% softirqs.CPU74.TIMER 215256 +7.8% 232072 ± 5% softirqs.CPU75.TIMER 11324 ± 4% -11.8% 9985 ± 3% softirqs.CPU76.SCHED 214170 +8.6% 232516 ± 5% softirqs.CPU76.TIMER 11646 ± 5% -15.7% 9824 ± 7% softirqs.CPU77.SCHED 213706 +8.0% 230880 ± 5% softirqs.CPU77.TIMER 11397 ± 6% -13.9% 9810 ± 10% softirqs.CPU78.SCHED 11179 ± 3% -11.5% 9894 ± 5% softirqs.CPU79.SCHED 213215 +8.4% 231081 ± 5% softirqs.CPU79.TIMER 11165 ± 4% -16.2% 9352 ± 9% softirqs.CPU8.SCHED 216508 ± 2% +7.9% 233692 ± 6% softirqs.CPU8.TIMER 11203 ± 2% -13.9% 9643 ± 4% softirqs.CPU80.SCHED 216446 +8.3% 234469 ± 5% softirqs.CPU80.TIMER 11151 ± 2% -12.8% 9726 ± 6% softirqs.CPU81.SCHED 11373 ± 3% -13.0% 9891 ± 6% softirqs.CPU82.SCHED 213597 +8.9% 232608 ± 5% softirqs.CPU82.TIMER 214308 +7.9% 231148 ± 5% softirqs.CPU83.TIMER 214384 +20.9% 259149 ± 15% softirqs.CPU84.TIMER 11458 -13.7% 9887 ± 7% softirqs.CPU86.SCHED 215459 +8.2% 233058 ± 5% softirqs.CPU86.TIMER 11303 ± 4% -12.4% 9904 ± 8% softirqs.CPU87.SCHED 214185 +7.9% 231116 ± 5% softirqs.CPU87.TIMER 11030 ± 2% -13.0% 9593 ± 8% softirqs.CPU9.SCHED 213229 +8.7% 231788 ± 6% softirqs.CPU9.TIMER 995635 -12.9% 867688 ± 5% softirqs.SCHED 19052052 +8.6% 20690944 ± 5% softirqs.TIMER 31.32 -1.6 29.74 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.24 -1.5 28.76 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 26.06 -0.9 25.17 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.35 ± 5% -0.9 1.48 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 25.36 -0.8 24.54 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 2.75 ± 4% -0.7 2.08 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 4.24 ± 2% -0.6 3.60 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64 3.84 ± 2% -0.6 3.23 ± 2% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 7.83 -0.5 7.30 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 1.68 ± 6% -0.5 1.17 ± 10% perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 8.02 -0.5 7.53 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 8.08 -0.5 7.60 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.63 ± 4% -0.4 1.19 ± 6% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write 1.71 ± 6% -0.4 1.28 ± 8% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait 2.60 -0.4 2.19 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.03 ± 2% -0.4 1.64 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.47 -0.4 3.08 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 1.95 ± 2% -0.3 1.61 ± 3% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 2.67 -0.3 2.35 ± 3% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 0.56 -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.22 ± 4% -0.3 0.94 ± 2% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 1.25 ± 2% -0.3 1.00 ± 5% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64 2.64 -0.2 2.41 ± 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 1.53 -0.2 1.34 ± 7% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.59 ± 4% -0.2 0.40 ± 57% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.34 -0.2 1.16 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.07 ± 2% -0.2 0.90 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 1.05 -0.2 0.88 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule 1.56 ± 2% -0.2 1.38 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read 1.17 ± 4% -0.2 1.01 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 0.98 ± 2% -0.2 0.82 ± 4% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.66 ± 2% -0.2 1.50 ± 5% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 0.73 ± 3% -0.2 0.57 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.96 ± 2% -0.1 0.82 ± 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write 0.89 ± 2% -0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.66 ± 6% -0.1 0.54 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64 2.42 -0.1 2.32 perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.pipe_wait.pipe_read 0.89 ± 4% -0.1 0.79 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write 0.74 ± 2% -0.1 0.66 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.84 -0.1 0.79 ± 3% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.56 -0.0 0.54 ± 2% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.77 ± 2% +0.1 0.82 perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function 0.90 ± 3% +0.1 0.97 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr 0.69 ± 3% +0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.80 ± 4% +0.1 0.89 ± 2% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 1.10 +0.1 1.20 perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.73 ± 3% +0.1 0.83 ± 2% perf-profile.calltrace.cycles-pp.__switch_to 1.05 +0.1 1.16 perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 0.63 ± 4% +0.1 0.75 ± 2% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 0.99 ± 5% +0.1 1.12 ± 5% perf-profile.calltrace.cycles-pp.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.07 ± 3% +0.1 1.21 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm 0.64 ± 6% +0.2 0.80 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read 1.41 ± 3% +0.2 1.59 ± 2% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.77 ± 5% +0.2 1.00 ± 5% perf-profile.calltrace.cycles-pp.load_new_mm_cr3.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop 1.73 ± 3% +0.3 2.06 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.14 ±173% +0.5 0.63 ± 5% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read 0.13 ±173% +0.5 0.64 ± 5% perf-profile.calltrace.cycles-pp.migrate_task_rq_fair.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common 1.85 ± 5% +0.5 2.38 ± 3% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.00 +0.6 0.59 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read 0.00 +0.6 0.60 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +0.7 0.67 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.15 ±173% +1.0 1.14 ± 12% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 87.66 +1.1 88.78 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 86.87 +1.1 88.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.59 ± 4% +1.4 6.96 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.72 ± 4% +1.4 7.12 ± 3% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.87 ± 4% +1.4 7.28 ± 3% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.70 ± 30% +3.3 4.98 ± 13% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 35.54 +3.3 38.87 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 34.28 +3.5 37.82 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 31.12 +4.1 35.22 ± 2% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.49 +4.1 34.63 ± 2% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 22.54 ± 2% +6.0 28.53 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 20.58 ± 2% +6.2 26.76 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 19.97 ± 2% +6.2 26.16 ± 3% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 19.73 ± 3% +6.2 25.92 ± 3% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 3.39 ± 20% +7.0 10.39 ± 12% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 4.89 ± 15% +7.1 11.98 ± 10% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 31.34 -1.6 29.75 perf-profile.children.cycles-pp.ksys_read 30.28 -1.5 28.79 perf-profile.children.cycles-pp.vfs_read 4.14 ± 5% -1.3 2.85 ± 7% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 3.51 ± 6% -1.0 2.54 ± 8% perf-profile.children.cycles-pp.update_cfs_group 26.10 -0.9 25.20 perf-profile.children.cycles-pp.new_sync_read 25.42 -0.8 24.59 perf-profile.children.cycles-pp.pipe_read 4.67 -0.8 3.86 ± 4% perf-profile.children.cycles-pp.security_file_permission 4.33 ± 2% -0.7 3.65 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret 4.24 ± 2% -0.6 3.60 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64 2.18 ± 4% -0.5 1.64 ± 6% perf-profile.children.cycles-pp.mutex_unlock 7.92 -0.5 7.40 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair 8.10 -0.5 7.61 ± 3% perf-profile.children.cycles-pp.activate_task 8.16 -0.5 7.67 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate 4.12 ± 4% -0.5 3.64 ± 5% perf-profile.children.cycles-pp._raw_spin_lock 2.83 -0.4 2.38 ± 6% perf-profile.children.cycles-pp.selinux_file_permission 3.53 -0.4 3.13 ± 4% perf-profile.children.cycles-pp.enqueue_entity 1.99 ± 2% -0.4 1.64 ± 3% perf-profile.children.cycles-pp.copy_page_from_iter 3.34 -0.4 2.99 ± 4% perf-profile.children.cycles-pp.update_load_avg 2.70 -0.3 2.39 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter 2.66 -0.3 2.35 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.93 ± 3% -0.3 0.64 ± 7% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 1.41 ± 4% -0.3 1.13 perf-profile.children.cycles-pp.file_has_perm 2.75 -0.3 2.49 ± 2% perf-profile.children.cycles-pp.dequeue_entity 2.29 ± 2% -0.3 2.04 perf-profile.children.cycles-pp.mutex_lock 0.85 ± 3% -0.2 0.61 ± 4% perf-profile.children.cycles-pp.__inode_security_revalidate 0.59 ± 3% -0.2 0.39 ± 6% perf-profile.children.cycles-pp.__mutex_lock 1.79 ± 2% -0.2 1.59 ± 4% perf-profile.children.cycles-pp.__fdget_pos 1.70 ± 2% -0.2 1.52 ± 3% perf-profile.children.cycles-pp.__fget_light 0.93 ± 3% -0.2 0.75 perf-profile.children.cycles-pp.avc_has_perm 1.09 ± 2% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.copyin 2.33 ± 2% -0.2 2.16 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.96 ± 2% -0.2 0.81 ± 2% perf-profile.children.cycles-pp.___might_sleep 1.67 ± 2% -0.2 1.52 ± 5% perf-profile.children.cycles-pp.copyout 0.83 ± 4% -0.1 0.68 ± 2% perf-profile.children.cycles-pp._cond_resched 0.33 ± 5% -0.1 0.19 ± 10% perf-profile.children.cycles-pp.preempt_schedule_common 0.42 ± 2% -0.1 0.28 ± 8% perf-profile.children.cycles-pp.wake_up_q 0.70 ± 3% -0.1 0.59 ± 3% perf-profile.children.cycles-pp.__might_sleep 0.65 ± 5% -0.1 0.54 ± 7% perf-profile.children.cycles-pp.__fsnotify_parent 0.52 ± 2% -0.1 0.43 perf-profile.children.cycles-pp.__might_fault 0.53 ± 5% -0.1 0.45 perf-profile.children.cycles-pp.current_time 0.84 ± 3% -0.1 0.77 perf-profile.children.cycles-pp.fsnotify 0.16 ± 7% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.15 ± 8% -0.1 0.08 ± 15% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.22 ± 3% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.wake_q_add 0.49 -0.1 0.44 perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.26 ± 4% -0.1 0.21 ± 2% perf-profile.children.cycles-pp.rcu_all_qs 0.08 ± 14% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_read 0.25 ± 4% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.finish_wait 0.38 -0.0 0.34 ± 2% perf-profile.children.cycles-pp.__x64_sys_read 0.43 ± 3% -0.0 0.39 ± 2% perf-profile.children.cycles-pp.prepare_to_wait 0.14 ± 20% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.task_tick_fair 0.16 ± 19% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.scheduler_tick 0.10 ± 8% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.mutex_spin_on_owner 0.18 ± 8% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.inode_has_perm 0.21 ± 2% -0.0 0.18 ± 6% perf-profile.children.cycles-pp.deactivate_task 0.20 ± 2% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.generic_pipe_buf_confirm 0.13 ± 3% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__sb_end_write 0.14 ± 9% -0.0 0.12 perf-profile.children.cycles-pp.timespec64_trunc 0.14 ± 3% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.native_irq_return_iret 0.11 ± 4% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.smp_reschedule_interrupt 0.17 ± 4% -0.0 0.15 perf-profile.children.cycles-pp.iov_iter_init 0.11 ± 4% -0.0 0.09 ± 10% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.09 ± 4% -0.0 0.07 perf-profile.children.cycles-pp.__x2apic_send_IPI_dest 0.15 -0.0 0.13 ± 3% perf-profile.children.cycles-pp.rb_insert_color 0.18 ± 2% -0.0 0.17 ± 2% perf-profile.children.cycles-pp.set_next_buddy 0.08 ± 5% -0.0 0.07 perf-profile.children.cycles-pp.interrupt_entry 0.23 ± 2% +0.0 0.24 perf-profile.children.cycles-pp.cpumask_next 0.67 +0.0 0.69 perf-profile.children.cycles-pp.__calc_delta 0.15 ± 3% +0.0 0.18 ± 6% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.17 ± 6% +0.1 0.23 ± 3% perf-profile.children.cycles-pp.remove_entity_load_avg 0.14 ± 6% +0.1 0.19 ± 7% perf-profile.children.cycles-pp.attach_entity_load_avg 0.80 ± 3% +0.1 0.86 perf-profile.children.cycles-pp.check_preempt_wakeup 0.37 ± 3% +0.1 0.45 ± 4% perf-profile.children.cycles-pp.account_entity_enqueue 0.79 ± 4% +0.1 0.88 ± 5% perf-profile.children.cycles-pp.finish_task_switch 0.80 ± 2% +0.1 0.89 ± 2% perf-profile.children.cycles-pp.put_prev_entity 0.54 ± 2% +0.1 0.65 perf-profile.children.cycles-pp.account_entity_dequeue 1.01 ± 5% +0.1 1.13 ± 5% perf-profile.children.cycles-pp.set_task_cpu 0.49 ± 7% +0.2 0.65 ± 5% perf-profile.children.cycles-pp.migrate_task_rq_fair 1.23 ± 4% +0.2 1.40 ± 2% perf-profile.children.cycles-pp.__switch_to_asm 1.15 ± 2% +0.2 1.33 ± 2% perf-profile.children.cycles-pp.__switch_to 1.42 ± 3% +0.2 1.59 ± 2% perf-profile.children.cycles-pp.switch_fpu_return 0.76 ± 3% +0.2 0.96 ± 3% perf-profile.children.cycles-pp.pick_next_entity 2.21 +0.2 2.42 perf-profile.children.cycles-pp.reweight_entity 2.20 ± 2% +0.2 2.43 perf-profile.children.cycles-pp.load_new_mm_cr3 1.24 ± 5% +0.2 1.48 ± 4% perf-profile.children.cycles-pp.update_rq_clock 1.43 ± 5% +0.3 1.73 ± 2% perf-profile.children.cycles-pp.___perf_sw_event 0.38 ± 15% +0.3 0.68 ± 9% perf-profile.children.cycles-pp.find_next_bit 3.96 +0.3 4.28 perf-profile.children.cycles-pp.pick_next_task_fair 4.43 ± 2% +0.4 4.82 perf-profile.children.cycles-pp.switch_mm_irqs_off 1.11 +0.7 1.83 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.46 ± 26% +0.7 1.20 ± 12% perf-profile.children.cycles-pp.cpumask_next_wrap 87.76 +1.1 88.86 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 86.98 +1.1 88.10 perf-profile.children.cycles-pp.do_syscall_64 21.71 +1.3 23.02 perf-profile.children.cycles-pp.__schedule 5.98 ± 4% +1.4 7.34 ± 3% perf-profile.children.cycles-pp.exit_to_usermode_loop 21.59 +1.4 22.99 perf-profile.children.cycles-pp.schedule 1.73 ± 29% +3.3 5.03 ± 13% perf-profile.children.cycles-pp.available_idle_cpu 35.55 +3.3 38.88 perf-profile.children.cycles-pp.ksys_write 34.31 +3.5 37.84 perf-profile.children.cycles-pp.vfs_write 31.14 +4.1 35.23 ± 2% perf-profile.children.cycles-pp.new_sync_write 30.56 +4.2 34.74 ± 2% perf-profile.children.cycles-pp.pipe_write 22.96 ± 2% +6.0 28.93 ± 3% perf-profile.children.cycles-pp.__wake_up_common_lock 20.11 ± 2% +6.1 26.19 ± 3% perf-profile.children.cycles-pp.try_to_wake_up 20.66 ± 2% +6.2 26.82 ± 3% perf-profile.children.cycles-pp.__wake_up_common 20.01 ± 2% +6.2 26.19 ± 3% perf-profile.children.cycles-pp.autoremove_wake_function 3.46 ± 20% +7.0 10.48 ± 12% perf-profile.children.cycles-pp.select_idle_sibling 4.92 ± 15% +7.1 12.01 ± 10% perf-profile.children.cycles-pp.select_task_rq_fair 11.85 ± 3% -2.1 9.72 ± 2% perf-profile.self.cycles-pp.do_syscall_64 4.13 ± 5% -1.3 2.85 ± 8% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 3.49 ± 6% -1.0 2.52 ± 9% perf-profile.self.cycles-pp.update_cfs_group 4.33 ± 2% -0.7 3.64 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret 4.24 ± 2% -0.6 3.60 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64 2.12 ± 4% -0.5 1.59 ± 6% perf-profile.self.cycles-pp.mutex_unlock 2.61 -0.3 2.31 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.60 ± 2% -0.3 1.30 ± 5% perf-profile.self.cycles-pp.update_load_avg 1.48 ± 3% -0.2 1.25 ± 2% perf-profile.self.cycles-pp.mutex_lock 1.95 -0.2 1.75 ± 7% perf-profile.self.cycles-pp.selinux_file_permission 1.06 ± 5% -0.2 0.86 ± 5% perf-profile.self.cycles-pp.pipe_write 1.46 ± 2% -0.2 1.28 ± 3% perf-profile.self.cycles-pp.pipe_read 1.65 ± 2% -0.2 1.49 ± 4% perf-profile.self.cycles-pp.__fget_light 0.91 ± 3% -0.2 0.74 perf-profile.self.cycles-pp.avc_has_perm 0.94 ± 2% -0.2 0.79 ± 2% perf-profile.self.cycles-pp.___might_sleep 1.84 -0.1 1.72 ± 2% perf-profile.self.cycles-pp.update_curr 1.05 -0.1 0.93 perf-profile.self.cycles-pp.enqueue_task_fair 0.63 ± 3% -0.1 0.52 ± 2% perf-profile.self.cycles-pp.__might_sleep 0.60 ± 6% -0.1 0.51 ± 7% perf-profile.self.cycles-pp.__fsnotify_parent 0.29 ± 3% -0.1 0.19 ± 5% perf-profile.self.cycles-pp.__mutex_lock 0.38 ± 4% -0.1 0.29 ± 4% perf-profile.self.cycles-pp.vfs_write 0.51 -0.1 0.43 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter 0.49 -0.1 0.41 ± 2% perf-profile.self.cycles-pp.new_sync_write 0.35 ± 4% -0.1 0.28 ± 2% perf-profile.self.cycles-pp.copy_page_from_iter 0.81 ± 3% -0.1 0.75 perf-profile.self.cycles-pp.fsnotify 0.26 -0.1 0.19 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate 0.20 ± 4% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.__mutex_unlock_slowpath 0.29 ± 5% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.file_has_perm 0.22 -0.1 0.16 ± 8% perf-profile.self.cycles-pp.wake_q_add 0.57 ± 2% -0.1 0.52 perf-profile.self.cycles-pp.new_sync_read 0.34 ± 2% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.dequeue_entity 0.26 ± 3% -0.1 0.21 ± 3% perf-profile.self.cycles-pp.ksys_read 0.28 ± 2% -0.0 0.23 perf-profile.self.cycles-pp.security_file_permission 0.22 ± 3% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.finish_wait 0.25 -0.0 0.21 ± 4% perf-profile.self.cycles-pp.ksys_write 0.88 -0.0 0.83 perf-profile.self.cycles-pp.__lock_text_start 0.66 -0.0 0.61 perf-profile.self.cycles-pp.vfs_read 1.37 -0.0 1.32 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.29 ± 4% -0.0 0.25 perf-profile.self.cycles-pp.current_time 0.20 ± 3% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.rcu_all_qs 0.10 ± 8% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.42 -0.0 0.39 ± 2% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.34 -0.0 0.31 ± 2% perf-profile.self.cycles-pp.__x64_sys_read 0.09 ± 5% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.wake_up_q 0.15 ± 8% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.inode_has_perm 0.14 ± 5% -0.0 0.11 ± 9% perf-profile.self.cycles-pp.rb_next 0.20 ± 2% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.deactivate_task 0.11 ± 4% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.__fdget_pos 0.24 ± 2% -0.0 0.22 ± 3% perf-profile.self.cycles-pp.prepare_to_wait 0.14 ± 3% -0.0 0.11 ± 3% perf-profile.self.cycles-pp.native_irq_return_iret 0.13 ± 7% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.timespec64_trunc 0.06 -0.0 0.04 ± 57% perf-profile.self.cycles-pp.copyout 0.11 ± 4% -0.0 0.09 ± 10% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.17 ± 4% -0.0 0.15 ± 2% perf-profile.self.cycles-pp.generic_pipe_buf_confirm 0.18 ± 2% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.__might_fault 0.17 ± 2% -0.0 0.16 ± 2% perf-profile.self.cycles-pp.set_next_buddy 0.15 ± 2% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.rb_insert_color 0.12 ± 3% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.__sb_end_write 0.10 ± 8% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.06 -0.0 0.05 perf-profile.self.cycles-pp.kill_fasync 0.14 ± 3% +0.0 0.16 ± 5% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.66 +0.0 0.68 perf-profile.self.cycles-pp.__calc_delta 0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.native_load_tls 0.14 ± 3% +0.0 0.17 ± 4% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.67 +0.0 0.70 perf-profile.self.cycles-pp.dequeue_task_fair 0.23 ± 3% +0.0 0.27 ± 3% perf-profile.self.cycles-pp._cond_resched 0.17 ± 4% +0.0 0.21 ± 6% perf-profile.self.cycles-pp.activate_task 0.35 ± 3% +0.0 0.40 perf-profile.self.cycles-pp.pick_next_entity 1.20 +0.1 1.25 perf-profile.self.cycles-pp.select_task_rq_fair 0.13 ± 6% +0.1 0.18 ± 6% perf-profile.self.cycles-pp.attach_entity_load_avg 0.87 ± 3% +0.1 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair 0.27 ± 6% +0.1 0.36 ± 7% perf-profile.self.cycles-pp.migrate_task_rq_fair 0.32 ± 2% +0.1 0.41 ± 4% perf-profile.self.cycles-pp.account_entity_enqueue 0.42 +0.1 0.56 ± 2% perf-profile.self.cycles-pp.account_entity_dequeue 2.24 ± 3% +0.1 2.39 ± 2% perf-profile.self.cycles-pp.switch_mm_irqs_off 1.09 ± 2% +0.2 1.26 ± 2% perf-profile.self.cycles-pp.__switch_to 1.20 ± 4% +0.2 1.37 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 1.40 ± 3% +0.2 1.58 ± 2% perf-profile.self.cycles-pp.switch_fpu_return 0.89 ± 6% +0.2 1.10 ± 5% perf-profile.self.cycles-pp.update_rq_clock 2.19 ± 2% +0.2 2.42 perf-profile.self.cycles-pp.load_new_mm_cr3 1.33 ± 4% +0.3 1.60 ± 3% perf-profile.self.cycles-pp.___perf_sw_event 0.34 ± 14% +0.3 0.66 ± 9% perf-profile.self.cycles-pp.find_next_bit 0.28 ± 23% +0.4 0.72 ± 12% perf-profile.self.cycles-pp.cpumask_next_wrap 0.89 +0.7 1.58 ± 4% perf-profile.self.cycles-pp._raw_spin_lock 1.04 +0.7 1.76 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.65 ± 12% +3.0 3.61 ± 13% perf-profile.self.cycles-pp.select_idle_sibling 1.71 ± 29% +3.3 4.97 ± 13% perf-profile.self.cycles-pp.available_idle_cpu *************************************************************************************************** lkp-cfl-e1: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory ========================================================================================= compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-2018-04-03.cgz/lkp-cfl-e1/hackbench/0xb8 commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:2 -50% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x 1:2 -50% :4 dmesg.WARNING:stack_recursion %stddev %change %stddev \ | \ 49845 -6.8% 46466 hackbench.throughput 8.776e+08 +13.0% 9.915e+08 hackbench.time.involuntary_context_switches 93623 -6.3% 87707 hackbench.time.minor_page_faults 740.08 -1.4% 729.54 hackbench.time.user_time 1.569e+09 +3.5% 1.623e+09 hackbench.time.voluntary_context_switches 3.072e+08 -6.2% 2.88e+08 hackbench.workload 21.45 -1.7% 21.09 boot-time.boot 1747212 ± 3% -31.0% 1205072 ± 14% cpuidle.POLL.time 1289000 ± 4% -36.6% 817407 ± 13% cpuidle.POLL.usage 4038486 +6.3% 4293331 vmstat.system.cs 394711 -10.2% 354353 ± 2% vmstat.system.in 10059 -15.4% 8506 ± 5% softirqs.CPU0.SCHED 309881 -27.3% 225159 ± 4% softirqs.CPU2.TIMER 220105 +20.8% 265807 ± 16% softirqs.CPU6.TIMER 120937 ± 3% -17.6% 99667 ± 2% softirqs.SCHED 18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.active_objs 18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.num_objs 8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.active_objs 8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.num_objs 5280 -8.8% 4814 slabinfo.task_delay_info.active_objs 5280 -8.8% 4814 slabinfo.task_delay_info.num_objs 65295 -1.9% 64034 proc-vmstat.nr_active_anon 60605 -1.5% 59671 proc-vmstat.nr_anon_pages 12935 -0.9% 12820 proc-vmstat.nr_slab_unreclaimable 65295 -1.9% 64034 proc-vmstat.nr_zone_active_anon 24774010 -16.6% 20673098 proc-vmstat.numa_hit 24774010 -16.6% 20673098 proc-vmstat.numa_local 24839243 -16.5% 20735391 proc-vmstat.pgalloc_normal 673024 -2.4% 657142 proc-vmstat.pgfault 24822112 -16.5% 20717243 proc-vmstat.pgfree 247.95 ± 14% -19.1% 200.59 ± 4% sched_debug.cfs_rq:/.load_avg.max 43.55 ± 3% -19.9% 34.86 ± 6% sched_debug.cfs_rq:/.load_avg.min 55.51 ± 6% -13.2% 48.19 ± 7% sched_debug.cfs_rq:/.load_avg.stddev 20116 ± 8% -12.8% 17531 ± 5% sched_debug.cfs_rq:/.runnable_weight.min 1288 +7.5% 1384 ± 3% sched_debug.cfs_rq:/.util_avg.max 54763 ± 9% +130.8% 126390 ± 25% sched_debug.cpu.avg_idle.avg 301502 ± 18% +84.9% 557406 ± 14% sched_debug.cpu.avg_idle.max 82233 ± 22% +122.7% 183153 ± 17% sched_debug.cpu.avg_idle.stddev 0.00 ± 12% -12.0% 0.00 ± 8% sched_debug.cpu.next_balance.stddev 16.92 ± 4% -7.5% 15.66 ± 3% sched_debug.cpu.nr_running.avg 2.92 ± 10% -20.9% 2.31 ± 16% sched_debug.cpu.nr_uninterruptible.avg 2977 ± 50% -73.0% 803.75 ± 43% interrupts.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1 546.00 ± 23% +119.5% 1198 ± 16% interrupts.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3 13831842 -11.5% 12245495 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts 45259 ± 5% +68.9% 76440 ± 15% interrupts.CPU0.TRM:Thermal_event_interrupts 2977 ± 50% -73.0% 803.75 ± 43% interrupts.CPU1.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1 45283 ± 5% +68.9% 76474 ± 15% interrupts.CPU1.TRM:Thermal_event_interrupts 13947887 -11.0% 12413074 interrupts.CPU10.RES:Rescheduling_interrupts 45275 ± 5% +68.9% 76468 ± 15% interrupts.CPU10.TRM:Thermal_event_interrupts 45286 ± 5% +68.9% 76468 ± 15% interrupts.CPU11.TRM:Thermal_event_interrupts 13579741 -11.8% 11974820 ± 5% interrupts.CPU12.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU12.TRM:Thermal_event_interrupts 13668073 -11.3% 12119346 interrupts.CPU13.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76474 ± 15% interrupts.CPU13.TRM:Thermal_event_interrupts 13578574 -7.8% 12518854 interrupts.CPU14.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76467 ± 15% interrupts.CPU14.TRM:Thermal_event_interrupts 13689117 -10.1% 12310290 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76473 ± 15% interrupts.CPU15.TRM:Thermal_event_interrupts 13924470 -10.5% 12461613 interrupts.CPU2.RES:Rescheduling_interrupts 45264 ± 5% +68.9% 76467 ± 15% interrupts.CPU2.TRM:Thermal_event_interrupts 546.00 ± 23% +119.5% 1198 ± 16% interrupts.CPU3.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3 13951884 -10.5% 12480739 interrupts.CPU3.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76469 ± 15% interrupts.CPU3.TRM:Thermal_event_interrupts 45285 ± 5% +68.9% 76471 ± 15% interrupts.CPU4.TRM:Thermal_event_interrupts 14030690 -10.7% 12530992 ± 3% interrupts.CPU5.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU5.TRM:Thermal_event_interrupts 45285 ± 5% +68.9% 76474 ± 15% interrupts.CPU6.TRM:Thermal_event_interrupts 45285 ± 5% +68.9% 76472 ± 15% interrupts.CPU7.TRM:Thermal_event_interrupts 13319036 -10.2% 11966842 ± 2% interrupts.CPU8.RES:Rescheduling_interrupts 45280 ± 5% +68.7% 76405 ± 15% interrupts.CPU8.TRM:Thermal_event_interrupts 13401328 -11.8% 11816711 ± 3% interrupts.CPU9.RES:Rescheduling_interrupts 45279 ± 5% +68.9% 76467 ± 15% interrupts.CPU9.TRM:Thermal_event_interrupts 2.19e+08 -9.8% 1.974e+08 ± 2% interrupts.RES:Rescheduling_interrupts 724498 ± 5% +68.9% 1223435 ± 15% interrupts.TRM:Thermal_event_interrupts 53.44 +4.2% 55.68 perf-stat.i.MPKI 0.12 -0.0 0.10 ± 4% perf-stat.i.cache-miss-rate% 1764490 -13.1% 1533362 ± 3% perf-stat.i.cache-misses 1.767e+09 +3.6% 1.832e+09 perf-stat.i.cache-references 4055785 +6.2% 4306694 perf-stat.i.context-switches 336154 +21.5% 408363 perf-stat.i.cpu-migrations 34528 ± 5% +10.3% 38089 ± 3% perf-stat.i.cycles-between-cache-misses 0.01 ± 6% -0.0 0.01 ± 6% perf-stat.i.dTLB-load-miss-rate% 899577 ± 6% -15.2% 762553 ± 5% perf-stat.i.dTLB-load-misses 0.00 ± 7% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate% 22816 ± 7% -13.2% 19813 ± 4% perf-stat.i.dTLB-store-misses 6.044e+09 -1.2% 5.969e+09 perf-stat.i.dTLB-stores 31331199 -3.4% 30272777 perf-stat.i.iTLB-load-misses 1061 +3.1% 1094 perf-stat.i.instructions-per-iTLB-miss 1077 -2.9% 1046 perf-stat.i.minor-faults 73018 -10.9% 65028 ± 2% perf-stat.i.node-loads 122632 ± 2% -9.3% 111181 ± 4% perf-stat.i.node-stores 1077 -2.9% 1046 perf-stat.i.page-faults 53.29 +4.2% 55.53 perf-stat.overall.MPKI 0.10 -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate% 25264 +15.3% 29120 ± 3% perf-stat.overall.cycles-between-cache-misses 0.01 ± 6% -0.0 0.01 ± 6% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 7% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate% 1058 +2.9% 1089 perf-stat.overall.instructions-per-iTLB-miss 65158 +6.6% 69487 perf-stat.overall.path-length 1761814 -13.1% 1530883 ± 3% perf-stat.ps.cache-misses 1.765e+09 +3.7% 1.829e+09 perf-stat.ps.cache-references 4048770 +6.2% 4299578 perf-stat.ps.context-switches 335565 +21.5% 407697 perf-stat.ps.cpu-migrations 898100 ± 6% -15.2% 761293 ± 5% perf-stat.ps.dTLB-load-misses 22777 ± 7% -13.2% 19781 ± 4% perf-stat.ps.dTLB-store-misses 6.034e+09 -1.2% 5.96e+09 perf-stat.ps.dTLB-stores 31277565 -3.4% 30222833 perf-stat.ps.iTLB-load-misses 1075 -2.9% 1044 perf-stat.ps.minor-faults 72896 -10.9% 64922 ± 2% perf-stat.ps.node-loads 122424 ± 2% -9.3% 111005 ± 4% perf-stat.ps.node-stores 1075 -2.9% 1044 perf-stat.ps.page-faults 39.37 ±100% -36.4 2.98 ±173% perf-profile.calltrace.cycles-pp.start_thread 21.68 ±100% -20.0 1.64 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread 20.14 ±100% -18.6 1.52 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 20.00 ±100% -18.5 1.51 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 18.05 ±100% -16.7 1.39 ±173% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 16.75 ±100% -15.5 1.22 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread 16.70 ±100% -15.4 1.26 ±173% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 14.98 ±100% -13.9 1.09 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 14.82 ±100% -13.7 1.08 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 13.95 ±100% -12.9 1.03 ±173% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 12.97 ±100% -12.0 0.95 ±173% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 3.69 ± 6% -1.1 2.58 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 4.25 ± 6% -1.0 3.29 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.54 ± 10% -0.5 1.05 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 1.07 ± 16% -0.5 0.60 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write 1.14 ± 2% -0.3 0.80 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.pipe_wait 0.55 ± 4% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read 1.90 ± 9% -0.2 1.67 ± 7% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.71 ± 7% -0.2 1.48 ± 7% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64 0.68 ± 19% +0.2 0.86 perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.67 ± 16% +0.2 0.88 ± 4% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait 0.29 ±100% +0.4 0.64 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 0.31 ±100% +0.4 0.67 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read 0.32 ±100% +0.4 0.68 ± 2% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.29 ±100% +0.4 0.68 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.35 ±100% +0.4 0.78 perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule 1.23 ± 19% +0.4 1.68 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 1.29 ± 18% +0.5 1.76 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.00 +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.update_load_avg.set_next_entity.pick_next_task_fair.__schedule.schedule 0.00 +0.6 0.59 ± 7% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule 8.82 +0.6 9.46 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 0.47 ±100% +0.7 1.14 ± 6% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 2.21 ± 10% +0.7 2.88 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 9.10 +0.7 9.79 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 9.04 +0.7 9.74 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.71 ±100% +0.9 1.61 ± 3% perf-profile.calltrace.cycles-pp.__switch_to 0.68 ± 99% +0.9 1.60 ± 3% perf-profile.calltrace.cycles-pp.native_write_msr 0.86 ±100% +0.9 1.79 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm 6.54 +1.0 7.53 ± 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read 26.96 +1.0 27.95 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 1.42 ± 63% +1.0 2.47 ± 3% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.54 ± 8% +1.2 5.79 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.99 +1.3 16.27 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read 15.35 +1.3 16.66 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read 27.37 +1.3 28.70 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 17.81 +1.6 19.38 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read 3.95 ± 11% +1.8 5.70 ± 6% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 23.38 +2.0 25.33 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 24.30 +2.0 26.27 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 23.73 +2.0 25.71 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 5.24 ± 10% +2.0 7.25 ± 6% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 3.23 ± 57% +2.7 5.98 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.34 ± 57% +2.8 6.18 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 39.37 ±100% -36.4 2.98 ±173% perf-profile.children.cycles-pp.start_thread 21.68 ±100% -20.0 1.67 ±173% perf-profile.children.cycles-pp.__GI___libc_write 16.99 ±100% -15.7 1.24 ±173% perf-profile.children.cycles-pp.__GI___libc_read 6.84 ± 10% -2.1 4.71 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 45.35 -1.0 44.33 perf-profile.children.cycles-pp.ksys_write 6.70 ± 5% -1.0 5.70 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 5.90 ± 4% -0.6 5.34 ± 6% perf-profile.children.cycles-pp.security_file_permission 3.35 ± 9% -0.6 2.80 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 42.14 -0.5 41.63 perf-profile.children.cycles-pp.vfs_write 3.41 -0.4 3.01 perf-profile.children.cycles-pp.__fdget_pos 3.30 -0.4 2.91 perf-profile.children.cycles-pp.__fget_light 2.71 ± 2% -0.4 2.32 perf-profile.children.cycles-pp.__fget 3.23 -0.3 2.93 ± 3% perf-profile.children.cycles-pp.mutex_lock 2.76 -0.3 2.47 ± 5% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 2.21 ± 3% -0.3 1.93 ± 10% perf-profile.children.cycles-pp.file_has_perm 3.14 -0.2 2.90 ± 4% perf-profile.children.cycles-pp.copy_page_to_iter 3.03 ± 5% -0.2 2.79 ± 5% perf-profile.children.cycles-pp.selinux_file_permission 2.10 ± 5% -0.2 1.86 ± 2% perf-profile.children.cycles-pp.mutex_unlock 1.50 ± 5% -0.2 1.27 ± 2% perf-profile.children.cycles-pp.fput_many 0.68 ± 17% -0.2 0.48 ± 4% perf-profile.children.cycles-pp.__mutex_lock 1.66 ± 2% -0.2 1.47 ± 4% perf-profile.children.cycles-pp.copyout 1.73 -0.2 1.57 ± 2% perf-profile.children.cycles-pp.update_rq_clock 1.30 -0.1 1.17 ± 6% perf-profile.children.cycles-pp.copyin 0.27 ± 11% -0.1 0.15 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common 1.25 ± 3% -0.1 1.15 ± 3% perf-profile.children.cycles-pp.fsnotify 0.79 -0.1 0.69 ± 7% perf-profile.children.cycles-pp.reschedule_interrupt 0.91 -0.1 0.82 ± 5% perf-profile.children.cycles-pp._cond_resched 0.80 ± 2% -0.1 0.71 ± 3% perf-profile.children.cycles-pp.file_update_time 0.29 ± 10% -0.1 0.24 ± 5% perf-profile.children.cycles-pp.rb_insert_color 0.26 -0.0 0.21 ± 2% perf-profile.children.cycles-pp.native_irq_return_iret 0.07 ± 14% -0.0 0.03 ±100% perf-profile.children.cycles-pp.scheduler_ipi 0.22 ± 6% -0.0 0.18 ± 4% perf-profile.children.cycles-pp.__x64_sys_read 0.25 ± 4% -0.0 0.21 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.27 -0.0 0.23 ± 9% perf-profile.children.cycles-pp.timespec64_trunc 0.23 ± 2% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.smp_reschedule_interrupt 0.22 -0.0 0.20 ± 5% perf-profile.children.cycles-pp.__x64_sys_write 0.24 -0.0 0.22 ± 3% perf-profile.children.cycles-pp.__list_add_valid 0.14 -0.0 0.12 ± 10% perf-profile.children.cycles-pp.kill_fasync 0.11 ± 4% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.perf_exclude_event 0.10 +0.0 0.12 ± 8% perf-profile.children.cycles-pp.is_cpu_allowed 0.09 ± 11% +0.0 0.12 ± 7% perf-profile.children.cycles-pp.rcu_note_context_switch 0.07 +0.0 0.10 ± 10% perf-profile.children.cycles-pp.pkg_thermal_notify 0.17 ± 3% +0.0 0.20 ± 9% perf-profile.children.cycles-pp.deactivate_task 0.16 +0.0 0.21 ± 10% perf-profile.children.cycles-pp.update_cfs_rq_h_load 0.08 ± 6% +0.0 0.12 ± 10% perf-profile.children.cycles-pp.intel_thermal_interrupt 0.08 ± 6% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.smp_thermal_interrupt 0.08 +0.1 0.13 ± 9% perf-profile.children.cycles-pp.thermal_interrupt 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.default_wake_function 0.19 ± 5% +0.1 0.25 ± 5% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.19 ± 5% +0.1 0.27 ± 4% perf-profile.children.cycles-pp.finish_wait 0.89 ± 8% +0.1 0.98 perf-profile.children.cycles-pp.__calc_delta 0.63 ± 4% +0.1 0.74 ± 4% perf-profile.children.cycles-pp.native_sched_clock 0.33 ± 4% +0.1 0.44 ± 5% perf-profile.children.cycles-pp.set_next_buddy 1.23 ± 3% +0.1 1.34 ± 3% perf-profile.children.cycles-pp.check_preempt_wakeup 0.69 ± 4% +0.1 0.81 ± 3% perf-profile.children.cycles-pp.sched_clock 1.52 ± 3% +0.1 1.65 ± 2% perf-profile.children.cycles-pp.set_next_entity 0.73 ± 4% +0.1 0.88 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.57 ± 4% +0.2 0.74 ± 4% perf-profile.children.cycles-pp.account_entity_dequeue 1.21 ± 4% +0.2 1.39 ± 2% perf-profile.children.cycles-pp.__update_load_avg_se 1.99 ± 3% +0.2 2.20 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 1.64 ± 6% +0.2 1.85 perf-profile.children.cycles-pp.__switch_to_asm 1.60 ± 4% +0.2 1.85 ± 3% perf-profile.children.cycles-pp.update_cfs_group 3.91 +0.2 4.15 perf-profile.children.cycles-pp.update_curr 1.04 ± 5% +0.3 1.30 ± 4% perf-profile.children.cycles-pp.___perf_sw_event 1.05 ± 9% +0.3 1.33 ± 2% perf-profile.children.cycles-pp.pick_next_entity 1.57 ± 3% +0.3 1.89 perf-profile.children.cycles-pp.native_write_msr 1.01 ± 9% +0.3 1.33 ± 5% perf-profile.children.cycles-pp.put_prev_entity 2.17 ± 7% +0.3 2.51 ± 2% perf-profile.children.cycles-pp.switch_fpu_return 1.93 ± 7% +0.4 2.36 perf-profile.children.cycles-pp.__switch_to 2.99 ± 4% +0.6 3.57 ± 2% perf-profile.children.cycles-pp.reweight_entity 8.89 +0.6 9.50 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair 9.16 +0.7 9.82 perf-profile.children.cycles-pp.ttwu_do_activate 9.09 +0.7 9.76 ± 2% perf-profile.children.cycles-pp.activate_task 5.18 ± 6% +0.9 6.04 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair 6.71 +0.9 7.63 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair 28.01 +1.4 29.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common_lock 4.91 ± 7% +1.4 6.28 ± 4% perf-profile.children.cycles-pp.exit_to_usermode_loop 18.19 +1.4 19.56 perf-profile.children.cycles-pp.pipe_wait 4.03 ± 11% +1.7 5.75 ± 6% perf-profile.children.cycles-pp.select_idle_sibling 23.86 ± 2% +1.8 25.71 ± 2% perf-profile.children.cycles-pp.try_to_wake_up 24.44 +1.9 26.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common 23.84 +1.9 25.77 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function 5.29 ± 10% +2.0 7.29 ± 6% perf-profile.children.cycles-pp.select_task_rq_fair 20.27 ± 2% +2.3 22.53 ± 2% perf-profile.children.cycles-pp.__schedule 20.36 ± 2% +2.5 22.82 ± 2% perf-profile.children.cycles-pp.schedule 6.82 ± 10% -2.1 4.69 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 2.67 ± 2% -0.4 2.29 perf-profile.self.cycles-pp.__fget 2.72 -0.3 2.44 ± 5% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.22 -0.2 0.98 ± 2% perf-profile.self.cycles-pp.update_rq_clock 1.36 ± 3% -0.2 1.13 ± 6% perf-profile.self.cycles-pp.pipe_write 1.48 ± 5% -0.2 1.24 ± 3% perf-profile.self.cycles-pp.fput_many 2.04 ± 5% -0.2 1.81 ± 2% perf-profile.self.cycles-pp.mutex_unlock 1.96 -0.2 1.73 ± 2% perf-profile.self.cycles-pp.mutex_lock 0.66 ± 12% -0.1 0.53 ± 10% perf-profile.self.cycles-pp.file_has_perm 0.38 ± 10% -0.1 0.28 ± 9% perf-profile.self.cycles-pp.ksys_read 0.34 ± 7% -0.1 0.25 ± 7% perf-profile.self.cycles-pp.__mutex_lock 1.21 ± 3% -0.1 1.11 ± 2% perf-profile.self.cycles-pp.fsnotify 1.00 -0.1 0.94 ± 5% perf-profile.self.cycles-pp.__might_sleep 0.60 ± 4% -0.1 0.55 ± 8% perf-profile.self.cycles-pp.vfs_write 0.29 ± 10% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.rb_insert_color 0.32 ± 6% -0.0 0.28 ± 6% perf-profile.self.cycles-pp.ksys_write 0.26 -0.0 0.21 ± 2% perf-profile.self.cycles-pp.native_irq_return_iret 0.07 ± 14% -0.0 0.03 ±100% perf-profile.self.cycles-pp.scheduler_ipi 0.36 ± 5% -0.0 0.32 ± 8% perf-profile.self.cycles-pp.__sb_start_write 0.29 -0.0 0.24 ± 4% perf-profile.self.cycles-pp.check_preempt_curr 0.22 ± 6% -0.0 0.17 ± 6% perf-profile.self.cycles-pp.__x64_sys_read 0.25 -0.0 0.21 ± 8% perf-profile.self.cycles-pp.timespec64_trunc 0.22 ± 4% -0.0 0.19 ± 11% perf-profile.self.cycles-pp.wake_q_add 0.21 ± 2% -0.0 0.18 ± 6% perf-profile.self.cycles-pp.__x64_sys_write 0.21 ± 2% -0.0 0.20 perf-profile.self.cycles-pp.__list_add_valid 0.11 ± 4% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.kill_fasync 0.06 ± 9% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.rcu_note_context_switch 0.15 +0.0 0.18 ± 6% perf-profile.self.cycles-pp.autoremove_wake_function 0.16 +0.0 0.19 ± 9% perf-profile.self.cycles-pp.deactivate_task 0.26 +0.0 0.29 ± 5% perf-profile.self.cycles-pp.rcu_all_qs 0.33 ± 3% +0.0 0.37 ± 4% perf-profile.self.cycles-pp.set_next_entity 0.18 ± 5% +0.0 0.22 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.03 ±100% +0.0 0.07 perf-profile.self.cycles-pp.check_cfs_rq_runtime 0.16 ± 6% +0.0 0.20 ± 6% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.16 +0.0 0.20 ± 12% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.20 ± 2% +0.1 0.25 ± 2% perf-profile.self.cycles-pp.activate_task 0.41 ± 6% +0.1 0.46 perf-profile.self.cycles-pp.schedule 1.96 +0.1 2.02 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.15 ± 6% +0.1 0.22 perf-profile.self.cycles-pp.finish_wait 0.41 ± 4% +0.1 0.48 ± 6% perf-profile.self.cycles-pp.account_entity_enqueue 0.61 ± 4% +0.1 0.71 ± 3% perf-profile.self.cycles-pp.native_sched_clock 0.83 ± 2% +0.1 0.93 perf-profile.self.cycles-pp.dequeue_task_fair 0.29 ± 3% +0.1 0.40 ± 5% perf-profile.self.cycles-pp.set_next_buddy 1.02 +0.1 1.15 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair 0.46 ± 12% +0.2 0.61 ± 3% perf-profile.self.cycles-pp.pick_next_entity 1.17 ± 4% +0.2 1.35 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se 0.42 ± 8% +0.2 0.61 ± 5% perf-profile.self.cycles-pp.account_entity_dequeue 1.64 ± 6% +0.2 1.85 perf-profile.self.cycles-pp.__switch_to_asm 1.23 ± 4% +0.2 1.46 ± 3% perf-profile.self.cycles-pp.reweight_entity 1.07 ± 5% +0.2 1.31 ± 4% perf-profile.self.cycles-pp.select_task_rq_fair 0.90 ± 5% +0.2 1.14 ± 5% perf-profile.self.cycles-pp.___perf_sw_event 1.57 ± 4% +0.2 1.82 ± 2% perf-profile.self.cycles-pp.update_cfs_group 1.17 ± 3% +0.3 1.42 ± 7% perf-profile.self.cycles-pp.update_load_avg 1.56 ± 4% +0.3 1.88 perf-profile.self.cycles-pp.native_write_msr 2.16 ± 7% +0.3 2.50 ± 2% perf-profile.self.cycles-pp.switch_fpu_return 1.84 ± 8% +0.4 2.25 ± 2% perf-profile.self.cycles-pp.__switch_to 1.22 +0.5 1.73 ± 4% perf-profile.self.cycles-pp._raw_spin_lock 0.92 ± 11% +1.3 2.27 ± 6% perf-profile.self.cycles-pp.select_idle_sibling Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

10 months

4
6
0 / 0

[btrfs] c75e839414: aim7.jobs-per-min -9.1% regression

by kernel test robot

Greeting, FYI, we noticed a -9.1% regression of aim7.jobs-per-min due to commit: commit: c75e839414d3610e6487ae3145199c500d55f7f7 ("btrfs: kill the subvol_srcu") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: aim7 on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory with following parameters: disk: 4BRD_12G md: RAID0 fs: btrfs test: disk_wrt load: 1500 cpufreq_governor: performance ucode: 0x500002c test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system. test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/4BRD_12G/btrfs/x86_64-rhel-7.6/1500/RAID0/debian-x86_64-20191114.cgz/lkp-csl-2ap2/disk_wrt/aim7/0x500002c commit: efc3453494 ("btrfs: make btrfs_cleanup_fs_roots use the radix tree lock") c75e839414 ("btrfs: kill the subvol_srcu") efc3453494af7818 c75e839414d3610e6487ae31451 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 3:9 -33% :8 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x %stddev %change %stddev \ | \ 29509 ± 2% -9.1% 26837 ± 2% aim7.jobs-per-min 305.28 ± 2% +10.0% 335.72 ± 2% aim7.time.elapsed_time 305.28 ± 2% +10.0% 335.72 ± 2% aim7.time.elapsed_time.max 4883135 ± 10% +37.9% 6735464 ± 7% aim7.time.involuntary_context_switches 56288 ± 2% +10.5% 62202 ± 2% aim7.time.system_time 2344783 +6.5% 2497364 ± 2% aim7.time.voluntary_context_switches 62337721 ± 2% +9.8% 68456490 ± 2% turbostat.IRQ 431.56 ± 6% +22.3% 527.88 ± 4% vmstat.procs.r 27340 ± 2% +11.2% 30397 ± 2% vmstat.system.cs 226804 ± 6% +21.7% 276057 ± 4% meminfo.Active(file) 221309 ± 6% +22.3% 270668 ± 4% meminfo.Dirty 720.89 ±111% +49.3% 1076 ± 73% meminfo.Mlocked 14278 ± 2% -8.3% 13094 ± 2% meminfo.max_used_kB 57228 ± 6% +22.7% 70195 ± 5% numa-meminfo.node0.Active(file) 55433 ± 6% +21.6% 67431 ± 4% numa-meminfo.node0.Dirty 56152 ± 6% +21.4% 68180 ± 5% numa-meminfo.node1.Active(file) 55001 ± 6% +22.5% 67397 ± 4% numa-meminfo.node1.Dirty 56373 ± 6% +21.7% 68594 ± 4% numa-meminfo.node2.Active(file) 55222 ± 7% +22.6% 67726 ± 4% numa-meminfo.node2.Dirty 56671 ± 6% +20.5% 68317 ± 3% numa-meminfo.node3.Active(file) 55285 ± 6% +21.8% 67355 ± 4% numa-meminfo.node3.Dirty 56694 ± 6% +21.7% 69019 ± 4% proc-vmstat.nr_active_file 55342 ± 6% +22.3% 67662 ± 4% proc-vmstat.nr_dirty 402316 +2.1% 410951 proc-vmstat.nr_file_pages 180.22 ±111% +49.4% 269.25 ± 73% proc-vmstat.nr_mlock 56694 ± 6% +21.7% 69019 ± 4% proc-vmstat.nr_zone_active_file 54680 ± 6% +22.8% 67168 ± 4% proc-vmstat.nr_zone_write_pending 3144381 ± 2% +6.1% 3335275 proc-vmstat.pgactivate 1387558 ± 2% +7.9% 1496754 ± 2% proc-vmstat.pgfault 983.33 ± 4% +5.4% 1036 proc-vmstat.unevictable_pgs_culled 14331 ± 6% +22.6% 17566 ± 5% numa-vmstat.node0.nr_active_file 13884 ± 6% +21.6% 16884 ± 4% numa-vmstat.node0.nr_dirty 14330 ± 6% +22.6% 17566 ± 5% numa-vmstat.node0.nr_zone_active_file 13714 ± 6% +22.2% 16755 ± 4% numa-vmstat.node0.nr_zone_write_pending 14047 ± 6% +21.3% 17043 ± 4% numa-vmstat.node1.nr_active_file 13763 ± 6% +22.3% 16838 ± 4% numa-vmstat.node1.nr_dirty 14047 ± 6% +21.3% 17043 ± 4% numa-vmstat.node1.nr_zone_active_file 13599 ± 6% +23.0% 16726 ± 4% numa-vmstat.node1.nr_zone_write_pending 14074 ± 5% +21.7% 17130 ± 4% numa-vmstat.node2.nr_active_file 13786 ± 6% +22.6% 16906 ± 4% numa-vmstat.node2.nr_dirty 14074 ± 5% +21.7% 17130 ± 4% numa-vmstat.node2.nr_zone_active_file 13617 ± 7% +23.3% 16789 ± 4% numa-vmstat.node2.nr_zone_write_pending 14151 ± 6% +20.7% 17074 ± 3% numa-vmstat.node3.nr_active_file 13790 ± 6% +22.1% 16832 ± 4% numa-vmstat.node3.nr_dirty 14152 ± 6% +20.7% 17074 ± 3% numa-vmstat.node3.nr_zone_active_file 13627 ± 6% +22.5% 16696 ± 4% numa-vmstat.node3.nr_zone_write_pending 828631 ± 16% +49.9% 1241934 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev 1.33 ± 30% +96.9% 2.61 ± 14% sched_debug.cfs_rq:/.nr_spread_over.avg 831152 ± 16% +49.7% 1244586 ± 8% sched_debug.cfs_rq:/.spread0.stddev 1.92 ± 14% +28.4% 2.47 ± 8% sched_debug.cpu.nr_running.avg 5.00 ± 9% +29.7% 6.48 ± 9% sched_debug.cpu.nr_running.max 0.90 ± 9% +16.0% 1.05 ± 3% sched_debug.cpu.nr_running.stddev 21790 ± 11% +16.7% 25440 ± 2% sched_debug.cpu.nr_switches.avg 19761 ± 11% +17.9% 23289 ± 2% sched_debug.cpu.nr_switches.min 20403 ± 12% +18.1% 24101 ± 2% sched_debug.cpu.sched_count.avg 27825 ± 10% +15.8% 32227 ± 4% sched_debug.cpu.sched_count.max 18995 ± 13% +18.3% 22463 ± 2% sched_debug.cpu.sched_count.min 1143 ± 11% +15.6% 1322 ± 4% sched_debug.cpu.sched_count.stddev 1404 ± 7% -31.0% 969.66 ± 11% sched_debug.cpu.sched_goidle.avg 1186 ± 8% -36.0% 759.25 ± 14% sched_debug.cpu.sched_goidle.min 16821 ± 14% +20.6% 20284 ± 6% sched_debug.cpu.ttwu_count.max 2790 ± 11% +21.7% 3395 ± 7% sched_debug.cpu.ttwu_count.stddev 398.19 ± 16% +28.3% 510.73 ± 10% sched_debug.cpu.ttwu_local.stddev 43238516 ± 5% +19.0% 51459028 ± 4% perf-stat.i.cache-misses 83870573 ± 4% +13.8% 95436555 ± 4% perf-stat.i.cache-references 27181 ± 2% +11.1% 30198 ± 2% perf-stat.i.context-switches 3008 ± 2% -7.1% 2794 perf-stat.i.cpu-migrations 12612 ± 5% -15.6% 10641 ± 3% perf-stat.i.cycles-between-cache-misses 1.1e+09 ± 2% -7.3% 1.019e+09 ± 2% perf-stat.i.dTLB-stores 10684629 ± 2% +10.8% 11842781 ± 2% perf-stat.i.node-load-misses 641429 ± 2% +24.4% 798190 ± 2% perf-stat.i.node-loads 5915955 ± 2% -6.3% 5543368 ± 2% perf-stat.i.node-store-misses 380971 -6.7% 355629 ± 2% perf-stat.i.node-stores 1.66 ± 4% +14.4% 1.90 ± 3% perf-stat.overall.MPKI 13064 ± 5% -15.8% 10999 ± 4% perf-stat.overall.cycles-between-cache-misses 43484558 ± 5% +19.2% 51837476 ± 4% perf-stat.ps.cache-misses 83278267 ± 4% +14.4% 95253129 ± 3% perf-stat.ps.cache-references 27414 ± 2% +11.2% 30486 ± 2% perf-stat.ps.context-switches 3028 ± 2% -7.0% 2817 perf-stat.ps.cpu-migrations 1.104e+09 ± 2% -7.3% 1.023e+09 ± 2% perf-stat.ps.dTLB-stores 4394 -1.8% 4315 perf-stat.ps.minor-faults 10766873 ± 2% +11.0% 11947268 ± 2% perf-stat.ps.node-load-misses 645940 ± 2% +24.7% 805392 ± 2% perf-stat.ps.node-loads 5965618 ± 2% -6.2% 5595562 ± 2% perf-stat.ps.node-store-misses 383092 -6.5% 358124 ± 2% perf-stat.ps.node-stores 4395 -1.8% 4315 perf-stat.ps.page-faults 1.538e+13 ± 2% +10.0% 1.692e+13 ± 2% perf-stat.total.instructions 27.39 -0.1 27.28 perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 35.13 +0.1 35.23 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 35.23 +0.1 35.34 perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 35.48 +0.1 35.60 perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 35.08 +0.1 35.22 perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter 34.97 +0.1 35.12 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write 35.57 +0.1 35.72 perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 35.10 +0.2 35.26 perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 35.11 +0.2 35.27 perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 98.71 +0.2 98.88 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 98.69 +0.2 98.87 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 98.74 +0.2 98.91 perf-profile.calltrace.cycles-pp.write 98.71 +0.2 98.89 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 98.68 +0.2 98.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 98.64 +0.2 98.82 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 98.63 +0.2 98.81 perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 98.58 +0.2 98.77 perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 27.39 -0.1 27.28 perf-profile.children.cycles-pp.btrfs_dirty_pages 0.56 ± 5% -0.1 0.46 ± 9% perf-profile.children.cycles-pp.btrfs_get_extent 0.51 ± 5% -0.1 0.41 ± 9% perf-profile.children.cycles-pp.btrfs_lookup_file_extent 0.52 ± 5% -0.1 0.43 ± 9% perf-profile.children.cycles-pp.btrfs_search_slot 0.44 ± 6% -0.1 0.35 ± 11% perf-profile.children.cycles-pp.btrfs_read_lock_root_node 0.43 ± 6% -0.1 0.34 ± 11% perf-profile.children.cycles-pp.btrfs_tree_read_lock 0.19 ± 15% -0.1 0.11 ± 27% perf-profile.children.cycles-pp.start_secondary 0.19 ± 16% -0.1 0.10 ± 27% perf-profile.children.cycles-pp.cpuidle_enter 0.19 ± 16% -0.1 0.10 ± 27% perf-profile.children.cycles-pp.cpuidle_enter_state 0.41 ± 7% -0.1 0.33 ± 8% perf-profile.children.cycles-pp.rwsem_optimistic_spin 0.46 ± 7% -0.1 0.38 ± 8% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.secondary_startup_64 0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.cpu_startup_entry 0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.do_idle 0.18 ± 15% -0.1 0.10 ± 24% perf-profile.children.cycles-pp.intel_idle 0.29 ± 9% -0.1 0.22 ± 11% perf-profile.children.cycles-pp.osq_lock 0.19 ± 6% -0.0 0.15 ± 16% perf-profile.children.cycles-pp.finish_wait 0.39 ± 5% -0.0 0.35 ± 6% perf-profile.children.cycles-pp.do_unlinkat 0.39 ± 6% -0.0 0.35 ± 7% perf-profile.children.cycles-pp.unlink 0.26 ± 5% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_sys_open 0.26 ± 5% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_sys_openat2 0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_filp_open 0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.path_openat 0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.creat 0.00 +0.0 0.05 perf-profile.children.cycles-pp.btrfs_calculate_inode_block_rsv_size 99.65 +0.1 99.73 perf-profile.children.cycles-pp.do_syscall_64 99.66 +0.1 99.74 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 35.62 +0.1 35.74 perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes 35.57 +0.1 35.72 perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 97.55 +0.2 97.72 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 98.74 +0.2 98.92 perf-profile.children.cycles-pp.write 98.70 +0.2 98.88 perf-profile.children.cycles-pp.ksys_write 98.69 +0.2 98.87 perf-profile.children.cycles-pp.vfs_write 98.65 +0.2 98.83 perf-profile.children.cycles-pp.new_sync_write 98.63 +0.2 98.81 perf-profile.children.cycles-pp.btrfs_file_write_iter 98.59 +0.2 98.78 perf-profile.children.cycles-pp.btrfs_buffered_write 97.60 +0.2 97.79 perf-profile.children.cycles-pp._raw_spin_lock 0.18 ± 15% -0.1 0.10 ± 24% perf-profile.self.cycles-pp.intel_idle 0.28 ± 9% -0.1 0.21 ± 12% perf-profile.self.cycles-pp.osq_lock 0.45 ± 2% -0.0 0.41 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 135788 ± 2% +13.1% 153639 ± 3% softirqs.CPU0.RCU 115547 ± 3% +8.0% 124768 ± 4% softirqs.CPU0.TIMER 135243 ± 3% +14.0% 154188 ± 2% softirqs.CPU1.RCU 136056 ± 3% +12.4% 152960 ± 3% softirqs.CPU10.RCU 134253 ± 3% +14.2% 153260 ± 3% softirqs.CPU100.RCU 134668 ± 4% +13.7% 153157 ± 2% softirqs.CPU101.RCU 109995 ± 3% +7.9% 118637 ± 3% softirqs.CPU101.TIMER 134989 ± 4% +13.9% 153700 ± 2% softirqs.CPU102.RCU 109876 ± 3% +8.9% 119608 ± 2% softirqs.CPU102.TIMER 134868 ± 4% +13.8% 153427 ± 3% softirqs.CPU103.RCU 109933 ± 3% +8.2% 118946 ± 3% softirqs.CPU103.TIMER 134454 ± 3% +14.0% 153285 ± 3% softirqs.CPU104.RCU 109731 ± 2% +8.4% 118965 ± 3% softirqs.CPU104.TIMER 134878 ± 3% +13.6% 153283 ± 3% softirqs.CPU105.RCU 134096 ± 3% +14.1% 153022 ± 3% softirqs.CPU106.RCU 109821 ± 2% +8.2% 118845 ± 3% softirqs.CPU106.TIMER 133905 ± 3% +15.0% 154004 ± 3% softirqs.CPU107.RCU 134117 ± 3% +14.3% 153232 ± 3% softirqs.CPU108.RCU 109545 ± 2% +8.8% 119182 ± 3% softirqs.CPU108.TIMER 134574 ± 3% +13.8% 153202 ± 3% softirqs.CPU109.RCU 109804 ± 3% +8.2% 118760 ± 3% softirqs.CPU109.TIMER 135106 ± 3% +14.8% 155124 ± 3% softirqs.CPU11.RCU 111746 ± 4% +7.8% 120460 ± 3% softirqs.CPU11.TIMER 134482 ± 4% +14.3% 153687 ± 3% softirqs.CPU110.RCU 110314 ± 3% +8.1% 119241 ± 3% softirqs.CPU110.TIMER 134464 ± 3% +14.3% 153634 ± 3% softirqs.CPU111.RCU 110626 ± 3% +8.3% 119815 ± 3% softirqs.CPU111.TIMER 134895 ± 3% +14.4% 154257 ± 3% softirqs.CPU112.RCU 110040 ± 3% +14.6% 126058 ± 16% softirqs.CPU112.TIMER 134644 ± 3% +13.6% 152993 ± 3% softirqs.CPU113.RCU 109693 ± 3% +8.1% 118569 ± 3% softirqs.CPU113.TIMER 134313 ± 3% +13.9% 152993 ± 3% softirqs.CPU114.RCU 109711 ± 3% +8.6% 119143 ± 3% softirqs.CPU114.TIMER 134559 ± 3% +13.8% 153115 ± 3% softirqs.CPU115.RCU 110226 ± 3% +8.2% 119221 ± 3% softirqs.CPU115.TIMER 134087 ± 3% +14.2% 153077 ± 3% softirqs.CPU116.RCU 110225 ± 3% +9.0% 120184 ± 3% softirqs.CPU116.TIMER 134271 ± 3% +14.1% 153220 ± 3% softirqs.CPU117.RCU 109179 ± 2% +8.3% 118274 ± 3% softirqs.CPU117.TIMER 134281 ± 3% +14.1% 153230 ± 3% softirqs.CPU118.RCU 134415 ± 3% +14.0% 153215 ± 3% softirqs.CPU119.RCU 109514 ± 2% +8.3% 118643 ± 3% softirqs.CPU119.TIMER 135560 ± 3% +13.5% 153874 ± 3% softirqs.CPU12.RCU 110316 ± 2% +8.5% 119734 ± 3% softirqs.CPU12.TIMER 134177 ± 3% +14.6% 153815 ± 3% softirqs.CPU120.RCU 134367 ± 3% +15.0% 154493 ± 3% softirqs.CPU121.RCU 133972 ± 3% +14.3% 153173 ± 3% softirqs.CPU122.RCU 134801 ± 4% +13.6% 153136 ± 3% softirqs.CPU123.RCU 134709 ± 3% +13.9% 153440 ± 3% softirqs.CPU124.RCU 135185 ± 3% +13.1% 152925 ± 3% softirqs.CPU125.RCU 134977 ± 3% +13.8% 153608 ± 3% softirqs.CPU126.RCU 133862 ± 4% +14.7% 153558 ± 3% softirqs.CPU127.RCU 134554 ± 3% +13.9% 153237 ± 3% softirqs.CPU128.RCU 134254 ± 3% +14.1% 153130 ± 3% softirqs.CPU129.RCU 135133 ± 3% +14.0% 154103 ± 3% softirqs.CPU13.RCU 110389 ± 2% +8.1% 119382 ± 3% softirqs.CPU13.TIMER 134319 ± 3% +13.8% 152909 ± 3% softirqs.CPU130.RCU 133590 ± 3% +14.5% 152925 ± 3% softirqs.CPU131.RCU 134416 ± 4% +14.0% 153223 ± 3% softirqs.CPU132.RCU 133931 ± 3% +14.5% 153359 ± 3% softirqs.CPU133.RCU 134031 ± 3% +14.5% 153527 ± 3% softirqs.CPU134.RCU 133851 ± 4% +14.3% 153052 ± 3% softirqs.CPU135.RCU 134235 ± 4% +13.9% 152959 ± 3% softirqs.CPU136.RCU 134003 ± 3% +14.2% 153037 ± 3% softirqs.CPU137.RCU 133453 ± 3% +14.4% 152625 ± 3% softirqs.CPU138.RCU 133514 ± 3% +14.8% 153289 ± 3% softirqs.CPU139.RCU 134952 ± 3% +13.6% 153372 ± 3% softirqs.CPU14.RCU 134045 ± 3% +13.7% 152361 ± 3% softirqs.CPU140.RCU 133662 ± 3% +14.1% 152528 ± 3% softirqs.CPU141.RCU 133643 ± 3% +14.7% 153283 ± 2% softirqs.CPU142.RCU 133313 ± 3% +15.0% 153254 ± 3% softirqs.CPU143.RCU 133585 ± 3% +14.4% 152837 ± 3% softirqs.CPU144.RCU 108590 ± 3% +12.7% 122339 ± 5% softirqs.CPU144.TIMER 133471 ± 3% +14.5% 152883 ± 3% softirqs.CPU145.RCU 108095 ± 3% +14.5% 123726 ± 6% softirqs.CPU145.TIMER 133588 ± 3% +15.4% 154173 ± 3% softirqs.CPU146.RCU 108268 ± 3% +19.8% 129684 ± 16% softirqs.CPU146.TIMER 134060 ± 3% +14.3% 153270 ± 3% softirqs.CPU147.RCU 108731 ± 3% +13.0% 122902 ± 5% softirqs.CPU147.TIMER 133778 ± 3% +14.1% 152681 ± 3% softirqs.CPU148.RCU 108208 ± 3% +12.6% 121875 ± 5% softirqs.CPU148.TIMER 133514 ± 3% +14.2% 152460 ± 3% softirqs.CPU149.RCU 108054 ± 3% +15.6% 124938 ± 9% softirqs.CPU149.TIMER 134677 ± 3% +14.5% 154138 ± 2% softirqs.CPU15.RCU 133245 ± 3% +14.4% 152373 ± 3% softirqs.CPU150.RCU 107788 ± 3% +13.1% 121957 ± 5% softirqs.CPU150.TIMER 133747 ± 3% +14.0% 152473 ± 3% softirqs.CPU151.RCU 108258 ± 4% +12.4% 121662 ± 5% softirqs.CPU151.TIMER 133819 ± 3% +14.1% 152726 ± 3% softirqs.CPU152.RCU 108163 ± 3% +12.7% 121940 ± 5% softirqs.CPU152.TIMER 134057 ± 3% +14.0% 152844 ± 3% softirqs.CPU153.RCU 107680 ± 3% +13.1% 121787 ± 5% softirqs.CPU153.TIMER 133660 ± 3% +14.2% 152675 ± 3% softirqs.CPU154.RCU 107911 ± 3% +15.4% 124485 ± 9% softirqs.CPU154.TIMER 133349 ± 3% +14.6% 152811 ± 3% softirqs.CPU155.RCU 107513 ± 3% +15.1% 123737 ± 6% softirqs.CPU155.TIMER 133901 ± 3% +14.0% 152615 ± 3% softirqs.CPU156.RCU 107853 ± 3% +13.0% 121904 ± 5% softirqs.CPU156.TIMER 133621 ± 3% +14.2% 152615 ± 3% softirqs.CPU157.RCU 108040 ± 3% +12.6% 121684 ± 5% softirqs.CPU157.TIMER 133891 ± 3% +14.2% 152862 ± 3% softirqs.CPU158.RCU 108534 ± 3% +12.7% 122300 ± 5% softirqs.CPU158.TIMER 133925 ± 3% +14.5% 153347 ± 3% softirqs.CPU159.RCU 108099 ± 3% +12.9% 122015 ± 5% softirqs.CPU159.TIMER 134893 ± 3% +14.5% 154389 ± 3% softirqs.CPU16.RCU 134169 ± 3% +13.9% 152812 ± 3% softirqs.CPU160.RCU 108300 ± 3% +12.9% 122318 ± 5% softirqs.CPU160.TIMER 133818 ± 3% +15.1% 154048 ± 4% softirqs.CPU161.RCU 107919 ± 3% +13.6% 122606 ± 6% softirqs.CPU161.TIMER 133867 ± 3% +14.2% 152856 ± 3% softirqs.CPU162.RCU 107991 ± 3% +13.2% 122209 ± 5% softirqs.CPU162.TIMER 133579 ± 3% +14.3% 152654 ± 3% softirqs.CPU163.RCU 108317 ± 3% +12.9% 122271 ± 5% softirqs.CPU163.TIMER 133914 ± 3% +14.4% 153158 ± 3% softirqs.CPU164.RCU 133684 ± 3% +14.4% 152895 ± 3% softirqs.CPU165.RCU 107435 ± 3% +12.5% 120877 ± 5% softirqs.CPU165.TIMER 133983 ± 3% +13.9% 152582 ± 3% softirqs.CPU166.RCU 107432 ± 3% +13.2% 121641 ± 5% softirqs.CPU166.TIMER 133947 ± 3% +13.6% 152137 ± 3% softirqs.CPU167.RCU 134005 ± 3% +14.5% 153385 ± 4% softirqs.CPU168.RCU 115752 ± 4% +12.0% 129634 ± 4% softirqs.CPU168.TIMER 133557 ± 3% +15.2% 153865 ± 4% softirqs.CPU169.RCU 115884 ± 3% +11.7% 129481 ± 4% softirqs.CPU169.TIMER 135348 ± 3% +13.8% 153977 ± 3% softirqs.CPU17.RCU 110589 ± 3% +7.9% 119298 ± 3% softirqs.CPU17.TIMER 134463 ± 3% +14.0% 153229 ± 3% softirqs.CPU170.RCU 133740 ± 3% +14.1% 152630 ± 3% softirqs.CPU171.RCU 116524 ± 4% +10.9% 129237 ± 3% softirqs.CPU171.TIMER 133716 ± 3% +13.8% 152196 ± 3% softirqs.CPU172.RCU 116393 ± 4% +10.9% 129087 ± 4% softirqs.CPU172.TIMER 133564 ± 3% +13.9% 152165 ± 3% softirqs.CPU173.RCU 116237 ± 3% +11.4% 129460 ± 3% softirqs.CPU173.TIMER 133411 ± 3% +14.5% 152764 ± 3% softirqs.CPU174.RCU 116018 ± 3% +11.2% 129006 ± 4% softirqs.CPU174.TIMER 133147 ± 3% +15.0% 153104 ± 2% softirqs.CPU175.RCU 116052 ± 4% +11.7% 129664 ± 3% softirqs.CPU175.TIMER 133989 ± 3% +13.8% 152504 ± 3% softirqs.CPU176.RCU 116071 ± 4% +10.9% 128756 ± 3% softirqs.CPU176.TIMER 133902 ± 3% +14.1% 152811 ± 3% softirqs.CPU177.RCU 115994 ± 3% +11.1% 128849 ± 4% softirqs.CPU177.TIMER 133679 ± 3% +14.4% 152887 ± 3% softirqs.CPU178.RCU 115942 ± 4% +13.4% 131439 ± 5% softirqs.CPU178.TIMER 133985 ± 3% +14.0% 152762 ± 3% softirqs.CPU179.RCU 116347 ± 4% +11.2% 129422 ± 4% softirqs.CPU179.TIMER 134982 ± 3% +14.0% 153838 ± 3% softirqs.CPU18.RCU 110310 ± 3% +8.6% 119851 ± 3% softirqs.CPU18.TIMER 133236 ± 3% +14.7% 152870 ± 3% softirqs.CPU180.RCU 115901 ± 4% +11.3% 129054 ± 4% softirqs.CPU180.TIMER 133615 ± 3% +14.4% 152876 ± 3% softirqs.CPU181.RCU 115505 ± 4% +11.3% 128560 ± 3% softirqs.CPU181.TIMER 133171 ± 3% +14.7% 152810 ± 3% softirqs.CPU182.RCU 115955 ± 4% +11.0% 128687 ± 3% softirqs.CPU182.TIMER 133413 ± 3% +14.4% 152671 ± 3% softirqs.CPU183.RCU 116313 ± 4% +11.0% 129140 ± 3% softirqs.CPU183.TIMER 133598 ± 3% +14.8% 153388 ± 3% softirqs.CPU184.RCU 115462 ± 3% +13.0% 130430 ± 6% softirqs.CPU184.TIMER 133884 ± 3% +13.9% 152525 ± 3% softirqs.CPU185.RCU 115598 ± 4% +11.1% 128397 ± 4% softirqs.CPU185.TIMER 133262 ± 3% +14.5% 152646 ± 3% softirqs.CPU186.RCU 115745 ± 3% +11.1% 128612 ± 4% softirqs.CPU186.TIMER 133430 ± 3% +14.5% 152767 ± 3% softirqs.CPU187.RCU 115769 ± 4% +11.0% 128464 ± 3% softirqs.CPU187.TIMER 133259 ± 3% +14.8% 153036 ± 3% softirqs.CPU188.RCU 116108 ± 4% +11.3% 129232 ± 3% softirqs.CPU188.TIMER 133609 ± 3% +14.1% 152436 ± 3% softirqs.CPU189.RCU 115984 ± 4% +11.0% 128710 ± 3% softirqs.CPU189.TIMER 135090 ± 3% +13.9% 153802 ± 3% softirqs.CPU19.RCU 110865 ± 2% +8.2% 119936 ± 3% softirqs.CPU19.TIMER 133685 ± 3% +14.2% 152631 ± 3% softirqs.CPU190.RCU 115642 ± 3% +11.2% 128581 ± 4% softirqs.CPU190.TIMER 131031 ± 3% +14.6% 150126 ± 3% softirqs.CPU191.RCU 114275 ± 4% +11.4% 127249 ± 4% softirqs.CPU191.TIMER 134928 ± 3% +14.3% 154165 ± 3% softirqs.CPU2.RCU 135126 ± 3% +14.0% 154083 ± 3% softirqs.CPU20.RCU 111013 ± 3% +8.6% 120549 ± 3% softirqs.CPU20.TIMER 134663 ± 3% +14.5% 154251 ± 3% softirqs.CPU21.RCU 109671 ± 2% +8.2% 118708 ± 3% softirqs.CPU21.TIMER 135047 ± 3% +13.9% 153766 ± 3% softirqs.CPU22.RCU 134437 ± 3% +14.2% 153492 ± 3% softirqs.CPU23.RCU 109675 ± 2% +8.3% 118824 ± 3% softirqs.CPU23.TIMER 136114 ± 3% +13.3% 154277 ± 3% softirqs.CPU24.RCU 134825 ± 3% +14.2% 154007 ± 3% softirqs.CPU25.RCU 139234 ± 4% +13.8% 158515 ± 3% softirqs.CPU26.RCU 135056 ± 3% +14.1% 154062 ± 3% softirqs.CPU27.RCU 135307 ± 3% +13.7% 153897 ± 3% softirqs.CPU28.RCU 134686 ± 3% +14.7% 154497 ± 3% softirqs.CPU29.RCU 134688 ± 3% +14.4% 154115 ± 2% softirqs.CPU3.RCU 134642 ± 3% +14.5% 154226 ± 3% softirqs.CPU30.RCU 134773 ± 3% +14.2% 153887 ± 3% softirqs.CPU31.RCU 134523 ± 3% +14.3% 153733 ± 3% softirqs.CPU32.RCU 134488 ± 3% +14.4% 153801 ± 3% softirqs.CPU33.RCU 134483 ± 3% +14.1% 153380 ± 3% softirqs.CPU34.RCU 134886 ± 3% +13.6% 153180 ± 3% softirqs.CPU35.RCU 135021 ± 3% +13.6% 153340 ± 3% softirqs.CPU36.RCU 134064 ± 3% +15.1% 154277 ± 3% softirqs.CPU37.RCU 134166 ± 3% +14.8% 153967 ± 3% softirqs.CPU38.RCU 134488 ± 3% +14.4% 153871 ± 3% softirqs.CPU39.RCU 135294 ± 2% +14.6% 155025 ± 3% softirqs.CPU4.RCU 134598 ± 4% +14.3% 153848 ± 3% softirqs.CPU40.RCU 134255 ± 3% +14.8% 154099 ± 3% softirqs.CPU41.RCU 134334 ± 3% +14.3% 153491 ± 3% softirqs.CPU42.RCU 134201 ± 3% +14.5% 153654 ± 3% softirqs.CPU43.RCU 134260 ± 3% +14.2% 153359 ± 3% softirqs.CPU44.RCU 134005 ± 3% +14.4% 153367 ± 3% softirqs.CPU45.RCU 134236 ± 3% +14.3% 153419 ± 3% softirqs.CPU46.RCU 134552 ± 3% +13.8% 153056 ± 3% softirqs.CPU47.RCU 134623 ± 3% +14.4% 154028 ± 3% softirqs.CPU48.RCU 134348 ± 3% +14.7% 154080 ± 3% softirqs.CPU49.RCU 109020 ± 3% +18.6% 129279 ± 14% softirqs.CPU49.TIMER 134334 ± 3% +15.0% 154455 ± 3% softirqs.CPU5.RCU 110260 ± 2% +8.8% 119951 ± 4% softirqs.CPU5.TIMER 134377 ± 3% +13.8% 152954 ± 3% softirqs.CPU50.RCU 109169 ± 3% +15.0% 125548 ± 9% softirqs.CPU50.TIMER 134780 ± 3% +14.2% 153967 ± 3% softirqs.CPU51.RCU 109951 ± 3% +12.5% 123641 ± 5% softirqs.CPU51.TIMER 133929 ± 3% +15.0% 154012 ± 3% softirqs.CPU52.RCU 108665 ± 3% +12.8% 122613 ± 5% softirqs.CPU52.TIMER 133761 ± 3% +14.9% 153675 ± 3% softirqs.CPU53.RCU 108606 ± 3% +19.6% 129861 ± 16% softirqs.CPU53.TIMER 134369 ± 3% +14.1% 153255 ± 3% softirqs.CPU54.RCU 108917 ± 3% +12.5% 122585 ± 5% softirqs.CPU54.TIMER 133810 ± 3% +14.6% 153282 ± 3% softirqs.CPU55.RCU 108875 ± 3% +12.5% 122468 ± 5% softirqs.CPU55.TIMER 134038 ± 3% +14.3% 153248 ± 3% softirqs.CPU56.RCU 108569 ± 3% +12.8% 122502 ± 5% softirqs.CPU56.TIMER 133993 ± 4% +14.5% 153426 ± 3% softirqs.CPU57.RCU 108465 ± 3% +12.8% 122399 ± 5% softirqs.CPU57.TIMER 134049 ± 3% +14.7% 153708 ± 3% softirqs.CPU58.RCU 108634 ± 4% +18.8% 129107 ± 16% softirqs.CPU58.TIMER 134025 ± 3% +14.9% 154015 ± 3% softirqs.CPU59.RCU 108501 ± 3% +18.6% 128721 ± 13% softirqs.CPU59.TIMER 135247 ± 3% +14.5% 154804 ± 3% softirqs.CPU6.RCU 133745 ± 4% +14.7% 153452 ± 3% softirqs.CPU60.RCU 108526 ± 3% +13.0% 122688 ± 5% softirqs.CPU60.TIMER 133931 ± 3% +14.6% 153549 ± 3% softirqs.CPU61.RCU 108764 ± 3% +12.9% 122787 ± 5% softirqs.CPU61.TIMER 134682 ± 3% +13.8% 153266 ± 3% softirqs.CPU62.RCU 109188 ± 3% +12.5% 122806 ± 5% softirqs.CPU62.TIMER 134190 ± 3% +14.6% 153761 ± 2% softirqs.CPU63.RCU 108489 ± 3% +12.8% 122426 ± 5% softirqs.CPU63.TIMER 134280 ± 3% +14.3% 153538 ± 3% softirqs.CPU64.RCU 108493 ± 3% +13.3% 122953 ± 5% softirqs.CPU64.TIMER 134074 ± 3% +14.7% 153765 ± 3% softirqs.CPU65.RCU 108331 ± 3% +13.0% 122440 ± 5% softirqs.CPU65.TIMER 134424 ± 4% +14.3% 153692 ± 3% softirqs.CPU66.RCU 108874 ± 3% +12.5% 122490 ± 5% softirqs.CPU66.TIMER 133801 ± 3% +14.9% 153746 ± 3% softirqs.CPU67.RCU 108903 ± 3% +13.2% 123240 ± 5% softirqs.CPU67.TIMER 134085 ± 3% +14.4% 153371 ± 3% softirqs.CPU68.RCU 109310 ± 3% +12.3% 122810 ± 5% softirqs.CPU68.TIMER 134091 ± 3% +14.5% 153515 ± 3% softirqs.CPU69.RCU 108193 ± 3% +12.5% 121716 ± 5% softirqs.CPU69.TIMER 134883 ± 3% +13.6% 153181 ± 3% softirqs.CPU7.RCU 134392 ± 3% +14.8% 154240 ± 3% softirqs.CPU70.RCU 108056 ± 3% +13.4% 122578 ± 5% softirqs.CPU70.TIMER 133901 ± 3% +14.2% 152873 ± 3% softirqs.CPU71.RCU 134273 ± 3% +14.4% 153630 ± 3% softirqs.CPU72.RCU 116189 ± 3% +11.5% 129495 ± 3% softirqs.CPU72.TIMER 134231 ± 3% +14.5% 153682 ± 3% softirqs.CPU73.RCU 116348 ± 3% +11.3% 129453 ± 3% softirqs.CPU73.TIMER 134235 ± 3% +14.1% 153158 ± 3% softirqs.CPU74.RCU 133896 ± 3% +14.6% 153477 ± 3% softirqs.CPU75.RCU 116933 ± 4% +11.2% 129980 ± 3% softirqs.CPU75.TIMER 134014 ± 3% +14.3% 153198 ± 3% softirqs.CPU76.RCU 116889 ± 3% +11.0% 129768 ± 3% softirqs.CPU76.TIMER 134608 ± 3% +13.7% 152994 ± 3% softirqs.CPU77.RCU 117342 ± 3% +11.0% 130274 ± 3% softirqs.CPU77.TIMER 134551 ± 3% +13.9% 153253 ± 3% softirqs.CPU78.RCU 117229 ± 4% +10.5% 129549 ± 4% softirqs.CPU78.TIMER 134118 ± 3% +14.3% 153325 ± 3% softirqs.CPU79.RCU 116982 ± 3% +11.0% 129802 ± 3% softirqs.CPU79.TIMER 135303 ± 3% +13.9% 154129 ± 3% softirqs.CPU8.RCU 134699 ± 3% +13.6% 153047 ± 3% softirqs.CPU80.RCU 117058 ± 4% +10.6% 129470 ± 3% softirqs.CPU80.TIMER 134088 ± 3% +15.6% 155063 ± 3% softirqs.CPU81.RCU 116539 ± 4% +12.3% 130886 ± 5% softirqs.CPU81.TIMER 133989 ± 3% +14.6% 153526 ± 3% softirqs.CPU82.RCU 116614 ± 4% +16.8% 136238 ± 12% softirqs.CPU82.TIMER 133966 ± 3% +14.3% 153142 ± 3% softirqs.CPU83.RCU 117140 ± 3% +11.2% 130205 ± 4% softirqs.CPU83.TIMER 133561 ± 3% +15.0% 153661 ± 3% softirqs.CPU84.RCU 116648 ± 4% +11.3% 129844 ± 4% softirqs.CPU84.TIMER 134217 ± 3% +14.2% 153330 ± 3% softirqs.CPU85.RCU 116247 ± 4% +11.3% 129336 ± 3% softirqs.CPU85.TIMER 133956 ± 3% +14.5% 153338 ± 3% softirqs.CPU86.RCU 116654 ± 4% +11.1% 129560 ± 3% softirqs.CPU86.TIMER 133834 ± 3% +15.1% 154049 ± 2% softirqs.CPU87.RCU 116898 ± 4% +12.0% 130912 ± 3% softirqs.CPU87.TIMER 133951 ± 3% +15.1% 154211 ± 3% softirqs.CPU88.RCU 116139 ± 3% +16.0% 134676 ± 13% softirqs.CPU88.TIMER 134288 ± 3% +14.3% 153433 ± 2% softirqs.CPU89.RCU 116227 ± 3% +11.0% 129064 ± 3% softirqs.CPU89.TIMER 135289 ± 3% +13.6% 153744 ± 3% softirqs.CPU9.RCU 133598 ± 3% +14.5% 153019 ± 3% softirqs.CPU90.RCU 116395 ± 3% +10.9% 129084 ± 4% softirqs.CPU90.TIMER 133854 ± 4% +14.9% 153799 ± 3% softirqs.CPU91.RCU 116562 ± 4% +11.1% 129452 ± 3% softirqs.CPU91.TIMER 134107 ± 3% +14.1% 152958 ± 3% softirqs.CPU92.RCU 117107 ± 4% +10.6% 129559 ± 3% softirqs.CPU92.TIMER 133800 ± 3% +14.7% 153465 ± 3% softirqs.CPU93.RCU 116415 ± 3% +11.5% 129823 ± 4% softirqs.CPU93.TIMER 134092 ± 3% +14.0% 152835 ± 3% softirqs.CPU94.RCU 116063 ± 3% +11.4% 129252 ± 4% softirqs.CPU94.TIMER 134284 ± 3% +14.1% 153230 ± 4% softirqs.CPU95.RCU 117908 ± 3% +11.0% 130904 ± 4% softirqs.CPU95.TIMER 134264 ± 3% +14.3% 153455 ± 3% softirqs.CPU96.RCU 109819 ± 3% +8.6% 119258 ± 3% softirqs.CPU96.TIMER 135000 ± 3% +13.7% 153529 ± 3% softirqs.CPU97.RCU 134163 ± 3% +14.2% 153253 ± 3% softirqs.CPU98.RCU 109990 ± 3% +8.1% 118896 ± 3% softirqs.CPU98.TIMER 134966 ± 3% +13.5% 153245 ± 3% softirqs.CPU99.RCU 25776297 ± 3% +14.3% 29452390 ± 3% softirqs.RCU 21445010 ± 3% +9.5% 23488096 ± 3% softirqs.TIMER 614.00 ± 2% +9.9% 674.50 ± 2% interrupts.9:IO-APIC.9-fasteoi.acpi 611854 ± 2% +10.1% 673945 ± 2% interrupts.CPU0.LOC:Local_timer_interrupts 614.00 ± 2% +9.9% 674.50 ± 2% interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 611835 ± 2% +10.1% 673883 ± 2% interrupts.CPU1.LOC:Local_timer_interrupts 611859 ± 2% +10.2% 673975 ± 2% interrupts.CPU10.LOC:Local_timer_interrupts 7508 ± 5% +15.5% 8675 ± 4% interrupts.CPU10.RES:Rescheduling_interrupts 611802 ± 2% +10.2% 673904 ± 2% interrupts.CPU100.LOC:Local_timer_interrupts 7323 ± 5% +16.1% 8503 ± 3% interrupts.CPU100.RES:Rescheduling_interrupts 611804 ± 2% +10.2% 673917 ± 2% interrupts.CPU101.LOC:Local_timer_interrupts 7342 ± 3% +13.8% 8352 ± 3% interrupts.CPU101.RES:Rescheduling_interrupts 611780 ± 2% +10.2% 673877 ± 2% interrupts.CPU102.LOC:Local_timer_interrupts 7506 ± 5% +12.2% 8424 ± 4% interrupts.CPU102.RES:Rescheduling_interrupts 611734 ± 2% +10.2% 673851 ± 2% interrupts.CPU103.LOC:Local_timer_interrupts 7436 ± 3% +15.8% 8607 ± 4% interrupts.CPU103.RES:Rescheduling_interrupts 611800 ± 2% +10.2% 673908 ± 2% interrupts.CPU104.LOC:Local_timer_interrupts 7519 ± 5% +13.1% 8502 ± 2% interrupts.CPU104.RES:Rescheduling_interrupts 611790 ± 2% +10.1% 673864 ± 2% interrupts.CPU105.LOC:Local_timer_interrupts 7346 ± 5% +16.4% 8553 ± 2% interrupts.CPU105.RES:Rescheduling_interrupts 611801 ± 2% +10.1% 673859 ± 2% interrupts.CPU106.LOC:Local_timer_interrupts 7352 ± 5% +16.5% 8567 ± 2% interrupts.CPU106.RES:Rescheduling_interrupts 611797 ± 2% +10.1% 673890 ± 2% interrupts.CPU107.LOC:Local_timer_interrupts 7238 ± 6% +15.3% 8344 ± 4% interrupts.CPU107.RES:Rescheduling_interrupts 611778 ± 2% +10.2% 673882 ± 2% interrupts.CPU108.LOC:Local_timer_interrupts 7310 ± 5% +16.3% 8503 ± 5% interrupts.CPU108.RES:Rescheduling_interrupts 611788 ± 2% +10.1% 673881 ± 2% interrupts.CPU109.LOC:Local_timer_interrupts 7324 ± 6% +13.4% 8307 interrupts.CPU109.RES:Rescheduling_interrupts 611827 ± 2% +10.2% 673960 ± 2% interrupts.CPU11.LOC:Local_timer_interrupts 7598 ± 6% +12.6% 8557 ± 5% interrupts.CPU11.RES:Rescheduling_interrupts 611796 ± 2% +10.1% 673865 ± 2% interrupts.CPU110.LOC:Local_timer_interrupts 7326 ± 4% +16.2% 8513 ± 4% interrupts.CPU110.RES:Rescheduling_interrupts 611804 ± 2% +10.1% 673890 ± 2% interrupts.CPU111.LOC:Local_timer_interrupts 7312 ± 3% +14.4% 8365 ± 3% interrupts.CPU111.RES:Rescheduling_interrupts 611801 ± 2% +10.1% 673890 ± 2% interrupts.CPU112.LOC:Local_timer_interrupts 7277 ± 2% +14.6% 8337 ± 5% interrupts.CPU112.RES:Rescheduling_interrupts 611789 ± 2% +10.2% 673896 ± 2% interrupts.CPU113.LOC:Local_timer_interrupts 7346 ± 4% +15.2% 8461 ± 4% interrupts.CPU113.RES:Rescheduling_interrupts 611759 ± 2% +10.2% 673861 ± 2% interrupts.CPU114.LOC:Local_timer_interrupts 7290 ± 4% +17.9% 8594 ± 4% interrupts.CPU114.RES:Rescheduling_interrupts 611783 ± 2% +10.2% 673883 ± 2% interrupts.CPU115.LOC:Local_timer_interrupts 7470 ± 4% +13.6% 8489 ± 5% interrupts.CPU115.RES:Rescheduling_interrupts 611807 ± 2% +10.2% 673925 ± 2% interrupts.CPU116.LOC:Local_timer_interrupts 7336 ± 4% +15.3% 8455 ± 2% interrupts.CPU116.RES:Rescheduling_interrupts 611778 ± 2% +10.2% 673890 ± 2% interrupts.CPU117.LOC:Local_timer_interrupts 7326 ± 5% +15.9% 8493 ± 3% interrupts.CPU117.RES:Rescheduling_interrupts 611782 ± 2% +10.1% 673849 ± 2% interrupts.CPU118.LOC:Local_timer_interrupts 7239 ± 4% +18.4% 8574 ± 5% interrupts.CPU118.RES:Rescheduling_interrupts 611778 ± 2% +10.2% 673881 ± 2% interrupts.CPU119.LOC:Local_timer_interrupts 7327 ± 4% +13.7% 8335 ± 6% interrupts.CPU119.RES:Rescheduling_interrupts 611853 ± 2% +10.1% 673944 ± 2% interrupts.CPU12.LOC:Local_timer_interrupts 7516 ± 4% +17.0% 8798 ± 6% interrupts.CPU12.RES:Rescheduling_interrupts 611417 ± 2% +10.2% 673606 ± 2% interrupts.CPU120.LOC:Local_timer_interrupts 7392 ± 4% +14.2% 8444 ± 5% interrupts.CPU120.RES:Rescheduling_interrupts 611457 ± 2% +10.2% 673608 ± 2% interrupts.CPU121.LOC:Local_timer_interrupts 7376 ± 3% +15.6% 8525 ± 4% interrupts.CPU121.RES:Rescheduling_interrupts 611426 ± 2% +10.2% 673610 ± 2% interrupts.CPU122.LOC:Local_timer_interrupts 7466 ± 5% +11.7% 8336 ± 4% interrupts.CPU122.RES:Rescheduling_interrupts 611384 ± 2% +10.2% 673577 ± 2% interrupts.CPU123.LOC:Local_timer_interrupts 7240 ± 5% +18.2% 8559 ± 3% interrupts.CPU123.RES:Rescheduling_interrupts 611388 ± 2% +10.2% 673597 ± 2% interrupts.CPU124.LOC:Local_timer_interrupts 7245 ± 5% +18.0% 8548 ± 5% interrupts.CPU124.RES:Rescheduling_interrupts 611391 ± 2% +10.2% 673567 ± 2% interrupts.CPU125.LOC:Local_timer_interrupts 7318 ± 3% +15.3% 8436 ± 5% interrupts.CPU125.RES:Rescheduling_interrupts 611421 ± 2% +10.2% 673570 ± 2% interrupts.CPU126.LOC:Local_timer_interrupts 7258 ± 5% +16.4% 8448 ± 4% interrupts.CPU126.RES:Rescheduling_interrupts 611468 ± 2% +10.2% 673614 ± 2% interrupts.CPU127.LOC:Local_timer_interrupts 7276 ± 5% +16.9% 8508 ± 5% interrupts.CPU127.RES:Rescheduling_interrupts 611422 ± 2% +10.2% 673591 ± 2% interrupts.CPU128.LOC:Local_timer_interrupts 7278 ± 4% +17.0% 8518 ± 3% interrupts.CPU128.RES:Rescheduling_interrupts 611407 ± 2% +10.2% 673562 ± 2% interrupts.CPU129.LOC:Local_timer_interrupts 7284 ± 3% +17.0% 8519 ± 5% interrupts.CPU129.RES:Rescheduling_interrupts 611825 ± 2% +10.1% 673920 ± 2% interrupts.CPU13.LOC:Local_timer_interrupts 7550 ± 4% +16.2% 8771 ± 3% interrupts.CPU13.RES:Rescheduling_interrupts 611428 ± 2% +10.2% 673571 ± 2% interrupts.CPU130.LOC:Local_timer_interrupts 7337 ± 5% +16.4% 8541 ± 5% interrupts.CPU130.RES:Rescheduling_interrupts 611406 ± 2% +10.2% 673582 ± 2% interrupts.CPU131.LOC:Local_timer_interrupts 7205 ± 5% +17.1% 8440 ± 6% interrupts.CPU131.RES:Rescheduling_interrupts 611449 ± 2% +10.2% 673586 ± 2% interrupts.CPU132.LOC:Local_timer_interrupts 7431 ± 4% +12.7% 8372 ± 3% interrupts.CPU132.RES:Rescheduling_interrupts 611464 ± 2% +10.2% 673588 ± 2% interrupts.CPU133.LOC:Local_timer_interrupts 7289 ± 4% +15.9% 8446 ± 3% interrupts.CPU133.RES:Rescheduling_interrupts 611437 ± 2% +10.2% 673583 ± 2% interrupts.CPU134.LOC:Local_timer_interrupts 7321 ± 4% +17.7% 8616 ± 4% interrupts.CPU134.RES:Rescheduling_interrupts 611401 ± 2% +10.2% 673595 ± 2% interrupts.CPU135.LOC:Local_timer_interrupts 7300 ± 4% +15.7% 8446 ± 4% interrupts.CPU135.RES:Rescheduling_interrupts 611462 ± 2% +10.2% 673602 ± 2% interrupts.CPU136.LOC:Local_timer_interrupts 7389 ± 4% +13.9% 8417 ± 5% interrupts.CPU136.RES:Rescheduling_interrupts 611424 ± 2% +10.2% 673588 ± 2% interrupts.CPU137.LOC:Local_timer_interrupts 7126 ± 4% +20.0% 8550 ± 5% interrupts.CPU137.RES:Rescheduling_interrupts 611383 ± 2% +10.2% 673556 ± 2% interrupts.CPU138.LOC:Local_timer_interrupts 7297 ± 5% +16.1% 8475 ± 5% interrupts.CPU138.RES:Rescheduling_interrupts 611422 ± 2% +10.2% 673603 ± 2% interrupts.CPU139.LOC:Local_timer_interrupts 7160 ± 4% +17.6% 8423 ± 4% interrupts.CPU139.RES:Rescheduling_interrupts 611815 ± 2% +10.1% 673901 ± 2% interrupts.CPU14.LOC:Local_timer_interrupts 7576 ± 5% +16.3% 8810 ± 3% interrupts.CPU14.RES:Rescheduling_interrupts 611388 ± 2% +10.2% 673577 ± 2% interrupts.CPU140.LOC:Local_timer_interrupts 7342 ± 6% +12.9% 8293 ± 4% interrupts.CPU140.RES:Rescheduling_interrupts 611394 ± 2% +10.2% 673596 ± 2% interrupts.CPU141.LOC:Local_timer_interrupts 7257 ± 4% +16.1% 8423 ± 5% interrupts.CPU141.RES:Rescheduling_interrupts 611411 ± 2% +10.2% 673590 ± 2% interrupts.CPU142.LOC:Local_timer_interrupts 7300 ± 6% +16.1% 8475 ± 3% interrupts.CPU142.RES:Rescheduling_interrupts 611396 ± 2% +10.2% 673622 ± 2% interrupts.CPU143.LOC:Local_timer_interrupts 7399 ± 5% +14.0% 8435 ± 4% interrupts.CPU143.RES:Rescheduling_interrupts 610929 ± 2% +10.3% 673686 ± 2% interrupts.CPU144.LOC:Local_timer_interrupts 7281 ± 5% +13.4% 8257 ± 5% interrupts.CPU144.RES:Rescheduling_interrupts 610918 ± 2% +10.3% 673755 ± 2% interrupts.CPU145.LOC:Local_timer_interrupts 7343 ± 5% +14.5% 8408 ± 4% interrupts.CPU145.RES:Rescheduling_interrupts 610910 ± 2% +10.3% 673705 ± 2% interrupts.CPU146.LOC:Local_timer_interrupts 7524 ± 4% +13.6% 8550 ± 6% interrupts.CPU146.RES:Rescheduling_interrupts 610858 ± 2% +10.3% 673769 ± 2% interrupts.CPU147.LOC:Local_timer_interrupts 7433 ± 4% +10.8% 8237 ± 3% interrupts.CPU147.RES:Rescheduling_interrupts 610872 ± 2% +10.3% 673727 ± 2% interrupts.CPU148.LOC:Local_timer_interrupts 7354 ± 3% +13.1% 8320 ± 4% interrupts.CPU148.RES:Rescheduling_interrupts 610909 ± 2% +10.3% 673750 ± 2% interrupts.CPU149.LOC:Local_timer_interrupts 7365 ± 3% +16.5% 8582 ± 3% interrupts.CPU149.RES:Rescheduling_interrupts 611818 ± 2% +10.2% 673937 ± 2% interrupts.CPU15.LOC:Local_timer_interrupts 7632 ± 4% +12.5% 8587 ± 4% interrupts.CPU15.RES:Rescheduling_interrupts 610898 ± 2% +10.3% 673770 ± 2% interrupts.CPU150.LOC:Local_timer_interrupts 7369 ± 4% +12.4% 8282 ± 3% interrupts.CPU150.RES:Rescheduling_interrupts 610887 ± 2% +10.3% 673754 ± 2% interrupts.CPU151.LOC:Local_timer_interrupts 7322 ± 4% +15.3% 8445 ± 3% interrupts.CPU151.RES:Rescheduling_interrupts 610849 ± 2% +10.3% 673762 ± 2% interrupts.CPU152.LOC:Local_timer_interrupts 7305 ± 4% +15.5% 8439 ± 5% interrupts.CPU152.RES:Rescheduling_interrupts 610909 ± 2% +10.3% 673759 ± 2% interrupts.CPU153.LOC:Local_timer_interrupts 7363 ± 6% +14.5% 8432 ± 2% interrupts.CPU153.RES:Rescheduling_interrupts 610878 ± 2% +10.3% 673730 ± 2% interrupts.CPU154.LOC:Local_timer_interrupts 7373 ± 4% +14.1% 8414 ± 4% interrupts.CPU154.RES:Rescheduling_interrupts 610891 ± 2% +10.3% 673720 ± 2% interrupts.CPU155.LOC:Local_timer_interrupts 7385 ± 5% +11.8% 8260 ± 4% interrupts.CPU155.RES:Rescheduling_interrupts 610909 ± 2% +10.3% 673753 ± 2% interrupts.CPU156.LOC:Local_timer_interrupts 7377 ± 5% +14.0% 8410 ± 5% interrupts.CPU156.RES:Rescheduling_interrupts 610886 ± 2% +10.3% 673754 ± 2% interrupts.CPU157.LOC:Local_timer_interrupts 7399 ± 5% +13.1% 8370 ± 4% interrupts.CPU157.RES:Rescheduling_interrupts 610854 ± 2% +10.3% 673699 ± 2% interrupts.CPU158.LOC:Local_timer_interrupts 7489 ± 4% +10.5% 8278 ± 2% interrupts.CPU158.RES:Rescheduling_interrupts 610921 ± 2% +10.3% 673726 ± 2% interrupts.CPU159.LOC:Local_timer_interrupts 7281 ± 5% +15.4% 8402 ± 4% interrupts.CPU159.RES:Rescheduling_interrupts 611836 ± 2% +10.1% 673889 ± 2% interrupts.CPU16.LOC:Local_timer_interrupts 7548 ± 4% +15.6% 8723 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts 610870 ± 2% +10.3% 673742 ± 2% interrupts.CPU160.LOC:Local_timer_interrupts 610856 ± 2% +10.3% 673722 ± 2% interrupts.CPU161.LOC:Local_timer_interrupts 7472 ± 5% +13.4% 8475 ± 5% interrupts.CPU161.RES:Rescheduling_interrupts 610891 ± 2% +10.3% 673708 ± 2% interrupts.CPU162.LOC:Local_timer_interrupts 7177 ± 5% +14.8% 8239 ± 5% interrupts.CPU162.RES:Rescheduling_interrupts 610864 ± 2% +10.3% 673696 ± 2% interrupts.CPU163.LOC:Local_timer_interrupts 7337 ± 5% +12.7% 8266 ± 4% interrupts.CPU163.RES:Rescheduling_interrupts 610858 ± 2% +10.3% 673724 ± 2% interrupts.CPU164.LOC:Local_timer_interrupts 7269 ± 5% +15.2% 8376 ± 4% interrupts.CPU164.RES:Rescheduling_interrupts 610861 ± 2% +10.3% 673711 ± 2% interrupts.CPU165.LOC:Local_timer_interrupts 7431 ± 4% +9.4% 8131 ± 4% interrupts.CPU165.RES:Rescheduling_interrupts 610867 ± 2% +10.3% 673684 ± 2% interrupts.CPU166.LOC:Local_timer_interrupts 7339 ± 2% +14.1% 8374 ± 4% interrupts.CPU166.RES:Rescheduling_interrupts 610913 ± 2% +10.3% 673730 ± 2% interrupts.CPU167.LOC:Local_timer_interrupts 7174 ± 3% +17.5% 8427 ± 4% interrupts.CPU167.RES:Rescheduling_interrupts 611289 ± 2% +10.2% 673888 ± 2% interrupts.CPU168.LOC:Local_timer_interrupts 7311 ± 5% +14.7% 8382 ± 3% interrupts.CPU168.RES:Rescheduling_interrupts 611333 ± 2% +10.2% 673887 ± 2% interrupts.CPU169.LOC:Local_timer_interrupts 7232 ± 2% +17.8% 8517 ± 5% interrupts.CPU169.RES:Rescheduling_interrupts 611838 ± 2% +10.1% 673879 ± 2% interrupts.CPU17.LOC:Local_timer_interrupts 7539 ± 2% +17.7% 8873 ± 10% interrupts.CPU17.RES:Rescheduling_interrupts 611349 ± 2% +10.2% 673889 ± 2% interrupts.CPU170.LOC:Local_timer_interrupts 7374 ± 2% +14.2% 8424 ± 5% interrupts.CPU170.RES:Rescheduling_interrupts 611297 ± 2% +10.2% 673873 ± 2% interrupts.CPU171.LOC:Local_timer_interrupts 7438 ± 4% +13.9% 8471 ± 5% interrupts.CPU171.RES:Rescheduling_interrupts 611304 ± 2% +10.2% 673845 ± 2% interrupts.CPU172.LOC:Local_timer_interrupts 7317 ± 3% +16.6% 8529 ± 5% interrupts.CPU172.RES:Rescheduling_interrupts 611297 ± 2% +10.2% 673887 ± 2% interrupts.CPU173.LOC:Local_timer_interrupts 611309 ± 2% +10.2% 673858 ± 2% interrupts.CPU174.LOC:Local_timer_interrupts 7369 ± 5% +15.8% 8531 ± 5% interrupts.CPU174.RES:Rescheduling_interrupts 611311 ± 2% +10.2% 673872 ± 2% interrupts.CPU175.LOC:Local_timer_interrupts 7352 ± 4% +15.1% 8465 ± 4% interrupts.CPU175.RES:Rescheduling_interrupts 611309 ± 2% +10.2% 673886 ± 2% interrupts.CPU176.LOC:Local_timer_interrupts 7395 ± 2% +14.0% 8434 ± 5% interrupts.CPU176.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673868 ± 2% interrupts.CPU177.LOC:Local_timer_interrupts 611248 ± 2% +10.2% 673805 ± 2% interrupts.CPU178.LOC:Local_timer_interrupts 7355 ± 5% +14.1% 8393 ± 4% interrupts.CPU178.RES:Rescheduling_interrupts 611319 ± 2% +10.2% 673906 ± 2% interrupts.CPU179.LOC:Local_timer_interrupts 7269 ± 4% +15.2% 8374 ± 6% interrupts.CPU179.RES:Rescheduling_interrupts 611799 ± 2% +10.1% 673864 ± 2% interrupts.CPU18.LOC:Local_timer_interrupts 7585 ± 4% +14.5% 8683 ± 4% interrupts.CPU18.RES:Rescheduling_interrupts 611330 ± 2% +10.2% 673869 ± 2% interrupts.CPU180.LOC:Local_timer_interrupts 7404 ± 4% +14.0% 8442 ± 5% interrupts.CPU180.RES:Rescheduling_interrupts 611324 ± 2% +10.2% 673889 ± 2% interrupts.CPU181.LOC:Local_timer_interrupts 7435 ± 2% +13.0% 8399 ± 3% interrupts.CPU181.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673874 ± 2% interrupts.CPU182.LOC:Local_timer_interrupts 7250 ± 4% +14.6% 8305 ± 4% interrupts.CPU182.RES:Rescheduling_interrupts 611307 ± 2% +10.2% 673852 ± 2% interrupts.CPU183.LOC:Local_timer_interrupts 7320 ± 4% +13.2% 8290 ± 5% interrupts.CPU183.RES:Rescheduling_interrupts 611329 ± 2% +10.2% 673877 ± 2% interrupts.CPU184.LOC:Local_timer_interrupts 7377 ± 4% +12.5% 8299 ± 5% interrupts.CPU184.RES:Rescheduling_interrupts 611312 ± 2% +10.2% 673866 ± 2% interrupts.CPU185.LOC:Local_timer_interrupts 7333 ± 4% +13.9% 8351 ± 5% interrupts.CPU185.RES:Rescheduling_interrupts 611290 ± 2% +10.2% 673868 ± 2% interrupts.CPU186.LOC:Local_timer_interrupts 7426 ± 4% +14.1% 8471 ± 3% interrupts.CPU186.RES:Rescheduling_interrupts 611304 ± 2% +10.2% 673853 ± 2% interrupts.CPU187.LOC:Local_timer_interrupts 7380 ± 3% +12.5% 8304 ± 5% interrupts.CPU187.RES:Rescheduling_interrupts 611307 ± 2% +10.2% 673881 ± 2% interrupts.CPU188.LOC:Local_timer_interrupts 7277 ± 5% +16.3% 8467 ± 6% interrupts.CPU188.RES:Rescheduling_interrupts 611299 ± 2% +10.2% 673879 ± 2% interrupts.CPU189.LOC:Local_timer_interrupts 7436 ± 3% +14.5% 8514 ± 6% interrupts.CPU189.RES:Rescheduling_interrupts 611761 ± 2% +10.2% 673928 ± 2% interrupts.CPU19.LOC:Local_timer_interrupts 7515 ± 6% +12.7% 8467 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts 611321 ± 2% +10.2% 673856 ± 2% interrupts.CPU190.LOC:Local_timer_interrupts 7188 ± 3% +16.4% 8366 ± 5% interrupts.CPU190.RES:Rescheduling_interrupts 611544 ± 2% +10.2% 674120 ± 2% interrupts.CPU191.LOC:Local_timer_interrupts 7207 ± 3% +15.3% 8312 ± 3% interrupts.CPU191.RES:Rescheduling_interrupts 611805 ± 2% +10.2% 673920 ± 2% interrupts.CPU2.LOC:Local_timer_interrupts 611781 ± 2% +10.2% 673897 ± 2% interrupts.CPU20.LOC:Local_timer_interrupts 7531 ± 5% +16.4% 8768 ± 3% interrupts.CPU20.RES:Rescheduling_interrupts 611715 ± 2% +10.2% 673897 ± 2% interrupts.CPU21.LOC:Local_timer_interrupts 7467 ± 3% +14.9% 8577 ± 4% interrupts.CPU21.RES:Rescheduling_interrupts 611834 ± 2% +10.1% 673900 ± 2% interrupts.CPU22.LOC:Local_timer_interrupts 7427 ± 6% +17.9% 8754 ± 4% interrupts.CPU22.RES:Rescheduling_interrupts 611788 ± 2% +10.1% 673869 ± 2% interrupts.CPU23.LOC:Local_timer_interrupts 7592 ± 4% +12.2% 8518 ± 4% interrupts.CPU23.RES:Rescheduling_interrupts 611426 ± 2% +10.2% 673619 ± 2% interrupts.CPU24.LOC:Local_timer_interrupts 7457 ± 4% +14.7% 8551 ± 3% interrupts.CPU24.RES:Rescheduling_interrupts 611439 ± 2% +10.2% 673636 ± 2% interrupts.CPU25.LOC:Local_timer_interrupts 611402 ± 2% +10.2% 673620 ± 2% interrupts.CPU26.LOC:Local_timer_interrupts 7463 ± 3% +18.4% 8838 ± 7% interrupts.CPU26.RES:Rescheduling_interrupts 611413 ± 2% +10.2% 673615 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts 7412 ± 4% +16.1% 8607 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts 611409 ± 2% +10.2% 673625 ± 2% interrupts.CPU28.LOC:Local_timer_interrupts 7478 ± 5% +15.6% 8648 ± 3% interrupts.CPU28.RES:Rescheduling_interrupts 611415 ± 2% +10.2% 673620 ± 2% interrupts.CPU29.LOC:Local_timer_interrupts 7431 ± 2% +15.5% 8585 ± 2% interrupts.CPU29.RES:Rescheduling_interrupts 611807 ± 2% +10.1% 673896 ± 2% interrupts.CPU3.LOC:Local_timer_interrupts 7659 ± 5% +20.9% 9264 ± 10% interrupts.CPU3.RES:Rescheduling_interrupts 611433 ± 2% +10.2% 673520 ± 2% interrupts.CPU30.LOC:Local_timer_interrupts 7432 ± 3% +16.5% 8659 ± 4% interrupts.CPU30.RES:Rescheduling_interrupts 611450 ± 2% +10.2% 673604 ± 2% interrupts.CPU31.LOC:Local_timer_interrupts 7490 ± 4% +14.6% 8585 ± 5% interrupts.CPU31.RES:Rescheduling_interrupts 611444 ± 2% +10.2% 673588 ± 2% interrupts.CPU32.LOC:Local_timer_interrupts 7393 ± 4% +19.0% 8799 ± 3% interrupts.CPU32.RES:Rescheduling_interrupts 611407 ± 2% +10.2% 673606 ± 2% interrupts.CPU33.LOC:Local_timer_interrupts 7398 ± 4% +16.8% 8644 ± 4% interrupts.CPU33.RES:Rescheduling_interrupts 611452 ± 2% +10.2% 673609 ± 2% interrupts.CPU34.LOC:Local_timer_interrupts 7414 ± 5% +18.8% 8806 ± 4% interrupts.CPU34.RES:Rescheduling_interrupts 611433 ± 2% +10.2% 673588 ± 2% interrupts.CPU35.LOC:Local_timer_interrupts 7486 ± 5% +18.6% 8878 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts 611450 ± 2% +10.2% 673603 ± 2% interrupts.CPU36.LOC:Local_timer_interrupts 7503 ± 3% +15.0% 8627 ± 3% interrupts.CPU36.RES:Rescheduling_interrupts 611465 ± 2% +10.2% 673611 ± 2% interrupts.CPU37.LOC:Local_timer_interrupts 7416 ± 3% +17.6% 8719 ± 2% interrupts.CPU37.RES:Rescheduling_interrupts 611445 ± 2% +10.2% 673575 ± 2% interrupts.CPU38.LOC:Local_timer_interrupts 7477 ± 5% +14.2% 8540 ± 3% interrupts.CPU38.RES:Rescheduling_interrupts 611420 ± 2% +10.2% 673600 ± 2% interrupts.CPU39.LOC:Local_timer_interrupts 7562 ± 5% +14.0% 8620 ± 4% interrupts.CPU39.RES:Rescheduling_interrupts 611823 ± 2% +10.2% 673949 ± 2% interrupts.CPU4.LOC:Local_timer_interrupts 7579 ± 4% +14.9% 8708 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts 611390 ± 2% +10.2% 673618 ± 2% interrupts.CPU40.LOC:Local_timer_interrupts 7444 ± 4% +15.9% 8626 ± 3% interrupts.CPU40.RES:Rescheduling_interrupts 611429 ± 2% +10.2% 673612 ± 2% interrupts.CPU41.LOC:Local_timer_interrupts 7464 ± 5% +15.3% 8605 ± 2% interrupts.CPU41.RES:Rescheduling_interrupts 611418 ± 2% +10.2% 673597 ± 2% interrupts.CPU42.LOC:Local_timer_interrupts 7394 ± 4% +18.6% 8772 ± 5% interrupts.CPU42.RES:Rescheduling_interrupts 611364 ± 2% +10.2% 673601 ± 2% interrupts.CPU43.LOC:Local_timer_interrupts 7483 ± 5% +15.3% 8630 ± 3% interrupts.CPU43.RES:Rescheduling_interrupts 611421 ± 2% +10.2% 673593 ± 2% interrupts.CPU44.LOC:Local_timer_interrupts 7463 ± 2% +16.0% 8656 ± 4% interrupts.CPU44.RES:Rescheduling_interrupts 611423 ± 2% +10.2% 673593 ± 2% interrupts.CPU45.LOC:Local_timer_interrupts 7455 ± 4% +14.8% 8558 ± 3% interrupts.CPU45.RES:Rescheduling_interrupts 611333 ± 2% +10.2% 673616 ± 2% interrupts.CPU46.LOC:Local_timer_interrupts 7406 ± 3% +17.4% 8696 ± 3% interrupts.CPU46.RES:Rescheduling_interrupts 611421 ± 2% +10.2% 673605 ± 2% interrupts.CPU47.LOC:Local_timer_interrupts 7482 ± 5% +15.7% 8660 ± 2% interrupts.CPU47.RES:Rescheduling_interrupts 610910 ± 2% +10.3% 673737 ± 2% interrupts.CPU48.LOC:Local_timer_interrupts 7521 ± 6% +15.1% 8655 ± 4% interrupts.CPU48.RES:Rescheduling_interrupts 610921 ± 2% +10.3% 673770 ± 2% interrupts.CPU49.LOC:Local_timer_interrupts 7594 ± 5% +12.7% 8558 ± 3% interrupts.CPU49.RES:Rescheduling_interrupts 611796 ± 2% +10.1% 673831 ± 2% interrupts.CPU5.LOC:Local_timer_interrupts 610930 ± 2% +10.3% 673671 ± 2% interrupts.CPU50.LOC:Local_timer_interrupts 7504 ± 2% +17.6% 8824 ± 4% interrupts.CPU50.RES:Rescheduling_interrupts 610882 ± 2% +10.3% 673738 ± 2% interrupts.CPU51.LOC:Local_timer_interrupts 610884 ± 2% +10.3% 673731 ± 2% interrupts.CPU52.LOC:Local_timer_interrupts 7436 ± 4% +18.1% 8783 ± 3% interrupts.CPU52.RES:Rescheduling_interrupts 610885 ± 2% +10.3% 673774 ± 2% interrupts.CPU53.LOC:Local_timer_interrupts 7544 ± 4% +15.3% 8700 ± 4% interrupts.CPU53.RES:Rescheduling_interrupts 610902 ± 2% +10.3% 673763 ± 2% interrupts.CPU54.LOC:Local_timer_interrupts 7534 ± 4% +15.6% 8709 ± 6% interrupts.CPU54.RES:Rescheduling_interrupts 610927 ± 2% +10.3% 673739 ± 2% interrupts.CPU55.LOC:Local_timer_interrupts 610883 ± 2% +10.3% 673719 ± 2% interrupts.CPU56.LOC:Local_timer_interrupts 610921 ± 2% +10.3% 673689 ± 2% interrupts.CPU57.LOC:Local_timer_interrupts 7473 ± 5% +14.8% 8577 ± 4% interrupts.CPU57.RES:Rescheduling_interrupts 610893 ± 2% +10.3% 673781 ± 2% interrupts.CPU58.LOC:Local_timer_interrupts 7407 ± 5% +16.6% 8635 ± 3% interrupts.CPU58.RES:Rescheduling_interrupts 610898 ± 2% +10.3% 673763 ± 2% interrupts.CPU59.LOC:Local_timer_interrupts 7642 ± 5% +13.1% 8643 ± 3% interrupts.CPU59.RES:Rescheduling_interrupts 611823 ± 2% +10.1% 673914 ± 2% interrupts.CPU6.LOC:Local_timer_interrupts 7542 ± 3% +17.3% 8845 ± 3% interrupts.CPU6.RES:Rescheduling_interrupts 610869 ± 2% +10.3% 673784 ± 2% interrupts.CPU60.LOC:Local_timer_interrupts 7535 ± 4% +14.8% 8647 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts 610906 ± 2% +10.3% 673701 ± 2% interrupts.CPU61.LOC:Local_timer_interrupts 7631 ± 4% +13.5% 8660 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts 610863 ± 2% +10.3% 673705 ± 2% interrupts.CPU62.LOC:Local_timer_interrupts 7634 ± 4% +11.8% 8532 ± 4% interrupts.CPU62.RES:Rescheduling_interrupts 610920 ± 2% +10.3% 673730 ± 2% interrupts.CPU63.LOC:Local_timer_interrupts 7549 ± 5% +14.8% 8663 ± 5% interrupts.CPU63.RES:Rescheduling_interrupts 610884 ± 2% +10.3% 673727 ± 2% interrupts.CPU64.LOC:Local_timer_interrupts 7644 ± 3% +13.4% 8672 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts 610853 ± 2% +10.3% 673708 ± 2% interrupts.CPU65.LOC:Local_timer_interrupts 7555 ± 5% +14.8% 8672 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts 610950 ± 2% +10.3% 673804 ± 2% interrupts.CPU66.LOC:Local_timer_interrupts 7457 ± 5% +16.8% 8710 ± 4% interrupts.CPU66.RES:Rescheduling_interrupts 610888 ± 2% +10.3% 673735 ± 2% interrupts.CPU67.LOC:Local_timer_interrupts 7550 ± 4% +11.5% 8415 ± 5% interrupts.CPU67.RES:Rescheduling_interrupts 610891 ± 2% +10.3% 673727 ± 2% interrupts.CPU68.LOC:Local_timer_interrupts 7612 ± 4% +12.1% 8534 ± 3% interrupts.CPU68.RES:Rescheduling_interrupts 610901 ± 2% +10.3% 673659 ± 2% interrupts.CPU69.LOC:Local_timer_interrupts 7493 ± 4% +14.8% 8601 ± 4% interrupts.CPU69.RES:Rescheduling_interrupts 611808 ± 2% +10.1% 673844 ± 2% interrupts.CPU7.LOC:Local_timer_interrupts 7673 ± 5% +15.0% 8821 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts 610906 ± 2% +10.3% 673733 ± 2% interrupts.CPU70.LOC:Local_timer_interrupts 7412 ± 3% +16.9% 8664 ± 4% interrupts.CPU70.RES:Rescheduling_interrupts 610871 ± 2% +10.3% 673671 ± 2% interrupts.CPU71.LOC:Local_timer_interrupts 7519 ± 4% +13.8% 8557 ± 5% interrupts.CPU71.RES:Rescheduling_interrupts 611325 ± 2% +10.2% 673906 ± 2% interrupts.CPU72.LOC:Local_timer_interrupts 7720 ± 4% +9.6% 8459 ± 4% interrupts.CPU72.RES:Rescheduling_interrupts 611328 ± 2% +10.2% 673908 ± 2% interrupts.CPU73.LOC:Local_timer_interrupts 7535 ± 4% +14.6% 8634 ± 4% interrupts.CPU73.RES:Rescheduling_interrupts 611311 ± 2% +10.2% 673883 ± 2% interrupts.CPU74.LOC:Local_timer_interrupts 7526 ± 5% +14.7% 8635 ± 6% interrupts.CPU74.RES:Rescheduling_interrupts 611315 ± 2% +10.2% 673868 ± 2% interrupts.CPU75.LOC:Local_timer_interrupts 7712 ± 5% +9.8% 8469 ± 4% interrupts.CPU75.RES:Rescheduling_interrupts 611304 ± 2% +10.2% 673860 ± 2% interrupts.CPU76.LOC:Local_timer_interrupts 7476 ± 5% +14.2% 8537 ± 5% interrupts.CPU76.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673834 ± 2% interrupts.CPU77.LOC:Local_timer_interrupts 7670 ± 3% +12.5% 8630 ± 5% interrupts.CPU77.RES:Rescheduling_interrupts 611326 ± 2% +10.2% 673851 ± 2% interrupts.CPU78.LOC:Local_timer_interrupts 7417 ± 3% +15.6% 8578 ± 4% interrupts.CPU78.RES:Rescheduling_interrupts 611328 ± 2% +10.2% 673820 ± 2% interrupts.CPU79.LOC:Local_timer_interrupts 7545 ± 4% +13.9% 8590 ± 4% interrupts.CPU79.RES:Rescheduling_interrupts 611842 ± 2% +10.1% 673928 ± 2% interrupts.CPU8.LOC:Local_timer_interrupts 7673 ± 3% +14.9% 8818 ± 4% interrupts.CPU8.RES:Rescheduling_interrupts 611366 ± 2% +10.2% 673868 ± 2% interrupts.CPU80.LOC:Local_timer_interrupts 7513 ± 3% +16.1% 8723 ± 5% interrupts.CPU80.RES:Rescheduling_interrupts 611322 ± 2% +10.2% 673860 ± 2% interrupts.CPU81.LOC:Local_timer_interrupts 611299 ± 2% +10.2% 673868 ± 2% interrupts.CPU82.LOC:Local_timer_interrupts 7523 ± 4% +13.8% 8562 ± 4% interrupts.CPU82.RES:Rescheduling_interrupts 611320 ± 2% +10.2% 673727 ± 2% interrupts.CPU83.LOC:Local_timer_interrupts 7518 ± 5% +12.4% 8454 ± 5% interrupts.CPU83.RES:Rescheduling_interrupts 611326 ± 2% +10.2% 673899 ± 2% interrupts.CPU84.LOC:Local_timer_interrupts 7452 ± 6% +12.3% 8365 ± 5% interrupts.CPU84.RES:Rescheduling_interrupts 611329 ± 2% +10.2% 673885 ± 2% interrupts.CPU85.LOC:Local_timer_interrupts 7399 ± 5% +11.1% 8218 ± 5% interrupts.CPU85.RES:Rescheduling_interrupts 611278 ± 2% +10.2% 673875 ± 2% interrupts.CPU86.LOC:Local_timer_interrupts 7376 ± 4% +16.3% 8581 ± 3% interrupts.CPU86.RES:Rescheduling_interrupts 611313 ± 2% +10.2% 673874 ± 2% interrupts.CPU87.LOC:Local_timer_interrupts 7544 ± 4% +12.8% 8508 ± 4% interrupts.CPU87.RES:Rescheduling_interrupts 611351 ± 2% +10.2% 673928 ± 2% interrupts.CPU88.LOC:Local_timer_interrupts 7418 ± 6% +13.3% 8406 ± 4% interrupts.CPU88.RES:Rescheduling_interrupts 611324 ± 2% +10.2% 673795 ± 2% interrupts.CPU89.LOC:Local_timer_interrupts 7426 ± 4% +13.6% 8439 ± 5% interrupts.CPU89.RES:Rescheduling_interrupts 611832 ± 2% +10.2% 673968 ± 2% interrupts.CPU9.LOC:Local_timer_interrupts 7538 ± 3% +15.6% 8715 ± 4% interrupts.CPU9.RES:Rescheduling_interrupts 611298 ± 2% +10.2% 673878 ± 2% interrupts.CPU90.LOC:Local_timer_interrupts 7421 ± 3% +12.7% 8363 ± 4% interrupts.CPU90.RES:Rescheduling_interrupts 611305 ± 2% +10.2% 673874 ± 2% interrupts.CPU91.LOC:Local_timer_interrupts 7377 ± 5% +15.6% 8528 ± 5% interrupts.CPU91.RES:Rescheduling_interrupts 611288 ± 2% +10.2% 673876 ± 2% interrupts.CPU92.LOC:Local_timer_interrupts 7349 ± 4% +12.0% 8234 ± 4% interrupts.CPU92.RES:Rescheduling_interrupts 611329 ± 2% +10.2% 673857 ± 2% interrupts.CPU93.LOC:Local_timer_interrupts 7438 ± 3% +14.7% 8530 ± 4% interrupts.CPU93.RES:Rescheduling_interrupts 611276 ± 2% +10.2% 673866 ± 2% interrupts.CPU94.LOC:Local_timer_interrupts 7363 ± 4% +12.7% 8300 ± 5% interrupts.CPU94.RES:Rescheduling_interrupts 611381 ± 2% +10.2% 673922 ± 2% interrupts.CPU95.LOC:Local_timer_interrupts 10549 ± 3% +14.9% 12117 ± 4% interrupts.CPU95.RES:Rescheduling_interrupts 611824 ± 2% +10.1% 673847 ± 2% interrupts.CPU96.LOC:Local_timer_interrupts 7416 ± 3% +12.9% 8375 ± 5% interrupts.CPU96.RES:Rescheduling_interrupts 611751 ± 2% +10.2% 673870 ± 2% interrupts.CPU97.LOC:Local_timer_interrupts 7437 ± 4% +14.0% 8477 ± 4% interrupts.CPU97.RES:Rescheduling_interrupts 611787 ± 2% +10.1% 673877 ± 2% interrupts.CPU98.LOC:Local_timer_interrupts 7555 ± 4% +13.0% 8534 ± 3% interrupts.CPU98.RES:Rescheduling_interrupts 611783 ± 2% +10.1% 673872 ± 2% interrupts.CPU99.LOC:Local_timer_interrupts 7452 ± 3% +12.8% 8404 ± 5% interrupts.CPU99.RES:Rescheduling_interrupts 1.174e+08 ± 2% +10.2% 1.294e+08 ± 2% interrupts.LOC:Local_timer_interrupts 1433875 ± 3% +14.6% 1643307 ± 3% interrupts.RES:Rescheduling_interrupts aim7.jobs-per-min 30500 +-------------------------------------------------------------------+ | ..+.... .| 30000 |-+ ...... ... ... | 29500 |................ ...... .. ... | | +.. ... .. | 29000 |-+ ... ... | 28500 |-+ . ... | | +. | 28000 |-+ O O | 27500 |-+ | | | 27000 |-+ | 26500 |-+ | | O | 26000 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

10 months, 3 weeks

2
3
0 / 0

[sched] c3a340f7e7: invalid_opcode:#[##]

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-4.9): commit: c3a340f7e7eadac7662ab104ceb16432e5a4c6b2 ("sched: Have sched_class_highest define by vmlinux.lds.h") https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git sched/core in testcase: trinity with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +------------------------------------------+------------+------------+ | | 590d697963 | c3a340f7e7 | +------------------------------------------+------------+------------+ | boot_successes | 4 | 0 | | boot_failures | 0 | 4 | | invalid_opcode:#[##] | 0 | 4 | | EIP:sched_init | 0 | 4 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 4 | | kernel_BUG_at_kernel/sched/core.c | 0 | 4 | +------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 1.813939] ** If you see this message and you are not debugging ** [ 1.814781] ** the kernel, report this immediately to your vendor! ** [ 1.815661] ** ** [ 1.816509] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 1.817362] ********************************************************** [ 1.820914] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC [ 1.821634] CPU: 0 PID: 0 Comm: swapper Not tainted 5.8.0-rc1-00024-gc3a340f7e7ead #2 [ 1.822630] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 1.823710] EIP: sched_init+0x3d/0x2fb [ 1.824191] Code: 20 a6 e5 c1 3d 40 a6 e5 c1 0f 95 c0 08 c2 75 1a b8 a0 a6 e5 c1 3d c0 a6 e5 c1 75 0e b8 20 a7 e5 c1 3d 40 a7 e5 c1 74 04 0f 0b <0f> 0b 55 89 e5 57 56 53 83 ec 08 e8 67 04 00 00 8b 3d 80 ad ec c1 [ 1.826607] EAX: c1e5a601 EBX: c1dac484 ECX: 00000000 EDX: 00000001 [ 1.827434] ESI: 00000000 EDI: 000003f8 EBP: c1e93f9c ESP: c1e93f78 [ 1.828241] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00210002 [ 1.829112] CR0: 80050033 CR2: ffc70000 CR3: 021ce000 CR4: 00000690 [ 1.829919] Call Trace: [ 1.830240] ? start_kernel+0x43f/0x677 [ 1.830738] i386_start_kernel+0x48/0x4a [ 1.831261] startup_32_smp+0x15f/0x164 [ 1.831757] Modules linked in: [ 1.832165] ---[ end trace db9d28bea02c9e7f ]--- [ 1.832761] EIP: sched_init+0x3d/0x2fb [ 1.833243] Code: 20 a6 e5 c1 3d 40 a6 e5 c1 0f 95 c0 08 c2 75 1a b8 a0 a6 e5 c1 3d c0 a6 e5 c1 75 0e b8 20 a7 e5 c1 3d 40 a7 e5 c1 74 04 0f 0b <0f> 0b 55 89 e5 57 56 53 83 ec 08 e8 67 04 00 00 8b 3d 80 ad ec c1 [ 1.835656] EAX: c1e5a601 EBX: c1dac484 ECX: 00000000 EDX: 00000001 [ 1.836463] ESI: 00000000 EDI: 000003f8 EBP: c1e93f9c ESP: c1e93f78 [ 1.838603] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00210002 [ 1.839496] CR0: 80050033 CR2: ffc70000 CR3: 021ce000 CR4: 00000690 [ 1.840300] Kernel panic - not syncing: Fatal exception [ 1.840969] ------------[ cut here ]------------ [ 1.840970] kernel BUG at kernel/sched/core.c:6652! Elapsed time: 60 qemu-img create -f qcow2 disk-vm-snb-94-0 256G qemu-img create -f qcow2 disk-vm-snb-94-1 256G To reproduce: # build kernel cd linux cp config-5.8.0-rc1-00024-gc3a340f7e7ead .config make HOSTCC=gcc-4.9 CC=gcc-4.9 ARCH=i386 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, lkp

11 months

6
12
0 / 0

[hugetlbfs] c0d0381ade: vm-scalability.throughput -33.4% regression

by kernel test robot

Greeting, FYI, we noticed a -33.4% regression of vm-scalability.throughput due to commit: commit: c0d0381ade79885c04a04c303284b040616b116e ("hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: vm-scalability on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory with following parameters: runtime: 300s size: 8T test: anon-cow-seq-hugetlb cpufreq_governor: performance ucode: 0x11 test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us. test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/8T/lkp-knm01/anon-cow-seq-hugetlb/vm-scalability/0x11 commit: 49aef7175c ("mm/memblock.c: remove redundant assignment to variable max_addr") c0d0381ade ("hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization") 49aef7175cc6eb70 c0d0381ade79885c04a04c30328 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x :4 25% 1:4 dmesg.WARNING:at_ip__mutex_lock/0x :4 25% 1:4 dmesg.WARNING:at_ip_perf_event_mmap_output/0x %stddev %change %stddev \ | \ 37643 -29.5% 26549 ± 4% vm-scalability.median 9.18 ± 16% +9.3 18.48 ± 21% vm-scalability.median_stddev% 12666048 -33.4% 8429873 ± 4% vm-scalability.throughput 1701516 -19.5% 1369683 ± 5% vm-scalability.time.minor_page_faults 13447 ± 3% -19.3% 10848 vm-scalability.time.percent_of_cpu_this_job_got 26620 ± 4% -10.5% 23830 vm-scalability.time.system_time 14297 ± 3% -35.3% 9255 ± 4% vm-scalability.time.user_time 125377 ± 2% +621.3% 904348 ± 4% vm-scalability.time.voluntary_context_switches 2.468e+09 -19.6% 1.984e+09 ± 5% vm-scalability.workload 3.807e+10 ± 4% +20.9% 4.605e+10 ± 2% cpuidle.C1.time 9524 ± 10% -17.3% 7873 ± 4% numa-vmstat.node0.nr_mapped 2494252 -12.4% 2184278 ± 3% numa-numastat.node0.local_node 2494194 -12.4% 2184223 ± 3% numa-numastat.node0.numa_hit 49.26 ± 3% +9.4 58.62 mpstat.cpu.all.idle% 32.89 ± 4% -3.2 29.70 mpstat.cpu.all.sys% 17.79 ± 3% -6.2 11.64 ± 4% mpstat.cpu.all.usr% 4856 +47.6% 7169 numa-meminfo.node0.HugePages_Surp 4856 +47.6% 7169 numa-meminfo.node0.HugePages_Total 37725 ± 10% -17.7% 31054 ± 4% numa-meminfo.node0.Mapped 12274212 +38.3% 16981169 numa-meminfo.node0.MemUsed 49.00 ± 3% +18.9% 58.25 vmstat.cpu.id 17.00 ± 4% -33.8% 11.25 ± 3% vmstat.cpu.us 138.25 ± 2% -19.5% 111.25 ± 2% vmstat.procs.r 5836 ± 2% +88.0% 10969 ± 3% vmstat.system.cs 4855 +48.0% 7186 meminfo.HugePages_Surp 4856 +48.0% 7186 meminfo.HugePages_Total 9946017 +48.0% 14718559 meminfo.Hugetlb 48593 ± 8% -13.9% 41859 ± 3% meminfo.Mapped 12877361 +36.8% 17619364 meminfo.Memused 81607 ± 2% +29.1% 105324 meminfo.max_used_kB 2890 ±100% +285.4% 11139 ± 3% perf-stat.i.context-switches 298.31 ±100% +243.7% 1025 ± 2% perf-stat.i.cpu-migrations 0.11 ±100% +119.3% 0.24 perf-stat.i.ipc 0.04 ±100% +129.7% 0.10 ± 4% perf-stat.i.metric.K/sec 0.11 ±100% +122.1% 0.24 perf-stat.overall.ipc 3107 ±100% +137.3% 7372 ± 3% perf-stat.overall.path-length 2836 ±100% +284.4% 10902 ± 3% perf-stat.ps.context-switches 275.01 ±100% +257.9% 984.31 ± 2% perf-stat.ps.cpu-migrations 111986 -11.7% 98924 ± 2% slabinfo.anon_vma.active_objs 2434 -11.7% 2150 ± 2% slabinfo.anon_vma.active_slabs 111986 -11.7% 98924 ± 2% slabinfo.anon_vma.num_objs 2434 -11.7% 2150 ± 2% slabinfo.anon_vma.num_slabs 258615 -9.1% 235038 ± 2% slabinfo.anon_vma_chain.active_objs 4041 -9.1% 3672 ± 2% slabinfo.anon_vma_chain.active_slabs 258670 -9.1% 235049 ± 2% slabinfo.anon_vma_chain.num_objs 4041 -9.1% 3672 ± 2% slabinfo.anon_vma_chain.num_slabs 1500 -17.8% 1232 ± 5% slabinfo.hugetlbfs_inode_cache.active_objs 1500 -17.8% 1232 ± 5% slabinfo.hugetlbfs_inode_cache.num_objs 682.00 ± 6% +15.9% 790.50 ± 6% slabinfo.numa_policy.active_objs 682.00 ± 6% +15.9% 790.50 ± 6% slabinfo.numa_policy.num_objs 1082230 -19.6% 869890 ± 5% proc-vmstat.htlb_buddy_alloc_success 1721553 -6.8% 1604364 proc-vmstat.nr_dirty_background_threshold 3447317 -6.8% 3212652 proc-vmstat.nr_dirty_threshold 17371789 -6.8% 16197198 proc-vmstat.nr_free_pages 351.00 +3.8% 364.25 proc-vmstat.nr_inactive_file 39705 +1.4% 40269 proc-vmstat.nr_kernel_stack 12252 ± 7% -13.7% 10570 ± 3% proc-vmstat.nr_mapped 2632 +8.8% 2863 proc-vmstat.nr_page_table_pages 98880 -1.6% 97341 proc-vmstat.nr_slab_unreclaimable 351.00 +3.8% 364.25 proc-vmstat.nr_zone_inactive_file 6285 ± 50% -90.8% 577.50 ± 47% proc-vmstat.numa_hint_faults 6285 ± 50% -90.8% 577.50 ± 47% proc-vmstat.numa_hint_faults_local 2514799 -12.3% 2205532 ± 3% proc-vmstat.numa_hit 2514798 -12.3% 2205531 ± 3% proc-vmstat.numa_local 5.557e+08 -19.6% 4.469e+08 ± 5% proc-vmstat.pgalloc_normal 2510883 -12.9% 2187221 ± 3% proc-vmstat.pgfault 5.553e+08 -19.5% 4.468e+08 ± 5% proc-vmstat.pgfree 60107 ± 2% -19.7% 48269 ± 17% sched_debug.cfs_rq:/.exec_clock.avg 71511 ± 2% -23.4% 54794 ± 16% sched_debug.cfs_rq:/.exec_clock.max 44464 ± 7% -16.9% 36954 ± 18% sched_debug.cfs_rq:/.exec_clock.min 4619 ± 4% -47.8% 2412 ± 11% sched_debug.cfs_rq:/.exec_clock.stddev 614.00 ± 10% +371.9% 2897 ±124% sched_debug.cfs_rq:/.load_avg.max 49.58 ± 11% +321.1% 208.79 ±123% sched_debug.cfs_rq:/.load_avg.stddev 11526482 ± 4% -37.6% 7189313 ± 17% sched_debug.cfs_rq:/.min_vruntime.avg 13399855 ± 4% -40.0% 8039878 ± 15% sched_debug.cfs_rq:/.min_vruntime.max 8425626 ± 8% -37.2% 5294770 ± 18% sched_debug.cfs_rq:/.min_vruntime.min 788640 ± 4% -56.4% 343805 ± 11% sched_debug.cfs_rq:/.min_vruntime.stddev 0.37 ± 8% -40.0% 0.22 ± 10% sched_debug.cfs_rq:/.nr_running.avg 21.17 ± 5% +134.9% 49.72 ± 21% sched_debug.cfs_rq:/.nr_spread_over.avg 56.15 ± 13% +60.7% 90.25 ± 19% sched_debug.cfs_rq:/.nr_spread_over.max 6.95 ± 7% +289.4% 27.06 ± 23% sched_debug.cfs_rq:/.nr_spread_over.min 7.32 ± 5% +39.4% 10.20 ± 19% sched_debug.cfs_rq:/.nr_spread_over.stddev 413.38 ± 11% -27.5% 299.85 ± 14% sched_debug.cfs_rq:/.runnable_avg.avg 834.44 ± 53% -65.7% 286.48 ± 5% sched_debug.cfs_rq:/.runnable_avg.stddev 2625208 ± 11% -29.7% 1845657 ± 13% sched_debug.cfs_rq:/.spread0.avg 4483653 ± 6% -39.8% 2699533 ± 11% sched_debug.cfs_rq:/.spread0.max -475138 -94.3% -26940 sched_debug.cfs_rq:/.spread0.min 801419 ± 4% -56.8% 345835 ± 11% sched_debug.cfs_rq:/.spread0.stddev 371.84 ± 8% -22.5% 288.10 ± 14% sched_debug.cfs_rq:/.util_avg.avg 382.64 ± 4% -32.5% 258.09 ± 4% sched_debug.cfs_rq:/.util_avg.stddev 330.75 ± 10% -54.8% 149.57 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.avg 358.21 ± 6% -27.6% 259.25 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev 4047967 ± 4% +17.6% 4760305 ± 4% sched_debug.cpu.avg_idle.avg 561715 ± 52% +116.4% 1215479 ± 7% sched_debug.cpu.avg_idle.min 731.80 ± 6% -38.6% 449.23 ± 8% sched_debug.cpu.clock.stddev 731.80 ± 6% -38.6% 449.23 ± 8% sched_debug.cpu.clock_task.stddev 4741 ± 13% -43.0% 2702 ± 18% sched_debug.cpu.curr->pid.avg 5519 ± 7% -15.8% 4650 ± 11% sched_debug.cpu.curr->pid.stddev 2119875 ± 5% +16.3% 2465873 ± 4% sched_debug.cpu.max_idle_balance_cost.avg 1107931 ± 2% +24.6% 1380330 ± 3% sched_debug.cpu.max_idle_balance_cost.min 581556 ± 5% +11.1% 646020 ± 5% sched_debug.cpu.max_idle_balance_cost.stddev 0.00 ± 4% -30.8% 0.00 ± 11% sched_debug.cpu.next_balance.stddev 0.31 ± 12% -32.6% 0.21 ± 10% sched_debug.cpu.nr_running.avg 5116 +46.1% 7477 ± 11% sched_debug.cpu.nr_switches.avg 2301 ± 5% +85.0% 4257 ± 22% sched_debug.cpu.nr_switches.min 0.09 ± 44% +300.2% 0.34 ± 17% sched_debug.cpu.nr_uninterruptible.avg 296.40 ± 3% +24.1% 367.95 ± 8% sched_debug.cpu.nr_uninterruptible.max -50.40 +38.8% -69.98 sched_debug.cpu.nr_uninterruptible.min 23.04 ± 3% +53.3% 35.32 ± 12% sched_debug.cpu.nr_uninterruptible.stddev 2343 +98.2% 4646 ± 18% sched_debug.cpu.sched_count.avg 1110 ± 2% +193.0% 3252 ± 23% sched_debug.cpu.sched_count.min 913.60 ± 2% +125.6% 2060 ± 18% sched_debug.cpu.sched_goidle.avg 356.95 +296.7% 1416 ± 25% sched_debug.cpu.sched_goidle.min 1148 +102.6% 2326 ± 18% sched_debug.cpu.ttwu_count.avg 510.15 ± 5% +80.8% 922.26 ± 21% sched_debug.cpu.ttwu_count.min 148.25 ± 6% -11.3% 131.52 ± 6% sched_debug.cpu.ttwu_local.stddev 21.96 ± 6% -10.0 11.99 ± 11% perf-profile.calltrace.cycles-pp.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault 19.05 ± 5% -8.1 10.96 ± 11% perf-profile.calltrace.cycles-pp.copy_user_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault 18.72 ± 5% -8.0 10.76 ± 11% perf-profile.calltrace.cycles-pp.copy_subpage.copy_user_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault 17.82 ± 4% -7.6 10.17 ± 10% perf-profile.calltrace.cycles-pp.copy_page.copy_subpage.copy_user_huge_page.hugetlb_cow.hugetlb_fault 2.22 ± 44% -1.5 0.68 ± 23% perf-profile.calltrace.cycles-pp.alloc_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault 1.03 ± 22% -0.6 0.40 ±102% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.do_access 0.91 ± 23% -0.6 0.32 ±103% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.do_access 0.81 ± 19% -0.2 0.60 ± 14% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit 0.81 ± 19% -0.2 0.60 ± 14% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit 0.80 ± 19% -0.2 0.60 ± 14% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput 0.79 ± 19% -0.2 0.59 ± 15% perf-profile.calltrace.cycles-pp.__free_huge_page.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap 1.44 ± 22% +0.6 2.00 ± 28% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 0.71 ± 80% +1.5 2.16 ± 33% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.osq_lock 1.03 ± 55% +1.6 2.67 ± 32% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.osq_lock.__mutex_lock 1.24 ± 57% +2.0 3.29 ± 26% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.osq_lock.__mutex_lock.hugetlb_fault 1.38 ± 55% +2.3 3.66 ± 25% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.osq_lock.__mutex_lock.hugetlb_fault.handle_mm_fault 29.76 ± 21% +20.1 49.83 ± 13% perf-profile.calltrace.cycles-pp.__mutex_lock.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault 15.71 ± 27% +23.2 38.88 ± 18% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.hugetlb_fault.handle_mm_fault.do_page_fault 21.96 ± 6% -10.0 12.00 ± 11% perf-profile.children.cycles-pp.hugetlb_cow 19.36 ± 5% -8.2 11.11 ± 11% perf-profile.children.cycles-pp.copy_user_huge_page 18.74 ± 5% -8.0 10.77 ± 11% perf-profile.children.cycles-pp.copy_subpage 18.31 ± 5% -7.8 10.54 ± 11% perf-profile.children.cycles-pp.copy_page 16.95 ± 20% -7.0 10.00 ± 53% perf-profile.children.cycles-pp.do_rw_once 2.46 ± 41% -1.7 0.80 ± 14% perf-profile.children.cycles-pp.alloc_huge_page 3.11 ± 36% -1.6 1.48 ± 4% perf-profile.children.cycles-pp._raw_spin_lock 2.64 ± 44% -1.5 1.10 ± 6% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 1.60 ± 38% -1.0 0.57 ± 14% perf-profile.children.cycles-pp.alloc_surplus_huge_page 1.81 ± 12% -0.5 1.34 ± 4% perf-profile.children.cycles-pp.do_syscall_64 1.81 ± 12% -0.5 1.35 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 0.76 ± 28% -0.4 0.38 ± 12% perf-profile.children.cycles-pp.alloc_fresh_huge_page 1.21 ± 10% -0.4 0.85 ± 11% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.40 ± 52% -0.3 0.09 ± 20% perf-profile.children.cycles-pp.prep_new_huge_page 0.81 ± 19% -0.2 0.60 ± 14% perf-profile.children.cycles-pp.tlb_finish_mmu 0.81 ± 19% -0.2 0.60 ± 14% perf-profile.children.cycles-pp.tlb_flush_mmu 0.80 ± 18% -0.2 0.60 ± 15% perf-profile.children.cycles-pp.release_pages 0.79 ± 19% -0.2 0.59 ± 15% perf-profile.children.cycles-pp.__free_huge_page 0.44 ± 13% -0.2 0.26 ± 14% perf-profile.children.cycles-pp.___might_sleep 0.20 ± 14% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.update_and_free_page 0.32 ± 12% -0.1 0.21 ± 6% perf-profile.children.cycles-pp.ksys_read 0.29 ± 13% -0.1 0.18 ± 10% perf-profile.children.cycles-pp.read 0.31 ± 11% -0.1 0.20 ± 8% perf-profile.children.cycles-pp.vfs_read 0.18 ± 17% -0.1 0.09 ± 8% perf-profile.children.cycles-pp._cond_resched 0.16 ± 9% -0.1 0.09 ± 17% perf-profile.children.cycles-pp.__libc_start_main 0.16 ± 11% -0.1 0.08 ± 17% perf-profile.children.cycles-pp.main 0.19 ± 14% -0.1 0.12 ± 17% perf-profile.children.cycles-pp.x86_pmu_disable 0.26 ± 8% -0.1 0.19 ± 6% perf-profile.children.cycles-pp.ksys_write 0.26 ± 9% -0.1 0.19 ± 8% perf-profile.children.cycles-pp.vfs_write 0.15 ± 12% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.new_sync_read 0.15 ± 11% -0.1 0.09 ± 17% perf-profile.children.cycles-pp.pipe_read 0.24 ± 9% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.new_sync_write 0.08 ± 6% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__perf_sw_event 0.11 ± 14% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.__wake_up_common 0.12 ± 3% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.schedule_timeout 0.12 ± 13% -0.0 0.07 ± 17% perf-profile.children.cycles-pp.__wake_up_common_lock 0.10 ± 15% -0.0 0.06 ± 15% perf-profile.children.cycles-pp.autoremove_wake_function 0.11 ± 29% -0.0 0.07 ± 10% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.26 ± 6% -0.0 0.22 ± 5% perf-profile.children.cycles-pp.tick_program_event 0.09 ± 31% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.09 ± 9% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.rcu_gp_kthread 0.08 ± 10% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.cmd_stat 0.01 ±173% +0.0 0.06 ± 11% perf-profile.children.cycles-pp.select_task_rq_fair 0.11 ± 11% +0.0 0.16 ± 9% perf-profile.children.cycles-pp.finish_task_switch 0.00 +0.1 0.06 ± 14% perf-profile.children.cycles-pp.rwsem_wake 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.arch_stack_walk 0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.dequeue_task_fair 0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.stack_trace_save_tsk 0.13 ± 6% +0.1 0.21 ± 10% perf-profile.children.cycles-pp.schedule_idle 0.06 ± 11% +0.1 0.14 ± 10% perf-profile.children.cycles-pp.enqueue_entity 0.03 ±100% +0.1 0.11 ± 7% perf-profile.children.cycles-pp.update_load_avg 0.00 +0.1 0.09 ± 12% perf-profile.children.cycles-pp.__account_scheduler_latency 0.07 ± 17% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.activate_task 0.07 ± 17% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.ttwu_do_activate 0.07 ± 17% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.enqueue_task_fair 0.00 +0.1 0.09 ± 8% perf-profile.children.cycles-pp.wake_up_q 0.14 ± 24% +0.1 0.27 ± 13% perf-profile.children.cycles-pp.schedule_preempt_disabled 0.00 +0.1 0.14 ± 15% perf-profile.children.cycles-pp.sched_ttwu_pending 1.17 ± 13% +0.4 1.56 ± 8% perf-profile.children.cycles-pp.load_balance 0.99 ± 11% +0.4 1.41 ± 9% perf-profile.children.cycles-pp.find_busiest_group 0.96 ± 11% +0.4 1.38 ± 9% perf-profile.children.cycles-pp.update_sd_lb_stats 0.66 ± 9% +0.5 1.20 ± 11% perf-profile.children.cycles-pp.newidle_balance 0.53 ± 7% +0.6 1.11 ± 12% perf-profile.children.cycles-pp.pick_next_task_fair 0.71 ± 8% +0.6 1.35 ± 10% perf-profile.children.cycles-pp.schedule 0.00 +0.7 0.68 ± 17% perf-profile.children.cycles-pp.rwsem_down_read_slowpath 0.86 ± 7% +0.7 1.58 ± 9% perf-profile.children.cycles-pp.__schedule 52.14 ± 14% +10.7 62.86 ± 9% perf-profile.children.cycles-pp.page_fault 52.09 ± 14% +10.7 62.83 ± 9% perf-profile.children.cycles-pp.do_page_fault 51.96 ± 14% +10.8 62.74 ± 9% perf-profile.children.cycles-pp.handle_mm_fault 51.86 ± 14% +10.8 62.67 ± 9% perf-profile.children.cycles-pp.hugetlb_fault 29.77 ± 21% +20.1 49.84 ± 13% perf-profile.children.cycles-pp.__mutex_lock 15.83 ± 27% +23.4 39.18 ± 18% perf-profile.children.cycles-pp.osq_lock 16.53 ± 4% -7.1 9.48 ± 9% perf-profile.self.cycles-pp.copy_page 15.27 ± 22% -6.4 8.90 ± 53% perf-profile.self.cycles-pp.do_rw_once 2.38 ± 42% -1.4 1.02 ± 7% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.70 ± 9% -0.2 0.48 ± 14% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler 0.38 ± 2% -0.2 0.21 ± 11% perf-profile.self.cycles-pp.copy_subpage 0.33 ± 4% -0.2 0.17 ± 11% perf-profile.self.cycles-pp.copy_user_huge_page 0.18 ± 18% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.update_and_free_page 0.19 ± 14% -0.1 0.11 ± 18% perf-profile.self.cycles-pp.x86_pmu_disable 0.19 ± 6% -0.1 0.12 ± 10% perf-profile.self.cycles-pp.___might_sleep 0.35 ± 14% -0.1 0.28 ± 8% perf-profile.self.cycles-pp.run_timer_softirq 0.18 ± 10% -0.1 0.12 ± 6% perf-profile.self.cycles-pp.get_page_from_freelist 0.26 ± 6% -0.0 0.22 ± 5% perf-profile.self.cycles-pp.tick_program_event 0.07 -0.0 0.06 ± 7% perf-profile.self.cycles-pp.try_to_wake_up 0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.update_load_avg 0.73 ± 12% +0.3 1.06 ± 8% perf-profile.self.cycles-pp.update_sd_lb_stats 14.37 ± 25% +21.2 35.60 ± 18% perf-profile.self.cycles-pp.osq_lock 22773 ± 3% +11.6% 25420 ± 2% softirqs.CPU0.SCHED 15462 ± 6% +24.5% 19248 ± 5% softirqs.CPU1.SCHED 8331 ± 11% +27.1% 10590 ± 11% softirqs.CPU10.RCU 14783 ± 4% +16.0% 17145 ± 4% softirqs.CPU10.SCHED 14236 ± 3% +15.6% 16462 ± 2% softirqs.CPU100.SCHED 14117 ± 3% +18.2% 16683 ± 3% softirqs.CPU101.SCHED 13918 ± 4% +18.4% 16472 ± 5% softirqs.CPU102.SCHED 14183 ± 6% +18.0% 16742 ± 3% softirqs.CPU103.SCHED 13795 ± 5% +19.8% 16528 softirqs.CPU104.SCHED 14326 ± 6% +17.7% 16859 softirqs.CPU105.SCHED 13842 ± 5% +19.4% 16530 ± 2% softirqs.CPU106.SCHED 14155 ± 4% +20.0% 16983 ± 2% softirqs.CPU107.SCHED 13907 ± 3% +18.6% 16492 ± 2% softirqs.CPU108.SCHED 14411 ± 7% +16.1% 16726 ± 2% softirqs.CPU109.SCHED 14354 +19.0% 17085 ± 2% softirqs.CPU11.SCHED 13689 ± 6% +21.8% 16667 ± 3% softirqs.CPU110.SCHED 13461 ± 6% +25.6% 16907 ± 3% softirqs.CPU111.SCHED 13738 ± 9% +23.2% 16925 softirqs.CPU112.SCHED 14135 ± 3% +20.5% 17032 softirqs.CPU113.SCHED 14477 ± 4% +20.8% 17491 ± 4% softirqs.CPU114.SCHED 123003 ± 2% -7.8% 113426 ± 2% softirqs.CPU114.TIMER 14014 ± 4% +23.4% 17299 softirqs.CPU115.SCHED 13528 ± 6% +24.4% 16824 ± 2% softirqs.CPU116.SCHED 13530 ± 6% +24.5% 16840 ± 2% softirqs.CPU117.SCHED 14263 ± 3% +16.1% 16557 ± 2% softirqs.CPU119.SCHED 7902 ± 2% +22.1% 9648 ± 6% softirqs.CPU12.RCU 14332 ± 3% +19.5% 17130 ± 4% softirqs.CPU12.SCHED 13476 ± 5% +26.5% 17047 softirqs.CPU120.SCHED 14240 ± 6% +20.7% 17183 softirqs.CPU121.SCHED 13161 ± 5% +27.0% 16708 ± 2% softirqs.CPU122.SCHED 13037 ± 7% +28.8% 16786 softirqs.CPU123.SCHED 14229 ± 5% +16.5% 16577 ± 3% softirqs.CPU124.SCHED 14427 ± 4% +15.4% 16652 softirqs.CPU125.SCHED 14426 ± 4% +16.3% 16771 softirqs.CPU126.SCHED 6680 +39.3% 9305 ± 22% softirqs.CPU127.RCU 13659 ± 3% +23.7% 16890 ± 3% softirqs.CPU128.SCHED 14396 ± 5% +17.9% 16979 ± 3% softirqs.CPU129.SCHED 14183 ± 7% +20.2% 17048 ± 2% softirqs.CPU13.SCHED 6700 ± 2% +35.8% 9097 ± 17% softirqs.CPU130.RCU 13949 ± 4% +22.2% 17051 ± 4% softirqs.CPU130.SCHED 14271 ± 4% +21.8% 17379 ± 2% softirqs.CPU131.SCHED 129315 ± 3% -7.0% 120290 softirqs.CPU131.TIMER 14135 ± 2% +16.0% 16396 ± 2% softirqs.CPU132.SCHED 13567 ± 4% +22.5% 16617 ± 3% softirqs.CPU133.SCHED 13912 ± 5% +20.1% 16712 ± 2% softirqs.CPU134.SCHED 14309 ± 5% +18.9% 17016 ± 2% softirqs.CPU135.SCHED 13637 ± 4% +20.3% 16402 ± 5% softirqs.CPU136.SCHED 13843 ± 4% +19.3% 16518 ± 2% softirqs.CPU137.SCHED 14049 ± 5% +19.2% 16750 ± 2% softirqs.CPU138.SCHED 14244 ± 7% +18.1% 16816 softirqs.CPU139.SCHED 14139 +22.1% 17271 ± 2% softirqs.CPU14.SCHED 13460 ± 8% +22.9% 16546 softirqs.CPU140.SCHED 13645 ± 6% +21.7% 16601 ± 2% softirqs.CPU141.SCHED 13278 ± 6% +25.5% 16661 ± 3% softirqs.CPU142.SCHED 12487 ± 5% +29.2% 16136 ± 2% softirqs.CPU143.SCHED 13440 ± 9% +22.8% 16502 softirqs.CPU144.SCHED 126064 ± 2% -8.1% 115836 softirqs.CPU144.TIMER 13995 ± 6% +21.2% 16966 ± 3% softirqs.CPU145.SCHED 13834 ± 3% +14.8% 15883 ± 6% softirqs.CPU146.SCHED 14297 ± 9% +17.1% 16735 softirqs.CPU147.SCHED 14500 ± 3% +15.2% 16711 softirqs.CPU148.SCHED 14419 ± 6% +17.5% 16945 softirqs.CPU149.SCHED 8018 ± 6% +24.4% 9977 ± 7% softirqs.CPU15.RCU 13832 ± 5% +28.2% 17737 ± 5% softirqs.CPU15.SCHED 14538 ± 5% +15.0% 16723 ± 2% softirqs.CPU150.SCHED 15092 +9.5% 16526 ± 4% softirqs.CPU152.SCHED 14533 ± 2% +15.6% 16804 softirqs.CPU153.SCHED 14415 ± 2% +14.8% 16554 ± 2% softirqs.CPU154.SCHED 14583 ± 2% +15.4% 16833 ± 2% softirqs.CPU155.SCHED 14851 ± 2% +11.8% 16598 ± 2% softirqs.CPU156.SCHED 6747 +35.2% 9124 ± 19% softirqs.CPU158.RCU 13839 ± 4% +25.9% 17429 ± 4% softirqs.CPU158.SCHED 15020 ± 5% +11.8% 16785 ± 2% softirqs.CPU159.SCHED 13468 ± 5% +25.8% 16945 ± 2% softirqs.CPU16.SCHED 128305 ± 3% -8.1% 117855 softirqs.CPU160.TIMER 15111 ± 2% +13.7% 17184 softirqs.CPU162.SCHED 14788 ± 2% +12.8% 16685 ± 3% softirqs.CPU163.SCHED 13903 ± 5% +21.5% 16889 softirqs.CPU164.SCHED 14547 ± 5% +13.5% 16510 ± 2% softirqs.CPU165.SCHED 14440 ± 4% +13.3% 16354 ± 3% softirqs.CPU166.SCHED 14450 ± 5% +14.1% 16494 ± 2% softirqs.CPU167.SCHED 131454 ± 3% -6.9% 122396 ± 2% softirqs.CPU167.TIMER 13683 ± 6% +23.0% 16824 ± 3% softirqs.CPU168.SCHED 14468 ± 5% +18.1% 17085 ± 3% softirqs.CPU169.SCHED 7703 +22.5% 9437 ± 4% softirqs.CPU17.RCU 14392 +19.8% 17237 softirqs.CPU17.SCHED 14570 ± 6% +16.9% 17038 ± 3% softirqs.CPU170.SCHED 14172 ± 5% +19.3% 16905 softirqs.CPU171.SCHED 14176 ± 3% +17.9% 16709 softirqs.CPU172.SCHED 127159 ± 2% -8.4% 116497 ± 2% softirqs.CPU172.TIMER 15034 ± 2% +11.1% 16698 ± 2% softirqs.CPU174.SCHED 14923 ± 8% +17.0% 17458 ± 5% softirqs.CPU175.SCHED 14164 ± 5% +18.8% 16833 ± 2% softirqs.CPU176.SCHED 127027 ± 2% -8.4% 116378 ± 2% softirqs.CPU176.TIMER 14359 ± 3% +16.9% 16788 ± 4% softirqs.CPU177.SCHED 14466 ± 3% +18.7% 17169 softirqs.CPU178.SCHED 14632 ± 2% +15.1% 16837 softirqs.CPU179.SCHED 123562 ± 3% -8.2% 113392 ± 2% softirqs.CPU179.TIMER 7677 +27.6% 9794 ± 2% softirqs.CPU18.RCU 14177 ± 2% +21.6% 17233 softirqs.CPU18.SCHED 14429 ± 7% +15.8% 16707 softirqs.CPU180.SCHED 15194 ± 5% +12.4% 17084 softirqs.CPU181.SCHED 13917 ± 3% +20.2% 16723 ± 3% softirqs.CPU182.SCHED 14587 ± 2% +15.1% 16786 ± 3% softirqs.CPU183.SCHED 14266 ± 3% +18.0% 16827 ± 2% softirqs.CPU184.SCHED 14661 ± 3% +16.3% 17053 softirqs.CPU185.SCHED 14222 ± 5% +21.0% 17202 ± 3% softirqs.CPU186.SCHED 15059 ± 2% +13.0% 17022 ± 3% softirqs.CPU187.SCHED 125388 ± 2% -7.6% 115887 ± 2% softirqs.CPU187.TIMER 13435 ± 4% +25.0% 16801 ± 3% softirqs.CPU188.SCHED 13434 ± 2% +28.8% 17297 ± 2% softirqs.CPU189.SCHED 7719 +27.0% 9806 ± 9% softirqs.CPU19.RCU 14075 ± 2% +20.7% 16986 softirqs.CPU19.SCHED 13735 ± 3% +20.4% 16536 ± 2% softirqs.CPU190.SCHED 13772 ± 3% +19.8% 16494 softirqs.CPU192.SCHED 13911 ± 2% +17.4% 16327 ± 2% softirqs.CPU193.SCHED 6417 +34.3% 8616 ± 17% softirqs.CPU194.RCU 14008 +17.6% 16468 ± 8% softirqs.CPU194.SCHED 14094 ± 4% +19.9% 16905 softirqs.CPU195.SCHED 14387 ± 7% +16.1% 16698 ± 5% softirqs.CPU196.SCHED 13704 ± 3% +23.7% 16952 softirqs.CPU198.SCHED 8214 ± 10% +23.7% 10157 ± 7% softirqs.CPU2.RCU 14431 +17.4% 16946 ± 4% softirqs.CPU2.SCHED 7770 +30.4% 10135 ± 8% softirqs.CPU20.RCU 13996 ± 2% +23.7% 17320 softirqs.CPU20.SCHED 14611 ± 5% +12.7% 16463 ± 3% softirqs.CPU200.SCHED 14897 ± 3% +15.5% 17199 softirqs.CPU201.SCHED 14105 ± 6% +21.7% 17167 ± 2% softirqs.CPU202.SCHED 127698 ± 3% -7.1% 118685 softirqs.CPU202.TIMER 14369 ± 6% +18.6% 17048 ± 2% softirqs.CPU203.SCHED 126668 ± 3% -8.8% 115539 softirqs.CPU203.TIMER 14461 ± 4% +13.1% 16356 ± 3% softirqs.CPU205.SCHED 13139 ± 10% +26.6% 16634 ± 3% softirqs.CPU206.SCHED 14808 ± 2% +11.8% 16550 ± 2% softirqs.CPU207.SCHED 13744 ± 5% +21.0% 16624 ± 4% softirqs.CPU208.SCHED 14039 ± 5% +18.7% 16661 ± 2% softirqs.CPU209.SCHED 127488 ± 2% -8.2% 116998 softirqs.CPU209.TIMER 7755 ± 2% +30.5% 10122 ± 12% softirqs.CPU21.RCU 13731 ± 5% +25.2% 17185 ± 2% softirqs.CPU21.SCHED 14755 ± 6% +16.1% 17123 ± 4% softirqs.CPU211.SCHED 13920 ± 3% +18.5% 16501 ± 2% softirqs.CPU212.SCHED 14435 ± 4% +16.1% 16765 ± 3% softirqs.CPU213.SCHED 13857 ± 6% +21.6% 16844 ± 2% softirqs.CPU214.SCHED 13417 ± 8% +16.5% 15635 ± 2% softirqs.CPU215.SCHED 127636 ± 3% -7.4% 118224 softirqs.CPU215.TIMER 13715 ± 3% +21.7% 16686 ± 3% softirqs.CPU216.SCHED 126421 ± 2% -7.8% 116544 softirqs.CPU216.TIMER 13885 ± 5% +18.2% 16416 softirqs.CPU217.SCHED 14003 ± 8% +20.5% 16875 softirqs.CPU218.SCHED 13891 ± 4% +23.9% 17210 ± 4% softirqs.CPU22.SCHED 14901 ± 2% +12.0% 16694 ± 2% softirqs.CPU220.SCHED 14623 ± 6% +17.3% 17153 softirqs.CPU221.SCHED 14613 ± 8% +13.9% 16643 ± 2% softirqs.CPU224.SCHED 14745 ± 4% +12.9% 16643 softirqs.CPU225.SCHED 126232 -8.3% 115717 softirqs.CPU225.TIMER 14290 ± 2% +17.2% 16753 softirqs.CPU226.SCHED 15229 ± 3% +10.3% 16800 ± 3% softirqs.CPU228.SCHED 14282 ± 6% +15.4% 16479 ± 3% softirqs.CPU23.SCHED 13785 ± 9% +24.2% 17126 ± 4% softirqs.CPU230.SCHED 126791 ± 4% -9.4% 114889 ± 3% softirqs.CPU231.TIMER 14749 ± 3% +15.0% 16958 ± 3% softirqs.CPU232.SCHED 15099 ± 5% +11.9% 16893 ± 2% softirqs.CPU233.SCHED 14825 ± 6% +12.9% 16738 softirqs.CPU234.SCHED 14272 ± 5% +18.7% 16945 ± 2% softirqs.CPU236.SCHED 15187 ± 2% +10.9% 16846 softirqs.CPU237.SCHED 15267 ± 7% +11.1% 16955 ± 3% softirqs.CPU239.SCHED 14156 ± 4% +16.3% 16466 ± 3% softirqs.CPU24.SCHED 14671 ± 3% +13.9% 16711 ± 3% softirqs.CPU240.SCHED 127318 ± 2% -7.6% 117651 ± 2% softirqs.CPU240.TIMER 14218 ± 5% +20.2% 17097 softirqs.CPU241.SCHED 14554 ± 3% +15.2% 16771 softirqs.CPU242.SCHED 14003 +19.9% 16785 softirqs.CPU243.SCHED 13761 ± 6% +21.5% 16724 ± 2% softirqs.CPU244.SCHED 14418 ± 6% +16.4% 16781 softirqs.CPU245.SCHED 14760 ± 7% +14.8% 16940 ± 4% softirqs.CPU246.SCHED 14891 ± 3% +12.3% 16722 ± 4% softirqs.CPU249.SCHED 13823 ± 4% +24.1% 17148 softirqs.CPU25.SCHED 14201 ± 4% +18.1% 16774 ± 3% softirqs.CPU250.SCHED 14620 ± 10% +18.5% 17331 softirqs.CPU251.SCHED 14877 ± 3% +11.3% 16564 ± 2% softirqs.CPU252.SCHED 14517 ± 6% +16.3% 16877 ± 3% softirqs.CPU254.SCHED 14284 ± 7% +18.0% 16858 ± 4% softirqs.CPU255.SCHED 121538 ± 3% -7.5% 112473 softirqs.CPU255.TIMER 14582 +14.8% 16736 softirqs.CPU256.SCHED 14057 ± 3% +22.8% 17259 softirqs.CPU257.SCHED 15076 ± 3% +13.4% 17097 ± 3% softirqs.CPU258.SCHED 14356 ± 3% +20.2% 17251 ± 3% softirqs.CPU259.SCHED 13998 ± 3% +20.1% 16806 ± 2% softirqs.CPU26.SCHED 13899 ± 4% +19.0% 16534 ± 5% softirqs.CPU260.SCHED 128277 ± 4% -7.4% 118758 softirqs.CPU260.TIMER 14288 ± 5% +20.7% 17251 softirqs.CPU261.SCHED 14456 ± 5% +15.1% 16644 softirqs.CPU262.SCHED 14114 ± 2% +17.8% 16624 ± 4% softirqs.CPU263.SCHED 13328 ± 4% +25.0% 16657 softirqs.CPU264.SCHED 13328 ± 4% +25.1% 16676 ± 4% softirqs.CPU266.SCHED 14367 ± 3% +19.5% 17165 ± 2% softirqs.CPU267.SCHED 122663 ± 3% -7.6% 113293 ± 2% softirqs.CPU268.TIMER 13527 ± 6% +25.2% 16940 softirqs.CPU27.SCHED 14341 ± 5% +19.8% 17187 softirqs.CPU270.SCHED 14345 ± 5% +14.3% 16403 ± 5% softirqs.CPU271.SCHED 14990 ± 3% +14.4% 17152 ± 4% softirqs.CPU272.SCHED 15007 ± 2% +13.4% 17012 softirqs.CPU273.SCHED 14668 ± 8% +17.0% 17157 ± 4% softirqs.CPU274.SCHED 14287 ± 6% +17.2% 16749 ± 4% softirqs.CPU277.SCHED 6301 +41.8% 8938 ± 16% softirqs.CPU278.RCU 13980 ± 10% +24.6% 17424 ± 2% softirqs.CPU278.SCHED 14763 ± 7% +16.0% 17119 softirqs.CPU279.SCHED 13682 ± 5% +23.9% 16950 softirqs.CPU28.SCHED 14204 ± 3% +17.0% 16617 ± 3% softirqs.CPU280.SCHED 14202 ± 3% +14.4% 16240 ± 2% softirqs.CPU281.SCHED 14629 ± 7% +16.3% 17013 ± 3% softirqs.CPU283.SCHED 13692 ± 6% +22.7% 16795 softirqs.CPU284.SCHED 14194 ± 7% +17.5% 16679 ± 3% softirqs.CPU285.SCHED 123721 ± 3% -8.1% 113741 ± 3% softirqs.CPU285.TIMER 14757 ± 7% +15.5% 17050 softirqs.CPU286.SCHED 13560 ± 6% +17.2% 15894 softirqs.CPU287.SCHED 14201 ± 4% +19.9% 17020 ± 2% softirqs.CPU29.SCHED 14345 ± 3% +20.5% 17292 softirqs.CPU3.SCHED 13910 ± 4% +20.9% 16814 ± 4% softirqs.CPU30.SCHED 7587 +41.8% 10759 ± 23% softirqs.CPU31.RCU 13401 ± 4% +26.6% 16967 ± 2% softirqs.CPU31.SCHED 13593 +24.6% 16937 softirqs.CPU32.SCHED 13175 ± 2% +29.0% 16989 softirqs.CPU33.SCHED 13491 ± 5% +25.8% 16974 ± 3% softirqs.CPU34.SCHED 13520 ± 7% +25.9% 17018 ± 2% softirqs.CPU35.SCHED 13723 ± 6% +22.1% 16763 ± 2% softirqs.CPU36.SCHED 13449 ± 3% +27.1% 17091 softirqs.CPU37.SCHED 7405 +40.0% 10368 ± 17% softirqs.CPU38.RCU 14299 ± 6% +22.4% 17509 ± 3% softirqs.CPU38.SCHED 14269 ± 3% +18.6% 16918 softirqs.CPU39.SCHED 7977 ± 2% +23.6% 9862 ± 5% softirqs.CPU4.RCU 14055 ± 2% +23.4% 17349 ± 3% softirqs.CPU4.SCHED 14257 ± 2% +19.4% 17030 ± 2% softirqs.CPU40.SCHED 13774 ± 2% +24.1% 17090 softirqs.CPU41.SCHED 14425 ± 8% +17.7% 16976 softirqs.CPU42.SCHED 14314 ± 2% +21.2% 17345 ± 2% softirqs.CPU43.SCHED 13274 ± 4% +27.7% 16954 softirqs.CPU44.SCHED 135612 ± 10% -15.7% 114264 ± 3% softirqs.CPU44.TIMER 13919 ± 2% +20.9% 16829 softirqs.CPU45.SCHED 13156 +27.9% 16834 ± 2% softirqs.CPU46.SCHED 13617 ± 2% +22.0% 16614 softirqs.CPU47.SCHED 13565 ± 3% +25.7% 17054 ± 2% softirqs.CPU48.SCHED 13791 ± 4% +21.6% 16769 softirqs.CPU49.SCHED 14260 ± 3% +20.1% 17126 ± 4% softirqs.CPU5.SCHED 13510 ± 7% +25.3% 16934 softirqs.CPU50.SCHED 12790 ± 10% +30.3% 16669 softirqs.CPU51.SCHED 13989 +19.9% 16774 ± 3% softirqs.CPU52.SCHED 14279 ± 3% +17.1% 16722 ± 2% softirqs.CPU53.SCHED 7263 +36.9% 9940 ± 21% softirqs.CPU54.RCU 14088 ± 2% +20.1% 16918 softirqs.CPU54.SCHED 13508 ± 6% +23.1% 16629 softirqs.CPU55.SCHED 130302 ± 6% -9.4% 118014 ± 7% softirqs.CPU55.TIMER 13313 ± 3% +27.9% 17030 ± 2% softirqs.CPU56.SCHED 7360 ± 2% +33.2% 9803 ± 12% softirqs.CPU57.RCU 13818 ± 2% +21.3% 16768 ± 2% softirqs.CPU57.SCHED 7296 +27.1% 9274 ± 7% softirqs.CPU58.RCU 13922 ± 3% +23.4% 17181 ± 2% softirqs.CPU58.SCHED 7271 +30.3% 9476 ± 11% softirqs.CPU59.RCU 14169 ± 5% +19.7% 16956 ± 2% softirqs.CPU59.SCHED 8176 ± 7% +16.5% 9524 ± 3% softirqs.CPU6.RCU 13689 ± 3% +25.2% 17132 ± 5% softirqs.CPU6.SCHED 13645 ± 2% +23.3% 16825 softirqs.CPU60.SCHED 13441 ± 6% +24.9% 16786 ± 2% softirqs.CPU61.SCHED 13497 ± 5% +25.7% 16959 softirqs.CPU62.SCHED 13682 ± 4% +25.5% 17170 softirqs.CPU63.SCHED 13656 ± 6% +22.3% 16695 ± 3% softirqs.CPU64.SCHED 13704 ± 7% +22.7% 16819 softirqs.CPU65.SCHED 13683 ± 3% +23.0% 16826 softirqs.CPU66.SCHED 14076 ± 2% +19.3% 16797 ± 2% softirqs.CPU67.SCHED 13713 ± 4% +18.2% 16215 ± 2% softirqs.CPU68.SCHED 13396 ± 3% +25.1% 16754 ± 2% softirqs.CPU69.SCHED 8042 ± 3% +21.0% 9730 ± 7% softirqs.CPU7.RCU 13604 ± 3% +27.3% 17320 ± 5% softirqs.CPU7.SCHED 13023 ± 2% +29.2% 16828 ± 2% softirqs.CPU70.SCHED 12307 ± 3% +32.1% 16255 softirqs.CPU71.SCHED 7727 ± 22% +37.3% 10612 ± 22% softirqs.CPU72.RCU 13320 ± 3% +28.0% 17050 ± 2% softirqs.CPU72.SCHED 13838 ± 3% +22.0% 16883 ± 3% softirqs.CPU73.SCHED 14604 ± 2% +14.4% 16707 ± 4% softirqs.CPU74.SCHED 14184 ± 4% +19.1% 16892 ± 2% softirqs.CPU75.SCHED 14138 ± 4% +19.0% 16830 softirqs.CPU76.SCHED 7098 +28.1% 9090 ± 12% softirqs.CPU77.RCU 13082 ± 4% +28.0% 16741 softirqs.CPU77.SCHED 126355 ± 4% -10.8% 112765 ± 2% softirqs.CPU78.TIMER 14718 ± 5% +12.6% 16576 ± 3% softirqs.CPU79.SCHED 7959 ± 2% +27.1% 10119 ± 10% softirqs.CPU8.RCU 13775 ± 4% +25.1% 17232 ± 5% softirqs.CPU8.SCHED 14397 +13.6% 16355 ± 2% softirqs.CPU80.SCHED 13999 ± 6% +20.7% 16896 ± 2% softirqs.CPU81.SCHED 14129 ± 7% +18.2% 16706 softirqs.CPU82.SCHED 13931 +17.6% 16385 ± 3% softirqs.CPU83.SCHED 14455 ± 5% +16.1% 16776 ± 2% softirqs.CPU84.SCHED 14426 ± 5% +16.4% 16799 softirqs.CPU86.SCHED 7049 +28.2% 9036 ± 10% softirqs.CPU88.RCU 13901 ± 3% +20.1% 16697 ± 2% softirqs.CPU88.SCHED 127258 ± 2% -7.3% 118017 softirqs.CPU88.TIMER 14353 ± 5% +17.4% 16855 ± 2% softirqs.CPU89.SCHED 14823 ± 4% +14.7% 17004 softirqs.CPU9.SCHED 14278 ± 4% +18.4% 16908 softirqs.CPU90.SCHED 14225 ± 2% +18.8% 16894 ± 2% softirqs.CPU91.SCHED 7091 +26.7% 8987 ± 10% softirqs.CPU92.RCU 13726 ± 3% +24.0% 17014 ± 3% softirqs.CPU92.SCHED 14051 ± 5% +20.6% 16939 softirqs.CPU93.SCHED 14123 ± 4% +17.9% 16649 ± 3% softirqs.CPU94.SCHED 130109 ± 5% -8.2% 119400 softirqs.CPU95.TIMER 13638 ± 4% +23.5% 16847 ± 2% softirqs.CPU96.SCHED 13134 ± 2% +29.4% 17001 ± 3% softirqs.CPU97.SCHED 13357 ± 3% +25.7% 16791 ± 2% softirqs.CPU98.SCHED 14112 ± 6% +20.0% 16938 softirqs.CPU99.SCHED 1991578 +21.8% 2426642 ± 3% softirqs.RCU 4100803 +18.4% 4855612 softirqs.SCHED 454.00 ± 27% +675.8% 3522 ±139% interrupts.30:IR-PCI-MSI.2097153-edge.eth0-TxRx-0 384622 -1.6% 378485 interrupts.CAL:Function_call_interrupts 930.25 ± 8% -46.7% 496.00 ± 10% interrupts.CPU0.TLB:TLB_shootdowns 758.50 ± 9% -44.8% 418.75 ± 11% interrupts.CPU1.TLB:TLB_shootdowns 454.00 ± 27% +675.8% 3522 ±139% interrupts.CPU10.30:IR-PCI-MSI.2097153-edge.eth0-TxRx-0 886.75 ± 5% -39.4% 537.00 ± 3% interrupts.CPU10.TLB:TLB_shootdowns 169.50 ± 11% +67.0% 283.00 ± 9% interrupts.CPU100.TLB:TLB_shootdowns 178.50 ± 20% +44.0% 257.00 ± 9% interrupts.CPU101.TLB:TLB_shootdowns 169.00 ± 23% +62.7% 275.00 ± 20% interrupts.CPU104.TLB:TLB_shootdowns 1463 ± 28% +88.1% 2753 ± 22% interrupts.CPU105.NMI:Non-maskable_interrupts 1463 ± 28% +88.1% 2753 ± 22% interrupts.CPU105.PMI:Performance_monitoring_interrupts 158.75 ± 21% +73.4% 275.25 ± 4% interrupts.CPU106.TLB:TLB_shootdowns 157.50 ± 24% +67.3% 263.50 ± 13% interrupts.CPU109.TLB:TLB_shootdowns 2884 ± 27% -43.7% 1625 ± 6% interrupts.CPU11.NMI:Non-maskable_interrupts 2884 ± 27% -43.7% 1625 ± 6% interrupts.CPU11.PMI:Performance_monitoring_interrupts 713.50 ± 5% -38.8% 436.50 ± 13% interrupts.CPU11.TLB:TLB_shootdowns 146.00 ± 38% +79.3% 261.75 ± 14% interrupts.CPU110.TLB:TLB_shootdowns 162.00 ± 14% +82.6% 295.75 ± 12% interrupts.CPU112.TLB:TLB_shootdowns 445.75 ± 52% +170.1% 1204 ± 91% interrupts.CPU114.RES:Rescheduling_interrupts 160.25 ± 37% +70.8% 273.75 ± 14% interrupts.CPU114.TLB:TLB_shootdowns 1417 ± 17% +66.0% 2352 ± 34% interrupts.CPU115.NMI:Non-maskable_interrupts 1417 ± 17% +66.0% 2352 ± 34% interrupts.CPU115.PMI:Performance_monitoring_interrupts 380.75 ± 9% +37.9% 525.00 ± 15% interrupts.CPU116.RES:Rescheduling_interrupts 183.00 ± 9% +48.2% 271.25 ± 14% interrupts.CPU116.TLB:TLB_shootdowns 170.50 ± 15% +74.5% 297.50 ± 5% interrupts.CPU118.TLB:TLB_shootdowns 454.25 ± 38% +251.3% 1596 ±103% interrupts.CPU12.RES:Rescheduling_interrupts 847.00 ± 12% -36.9% 534.25 ± 10% interrupts.CPU12.TLB:TLB_shootdowns 173.25 ± 20% +65.7% 287.00 ± 13% interrupts.CPU122.TLB:TLB_shootdowns 420.50 ± 32% +80.5% 759.00 ± 37% interrupts.CPU123.RES:Rescheduling_interrupts 247.75 ± 22% +75.9% 435.75 ± 14% interrupts.CPU124.RES:Rescheduling_interrupts 181.75 ± 33% +58.7% 288.50 ± 15% interrupts.CPU126.TLB:TLB_shootdowns 364.75 ± 21% +39.1% 507.50 ± 11% interrupts.CPU128.RES:Rescheduling_interrupts 337.75 ± 24% +248.5% 1177 ± 74% interrupts.CPU13.RES:Rescheduling_interrupts 783.25 ± 14% -47.8% 409.00 ± 3% interrupts.CPU13.TLB:TLB_shootdowns 277.00 ± 14% +84.9% 512.25 ± 10% interrupts.CPU131.RES:Rescheduling_interrupts 155.75 ± 24% +71.7% 267.50 ± 21% interrupts.CPU132.TLB:TLB_shootdowns 174.25 ± 24% +50.4% 262.00 ± 19% interrupts.CPU133.TLB:TLB_shootdowns 137.00 ± 7% +116.4% 296.50 ± 11% interrupts.CPU134.TLB:TLB_shootdowns 152.00 ± 24% +86.3% 283.25 ± 17% interrupts.CPU138.TLB:TLB_shootdowns 820.00 ± 10% -32.5% 553.75 ± 3% interrupts.CPU14.TLB:TLB_shootdowns 277.75 ± 19% +81.1% 503.00 ± 7% interrupts.CPU140.RES:Rescheduling_interrupts 298.25 ± 31% +59.3% 475.00 ± 15% interrupts.CPU142.RES:Rescheduling_interrupts 192.50 ± 22% +38.6% 266.75 ± 4% interrupts.CPU143.TLB:TLB_shootdowns 290.00 ± 38% +103.4% 590.00 ± 18% interrupts.CPU145.RES:Rescheduling_interrupts 104.50 ± 34% +90.4% 199.00 ± 16% interrupts.CPU145.TLB:TLB_shootdowns 226.50 ± 10% +96.9% 446.00 ± 15% interrupts.CPU146.RES:Rescheduling_interrupts 95.75 ± 8% +145.4% 235.00 ± 12% interrupts.CPU146.TLB:TLB_shootdowns 145.00 ± 22% +65.2% 239.50 ± 16% interrupts.CPU147.TLB:TLB_shootdowns 90.25 ± 23% +212.2% 281.75 ± 19% interrupts.CPU148.TLB:TLB_shootdowns 104.25 ± 22% +142.7% 253.00 ± 11% interrupts.CPU149.TLB:TLB_shootdowns 2888 ± 16% -35.6% 1860 ± 30% interrupts.CPU15.NMI:Non-maskable_interrupts 2888 ± 16% -35.6% 1860 ± 30% interrupts.CPU15.PMI:Performance_monitoring_interrupts 472.25 ± 31% +275.2% 1771 ±106% interrupts.CPU15.RES:Rescheduling_interrupts 816.50 ± 16% -50.0% 408.50 ± 11% interrupts.CPU15.TLB:TLB_shootdowns 84.75 ± 24% +207.4% 260.50 ± 10% interrupts.CPU150.TLB:TLB_shootdowns 133.50 ± 16% +108.6% 278.50 ± 17% interrupts.CPU151.TLB:TLB_shootdowns 103.75 ± 25% +132.0% 240.75 ± 8% interrupts.CPU152.TLB:TLB_shootdowns 105.00 ± 60% +155.0% 267.75 ± 7% interrupts.CPU153.TLB:TLB_shootdowns 100.50 ± 28% +133.1% 234.25 ± 8% interrupts.CPU154.TLB:TLB_shootdowns 73.50 ± 36% +220.4% 235.50 ± 11% interrupts.CPU155.TLB:TLB_shootdowns 229.50 ± 18% +182.1% 647.50 ± 8% interrupts.CPU156.RES:Rescheduling_interrupts 114.50 ± 20% +110.9% 241.50 ± 29% interrupts.CPU156.TLB:TLB_shootdowns 125.75 ± 27% +79.7% 226.00 ± 8% interrupts.CPU157.TLB:TLB_shootdowns 97.50 ± 21% +143.1% 237.00 ± 16% interrupts.CPU158.TLB:TLB_shootdowns 94.75 ± 49% +136.4% 224.00 ± 21% interrupts.CPU159.TLB:TLB_shootdowns 452.00 ± 36% +89.2% 855.25 ± 39% interrupts.CPU16.RES:Rescheduling_interrupts 959.00 ± 16% -54.3% 438.25 ± 8% interrupts.CPU16.TLB:TLB_shootdowns 96.75 ± 9% +145.0% 237.00 ± 9% interrupts.CPU161.TLB:TLB_shootdowns 94.25 ± 24% +169.5% 254.00 ± 10% interrupts.CPU162.TLB:TLB_shootdowns 100.50 ± 8% +141.8% 243.00 ± 14% interrupts.CPU163.TLB:TLB_shootdowns 87.00 ± 42% +186.5% 249.25 ± 8% interrupts.CPU164.TLB:TLB_shootdowns 93.50 ± 29% +145.7% 229.75 ± 9% interrupts.CPU165.TLB:TLB_shootdowns 110.50 ± 33% +165.4% 293.25 ± 15% interrupts.CPU166.TLB:TLB_shootdowns 323.25 ± 37% +105.7% 665.00 ± 39% interrupts.CPU167.RES:Rescheduling_interrupts 96.75 ± 48% +140.3% 232.50 ± 14% interrupts.CPU167.TLB:TLB_shootdowns 224.00 ± 22% +140.8% 539.50 ± 40% interrupts.CPU168.RES:Rescheduling_interrupts 87.50 ± 26% +151.1% 219.75 ± 14% interrupts.CPU168.TLB:TLB_shootdowns 1120 ± 47% +79.9% 2015 ± 27% interrupts.CPU169.NMI:Non-maskable_interrupts 1120 ± 47% +79.9% 2015 ± 27% interrupts.CPU169.PMI:Performance_monitoring_interrupts 100.25 ± 22% +117.5% 218.00 ± 9% interrupts.CPU169.TLB:TLB_shootdowns 667.75 ± 12% -30.3% 465.50 ± 14% interrupts.CPU17.TLB:TLB_shootdowns 75.00 ± 28% +252.7% 264.50 ± 22% interrupts.CPU170.TLB:TLB_shootdowns 94.25 ± 19% +126.0% 213.00 ± 13% interrupts.CPU171.TLB:TLB_shootdowns 318.25 ± 28% +53.3% 488.00 ± 18% interrupts.CPU172.RES:Rescheduling_interrupts 85.25 ± 33% +221.1% 273.75 ± 19% interrupts.CPU172.TLB:TLB_shootdowns 82.25 ± 18% +130.7% 189.75 ± 20% interrupts.CPU173.TLB:TLB_shootdowns 67.75 ± 55% +261.6% 245.00 ± 12% interrupts.CPU174.TLB:TLB_shootdowns 227.75 ± 41% +588.6% 1568 ± 95% interrupts.CPU175.RES:Rescheduling_interrupts 91.50 ± 10% +150.8% 229.50 ± 10% interrupts.CPU175.TLB:TLB_shootdowns 99.50 ± 17% +147.2% 246.00 ± 22% interrupts.CPU176.TLB:TLB_shootdowns 79.25 ± 18% +200.0% 237.75 ± 27% interrupts.CPU177.TLB:TLB_shootdowns 87.50 ± 36% +129.7% 201.00 ± 4% interrupts.CPU178.TLB:TLB_shootdowns 127.25 ± 44% +82.7% 232.50 ± 15% interrupts.CPU179.TLB:TLB_shootdowns 1161 ± 15% +137.6% 2758 ± 27% interrupts.CPU18.NMI:Non-maskable_interrupts 1161 ± 15% +137.6% 2758 ± 27% interrupts.CPU18.PMI:Performance_monitoring_interrupts 425.75 ± 15% +149.5% 1062 ± 64% interrupts.CPU18.RES:Rescheduling_interrupts 985.50 ± 13% -50.1% 492.25 ± 13% interrupts.CPU18.TLB:TLB_shootdowns 112.25 ± 33% +120.7% 247.75 ± 8% interrupts.CPU180.TLB:TLB_shootdowns 126.25 ± 15% +96.0% 247.50 ± 9% interrupts.CPU181.TLB:TLB_shootdowns 229.25 ± 25% +74.6% 400.25 ± 7% interrupts.CPU182.RES:Rescheduling_interrupts 102.50 ± 41% +173.4% 280.25 ± 4% interrupts.CPU182.TLB:TLB_shootdowns 253.25 ± 13% +52.6% 386.50 ± 8% interrupts.CPU183.RES:Rescheduling_interrupts 97.00 ± 50% +153.4% 245.75 ± 4% interrupts.CPU183.TLB:TLB_shootdowns 99.00 ± 14% +143.7% 241.25 ± 18% interrupts.CPU184.TLB:TLB_shootdowns 287.50 ± 37% +57.0% 451.25 ± 32% interrupts.CPU185.RES:Rescheduling_interrupts 92.50 ± 27% +139.2% 221.25 ± 5% interrupts.CPU185.TLB:TLB_shootdowns 2302 ± 26% +37.4% 3162 ± 5% interrupts.CPU186.NMI:Non-maskable_interrupts 2302 ± 26% +37.4% 3162 ± 5% interrupts.CPU186.PMI:Performance_monitoring_interrupts 87.75 ± 16% +204.0% 266.75 ± 20% interrupts.CPU186.TLB:TLB_shootdowns 97.00 ± 31% +157.0% 249.25 ± 10% interrupts.CPU187.TLB:TLB_shootdowns 261.50 ± 10% +54.2% 403.25 ± 15% interrupts.CPU188.RES:Rescheduling_interrupts 108.50 ± 42% +103.7% 221.00 ± 26% interrupts.CPU188.TLB:TLB_shootdowns 223.75 ± 21% +88.4% 421.50 ± 18% interrupts.CPU189.RES:Rescheduling_interrupts 119.25 ± 11% +108.4% 248.50 ± 25% interrupts.CPU189.TLB:TLB_shootdowns 400.25 ± 27% +117.4% 870.25 ± 52% interrupts.CPU19.RES:Rescheduling_interrupts 699.50 ± 16% -31.8% 476.75 ± 10% interrupts.CPU19.TLB:TLB_shootdowns 99.75 ± 43% +148.6% 248.00 ± 12% interrupts.CPU190.TLB:TLB_shootdowns 98.25 ± 10% +146.6% 242.25 ± 17% interrupts.CPU191.TLB:TLB_shootdowns 105.25 ± 29% +145.4% 258.25 ± 19% interrupts.CPU192.TLB:TLB_shootdowns 89.50 ± 23% +191.3% 260.75 ± 17% interrupts.CPU193.TLB:TLB_shootdowns 66.25 ± 25% +285.7% 255.50 ± 17% interrupts.CPU194.TLB:TLB_shootdowns 278.75 ± 10% +84.9% 515.50 ± 32% interrupts.CPU195.RES:Rescheduling_interrupts 80.00 ± 18% +247.8% 278.25 ± 14% interrupts.CPU195.TLB:TLB_shootdowns 70.25 ± 12% +246.3% 243.25 ± 11% interrupts.CPU196.TLB:TLB_shootdowns 104.75 ± 26% +147.0% 258.75 ± 23% interrupts.CPU197.TLB:TLB_shootdowns 86.75 ± 62% +188.2% 250.00 ± 20% interrupts.CPU198.TLB:TLB_shootdowns 94.25 ± 17% +150.1% 235.75 ± 19% interrupts.CPU199.TLB:TLB_shootdowns 3084 ± 4% -52.9% 1453 ± 6% interrupts.CPU2.NMI:Non-maskable_interrupts 3084 ± 4% -52.9% 1453 ± 6% interrupts.CPU2.PMI:Performance_monitoring_interrupts 556.25 ± 46% +179.6% 1555 ± 83% interrupts.CPU2.RES:Rescheduling_interrupts 891.00 ± 15% -42.6% 511.50 ± 13% interrupts.CPU2.TLB:TLB_shootdowns 364.75 ± 16% +193.9% 1072 ± 40% interrupts.CPU20.RES:Rescheduling_interrupts 825.75 ± 13% -35.7% 530.75 ± 6% interrupts.CPU20.TLB:TLB_shootdowns 129.25 ± 17% +83.0% 236.50 ± 7% interrupts.CPU200.TLB:TLB_shootdowns 88.50 ± 58% +200.0% 265.50 ± 12% interrupts.CPU201.TLB:TLB_shootdowns 116.75 ± 24% +99.8% 233.25 ± 24% interrupts.CPU202.TLB:TLB_shootdowns 254.25 ± 35% +121.8% 564.00 ± 14% interrupts.CPU203.RES:Rescheduling_interrupts 83.25 ± 27% +203.0% 252.25 ± 18% interrupts.CPU203.TLB:TLB_shootdowns 110.25 ± 16% +195.2% 325.50 ± 35% interrupts.CPU204.TLB:TLB_shootdowns 332.00 ± 44% +60.5% 532.75 ± 15% interrupts.CPU205.RES:Rescheduling_interrupts 89.00 ± 32% +165.4% 236.25 ± 11% interrupts.CPU205.TLB:TLB_shootdowns 236.25 ± 23% +112.8% 502.75 ± 10% interrupts.CPU206.RES:Rescheduling_interrupts 82.75 ± 32% +195.2% 244.25 ± 13% interrupts.CPU206.TLB:TLB_shootdowns 121.25 ± 15% +86.6% 226.25 ± 18% interrupts.CPU207.TLB:TLB_shootdowns 83.75 ± 24% +170.7% 226.75 ± 5% interrupts.CPU208.TLB:TLB_shootdowns 330.25 ± 13% +131.5% 764.50 ± 44% interrupts.CPU209.RES:Rescheduling_interrupts 75.75 ± 20% +217.8% 240.75 ± 9% interrupts.CPU209.TLB:TLB_shootdowns 778.00 ± 16% -39.2% 473.00 ± 34% interrupts.CPU21.TLB:TLB_shootdowns 268.75 ± 27% +178.4% 748.25 ± 37% interrupts.CPU210.RES:Rescheduling_interrupts 114.75 ± 16% +100.0% 229.50 ± 13% interrupts.CPU210.TLB:TLB_shootdowns 112.00 ± 29% +101.1% 225.25 ± 15% interrupts.CPU211.TLB:TLB_shootdowns 97.00 ± 58% +159.3% 251.50 ± 21% interrupts.CPU212.TLB:TLB_shootdowns 263.75 ± 29% +80.3% 475.50 ± 15% interrupts.CPU213.RES:Rescheduling_interrupts 101.25 ± 25% +98.5% 201.00 ± 28% interrupts.CPU215.TLB:TLB_shootdowns 1275 ± 12% +70.6% 2177 ± 30% interrupts.CPU216.NMI:Non-maskable_interrupts 1275 ± 12% +70.6% 2177 ± 30% interrupts.CPU216.PMI:Performance_monitoring_interrupts 278.75 ± 15% +48.7% 414.50 ± 14% interrupts.CPU216.RES:Rescheduling_interrupts 95.50 ± 26% +118.3% 208.50 ± 11% interrupts.CPU216.TLB:TLB_shootdowns 1391 ± 30% +54.2% 2146 ± 33% interrupts.CPU217.NMI:Non-maskable_interrupts 1391 ± 30% +54.2% 2146 ± 33% interrupts.CPU217.PMI:Performance_monitoring_interrupts 114.50 ± 10% +84.9% 211.75 ± 16% interrupts.CPU217.TLB:TLB_shootdowns 77.00 ± 48% +308.4% 314.50 ± 55% interrupts.CPU218.TLB:TLB_shootdowns 70.75 ± 41% +259.0% 254.00 ± 21% interrupts.CPU219.TLB:TLB_shootdowns 884.75 ± 18% -36.6% 561.25 ± 12% interrupts.CPU22.TLB:TLB_shootdowns 58.25 ± 36% +315.9% 242.25 ± 13% interrupts.CPU220.TLB:TLB_shootdowns 230.50 ± 35% +103.9% 470.00 ± 18% interrupts.CPU221.RES:Rescheduling_interrupts 61.00 ± 40% +254.1% 216.00 ± 3% interrupts.CPU221.TLB:TLB_shootdowns 114.75 ± 47% +102.6% 232.50 ± 22% interrupts.CPU223.TLB:TLB_shootdowns 94.25 ± 19% +146.9% 232.75 ± 15% interrupts.CPU224.TLB:TLB_shootdowns 92.75 ± 47% +142.6% 225.00 ± 12% interrupts.CPU225.TLB:TLB_shootdowns 84.75 ± 19% +182.0% 239.00 ± 4% interrupts.CPU226.TLB:TLB_shootdowns 91.00 ± 34% +164.8% 241.00 ± 20% interrupts.CPU227.TLB:TLB_shootdowns 2345 ± 32% -40.0% 1407 ± 17% interrupts.CPU228.NMI:Non-maskable_interrupts 2345 ± 32% -40.0% 1407 ± 17% interrupts.CPU228.PMI:Performance_monitoring_interrupts 61.75 ± 24% +232.0% 205.00 ± 10% interrupts.CPU228.TLB:TLB_shootdowns 75.00 ± 49% +188.7% 216.50 ± 14% interrupts.CPU229.TLB:TLB_shootdowns 706.25 ± 17% -38.5% 434.00 ± 19% interrupts.CPU23.TLB:TLB_shootdowns 78.50 ± 20% +208.6% 242.25 ± 11% interrupts.CPU230.TLB:TLB_shootdowns 67.50 ± 35% +214.4% 212.25 ± 20% interrupts.CPU231.TLB:TLB_shootdowns 204.25 ± 15% +141.9% 494.00 ± 11% interrupts.CPU232.RES:Rescheduling_interrupts 65.25 ± 19% +254.0% 231.00 ± 13% interrupts.CPU232.TLB:TLB_shootdowns 104.75 ± 28% +106.9% 216.75 ± 26% interrupts.CPU233.TLB:TLB_shootdowns 65.00 ± 44% +226.9% 212.50 ± 8% interrupts.CPU234.TLB:TLB_shootdowns 72.00 ± 53% +186.8% 206.50 ± 20% interrupts.CPU235.TLB:TLB_shootdowns 262.50 ± 38% +75.3% 460.25 ± 13% interrupts.CPU236.RES:Rescheduling_interrupts 72.75 ± 61% +195.5% 215.00 ± 7% interrupts.CPU236.TLB:TLB_shootdowns 78.25 ± 21% +187.2% 224.75 ± 32% interrupts.CPU237.TLB:TLB_shootdowns 92.25 ± 27% +166.9% 246.25 ± 17% interrupts.CPU238.TLB:TLB_shootdowns 120.50 ± 37% +118.5% 263.25 ± 19% interrupts.CPU239.TLB:TLB_shootdowns 871.75 ± 7% -42.3% 502.75 ± 12% interrupts.CPU24.TLB:TLB_shootdowns 63.75 ± 34% +271.0% 236.50 ± 6% interrupts.CPU240.TLB:TLB_shootdowns 77.25 ± 29% +207.4% 237.50 ± 15% interrupts.CPU241.TLB:TLB_shootdowns 49.75 ± 29% +310.6% 204.25 ± 13% interrupts.CPU242.TLB:TLB_shootdowns 76.00 ± 19% +194.7% 224.00 ± 17% interrupts.CPU243.TLB:TLB_shootdowns 80.75 ± 10% +183.6% 229.00 ± 17% interrupts.CPU244.TLB:TLB_shootdowns 86.75 ± 57% +171.8% 235.75 ± 8% interrupts.CPU245.TLB:TLB_shootdowns 56.50 ± 51% +344.2% 251.00 ± 13% interrupts.CPU246.TLB:TLB_shootdowns 270.50 ± 24% +153.1% 684.75 ± 34% interrupts.CPU247.RES:Rescheduling_interrupts 88.25 ± 24% +180.2% 247.25 ± 18% interrupts.CPU247.TLB:TLB_shootdowns 81.25 ± 37% +167.1% 217.00 ± 20% interrupts.CPU248.TLB:TLB_shootdowns 67.00 ± 26% +216.4% 212.00 ± 11% interrupts.CPU249.TLB:TLB_shootdowns 740.75 ± 11% -36.9% 467.25 ± 7% interrupts.CPU25.TLB:TLB_shootdowns 67.50 ± 29% +247.0% 234.25 ± 11% interrupts.CPU250.TLB:TLB_shootdowns 92.00 ± 63% +111.1% 194.25 ± 24% interrupts.CPU251.TLB:TLB_shootdowns 72.25 ± 45% +223.5% 233.75 ± 16% interrupts.CPU252.TLB:TLB_shootdowns 78.75 ± 29% +178.7% 219.50 ± 12% interrupts.CPU253.TLB:TLB_shootdowns 64.25 ± 58% +258.4% 230.25 ± 11% interrupts.CPU254.TLB:TLB_shootdowns 267.00 ± 17% +44.3% 385.25 ± 13% interrupts.CPU255.RES:Rescheduling_interrupts 114.25 ± 36% +106.1% 235.50 ± 12% interrupts.CPU255.TLB:TLB_shootdowns 74.00 ± 33% +213.9% 232.25 ± 13% interrupts.CPU256.TLB:TLB_shootdowns 86.00 ± 81% +158.4% 222.25 ± 19% interrupts.CPU257.TLB:TLB_shootdowns 1565 ± 38% +54.4% 2417 ± 27% interrupts.CPU258.NMI:Non-maskable_interrupts 1565 ± 38% +54.4% 2417 ± 27% interrupts.CPU258.PMI:Performance_monitoring_interrupts 252.00 ± 25% +118.6% 550.75 ± 54% interrupts.CPU258.RES:Rescheduling_interrupts 81.75 ± 21% +192.7% 239.25 ± 9% interrupts.CPU258.TLB:TLB_shootdowns 106.75 ± 37% +109.1% 223.25 ± 20% interrupts.CPU259.TLB:TLB_shootdowns 899.75 ± 8% -42.9% 514.00 ± 6% interrupts.CPU26.TLB:TLB_shootdowns 67.25 ± 41% +291.8% 263.50 ± 22% interrupts.CPU260.TLB:TLB_shootdowns 59.25 ± 34% +289.0% 230.50 ± 8% interrupts.CPU261.TLB:TLB_shootdowns 292.75 ± 12% +61.0% 471.25 ± 16% interrupts.CPU262.RES:Rescheduling_interrupts 49.75 ± 45% +400.0% 248.75 ± 14% interrupts.CPU262.TLB:TLB_shootdowns 325.00 ± 17% +52.1% 494.25 ± 27% interrupts.CPU263.RES:Rescheduling_interrupts 87.00 ± 32% +143.1% 211.50 ± 11% interrupts.CPU263.TLB:TLB_shootdowns 218.25 ± 14% +71.7% 374.75 ± 5% interrupts.CPU264.RES:Rescheduling_interrupts 75.75 ± 28% +188.8% 218.75 ± 9% interrupts.CPU264.TLB:TLB_shootdowns 56.25 ± 55% +306.2% 228.50 ± 20% interrupts.CPU265.TLB:TLB_shootdowns 80.75 ± 53% +221.1% 259.25 ± 12% interrupts.CPU266.TLB:TLB_shootdowns 271.25 ± 27% +56.2% 423.75 ± 22% interrupts.CPU267.RES:Rescheduling_interrupts 60.25 ± 48% +352.7% 272.75 ± 16% interrupts.CPU267.TLB:TLB_shootdowns 56.00 ± 43% +295.5% 221.50 ± 20% interrupts.CPU268.TLB:TLB_shootdowns 97.25 ± 14% +151.2% 244.25 ± 17% interrupts.CPU269.TLB:TLB_shootdowns 688.00 ± 13% -32.2% 466.25 ± 11% interrupts.CPU27.TLB:TLB_shootdowns 208.00 ± 27% +93.6% 402.75 ± 8% interrupts.CPU270.RES:Rescheduling_interrupts 57.50 ± 30% +368.3% 269.25 ± 19% interrupts.CPU270.TLB:TLB_shootdowns 73.75 ± 34% +288.5% 286.50 ± 14% interrupts.CPU271.TLB:TLB_shootdowns 279.25 ± 32% +37.3% 383.50 ± 10% interrupts.CPU272.RES:Rescheduling_interrupts 79.25 ± 14% +189.6% 229.50 ± 28% interrupts.CPU272.TLB:TLB_shootdowns 216.50 ± 15% +201.8% 653.50 ± 66% interrupts.CPU273.RES:Rescheduling_interrupts 65.50 ± 22% +256.1% 233.25 ± 11% interrupts.CPU273.TLB:TLB_shootdowns 74.75 ± 68% +198.3% 223.00 ± 17% interrupts.CPU274.TLB:TLB_shootdowns 68.50 ± 42% +204.4% 208.50 ± 10% interrupts.CPU275.TLB:TLB_shootdowns 59.75 ± 35% +226.8% 195.25 ± 13% interrupts.CPU276.TLB:TLB_shootdowns 292.25 ± 35% +80.2% 526.75 ± 32% interrupts.CPU277.RES:Rescheduling_interrupts 73.25 ± 39% +233.8% 244.50 ± 14% interrupts.CPU277.TLB:TLB_shootdowns 61.50 ± 60% +248.0% 214.00 ± 9% interrupts.CPU278.TLB:TLB_shootdowns 163.25 ± 15% +188.1% 470.25 ± 18% interrupts.CPU279.RES:Rescheduling_interrupts 81.50 ± 10% +142.0% 197.25 ± 8% interrupts.CPU279.TLB:TLB_shootdowns 372.75 ± 20% +139.3% 892.00 ± 59% interrupts.CPU28.RES:Rescheduling_interrupts 891.00 ± 4% -41.6% 520.50 ± 10% interrupts.CPU28.TLB:TLB_shootdowns 75.75 ± 23% +223.8% 245.25 ± 12% interrupts.CPU280.TLB:TLB_shootdowns 66.25 ± 39% +234.0% 221.25 ± 12% interrupts.CPU281.TLB:TLB_shootdowns 239.25 ± 24% +90.9% 456.75 ± 15% interrupts.CPU282.RES:Rescheduling_interrupts 63.50 ± 36% +281.9% 242.50 ± 11% interrupts.CPU282.TLB:TLB_shootdowns 75.50 ± 39% +215.2% 238.00 ± 10% interrupts.CPU283.TLB:TLB_shootdowns 269.75 ± 41% +72.2% 464.50 ± 20% interrupts.CPU284.RES:Rescheduling_interrupts 66.25 ± 51% +251.7% 233.00 ± 22% interrupts.CPU284.TLB:TLB_shootdowns 103.75 ± 33% +117.3% 225.50 ± 6% interrupts.CPU285.TLB:TLB_shootdowns 274.75 ± 25% +42.6% 391.75 ± 9% interrupts.CPU286.RES:Rescheduling_interrupts 102.50 ± 22% +106.1% 211.25 ± 3% interrupts.CPU286.TLB:TLB_shootdowns 1604 ± 9% -17.7% 1320 interrupts.CPU287.CAL:Function_call_interrupts 1860 ± 18% -26.0% 1376 ± 16% interrupts.CPU287.NMI:Non-maskable_interrupts 1860 ± 18% -26.0% 1376 ± 16% interrupts.CPU287.PMI:Performance_monitoring_interrupts 245.50 ± 19% +54.3% 378.75 ± 8% interrupts.CPU287.RES:Rescheduling_interrupts 685.00 ± 13% -35.5% 441.75 ± 4% interrupts.CPU29.TLB:TLB_shootdowns 689.75 ± 18% -32.4% 466.00 ± 6% interrupts.CPU3.TLB:TLB_shootdowns 399.75 ± 15% +149.8% 998.75 ± 69% interrupts.CPU30.RES:Rescheduling_interrupts 793.00 ± 16% -36.1% 506.75 ± 7% interrupts.CPU30.TLB:TLB_shootdowns 3182 ± 24% -56.9% 1372 ± 6% interrupts.CPU31.NMI:Non-maskable_interrupts 3182 ± 24% -56.9% 1372 ± 6% interrupts.CPU31.PMI:Performance_monitoring_interrupts 283.00 ± 10% +134.5% 663.50 ± 31% interrupts.CPU31.RES:Rescheduling_interrupts 783.50 ± 20% -43.8% 440.25 ± 3% interrupts.CPU31.TLB:TLB_shootdowns 905.25 ± 8% -39.7% 546.25 ± 19% interrupts.CPU32.TLB:TLB_shootdowns 709.25 ± 4% -35.0% 461.00 ± 14% interrupts.CPU33.TLB:TLB_shootdowns 474.50 ± 31% +92.7% 914.50 ± 50% interrupts.CPU34.RES:Rescheduling_interrupts 879.50 ± 11% -43.0% 501.50 ± 9% interrupts.CPU34.TLB:TLB_shootdowns 750.00 ± 12% -42.5% 431.25 ± 10% interrupts.CPU35.TLB:TLB_shootdowns 889.00 ± 5% -45.4% 485.25 ± 6% interrupts.CPU36.TLB:TLB_shootdowns 415.25 ± 26% +34.3% 557.75 ± 9% interrupts.CPU37.RES:Rescheduling_interrupts 818.50 ± 14% -45.1% 449.25 ± 14% interrupts.CPU37.TLB:TLB_shootdowns 453.50 ± 19% +23.7% 561.00 ± 11% interrupts.CPU38.RES:Rescheduling_interrupts 899.00 ± 5% -37.8% 558.75 ± 7% interrupts.CPU38.TLB:TLB_shootdowns 752.25 ± 14% -41.3% 441.25 ± 11% interrupts.CPU39.TLB:TLB_shootdowns 1502 ± 7% +48.7% 2234 ± 24% interrupts.CPU4.NMI:Non-maskable_interrupts 1502 ± 7% +48.7% 2234 ± 24% interrupts.CPU4.PMI:Performance_monitoring_interrupts 1012 ± 26% -47.3% 534.00 ± 6% interrupts.CPU4.TLB:TLB_shootdowns 882.25 ± 14% -41.4% 517.00 ± 17% interrupts.CPU40.TLB:TLB_shootdowns 762.50 ± 13% -35.8% 489.25 ± 4% interrupts.CPU41.TLB:TLB_shootdowns 995.75 ± 12% -49.2% 506.00 ± 8% interrupts.CPU42.TLB:TLB_shootdowns 652.50 ± 12% -35.4% 421.25 ± 11% interrupts.CPU43.TLB:TLB_shootdowns 421.25 ± 18% +102.1% 851.25 ± 47% interrupts.CPU44.RES:Rescheduling_interrupts 819.00 ± 10% -39.3% 497.25 ± 7% interrupts.CPU44.TLB:TLB_shootdowns 369.00 ± 21% +79.1% 660.75 ± 22% interrupts.CPU45.RES:Rescheduling_interrupts 801.00 ± 7% -41.6% 467.75 ± 8% interrupts.CPU45.TLB:TLB_shootdowns 786.75 ± 15% -38.5% 483.75 ± 12% interrupts.CPU46.TLB:TLB_shootdowns 716.75 ± 13% -35.4% 463.25 ± 11% interrupts.CPU47.TLB:TLB_shootdowns 813.25 ± 9% -34.6% 531.75 ± 6% interrupts.CPU48.TLB:TLB_shootdowns 658.50 ± 6% -26.2% 486.25 ± 6% interrupts.CPU49.TLB:TLB_shootdowns 735.25 ± 20% -36.6% 466.00 ± 18% interrupts.CPU5.TLB:TLB_shootdowns 852.50 ± 17% -43.0% 485.75 ± 9% interrupts.CPU50.TLB:TLB_shootdowns 815.75 ± 8% -40.3% 487.25 ± 12% interrupts.CPU51.TLB:TLB_shootdowns 899.50 ± 9% -42.5% 517.50 ± 10% interrupts.CPU52.TLB:TLB_shootdowns 654.50 ± 9% -33.8% 433.00 ± 19% interrupts.CPU53.TLB:TLB_shootdowns 920.00 ± 8% -40.9% 543.50 ± 6% interrupts.CPU54.TLB:TLB_shootdowns 687.25 ± 19% -31.5% 470.75 ± 16% interrupts.CPU55.TLB:TLB_shootdowns 2467 ± 22% -41.5% 1442 ± 13% interrupts.CPU56.NMI:Non-maskable_interrupts 2467 ± 22% -41.5% 1442 ± 13% interrupts.CPU56.PMI:Performance_monitoring_interrupts 342.00 ± 7% +106.1% 705.00 ± 30% interrupts.CPU56.RES:Rescheduling_interrupts 884.75 ± 19% -45.2% 485.25 ± 8% interrupts.CPU56.TLB:TLB_shootdowns 772.00 ± 11% -43.0% 440.00 ± 11% interrupts.CPU57.TLB:TLB_shootdowns 942.50 ± 11% -46.3% 506.00 ± 10% interrupts.CPU58.TLB:TLB_shootdowns 754.25 ± 5% -38.0% 467.50 interrupts.CPU59.TLB:TLB_shootdowns 393.75 ± 9% +311.4% 1619 ±109% interrupts.CPU6.RES:Rescheduling_interrupts 860.25 ± 12% -46.6% 459.00 ± 7% interrupts.CPU6.TLB:TLB_shootdowns 894.50 ± 16% -43.4% 506.00 ± 13% interrupts.CPU60.TLB:TLB_shootdowns 1684 ± 32% +78.1% 2999 ± 14% interrupts.CPU61.NMI:Non-maskable_interrupts 1684 ± 32% +78.1% 2999 ± 14% interrupts.CPU61.PMI:Performance_monitoring_interrupts 693.50 ± 17% -34.9% 451.75 ± 9% interrupts.CPU61.TLB:TLB_shootdowns 936.25 ± 16% -49.3% 474.75 ± 8% interrupts.CPU62.TLB:TLB_shootdowns 421.25 ± 44% +112.9% 897.00 ± 11% interrupts.CPU63.RES:Rescheduling_interrupts 731.25 ± 8% -41.1% 431.00 ± 7% interrupts.CPU63.TLB:TLB_shootdowns 883.00 ± 15% -43.7% 496.75 ± 7% interrupts.CPU64.TLB:TLB_shootdowns 736.00 ± 11% -36.8% 465.50 ± 5% interrupts.CPU65.TLB:TLB_shootdowns 880.00 ± 7% -42.6% 505.25 ± 8% interrupts.CPU66.TLB:TLB_shootdowns 943.50 ± 11% -40.5% 561.75 ± 10% interrupts.CPU68.TLB:TLB_shootdowns 393.25 ± 36% +100.5% 788.50 ± 26% interrupts.CPU69.RES:Rescheduling_interrupts 790.25 ± 9% -38.3% 487.50 ± 8% interrupts.CPU69.TLB:TLB_shootdowns 415.50 ± 24% +407.0% 2106 ±123% interrupts.CPU7.RES:Rescheduling_interrupts 679.00 ± 14% -36.8% 429.25 interrupts.CPU7.TLB:TLB_shootdowns 350.25 ± 13% +92.1% 672.75 ± 12% interrupts.CPU70.RES:Rescheduling_interrupts 778.75 ± 3% -41.8% 453.50 ± 11% interrupts.CPU70.TLB:TLB_shootdowns 673.50 ± 8% -39.8% 405.75 ± 11% interrupts.CPU71.TLB:TLB_shootdowns 460.75 ± 74% +95.7% 901.75 ± 43% interrupts.CPU74.RES:Rescheduling_interrupts 160.75 ± 33% +82.1% 292.75 ± 15% interrupts.CPU76.TLB:TLB_shootdowns 198.00 ± 22% +45.8% 288.75 ± 9% interrupts.CPU77.TLB:TLB_shootdowns 2305 ± 26% -35.5% 1486 ± 12% interrupts.CPU79.NMI:Non-maskable_interrupts 2305 ± 26% -35.5% 1486 ± 12% interrupts.CPU79.PMI:Performance_monitoring_interrupts 155.25 ± 31% +72.9% 268.50 ± 8% interrupts.CPU79.TLB:TLB_shootdowns 887.25 ± 17% -45.4% 484.50 ± 10% interrupts.CPU8.TLB:TLB_shootdowns 192.00 ± 18% +32.9% 255.25 ± 12% interrupts.CPU80.TLB:TLB_shootdowns 158.75 ± 30% +95.1% 309.75 ± 12% interrupts.CPU82.TLB:TLB_shootdowns 265.75 ± 35% +120.8% 586.75 ± 30% interrupts.CPU84.RES:Rescheduling_interrupts 146.75 ± 29% +80.2% 264.50 ± 20% interrupts.CPU84.TLB:TLB_shootdowns 186.00 ± 31% +56.0% 290.25 ± 9% interrupts.CPU86.TLB:TLB_shootdowns 169.25 ± 15% +66.5% 281.75 ± 2% interrupts.CPU88.TLB:TLB_shootdowns 879.75 ± 10% -47.5% 462.00 ± 4% interrupts.CPU9.TLB:TLB_shootdowns 161.00 ± 20% +75.8% 283.00 ± 5% interrupts.CPU90.TLB:TLB_shootdowns 194.75 ± 12% +50.3% 292.75 ± 11% interrupts.CPU91.TLB:TLB_shootdowns 2722 ± 34% -37.8% 1692 ± 6% interrupts.CPU92.NMI:Non-maskable_interrupts 2722 ± 34% -37.8% 1692 ± 6% interrupts.CPU92.PMI:Performance_monitoring_interrupts 190.75 ± 14% +36.3% 260.00 ± 22% interrupts.CPU92.TLB:TLB_shootdowns 188.00 ± 15% +57.0% 295.25 ± 10% interrupts.CPU94.TLB:TLB_shootdowns 200.25 ± 11% +59.6% 319.50 ± 16% interrupts.CPU95.TLB:TLB_shootdowns 316.25 ± 55% +105.1% 648.50 ± 19% interrupts.CPU96.RES:Rescheduling_interrupts 143.75 ± 42% +91.0% 274.50 ± 8% interrupts.CPU96.TLB:TLB_shootdowns 2409 ± 41% -34.2% 1586 ± 46% interrupts.CPU97.NMI:Non-maskable_interrupts 2409 ± 41% -34.2% 1586 ± 46% interrupts.CPU97.PMI:Performance_monitoring_interrupts 186.00 ± 33% +56.0% 290.25 ± 16% interrupts.CPU98.TLB:TLB_shootdowns vm-scalability.time.user_time 16000 +-------------------------------------------------------------------+ | + | 15000 |-+ :+ + + + | 14000 |-+ .++. +. + + +.+ :+ .++ ++. +.++ + + :+ + +| |+.+++ ++ + + + ++.+++ +.+++.++.+ ++ + +.+ + :+ | 13000 |-+ + | | | 12000 |-+ | | | 11000 |-+ | 10000 |-+ | | OO O O | 9000 |-+ OO O OO O OO O | |O OOO O O OO | 8000 +-------------------------------------------------------------------+ vm-scalability.time.voluntary_context_switches 1e+06 +------------------------------------------------------------------+ | O O | 900000 |O+OOO OO O OOO OOO O O OO | 800000 |-+ | | | 700000 |-+ | 600000 |-+ | | | 500000 |-+ | 400000 |-+ | | | 300000 |-+ | 200000 |-+ | |+.+++.+++.+ + +.+++.++.+++.+++.+++.+++. .+ +. | 100000 +------------------------------------------------------------------+ vm-scalability.throughput 1.6e+07 +-----------------------------------------------------------------+ | | 1.5e+07 |-+ + + + | 1.4e+07 |+. + + +. + + +. + + | | +++ +.+ +.+++ .++ +++ +.++ +++ +.+ +. | 1.3e+07 |-+ + +.+++ +++.++ +++.+++.+| 1.2e+07 |-+ | | | 1.1e+07 |-+ | 1e+07 |-+ | | | 9e+06 |-+ O O O O | 8e+06 |-+ O OO O OO O OO OO O | |O O O O O | 7e+06 +-----------------------------------------------------------------+ vm-scalability.median 44000 +-------------------------------------------------------------------+ 42000 |-+ + + + | | : : : : :: | 40000 |+. : +. + ++.+ .++. : + .+++. : +. + | 38000 |-++++ + +.+ ++.+++ +++ + ++.+ + +. .+++. +. ++. | | +++ + + +| 36000 |-+ | 34000 |-+ | 32000 |-+ | | | 30000 |-+ | 28000 |-+ O O O | | O O O O | 26000 |O+OOO O O OO OOO O O O | 24000 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

11 months, 1 week

3
10
0 / 0

[fs] b6509f6a8c: will-it-scale.per_thread_ops -12.6% regression

by kernel test robot

Greeting, FYI, we noticed a -12.6% regression of will-it-scale.per_thread_ops due to commit: commit: b6509f6a8c4313c068c69785c001451415969e44 ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory with following parameters: nr_task: 100% mode: thread test: eventfd1 cpufreq_governor: performance ucode: 0x5002f01 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_process_ops -6.4% regression | | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=process | | | nr_task=100% | | | test=unix1 | | | ucode=0x5002f01 | +------------------+---------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops -2.3% regression | | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=100% | | | test=pipe1 | | | ucode=0x5002f01 | +------------------+---------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/eventfd1/will-it-scale/0x5002f01 commit: v5.8-rc3 b6509f6a8c ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"") v5.8-rc3 b6509f6a8c4313c068c69785c00 ---------------- --------------------------- %stddev %change %stddev \ | \ 1652352 -12.6% 1444002 ± 2% will-it-scale.per_thread_ops 3.173e+08 -12.6% 2.772e+08 ± 2% will-it-scale.workload 23.95 -3.1 20.90 ± 2% mpstat.cpu.all.usr% 23.00 -12.0% 20.25 ± 4% vmstat.cpu.us 578849 ± 4% +8.6% 628428 ± 4% sched_debug.cpu.max_idle_balance_cost.max 0.00 ± 9% -14.4% 0.00 ± 13% sched_debug.cpu.next_balance.stddev 2207 ± 24% +34.7% 2973 ± 20% numa-vmstat.node0.nr_mapped 6949 ± 22% +43.7% 9984 ± 10% numa-vmstat.node0.nr_slab_reclaimable 9739 ± 13% -29.0% 6913 ± 12% numa-vmstat.node1.nr_slab_reclaimable 7247 ± 13% -28.4% 5190 ± 19% slabinfo.eventpoll_pwq.active_objs 7247 ± 13% -28.4% 5190 ± 19% slabinfo.eventpoll_pwq.num_objs 960.00 ± 11% -40.0% 576.00 ± 26% slabinfo.kmalloc-rcl-128.active_objs 960.00 ± 11% -40.0% 576.00 ± 26% slabinfo.kmalloc-rcl-128.num_objs 27801 ± 22% +43.7% 39940 ± 10% numa-meminfo.node0.KReclaimable 8718 ± 25% +35.0% 11773 ± 20% numa-meminfo.node0.Mapped 27801 ± 22% +43.7% 39940 ± 10% numa-meminfo.node0.SReclaimable 38959 ± 13% -29.0% 27656 ± 12% numa-meminfo.node1.KReclaimable 38959 ± 13% -29.0% 27656 ± 12% numa-meminfo.node1.SReclaimable 107406 ± 2% -13.6% 92844 ± 6% numa-meminfo.node1.Slab 0.00 +1.1 1.09 ± 16% perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.3 1.26 ± 18% perf-profile.calltrace.cycles-pp.fsnotify_parent.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.00 +1.8 1.76 ± 21% perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +3.7 3.70 ± 20% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +3.0 3.02 ± 17% perf-profile.children.cycles-pp.fsnotify_parent 0.00 +5.5 5.55 ± 20% perf-profile.children.cycles-pp.fsnotify 0.00 +2.7 2.66 ± 17% perf-profile.self.cycles-pp.fsnotify_parent 0.00 +5.4 5.35 ± 20% perf-profile.self.cycles-pp.fsnotify 10890 ± 21% +53.7% 16738 ± 12% interrupts.CPU168.CAL:Function_call_interrupts 4602 ± 5% +22.7% 5649 ± 13% interrupts.CPU169.CAL:Function_call_interrupts 4920 ± 13% -21.3% 3874 ± 2% interrupts.CPU170.CAL:Function_call_interrupts 463.80 ± 11% -25.1% 347.50 ± 4% interrupts.CPU170.RES:Rescheduling_interrupts 3619 ± 2% +10.2% 3990 ± 9% interrupts.CPU36.CAL:Function_call_interrupts 5768 ± 34% +43.3% 8267 ± 2% interrupts.CPU74.NMI:Non-maskable_interrupts 5768 ± 34% +43.3% 8267 ± 2% interrupts.CPU74.PMI:Performance_monitoring_interrupts 5769 ± 34% +43.3% 8268 ± 2% interrupts.CPU75.NMI:Non-maskable_interrupts 5769 ± 34% +43.3% 8268 ± 2% interrupts.CPU75.PMI:Performance_monitoring_interrupts 5769 ± 34% +43.3% 8267 ± 2% interrupts.CPU76.NMI:Non-maskable_interrupts 5769 ± 34% +43.3% 8267 ± 2% interrupts.CPU76.PMI:Performance_monitoring_interrupts 5768 ± 34% +43.3% 8268 ± 2% interrupts.CPU77.NMI:Non-maskable_interrupts 5768 ± 34% +43.3% 8268 ± 2% interrupts.CPU77.PMI:Performance_monitoring_interrupts 3602 ± 2% +12.9% 4068 ± 13% interrupts.CPU81.CAL:Function_call_interrupts 0.05 ± 32% +121.5% 0.10 ± 39% perf-stat.i.MPKI 1.06 -0.1 0.94 perf-stat.i.branch-miss-rate% 1.107e+09 -13.3% 9.596e+08 ± 2% perf-stat.i.branch-misses 199835 -11.5% 176905 ± 2% perf-stat.i.dTLB-store-misses 9.558e+10 +9.4% 1.046e+11 ± 2% perf-stat.i.dTLB-stores 9.687e+08 -13.8% 8.345e+08 ± 2% perf-stat.i.iTLB-load-misses 535.53 +20.3% 644.34 perf-stat.i.instructions-per-iTLB-miss 1.10 ± 12% +79.6% 1.98 ± 2% perf-stat.i.metric.K/sec 1.05 -0.1 0.92 perf-stat.overall.branch-miss-rate% 0.00 ± 13% -0.0 0.00 ± 4% perf-stat.overall.dTLB-load-miss-rate% 0.00 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate% 528.82 +20.8% 638.59 perf-stat.overall.instructions-per-iTLB-miss 487060 +20.3% 586095 perf-stat.overall.path-length 1.103e+09 -13.3% 9.564e+08 ± 2% perf-stat.ps.branch-misses 199661 -11.5% 176727 ± 2% perf-stat.ps.dTLB-store-misses 9.525e+10 +9.5% 1.043e+11 ± 2% perf-stat.ps.dTLB-stores 9.67e+08 -13.9% 8.33e+08 ± 3% perf-stat.ps.iTLB-load-misses will-it-scale.per_thread_ops 1.7e+06 +----------------------------------------------------------------+ | .+.. .+.+.. .+.. .+ | 1.65e+06 |.+ .+.+ + +.+. .+.+ +. .+.+. | | +. +. + | 1.6e+06 |-+ | | | 1.55e+06 |-+ | | | 1.5e+06 |-+ | | O O O O O O | 1.45e+06 |-O O O O O O O O O O O O O O O O O O O O | | O | 1.4e+06 |-+ | | | 1.35e+06 +----------------------------------------------------------------+ will-it-scale.workload 3.3e+08 +-----------------------------------------------------------------+ | | 3.2e+08 |.+ .+.+..+.+..+.+.+.. .+..+.+. .+.+ | | +..+ +.+ +..+ | 3.1e+08 |-+ | | | 3e+08 |-+ | | | 2.9e+08 |-+ | | | 2.8e+08 |-O O O O O O O O O O O O O O O O O O O O O O O O | | O O O | 2.7e+08 |-+ | | | 2.6e+08 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-csl-2ap4: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/unix1/will-it-scale/0x5002f01 commit: v5.8-rc3 b6509f6a8c ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"") v5.8-rc3 b6509f6a8c4313c068c69785c00 ---------------- --------------------------- %stddev %change %stddev \ | \ 720286 -6.4% 674507 will-it-scale.per_process_ops 1.383e+08 -6.4% 1.295e+08 will-it-scale.workload 97977 +1.4% 99342 proc-vmstat.nr_anon_pages 8.31 ± 41% +48.5% 12.33 ± 3% sched_debug.cpu.sched_goidle.min 1097 ± 8% +19.8% 1315 ± 9% slabinfo.task_group.active_objs 1097 ± 8% +19.8% 1315 ± 9% slabinfo.task_group.num_objs 13194 ± 25% -29.6% 9295 ± 24% softirqs.CPU124.RCU 9178 ± 8% -7.8% 8467 ± 4% softirqs.CPU48.RCU 1229 ± 68% +156.3% 3151 ± 70% interrupts.CPU0.CAL:Function_call_interrupts 8259 -37.6% 5157 ± 34% interrupts.CPU163.NMI:Non-maskable_interrupts 8259 -37.6% 5157 ± 34% interrupts.CPU163.PMI:Performance_monitoring_interrupts 15337 ± 81% +146.4% 37788 ± 45% numa-vmstat.node2.nr_active_anon 15187 ± 81% +147.5% 37594 ± 45% numa-vmstat.node2.nr_anon_pages 15337 ± 81% +146.4% 37788 ± 45% numa-vmstat.node2.nr_zone_active_anon 127540 ± 6% -8.0% 117311 numa-vmstat.node2.numa_other 62308 ± 81% +144.0% 152033 ± 45% numa-meminfo.node2.Active 61336 ± 81% +146.7% 151288 ± 45% numa-meminfo.node2.Active(anon) 15647 ±195% +497.8% 93536 ± 57% numa-meminfo.node2.AnonHugePages 60733 ± 82% +147.8% 150509 ± 45% numa-meminfo.node2.AnonPages 803985 ± 8% +10.5% 888495 ± 9% numa-meminfo.node2.MemUsed 1.14 -0.1 1.07 perf-stat.i.branch-miss-rate% 1.223e+09 -6.3% 1.146e+09 perf-stat.i.branch-misses 0.99 -1.5% 0.98 perf-stat.i.cpi 1.619e+11 +1.4% 1.641e+11 perf-stat.i.dTLB-loads 1e+11 +4.0% 1.04e+11 perf-stat.i.dTLB-stores 1.06e+09 -7.8% 9.779e+08 perf-stat.i.iTLB-load-misses 5.531e+11 +1.3% 5.603e+11 perf-stat.i.instructions 527.69 +9.7% 578.78 perf-stat.i.instructions-per-iTLB-miss 1.01 +1.4% 1.02 perf-stat.i.ipc 1931 +1.4% 1958 perf-stat.i.metric.M/sec 1.12 -0.1 1.06 perf-stat.overall.branch-miss-rate% 0.99 -1.4% 0.97 perf-stat.overall.cpi 0.00 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate% 521.71 +9.8% 572.92 perf-stat.overall.instructions-per-iTLB-miss 1.01 +1.4% 1.03 perf-stat.overall.ipc 1206072 +8.1% 1303735 perf-stat.overall.path-length 1.218e+09 -6.3% 1.142e+09 perf-stat.ps.branch-misses 1.613e+11 +1.4% 1.635e+11 perf-stat.ps.dTLB-loads 9.97e+10 +4.0% 1.036e+11 perf-stat.ps.dTLB-stores 1.057e+09 -7.8% 9.747e+08 perf-stat.ps.iTLB-load-misses 5.512e+11 +1.3% 5.584e+11 perf-stat.ps.instructions 1.668e+14 +1.2% 1.688e+14 perf-stat.total.instructions 38.39 -2.2 36.14 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 36.80 -2.1 34.66 perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 34.46 -2.0 32.44 perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write 33.29 -1.8 31.47 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.96 -1.8 29.19 perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write 31.72 -1.7 30.05 perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64 25.95 -1.5 24.46 perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read 53.97 -1.5 52.49 perf-profile.calltrace.cycles-pp.write 24.74 -1.4 23.30 perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read 47.12 -1.0 46.10 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 46.34 -1.0 45.38 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 44.29 -0.9 43.43 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 42.43 -0.8 41.64 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 13.05 -0.8 12.28 perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 11.31 -0.7 10.65 perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 10.31 -0.6 9.71 perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg 8.93 -0.5 8.39 perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 8.64 -0.5 8.14 perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter 8.31 -0.5 7.85 perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg 7.91 -0.5 7.45 perf-profile.calltrace.cycles-pp.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 6.55 -0.4 6.19 perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 0.54 ± 2% -0.3 0.25 ±100% perf-profile.calltrace.cycles-pp.___might_sleep.__might_fault._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg 3.81 -0.2 3.58 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.write 3.98 -0.2 3.75 perf-profile.calltrace.cycles-pp.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg 3.63 -0.2 3.41 perf-profile.calltrace.cycles-pp._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 3.88 -0.2 3.65 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.read 3.82 -0.2 3.60 perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter 2.87 -0.2 2.66 perf-profile.calltrace.cycles-pp.security_socket_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write 3.71 -0.2 3.50 perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg 3.62 -0.2 3.42 perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic 3.91 -0.2 3.71 perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic 2.95 -0.2 2.76 perf-profile.calltrace.cycles-pp.security_socket_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read.vfs_read 3.50 -0.2 3.31 perf-profile.calltrace.cycles-pp.__check_object_size.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 3.33 -0.2 3.15 perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb 3.44 -0.2 3.25 perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic 3.09 -0.2 2.91 perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg 3.65 -0.2 3.47 perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor 2.38 -0.2 2.21 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.08 -0.1 1.94 perf-profile.calltrace.cycles-pp.aa_sk_perm.security_socket_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 2.63 -0.1 2.50 perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb 3.44 -0.1 3.31 perf-profile.calltrace.cycles-pp.sock_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read 1.71 -0.1 1.59 perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64 1.69 -0.1 1.58 perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64 1.96 -0.1 1.85 perf-profile.calltrace.cycles-pp.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read 1.44 -0.1 1.34 perf-profile.calltrace.cycles-pp.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 1.57 -0.1 1.48 perf-profile.calltrace.cycles-pp.copyin._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg 1.47 -0.1 1.39 perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor 1.20 -0.1 1.13 perf-profile.calltrace.cycles-pp.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 0.70 -0.1 0.63 ± 4% perf-profile.calltrace.cycles-pp.__might_sleep.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb 1.10 ± 2% -0.1 1.03 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.98 -0.1 0.91 perf-profile.calltrace.cycles-pp.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 1.01 ± 2% -0.1 0.94 ± 3% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.01 -0.1 0.94 perf-profile.calltrace.cycles-pp.__might_fault._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg 1.13 -0.1 1.07 perf-profile.calltrace.cycles-pp.unix_write_space.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all 1.05 -0.1 0.99 ± 3% perf-profile.calltrace.cycles-pp.__ksize.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg 0.96 -0.1 0.91 perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 0.96 -0.1 0.90 perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 0.98 -0.1 0.93 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.write 1.04 -0.1 0.98 perf-profile.calltrace.cycles-pp.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 1.00 -0.1 0.94 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.read 0.80 -0.1 0.75 perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 1.02 -0.0 0.97 perf-profile.calltrace.cycles-pp.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write 0.85 -0.0 0.80 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter 0.68 ± 2% -0.0 0.64 perf-profile.calltrace.cycles-pp.__prepare_exit_to_usermode.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.71 -0.0 0.67 perf-profile.calltrace.cycles-pp.___might_sleep.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags 0.70 ± 2% -0.0 0.66 ± 2% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyin._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg 0.85 -0.0 0.81 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 0.56 -0.0 0.52 ± 2% perf-profile.calltrace.cycles-pp.apparmor_socket_recvmsg.security_socket_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read 0.57 -0.0 0.54 perf-profile.calltrace.cycles-pp.___might_sleep.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg.sock_write_iter 0.68 -0.0 0.64 ± 2% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter 0.57 -0.0 0.54 perf-profile.calltrace.cycles-pp.__might_sleep.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags 1.68 -0.0 1.64 perf-profile.calltrace.cycles-pp.mutex_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 0.65 -0.0 0.62 perf-profile.calltrace.cycles-pp.mutex_unlock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read 0.00 +0.6 0.57 perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.6 0.62 perf-profile.calltrace.cycles-pp.fsnotify_parent.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.00 +0.6 0.64 perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.50 +0.7 3.20 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.9 0.93 perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 48.49 +1.3 49.77 perf-profile.calltrace.cycles-pp.read 41.56 +1.7 43.30 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 40.78 +1.8 42.56 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 38.81 +1.9 40.68 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 37.05 +1.9 38.94 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 0.00 +1.9 1.90 perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 38.49 -2.2 36.24 perf-profile.children.cycles-pp.new_sync_write 36.86 -2.1 34.71 perf-profile.children.cycles-pp.sock_write_iter 34.57 -2.0 32.53 perf-profile.children.cycles-pp.sock_sendmsg 33.36 -1.8 31.55 perf-profile.children.cycles-pp.new_sync_read 31.28 -1.8 29.50 perf-profile.children.cycles-pp.unix_stream_sendmsg 31.77 -1.7 30.10 perf-profile.children.cycles-pp.sock_read_iter 26.00 -1.5 24.50 perf-profile.children.cycles-pp.unix_stream_recvmsg 54.00 -1.5 52.53 perf-profile.children.cycles-pp.write 24.99 -1.5 23.53 perf-profile.children.cycles-pp.unix_stream_read_generic 44.34 -0.9 43.48 perf-profile.children.cycles-pp.ksys_write 13.14 -0.8 12.36 perf-profile.children.cycles-pp.sock_alloc_send_pskb 42.57 -0.7 41.85 perf-profile.children.cycles-pp.vfs_write 11.37 -0.7 10.71 perf-profile.children.cycles-pp.alloc_skb_with_frags 10.51 -0.6 9.90 perf-profile.children.cycles-pp.__alloc_skb 9.00 -0.5 8.46 perf-profile.children.cycles-pp.unix_stream_read_actor 8.72 -0.5 8.21 perf-profile.children.cycles-pp.skb_copy_datagram_iter 8.00 -0.5 7.53 perf-profile.children.cycles-pp.skb_copy_datagram_from_iter 8.38 -0.5 7.91 perf-profile.children.cycles-pp.__skb_datagram_iter 7.43 -0.4 7.04 perf-profile.children.cycles-pp.__check_object_size 6.57 -0.4 6.21 perf-profile.children.cycles-pp.consume_skb 5.14 -0.3 4.83 perf-profile.children.cycles-pp.entry_SYSCALL_64 4.60 -0.3 4.32 perf-profile.children.cycles-pp.syscall_return_via_sysret 4.16 -0.3 3.89 perf-profile.children.cycles-pp.aa_sk_perm 4.22 -0.3 3.96 perf-profile.children.cycles-pp.___might_sleep 3.50 -0.2 3.26 perf-profile.children.cycles-pp.common_file_perm 4.07 -0.2 3.83 perf-profile.children.cycles-pp.__kmalloc_reserve 3.71 -0.2 3.48 perf-profile.children.cycles-pp._copy_from_iter 3.85 -0.2 3.63 perf-profile.children.cycles-pp.skb_release_all 2.95 -0.2 2.73 perf-profile.children.cycles-pp.__might_sleep 2.91 -0.2 2.70 perf-profile.children.cycles-pp.security_socket_sendmsg 3.72 -0.2 3.51 perf-profile.children.cycles-pp._copy_to_iter 3.76 -0.2 3.55 perf-profile.children.cycles-pp.skb_release_head_state 3.01 -0.2 2.81 perf-profile.children.cycles-pp.security_socket_recvmsg 3.54 -0.2 3.33 perf-profile.children.cycles-pp.__kmalloc_node_track_caller 3.97 -0.2 3.77 perf-profile.children.cycles-pp.simple_copy_to_iter 3.28 -0.2 3.09 perf-profile.children.cycles-pp.kmem_cache_alloc_node 3.54 -0.2 3.35 perf-profile.children.cycles-pp.unix_destruct_scm 2.36 ± 2% -0.2 2.21 ± 2% perf-profile.children.cycles-pp.__fdget_pos 2.64 -0.1 2.50 perf-profile.children.cycles-pp.sock_wfree 3.50 -0.1 3.37 perf-profile.children.cycles-pp.sock_recvmsg 2.01 ± 2% -0.1 1.88 ± 3% perf-profile.children.cycles-pp.__fget_light 2.10 ± 2% -0.1 1.97 ± 2% perf-profile.children.cycles-pp.__might_fault 1.97 -0.1 1.85 perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 1.54 -0.1 1.44 perf-profile.children.cycles-pp.wait_for_unix_gc 1.95 -0.1 1.85 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 1.43 -0.1 1.34 perf-profile.children.cycles-pp._cond_resched 1.60 -0.1 1.51 perf-profile.children.cycles-pp.copyin 1.52 -0.1 1.43 perf-profile.children.cycles-pp.__prepare_exit_to_usermode 1.67 ± 3% -0.1 1.58 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 1.64 -0.1 1.55 perf-profile.children.cycles-pp.copy_user_generic_unrolled 1.50 -0.1 1.42 perf-profile.children.cycles-pp.copyout 1.21 -0.1 1.13 perf-profile.children.cycles-pp.sock_def_readable 0.98 -0.1 0.91 perf-profile.children.cycles-pp.skb_set_owner_w 0.95 -0.1 0.89 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 1.05 -0.1 0.99 ± 2% perf-profile.children.cycles-pp.__ksize 0.84 -0.1 0.78 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 1.13 -0.1 1.08 perf-profile.children.cycles-pp.unix_write_space 1.05 -0.1 0.99 perf-profile.children.cycles-pp.skb_unlink 0.74 ± 2% -0.1 0.69 perf-profile.children.cycles-pp.apparmor_file_permission 1.02 -0.1 0.97 perf-profile.children.cycles-pp.skb_queue_tail 1.75 -0.0 1.70 perf-profile.children.cycles-pp.mutex_lock 0.56 -0.0 0.52 ± 2% perf-profile.children.cycles-pp.apparmor_socket_recvmsg 0.71 -0.0 0.67 perf-profile.children.cycles-pp.rcu_all_qs 0.67 -0.0 0.63 perf-profile.children.cycles-pp.__syscall_return_slowpath 0.67 -0.0 0.64 perf-profile.children.cycles-pp.check_stack_object 0.44 -0.0 0.41 perf-profile.children.cycles-pp.apparmor_socket_sendmsg 0.67 -0.0 0.65 perf-profile.children.cycles-pp.mutex_unlock 0.30 -0.0 0.28 ± 2% perf-profile.children.cycles-pp.put_pid 0.23 -0.0 0.21 ± 2% perf-profile.children.cycles-pp.skb_release_data 0.43 +0.0 0.46 perf-profile.children.cycles-pp.apparmor_socket_getpeersec_dgram 4.96 +0.6 5.57 perf-profile.children.cycles-pp.security_file_permission 88.79 +0.7 89.51 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 87.30 +0.8 88.11 perf-profile.children.cycles-pp.do_syscall_64 48.55 +1.3 49.85 perf-profile.children.cycles-pp.read 0.00 +1.9 1.87 perf-profile.children.cycles-pp.fsnotify_parent 38.85 +1.9 40.72 perf-profile.children.cycles-pp.ksys_read 37.17 +2.0 39.17 perf-profile.children.cycles-pp.vfs_read 0.00 +2.9 2.90 perf-profile.children.cycles-pp.fsnotify 4.56 -0.3 4.28 perf-profile.self.cycles-pp.syscall_return_via_sysret 4.10 -0.2 3.85 perf-profile.self.cycles-pp.___might_sleep 2.96 -0.2 2.75 perf-profile.self.cycles-pp.unix_stream_read_generic 2.65 -0.2 2.46 perf-profile.self.cycles-pp.__might_sleep 2.94 -0.2 2.75 perf-profile.self.cycles-pp.unix_stream_sendmsg 2.85 ± 2% -0.2 2.67 perf-profile.self.cycles-pp.common_file_perm 1.40 ± 4% -0.2 1.22 perf-profile.self.cycles-pp.new_sync_read 2.32 -0.2 2.17 perf-profile.self.cycles-pp.aa_sk_perm 2.56 -0.1 2.41 perf-profile.self.cycles-pp.entry_SYSCALL_64 1.90 ± 2% -0.1 1.77 ± 3% perf-profile.self.cycles-pp.__fget_light 2.13 -0.1 2.00 perf-profile.self.cycles-pp.__alloc_skb 2.29 -0.1 2.17 perf-profile.self.cycles-pp.sock_write_iter 1.93 -0.1 1.81 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 1.79 -0.1 1.67 perf-profile.self.cycles-pp.write 1.83 -0.1 1.72 perf-profile.self.cycles-pp.read 1.52 -0.1 1.42 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 1.71 -0.1 1.61 perf-profile.self.cycles-pp.__kmalloc_node_track_caller 1.93 -0.1 1.84 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 1.60 ± 3% -0.1 1.52 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 1.81 -0.1 1.73 perf-profile.self.cycles-pp.kmem_cache_alloc_node 1.17 -0.1 1.09 perf-profile.self.cycles-pp.sock_def_readable 1.55 -0.1 1.47 perf-profile.self.cycles-pp.copy_user_generic_unrolled 1.25 ± 2% -0.1 1.17 ± 2% perf-profile.self.cycles-pp._copy_to_iter 0.61 -0.1 0.54 perf-profile.self.cycles-pp.ksys_write 1.44 -0.1 1.37 perf-profile.self.cycles-pp.sock_wfree 0.95 -0.1 0.88 perf-profile.self.cycles-pp.skb_set_owner_w 1.06 -0.1 0.99 ± 2% perf-profile.self.cycles-pp._copy_from_iter 0.58 ± 5% -0.1 0.52 ± 5% perf-profile.self.cycles-pp.aa_file_perm 1.04 -0.1 0.97 ± 2% perf-profile.self.cycles-pp.__ksize 1.10 -0.1 1.04 perf-profile.self.cycles-pp.unix_write_space 0.68 ± 2% -0.1 0.62 perf-profile.self.cycles-pp.__skb_datagram_iter 0.93 -0.1 0.87 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.52 -0.1 0.47 perf-profile.self.cycles-pp.ksys_read 0.68 ± 2% -0.1 0.63 ± 2% perf-profile.self.cycles-pp.apparmor_file_permission 0.83 -0.0 0.78 perf-profile.self.cycles-pp.unix_destruct_scm 0.87 -0.0 0.82 perf-profile.self.cycles-pp.consume_skb 0.83 -0.0 0.79 perf-profile.self.cycles-pp.sock_alloc_send_pskb 0.71 -0.0 0.67 perf-profile.self.cycles-pp._cond_resched 0.82 -0.0 0.78 perf-profile.self.cycles-pp.alloc_skb_with_frags 0.29 ± 3% -0.0 0.25 perf-profile.self.cycles-pp.unix_stream_read_actor 0.66 -0.0 0.62 perf-profile.self.cycles-pp.skb_copy_datagram_from_iter 0.53 -0.0 0.49 ± 2% perf-profile.self.cycles-pp.apparmor_socket_recvmsg 0.64 -0.0 0.60 perf-profile.self.cycles-pp.__syscall_return_slowpath 0.60 ± 2% -0.0 0.56 ± 4% perf-profile.self.cycles-pp.__might_fault 0.50 -0.0 0.46 perf-profile.self.cycles-pp.__kmalloc_reserve 0.37 -0.0 0.34 perf-profile.self.cycles-pp.security_socket_recvmsg 2.25 -0.0 2.21 perf-profile.self.cycles-pp.sock_read_iter 0.54 -0.0 0.51 perf-profile.self.cycles-pp.rcu_all_qs 0.37 ± 5% -0.0 0.34 ± 4% perf-profile.self.cycles-pp.sock_sendmsg 0.43 -0.0 0.40 perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.98 -0.0 0.95 perf-profile.self.cycles-pp.unix_stream_recvmsg 0.35 -0.0 0.32 perf-profile.self.cycles-pp.skb_copy_datagram_iter 0.41 -0.0 0.38 perf-profile.self.cycles-pp.apparmor_socket_sendmsg 0.56 -0.0 0.54 perf-profile.self.cycles-pp.check_stack_object 0.44 -0.0 0.41 perf-profile.self.cycles-pp.wait_for_unix_gc 0.29 ± 3% -0.0 0.26 ± 2% perf-profile.self.cycles-pp.security_socket_sendmsg 0.42 -0.0 0.40 perf-profile.self.cycles-pp.security_socket_getpeersec_dgram 0.63 -0.0 0.61 perf-profile.self.cycles-pp.mutex_unlock 0.40 ± 2% -0.0 0.38 perf-profile.self.cycles-pp.__fdget_pos 0.31 -0.0 0.29 perf-profile.self.cycles-pp.copyin 0.23 -0.0 0.21 ± 2% perf-profile.self.cycles-pp.skb_release_data 0.19 -0.0 0.17 ± 2% perf-profile.self.cycles-pp.skb_release_head_state 0.24 -0.0 0.23 perf-profile.self.cycles-pp.copyout 0.21 ± 2% -0.0 0.19 ± 2% perf-profile.self.cycles-pp.simple_copy_to_iter 0.26 -0.0 0.25 perf-profile.self.cycles-pp.kmalloc_slab 0.17 ± 2% +0.0 0.20 ± 2% perf-profile.self.cycles-pp.__vfs_read 0.40 +0.0 0.43 perf-profile.self.cycles-pp.apparmor_socket_getpeersec_dgram 0.73 +0.0 0.77 perf-profile.self.cycles-pp.mutex_lock 0.51 +0.1 0.57 perf-profile.self.cycles-pp.sock_recvmsg 1.33 +0.2 1.57 perf-profile.self.cycles-pp.vfs_write 0.75 ± 6% +0.4 1.10 ± 3% perf-profile.self.cycles-pp.security_file_permission 0.97 +0.5 1.49 ± 2% perf-profile.self.cycles-pp.vfs_read 0.00 +1.7 1.65 perf-profile.self.cycles-pp.fsnotify_parent 0.00 +2.8 2.80 perf-profile.self.cycles-pp.fsnotify *************************************************************************************************** lkp-csl-2ap4: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/pipe1/will-it-scale/0x5002f01 commit: v5.8-rc3 b6509f6a8c ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"") v5.8-rc3 b6509f6a8c4313c068c69785c00 ---------------- --------------------------- %stddev %change %stddev \ | \ 1045643 -2.3% 1021331 will-it-scale.per_thread_ops 2.008e+08 -2.3% 1.961e+08 will-it-scale.workload 29490176 ± 3% -9.8% 26607616 ± 3% meminfo.DirectMap2M 202721 ± 4% +11.5% 225943 ± 3% numa-numastat.node2.local_node 25892 ± 28% -75.5% 6341 ± 97% numa-numastat.node2.other_node 1843 +8.9% 2007 ± 5% numa-vmstat.node2.nr_mapped 130681 ± 5% -14.7% 111436 ± 5% numa-vmstat.node2.numa_other 9801 ± 5% +16.0% 11370 ± 7% sched_debug.cpu.ttwu_count.max 3052 ± 12% +23.6% 3771 ± 11% sched_debug.cpu.ttwu_local.max 136156 +1.4% 138002 proc-vmstat.nr_active_anon 109009 +1.1% 110229 proc-vmstat.nr_anon_pages 136156 +1.4% 138002 proc-vmstat.nr_zone_active_anon 1120468 +1.6% 1137854 proc-vmstat.pgalloc_normal 337.83 ± 61% +53.8% 519.75 ± 47% interrupts.39:PCI-MSI.524290-edge.eth0-TxRx-1 337.83 ± 61% +53.8% 519.75 ± 47% interrupts.CPU11.39:PCI-MSI.524290-edge.eth0-TxRx-1 303.33 +45.7% 442.00 ± 44% interrupts.CPU148.RES:Rescheduling_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU155.NMI:Non-maskable_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU155.PMI:Performance_monitoring_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU157.NMI:Non-maskable_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU157.PMI:Performance_monitoring_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU158.NMI:Non-maskable_interrupts 5509 ± 35% +50.1% 8271 interrupts.CPU158.PMI:Performance_monitoring_interrupts 15130 ± 14% -18.1% 12396 ± 8% interrupts.CPU168.CAL:Function_call_interrupts 4137 ± 15% +44.3% 5970 ± 26% interrupts.CPU48.CAL:Function_call_interrupts 336.00 ± 6% +60.0% 537.75 ± 31% interrupts.CPU48.RES:Rescheduling_interrupts 0.75 -0.0 0.73 perf-stat.i.branch-miss-rate% 8.192e+08 -2.2% 8.01e+08 perf-stat.i.branch-misses 0.98 -1.4% 0.97 perf-stat.i.cpi 1.7e+11 +1.5% 1.725e+11 perf-stat.i.dTLB-loads 1.055e+11 +2.7% 1.084e+11 perf-stat.i.dTLB-stores 7.304e+08 -2.5% 7.123e+08 perf-stat.i.iTLB-load-misses 5.6e+11 +1.6% 5.688e+11 perf-stat.i.instructions 775.04 +4.2% 807.76 perf-stat.i.instructions-per-iTLB-miss 1.02 +1.5% 1.04 perf-stat.i.ipc 2015 +1.5% 2046 perf-stat.i.metric.M/sec 0.73 -0.0 0.72 perf-stat.overall.branch-miss-rate% 0.98 -1.5% 0.96 perf-stat.overall.cpi 0.00 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate% 764.55 +4.1% 796.24 perf-stat.overall.instructions-per-iTLB-miss 1.02 +1.5% 1.04 perf-stat.overall.ipc 841233 +4.0% 874479 perf-stat.overall.path-length 8.164e+08 -2.2% 7.983e+08 perf-stat.ps.branch-misses 1.695e+11 +1.5% 1.72e+11 perf-stat.ps.dTLB-loads 1.052e+11 +2.7% 1.08e+11 perf-stat.ps.dTLB-stores 7.301e+08 -2.5% 7.119e+08 perf-stat.ps.iTLB-load-misses 5.582e+11 +1.6% 5.668e+11 perf-stat.ps.instructions 1.689e+14 +1.5% 1.715e+14 perf-stat.total.instructions 50.20 -1.4 48.82 perf-profile.calltrace.cycles-pp.__GI___libc_read 40.68 -1.2 39.52 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read 39.53 -1.1 38.39 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 36.75 -1.1 35.70 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 32.27 -0.9 31.36 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 24.69 -0.6 24.14 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 22.23 -0.5 21.72 perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 21.04 -0.5 20.55 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 18.70 -0.4 18.27 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 6.80 -0.1 6.65 perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 5.39 -0.1 5.26 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__GI___libc_write 2.96 -0.1 2.84 ± 2% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.27 -0.1 2.15 perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 2.42 -0.1 2.31 ± 2% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 5.38 -0.1 5.28 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__GI___libc_read 2.61 -0.1 2.52 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.new_sync_read.vfs_read.ksys_read 3.46 -0.1 3.38 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 2.62 -0.1 2.55 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 2.08 -0.1 2.02 perf-profile.calltrace.cycles-pp.__pthread_disable_asynccancel 0.99 -0.1 0.94 perf-profile.calltrace.cycles-pp.fput_many.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 1.04 -0.0 1.00 perf-profile.calltrace.cycles-pp.__prepare_exit_to_usermode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 0.97 -0.0 0.93 ± 2% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyin.copy_page_from_iter.pipe_write.new_sync_write 1.23 -0.0 1.19 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 1.11 -0.0 1.08 perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyout.copy_page_to_iter.pipe_read.new_sync_read 1.20 -0.0 1.17 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.pipe_write.new_sync_write.vfs_write.ksys_write 1.01 -0.0 0.98 perf-profile.calltrace.cycles-pp.__prepare_exit_to_usermode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 0.79 -0.0 0.76 perf-profile.calltrace.cycles-pp.___might_sleep.mutex_lock.pipe_read.new_sync_read.vfs_read 1.50 -0.0 1.47 perf-profile.calltrace.cycles-pp.__might_fault.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 1.13 -0.0 1.11 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.pipe_read.new_sync_read.vfs_read.ksys_read 0.79 -0.0 0.77 perf-profile.calltrace.cycles-pp.___might_sleep.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 0.72 -0.0 0.70 perf-profile.calltrace.cycles-pp.__sb_end_write.pipe_write.new_sync_write.vfs_write.ksys_write 0.99 -0.0 0.97 perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +0.9 0.92 perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 49.09 +1.4 50.48 perf-profile.calltrace.cycles-pp.__GI___libc_write 0.00 +1.5 1.50 ± 5% perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 39.33 +1.6 40.95 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write 38.18 +1.7 39.84 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 35.19 +1.8 37.00 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 30.77 +1.8 32.61 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 50.18 -1.4 48.80 perf-profile.children.cycles-pp.__GI___libc_read 36.87 -1.1 35.81 perf-profile.children.cycles-pp.ksys_read 32.66 -0.9 31.73 perf-profile.children.cycles-pp.vfs_read 24.84 -0.6 24.29 perf-profile.children.cycles-pp.new_sync_write 22.51 -0.5 22.00 perf-profile.children.cycles-pp.pipe_write 21.18 -0.5 20.69 perf-profile.children.cycles-pp.new_sync_read 18.94 -0.4 18.50 perf-profile.children.cycles-pp.pipe_read 5.44 -0.2 5.27 perf-profile.children.cycles-pp.mutex_lock 7.06 -0.2 6.90 perf-profile.children.cycles-pp.copy_page_from_iter 6.34 -0.2 6.19 perf-profile.children.cycles-pp.syscall_return_via_sysret 7.28 -0.1 7.13 perf-profile.children.cycles-pp.entry_SYSCALL_64 2.33 -0.1 2.21 perf-profile.children.cycles-pp.copyin 7.14 -0.1 7.03 perf-profile.children.cycles-pp.copy_page_to_iter 2.48 -0.1 2.36 ± 2% perf-profile.children.cycles-pp.copyout 4.78 -0.1 4.67 perf-profile.children.cycles-pp.___might_sleep 2.48 -0.1 2.39 perf-profile.children.cycles-pp.copy_user_generic_unrolled 2.23 -0.1 2.14 perf-profile.children.cycles-pp.__prepare_exit_to_usermode 3.56 -0.1 3.48 perf-profile.children.cycles-pp.__wake_up_common_lock 2.69 -0.1 2.63 perf-profile.children.cycles-pp.__might_sleep 2.41 -0.1 2.35 perf-profile.children.cycles-pp._raw_spin_lock_irq 2.08 -0.1 2.02 perf-profile.children.cycles-pp.__pthread_disable_asynccancel 1.88 -0.1 1.82 perf-profile.children.cycles-pp.fput_many 1.40 -0.0 1.35 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.34 -0.0 0.29 perf-profile.children.cycles-pp.fput 1.38 -0.0 1.34 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.24 ± 7% -0.0 0.22 perf-profile.children.cycles-pp.tick_sched_handle 0.92 -0.0 0.90 perf-profile.children.cycles-pp.fpregs_assert_state_consistent 1.28 -0.0 1.26 perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.75 -0.0 0.73 perf-profile.children.cycles-pp.__sb_end_write 80.17 +0.5 80.63 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 77.95 +0.5 78.47 perf-profile.children.cycles-pp.do_syscall_64 1.98 +0.9 2.88 perf-profile.children.cycles-pp.fsnotify_parent 3.04 +1.4 4.42 ± 2% perf-profile.children.cycles-pp.fsnotify 49.09 +1.4 50.48 perf-profile.children.cycles-pp.__GI___libc_write 35.45 +1.8 37.22 perf-profile.children.cycles-pp.ksys_write 30.96 +2.0 32.93 perf-profile.children.cycles-pp.vfs_write 6.28 -0.1 6.13 perf-profile.self.cycles-pp.syscall_return_via_sysret 4.68 -0.1 4.58 perf-profile.self.cycles-pp.___might_sleep 2.35 -0.1 2.26 perf-profile.self.cycles-pp.copy_user_generic_unrolled 2.26 -0.1 2.18 perf-profile.self.cycles-pp.mutex_lock 2.61 -0.1 2.54 perf-profile.self.cycles-pp.__GI___libc_write 3.09 -0.1 3.02 perf-profile.self.cycles-pp.pipe_write 2.57 -0.1 2.50 perf-profile.self.cycles-pp.__GI___libc_read 2.26 -0.1 2.19 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 3.74 -0.1 3.68 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.93 -0.1 0.87 perf-profile.self.cycles-pp.ksys_write 1.38 -0.1 1.32 perf-profile.self.cycles-pp.__prepare_exit_to_usermode 2.31 -0.1 2.25 perf-profile.self.cycles-pp._raw_spin_lock_irq 1.78 -0.1 1.72 perf-profile.self.cycles-pp.fput_many 1.99 -0.1 1.94 perf-profile.self.cycles-pp.__pthread_disable_asynccancel 2.37 -0.0 2.32 perf-profile.self.cycles-pp.__might_sleep 1.14 -0.0 1.10 perf-profile.self.cycles-pp.do_syscall_64 1.36 -0.0 1.31 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 2.22 -0.0 2.18 perf-profile.self.cycles-pp.__pthread_enable_asynccancel 1.87 -0.0 1.83 perf-profile.self.cycles-pp.mutex_unlock 1.34 -0.0 1.31 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.91 -0.0 0.89 perf-profile.self.cycles-pp.file_update_time 0.85 -0.0 0.82 perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.44 -0.0 0.41 ± 2% perf-profile.self.cycles-pp.touch_atime 0.73 -0.0 0.71 perf-profile.self.cycles-pp.__sb_end_write 1.23 -0.0 1.21 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.21 -0.0 0.20 perf-profile.self.cycles-pp.kill_fasync 2.08 ± 2% +0.4 2.49 ± 4% perf-profile.self.cycles-pp.vfs_write 1.71 +0.8 2.51 perf-profile.self.cycles-pp.fsnotify_parent 2.94 +1.3 4.25 ± 2% perf-profile.self.cycles-pp.fsnotify Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

11 months, 1 week

3
3
0 / 0

[PCI] 3233e41d3e: WARNING:at_drivers/pci/pci.c:#pci_reset_hotplug_slot

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 3233e41d3e8ebcd44e92da47ffed97fd49b84278 ("[PATCH] PCI: pciehp: Fix AB-BA deadlock between reset_lock and device_lock") url: https://github.com/0day-ci/linux/commits/Lukas-Wunner/PCI-pciehp-Fix-AB-B... base: https://git.kernel.org/cgit/linux/kernel/git/helgaas/pci.git next in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 8 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +------------------------------------------------------+------------+------------+ | | 8a445afd71 | 3233e41d3e | +------------------------------------------------------+------------+------------+ | boot_successes | 4 | 0 | | boot_failures | 0 | 4 | | WARNING:at_drivers/pci/pci.c:#pci_reset_hotplug_slot | 0 | 4 | | RIP:pci_reset_hotplug_slot | 0 | 4 | +------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 0.971752] WARNING: CPU: 0 PID: 1 at drivers/pci/pci.c:4905 pci_reset_hotplug_slot+0x70/0x80 [ 0.971753] Modules linked in: [ 0.971755] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.8.0-rc1-00053-g3233e41d3e8eb #1 [ 0.971756] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 0.971757] RIP: 0010:pci_reset_hotplug_slot+0x70/0x80 [ 0.971759] Code: 41 89 c4 48 8b 7b 20 e8 4e 3a c1 ff 44 89 e0 5b 5d 41 5c c3 48 8b 43 18 31 f6 48 8d b8 90 00 00 00 e8 e4 80 70 00 85 c0 75 ba <0f> 0b eb b6 41 bc e7 ff ff ff eb d6 0f 1f 40 00 66 66 66 66 90 48 [ 0.971759] RSP: 0000:ffffbd73c0013ab0 EFLAGS: 00010246 [ 0.971761] RAX: 0000000000000000 RBX: ffff9700475024c0 RCX: ffff9701048c9900 [ 0.971762] RDX: ffff970047e40000 RSI: ffff9701048c9990 RDI: 0000000000000246 [ 0.971763] RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000 [ 0.971763] R10: 0000000000000001 R11: 0000000000000000 R12: ffff9700477320b0 [ 0.971764] R13: ffff970047732000 R14: ffff97004769c238 R15: 0000000000000000 [ 0.971765] FS: 0000000000000000(0000) GS:ffff97036fc00000(0000) knlGS:0000000000000000 [ 0.971766] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 0.971767] CR2: 0000000000000000 CR3: 00000001c2c10000 CR4: 00000000000006f0 [ 0.971767] Call Trace: [ 0.971768] pci_probe_reset_function+0xc4/0xe0 [ 0.971769] pci_device_add+0x13f/0x2a0 [ 0.971769] pci_scan_single_device+0xa4/0xc0 [ 0.971770] pci_scan_slot+0x52/0x110 [ 0.971771] pci_scan_child_bus_extend+0x3a/0x2a0 [ 0.971771] acpi_pci_root_create+0x1f7/0x250 [ 0.971772] pci_acpi_scan_root+0x182/0x1b0 [ 0.971773] acpi_pci_root_add.cold+0x59/0x1b0 [ 0.971773] ? acpi_device_always_present+0x20/0x90 [ 0.971774] acpi_bus_attach+0xf6/0x200 [ 0.971775] acpi_bus_attach+0x6b/0x200 [ 0.971776] acpi_bus_scan+0x43/0x90 [ 0.971776] ? acpi_sleep_proc_init+0x24/0x24 [ 0.971777] acpi_scan_init+0x102/0x24b [ 0.971778] acpi_init+0x2c7/0x329 [ 0.971778] do_one_initcall+0x5d/0x330 [ 0.971779] ? rcu_read_lock_sched_held+0x52/0x90 [ 0.971780] kernel_init_freeable+0x248/0x2c9 [ 0.971780] ? rest_init+0x23e/0x23e [ 0.971781] kernel_init+0xa/0x112 [ 0.971782] ret_from_fork+0x22/0x30 [ 0.971782] irq event stamp: 109115 [ 0.971783] hardirqs last enabled at (109115): [<ffffffffaa71a514>] _raw_spin_unlock_irqrestore+0x54/0x70 [ 0.971784] hardirqs last disabled at (109114): [<ffffffffaa71ab01>] _raw_spin_lock_irqsave+0x21/0x60 [ 0.971785] softirqs last enabled at (109060): [<ffffffffaaa003aa>] __do_softirq+0x3aa/0x4af [ 0.971786] softirqs last disabled at (109053): [<ffffffffaa8010b2>] asm_call_on_stack+0x12/0x20 [ 0.971787] ---[ end trace c3e4ce92dee5df5f ]--- To reproduce: # build kernel cd linux cp config-5.8.0-rc1-00053-g3233e41d3e8eb .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, lkp

1 year

3
2
0 / 0

[ptrace] 201766a20e: kernel_selftests.seccomp.make_fail

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: 201766a20e30f982ccfe36bebfad9602c3ff574a ("ptrace: add PTRACE_GET_SYSCALL_INFO request") https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master in testcase: kernel_selftests with following parameters: group: kselftests-02 test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. test-url: https://www.kernel.org/doc/Documentation/kselftest.txt on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> 2019-07-26 16:52:03 make run_tests -C seccomp make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp' gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf In file included from seccomp_bpf.c:51:0: seccomp_bpf.c: In function ‘tracer_ptrace’: seccomp_bpf.c:1787:20: error: ‘PTRACE_EVENTMSG_SYSCALL_ENTRY’ undeclared (first use in this function) EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^ ../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’ __typeof__(_expected) __exp = (_expected); \ ^~~~~~~~~ seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’ EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^~~~~~~~~ seccomp_bpf.c:1787:20: note: each undeclared identifier is reported only once for each function it appears in EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^ ../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’ __typeof__(_expected) __exp = (_expected); \ ^~~~~~~~~ seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’ EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^~~~~~~~~ seccomp_bpf.c:1788:6: error: ‘PTRACE_EVENTMSG_SYSCALL_EXIT’ undeclared (first use in this function) : PTRACE_EVENTMSG_SYSCALL_EXIT, msg); ^ ../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’ __typeof__(_expected) __exp = (_expected); \ ^~~~~~~~~ seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’ EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^~~~~~~~~ Makefile:12: recipe for target 'seccomp_bpf' failed make: *** [seccomp_bpf] Error 1 make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp' To reproduce: # build kernel cd linux cp config-5.2.0-10889-g201766a20e30f9 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

1 year

3
3
0 / 0

[x86/mce] 1de08dccd3: will-it-scale.per_process_ops -14.1% regression

by kernel test robot

Greeting, FYI, we noticed a -14.1% regression of will-it-scale.per_process_ops due to commit: commit: 1de08dccd383482a3e88845d3554094d338f5ff9 ("x86/mce: Add a struct mce.kflags field") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: will-it-scale on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory with following parameters: nr_task: 100% mode: process test: malloc1 cpufreq_governor: performance ucode: 0x11 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-knm01/malloc1/will-it-scale/0x11 commit: 9554bfe403 ("x86/mce: Convert the CEC to use the MCE notifier") 1de08dccd3 ("x86/mce: Add a struct mce.kflags field") 9554bfe403bdfc08 1de08dccd383482a3e88845d355 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x :4 25% 1:4 dmesg.WARNING:at_ip___perf_sw_event/0x %stddev %change %stddev \ | \ 668.00 -14.1% 573.75 will-it-scale.per_process_ops 192559 -14.1% 165344 will-it-scale.workload 424371 -20.3% 338331 ± 8% vmstat.system.in 0.00 ± 13% +0.0 0.00 ± 22% mpstat.cpu.all.soft% 0.54 -0.1 0.47 ± 3% mpstat.cpu.all.usr% 1.205e+08 -13.7% 1.039e+08 numa-numastat.node0.local_node 1.205e+08 -13.7% 1.039e+08 numa-numastat.node0.numa_hit 61585280 -13.1% 53521568 numa-vmstat.node0.numa_hit 61585799 -13.1% 53522027 numa-vmstat.node0.numa_local 1.203e+08 -13.8% 1.037e+08 proc-vmstat.numa_hit 1.203e+08 -13.8% 1.037e+08 proc-vmstat.numa_local 1.205e+08 -13.7% 1.04e+08 proc-vmstat.pgalloc_normal 60608363 -13.6% 52339576 proc-vmstat.pgfault 1.204e+08 -13.8% 1.038e+08 proc-vmstat.pgfree 0.04 ± 9% +17.4% 0.05 ± 7% sched_debug.cfs_rq:/.nr_running.stddev 52.52 ± 5% +11.9% 58.79 ± 2% sched_debug.cfs_rq:/.util_avg.stddev 2049590 ± 7% -12.8% 1788136 ± 4% sched_debug.cpu.avg_idle.avg 418.68 ± 2% -22.1% 326.13 ± 2% sched_debug.cpu.clock.stddev 418.68 ± 2% -22.1% 326.14 ± 2% sched_debug.cpu.clock_task.stddev 158439 ± 8% +29.6% 205376 ± 16% sched_debug.cpu.max_idle_balance_cost.stddev 0.00 -18.8% 0.00 ± 2% sched_debug.cpu.next_balance.stddev 0.00 ±173% +500.0% 0.00 ± 33% sched_debug.cpu.nr_uninterruptible.avg -49.04 +76.9% -86.75 sched_debug.cpu.nr_uninterruptible.min 1117 +13.3% 1266 ± 3% sched_debug.cpu.sched_count.min 36.92 -2.4 34.52 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 36.91 -2.4 34.51 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 36.30 -2.3 33.98 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 36.30 -2.3 33.99 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.55 -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore.pagevec_lru_move_fn 0.54 ± 2% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore.release_pages 1.02 -0.1 0.94 perf-profile.calltrace.cycles-pp.page_fault 0.81 -0.1 0.73 perf-profile.calltrace.cycles-pp.handle_mm_fault.do_page_fault.page_fault 0.98 -0.1 0.90 perf-profile.calltrace.cycles-pp.do_page_fault.page_fault 0.77 -0.1 0.68 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault 0.69 -0.1 0.61 perf-profile.calltrace.cycles-pp.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault.page_fault 0.65 -0.1 0.58 ± 5% perf-profile.calltrace.cycles-pp.apic_timer_interrupt._raw_spin_unlock_irqrestore.release_pages.tlb_flush_mmu.tlb_finish_mmu 0.67 ± 2% -0.1 0.60 ± 6% perf-profile.calltrace.cycles-pp.apic_timer_interrupt._raw_spin_unlock_irqrestore.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain 0.66 -0.1 0.59 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region 0.62 -0.1 0.56 ± 4% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore.release_pages.tlb_flush_mmu 0.64 ± 2% -0.1 0.57 ± 6% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt._raw_spin_unlock_irqrestore.pagevec_lru_move_fn.lru_add_drain_cpu 0.66 -0.1 0.60 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain.unmap_region 47.88 +0.1 48.00 perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 47.86 +0.1 48.00 perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap 47.74 +0.1 47.89 perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap 96.85 +0.3 97.18 perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 46.33 +0.3 46.67 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region 46.30 +0.3 46.64 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.tlb_finish_mmu 47.55 +0.3 47.90 perf-profile.calltrace.cycles-pp.lru_add_drain.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 47.55 +0.4 47.90 perf-profile.calltrace.cycles-pp.lru_add_drain_cpu.lru_add_drain.unmap_region.__do_munmap.__vm_munmap 47.52 +0.4 47.87 perf-profile.calltrace.cycles-pp.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain.unmap_region.__do_munmap 96.64 +0.4 97.01 perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 46.22 +0.5 46.74 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain.unmap_region 46.20 +0.5 46.72 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.pagevec_lru_move_fn.lru_add_drain_cpu.lru_add_drain 60.91 +2.6 63.55 perf-profile.calltrace.cycles-pp.munmap 60.60 +2.6 63.24 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 60.84 +2.6 63.47 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap 60.59 +2.6 63.23 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 60.83 +2.6 63.47 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 2.21 -0.3 1.90 ± 6% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 2.24 -0.3 1.93 ± 5% perf-profile.children.cycles-pp.apic_timer_interrupt 2.13 -0.3 1.85 ± 5% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 1.84 -0.2 1.60 ± 5% perf-profile.children.cycles-pp.hrtimer_interrupt 1.57 -0.2 1.39 ± 4% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.66 -0.1 0.55 perf-profile.children.cycles-pp.vm_mmap_pgoff 0.67 -0.1 0.56 perf-profile.children.cycles-pp.ksys_mmap_pgoff 1.07 -0.1 0.96 ± 5% perf-profile.children.cycles-pp.tick_sched_timer 1.03 -0.1 0.93 ± 5% perf-profile.children.cycles-pp.tick_sched_handle 1.01 -0.1 0.91 ± 5% perf-profile.children.cycles-pp.update_process_times 0.58 -0.1 0.49 perf-profile.children.cycles-pp.do_mmap 1.01 -0.1 0.92 perf-profile.children.cycles-pp.do_page_fault 1.05 -0.1 0.97 perf-profile.children.cycles-pp.page_fault 0.83 -0.1 0.74 ± 2% perf-profile.children.cycles-pp.handle_mm_fault 0.79 ± 2% -0.1 0.71 ± 6% perf-profile.children.cycles-pp.scheduler_tick 0.92 ± 2% -0.1 0.84 ± 3% perf-profile.children.cycles-pp.unmap_vmas 0.78 -0.1 0.70 ± 2% perf-profile.children.cycles-pp.__handle_mm_fault 0.88 -0.1 0.80 ± 3% perf-profile.children.cycles-pp.unmap_page_range 0.70 -0.1 0.62 perf-profile.children.cycles-pp.handle_pte_fault 0.43 -0.1 0.36 perf-profile.children.cycles-pp.mmap_region 0.47 ± 2% -0.1 0.41 ± 2% perf-profile.children.cycles-pp.mmap64 0.55 -0.0 0.50 ± 4% perf-profile.children.cycles-pp.task_tick_fair 0.18 ± 2% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.perf_event_mmap 0.11 -0.0 0.08 ± 6% perf-profile.children.cycles-pp.perf_iterate_sb 0.31 -0.0 0.27 ± 3% perf-profile.children.cycles-pp.__alloc_pages_nodemask 0.26 -0.0 0.23 perf-profile.children.cycles-pp.get_page_from_freelist 0.22 -0.0 0.19 ± 3% perf-profile.children.cycles-pp.pte_alloc_one 0.24 ± 2% -0.0 0.21 ± 4% perf-profile.children.cycles-pp.__pte_alloc 0.50 -0.0 0.47 ± 2% perf-profile.children.cycles-pp.change_protection 0.50 -0.0 0.47 ± 2% perf-profile.children.cycles-pp.change_p4d_range 0.50 -0.0 0.47 ± 2% perf-profile.children.cycles-pp.change_prot_numa 0.11 ± 4% -0.0 0.09 ± 4% perf-profile.children.cycles-pp.free_unref_page_list 0.50 -0.0 0.47 ± 2% perf-profile.children.cycles-pp.task_work_run 0.50 -0.0 0.48 perf-profile.children.cycles-pp.exit_to_usermode_loop 0.32 ± 2% -0.0 0.29 ± 4% perf-profile.children.cycles-pp.___might_sleep 0.50 -0.0 0.47 ± 2% perf-profile.children.cycles-pp.task_numa_work 0.16 ± 2% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.prep_new_page 0.16 ± 2% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.alloc_pages_vma 0.14 ± 3% -0.0 0.12 ± 4% perf-profile.children.cycles-pp.clear_page_erms 0.07 ± 6% -0.0 0.05 perf-profile.children.cycles-pp.kmem_cache_free 0.07 -0.0 0.05 ± 8% perf-profile.children.cycles-pp.percpu_counter_add_batch 0.17 ± 2% -0.0 0.15 ± 4% perf-profile.children.cycles-pp._cond_resched 0.12 -0.0 0.10 ± 4% perf-profile.children.cycles-pp.get_unmapped_area 0.09 ± 4% -0.0 0.07 ± 10% perf-profile.children.cycles-pp._raw_spin_lock 0.13 -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__anon_vma_prepare 0.13 ± 3% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.free_pgtables 0.10 ± 4% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown 0.16 -0.0 0.15 ± 3% perf-profile.children.cycles-pp.irq_exit 0.13 -0.0 0.12 ± 3% perf-profile.children.cycles-pp.free_pgd_range 0.10 -0.0 0.09 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.05 +0.0 0.08 ± 6% perf-profile.children.cycles-pp.__mod_memcg_state 0.07 +0.0 0.10 perf-profile.children.cycles-pp.__mod_lruvec_state 0.14 ± 13% +0.0 0.18 ± 6% perf-profile.children.cycles-pp.__remove_hrtimer 47.91 +0.1 48.04 perf-profile.children.cycles-pp.tlb_finish_mmu 47.90 +0.1 48.03 perf-profile.children.cycles-pp.tlb_flush_mmu 47.81 +0.1 47.96 perf-profile.children.cycles-pp.release_pages 98.55 +0.1 98.70 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 98.53 +0.1 98.68 perf-profile.children.cycles-pp.do_syscall_64 96.91 +0.3 97.24 perf-profile.children.cycles-pp.__x64_sys_munmap 96.89 +0.3 97.22 perf-profile.children.cycles-pp.__vm_munmap 47.73 +0.3 48.06 perf-profile.children.cycles-pp.pagevec_lru_move_fn 96.88 +0.3 97.21 perf-profile.children.cycles-pp.__do_munmap 47.60 +0.4 47.95 perf-profile.children.cycles-pp.lru_add_drain 47.59 +0.4 47.94 perf-profile.children.cycles-pp.lru_add_drain_cpu 96.67 +0.4 97.03 perf-profile.children.cycles-pp.unmap_region 92.78 +0.8 93.60 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 92.83 +0.8 93.67 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 60.92 +2.6 63.56 perf-profile.children.cycles-pp.munmap 0.17 ± 6% -0.1 0.06 ± 13% perf-profile.self.cycles-pp.__hrtimer_run_queues 0.47 ± 2% -0.1 0.42 ± 4% perf-profile.self.cycles-pp.unmap_page_range 0.10 ± 4% -0.0 0.07 ± 12% perf-profile.self.cycles-pp.hrtimer_interrupt 0.11 -0.0 0.08 ± 10% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.44 -0.0 0.41 ± 3% perf-profile.self.cycles-pp.change_p4d_range 0.08 -0.0 0.06 ± 9% perf-profile.self.cycles-pp.perf_iterate_sb 0.14 -0.0 0.12 ± 3% perf-profile.self.cycles-pp.clear_page_erms 0.08 ± 5% -0.0 0.07 ± 6% perf-profile.self.cycles-pp._raw_spin_lock 0.07 ± 7% -0.0 0.05 perf-profile.self.cycles-pp.kmem_cache_free 0.08 -0.0 0.07 ± 6% perf-profile.self.cycles-pp.release_pages 0.09 -0.0 0.08 ± 5% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.05 ± 8% +0.0 0.07 ± 14% perf-profile.self.cycles-pp.___perf_sw_event 0.05 +0.0 0.07 ± 5% perf-profile.self.cycles-pp.__mod_memcg_state 0.00 +0.1 0.14 ± 9% perf-profile.self.cycles-pp.__remove_hrtimer 92.77 +0.8 93.60 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 124732 +11.7% 139275 softirqs.CPU0.TIMER 125382 +13.1% 141792 ± 2% softirqs.CPU10.TIMER 124413 +11.5% 138750 softirqs.CPU100.TIMER 123657 +12.2% 138733 softirqs.CPU101.TIMER 123899 +12.1% 138896 softirqs.CPU102.TIMER 124228 +11.7% 138717 softirqs.CPU103.TIMER 123619 +12.3% 138882 softirqs.CPU104.TIMER 123585 +12.7% 139247 softirqs.CPU105.TIMER 123593 +12.0% 138384 softirqs.CPU106.TIMER 123642 +12.2% 138741 softirqs.CPU107.TIMER 123926 +11.7% 138456 softirqs.CPU108.TIMER 123798 +12.0% 138608 softirqs.CPU109.TIMER 124136 +12.0% 139018 softirqs.CPU11.TIMER 123935 +12.5% 139487 softirqs.CPU110.TIMER 124699 +11.1% 138505 softirqs.CPU111.TIMER 123708 +12.3% 138865 softirqs.CPU112.TIMER 123122 +13.1% 139227 softirqs.CPU113.TIMER 123665 +16.1% 143543 ± 7% softirqs.CPU114.TIMER 123887 +11.6% 138272 softirqs.CPU115.TIMER 123873 +11.8% 138498 softirqs.CPU116.TIMER 123987 +11.6% 138362 softirqs.CPU117.TIMER 123891 +11.6% 138254 softirqs.CPU118.TIMER 123315 +16.1% 143144 ± 4% softirqs.CPU119.TIMER 123661 +14.9% 142093 ± 3% softirqs.CPU12.TIMER 123199 +12.3% 138367 softirqs.CPU120.TIMER 123372 +12.1% 138262 softirqs.CPU121.TIMER 123641 +11.6% 137956 softirqs.CPU122.TIMER 123389 +12.6% 138884 softirqs.CPU123.TIMER 123309 +12.4% 138608 softirqs.CPU124.TIMER 123604 +11.9% 138283 softirqs.CPU125.TIMER 123456 +12.2% 138513 softirqs.CPU126.TIMER 123526 +12.8% 139288 softirqs.CPU127.TIMER 123496 +11.9% 138241 softirqs.CPU128.TIMER 123381 +12.9% 139252 softirqs.CPU129.TIMER 125036 +11.1% 138906 softirqs.CPU13.TIMER 123722 +12.5% 139159 softirqs.CPU130.TIMER 123523 +12.1% 138454 softirqs.CPU131.TIMER 123760 +11.8% 138329 softirqs.CPU132.TIMER 123569 +12.2% 138624 softirqs.CPU133.TIMER 123669 +12.1% 138657 softirqs.CPU134.TIMER 122996 +12.7% 138591 softirqs.CPU135.TIMER 123011 +12.6% 138503 softirqs.CPU136.TIMER 123141 +12.6% 138624 softirqs.CPU137.TIMER 123505 +12.0% 138360 softirqs.CPU138.TIMER 123513 +12.0% 138343 softirqs.CPU139.TIMER 124622 +18.6% 147768 ± 6% softirqs.CPU14.TIMER 123088 +12.3% 138241 softirqs.CPU140.TIMER 123186 +13.2% 139395 softirqs.CPU141.TIMER 123249 +12.8% 139061 ± 2% softirqs.CPU142.TIMER 123274 +12.2% 138333 softirqs.CPU143.TIMER 123241 +12.5% 138599 ± 2% softirqs.CPU144.TIMER 123183 +12.1% 138053 softirqs.CPU145.TIMER 124668 +10.7% 137947 softirqs.CPU146.TIMER 123275 +12.7% 138880 softirqs.CPU147.TIMER 123278 +12.3% 138403 softirqs.CPU148.TIMER 123611 +12.3% 138764 softirqs.CPU149.TIMER 124133 +12.0% 139055 softirqs.CPU15.TIMER 123195 +12.1% 138071 softirqs.CPU150.TIMER 123687 +11.7% 138126 softirqs.CPU151.TIMER 123483 +11.9% 138191 softirqs.CPU152.TIMER 123281 +12.1% 138248 softirqs.CPU153.TIMER 124097 +11.3% 138090 softirqs.CPU154.TIMER 123803 +11.7% 138348 softirqs.CPU155.TIMER 122720 +12.6% 138171 softirqs.CPU157.TIMER 123267 +12.7% 138929 softirqs.CPU158.TIMER 123387 +12.0% 138189 softirqs.CPU159.TIMER 124075 +12.0% 138944 softirqs.CPU16.TIMER 123571 +12.4% 138834 softirqs.CPU160.TIMER 123578 +12.0% 138378 softirqs.CPU161.TIMER 123664 +11.9% 138397 softirqs.CPU162.TIMER 123363 +11.9% 138061 softirqs.CPU163.TIMER 123053 +12.4% 138264 softirqs.CPU164.TIMER 123412 +12.1% 138367 softirqs.CPU165.TIMER 123721 +11.7% 138219 softirqs.CPU166.TIMER 123512 +11.8% 138098 softirqs.CPU167.TIMER 123392 +12.1% 138382 softirqs.CPU168.TIMER 123559 +11.9% 138276 softirqs.CPU169.TIMER 124075 +12.5% 139563 softirqs.CPU17.TIMER 123019 +12.5% 138428 softirqs.CPU170.TIMER 123467 +12.0% 138328 softirqs.CPU171.TIMER 123040 +12.3% 138224 softirqs.CPU173.TIMER 123997 +11.6% 138334 softirqs.CPU174.TIMER 123787 +11.8% 138437 softirqs.CPU175.TIMER 123315 +12.2% 138419 softirqs.CPU176.TIMER 123771 +12.1% 138716 softirqs.CPU177.TIMER 123016 +12.2% 138058 softirqs.CPU178.TIMER 122844 +12.4% 138072 softirqs.CPU179.TIMER 123981 +12.1% 138975 softirqs.CPU18.TIMER 123511 +11.8% 138041 softirqs.CPU180.TIMER 123415 +12.0% 138171 softirqs.CPU181.TIMER 122954 +12.9% 138845 softirqs.CPU182.TIMER 123291 +12.0% 138113 softirqs.CPU183.TIMER 122910 +12.4% 138175 softirqs.CPU184.TIMER 123015 +12.8% 138812 ± 2% softirqs.CPU185.TIMER 123197 +11.5% 137396 softirqs.CPU186.TIMER 122914 +12.2% 137870 softirqs.CPU187.TIMER 122854 +12.7% 138509 softirqs.CPU188.TIMER 122864 +12.5% 138211 softirqs.CPU189.TIMER 124311 +11.5% 138654 softirqs.CPU19.TIMER 122961 +12.4% 138166 softirqs.CPU190.TIMER 123015 +12.3% 138134 softirqs.CPU191.TIMER 122906 +12.3% 138029 softirqs.CPU192.TIMER 122988 +12.3% 138098 softirqs.CPU193.TIMER 122863 +11.9% 137464 softirqs.CPU194.TIMER 122883 +12.3% 138039 softirqs.CPU195.TIMER 123157 +12.1% 138023 softirqs.CPU196.TIMER 122831 +12.5% 138204 softirqs.CPU197.TIMER 123113 +11.9% 137755 softirqs.CPU198.TIMER 122809 +12.2% 137771 softirqs.CPU199.TIMER 123833 +12.3% 139118 softirqs.CPU20.TIMER 122908 +12.1% 137806 softirqs.CPU200.TIMER 122641 +12.4% 137887 softirqs.CPU201.TIMER 123187 +12.2% 138253 softirqs.CPU202.TIMER 122997 +12.2% 138012 softirqs.CPU203.TIMER 123088 +11.7% 137542 softirqs.CPU204.TIMER 122928 +12.2% 137903 softirqs.CPU205.TIMER 122990 +12.1% 137911 softirqs.CPU206.TIMER 123028 +12.2% 138095 softirqs.CPU207.TIMER 122473 +12.7% 138050 softirqs.CPU208.TIMER 122665 +12.8% 138325 softirqs.CPU209.TIMER 124436 +11.5% 138798 softirqs.CPU21.TIMER 122387 +12.7% 137900 softirqs.CPU210.TIMER 122693 +12.3% 137746 softirqs.CPU211.TIMER 122651 +12.3% 137710 softirqs.CPU212.TIMER 122995 +11.9% 137644 softirqs.CPU213.TIMER 122918 +11.9% 137535 softirqs.CPU214.TIMER 122662 +12.0% 137431 softirqs.CPU215.TIMER 122226 +12.3% 137244 softirqs.CPU216.TIMER 122549 +12.1% 137338 softirqs.CPU217.TIMER 123475 +11.5% 137651 softirqs.CPU218.TIMER 123553 +11.4% 137635 softirqs.CPU219.TIMER 123621 +13.3% 140099 ± 2% softirqs.CPU22.TIMER 122924 +12.0% 137726 softirqs.CPU220.TIMER 123174 +12.3% 138348 softirqs.CPU221.TIMER 122630 +12.5% 137977 softirqs.CPU222.TIMER 123731 +11.8% 138294 softirqs.CPU223.TIMER 123492 +11.8% 138023 softirqs.CPU224.TIMER 123019 +11.9% 137651 softirqs.CPU225.TIMER 123061 +12.0% 137781 softirqs.CPU226.TIMER 123436 +11.8% 137983 softirqs.CPU227.TIMER 122711 +12.3% 137768 softirqs.CPU228.TIMER 122361 +13.2% 138515 softirqs.CPU229.TIMER 124044 +12.2% 139183 softirqs.CPU23.TIMER 122462 +13.2% 138573 softirqs.CPU230.TIMER 122970 +12.5% 138298 softirqs.CPU231.TIMER 123005 +12.2% 137962 softirqs.CPU232.TIMER 122716 +12.4% 137939 softirqs.CPU233.TIMER 122591 +12.4% 137826 softirqs.CPU234.TIMER 122794 +12.4% 138058 softirqs.CPU235.TIMER 122607 +12.6% 138015 softirqs.CPU236.TIMER 122744 +12.8% 138401 softirqs.CPU237.TIMER 122680 +12.2% 137683 softirqs.CPU238.TIMER 122589 +12.3% 137729 softirqs.CPU239.TIMER 122741 +12.5% 138046 softirqs.CPU240.TIMER 124001 +11.3% 137964 softirqs.CPU241.TIMER 122283 +12.5% 137507 softirqs.CPU242.TIMER 122722 +12.4% 137958 softirqs.CPU243.TIMER 123393 +11.4% 137463 softirqs.CPU244.TIMER 122456 +12.4% 137610 softirqs.CPU245.TIMER 122995 +12.3% 138090 softirqs.CPU246.TIMER 123687 +11.4% 137814 softirqs.CPU247.TIMER 122494 +12.9% 138288 softirqs.CPU248.TIMER 122634 +12.6% 138146 softirqs.CPU249.TIMER 124923 +11.8% 139698 softirqs.CPU25.TIMER 122661 +12.3% 137714 softirqs.CPU250.TIMER 122343 +12.5% 137689 softirqs.CPU251.TIMER 122846 +11.8% 137353 softirqs.CPU252.TIMER 122455 +12.3% 137525 softirqs.CPU253.TIMER 122334 +13.1% 138369 softirqs.CPU254.TIMER 122023 +13.2% 138083 ± 2% softirqs.CPU256.TIMER 122317 +12.2% 137246 softirqs.CPU257.TIMER 122558 +11.6% 136825 softirqs.CPU258.TIMER 122482 +15.8% 141829 ± 6% softirqs.CPU259.TIMER 123637 +12.2% 138768 softirqs.CPU26.TIMER 122120 +12.5% 137405 softirqs.CPU260.TIMER 122386 +12.2% 137365 softirqs.CPU262.TIMER 122398 +12.4% 137560 softirqs.CPU263.TIMER 122108 +12.5% 137359 softirqs.CPU264.TIMER 122101 +15.4% 140873 ± 5% softirqs.CPU265.TIMER 122163 +12.2% 137050 softirqs.CPU266.TIMER 122077 +12.6% 137416 softirqs.CPU267.TIMER 122318 +12.3% 137341 softirqs.CPU268.TIMER 122063 +12.5% 137290 softirqs.CPU269.TIMER 123854 +12.0% 138752 softirqs.CPU27.TIMER 122323 +12.0% 137011 softirqs.CPU270.TIMER 122091 +12.4% 137199 softirqs.CPU271.TIMER 122042 +13.0% 137944 softirqs.CPU272.TIMER 122898 +11.4% 136920 softirqs.CPU273.TIMER 122100 +12.5% 137340 softirqs.CPU274.TIMER 122223 +12.2% 137141 softirqs.CPU275.TIMER 122367 +11.8% 136818 softirqs.CPU276.TIMER 122216 +12.2% 137067 softirqs.CPU277.TIMER 122035 +12.2% 136880 softirqs.CPU278.TIMER 122296 +12.4% 137442 softirqs.CPU279.TIMER 124072 +12.2% 139264 softirqs.CPU28.TIMER 122058 +12.3% 137060 softirqs.CPU280.TIMER 121889 +12.5% 137107 softirqs.CPU281.TIMER 121953 +12.4% 137058 softirqs.CPU282.TIMER 122136 +12.3% 137153 softirqs.CPU283.TIMER 122152 +11.9% 136661 softirqs.CPU284.TIMER 122126 +12.1% 136891 softirqs.CPU285.TIMER 121523 +12.7% 136947 softirqs.CPU286.TIMER 117427 +12.6% 132264 ± 2% softirqs.CPU287.TIMER 124082 +11.7% 138601 softirqs.CPU29.TIMER 124520 +11.4% 138684 softirqs.CPU3.TIMER 125786 ± 3% +10.4% 138855 softirqs.CPU30.TIMER 124607 +11.5% 138958 softirqs.CPU31.TIMER 123701 +13.1% 139886 softirqs.CPU32.TIMER 124593 +11.9% 139391 softirqs.CPU33.TIMER 123719 +12.0% 138526 softirqs.CPU34.TIMER 123959 +11.9% 138665 softirqs.CPU35.TIMER 123758 +12.0% 138556 softirqs.CPU36.TIMER 123856 +11.9% 138597 softirqs.CPU37.TIMER 124053 +16.7% 144775 ± 6% softirqs.CPU38.TIMER 123675 +11.8% 138281 softirqs.CPU39.TIMER 124228 +12.0% 139164 softirqs.CPU4.TIMER 123900 +12.3% 139175 softirqs.CPU40.TIMER 123892 +12.4% 139211 softirqs.CPU41.TIMER 127063 ± 3% +9.1% 138615 softirqs.CPU42.TIMER 123679 +12.2% 138760 softirqs.CPU43.TIMER 124702 +11.9% 139566 softirqs.CPU44.TIMER 123975 +11.9% 138712 softirqs.CPU45.TIMER 124174 +11.6% 138531 softirqs.CPU46.TIMER 123644 +12.1% 138571 softirqs.CPU48.TIMER 123687 +12.3% 138843 softirqs.CPU49.TIMER 124610 +11.6% 139078 softirqs.CPU5.TIMER 124146 +15.8% 143709 ± 3% softirqs.CPU51.TIMER 123635 +12.8% 139412 softirqs.CPU52.TIMER 124065 +12.1% 139088 softirqs.CPU53.TIMER 124147 +14.2% 141788 ± 3% softirqs.CPU54.TIMER 123762 +12.2% 138905 softirqs.CPU55.TIMER 125582 +10.6% 138868 softirqs.CPU57.TIMER 125328 +18.7% 148744 ± 11% softirqs.CPU58.TIMER 123995 +12.1% 138967 softirqs.CPU59.TIMER 124120 +12.1% 139157 softirqs.CPU6.TIMER 124023 +12.3% 139227 softirqs.CPU60.TIMER 123781 +12.1% 138727 softirqs.CPU61.TIMER 123569 +12.5% 138970 softirqs.CPU62.TIMER 123608 +12.5% 139074 softirqs.CPU63.TIMER 123407 +13.1% 139543 softirqs.CPU64.TIMER 127045 ± 4% +9.2% 138760 softirqs.CPU65.TIMER 126610 ± 5% +9.5% 138633 softirqs.CPU66.TIMER 123612 +12.1% 138603 softirqs.CPU67.TIMER 123604 +12.5% 139113 softirqs.CPU68.TIMER 123505 +12.2% 138554 softirqs.CPU69.TIMER 124325 +11.7% 138838 softirqs.CPU7.TIMER 124090 +12.5% 139591 softirqs.CPU70.TIMER 123699 +11.9% 138451 softirqs.CPU71.TIMER 124613 +13.9% 141945 ± 3% softirqs.CPU72.TIMER 123431 +12.3% 138639 softirqs.CPU73.TIMER 123853 +12.1% 138852 softirqs.CPU75.TIMER 124259 +11.8% 138957 softirqs.CPU76.TIMER 123966 +12.1% 138962 softirqs.CPU77.TIMER 123577 +12.1% 138498 softirqs.CPU78.TIMER 123749 +12.0% 138577 softirqs.CPU79.TIMER 124280 +11.7% 138772 softirqs.CPU8.TIMER 124029 +11.7% 138583 softirqs.CPU80.TIMER 123492 +12.4% 138826 softirqs.CPU81.TIMER 124304 +11.6% 138753 softirqs.CPU82.TIMER 123952 +12.0% 138766 softirqs.CPU83.TIMER 123672 +15.0% 142254 ± 3% softirqs.CPU85.TIMER 123910 +12.4% 139236 softirqs.CPU86.TIMER 123664 +12.3% 138908 softirqs.CPU87.TIMER 124068 +12.1% 139076 softirqs.CPU88.TIMER 123802 +12.2% 138885 softirqs.CPU89.TIMER 123826 +12.0% 138649 softirqs.CPU90.TIMER 124086 +11.7% 138621 softirqs.CPU91.TIMER 123586 +12.4% 138925 softirqs.CPU92.TIMER 123696 +12.5% 139160 softirqs.CPU93.TIMER 123720 +11.8% 138367 softirqs.CPU94.TIMER 124536 +11.3% 138666 softirqs.CPU95.TIMER 123950 +12.2% 139071 softirqs.CPU96.TIMER 123876 +12.1% 138860 softirqs.CPU97.TIMER 123700 +12.2% 138807 softirqs.CPU98.TIMER 123452 +12.5% 138905 softirqs.CPU99.TIMER 35617264 +12.0% 39908742 softirqs.TIMER 554.75 ± 65% +547.1% 3590 ± 96% interrupts.32:IR-PCI-MSI.2097155-edge.eth0-TxRx-2 452452 -21.0% 357575 ± 8% interrupts.CPU0.LOC:Local_timer_interrupts 1315 ± 10% +55.9% 2051 ± 16% interrupts.CPU0.RES:Rescheduling_interrupts 452347 -21.1% 357112 ± 8% interrupts.CPU1.LOC:Local_timer_interrupts 452532 -21.3% 356223 ± 8% interrupts.CPU10.LOC:Local_timer_interrupts 448763 -20.9% 354890 ± 8% interrupts.CPU100.LOC:Local_timer_interrupts 448680 -20.8% 355364 ± 8% interrupts.CPU101.LOC:Local_timer_interrupts 447608 -20.7% 355149 ± 8% interrupts.CPU102.LOC:Local_timer_interrupts 448227 -20.9% 354352 ± 8% interrupts.CPU103.LOC:Local_timer_interrupts 448996 -20.6% 356349 ± 8% interrupts.CPU104.LOC:Local_timer_interrupts 448510 -20.7% 355821 ± 8% interrupts.CPU105.LOC:Local_timer_interrupts 447226 -20.7% 354825 ± 8% interrupts.CPU106.LOC:Local_timer_interrupts 446828 -20.9% 353563 ± 8% interrupts.CPU107.LOC:Local_timer_interrupts 446384 -20.5% 355009 ± 8% interrupts.CPU108.LOC:Local_timer_interrupts 446823 -20.7% 354511 ± 8% interrupts.CPU109.LOC:Local_timer_interrupts 452246 -21.1% 356984 ± 8% interrupts.CPU11.LOC:Local_timer_interrupts 447034 -20.5% 355451 ± 8% interrupts.CPU110.LOC:Local_timer_interrupts 447783 -20.9% 354154 ± 8% interrupts.CPU111.LOC:Local_timer_interrupts 446657 -20.5% 355120 ± 8% interrupts.CPU112.LOC:Local_timer_interrupts 445392 -20.3% 354902 ± 8% interrupts.CPU113.LOC:Local_timer_interrupts 448354 -20.7% 355621 ± 8% interrupts.CPU114.LOC:Local_timer_interrupts 72.00 ± 84% +293.8% 283.50 ± 28% interrupts.CPU114.RES:Rescheduling_interrupts 447962 -20.8% 354707 ± 8% interrupts.CPU115.LOC:Local_timer_interrupts 447215 -20.7% 354848 ± 8% interrupts.CPU116.LOC:Local_timer_interrupts 447530 -20.8% 354411 ± 8% interrupts.CPU117.LOC:Local_timer_interrupts 448647 -20.9% 354998 ± 8% interrupts.CPU118.LOC:Local_timer_interrupts 447753 -20.6% 355316 ± 8% interrupts.CPU119.LOC:Local_timer_interrupts 554.75 ± 65% +547.1% 3590 ± 96% interrupts.CPU12.32:IR-PCI-MSI.2097155-edge.eth0-TxRx-2 451965 -20.9% 357310 ± 8% interrupts.CPU12.LOC:Local_timer_interrupts 448234 -20.7% 355312 ± 8% interrupts.CPU120.LOC:Local_timer_interrupts 447794 -20.7% 354971 ± 8% interrupts.CPU121.LOC:Local_timer_interrupts 448691 -20.4% 357243 ± 8% interrupts.CPU122.LOC:Local_timer_interrupts 5632 ± 32% -33.5% 3745 interrupts.CPU122.NMI:Non-maskable_interrupts 5632 ± 32% -33.5% 3745 interrupts.CPU122.PMI:Performance_monitoring_interrupts 447313 -20.2% 356771 ± 9% interrupts.CPU123.LOC:Local_timer_interrupts 91.50 ± 57% -59.0% 37.50 ±128% interrupts.CPU123.RES:Rescheduling_interrupts 448249 -20.3% 357447 ± 8% interrupts.CPU124.LOC:Local_timer_interrupts 448297 -20.6% 355804 ± 8% interrupts.CPU125.LOC:Local_timer_interrupts 449032 -21.0% 354647 ± 8% interrupts.CPU126.LOC:Local_timer_interrupts 447604 -20.8% 354651 ± 8% interrupts.CPU127.LOC:Local_timer_interrupts 447392 -20.4% 356084 ± 8% interrupts.CPU128.LOC:Local_timer_interrupts 169.25 ± 48% +405.2% 855.00 ±119% interrupts.CPU128.RES:Rescheduling_interrupts 447344 -20.7% 354766 ± 8% interrupts.CPU129.LOC:Local_timer_interrupts 6632 ± 24% -29.3% 4686 ± 33% interrupts.CPU129.NMI:Non-maskable_interrupts 6632 ± 24% -29.3% 4686 ± 33% interrupts.CPU129.PMI:Performance_monitoring_interrupts 452316 -21.2% 356567 ± 8% interrupts.CPU13.LOC:Local_timer_interrupts 449103 -20.4% 357697 ± 8% interrupts.CPU130.LOC:Local_timer_interrupts 449086 -20.5% 357205 ± 8% interrupts.CPU131.LOC:Local_timer_interrupts 450238 -21.0% 355846 ± 8% interrupts.CPU132.LOC:Local_timer_interrupts 447135 -20.8% 354142 ± 8% interrupts.CPU133.LOC:Local_timer_interrupts 446450 -20.1% 356602 ± 8% interrupts.CPU134.LOC:Local_timer_interrupts 449356 -20.5% 357068 ± 9% interrupts.CPU135.LOC:Local_timer_interrupts 447938 -20.5% 356253 ± 8% interrupts.CPU136.LOC:Local_timer_interrupts 447648 -20.6% 355408 ± 8% interrupts.CPU137.LOC:Local_timer_interrupts 3763 +74.3% 6559 ± 24% interrupts.CPU137.NMI:Non-maskable_interrupts 3763 +74.3% 6559 ± 24% interrupts.CPU137.PMI:Performance_monitoring_interrupts 446747 -20.3% 355856 ± 8% interrupts.CPU138.LOC:Local_timer_interrupts 447237 -20.8% 354392 ± 8% interrupts.CPU139.LOC:Local_timer_interrupts 452138 -21.0% 357163 ± 8% interrupts.CPU14.LOC:Local_timer_interrupts 448154 -20.5% 356451 ± 8% interrupts.CPU140.LOC:Local_timer_interrupts 176.25 ± 53% +131.8% 408.50 ± 38% interrupts.CPU140.RES:Rescheduling_interrupts 447880 -20.6% 355665 ± 8% interrupts.CPU141.LOC:Local_timer_interrupts 447680 -20.7% 354855 ± 8% interrupts.CPU142.LOC:Local_timer_interrupts 446659 -20.6% 354589 ± 8% interrupts.CPU143.LOC:Local_timer_interrupts 445820 -20.5% 354379 ± 8% interrupts.CPU144.LOC:Local_timer_interrupts 6541 ± 24% -42.8% 3738 interrupts.CPU144.NMI:Non-maskable_interrupts 6541 ± 24% -42.8% 3738 interrupts.CPU144.PMI:Performance_monitoring_interrupts 447253 -20.6% 355128 ± 8% interrupts.CPU145.LOC:Local_timer_interrupts 447763 -20.8% 354763 ± 8% interrupts.CPU146.LOC:Local_timer_interrupts 447772 -20.7% 354890 ± 8% interrupts.CPU147.LOC:Local_timer_interrupts 446939 -20.7% 354440 ± 8% interrupts.CPU148.LOC:Local_timer_interrupts 447464 -20.7% 354793 ± 8% interrupts.CPU149.LOC:Local_timer_interrupts 452738 -21.2% 356559 ± 8% interrupts.CPU15.LOC:Local_timer_interrupts 446799 -20.6% 354599 ± 8% interrupts.CPU150.LOC:Local_timer_interrupts 447879 -20.9% 354287 ± 8% interrupts.CPU151.LOC:Local_timer_interrupts 448426 -20.6% 355888 ± 8% interrupts.CPU152.LOC:Local_timer_interrupts 449366 -20.7% 356542 ± 8% interrupts.CPU153.LOC:Local_timer_interrupts 447867 -20.7% 355067 ± 8% interrupts.CPU154.LOC:Local_timer_interrupts 447719 -20.7% 355013 ± 8% interrupts.CPU155.LOC:Local_timer_interrupts 446511 -20.3% 355972 ± 8% interrupts.CPU156.LOC:Local_timer_interrupts 449192 -20.9% 355497 ± 8% interrupts.CPU157.LOC:Local_timer_interrupts 446758 -20.5% 355083 ± 8% interrupts.CPU158.LOC:Local_timer_interrupts 447084 -20.6% 355107 ± 8% interrupts.CPU159.LOC:Local_timer_interrupts 452662 -21.1% 357106 ± 8% interrupts.CPU16.LOC:Local_timer_interrupts 447831 -20.8% 354684 ± 8% interrupts.CPU160.LOC:Local_timer_interrupts 109.50 ± 15% -37.2% 68.75 ± 22% interrupts.CPU160.RES:Rescheduling_interrupts 447477 -20.6% 355188 ± 8% interrupts.CPU161.LOC:Local_timer_interrupts 448495 -20.8% 355285 ± 8% interrupts.CPU162.LOC:Local_timer_interrupts 449288 -20.9% 355477 ± 8% interrupts.CPU163.LOC:Local_timer_interrupts 447985 -20.6% 355568 ± 8% interrupts.CPU164.LOC:Local_timer_interrupts 446601 -20.5% 355191 ± 8% interrupts.CPU165.LOC:Local_timer_interrupts 448135 -20.6% 355818 ± 8% interrupts.CPU166.LOC:Local_timer_interrupts 448461 -20.6% 356227 ± 8% interrupts.CPU167.LOC:Local_timer_interrupts 447391 -20.7% 354733 ± 8% interrupts.CPU168.LOC:Local_timer_interrupts 446612 -20.5% 354909 ± 8% interrupts.CPU169.LOC:Local_timer_interrupts 452748 -21.3% 356314 ± 8% interrupts.CPU17.LOC:Local_timer_interrupts 447238 -20.4% 355830 ± 8% interrupts.CPU170.LOC:Local_timer_interrupts 448308 -20.9% 354608 ± 8% interrupts.CPU171.LOC:Local_timer_interrupts 449043 -21.2% 354035 ± 8% interrupts.CPU172.LOC:Local_timer_interrupts 449395 -20.9% 355374 ± 9% interrupts.CPU173.LOC:Local_timer_interrupts 5700 ± 32% -17.8% 4687 ± 33% interrupts.CPU173.NMI:Non-maskable_interrupts 5700 ± 32% -17.8% 4687 ± 33% interrupts.CPU173.PMI:Performance_monitoring_interrupts 446781 -20.4% 355457 ± 8% interrupts.CPU174.LOC:Local_timer_interrupts 446541 -20.7% 354087 ± 8% interrupts.CPU175.LOC:Local_timer_interrupts 447728 -20.6% 355499 ± 8% interrupts.CPU176.LOC:Local_timer_interrupts 447740 -20.5% 355885 ± 8% interrupts.CPU177.LOC:Local_timer_interrupts 6623 ± 24% -43.3% 3758 interrupts.CPU177.NMI:Non-maskable_interrupts 6623 ± 24% -43.3% 3758 interrupts.CPU177.PMI:Performance_monitoring_interrupts 447747 -20.7% 355148 ± 8% interrupts.CPU178.LOC:Local_timer_interrupts 447285 -20.6% 354994 ± 8% interrupts.CPU179.LOC:Local_timer_interrupts 450911 -21.1% 355809 ± 8% interrupts.CPU18.LOC:Local_timer_interrupts 447180 -20.7% 354485 ± 8% interrupts.CPU180.LOC:Local_timer_interrupts 447702 -20.7% 354973 ± 8% interrupts.CPU181.LOC:Local_timer_interrupts 447897 -20.7% 355132 ± 8% interrupts.CPU182.LOC:Local_timer_interrupts 449321 -21.0% 355184 ± 8% interrupts.CPU183.LOC:Local_timer_interrupts 448357 -20.8% 354979 ± 8% interrupts.CPU184.LOC:Local_timer_interrupts 165.00 ± 61% -81.2% 31.00 ± 63% interrupts.CPU184.RES:Rescheduling_interrupts 447698 -20.4% 356305 ± 8% interrupts.CPU185.LOC:Local_timer_interrupts 446780 -20.6% 354611 ± 8% interrupts.CPU186.LOC:Local_timer_interrupts 447678 -20.6% 355625 ± 8% interrupts.CPU187.LOC:Local_timer_interrupts 447756 -20.3% 356660 ± 8% interrupts.CPU188.LOC:Local_timer_interrupts 448842 -20.5% 356728 ± 8% interrupts.CPU189.LOC:Local_timer_interrupts 452463 -21.2% 356696 ± 8% interrupts.CPU19.LOC:Local_timer_interrupts 448558 -20.6% 355985 ± 8% interrupts.CPU190.LOC:Local_timer_interrupts 6581 ± 24% -28.9% 4680 ± 34% interrupts.CPU190.NMI:Non-maskable_interrupts 6581 ± 24% -28.9% 4680 ± 34% interrupts.CPU190.PMI:Performance_monitoring_interrupts 448186 -20.7% 355605 ± 8% interrupts.CPU191.LOC:Local_timer_interrupts 447640 -20.7% 354849 ± 8% interrupts.CPU192.LOC:Local_timer_interrupts 447828 -20.5% 355818 ± 8% interrupts.CPU193.LOC:Local_timer_interrupts 449769 -20.7% 356689 ± 8% interrupts.CPU194.LOC:Local_timer_interrupts 449120 -20.4% 357570 ± 9% interrupts.CPU195.LOC:Local_timer_interrupts 5641 ± 32% -33.7% 3738 interrupts.CPU195.NMI:Non-maskable_interrupts 5641 ± 32% -33.7% 3738 interrupts.CPU195.PMI:Performance_monitoring_interrupts 448037 -20.4% 356497 ± 7% interrupts.CPU196.LOC:Local_timer_interrupts 446302 -20.4% 355172 ± 8% interrupts.CPU197.LOC:Local_timer_interrupts 451541 -21.3% 355341 ± 8% interrupts.CPU198.LOC:Local_timer_interrupts 449452 -21.1% 354503 ± 9% interrupts.CPU199.LOC:Local_timer_interrupts 450751 -20.8% 356778 ± 8% interrupts.CPU2.LOC:Local_timer_interrupts 451672 -21.2% 356011 ± 8% interrupts.CPU20.LOC:Local_timer_interrupts 447614 -20.9% 354143 ± 8% interrupts.CPU200.LOC:Local_timer_interrupts 446364 -20.6% 354456 ± 8% interrupts.CPU201.LOC:Local_timer_interrupts 447150 -20.4% 355847 ± 8% interrupts.CPU202.LOC:Local_timer_interrupts 5662 ± 32% -17.4% 4678 ± 34% interrupts.CPU202.NMI:Non-maskable_interrupts 5662 ± 32% -17.4% 4678 ± 34% interrupts.CPU202.PMI:Performance_monitoring_interrupts 447324 -20.5% 355784 ± 8% interrupts.CPU203.LOC:Local_timer_interrupts 450353 -21.1% 355551 ± 8% interrupts.CPU204.LOC:Local_timer_interrupts 449486 -21.1% 354766 ± 8% interrupts.CPU205.LOC:Local_timer_interrupts 47.50 ±118% -84.7% 7.25 ± 15% interrupts.CPU205.RES:Rescheduling_interrupts 447640 -20.7% 355083 ± 8% interrupts.CPU206.LOC:Local_timer_interrupts 447443 -20.4% 356366 ± 8% interrupts.CPU207.LOC:Local_timer_interrupts 446651 -20.5% 355032 ± 8% interrupts.CPU208.LOC:Local_timer_interrupts 446807 -20.4% 355505 ± 8% interrupts.CPU209.LOC:Local_timer_interrupts 5643 ± 32% -33.5% 3751 interrupts.CPU209.NMI:Non-maskable_interrupts 5643 ± 32% -33.5% 3751 interrupts.CPU209.PMI:Performance_monitoring_interrupts 452259 -21.0% 357396 ± 8% interrupts.CPU21.LOC:Local_timer_interrupts 447655 -20.7% 354771 ± 8% interrupts.CPU210.LOC:Local_timer_interrupts 447098 -20.7% 354473 ± 8% interrupts.CPU211.LOC:Local_timer_interrupts 446681 -20.6% 354768 ± 8% interrupts.CPU212.LOC:Local_timer_interrupts 446397 -20.4% 355254 ± 8% interrupts.CPU213.LOC:Local_timer_interrupts 7.75 ± 5% +1032.3% 87.75 ±105% interrupts.CPU213.RES:Rescheduling_interrupts 449196 -20.8% 355834 ± 8% interrupts.CPU214.LOC:Local_timer_interrupts 447889 -20.7% 355386 ± 8% interrupts.CPU215.LOC:Local_timer_interrupts 448595 -21.1% 353795 ± 8% interrupts.CPU216.LOC:Local_timer_interrupts 448232 -21.0% 354308 ± 8% interrupts.CPU217.LOC:Local_timer_interrupts 446761 -20.6% 354787 ± 8% interrupts.CPU218.LOC:Local_timer_interrupts 446095 -20.5% 354569 ± 8% interrupts.CPU219.LOC:Local_timer_interrupts 451994 -21.1% 356839 ± 8% interrupts.CPU22.LOC:Local_timer_interrupts 448837 -21.0% 354621 ± 8% interrupts.CPU220.LOC:Local_timer_interrupts 448495 -21.1% 353771 ± 8% interrupts.CPU221.LOC:Local_timer_interrupts 446731 -20.6% 354886 ± 8% interrupts.CPU222.LOC:Local_timer_interrupts 446768 -20.7% 354324 ± 8% interrupts.CPU223.LOC:Local_timer_interrupts 446684 -20.6% 354750 ± 8% interrupts.CPU224.LOC:Local_timer_interrupts 446780 -20.6% 354621 ± 8% interrupts.CPU225.LOC:Local_timer_interrupts 448071 -20.6% 355846 ± 8% interrupts.CPU226.LOC:Local_timer_interrupts 447211 -20.6% 355043 ± 8% interrupts.CPU227.LOC:Local_timer_interrupts 447185 -20.7% 354717 ± 8% interrupts.CPU228.LOC:Local_timer_interrupts 446926 -20.8% 353965 ± 8% interrupts.CPU229.LOC:Local_timer_interrupts 452110 -20.8% 358288 ± 8% interrupts.CPU23.LOC:Local_timer_interrupts 448401 -20.3% 357175 ± 7% interrupts.CPU230.LOC:Local_timer_interrupts 449244 -20.8% 355611 ± 8% interrupts.CPU231.LOC:Local_timer_interrupts 449265 -21.1% 354397 ± 8% interrupts.CPU232.LOC:Local_timer_interrupts 5647 ± 33% -33.6% 3747 interrupts.CPU232.NMI:Non-maskable_interrupts 5647 ± 33% -33.6% 3747 interrupts.CPU232.PMI:Performance_monitoring_interrupts 447186 -20.7% 354486 ± 8% interrupts.CPU233.LOC:Local_timer_interrupts 448107 -21.0% 354228 ± 8% interrupts.CPU234.LOC:Local_timer_interrupts 7537 -37.9% 4681 ± 35% interrupts.CPU234.NMI:Non-maskable_interrupts 7537 -37.9% 4681 ± 35% interrupts.CPU234.PMI:Performance_monitoring_interrupts 447040 -20.7% 354506 ± 8% interrupts.CPU235.LOC:Local_timer_interrupts 447193 -20.9% 353777 ± 8% interrupts.CPU236.LOC:Local_timer_interrupts 446268 -20.6% 354295 ± 8% interrupts.CPU237.LOC:Local_timer_interrupts 449634 -20.8% 356255 ± 8% interrupts.CPU238.LOC:Local_timer_interrupts 449337 -20.8% 355992 ± 8% interrupts.CPU239.LOC:Local_timer_interrupts 451802 -20.9% 357381 ± 8% interrupts.CPU24.LOC:Local_timer_interrupts 447287 -20.7% 354800 ± 8% interrupts.CPU240.LOC:Local_timer_interrupts 446264 -20.8% 353437 ± 8% interrupts.CPU241.LOC:Local_timer_interrupts 447521 -20.6% 355538 ± 8% interrupts.CPU242.LOC:Local_timer_interrupts 448368 -20.8% 355108 ± 8% interrupts.CPU243.LOC:Local_timer_interrupts 448794 -21.2% 353861 ± 8% interrupts.CPU244.LOC:Local_timer_interrupts 448265 -21.1% 353723 ± 8% interrupts.CPU245.LOC:Local_timer_interrupts 447404 -20.6% 355385 ± 8% interrupts.CPU246.LOC:Local_timer_interrupts 448265 -20.7% 355456 ± 8% interrupts.CPU247.LOC:Local_timer_interrupts 447316 -20.7% 354842 ± 8% interrupts.CPU248.LOC:Local_timer_interrupts 447886 -20.7% 355020 ± 8% interrupts.CPU249.LOC:Local_timer_interrupts 452147 -21.3% 355786 ± 8% interrupts.CPU25.LOC:Local_timer_interrupts 448406 -20.8% 355289 ± 8% interrupts.CPU250.LOC:Local_timer_interrupts 448566 -20.9% 354726 ± 8% interrupts.CPU251.LOC:Local_timer_interrupts 447607 -21.0% 353757 ± 8% interrupts.CPU252.LOC:Local_timer_interrupts 28.50 ± 68% +173.7% 78.00 ± 39% interrupts.CPU252.RES:Rescheduling_interrupts 448037 -20.9% 354453 ± 8% interrupts.CPU253.LOC:Local_timer_interrupts 449413 -20.8% 356025 ± 8% interrupts.CPU254.LOC:Local_timer_interrupts 449548 -20.8% 356202 ± 8% interrupts.CPU255.LOC:Local_timer_interrupts 41.00 ±110% -84.8% 6.25 ± 13% interrupts.CPU255.RES:Rescheduling_interrupts 449239 -21.0% 354887 ± 8% interrupts.CPU256.LOC:Local_timer_interrupts 447192 -20.8% 354224 ± 8% interrupts.CPU257.LOC:Local_timer_interrupts 448177 -20.7% 355300 ± 8% interrupts.CPU258.LOC:Local_timer_interrupts 448275 -20.6% 355711 ± 8% interrupts.CPU259.LOC:Local_timer_interrupts 451573 -21.3% 355400 ± 8% interrupts.CPU26.LOC:Local_timer_interrupts 449188 -20.7% 356207 ± 8% interrupts.CPU260.LOC:Local_timer_interrupts 449309 -20.8% 355852 ± 8% interrupts.CPU261.LOC:Local_timer_interrupts 447847 -20.8% 354847 ± 8% interrupts.CPU262.LOC:Local_timer_interrupts 446950 -20.5% 355229 ± 8% interrupts.CPU263.LOC:Local_timer_interrupts 6496 ± 24% -42.7% 3722 interrupts.CPU263.NMI:Non-maskable_interrupts 6496 ± 24% -42.7% 3722 interrupts.CPU263.PMI:Performance_monitoring_interrupts 449945 -20.8% 356417 ± 8% interrupts.CPU264.LOC:Local_timer_interrupts 450439 -21.0% 355899 ± 8% interrupts.CPU265.LOC:Local_timer_interrupts 450339 -20.7% 357335 ± 8% interrupts.CPU266.LOC:Local_timer_interrupts 450174 -20.2% 359229 ± 9% interrupts.CPU267.LOC:Local_timer_interrupts 451002 -20.9% 356683 ± 8% interrupts.CPU268.LOC:Local_timer_interrupts 450462 -21.1% 355569 ± 8% interrupts.CPU269.LOC:Local_timer_interrupts 452188 -21.0% 357345 ± 8% interrupts.CPU27.LOC:Local_timer_interrupts 453310 -21.7% 355167 ± 8% interrupts.CPU270.LOC:Local_timer_interrupts 451241 -21.2% 355504 ± 8% interrupts.CPU271.LOC:Local_timer_interrupts 451114 -21.3% 355154 ± 8% interrupts.CPU272.LOC:Local_timer_interrupts 450134 -21.1% 355326 ± 8% interrupts.CPU273.LOC:Local_timer_interrupts 450431 -20.9% 356194 ± 8% interrupts.CPU274.LOC:Local_timer_interrupts 19.00 ± 45% +173.7% 52.00 ± 62% interrupts.CPU274.RES:Rescheduling_interrupts 450807 -20.8% 357003 ± 8% interrupts.CPU275.LOC:Local_timer_interrupts 453075 -21.5% 355714 ± 8% interrupts.CPU276.LOC:Local_timer_interrupts 450048 -21.0% 355644 ± 8% interrupts.CPU277.LOC:Local_timer_interrupts 449561 -20.7% 356522 ± 8% interrupts.CPU278.LOC:Local_timer_interrupts 450017 -20.4% 358363 ± 9% interrupts.CPU279.LOC:Local_timer_interrupts 451493 -21.2% 355798 ± 8% interrupts.CPU28.LOC:Local_timer_interrupts 160.00 ± 5% +51.4% 242.25 ± 9% interrupts.CPU28.RES:Rescheduling_interrupts 450158 -21.0% 355626 ± 8% interrupts.CPU280.LOC:Local_timer_interrupts 450175 -21.0% 355780 ± 8% interrupts.CPU281.LOC:Local_timer_interrupts 450065 -20.9% 356051 ± 8% interrupts.CPU282.LOC:Local_timer_interrupts 447777 -20.5% 355761 ± 8% interrupts.CPU283.LOC:Local_timer_interrupts 450110 -20.9% 356052 ± 8% interrupts.CPU284.LOC:Local_timer_interrupts 6524 ± 24% -28.6% 4657 ± 34% interrupts.CPU284.NMI:Non-maskable_interrupts 6524 ± 24% -28.6% 4657 ± 34% interrupts.CPU284.PMI:Performance_monitoring_interrupts 449420 -20.9% 355525 ± 8% interrupts.CPU285.LOC:Local_timer_interrupts 449800 -20.7% 356713 ± 8% interrupts.CPU286.LOC:Local_timer_interrupts 459915 -20.2% 366810 ± 9% interrupts.CPU287.LOC:Local_timer_interrupts 451800 -21.2% 355897 ± 8% interrupts.CPU29.LOC:Local_timer_interrupts 451158 -21.0% 356357 ± 8% interrupts.CPU3.LOC:Local_timer_interrupts 452347 -21.3% 355924 ± 8% interrupts.CPU30.LOC:Local_timer_interrupts 3777 +98.4% 7492 interrupts.CPU30.NMI:Non-maskable_interrupts 3777 +98.4% 7492 interrupts.CPU30.PMI:Performance_monitoring_interrupts 128.75 ± 31% +86.2% 239.75 ± 10% interrupts.CPU30.RES:Rescheduling_interrupts 451634 -20.8% 357831 ± 8% interrupts.CPU31.LOC:Local_timer_interrupts 452787 -20.9% 358017 ± 8% interrupts.CPU32.LOC:Local_timer_interrupts 70.50 ± 38% +279.8% 267.75 ± 54% interrupts.CPU32.RES:Rescheduling_interrupts 451923 -20.9% 357380 ± 8% interrupts.CPU33.LOC:Local_timer_interrupts 452722 -21.4% 355941 ± 8% interrupts.CPU34.LOC:Local_timer_interrupts 448603 -20.4% 357004 ± 8% interrupts.CPU35.LOC:Local_timer_interrupts 452210 -21.3% 355998 ± 8% interrupts.CPU36.LOC:Local_timer_interrupts 451882 -21.3% 355544 ± 8% interrupts.CPU37.LOC:Local_timer_interrupts 453383 -21.4% 356383 ± 8% interrupts.CPU38.LOC:Local_timer_interrupts 452809 -21.0% 357598 ± 8% interrupts.CPU39.LOC:Local_timer_interrupts 452315 -21.0% 357529 ± 8% interrupts.CPU4.LOC:Local_timer_interrupts 450608 -21.2% 354930 ± 8% interrupts.CPU40.LOC:Local_timer_interrupts 450081 -20.9% 355990 ± 8% interrupts.CPU41.LOC:Local_timer_interrupts 450043 -20.8% 356507 ± 8% interrupts.CPU42.LOC:Local_timer_interrupts 22.75 ± 48% +236.3% 76.50 ± 40% interrupts.CPU42.RES:Rescheduling_interrupts 450140 -20.8% 356378 ± 8% interrupts.CPU43.LOC:Local_timer_interrupts 449695 -20.7% 356421 ± 8% interrupts.CPU44.LOC:Local_timer_interrupts 18.50 ± 43% +625.7% 134.25 ± 80% interrupts.CPU44.RES:Rescheduling_interrupts 450168 -20.8% 356636 ± 8% interrupts.CPU45.LOC:Local_timer_interrupts 451761 -21.3% 355664 ± 8% interrupts.CPU46.LOC:Local_timer_interrupts 450482 -20.8% 356781 ± 8% interrupts.CPU47.LOC:Local_timer_interrupts 53.75 ± 80% +245.6% 185.75 ± 57% interrupts.CPU47.RES:Rescheduling_interrupts 450849 -21.0% 356318 ± 8% interrupts.CPU48.LOC:Local_timer_interrupts 449530 -20.8% 356024 ± 8% interrupts.CPU49.LOC:Local_timer_interrupts 452185 -21.2% 356424 ± 8% interrupts.CPU5.LOC:Local_timer_interrupts 452374 -21.2% 356685 ± 8% interrupts.CPU50.LOC:Local_timer_interrupts 451158 -20.7% 357779 ± 9% interrupts.CPU51.LOC:Local_timer_interrupts 451039 -20.6% 357966 ± 8% interrupts.CPU52.LOC:Local_timer_interrupts 5682 ± 32% -34.2% 3740 interrupts.CPU52.NMI:Non-maskable_interrupts 5682 ± 32% -34.2% 3740 interrupts.CPU52.PMI:Performance_monitoring_interrupts 450978 -21.0% 356243 ± 8% interrupts.CPU53.LOC:Local_timer_interrupts 452251 -21.5% 355052 ± 9% interrupts.CPU54.LOC:Local_timer_interrupts 450986 -21.2% 355254 ± 8% interrupts.CPU55.LOC:Local_timer_interrupts 450685 -21.0% 356148 ± 8% interrupts.CPU56.LOC:Local_timer_interrupts 449475 -20.9% 355563 ± 8% interrupts.CPU57.LOC:Local_timer_interrupts 451278 -20.7% 357853 ± 8% interrupts.CPU58.LOC:Local_timer_interrupts 169.75 ± 60% -63.6% 61.75 ± 99% interrupts.CPU58.RES:Rescheduling_interrupts 450102 -20.5% 357925 ± 8% interrupts.CPU59.LOC:Local_timer_interrupts 451723 -21.3% 355340 ± 8% interrupts.CPU6.LOC:Local_timer_interrupts 454798 -21.5% 356820 ± 8% interrupts.CPU60.LOC:Local_timer_interrupts 7545 -50.2% 3758 interrupts.CPU60.NMI:Non-maskable_interrupts 7545 -50.2% 3758 interrupts.CPU60.PMI:Performance_monitoring_interrupts 451436 -21.3% 355493 ± 8% interrupts.CPU61.LOC:Local_timer_interrupts 450910 -20.8% 357077 ± 8% interrupts.CPU62.LOC:Local_timer_interrupts 450686 -20.3% 359334 ± 9% interrupts.CPU63.LOC:Local_timer_interrupts 450216 -20.8% 356564 ± 8% interrupts.CPU64.LOC:Local_timer_interrupts 449965 -21.1% 354963 ± 8% interrupts.CPU65.LOC:Local_timer_interrupts 450709 -21.0% 355952 ± 8% interrupts.CPU66.LOC:Local_timer_interrupts 450160 -21.0% 355775 ± 8% interrupts.CPU67.LOC:Local_timer_interrupts 450297 -21.1% 355375 ± 8% interrupts.CPU68.LOC:Local_timer_interrupts 449937 -20.9% 356013 ± 8% interrupts.CPU69.LOC:Local_timer_interrupts 5674 ± 32% -17.2% 4700 ± 34% interrupts.CPU69.NMI:Non-maskable_interrupts 5674 ± 32% -17.2% 4700 ± 34% interrupts.CPU69.PMI:Performance_monitoring_interrupts 451977 -21.0% 356885 ± 8% interrupts.CPU7.LOC:Local_timer_interrupts 449651 -20.9% 355504 ± 8% interrupts.CPU70.LOC:Local_timer_interrupts 451073 -20.9% 356589 ± 8% interrupts.CPU71.LOC:Local_timer_interrupts 447324 -20.8% 354292 ± 8% interrupts.CPU72.LOC:Local_timer_interrupts 6588 ± 24% -43.2% 3740 interrupts.CPU72.NMI:Non-maskable_interrupts 6588 ± 24% -43.2% 3740 interrupts.CPU72.PMI:Performance_monitoring_interrupts 240.50 ± 7% -36.0% 154.00 ± 50% interrupts.CPU72.RES:Rescheduling_interrupts 447170 -20.7% 354707 ± 8% interrupts.CPU73.LOC:Local_timer_interrupts 448611 -20.8% 355143 ± 8% interrupts.CPU74.LOC:Local_timer_interrupts 447417 -20.8% 354345 ± 8% interrupts.CPU75.LOC:Local_timer_interrupts 448190 -20.8% 355035 ± 8% interrupts.CPU76.LOC:Local_timer_interrupts 448346 -20.9% 354442 ± 8% interrupts.CPU77.LOC:Local_timer_interrupts 6594 ± 24% -43.4% 3735 interrupts.CPU77.NMI:Non-maskable_interrupts 6594 ± 24% -43.4% 3735 interrupts.CPU77.PMI:Performance_monitoring_interrupts 448452 -20.6% 355866 ± 8% interrupts.CPU78.LOC:Local_timer_interrupts 447509 -20.7% 354660 ± 8% interrupts.CPU79.LOC:Local_timer_interrupts 451912 -20.9% 357687 ± 8% interrupts.CPU8.LOC:Local_timer_interrupts 449808 -20.9% 355596 ± 8% interrupts.CPU80.LOC:Local_timer_interrupts 449111 -21.0% 354994 ± 8% interrupts.CPU81.LOC:Local_timer_interrupts 6642 ± 24% -29.3% 4699 ± 34% interrupts.CPU81.NMI:Non-maskable_interrupts 6642 ± 24% -29.3% 4699 ± 34% interrupts.CPU81.PMI:Performance_monitoring_interrupts 447536 -20.8% 354299 ± 8% interrupts.CPU82.LOC:Local_timer_interrupts 448241 -21.0% 354323 ± 8% interrupts.CPU83.LOC:Local_timer_interrupts 447498 -20.7% 355042 ± 8% interrupts.CPU84.LOC:Local_timer_interrupts 484.50 ± 82% -80.6% 94.00 ± 47% interrupts.CPU84.RES:Rescheduling_interrupts 446794 -20.5% 355249 ± 8% interrupts.CPU85.LOC:Local_timer_interrupts 448191 -20.9% 354594 ± 8% interrupts.CPU86.LOC:Local_timer_interrupts 447635 -20.9% 353877 ± 8% interrupts.CPU87.LOC:Local_timer_interrupts 448706 -20.8% 355250 ± 8% interrupts.CPU88.LOC:Local_timer_interrupts 3738 +50.5% 5627 ± 33% interrupts.CPU88.NMI:Non-maskable_interrupts 3738 +50.5% 5627 ± 33% interrupts.CPU88.PMI:Performance_monitoring_interrupts 448312 -20.9% 354709 ± 8% interrupts.CPU89.LOC:Local_timer_interrupts 452329 -21.0% 357333 ± 8% interrupts.CPU9.LOC:Local_timer_interrupts 449864 -21.1% 354745 ± 8% interrupts.CPU90.LOC:Local_timer_interrupts 451193 -21.5% 353997 ± 8% interrupts.CPU91.LOC:Local_timer_interrupts 447166 -20.4% 356159 ± 8% interrupts.CPU92.LOC:Local_timer_interrupts 447364 -20.7% 354752 ± 8% interrupts.CPU93.LOC:Local_timer_interrupts 449764 -21.0% 355410 ± 8% interrupts.CPU94.LOC:Local_timer_interrupts 448393 -20.7% 355788 ± 8% interrupts.CPU95.LOC:Local_timer_interrupts 450191 -21.1% 355404 ± 8% interrupts.CPU96.LOC:Local_timer_interrupts 447651 -20.6% 355343 ± 8% interrupts.CPU97.LOC:Local_timer_interrupts 447679 -20.8% 354531 ± 8% interrupts.CPU98.LOC:Local_timer_interrupts 447448 -20.9% 353795 ± 8% interrupts.CPU99.LOC:Local_timer_interrupts 1.293e+08 -20.8% 1.024e+08 ± 8% interrupts.LOC:Local_timer_interrupts will-it-scale.per_process_ops 680 +---------------------------------------------------------------------+ | ++.++. .+ .++. + +.++.+ +.++.+.++. : ++.+.+ ++.++.+.++.++.| 660 |-+ + + +.+ ++.+.+ | 640 |-+ | | | 620 |-+ | | | 600 |-+ | | O O OO O O | 580 |-+O OO O O O OO | 560 |-O O O | | O O | 540 |-+ OO O O O OO O OO O | | O | 520 +---------------------------------------------------------------------+ will-it-scale.workload 200000 +------------------------------------------------------------------+ 195000 |.+ .+ +. +. .+ | | ++.++. +. +.+ + +.++.+ ++.+.++.+ : ++.++ +.++.++.++.++.| 190000 |-+ + + +.+ +.++.+ | 185000 |-+ | | | 180000 |-+ | 175000 |-+ | 170000 |-+ O O O | | O O O O O O | 165000 |-+ O O O O OO O | 160000 |-O | | O O OO OO OO OO O | 155000 |-+ O OO | 150000 +------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year

6
24
0 / 0

Re: [ext4] d3b6f23f71: stress-ng.fiemap.ops_per_sec -60.5% regression

by Xing Zhengjun

On 7/15/2020 7:04 PM, Ritesh Harjani wrote: > Hello Xing, > > On 4/7/20 1:30 PM, kernel test robot wrote: >> Greeting, >> >> FYI, we noticed a -60.5% regression of stress-ng.fiemap.ops_per_sec >> due to commit: >> >> >> commit: d3b6f23f71670007817a5d59f3fbafab2b794e8c ("ext4: move >> ext4_fiemap to use iomap framework") >> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master >> >> in testcase: stress-ng >> on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz >> with 192G memory >> with following parameters: >> >> nr_threads: 10% >> disk: 1HDD >> testtime: 1s >> class: os >> cpufreq_governor: performance >> ucode: 0x500002c >> fs: ext4 > > I started looking into this issue. But with my unit testing, I didn't > find any perf issue with fiemap ioctl call. I haven't yet explored about > how stress-ng take fiemap performance numbers, it could be doing > something differently. But in my testing I just made sure to create a > file with large number of extents and used xfs_io -c "fiemap -v" cmd > to check how much time it takes to read all the entries in 1st > and subsequent iterations. > > > Setup comprised of qemu machine on x86_64 with latest linux branch. > > 1. created a file of 10G using fallocate. (this allocated unwritten > extents for this file). > > 2. Then I punched hole on every alternate block of file. This step took > a long time. And after sufficiently long time, I had to cancel it. > for i in $(seq 1 2 xxxxx); do echo $i; fallocate -p -o $(($i*4096)) -l > 4096; done > > 3. Then issued fiemap call via xfs_io and took the time measurement. > time xfs_io -c "fiemap -v" bigfile > /dev/null > > > Perf numbers on latest default kernel build for above cmd. > > 1st iteration > ============== > real 0m31.684s > user 0m1.593s > sys 0m24.174s > > 2nd and subsequent iteration > ============================ > real 0m3.379s > user 0m1.300s > sys 0m2.080s > > > 4. Then I reverted all the iomap_fiemap patches and re-tested this. > With this the older ext4_fiemap implementation will be tested:- > > > 1st iteration > ============== > real 0m31.591s > user 0m1.400s > sys 0m24.243s > > > 2nd and subsequent iteration (had to cancel it since it was taking more > time then 15m) > ============================ > ^C^C > real 15m49.884s > user 0m0.032s > sys 15m49.722s > > I guess the reason why 2nd iteration with older implementation is taking > too much time is since with previous implementation we never cached > extent entries in extent_status tree. And also in 1st iteration the page > cache may get filled with lot of buffer_head entries. So maybe page > reclaims are taking more time. > > While with the latest implementation using iomap_fiemap(), the call to > query extent blocks is done using ext4_map_blocks(). ext4_map_blocks() > by default will also cache the extent entries into extent_status tree. > Hence during 2nd iteration, we will directly read the entries from > extent_status tree and will not do any disk I/O. > > -ritesh Could you try stress-ng( https://kernel.ubuntu.com/~cking/stress-ng/) test cases? The tarballs can be get from https://kernel.ubuntu.com/~cking/tarballs/stress-ng/. The command for this case you can try "stress-ng --timeout 1 --times --verify --metrics-brief --sequential 9 --class os --minimize --exclude spawn,exec,swap" I re-test it on the v5.8-rc6, the regression still existed. ========================================================================================= tbox_group/testcase/rootfs/kconfig/compiler/debug-setup/nr_threads/disk/testtime/fs/class/cpufreq_governor/ucode: lkp-csl-2sp5/stress-ng/debian-10.4-x86_64-20200603.cgz/x86_64-rhel-8.3/gcc-9/test/10%/1HDD/1s/ext4/os/performance/0x5002f01 commit: b2c5764262edded1b1cfff5a6ca82c3d61bb4a4a d3b6f23f71670007817a5d59f3fbafab2b794e8c v5.8-rc6 b2c5764262edded1 d3b6f23f71670007817a5d59f3f v5.8-rc6 ---------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev \ | \ | \ 20419 ± 3% -4.9% 19423 ± 4% +27.1% 25959 stress-ng.af-alg.ops 19655 ± 3% -5.7% 18537 ± 4% +27.8% 25111 stress-ng.af-alg.ops_per_sec 64.67 ± 5% -17.0% 53.67 ± 38% +22.2% 79.00 ± 9% stress-ng.chdir.ops 55.34 ± 3% -13.3% 47.99 ± 38% +26.4% 69.96 ± 10% stress-ng.chdir.ops_per_sec 64652 ± 7% -14.1% 55545 ± 11% -13.6% 55842 ± 6% stress-ng.chown.ops 64683 ± 7% -14.1% 55565 ± 11% -13.6% 55858 ± 6% stress-ng.chown.ops_per_sec 2805 ± 2% +0.6% 2820 ± 2% +130.0% 6452 stress-ng.clone.ops 2802 ± 2% +0.6% 2818 ± 2% +129.9% 6443 stress-ng.clone.ops_per_sec 34.67 +1.9% 35.33 ± 3% -9.6% 31.33 ± 3% stress-ng.copy-file.ops 22297 ± 23% +26.7% 28258 ± 2% +38.1% 30783 ± 14% stress-ng.dir.ops_per_sec 4743 ± 7% -12.6% 4147 ± 4% -19.9% 3801 ± 14% stress-ng.dirdeep.ops_per_sec 24499 ± 8% -9.3% 22223 ± 3% -19.8% 19647 ± 2% stress-ng.enosys.ops 24494 ± 8% -9.3% 22207 ± 3% -19.8% 19637 ± 2% stress-ng.enosys.ops_per_sec 76618 -63.8% 27756 ± 5% -55.5% 34092 ± 2% stress-ng.fiemap.ops 76042 -63.8% 27492 ± 5% -55.4% 33899 ± 2% stress-ng.fiemap.ops_per_sec 3816791 -6.2% 3581880 ± 2% -10.7% 3409584 ± 3% stress-ng.file-ioctl.ops 3817421 -6.2% 3582248 ± 2% -10.7% 3409544 ± 3% stress-ng.file-ioctl.ops_per_sec 27332 ± 17% +30.2% 35574 +37.2% 37506 ± 6% stress-ng.filename.ops 26222 ± 16% +28.6% 33722 +35.6% 35567 ± 6% stress-ng.filename.ops_per_sec 23431 ± 2% +0.5% 23559 +22.5% 28711 ± 4% stress-ng.fork.ops 23433 ± 2% +0.5% 23558 +22.5% 28709 ± 4% stress-ng.fork.ops_per_sec 48074 ± 15% -0.6% 47786 ± 9% -42.5% 27623 stress-ng.get.ops 48088 ± 15% -0.6% 47793 ± 9% -42.6% 27624 stress-ng.get.ops_per_sec 111379 -1.4% 109815 ± 2% -7.8% 102662 ± 4% stress-ng.getdent.ops 111398 -1.4% 109818 ± 2% -7.9% 102651 ± 4% stress-ng.getdent.ops_per_sec 226.67 ± 2% -1.0% 224.33 -15.1% 192.33 ± 6% stress-ng.inotify.ops 226.39 ± 2% -1.3% 223.53 -15.3% 191.82 ± 6% stress-ng.inotify.ops_per_sec 18.00 +9.3% 19.67 ± 2% +3.7% 18.67 ± 2% stress-ng.iomix.ops 4846 ± 10% +2.9% 4984 ± 7% -36.8% 3062 ± 7% stress-ng.kill.ops 4846 ± 10% +2.8% 4982 ± 7% -36.9% 3058 ± 7% stress-ng.kill.ops_per_sec 851324 ± 3% -9.9% 766988 ± 2% +2.3% 870703 stress-ng.lease.ops 850238 ± 3% -9.9% 766255 ± 2% +2.3% 870047 stress-ng.lease.ops_per_sec 134492 ± 14% -7.0% 125085 ± 16% -19.9% 107736 ± 3% stress-ng.mlock.ops_per_sec 1.38 +2.9% 1.42 ± 5% -20.2% 1.10 ± 12% stress-ng.mmapfork.ops_per_sec 1117721 ± 5% -9.2% 1015321 ± 20% -36.2% 713598 stress-ng.mq.ops 1117186 ± 5% -9.2% 1014593 ± 20% -36.2% 712387 stress-ng.mq.ops_per_sec 4584592 ± 2% -2.7% 4460470 ± 7% -12.0% 4036627 ± 8% stress-ng.msg.ops_per_sec 8179 ± 10% +7.3% 8772 ± 9% +24.0% 10144 stress-ng.opcode.ops 8177 ± 10% +7.2% 8770 ± 9% +24.0% 10141 stress-ng.opcode.ops_per_sec 5218 ± 3% +3.6% 5408 ± 2% +27.3% 6642 stress-ng.reboot.ops 5217 ± 3% +3.6% 5407 ± 2% +27.3% 6640 stress-ng.reboot.ops_per_sec 1724 ± 21% +27.9% 2205 ± 18% +54.3% 2660 ± 19% stress-ng.resources.ops 1217728 -31.4% 835808 ± 14% -18.7% 989423 ± 10% stress-ng.revio.ops 1217899 -31.4% 835846 ± 14% -18.8% 989313 ± 10% stress-ng.revio.ops_per_sec 752664 ± 9% -7.6% 695116 ± 10% -0.6% 748117 ± 6% stress-ng.seal.ops 752957 ± 9% -7.7% 695278 ± 10% -0.6% 748068 ± 6% stress-ng.seal.ops_per_sec 628.00 ± 83% -20.1% 502.00 ± 8% +176.2% 1734 stress-ng.seccomp.ops 628.22 ± 83% -20.1% 502.08 ± 8% +176.1% 1734 stress-ng.seccomp.ops_per_sec 1361741 ± 7% +4.0% 1416628 ± 8% +17.2% 1596549 ± 9% stress-ng.seek.ops 1361799 ± 7% +4.0% 1416578 ± 8% +17.2% 1596376 ± 9% stress-ng.seek.ops_per_sec 1537496 +1.2% 1556468 -9.9% 1385679 stress-ng.sigrt.ops 1532297 +1.3% 1551993 -9.9% 1381175 stress-ng.sigrt.ops_per_sec 1324589 ± 12% -26.6% 972626 ± 9% -14.8% 1128923 ± 4% stress-ng.sigsuspend.ops 1323957 ± 12% -26.6% 972084 ± 9% -14.8% 1128174 ± 4% stress-ng.sigsuspend.ops_per_sec 806.67 ± 13% +11.2% 896.67 ± 10% +31.3% 1059 ± 4% stress-ng.sockabuse.ops 806.73 ± 13% +11.1% 896.44 ± 10% +31.3% 1059 ± 4% stress-ng.sockabuse.ops_per_sec 6224635 ± 2% +0.8% 6274521 ± 2% -12.8% 5427883 ± 2% stress-ng.sockpair.ops 5383397 -1.9% 5279526 ± 2% -14.9% 4579084 ± 4% stress-ng.sockpair.ops_per_sec 22409 ± 6% -2.8% 21784 ± 5% -41.2% 13172 ± 9% stress-ng.sysbadaddr.ops 22414 ± 6% -2.8% 21784 ± 5% -41.3% 13167 ± 9% stress-ng.sysbadaddr.ops_per_sec 525.33 ± 13% +2.6% 539.00 ± 15% -42.1% 304.33 ± 25% stress-ng.sysfs.ops 524.98 ± 13% +2.6% 538.58 ± 15% -42.1% 303.79 ± 25% stress-ng.sysfs.ops_per_sec 3077 ± 12% +19.9% 3689 ± 4% +23.0% 3786 ± 4% stress-ng.sysinfo.ops 3009 ± 11% +22.6% 3688 ± 4% +25.7% 3783 ± 4% stress-ng.sysinfo.ops_per_sec 3001932 +1.2% 3038017 +15.7% 3474274 stress-ng.udp.ops 3002618 +1.2% 3037939 +15.7% 3473606 stress-ng.udp.ops_per_sec 184634 ± 12% -6.9% 171823 +24.5% 229788 ± 4% stress-ng.utime.ops 184678 ± 12% -7.0% 171836 +24.4% 229784 ± 4% stress-ng.utime.ops_per_sec 1835 ± 5% +2.2% 1875 ± 4% +8.2% 1986 ± 3% stress-ng.vforkmany.ops 1829 ± 5% +1.9% 1863 ± 4% +8.1% 1978 ± 3% stress-ng.vforkmany.ops_per_sec 14829 -1.0% 14682 ± 2% -20.1% 11848 ± 3% stress-ng.vm-segv.ops 14830 -1.0% 14682 ± 2% -29.7% 10433 ± 13% stress-ng.vm-segv.ops_per_sec 3527835 ± 3% +1.9% 3596613 ± 5% +495.6% 21010377 stress-ng.vm-splice.ops 3528845 ± 3% +1.9% 3597219 ± 5% +495.4% 21011045 stress-ng.vm-splice.ops_per_sec -- Zhengjun Xing

1 year, 1 month

2
2
0 / 0

[rcu] 276c410448: will-it-scale.per_thread_ops -12.3% regression

by kernel test robot

Greeting, FYI, we noticed a -12.3% regression of will-it-scale.per_thread_ops due to commit: commit: 276c410448dbca357a2bc3539acfe04862e5f172 ("rcu-tasks: Split ->trc_reader_need_end") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory with following parameters: nr_task: 100% mode: thread test: page_fault3 cpufreq_governor: performance ucode: 0x11 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-knm01/page_fault3/will-it-scale/0x11 commit: b0afa0f056 ("rcu-tasks: Provide boot parameter to delay IPIs until late in grace period") 276c410448 ("rcu-tasks: Split ->trc_reader_need_end") b0afa0f056676ffe 276c410448dbca357a2bc3539ac ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 2:4 -50% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x :4 28% 1:4 perf-profile.calltrace.cycles-pp.error_entry 0:4 0% 0:4 perf-profile.children.cycles-pp.error_exit 1:4 7% 2:4 perf-profile.children.cycles-pp.error_entry 0:4 4% 1:4 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 1414 -12.3% 1241 ± 2% will-it-scale.per_thread_ops 463.32 +1.7% 470.99 will-it-scale.time.elapsed_time 463.32 +1.7% 470.99 will-it-scale.time.elapsed_time.max 407566 -12.3% 357573 ± 2% will-it-scale.workload 48.51 -1.5% 47.77 boot-time.boot 7.203e+10 +20.0% 8.64e+10 ± 2% cpuidle.C1.time 2.162e+08 ± 2% +27.7% 2.761e+08 ± 2% cpuidle.C1.usage 60.50 +12.2 72.74 ± 2% mpstat.cpu.all.idle% 39.17 -12.2 26.97 ± 6% mpstat.cpu.all.sys% 2334 ± 12% +18.8% 2772 ± 5% slabinfo.khugepaged_mm_slot.active_objs 2334 ± 12% +18.8% 2772 ± 5% slabinfo.khugepaged_mm_slot.num_objs 60.25 +20.3% 72.50 ± 2% vmstat.cpu.id 92.75 ± 3% -21.6% 72.75 ± 5% vmstat.procs.r 223709 +41.8% 317250 ± 3% vmstat.system.cs 641687 ± 3% +8.0% 693245 ± 2% proc-vmstat.nr_inactive_anon 641688 ± 3% +8.0% 693245 ± 2% proc-vmstat.nr_zone_inactive_anon 166782 -3.7% 160632 proc-vmstat.numa_hint_faults 166782 -3.7% 160632 proc-vmstat.numa_hint_faults_local 984.25 -14.2% 844.75 ± 2% proc-vmstat.numa_huge_pte_updates 710979 -11.2% 631134 proc-vmstat.numa_pte_updates 1.967e+08 -10.9% 1.752e+08 proc-vmstat.pgfault 58.18 +3.4% 60.17 perf-stat.i.MPKI 1.173e+09 +10.5% 1.296e+09 perf-stat.i.branch-instructions 6.74 -0.1 6.68 perf-stat.i.branch-miss-rate% 72495831 +10.7% 80219684 perf-stat.i.branch-misses 14.68 -0.6 14.06 perf-stat.i.cache-miss-rate% 43014696 +10.5% 47551690 perf-stat.i.cache-misses 2.936e+08 +15.6% 3.393e+08 perf-stat.i.cache-references 227441 +42.0% 323034 ± 3% perf-stat.i.context-switches 37.22 -29.0% 26.44 ± 5% perf-stat.i.cpi 1.828e+11 -22.3% 1.421e+11 ± 3% perf-stat.i.cpu-cycles 513.71 +13.6% 583.63 perf-stat.i.cpu-migrations 4303 -27.8% 3107 ± 5% perf-stat.i.cycles-between-cache-misses 1.78 -0.0 1.74 perf-stat.i.iTLB-load-miss-rate% 90757979 +9.1% 98982336 perf-stat.i.iTLB-load-misses 5.518e+09 +10.1% 6.076e+09 perf-stat.i.iTLB-loads 5.501e+09 +10.3% 6.065e+09 perf-stat.i.instructions 59.50 +2.4% 60.91 perf-stat.i.instructions-per-iTLB-miss 0.03 +48.9% 0.05 ± 3% perf-stat.i.ipc 0.61 -22.2% 0.47 ± 3% perf-stat.i.metric.GHz 0.85 ± 2% +6.3% 0.90 ± 3% perf-stat.i.metric.K/sec 23.19 +10.3% 25.58 perf-stat.i.metric.M/sec 398027 -13.3% 345187 ± 2% perf-stat.i.minor-faults 398027 -13.3% 345187 ± 2% perf-stat.i.page-faults 51.63 +4.7% 54.06 perf-stat.overall.MPKI 14.82 -0.6 14.26 perf-stat.overall.cache-miss-rate% 32.82 -28.9% 23.34 ± 4% perf-stat.overall.cpi 4290 -29.4% 3029 ± 5% perf-stat.overall.cycles-between-cache-misses 0.03 +40.9% 0.04 ± 4% perf-stat.overall.ipc 6447068 +28.0% 8254132 ± 3% perf-stat.overall.path-length 1.214e+09 +10.5% 1.341e+09 perf-stat.ps.branch-instructions 72454767 +10.7% 80188768 perf-stat.ps.branch-misses 43379426 +11.2% 48228470 perf-stat.ps.cache-misses 2.927e+08 +15.5% 3.382e+08 ± 2% perf-stat.ps.cache-references 223136 +42.0% 316818 ± 3% perf-stat.ps.context-switches 1.861e+11 -21.6% 1.459e+11 ± 3% perf-stat.ps.cpu-cycles 498.80 +14.0% 568.54 perf-stat.ps.cpu-migrations 90216145 +8.9% 98279965 perf-stat.ps.iTLB-load-misses 5.688e+09 +10.2% 6.267e+09 perf-stat.ps.iTLB-loads 5.67e+09 +10.3% 6.256e+09 perf-stat.ps.instructions 404137 -13.0% 351604 ± 2% perf-stat.ps.minor-faults 404137 -13.0% 351604 ± 2% perf-stat.ps.page-faults 2.627e+12 +12.2% 2.949e+12 perf-stat.total.instructions 84210 -25.9% 62375 ± 5% sched_debug.cfs_rq:/.exec_clock.avg 91870 ± 3% -23.3% 70486 ± 5% sched_debug.cfs_rq:/.exec_clock.max 79683 -23.1% 61285 ± 6% sched_debug.cfs_rq:/.exec_clock.min 2299 ± 7% +36.5% 3139 ± 12% sched_debug.cfs_rq:/.load.avg 216559 +81.8% 393746 ± 14% sched_debug.cfs_rq:/.load.max 16561 ± 3% +64.0% 27165 ± 14% sched_debug.cfs_rq:/.load.stddev 2.32 ± 11% -19.6% 1.87 ± 14% sched_debug.cfs_rq:/.load_avg.avg 18373664 ± 3% -19.7% 14762699 ± 8% sched_debug.cfs_rq:/.min_vruntime.avg 18652687 ± 3% -19.9% 14948853 ± 8% sched_debug.cfs_rq:/.min_vruntime.max 17259651 ± 3% -20.6% 13706324 ± 9% sched_debug.cfs_rq:/.min_vruntime.min 0.09 ± 4% -30.2% 0.06 ± 8% sched_debug.cfs_rq:/.nr_running.avg 0.27 ± 2% -14.3% 0.24 ± 5% sched_debug.cfs_rq:/.nr_running.stddev 18.07 ± 2% -31.0% 12.47 ± 5% sched_debug.cfs_rq:/.nr_spread_over.avg 1.97 ± 34% -49.2% 1.00 ± 57% sched_debug.cfs_rq:/.nr_spread_over.min 258.66 ± 6% -38.4% 159.22 ± 5% sched_debug.cfs_rq:/.runnable_avg.avg 231.14 -13.6% 199.77 ± 7% sched_debug.cfs_rq:/.runnable_avg.stddev 302282 ± 40% -44.4% 168177 ± 29% sched_debug.cfs_rq:/.spread0.avg 568464 ± 36% -39.3% 344858 ± 16% sched_debug.cfs_rq:/.spread0.max 252.26 ± 6% -38.7% 154.72 ± 5% sched_debug.cfs_rq:/.util_avg.avg 217.63 -15.4% 184.13 ± 6% sched_debug.cfs_rq:/.util_avg.stddev 18.44 ± 12% -50.3% 9.17 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.avg 575.84 ± 17% -22.2% 447.81 ± 16% sched_debug.cfs_rq:/.util_est_enqueued.max 67.91 ± 9% -35.4% 43.85 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.stddev 417156 ± 7% -15.2% 353896 ± 5% sched_debug.cpu.avg_idle.stddev 273.72 ± 3% -11.3% 242.79 ± 4% sched_debug.cpu.clock.stddev 273.72 ± 3% -11.3% 242.79 ± 4% sched_debug.cpu.clock_task.stddev 426.78 ± 3% -27.6% 309.10 ± 12% sched_debug.cpu.curr->pid.avg 1282 ± 2% -12.4% 1123 ± 6% sched_debug.cpu.curr->pid.stddev 351857 ± 3% -16.4% 294246 ± 4% sched_debug.cpu.max_idle_balance_cost.stddev 0.00 -11.8% 0.00 ± 3% sched_debug.cpu.next_balance.stddev 0.09 ± 3% -32.0% 0.06 ± 12% sched_debug.cpu.nr_running.avg 0.28 -14.6% 0.24 ± 7% sched_debug.cpu.nr_running.stddev 163725 +37.4% 225040 ± 4% sched_debug.cpu.nr_switches.avg 174783 +35.4% 236604 ± 4% sched_debug.cpu.nr_switches.max 151538 +36.4% 206743 ± 4% sched_debug.cpu.nr_switches.min 2143 ± 4% +28.5% 2753 ± 4% sched_debug.cpu.nr_switches.stddev 160974 +38.1% 222280 ± 4% sched_debug.cpu.sched_count.avg 170145 +36.7% 232581 ± 4% sched_debug.cpu.sched_count.max 148944 +37.6% 204888 ± 4% sched_debug.cpu.sched_count.min 1847 ± 8% +38.7% 2563 ± 6% sched_debug.cpu.sched_count.stddev 80136 +38.3% 110820 ± 4% sched_debug.cpu.sched_goidle.avg 84559 +36.9% 115782 ± 4% sched_debug.cpu.sched_goidle.max 74091 +37.8% 102072 ± 4% sched_debug.cpu.sched_goidle.min 891.35 ± 6% +41.4% 1260 ± 6% sched_debug.cpu.sched_goidle.stddev 80346 +38.2% 111003 ± 4% sched_debug.cpu.ttwu_count.avg 114308 ± 2% +42.7% 163080 ± 6% sched_debug.cpu.ttwu_count.max 62759 ± 3% +38.4% 86881 ± 2% sched_debug.cpu.ttwu_count.min 8223 ± 7% +53.5% 12623 ± 13% sched_debug.cpu.ttwu_count.stddev 2.35 ± 10% -0.7 1.65 ± 14% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.94 ± 15% -0.5 0.44 ± 58% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.rwsem_down_read_slowpath.do_user_addr_fault.page_fault 0.90 ± 15% -0.5 0.42 ± 58% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.rwsem_down_read_slowpath.do_user_addr_fault.page_fault 1.38 ± 4% -0.4 0.94 ± 12% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 0.81 ± 4% +0.1 0.91 ± 4% perf-profile.calltrace.cycles-pp.file_update_time.fault_dirty_shared_page.do_fault.__handle_mm_fault.handle_mm_fault 1.35 ± 6% +0.2 1.51 ± 3% perf-profile.calltrace.cycles-pp.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault 1.16 ± 7% +0.2 1.33 ± 5% perf-profile.calltrace.cycles-pp.unlock_page.fault_dirty_shared_page.do_fault.__handle_mm_fault.handle_mm_fault 1.29 ± 8% +0.2 1.46 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault 1.76 ± 7% +0.3 2.04 ± 5% perf-profile.calltrace.cycles-pp.page_add_file_rmap.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault 2.13 ± 5% +0.3 2.41 ± 4% perf-profile.calltrace.cycles-pp.fault_dirty_shared_page.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault 2.46 ± 8% +0.4 2.81 ± 7% perf-profile.calltrace.cycles-pp.up_read.do_user_addr_fault.page_fault 2.86 ± 6% +0.4 3.21 ± 4% perf-profile.calltrace.cycles-pp.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault.do_fault 3.12 ± 6% +0.4 3.52 ± 4% perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_fault.__do_fault.do_fault.__handle_mm_fault 3.27 ± 6% +0.4 3.67 ± 4% perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.do_fault.__handle_mm_fault.handle_mm_fault 3.33 ± 6% +0.4 3.75 ± 4% perf-profile.calltrace.cycles-pp.__do_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault 0.13 ±173% +0.4 0.55 ± 8% perf-profile.calltrace.cycles-pp.swapgs_restore_regs_and_return_to_usermode 3.39 ± 7% +0.5 3.89 ± 5% perf-profile.calltrace.cycles-pp.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault.handle_mm_fault 3.44 ± 7% +0.5 3.94 ± 5% perf-profile.calltrace.cycles-pp.finish_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault 8.99 ± 6% +1.2 10.20 ± 4% perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.page_fault 12.23 ± 6% +1.6 13.84 ± 5% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.page_fault 12.68 ± 6% +1.7 14.38 ± 5% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.page_fault 45.14 ± 9% +6.1 51.25 ± 6% perf-profile.calltrace.cycles-pp.down_read_trylock.do_user_addr_fault.page_fault 66.96 ± 7% +7.4 74.37 ± 4% perf-profile.calltrace.cycles-pp.do_user_addr_fault.page_fault 69.19 ± 7% +7.7 76.89 ± 4% perf-profile.calltrace.cycles-pp.page_fault 17.74 ± 8% -3.3 14.46 ± 20% perf-profile.children.cycles-pp.apic_timer_interrupt 17.02 ± 8% -3.1 13.89 ± 20% perf-profile.children.cycles-pp.smp_apic_timer_interrupt 2.97 ± 8% -0.7 2.25 ± 12% perf-profile.children.cycles-pp.irq_exit 1.88 ± 11% -0.5 1.41 ± 11% perf-profile.children.cycles-pp.__softirqentry_text_start 1.02 ± 14% -0.4 0.59 ± 15% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 1.75 ± 12% -0.4 1.35 ± 14% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.92 ± 12% -0.2 0.68 ± 17% perf-profile.children.cycles-pp.ktime_get 0.32 ± 20% -0.2 0.13 ± 30% perf-profile.children.cycles-pp.munmap 0.32 ± 22% -0.2 0.14 ± 27% perf-profile.children.cycles-pp.__x64_sys_munmap 0.32 ± 22% -0.2 0.14 ± 28% perf-profile.children.cycles-pp.__vm_munmap 0.29 ± 21% -0.2 0.13 ± 23% perf-profile.children.cycles-pp.__do_munmap 0.28 ± 21% -0.2 0.12 ± 25% perf-profile.children.cycles-pp.unmap_region 0.28 ± 22% -0.2 0.13 ± 23% perf-profile.children.cycles-pp.unmap_vmas 0.28 ± 21% -0.2 0.13 ± 23% perf-profile.children.cycles-pp.unmap_page_range 0.48 ± 6% -0.1 0.34 ± 15% perf-profile.children.cycles-pp.rebalance_domains 0.56 ± 12% -0.1 0.43 ± 16% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.44 ± 11% -0.1 0.33 ± 25% perf-profile.children.cycles-pp.clockevents_program_event 0.31 ± 14% -0.1 0.21 ± 16% perf-profile.children.cycles-pp.run_rebalance_domains 0.34 ± 9% -0.1 0.25 ± 16% perf-profile.children.cycles-pp.__hrtimer_next_event_base 0.28 ± 14% -0.1 0.21 ± 22% perf-profile.children.cycles-pp.rcu_dynticks_eqs_exit 0.12 ± 21% -0.1 0.04 ± 59% perf-profile.children.cycles-pp.smp_call_function_many_cond 0.12 ± 21% -0.1 0.04 ± 59% perf-profile.children.cycles-pp.on_each_cpu_cond_mask 0.12 ± 21% -0.1 0.05 ± 60% perf-profile.children.cycles-pp.flush_tlb_mm_range 0.19 ± 16% -0.1 0.12 ± 25% perf-profile.children.cycles-pp.irq_work_tick 0.31 ± 10% -0.1 0.24 ± 21% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.17 ± 13% -0.0 0.12 ± 15% perf-profile.children.cycles-pp.rcu_idle_exit 0.12 ± 12% -0.0 0.08 ± 30% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.14 ± 8% -0.0 0.10 ± 12% perf-profile.children.cycles-pp.raise_softirq 0.09 ± 7% +0.0 0.11 ± 4% perf-profile.children.cycles-pp._cond_resched 0.06 +0.0 0.08 ± 11% perf-profile.children.cycles-pp.lock_page_memcg 0.15 ± 7% +0.0 0.17 ± 2% perf-profile.children.cycles-pp.__mod_memcg_state 0.07 +0.0 0.09 ± 4% perf-profile.children.cycles-pp.perf_swevent_event 0.18 ± 7% +0.0 0.21 ± 4% perf-profile.children.cycles-pp.current_time 0.23 ± 3% +0.0 0.26 ± 4% perf-profile.children.cycles-pp.xas_load 0.14 ± 9% +0.0 0.18 ± 8% perf-profile.children.cycles-pp.__count_memcg_events 0.46 ± 5% +0.1 0.51 ± 4% perf-profile.children.cycles-pp.__mod_lruvec_state 0.42 ± 5% +0.1 0.48 ± 7% perf-profile.children.cycles-pp.__mark_inode_dirty 0.55 ± 5% +0.1 0.61 ± 3% perf-profile.children.cycles-pp.___perf_sw_event 0.47 ± 4% +0.1 0.55 ± 8% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.42 ± 9% +0.1 0.50 ± 8% perf-profile.children.cycles-pp.__might_sleep 0.81 ± 5% +0.1 0.92 ± 4% perf-profile.children.cycles-pp.file_update_time 1.36 ± 6% +0.2 1.52 ± 3% perf-profile.children.cycles-pp.find_get_entry 1.17 ± 7% +0.2 1.34 ± 5% perf-profile.children.cycles-pp.unlock_page 1.77 ± 7% +0.3 2.05 ± 5% perf-profile.children.cycles-pp.page_add_file_rmap 2.14 ± 5% +0.3 2.42 ± 4% perf-profile.children.cycles-pp.fault_dirty_shared_page 2.51 ± 8% +0.4 2.86 ± 7% perf-profile.children.cycles-pp.up_read 2.90 ± 6% +0.4 3.26 ± 4% perf-profile.children.cycles-pp.find_lock_entry 3.12 ± 6% +0.4 3.52 ± 4% perf-profile.children.cycles-pp.shmem_getpage_gfp 3.27 ± 6% +0.4 3.68 ± 4% perf-profile.children.cycles-pp.shmem_fault 3.33 ± 6% +0.4 3.75 ± 3% perf-profile.children.cycles-pp.__do_fault 3.45 ± 7% +0.5 3.95 ± 5% perf-profile.children.cycles-pp.finish_fault 3.42 ± 7% +0.5 3.92 ± 5% perf-profile.children.cycles-pp.alloc_set_pte 9.02 ± 6% +1.2 10.23 ± 4% perf-profile.children.cycles-pp.do_fault 12.26 ± 6% +1.6 13.88 ± 5% perf-profile.children.cycles-pp.__handle_mm_fault 12.72 ± 6% +1.7 14.43 ± 5% perf-profile.children.cycles-pp.handle_mm_fault 45.27 ± 9% +6.1 51.39 ± 6% perf-profile.children.cycles-pp.down_read_trylock 67.33 ± 7% +7.5 74.81 ± 4% perf-profile.children.cycles-pp.do_user_addr_fault 69.24 ± 7% +7.7 76.95 ± 4% perf-profile.children.cycles-pp.page_fault 1.02 ± 14% -0.4 0.59 ± 15% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.74 ± 13% -0.2 0.55 ± 18% perf-profile.self.cycles-pp.ktime_get 0.28 ± 10% -0.1 0.21 ± 17% perf-profile.self.cycles-pp.__hrtimer_next_event_base 0.28 ± 13% -0.1 0.20 ± 21% perf-profile.self.cycles-pp.rcu_dynticks_eqs_exit 0.19 ± 16% -0.1 0.12 ± 25% perf-profile.self.cycles-pp.irq_work_tick 0.26 ± 7% -0.1 0.21 ± 12% perf-profile.self.cycles-pp.__softirqentry_text_start 0.20 ± 15% -0.0 0.16 ± 11% perf-profile.self.cycles-pp.__run_timers 0.14 ± 7% -0.0 0.10 ± 12% perf-profile.self.cycles-pp.raise_softirq 0.13 ± 9% -0.0 0.10 ± 15% perf-profile.self.cycles-pp.tick_sched_timer 0.07 ± 10% -0.0 0.04 ± 58% perf-profile.self.cycles-pp.clockevents_program_event 0.09 ± 5% -0.0 0.07 ± 17% perf-profile.self.cycles-pp.apic_timer_interrupt 0.05 ± 8% +0.0 0.07 ± 12% perf-profile.self.cycles-pp.lock_page_memcg 0.11 ± 4% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.file_update_time 0.10 ± 7% +0.0 0.12 ± 7% perf-profile.self.cycles-pp.___might_sleep 0.06 ± 6% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.perf_swevent_event 0.18 ± 8% +0.0 0.20 ± 6% perf-profile.self.cycles-pp.__mod_lruvec_state 0.13 ± 9% +0.0 0.15 ± 3% perf-profile.self.cycles-pp.__mod_memcg_state 0.24 ± 6% +0.0 0.27 ± 6% perf-profile.self.cycles-pp.handle_mm_fault 0.41 ± 4% +0.0 0.44 ± 3% perf-profile.self.cycles-pp.___perf_sw_event 0.14 ± 9% +0.0 0.17 ± 6% perf-profile.self.cycles-pp.__count_memcg_events 0.21 ± 6% +0.0 0.25 ± 4% perf-profile.self.cycles-pp.shmem_getpage_gfp 0.23 ± 10% +0.1 0.29 ± 13% perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode 1.02 ± 6% +0.1 1.13 ± 6% perf-profile.self.cycles-pp.find_get_entry 0.86 ± 4% +0.1 1.00 ± 6% perf-profile.self.cycles-pp.do_user_addr_fault 1.10 ± 6% +0.2 1.26 ± 7% perf-profile.self.cycles-pp.unlock_page 1.26 ± 7% +0.2 1.44 ± 8% perf-profile.self.cycles-pp.find_lock_entry 1.20 ± 6% +0.2 1.38 ± 8% perf-profile.self.cycles-pp.page_add_file_rmap 2.35 ± 7% +0.3 2.69 ± 8% perf-profile.self.cycles-pp.up_read 2.97 ± 6% +0.4 3.35 ± 8% perf-profile.self.cycles-pp.__handle_mm_fault 42.09 ± 8% +5.7 47.79 ± 8% perf-profile.self.cycles-pp.down_read_trylock 406.00 ± 37% +1859.5% 7955 ±151% interrupts.32:IR-PCI-MSI.2097155-edge.eth0-TxRx-2 3009113 -4.3% 2880393 ± 2% interrupts.CAL:Function_call_interrupts 1445 ± 50% +93.0% 2788 ± 9% interrupts.CPU0.NMI:Non-maskable_interrupts 1445 ± 50% +93.0% 2788 ± 9% interrupts.CPU0.PMI:Performance_monitoring_interrupts 939.50 ± 16% +23.2% 1157 ± 5% interrupts.CPU100.RES:Rescheduling_interrupts 1223 ± 11% +43.3% 1753 ± 23% interrupts.CPU103.NMI:Non-maskable_interrupts 1223 ± 11% +43.3% 1753 ± 23% interrupts.CPU103.PMI:Performance_monitoring_interrupts 1163 ± 6% +24.5% 1448 ± 6% interrupts.CPU104.RES:Rescheduling_interrupts 1062 ± 15% +40.5% 1493 ± 5% interrupts.CPU105.RES:Rescheduling_interrupts 1490 ± 28% +70.1% 2536 ± 20% interrupts.CPU106.NMI:Non-maskable_interrupts 1490 ± 28% +70.1% 2536 ± 20% interrupts.CPU106.PMI:Performance_monitoring_interrupts 1186 ± 3% +26.4% 1499 ± 11% interrupts.CPU106.RES:Rescheduling_interrupts 1186 ± 5% +36.5% 1618 ± 9% interrupts.CPU107.RES:Rescheduling_interrupts 1484 ± 29% +98.1% 2941 ± 9% interrupts.CPU108.NMI:Non-maskable_interrupts 1484 ± 29% +98.1% 2941 ± 9% interrupts.CPU108.PMI:Performance_monitoring_interrupts 1087 ± 11% +35.2% 1470 ± 11% interrupts.CPU108.RES:Rescheduling_interrupts 905.00 ± 14% +26.9% 1148 ± 8% interrupts.CPU109.RES:Rescheduling_interrupts 1534 ± 35% +91.6% 2939 ± 9% interrupts.CPU110.NMI:Non-maskable_interrupts 1534 ± 35% +91.6% 2939 ± 9% interrupts.CPU110.PMI:Performance_monitoring_interrupts 1222 ± 11% +54.3% 1885 ± 44% interrupts.CPU111.NMI:Non-maskable_interrupts 1222 ± 11% +54.3% 1885 ± 44% interrupts.CPU111.PMI:Performance_monitoring_interrupts 1066 +27.7% 1360 ± 5% interrupts.CPU111.RES:Rescheduling_interrupts 1214 ± 12% +51.1% 1833 ± 35% interrupts.CPU112.NMI:Non-maskable_interrupts 1214 ± 12% +51.1% 1833 ± 35% interrupts.CPU112.PMI:Performance_monitoring_interrupts 1206 ± 12% +30.8% 1577 ± 9% interrupts.CPU116.RES:Rescheduling_interrupts 1102 ± 10% +46.7% 1616 ± 11% interrupts.CPU117.RES:Rescheduling_interrupts 1211 ± 3% +35.2% 1638 ± 15% interrupts.CPU118.RES:Rescheduling_interrupts 1217 ± 11% +115.4% 2623 ± 29% interrupts.CPU119.NMI:Non-maskable_interrupts 1217 ± 11% +115.4% 2623 ± 29% interrupts.CPU119.PMI:Performance_monitoring_interrupts 406.00 ± 37% +1859.5% 7955 ±151% interrupts.CPU12.32:IR-PCI-MSI.2097155-edge.eth0-TxRx-2 1125 ± 12% +27.3% 1432 ± 10% interrupts.CPU12.NMI:Non-maskable_interrupts 1125 ± 12% +27.3% 1432 ± 10% interrupts.CPU12.PMI:Performance_monitoring_interrupts 1173 ± 12% +55.5% 1823 ± 35% interrupts.CPU120.NMI:Non-maskable_interrupts 1173 ± 12% +55.5% 1823 ± 35% interrupts.CPU120.PMI:Performance_monitoring_interrupts 1112 ± 5% +40.2% 1559 ± 6% interrupts.CPU121.RES:Rescheduling_interrupts 1227 ± 11% +50.5% 1846 ± 34% interrupts.CPU122.NMI:Non-maskable_interrupts 1227 ± 11% +50.5% 1846 ± 34% interrupts.CPU122.PMI:Performance_monitoring_interrupts 1036 ± 11% +36.6% 1416 ± 4% interrupts.CPU122.RES:Rescheduling_interrupts 1164 ± 22% +40.8% 1639 ± 16% interrupts.CPU123.RES:Rescheduling_interrupts 1419 ± 13% +20.7% 1712 ± 9% interrupts.CPU124.RES:Rescheduling_interrupts 1209 ± 11% +21.5% 1469 ± 9% interrupts.CPU126.NMI:Non-maskable_interrupts 1209 ± 11% +21.5% 1469 ± 9% interrupts.CPU126.PMI:Performance_monitoring_interrupts 1266 ± 16% +34.9% 1708 ± 13% interrupts.CPU126.RES:Rescheduling_interrupts 1207 ± 24% +36.5% 1648 ± 11% interrupts.CPU127.RES:Rescheduling_interrupts 1298 ± 12% +30.8% 1698 interrupts.CPU128.RES:Rescheduling_interrupts 1354 ± 7% +33.5% 1808 ± 9% interrupts.CPU129.RES:Rescheduling_interrupts 1046 ± 7% +42.3% 1488 ± 17% interrupts.CPU130.RES:Rescheduling_interrupts 1068 ± 6% +45.9% 1558 ± 13% interrupts.CPU131.RES:Rescheduling_interrupts 1200 ± 2% +41.7% 1701 ± 14% interrupts.CPU132.RES:Rescheduling_interrupts 1191 ± 11% +85.4% 2208 ± 39% interrupts.CPU133.NMI:Non-maskable_interrupts 1191 ± 11% +85.4% 2208 ± 39% interrupts.CPU133.PMI:Performance_monitoring_interrupts 1327 ± 9% +13.8% 1510 ± 4% interrupts.CPU134.RES:Rescheduling_interrupts 1203 ± 12% +52.8% 1837 ± 34% interrupts.CPU136.NMI:Non-maskable_interrupts 1203 ± 12% +52.8% 1837 ± 34% interrupts.CPU136.PMI:Performance_monitoring_interrupts 976.75 ± 8% +19.0% 1162 ± 8% interrupts.CPU136.RES:Rescheduling_interrupts 1438 ± 34% +75.9% 2530 ± 27% interrupts.CPU137.NMI:Non-maskable_interrupts 1438 ± 34% +75.9% 2530 ± 27% interrupts.CPU137.PMI:Performance_monitoring_interrupts 950.75 ± 5% +33.1% 1265 ± 14% interrupts.CPU139.RES:Rescheduling_interrupts 1054 ± 6% +26.9% 1338 ± 8% interrupts.CPU140.RES:Rescheduling_interrupts 10703 ± 3% -8.0% 9849 ± 4% interrupts.CPU140.TLB:TLB_shootdowns 1509 ± 29% +71.5% 2589 ± 27% interrupts.CPU141.NMI:Non-maskable_interrupts 1509 ± 29% +71.5% 2589 ± 27% interrupts.CPU141.PMI:Performance_monitoring_interrupts 1334 ± 7% +14.9% 1533 ± 8% interrupts.CPU141.RES:Rescheduling_interrupts 1505 ± 36% +71.7% 2585 ± 27% interrupts.CPU142.NMI:Non-maskable_interrupts 1505 ± 36% +71.7% 2585 ± 27% interrupts.CPU142.PMI:Performance_monitoring_interrupts 1206 ± 7% +32.3% 1596 ± 13% interrupts.CPU142.RES:Rescheduling_interrupts 1218 ± 11% +55.0% 1888 ± 44% interrupts.CPU143.NMI:Non-maskable_interrupts 1218 ± 11% +55.0% 1888 ± 44% interrupts.CPU143.PMI:Performance_monitoring_interrupts 1883 ± 5% +37.7% 2594 ± 12% interrupts.CPU143.RES:Rescheduling_interrupts 1211 ± 11% +51.1% 1831 ± 35% interrupts.CPU144.NMI:Non-maskable_interrupts 1211 ± 11% +51.1% 1831 ± 35% interrupts.CPU144.PMI:Performance_monitoring_interrupts 1266 ± 15% +43.3% 1815 ± 7% interrupts.CPU144.RES:Rescheduling_interrupts 1226 ± 11% +109.7% 2572 ± 27% interrupts.CPU145.NMI:Non-maskable_interrupts 1226 ± 11% +109.7% 2572 ± 27% interrupts.CPU145.PMI:Performance_monitoring_interrupts 1370 ± 27% +54.3% 2114 ± 35% interrupts.CPU15.NMI:Non-maskable_interrupts 1370 ± 27% +54.3% 2114 ± 35% interrupts.CPU15.PMI:Performance_monitoring_interrupts 685.25 ± 6% +33.2% 913.00 ± 9% interrupts.CPU15.RES:Rescheduling_interrupts 1197 ± 11% +84.2% 2206 ± 39% interrupts.CPU150.NMI:Non-maskable_interrupts 1197 ± 11% +84.2% 2206 ± 39% interrupts.CPU150.PMI:Performance_monitoring_interrupts 1097 ± 9% +33.0% 1459 ± 13% interrupts.CPU152.RES:Rescheduling_interrupts 1263 ± 7% +31.8% 1665 ± 8% interrupts.CPU154.RES:Rescheduling_interrupts 1215 ± 12% +49.4% 1815 ± 33% interrupts.CPU155.NMI:Non-maskable_interrupts 1215 ± 12% +49.4% 1815 ± 33% interrupts.CPU155.PMI:Performance_monitoring_interrupts 1222 ± 5% +29.1% 1578 ± 9% interrupts.CPU155.RES:Rescheduling_interrupts 1065 ± 9% +30.1% 1386 ± 13% interrupts.CPU156.RES:Rescheduling_interrupts 1017 ± 13% +23.8% 1259 ± 7% interrupts.CPU157.RES:Rescheduling_interrupts 1055 ± 11% +28.5% 1356 ± 11% interrupts.CPU158.RES:Rescheduling_interrupts 1016 ± 9% +39.7% 1420 ± 13% interrupts.CPU159.RES:Rescheduling_interrupts 1084 ± 2% +24.3% 1347 ± 13% interrupts.CPU161.RES:Rescheduling_interrupts 918.75 ± 13% +20.4% 1106 ± 11% interrupts.CPU162.RES:Rescheduling_interrupts 10725 -8.9% 9774 ± 2% interrupts.CPU162.TLB:TLB_shootdowns 959.75 ± 9% +24.8% 1197 ± 11% interrupts.CPU163.RES:Rescheduling_interrupts 1177 ± 8% +36.8% 1610 ± 15% interrupts.CPU167.RES:Rescheduling_interrupts 1211 ± 12% +84.4% 2233 ± 39% interrupts.CPU168.NMI:Non-maskable_interrupts 1211 ± 12% +84.4% 2233 ± 39% interrupts.CPU168.PMI:Performance_monitoring_interrupts 1220 ± 12% +22.2% 1491 ± 8% interrupts.CPU169.RES:Rescheduling_interrupts 711.00 ± 6% +33.3% 948.00 ± 13% interrupts.CPU17.RES:Rescheduling_interrupts 1254 ± 8% +32.6% 1663 ± 18% interrupts.CPU170.RES:Rescheduling_interrupts 1149 ± 10% +46.0% 1677 ± 11% interrupts.CPU171.RES:Rescheduling_interrupts 1578 ± 49% +41.6% 2234 ± 40% interrupts.CPU173.NMI:Non-maskable_interrupts 1578 ± 49% +41.6% 2234 ± 40% interrupts.CPU173.PMI:Performance_monitoring_interrupts 1242 ± 9% +17.3% 1456 ± 9% interrupts.CPU174.NMI:Non-maskable_interrupts 1242 ± 9% +17.3% 1456 ± 9% interrupts.CPU174.PMI:Performance_monitoring_interrupts 1008 ± 7% +26.7% 1278 interrupts.CPU176.RES:Rescheduling_interrupts 1015 ± 4% +33.9% 1360 ± 8% interrupts.CPU177.RES:Rescheduling_interrupts 929.50 ± 13% +27.4% 1183 ± 6% interrupts.CPU178.RES:Rescheduling_interrupts 1051 ± 8% +20.7% 1268 interrupts.CPU179.RES:Rescheduling_interrupts 1149 ± 13% +87.3% 2153 ± 34% interrupts.CPU18.NMI:Non-maskable_interrupts 1149 ± 13% +87.3% 2153 ± 34% interrupts.CPU18.PMI:Performance_monitoring_interrupts 832.75 ± 4% +23.5% 1028 interrupts.CPU18.RES:Rescheduling_interrupts 1215 ± 11% +21.9% 1481 ± 9% interrupts.CPU180.NMI:Non-maskable_interrupts 1215 ± 11% +21.9% 1481 ± 9% interrupts.CPU180.PMI:Performance_monitoring_interrupts 1050 ± 15% +39.3% 1463 ± 6% interrupts.CPU181.RES:Rescheduling_interrupts 1093 ± 5% +28.7% 1407 ± 11% interrupts.CPU182.RES:Rescheduling_interrupts 1082 ± 5% +27.6% 1381 ± 3% interrupts.CPU183.RES:Rescheduling_interrupts 990.50 ± 10% +28.1% 1269 ± 8% interrupts.CPU184.RES:Rescheduling_interrupts 964.00 ± 13% +28.2% 1235 ± 8% interrupts.CPU186.RES:Rescheduling_interrupts 1498 ± 29% +68.5% 2524 ± 19% interrupts.CPU187.NMI:Non-maskable_interrupts 1498 ± 29% +68.5% 2524 ± 19% interrupts.CPU187.PMI:Performance_monitoring_interrupts 1084 ± 11% +35.9% 1473 ± 9% interrupts.CPU188.RES:Rescheduling_interrupts 1537 ± 34% +68.0% 2582 ± 26% interrupts.CPU190.NMI:Non-maskable_interrupts 1537 ± 34% +68.0% 2582 ± 26% interrupts.CPU190.PMI:Performance_monitoring_interrupts 1247 ± 2% +32.7% 1655 ± 5% interrupts.CPU190.RES:Rescheduling_interrupts 1292 ± 11% +26.0% 1628 ± 6% interrupts.CPU191.RES:Rescheduling_interrupts 1223 ± 13% +50.2% 1837 ± 33% interrupts.CPU192.NMI:Non-maskable_interrupts 1223 ± 13% +50.2% 1837 ± 33% interrupts.CPU192.PMI:Performance_monitoring_interrupts 1263 ± 8% +30.8% 1651 ± 8% interrupts.CPU193.RES:Rescheduling_interrupts 1369 ± 8% +25.9% 1724 ± 13% interrupts.CPU196.RES:Rescheduling_interrupts 1127 ± 9% +27.5% 1437 ± 6% interrupts.CPU197.RES:Rescheduling_interrupts 1134 ± 10% +25.8% 1427 ± 19% interrupts.CPU198.RES:Rescheduling_interrupts 1205 ± 12% +47.0% 1772 ± 24% interrupts.CPU199.NMI:Non-maskable_interrupts 1205 ± 12% +47.0% 1772 ± 24% interrupts.CPU199.PMI:Performance_monitoring_interrupts 1160 ± 12% +117.2% 2520 ± 25% interrupts.CPU2.NMI:Non-maskable_interrupts 1160 ± 12% +117.2% 2520 ± 25% interrupts.CPU2.PMI:Performance_monitoring_interrupts 1408 ± 27% +77.7% 2502 ± 27% interrupts.CPU20.NMI:Non-maskable_interrupts 1408 ± 27% +77.7% 2502 ± 27% interrupts.CPU20.PMI:Performance_monitoring_interrupts 1077 ± 13% +28.2% 1381 ± 9% interrupts.CPU202.RES:Rescheduling_interrupts 1512 ± 36% +41.9% 2147 ± 26% interrupts.CPU203.NMI:Non-maskable_interrupts 1512 ± 36% +41.9% 2147 ± 26% interrupts.CPU203.PMI:Performance_monitoring_interrupts 1052 ± 5% +35.0% 1420 ± 20% interrupts.CPU203.RES:Rescheduling_interrupts 1001 ± 3% +41.0% 1411 ± 19% interrupts.CPU204.RES:Rescheduling_interrupts 1140 ± 4% +31.4% 1498 ± 7% interrupts.CPU206.RES:Rescheduling_interrupts 1288 ± 5% +27.6% 1643 ± 3% interrupts.CPU208.RES:Rescheduling_interrupts 1131 ± 13% +60.9% 1821 ± 44% interrupts.CPU21.NMI:Non-maskable_interrupts 1131 ± 13% +60.9% 1821 ± 44% interrupts.CPU21.PMI:Performance_monitoring_interrupts 1126 ± 2% +29.1% 1454 ± 13% interrupts.CPU210.RES:Rescheduling_interrupts 10556 ± 3% -8.9% 9618 ± 2% interrupts.CPU210.TLB:TLB_shootdowns 1042 ± 12% +42.8% 1489 ± 9% interrupts.CPU211.RES:Rescheduling_interrupts 1156 ± 10% +18.5% 1371 ± 9% interrupts.CPU212.RES:Rescheduling_interrupts 1336 ± 4% +13.8% 1520 ± 5% interrupts.CPU213.RES:Rescheduling_interrupts 1201 ± 5% +23.0% 1477 ± 8% interrupts.CPU214.RES:Rescheduling_interrupts 1622 ± 6% +36.7% 2219 ± 17% interrupts.CPU215.RES:Rescheduling_interrupts 1178 ± 16% +30.5% 1538 ± 6% interrupts.CPU216.RES:Rescheduling_interrupts 1006 ± 13% +48.5% 1494 ± 7% interrupts.CPU217.RES:Rescheduling_interrupts 1603 ± 48% +40.3% 2248 ± 38% interrupts.CPU218.NMI:Non-maskable_interrupts 1603 ± 48% +40.3% 2248 ± 38% interrupts.CPU218.PMI:Performance_monitoring_interrupts 1235 ± 9% +36.4% 1684 ± 6% interrupts.CPU218.RES:Rescheduling_interrupts 1219 ± 13% +29.3% 1576 ± 8% interrupts.CPU219.RES:Rescheduling_interrupts 1153 ± 12% +145.2% 2828 ± 10% interrupts.CPU22.NMI:Non-maskable_interrupts 1153 ± 12% +145.2% 2828 ± 10% interrupts.CPU22.PMI:Performance_monitoring_interrupts 849.50 ± 4% +25.5% 1066 ± 6% interrupts.CPU22.RES:Rescheduling_interrupts 1209 ± 12% +84.1% 2226 ± 40% interrupts.CPU220.NMI:Non-maskable_interrupts 1209 ± 12% +84.1% 2226 ± 40% interrupts.CPU220.PMI:Performance_monitoring_interrupts 1037 ± 10% +14.3% 1186 ± 3% interrupts.CPU220.RES:Rescheduling_interrupts 1092 ± 8% +31.8% 1440 ± 14% interrupts.CPU221.RES:Rescheduling_interrupts 1112 ± 13% +28.8% 1433 ± 9% interrupts.CPU222.RES:Rescheduling_interrupts 1220 ± 9% +32.8% 1620 ± 7% interrupts.CPU223.RES:Rescheduling_interrupts 1515 ± 34% +42.5% 2159 ± 28% interrupts.CPU226.NMI:Non-maskable_interrupts 1515 ± 34% +42.5% 2159 ± 28% interrupts.CPU226.PMI:Performance_monitoring_interrupts 1083 ± 8% +33.0% 1441 ± 13% interrupts.CPU226.RES:Rescheduling_interrupts 956.50 ± 11% +23.7% 1183 ± 8% interrupts.CPU227.RES:Rescheduling_interrupts 1180 ± 3% +38.2% 1631 ± 18% interrupts.CPU228.RES:Rescheduling_interrupts 1223 ± 11% +20.1% 1468 ± 9% interrupts.CPU229.NMI:Non-maskable_interrupts 1223 ± 11% +20.1% 1468 ± 9% interrupts.CPU229.PMI:Performance_monitoring_interrupts 1149 ± 12% +58.3% 1819 ± 44% interrupts.CPU23.NMI:Non-maskable_interrupts 1149 ± 12% +58.3% 1819 ± 44% interrupts.CPU23.PMI:Performance_monitoring_interrupts 726.75 ± 4% +32.6% 963.50 ± 6% interrupts.CPU23.RES:Rescheduling_interrupts 1013 ± 7% +35.2% 1370 ± 14% interrupts.CPU230.RES:Rescheduling_interrupts 1004 ± 5% +29.5% 1300 ± 8% interrupts.CPU231.RES:Rescheduling_interrupts 1213 ± 13% +141.5% 2929 ± 9% interrupts.CPU233.NMI:Non-maskable_interrupts 1213 ± 13% +141.5% 2929 ± 9% interrupts.CPU233.PMI:Performance_monitoring_interrupts 1013 ± 9% +22.5% 1241 ± 8% interrupts.CPU233.RES:Rescheduling_interrupts 1177 ± 13% +80.2% 2121 ± 29% interrupts.CPU234.NMI:Non-maskable_interrupts 1177 ± 13% +80.2% 2121 ± 29% interrupts.CPU234.PMI:Performance_monitoring_interrupts 1515 ± 35% +68.0% 2545 ± 26% interrupts.CPU237.NMI:Non-maskable_interrupts 1515 ± 35% +68.0% 2545 ± 26% interrupts.CPU237.PMI:Performance_monitoring_interrupts 1770 ± 29% +64.4% 2910 ± 10% interrupts.CPU238.NMI:Non-maskable_interrupts 1770 ± 29% +64.4% 2910 ± 10% interrupts.CPU238.PMI:Performance_monitoring_interrupts 1000 ± 10% +23.4% 1233 ± 3% interrupts.CPU238.RES:Rescheduling_interrupts 1178 ± 6% +23.5% 1456 ± 8% interrupts.CPU239.RES:Rescheduling_interrupts 1151 ± 14% +115.7% 2483 ± 27% interrupts.CPU24.NMI:Non-maskable_interrupts 1151 ± 14% +115.7% 2483 ± 27% interrupts.CPU24.PMI:Performance_monitoring_interrupts 823.00 ± 9% +51.7% 1248 ± 19% interrupts.CPU24.RES:Rescheduling_interrupts 1217 ± 15% +22.9% 1496 ± 8% interrupts.CPU240.RES:Rescheduling_interrupts 1014 ± 4% +54.4% 1565 ± 20% interrupts.CPU242.RES:Rescheduling_interrupts 10616 ± 3% -8.3% 9731 interrupts.CPU242.TLB:TLB_shootdowns 1043 ± 11% +36.3% 1422 ± 16% interrupts.CPU243.RES:Rescheduling_interrupts 1215 ± 11% +83.3% 2227 ± 40% interrupts.CPU247.NMI:Non-maskable_interrupts 1215 ± 11% +83.3% 2227 ± 40% interrupts.CPU247.PMI:Performance_monitoring_interrupts 1499 ± 34% +68.8% 2531 ± 20% interrupts.CPU248.NMI:Non-maskable_interrupts 1499 ± 34% +68.8% 2531 ± 20% interrupts.CPU248.PMI:Performance_monitoring_interrupts 1034 ± 5% +28.3% 1327 ± 15% interrupts.CPU248.RES:Rescheduling_interrupts 1103 ± 14% +25.1% 1380 ± 8% interrupts.CPU249.RES:Rescheduling_interrupts 1143 ± 13% +49.8% 1712 ± 24% interrupts.CPU25.NMI:Non-maskable_interrupts 1143 ± 13% +49.8% 1712 ± 24% interrupts.CPU25.PMI:Performance_monitoring_interrupts 749.50 ± 9% +24.9% 936.25 ± 4% interrupts.CPU25.RES:Rescheduling_interrupts 10842 ± 2% -8.8% 9885 ± 2% interrupts.CPU25.TLB:TLB_shootdowns 1099 ± 5% +18.0% 1297 ± 3% interrupts.CPU251.RES:Rescheduling_interrupts 993.25 ± 11% +23.1% 1223 ± 5% interrupts.CPU252.RES:Rescheduling_interrupts 1017 ± 12% +34.7% 1369 ± 11% interrupts.CPU253.RES:Rescheduling_interrupts 1532 ± 49% +44.3% 2211 ± 39% interrupts.CPU254.NMI:Non-maskable_interrupts 1532 ± 49% +44.3% 2211 ± 39% interrupts.CPU254.PMI:Performance_monitoring_interrupts 979.50 ± 10% +29.8% 1271 ± 10% interrupts.CPU254.RES:Rescheduling_interrupts 928.50 ± 8% +43.7% 1334 ± 17% interrupts.CPU256.RES:Rescheduling_interrupts 1049 ± 12% +23.9% 1300 ± 8% interrupts.CPU257.RES:Rescheduling_interrupts 1183 ± 4% +12.5% 1331 ± 10% interrupts.CPU258.RES:Rescheduling_interrupts 796.50 ± 6% +38.3% 1101 ± 23% interrupts.CPU26.RES:Rescheduling_interrupts 1087 ± 8% +28.4% 1396 ± 6% interrupts.CPU260.RES:Rescheduling_interrupts 1081 ± 11% +40.3% 1517 ± 6% interrupts.CPU261.RES:Rescheduling_interrupts 1553 ± 48% +43.8% 2233 ± 39% interrupts.CPU262.NMI:Non-maskable_interrupts 1553 ± 48% +43.8% 2233 ± 39% interrupts.CPU262.PMI:Performance_monitoring_interrupts 1270 ± 13% +29.0% 1638 ± 10% interrupts.CPU262.RES:Rescheduling_interrupts 1170 ± 13% +57.6% 1844 ± 44% interrupts.CPU264.NMI:Non-maskable_interrupts 1170 ± 13% +57.6% 1844 ± 44% interrupts.CPU264.PMI:Performance_monitoring_interrupts 899.00 ± 10% +38.8% 1247 ± 14% interrupts.CPU264.RES:Rescheduling_interrupts 912.25 ± 11% +18.3% 1079 ± 7% interrupts.CPU267.RES:Rescheduling_interrupts 703.25 ± 9% +48.5% 1044 ± 21% interrupts.CPU27.RES:Rescheduling_interrupts 1198 ± 12% +82.4% 2185 ± 40% interrupts.CPU273.NMI:Non-maskable_interrupts 1198 ± 12% +82.4% 2185 ± 40% interrupts.CPU273.PMI:Performance_monitoring_interrupts 847.50 ± 10% +44.9% 1228 ± 18% interrupts.CPU273.RES:Rescheduling_interrupts 900.50 ± 19% +34.2% 1208 ± 17% interrupts.CPU274.RES:Rescheduling_interrupts 1175 ± 13% +82.5% 2145 ± 36% interrupts.CPU275.NMI:Non-maskable_interrupts 1175 ± 13% +82.5% 2145 ± 36% interrupts.CPU275.PMI:Performance_monitoring_interrupts 800.50 ± 8% +35.5% 1084 ± 14% interrupts.CPU275.RES:Rescheduling_interrupts 803.00 ± 7% +55.0% 1244 ± 20% interrupts.CPU276.RES:Rescheduling_interrupts 888.50 ± 11% +40.6% 1249 ± 9% interrupts.CPU277.RES:Rescheduling_interrupts 10916 -10.1% 9813 ± 2% interrupts.CPU277.TLB:TLB_shootdowns 1035 ± 2% +23.6% 1280 ± 5% interrupts.CPU278.RES:Rescheduling_interrupts 926.75 ± 8% +37.3% 1272 ± 16% interrupts.CPU279.RES:Rescheduling_interrupts 753.25 ± 21% +42.8% 1076 ± 8% interrupts.CPU282.RES:Rescheduling_interrupts 1183 ± 12% +55.6% 1841 ± 44% interrupts.CPU283.NMI:Non-maskable_interrupts 1183 ± 12% +55.6% 1841 ± 44% interrupts.CPU283.PMI:Performance_monitoring_interrupts 1187 ± 11% +112.7% 2525 ± 27% interrupts.CPU284.NMI:Non-maskable_interrupts 1187 ± 11% +112.7% 2525 ± 27% interrupts.CPU284.PMI:Performance_monitoring_interrupts 2063 ± 28% +38.7% 2863 ± 9% interrupts.CPU285.NMI:Non-maskable_interrupts 2063 ± 28% +38.7% 2863 ± 9% interrupts.CPU285.PMI:Performance_monitoring_interrupts 854.25 ± 3% +27.0% 1084 ± 5% interrupts.CPU286.RES:Rescheduling_interrupts 729.50 ± 11% +33.3% 972.75 ± 5% interrupts.CPU287.RES:Rescheduling_interrupts 1473 ± 50% +72.6% 2542 ± 30% interrupts.CPU29.NMI:Non-maskable_interrupts 1473 ± 50% +72.6% 2542 ± 30% interrupts.CPU29.PMI:Performance_monitoring_interrupts 1133 ± 13% +92.6% 2183 ± 39% interrupts.CPU3.NMI:Non-maskable_interrupts 1133 ± 13% +92.6% 2183 ± 39% interrupts.CPU3.PMI:Performance_monitoring_interrupts 853.00 ± 5% +19.3% 1017 ± 4% interrupts.CPU3.RES:Rescheduling_interrupts 865.00 ± 5% +27.6% 1103 ± 9% interrupts.CPU30.RES:Rescheduling_interrupts 750.50 ± 10% +20.9% 907.25 ± 12% interrupts.CPU31.RES:Rescheduling_interrupts 1140 ± 13% +93.0% 2201 ± 40% interrupts.CPU32.NMI:Non-maskable_interrupts 1140 ± 13% +93.0% 2201 ± 40% interrupts.CPU32.PMI:Performance_monitoring_interrupts 800.50 ± 10% +17.9% 944.00 ± 4% interrupts.CPU32.RES:Rescheduling_interrupts 836.50 ± 5% +18.9% 994.50 ± 4% interrupts.CPU34.RES:Rescheduling_interrupts 797.00 ± 4% +21.9% 971.75 ± 6% interrupts.CPU35.RES:Rescheduling_interrupts 754.00 ± 4% +21.2% 914.00 ± 5% interrupts.CPU36.RES:Rescheduling_interrupts 616.25 ± 8% +44.8% 892.25 ± 15% interrupts.CPU37.RES:Rescheduling_interrupts 791.25 ± 6% +32.6% 1049 ± 16% interrupts.CPU38.RES:Rescheduling_interrupts 686.75 ± 5% +34.9% 926.50 ± 14% interrupts.CPU39.RES:Rescheduling_interrupts 878.50 ± 6% +31.6% 1156 ± 16% interrupts.CPU4.RES:Rescheduling_interrupts 1180 ± 12% +85.2% 2185 ± 34% interrupts.CPU42.NMI:Non-maskable_interrupts 1180 ± 12% +85.2% 2185 ± 34% interrupts.CPU42.PMI:Performance_monitoring_interrupts 980.75 ± 11% +39.5% 1367 ± 14% interrupts.CPU45.RES:Rescheduling_interrupts 1103 ± 14% +22.1% 1347 ± 6% interrupts.CPU46.RES:Rescheduling_interrupts 925.00 ± 26% +42.2% 1315 ± 10% interrupts.CPU47.RES:Rescheduling_interrupts 960.25 ± 15% +41.2% 1356 ± 12% interrupts.CPU48.RES:Rescheduling_interrupts 921.75 ± 8% +23.2% 1135 ± 11% interrupts.CPU49.RES:Rescheduling_interrupts 1067 ± 15% +25.3% 1337 ± 10% interrupts.CPU50.RES:Rescheduling_interrupts 1076 ± 7% +49.1% 1605 ± 17% interrupts.CPU56.RES:Rescheduling_interrupts 10449 ± 2% -7.7% 9646 ± 3% interrupts.CPU56.TLB:TLB_shootdowns 913.75 ± 13% +40.9% 1287 ± 13% interrupts.CPU58.RES:Rescheduling_interrupts 767.50 ± 8% +54.1% 1182 ± 14% interrupts.CPU59.RES:Rescheduling_interrupts 1374 ± 27% +79.9% 2473 ± 26% interrupts.CPU6.NMI:Non-maskable_interrupts 1374 ± 27% +79.9% 2473 ± 26% interrupts.CPU6.PMI:Performance_monitoring_interrupts 825.00 ± 11% +38.8% 1145 ± 18% interrupts.CPU6.RES:Rescheduling_interrupts 1016 ± 21% +32.2% 1343 ± 12% interrupts.CPU60.RES:Rescheduling_interrupts 802.50 ± 12% +52.6% 1225 ± 16% interrupts.CPU61.RES:Rescheduling_interrupts 895.50 ± 8% +21.4% 1087 ± 2% interrupts.CPU62.RES:Rescheduling_interrupts 973.25 ± 22% +41.4% 1376 ± 6% interrupts.CPU66.RES:Rescheduling_interrupts 854.75 ± 10% +33.8% 1143 ± 13% interrupts.CPU67.RES:Rescheduling_interrupts 771.25 ± 12% +40.2% 1081 ± 13% interrupts.CPU69.RES:Rescheduling_interrupts 621.50 ± 7% +45.4% 903.50 ± 12% interrupts.CPU7.RES:Rescheduling_interrupts 1169 ± 7% +24.5% 1455 ± 5% interrupts.CPU70.RES:Rescheduling_interrupts 10547 ± 2% -8.5% 9654 interrupts.CPU70.TLB:TLB_shootdowns 1574 ± 12% +34.4% 2115 ± 11% interrupts.CPU71.RES:Rescheduling_interrupts 1456 ± 29% +49.9% 2183 ± 35% interrupts.CPU72.NMI:Non-maskable_interrupts 1456 ± 29% +49.9% 2183 ± 35% interrupts.CPU72.PMI:Performance_monitoring_interrupts 1183 ± 12% +25.6% 1485 ± 8% interrupts.CPU72.RES:Rescheduling_interrupts 1514 ± 33% +41.4% 2142 ± 27% interrupts.CPU73.NMI:Non-maskable_interrupts 1514 ± 33% +41.4% 2142 ± 27% interrupts.CPU73.PMI:Performance_monitoring_interrupts 986.75 ± 21% +56.7% 1546 ± 14% interrupts.CPU73.RES:Rescheduling_interrupts 1001 ± 4% +35.5% 1356 ± 7% interrupts.CPU76.RES:Rescheduling_interrupts 1004 ± 10% +22.4% 1229 ± 11% interrupts.CPU77.RES:Rescheduling_interrupts 897.25 ± 9% +34.5% 1206 ± 19% interrupts.CPU78.RES:Rescheduling_interrupts 785.75 ± 11% +25.6% 987.00 ± 4% interrupts.CPU8.RES:Rescheduling_interrupts 943.75 ± 9% +29.1% 1218 ± 12% interrupts.CPU81.RES:Rescheduling_interrupts 991.75 ± 2% +37.9% 1367 ± 17% interrupts.CPU82.RES:Rescheduling_interrupts 1177 ± 12% +58.9% 1870 ± 44% interrupts.CPU83.NMI:Non-maskable_interrupts 1177 ± 12% +58.9% 1870 ± 44% interrupts.CPU83.PMI:Performance_monitoring_interrupts 1170 ± 15% +17.1% 1370 ± 6% interrupts.CPU83.RES:Rescheduling_interrupts 1499 ± 30% +45.9% 2188 ± 34% interrupts.CPU86.NMI:Non-maskable_interrupts 1499 ± 30% +45.9% 2188 ± 34% interrupts.CPU86.PMI:Performance_monitoring_interrupts 1193 ± 10% +109.8% 2502 ± 20% interrupts.CPU87.NMI:Non-maskable_interrupts 1193 ± 10% +109.8% 2502 ± 20% interrupts.CPU87.PMI:Performance_monitoring_interrupts 1145 ± 14% +31.2% 1502 ± 7% interrupts.CPU87.RES:Rescheduling_interrupts 1177 ± 6% +21.1% 1426 ± 8% interrupts.CPU88.RES:Rescheduling_interrupts 1140 ± 14% +91.1% 2179 ± 40% interrupts.CPU9.NMI:Non-maskable_interrupts 1140 ± 14% +91.1% 2179 ± 40% interrupts.CPU9.PMI:Performance_monitoring_interrupts 676.75 ± 10% +41.2% 955.25 ± 11% interrupts.CPU9.RES:Rescheduling_interrupts 1195 ± 11% +56.7% 1872 ± 43% interrupts.CPU90.NMI:Non-maskable_interrupts 1195 ± 11% +56.7% 1872 ± 43% interrupts.CPU90.PMI:Performance_monitoring_interrupts 2083 ± 27% +40.0% 2917 ± 10% interrupts.CPU95.NMI:Non-maskable_interrupts 2083 ± 27% +40.0% 2917 ± 10% interrupts.CPU95.PMI:Performance_monitoring_interrupts 1110 ± 6% +14.4% 1270 ± 11% interrupts.CPU95.RES:Rescheduling_interrupts 1110 ± 6% +41.0% 1566 ± 18% interrupts.CPU98.RES:Rescheduling_interrupts 984.75 ± 9% +58.2% 1557 ± 19% interrupts.CPU99.RES:Rescheduling_interrupts 310787 ± 6% +23.6% 384165 ± 4% interrupts.RES:Rescheduling_interrupts 33134 ± 2% +14.7% 38003 ± 6% softirqs.CPU0.SCHED 27079 ± 2% +17.4% 31779 ± 3% softirqs.CPU1.SCHED 162495 +8.8% 176829 ± 4% softirqs.CPU1.TIMER 26465 +19.2% 31556 ± 3% softirqs.CPU10.SCHED 25261 ± 3% +16.4% 29411 ± 3% softirqs.CPU100.SCHED 25450 +18.5% 30153 ± 4% softirqs.CPU101.SCHED 25169 ± 2% +19.3% 30020 ± 3% softirqs.CPU102.SCHED 24805 ± 3% +20.5% 29893 ± 3% softirqs.CPU103.SCHED 25132 ± 3% +18.9% 29875 ± 3% softirqs.CPU104.SCHED 177301 ± 2% +11.3% 197274 ± 4% softirqs.CPU104.TIMER 24869 ± 3% +18.8% 29537 ± 2% softirqs.CPU105.SCHED 25032 ± 2% +17.7% 29462 ± 3% softirqs.CPU106.SCHED 24317 +22.3% 29729 ± 3% softirqs.CPU107.SCHED 25241 ± 3% +18.5% 29917 ± 2% softirqs.CPU108.SCHED 25571 ± 2% +17.6% 30075 ± 3% softirqs.CPU109.SCHED 25910 +17.9% 30551 ± 3% softirqs.CPU11.SCHED 24927 ± 2% +18.5% 29538 ± 4% softirqs.CPU110.SCHED 25019 ± 2% +19.2% 29833 ± 4% softirqs.CPU111.SCHED 25067 ± 2% +18.7% 29743 ± 4% softirqs.CPU112.SCHED 25105 ± 2% +19.1% 29892 ± 2% softirqs.CPU113.SCHED 25603 ± 2% +16.2% 29753 ± 4% softirqs.CPU114.SCHED 25517 ± 2% +17.9% 30096 ± 3% softirqs.CPU115.SCHED 176346 ± 5% +11.4% 196390 ± 4% softirqs.CPU115.TIMER 24988 ± 2% +17.3% 29299 ± 4% softirqs.CPU116.SCHED 24362 +22.1% 29748 ± 2% softirqs.CPU117.SCHED 25420 ± 2% +18.4% 30101 ± 3% softirqs.CPU118.SCHED 25202 +17.2% 29531 ± 3% softirqs.CPU119.SCHED 246.00 ± 32% +5886.8% 14727 ±166% softirqs.CPU12.NET_RX 25808 ± 2% +19.6% 30866 ± 4% softirqs.CPU12.SCHED 161592 ± 4% +26.1% 203759 ± 17% softirqs.CPU12.TIMER 25550 +16.3% 29721 ± 3% softirqs.CPU120.SCHED 172644 ± 4% +13.1% 195269 ± 4% softirqs.CPU120.TIMER 24062 ± 5% +24.3% 29898 ± 3% softirqs.CPU121.SCHED 173612 ± 4% +12.4% 195124 ± 3% softirqs.CPU121.TIMER 24774 ± 3% +20.1% 29742 ± 2% softirqs.CPU122.SCHED 25233 ± 2% +18.4% 29878 ± 2% softirqs.CPU123.SCHED 25038 ± 2% +18.0% 29548 ± 4% softirqs.CPU124.SCHED 24756 +19.1% 29485 ± 3% softirqs.CPU125.SCHED 25079 +19.2% 29903 ± 3% softirqs.CPU126.SCHED 25110 +17.9% 29598 ± 3% softirqs.CPU127.SCHED 24869 ± 3% +19.6% 29754 ± 3% softirqs.CPU128.SCHED 25255 ± 2% +17.6% 29710 ± 3% softirqs.CPU129.SCHED 26257 +22.8% 32254 ± 3% softirqs.CPU13.SCHED 25274 ± 2% +19.3% 30153 ± 2% softirqs.CPU130.SCHED 25219 ± 2% +15.4% 29103 ± 2% softirqs.CPU131.SCHED 25034 ± 2% +19.3% 29869 ± 2% softirqs.CPU132.SCHED 25448 +17.1% 29792 softirqs.CPU133.SCHED 25064 +17.5% 29450 ± 4% softirqs.CPU134.SCHED 25561 ± 2% +16.4% 29742 ± 4% softirqs.CPU135.SCHED 25620 +17.3% 30041 ± 3% softirqs.CPU136.SCHED 174761 ± 6% +11.1% 194097 ± 7% softirqs.CPU136.TIMER 26235 ± 2% +14.8% 30124 ± 2% softirqs.CPU137.SCHED 169540 ± 4% +13.3% 192089 ± 3% softirqs.CPU137.TIMER 25194 +18.3% 29808 ± 3% softirqs.CPU138.SCHED 25038 ± 4% +19.7% 29982 ± 4% softirqs.CPU139.SCHED 26136 ± 2% +20.5% 31494 ± 2% softirqs.CPU14.SCHED 25185 +20.2% 30269 ± 3% softirqs.CPU140.SCHED 25032 +20.2% 30098 ± 2% softirqs.CPU141.SCHED 25247 ± 2% +16.9% 29514 softirqs.CPU142.SCHED 24775 +12.9% 27962 ± 6% softirqs.CPU143.SCHED 25196 +19.1% 30016 ± 3% softirqs.CPU144.SCHED 24907 ± 2% +18.4% 29495 ± 3% softirqs.CPU145.SCHED 25254 +16.3% 29359 ± 4% softirqs.CPU146.SCHED 25123 +19.8% 30089 ± 3% softirqs.CPU147.SCHED 25311 ± 6% +17.6% 29757 ± 5% softirqs.CPU148.SCHED 25020 ± 2% +20.4% 30121 ± 2% softirqs.CPU149.SCHED 26013 +19.5% 31094 ± 2% softirqs.CPU15.SCHED 25040 +19.1% 29828 ± 4% softirqs.CPU150.SCHED 25358 +20.1% 30458 ± 2% softirqs.CPU151.SCHED 175966 ± 6% +11.6% 196369 ± 4% softirqs.CPU151.TIMER 24847 ± 2% +20.7% 30003 ± 2% softirqs.CPU152.SCHED 24860 ± 4% +19.9% 29811 ± 3% softirqs.CPU153.SCHED 25102 ± 3% +18.0% 29611 ± 3% softirqs.CPU154.SCHED 24819 +20.0% 29792 ± 3% softirqs.CPU155.SCHED 25414 ± 2% +17.3% 29808 softirqs.CPU156.SCHED 176417 ± 5% +15.6% 203967 ± 5% softirqs.CPU156.TIMER 25556 +16.1% 29663 ± 4% softirqs.CPU157.SCHED 25241 ± 2% +19.9% 30256 ± 3% softirqs.CPU158.SCHED 25462 +17.9% 30017 ± 2% softirqs.CPU159.SCHED 25909 +18.5% 30694 ± 3% softirqs.CPU16.SCHED 25139 +18.1% 29684 ± 4% softirqs.CPU160.SCHED 25431 +17.2% 29812 ± 4% softirqs.CPU161.SCHED 25712 +18.6% 30483 ± 5% softirqs.CPU162.SCHED 172304 ± 4% +13.1% 194925 ± 4% softirqs.CPU162.TIMER 25736 +16.7% 30040 ± 3% softirqs.CPU163.SCHED 169697 ± 6% +16.0% 196768 ± 7% softirqs.CPU163.TIMER 25196 ± 2% +17.4% 29591 ± 2% softirqs.CPU164.SCHED 24976 +19.1% 29753 ± 2% softirqs.CPU165.SCHED 25153 ± 2% +18.0% 29682 ± 2% softirqs.CPU166.SCHED 24977 +19.7% 29886 ± 2% softirqs.CPU167.SCHED 24784 ± 3% +18.9% 29457 ± 3% softirqs.CPU168.SCHED 25266 ± 2% +17.1% 29596 ± 4% softirqs.CPU169.SCHED 25886 ± 2% +19.3% 30876 ± 3% softirqs.CPU17.SCHED 24920 +17.6% 29295 ± 3% softirqs.CPU170.SCHED 24993 +18.7% 29660 ± 3% softirqs.CPU171.SCHED 25422 +17.1% 29763 ± 2% softirqs.CPU172.SCHED 25491 ± 2% +17.7% 29994 ± 3% softirqs.CPU173.SCHED 25264 ± 2% +16.8% 29518 ± 6% softirqs.CPU174.SCHED 25266 ± 2% +17.9% 29791 ± 2% softirqs.CPU175.SCHED 25231 ± 2% +17.9% 29754 ± 3% softirqs.CPU176.SCHED 183212 ± 2% +10.0% 201572 ± 2% softirqs.CPU176.TIMER 25273 ± 2% +18.3% 29899 ± 3% softirqs.CPU177.SCHED 25155 +19.8% 30145 ± 2% softirqs.CPU178.SCHED 25499 ± 2% +16.0% 29576 ± 5% softirqs.CPU179.SCHED 26339 +16.3% 30634 ± 2% softirqs.CPU18.SCHED 24757 ± 4% +19.2% 29500 ± 2% softirqs.CPU180.SCHED 25192 ± 3% +14.7% 28898 ± 5% softirqs.CPU181.SCHED 25166 ± 2% +19.2% 29999 ± 2% softirqs.CPU182.SCHED 25259 ± 2% +18.4% 29910 ± 4% softirqs.CPU183.SCHED 25247 ± 2% +16.9% 29519 ± 4% softirqs.CPU184.SCHED 25415 ± 2% +16.9% 29715 ± 2% softirqs.CPU185.SCHED 180421 ± 5% +11.4% 200946 ± 3% softirqs.CPU185.TIMER 24973 ± 3% +16.3% 29053 softirqs.CPU186.SCHED 25525 ± 2% +16.1% 29625 ± 2% softirqs.CPU187.SCHED 25461 +18.2% 30088 ± 2% softirqs.CPU188.SCHED 24833 +20.5% 29936 ± 2% softirqs.CPU189.SCHED 25894 +17.1% 30332 ± 4% softirqs.CPU19.SCHED 162878 ± 4% +18.0% 192168 ± 14% softirqs.CPU19.TIMER 25062 +18.4% 29663 ± 2% softirqs.CPU190.SCHED 24867 +18.3% 29425 ± 3% softirqs.CPU191.SCHED 24926 +18.8% 29620 ± 2% softirqs.CPU192.SCHED 183126 ± 5% +11.9% 204971 ± 2% softirqs.CPU192.TIMER 24864 ± 2% +16.2% 28893 ± 4% softirqs.CPU193.SCHED 25242 +17.7% 29702 ± 2% softirqs.CPU194.SCHED 25230 ± 2% +17.4% 29626 ± 3% softirqs.CPU195.SCHED 24981 ± 2% +18.8% 29680 ± 2% softirqs.CPU196.SCHED 25020 +17.3% 29348 ± 6% softirqs.CPU197.SCHED 25272 +18.5% 29942 ± 3% softirqs.CPU198.SCHED 25221 ± 2% +18.8% 29968 ± 3% softirqs.CPU199.SCHED 182100 ± 4% +8.3% 197211 ± 3% softirqs.CPU199.TIMER 26451 +18.1% 31251 ± 3% softirqs.CPU2.SCHED 26194 +18.4% 31019 ± 2% softirqs.CPU20.SCHED 25645 ± 2% +17.6% 30168 ± 3% softirqs.CPU200.SCHED 25337 +20.0% 30393 ± 3% softirqs.CPU201.SCHED 25160 +19.0% 29945 ± 2% softirqs.CPU202.SCHED 25322 ± 2% +18.3% 29944 ± 3% softirqs.CPU203.SCHED 25076 +20.2% 30130 ± 3% softirqs.CPU204.SCHED 25542 +19.2% 30455 ± 2% softirqs.CPU205.SCHED 25245 ± 2% +16.7% 29455 ± 2% softirqs.CPU206.SCHED 25317 ± 2% +17.5% 29743 ± 3% softirqs.CPU207.SCHED 25283 ± 2% +18.5% 29958 ± 4% softirqs.CPU208.SCHED 25533 +12.9% 28839 ± 6% softirqs.CPU209.SCHED 176766 ± 3% +13.1% 199978 ± 2% softirqs.CPU209.TIMER 26093 +18.6% 30954 ± 3% softirqs.CPU21.SCHED 25426 ± 2% +17.8% 29963 ± 3% softirqs.CPU210.SCHED 25363 ± 2% +17.5% 29794 softirqs.CPU211.SCHED 25056 ± 2% +18.3% 29644 ± 5% softirqs.CPU212.SCHED 24915 ± 2% +18.8% 29602 ± 3% softirqs.CPU213.SCHED 25269 ± 2% +19.9% 30296 ± 2% softirqs.CPU214.SCHED 24759 ± 2% +16.4% 28808 ± 5% softirqs.CPU215.SCHED 25090 +19.8% 30061 ± 2% softirqs.CPU216.SCHED 24887 ± 2% +17.2% 29156 softirqs.CPU217.SCHED 25021 +17.2% 29336 ± 4% softirqs.CPU218.SCHED 25320 ± 2% +17.7% 29806 ± 2% softirqs.CPU219.SCHED 25672 ± 2% +20.9% 31048 softirqs.CPU22.SCHED 25182 ± 2% +18.8% 29919 ± 3% softirqs.CPU220.SCHED 25152 ± 2% +20.3% 30248 ± 4% softirqs.CPU221.SCHED 24777 +19.1% 29504 ± 5% softirqs.CPU222.SCHED 25122 ± 3% +19.6% 30041 ± 2% softirqs.CPU223.SCHED 25079 ± 2% +19.4% 29954 ± 2% softirqs.CPU224.SCHED 25236 ± 2% +18.8% 29969 ± 2% softirqs.CPU225.SCHED 25105 ± 3% +19.0% 29876 ± 2% softirqs.CPU226.SCHED 25609 +15.6% 29613 ± 4% softirqs.CPU227.SCHED 180136 ± 8% +11.0% 200040 ± 5% softirqs.CPU227.TIMER 25435 +15.9% 29476 ± 2% softirqs.CPU228.SCHED 175813 ± 3% +16.7% 205108 ± 5% softirqs.CPU228.TIMER 25142 ± 2% +14.1% 28699 ± 5% softirqs.CPU229.SCHED 25831 +19.2% 30787 ± 2% softirqs.CPU23.SCHED 25460 ± 2% +18.8% 30242 ± 3% softirqs.CPU230.SCHED 25200 ± 3% +20.1% 30259 ± 2% softirqs.CPU231.SCHED 168374 ± 3% +8.7% 183098 ± 3% softirqs.CPU231.TIMER 25379 ± 2% +17.1% 29711 ± 2% softirqs.CPU232.SCHED 25524 ± 2% +18.5% 30246 ± 3% softirqs.CPU233.SCHED 25686 ± 2% +17.5% 30170 ± 3% softirqs.CPU234.SCHED 170330 ± 4% +11.3% 189542 ± 5% softirqs.CPU234.TIMER 25635 ± 2% +14.7% 29410 ± 4% softirqs.CPU235.SCHED 176292 ± 7% +15.9% 204389 ± 6% softirqs.CPU235.TIMER 25549 ± 2% +16.3% 29718 ± 2% softirqs.CPU236.SCHED 178538 ± 4% +9.2% 194880 ± 4% softirqs.CPU236.TIMER 25386 ± 2% +18.4% 30055 ± 2% softirqs.CPU237.SCHED 25191 ± 2% +17.9% 29707 softirqs.CPU238.SCHED 25186 ± 3% +20.3% 30293 softirqs.CPU239.SCHED 25877 +19.0% 30799 ± 3% softirqs.CPU24.SCHED 25506 +17.5% 29967 ± 3% softirqs.CPU240.SCHED 24964 ± 2% +21.1% 30222 ± 3% softirqs.CPU241.SCHED 25258 +17.7% 29719 ± 2% softirqs.CPU242.SCHED 25162 +18.2% 29740 ± 4% softirqs.CPU243.SCHED 25808 +17.3% 30285 ± 3% softirqs.CPU244.SCHED 25679 +17.4% 30150 ± 3% softirqs.CPU245.SCHED 175664 ± 3% +8.3% 190327 ± 4% softirqs.CPU245.TIMER 25134 ± 2% +19.7% 30079 ± 4% softirqs.CPU246.SCHED 24942 +19.9% 29913 ± 3% softirqs.CPU247.SCHED 25395 ± 2% +17.8% 29906 ± 2% softirqs.CPU248.SCHED 180993 ± 3% +10.8% 200584 ± 4% softirqs.CPU248.TIMER 25491 ± 2% +18.0% 30075 ± 4% softirqs.CPU249.SCHED 26099 +17.2% 30595 ± 3% softirqs.CPU25.SCHED 25332 +18.7% 30060 ± 2% softirqs.CPU250.SCHED 25471 ± 2% +17.4% 29892 ± 3% softirqs.CPU251.SCHED 180242 ± 4% +9.8% 197819 ± 4% softirqs.CPU251.TIMER 25406 ± 2% +16.9% 29694 ± 3% softirqs.CPU252.SCHED 25317 ± 3% +18.2% 29922 ± 3% softirqs.CPU253.SCHED 25103 ± 4% +19.9% 30106 ± 3% softirqs.CPU254.SCHED 25231 ± 2% +19.8% 30227 ± 3% softirqs.CPU255.SCHED 25509 ± 2% +18.2% 30161 ± 3% softirqs.CPU256.SCHED 25559 ± 3% +16.9% 29888 ± 3% softirqs.CPU257.SCHED 25277 ± 2% +16.8% 29523 ± 2% softirqs.CPU258.SCHED 24695 ± 5% +20.4% 29721 ± 2% softirqs.CPU259.SCHED 182550 ± 5% +10.5% 201781 ± 3% softirqs.CPU259.TIMER 25905 +19.5% 30970 ± 2% softirqs.CPU26.SCHED 164306 ± 3% +7.6% 176738 ± 5% softirqs.CPU26.TIMER 24829 ± 2% +20.8% 29990 ± 2% softirqs.CPU260.SCHED 25156 +19.1% 29971 ± 2% softirqs.CPU261.SCHED 25098 ± 3% +17.6% 29515 ± 4% softirqs.CPU262.SCHED 180807 ± 5% +9.4% 197856 ± 3% softirqs.CPU262.TIMER 25053 ± 2% +18.2% 29613 ± 2% softirqs.CPU263.SCHED 25505 +18.9% 30336 ± 3% softirqs.CPU264.SCHED 169958 ± 5% +11.8% 189979 ± 4% softirqs.CPU264.TIMER 25011 ± 3% +18.5% 29640 ± 4% softirqs.CPU265.SCHED 169719 ± 4% +22.2% 207453 ± 8% softirqs.CPU265.TIMER 25489 ± 2% +18.6% 30231 ± 3% softirqs.CPU266.SCHED 25727 +18.8% 30554 ± 2% softirqs.CPU267.SCHED 25423 +18.1% 30020 ± 2% softirqs.CPU268.SCHED 25451 ± 2% +19.0% 30292 ± 4% softirqs.CPU269.SCHED 26042 +18.5% 30852 ± 3% softirqs.CPU27.SCHED 163812 ± 3% +8.6% 177825 ± 5% softirqs.CPU27.TIMER 25649 ± 2% +18.3% 30337 ± 3% softirqs.CPU270.SCHED 25318 +19.3% 30212 ± 3% softirqs.CPU271.SCHED 25592 ± 2% +19.6% 30603 ± 3% softirqs.CPU272.SCHED 25440 ± 2% +20.0% 30520 ± 3% softirqs.CPU273.SCHED 26007 +18.2% 30741 ± 2% softirqs.CPU274.SCHED 25661 +18.7% 30454 ± 3% softirqs.CPU275.SCHED 25227 +21.0% 30536 ± 2% softirqs.CPU276.SCHED 25923 +17.9% 30560 ± 3% softirqs.CPU277.SCHED 25973 +14.6% 29757 ± 5% softirqs.CPU278.SCHED 25905 ± 2% +16.2% 30089 ± 3% softirqs.CPU279.SCHED 26091 +18.3% 30875 ± 3% softirqs.CPU28.SCHED 25736 ± 2% +17.4% 30217 ± 4% softirqs.CPU280.SCHED 25964 +13.5% 29482 ± 6% softirqs.CPU281.SCHED 170413 ± 4% +11.8% 190550 ± 4% softirqs.CPU281.TIMER 25669 +18.8% 30484 ± 3% softirqs.CPU282.SCHED 25462 ± 2% +18.6% 30208 softirqs.CPU283.SCHED 174059 ± 6% +10.6% 192446 ± 5% softirqs.CPU283.TIMER 25578 +18.4% 30287 ± 3% softirqs.CPU284.SCHED 25703 +19.1% 30617 ± 3% softirqs.CPU285.SCHED 25775 ± 2% +16.0% 29891 ± 2% softirqs.CPU286.SCHED 25175 ± 2% +16.0% 29204 ± 4% softirqs.CPU287.SCHED 26076 +17.7% 30684 ± 3% softirqs.CPU29.SCHED 26116 +18.8% 31032 ± 2% softirqs.CPU3.SCHED 26043 +18.2% 30789 ± 3% softirqs.CPU30.SCHED 25884 +18.5% 30681 ± 2% softirqs.CPU31.SCHED 167289 ± 2% +6.3% 177832 ± 6% softirqs.CPU31.TIMER 26095 +17.1% 30563 ± 3% softirqs.CPU32.SCHED 161186 ± 2% +10.6% 178192 ± 4% softirqs.CPU32.TIMER 26124 +18.2% 30887 ± 3% softirqs.CPU33.SCHED 25966 +18.0% 30651 ± 2% softirqs.CPU34.SCHED 167402 ± 4% +6.6% 178425 ± 4% softirqs.CPU34.TIMER 25944 +18.0% 30622 ± 3% softirqs.CPU35.SCHED 26167 +17.9% 30861 ± 2% softirqs.CPU36.SCHED 25900 +20.7% 31271 ± 3% softirqs.CPU37.SCHED 25475 ± 3% +21.1% 30843 ± 3% softirqs.CPU38.SCHED 25642 ± 2% +19.4% 30627 ± 3% softirqs.CPU39.SCHED 25494 ± 2% +20.7% 30774 ± 2% softirqs.CPU4.SCHED 25610 +18.1% 30236 ± 4% softirqs.CPU40.SCHED 26064 ± 3% +17.2% 30546 ± 3% softirqs.CPU41.SCHED 25739 ± 2% +15.2% 29643 ± 4% softirqs.CPU42.SCHED 25730 ± 2% +18.5% 30493 ± 3% softirqs.CPU43.SCHED 175037 ± 6% +10.6% 193664 ± 4% softirqs.CPU43.TIMER 25729 +18.4% 30470 ± 3% softirqs.CPU44.SCHED 25114 ± 2% +21.3% 30475 ± 2% softirqs.CPU45.SCHED 25415 ± 3% +20.1% 30511 ± 2% softirqs.CPU46.SCHED 25558 +17.4% 29994 softirqs.CPU47.SCHED 25624 +16.7% 29897 ± 4% softirqs.CPU48.SCHED 167429 ± 5% +13.0% 189268 ± 5% softirqs.CPU48.TIMER 25871 +17.3% 30344 ± 3% softirqs.CPU49.SCHED 167694 ± 3% +12.5% 188683 ± 4% softirqs.CPU49.TIMER 25681 +20.0% 30819 ± 3% softirqs.CPU5.SCHED 25661 +17.8% 30220 softirqs.CPU50.SCHED 25857 +17.6% 30408 ± 3% softirqs.CPU51.SCHED 25943 +17.7% 30544 ± 3% softirqs.CPU52.SCHED 25398 ± 2% +19.6% 30387 ± 3% softirqs.CPU53.SCHED 25599 ± 2% +18.9% 30428 ± 3% softirqs.CPU54.SCHED 25454 ± 2% +20.5% 30663 ± 3% softirqs.CPU55.SCHED 25603 ± 2% +18.6% 30374 ± 3% softirqs.CPU56.SCHED 25665 +18.7% 30453 ± 3% softirqs.CPU57.SCHED 25853 +18.9% 30740 ± 2% softirqs.CPU58.SCHED 25916 +17.5% 30454 ± 3% softirqs.CPU59.SCHED 26202 +16.7% 30570 ± 3% softirqs.CPU6.SCHED 25549 ± 2% +20.5% 30778 ± 2% softirqs.CPU60.SCHED 25854 +18.5% 30626 ± 2% softirqs.CPU61.SCHED 25516 ± 2% +17.1% 29871 ± 4% softirqs.CPU62.SCHED 25833 +17.2% 30284 ± 3% softirqs.CPU63.SCHED 25510 ± 3% +19.2% 30420 ± 3% softirqs.CPU64.SCHED 26177 +16.5% 30487 ± 3% softirqs.CPU65.SCHED 166747 ± 3% +11.5% 185900 ± 3% softirqs.CPU65.TIMER 25637 ± 2% +19.0% 30503 ± 3% softirqs.CPU66.SCHED 25644 ± 2% +15.4% 29588 softirqs.CPU67.SCHED 25796 +18.6% 30585 ± 3% softirqs.CPU68.SCHED 25503 +20.2% 30654 ± 3% softirqs.CPU69.SCHED 25912 +16.4% 30171 ± 4% softirqs.CPU7.SCHED 25994 ± 6% +16.2% 30203 softirqs.CPU70.SCHED 25255 +17.9% 29766 ± 3% softirqs.CPU71.SCHED 25057 +19.1% 29854 ± 3% softirqs.CPU72.SCHED 24925 ± 2% +18.7% 29588 ± 2% softirqs.CPU73.SCHED 25774 +16.1% 29929 ± 3% softirqs.CPU74.SCHED 177035 ± 2% +7.9% 190942 ± 4% softirqs.CPU74.TIMER 25433 +19.2% 30327 ± 3% softirqs.CPU75.SCHED 25274 ± 2% +15.5% 29184 ± 3% softirqs.CPU76.SCHED 25297 +17.7% 29765 ± 4% softirqs.CPU77.SCHED 25557 ± 2% +18.2% 30203 ± 3% softirqs.CPU78.SCHED 25604 ± 2% +17.3% 30034 ± 3% softirqs.CPU79.SCHED 175758 ± 6% +9.8% 193033 ± 3% softirqs.CPU79.TIMER 26086 +18.4% 30878 ± 2% softirqs.CPU8.SCHED 25297 +18.6% 30013 ± 3% softirqs.CPU80.SCHED 25450 +17.8% 29987 ± 2% softirqs.CPU81.SCHED 25125 ± 3% +17.0% 29404 ± 3% softirqs.CPU82.SCHED 24840 ± 2% +18.7% 29490 ± 6% softirqs.CPU83.SCHED 25920 +14.9% 29770 ± 2% softirqs.CPU84.SCHED 168708 ± 5% +18.4% 199703 ± 7% softirqs.CPU84.TIMER 24980 ± 3% +19.5% 29850 ± 3% softirqs.CPU85.SCHED 25338 +18.6% 30042 ± 2% softirqs.CPU86.SCHED 25404 ± 2% +18.6% 30142 ± 2% softirqs.CPU87.SCHED 177099 ± 3% +9.0% 193072 softirqs.CPU87.TIMER 24855 ± 2% +20.7% 30000 ± 3% softirqs.CPU88.SCHED 25532 ± 2% +17.2% 29916 ± 3% softirqs.CPU89.SCHED 26321 ± 2% +14.7% 30186 ± 4% softirqs.CPU9.SCHED 25622 ± 2% +17.4% 30077 ± 3% softirqs.CPU90.SCHED 177491 ± 4% +10.9% 196822 ± 4% softirqs.CPU90.TIMER 25638 ± 2% +14.0% 29228 ± 5% softirqs.CPU91.SCHED 175040 ± 5% +16.7% 204221 ± 6% softirqs.CPU91.TIMER 25387 ± 2% +18.0% 29968 ± 2% softirqs.CPU92.SCHED 25316 ± 2% +19.5% 30262 ± 2% softirqs.CPU93.SCHED 25125 ± 2% +19.6% 30051 softirqs.CPU94.SCHED 25303 ± 2% +18.1% 29875 ± 2% softirqs.CPU95.SCHED 25481 +18.3% 30133 ± 3% softirqs.CPU96.SCHED 25533 +18.1% 30146 ± 3% softirqs.CPU97.SCHED 24892 ± 3% +20.4% 29982 ± 3% softirqs.CPU98.SCHED 24611 +20.3% 29607 ± 3% softirqs.CPU99.SCHED 7325587 +18.2% 8660780 ± 3% softirqs.SCHED will-it-scale.per_thread_ops 1500 +--------------------------------------------------------------------+ 1450 |+. +. .+ +++. +. .+++. +. .++. + | | + ++ + ++ + ++ + + .++.+ | 1400 |-+ : + + | 1350 |-+ : : OO | 1300 |-+ + O | 1250 |-+ OO O OO OO O| | O OO O O OO OO O | 1200 |-+ O O OO OO OO O | 1150 |-+ O | 1100 |-+ O O O O | 1050 |O+O O O O O | | O O O O | 1000 |-+ O O | 950 +--------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

1 year, 1 month

3
3
0 / 0

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP July 2020