February 2021 - LKP - Ml01.01.Org

[proc/wchan] 30a3a19273: leaking-addresses.proc.wchan./proc/bus/input/devices:B:KEY=1000000000007ff980000000007fffebeffdfffeffffffffffffffffffffe

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 30a3a192730a997bc4afff5765254175b6fb64f3 ("[PATCH] proc/wchan: Use printk format instead of lookup_symbol_name()") url: https://github.com/0day-ci/linux/commits/Helge-Deller/proc-wchan-Use-prin... base: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git 09162bc32c880a791c6c0668ce0745cf7958f576 in testcase: leaking-addresses version: leaking-addresses-x86_64-4f19048-1_20201111 with following parameters: ucode: 0xde on test machine: 4 threads Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz with 32G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> 2021-01-01 01:52:25 ./leaking_addresses.pl --output-raw result/scan.out 2021-01-01 01:52:49 ./leaking_addresses.pl --input-raw result/scan.out --squash-by-filename Total number of results from scan (incl dmesg): 156538 dmesg output: [ 0.058490] mapped IOAPIC to ffffffffff5fb000 (fec00000) Results squashed by filename (excl dmesg). Displaying [<number of results> <filename>], <example result> [1 _error_injection_whitelist] 0xffffffffc0a254b0 [25 __bug_table] 0xffffffffc01e0070 [46 .orc_unwind_ip] 0xffffffffc009f3a0 [6 __tracepoints_strings] 0xffffffffc027d7d0 [50 .strtab] 0xffffffffc00b9b88 [1 .rodata.cst16.mask2] 0xffffffffc00a70e0 [1 key] 1000000000007 ff980000000007ff febeffdfffefffff fffffffffffffffe [50 .note.Linux] 0xffffffffc009f024 [41 .data] 0xffffffffc00a1000 [6 .static_call.text] 0xffffffffc0274b44 [1 _ftrace_eval_map] 0xffffffffc0a20148 [10 .data.once] 0xffffffffc04475b4 [7 .static_call_sites] 0xffffffffc0a22088 [6 __tracepoints_ptrs] 0xffffffffc027d7bc [7 .fixup] 0xffffffffc00852ea [49 __mcount_loc] 0xffffffffc009f03c [19 __param] 0xffffffffc009f378 [38 .rodata.str1.8] 0xffffffffc009f170 [1 ___srcu_struct_ptrs] 0xffffffffc0355000 [14 .altinstr_replacement] 0xffffffffc04349ca [154936 kallsyms] ffffffff81000000 T startup_64 [50 .gnu.linkonce.this_module] 0xffffffffc00a1140 [24 __ksymtab_strings] 0xffffffffc00e2048 [31 .bss] 0xffffffffc00a1500 [42 .rodata.str1.1] 0xffffffffc009f09c [9 .init.rodata] 0xffffffffc00b8000 [11 __ex_table] 0xffffffffc00bd128 [14 .parainstructions] 0xffffffffc03b5d88 [6 __tracepoints] 0xffffffffc02818c0 [1 .rodata.cst16.mask1] 0xffffffffc00a70d0 [18 __dyndbg] 0xffffffffc00a10c8 [5 .altinstr_aux] 0xffffffffc0143a49 [22 .smp_locks] 0xffffffffc009f094 [2 .rodata.cst16.bswap_mask] 0xffffffffc005e070 [40 .init.text] 0xffffffffc00b7000 [4 .init.data] 0xffffffffc00e7000 [10 .data..read_mostly] 0xffffffffc00a1100 [14 .altinstructions] 0xffffffffc0446846 [6 __bpf_raw_tp_map] 0xffffffffc0281720 [50 .note.gnu.build-id] 0xffffffffc009f000 [6 _ftrace_events] 0xffffffffc0281780 [140 printk_formats] 0xffffffff82341767 : "CPU_ON" [25 __jump_table] 0xffffffffc00a0000 [37 .exit.text] 0xffffffffc009ec70 [50 .text] 0xffffffffc009e000 [35 .text.unlikely] 0xffffffffc009ebaf [18 __ksymtab] 0xffffffffc00e203c [46 .orc_unwind] 0xffffffffc009f544 [1 .data..cacheline_aligned] 0xffffffffc081d8c0 [2 .noinstr.text] 0xffffffffc04b8d00 [1 uevent] KEY=1000000000007 ff980000000007ff febeffdfffefffff fffffffffffffffe [50 modules] netconsole 20480 0 - Live 0xffffffffc00cb000 [337 blacklist] 0xffffffff81c00880-0xffffffff81c008a0 asm_exc_overflow [1 .rodata.cst32.byteshift_table] 0xffffffffc00a7100 [2 wchan] 0xffffc9000000003c/proc/bus/input/devices: B: KEY=1000000000007 ff980000000007ff febeffdfffefffff fffffffffffffffe [6 .ref.data] 0xffffffffc02817a0 [14 __ksymtab_gpl] 0xffffffffc03b503c [42 .rodata] 0xffffffffc009f2c0 [50 .symtab] 0xffffffffc00b9000 To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml Thanks, Oliver Sang

1 year

4
3
0 / 0

[ext4] 05c2c00f37: aim7.jobs-per-min -11.8% regression

by kernel test robot

Greeting, FYI, we noticed a -11.8% regression of aim7.jobs-per-min due to commit: commit: 05c2c00f3769abb9e323fcaca70d2de0b48af7ba ("ext4: protect superblock modifications with a buffer lock") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: aim7 on test machine: 88 threads Intel(R) Xeon(R) Gold 6238M CPU @ 2.10GHz with 128G memory with following parameters: disk: 4BRD_12G md: RAID1 fs: ext4 test: creat-clo load: 1000 cpufreq_governor: performance ucode: 0x5003006 test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system. test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/ In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------+ | testcase: change | aim9: aim9.udp_test.ops_per_sec 2.1% improvement | | test machine | 256 threads Intel(R) Genuine Intel(R) CPU 0000 @ 1.30GHz with 112G memory | | test parameters | cpufreq_governor=performance | | | test=all | | | testtime=5s | | | ucode=0xffff0190 | +------------------+---------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml bin/lkp run compatible-job.yaml ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/4BRD_12G/ext4/x86_64-rhel-8.3/1000/RAID1/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp9/creat-clo/aim7/0x5003006 commit: 4392fbc4ba ("ext4: drop sync argument of ext4_commit_super()") 05c2c00f37 ("ext4: protect superblock modifications with a buffer lock") 4392fbc4bab57db3 05c2c00f3769abb9e323fcaca70 ---------------- --------------------------- %stddev %change %stddev \ | \ 13342 -11.8% 11771 ± 2% aim7.jobs-per-min 449.81 +13.4% 510.10 ± 2% aim7.time.elapsed_time 449.81 +13.4% 510.10 ± 2% aim7.time.elapsed_time.max 1246834 ± 2% +13.8% 1418574 ± 3% aim7.time.involuntary_context_switches 204562 ± 4% +12.2% 229478 ± 3% aim7.time.minor_page_faults 35777 +12.9% 40390 ± 2% aim7.time.system_time 6270530 ± 3% +34.5% 8435526 ± 2% aim7.time.voluntary_context_switches 8686688 ± 12% +24.1% 10784294 ± 4% cpuidle.C1E.usage 8.74 +4.5% 9.14 iostat.cpu.idle 0.02 ± 12% -0.0 0.02 ± 9% mpstat.cpu.all.iowait% 6988 ± 11% +43.1% 9999 ± 9% numa-meminfo.node0.Active(file) 1440589 ± 7% +7.8% 1552890 ± 6% numa-numastat.node1.numa_hit 1747 ± 11% +43.0% 2499 ± 9% numa-vmstat.node0.nr_active_file 1747 ± 11% +43.0% 2499 ± 9% numa-vmstat.node0.nr_zone_active_file 481.09 +12.6% 541.71 ± 2% uptime.boot 5541 +10.8% 6140 uptime.idle 42743 -10.6% 38207 slabinfo.filp.active_objs 1430 -10.0% 1287 slabinfo.filp.active_slabs 45785 -10.0% 41214 slabinfo.filp.num_objs 1430 -10.0% 1287 slabinfo.filp.num_slabs 91.00 -1.1% 90.00 vmstat.cpu.sy 2850 ± 2% +14.5% 3263 vmstat.io.bo 13739 ± 3% +29.2% 17754 ± 2% vmstat.memory.buff 25822 ± 2% +14.8% 29645 vmstat.system.cs 452981 +11.7% 505788 ± 3% meminfo.Active 439264 +11.1% 488079 ± 3% meminfo.Active(anon) 13715 ± 3% +29.1% 17708 ± 2% meminfo.Active(file) 13713 ± 3% +29.2% 17723 ± 2% meminfo.Buffers 459294 +10.6% 507955 ± 3% meminfo.Shmem 109825 +11.1% 121996 ± 3% proc-vmstat.nr_active_anon 3428 ± 3% +29.1% 4426 ± 2% proc-vmstat.nr_active_file 1427 +13.3% 1618 ± 2% proc-vmstat.nr_dirtied 376669 +3.5% 389792 proc-vmstat.nr_file_pages 114820 +10.6% 126987 ± 3% proc-vmstat.nr_shmem 1372 +15.1% 1579 ± 4% proc-vmstat.nr_written 109825 +11.1% 121996 ± 3% proc-vmstat.nr_zone_active_anon 3428 ± 3% +29.1% 4426 ± 2% proc-vmstat.nr_zone_active_file 151153 ± 5% +16.9% 176679 ± 3% proc-vmstat.numa_hint_faults 106816 ± 7% +17.4% 125437 ± 4% proc-vmstat.numa_hint_faults_local 2915186 +5.4% 3071445 proc-vmstat.numa_hit 2835807 +5.4% 2988976 proc-vmstat.numa_local 129566 ± 2% +7.4% 139212 ± 4% proc-vmstat.pgactivate 4233617 +2.8% 4353951 proc-vmstat.pgalloc_normal 1622353 +12.0% 1816326 proc-vmstat.pgfault 3863258 +2.8% 3970690 proc-vmstat.pgfree 1290723 ± 3% +29.8% 1675288 ± 2% proc-vmstat.pgpgout 86298 +10.3% 95227 ± 3% proc-vmstat.pgreuse 189.67 ± 5% -9.6% 171.45 ± 5% sched_debug.cfs_rq:/.load_avg.stddev 15090675 +13.9% 17181333 sched_debug.cfs_rq:/.min_vruntime.avg 15410230 +14.0% 17563602 sched_debug.cfs_rq:/.min_vruntime.max 14179241 +13.8% 16142220 sched_debug.cfs_rq:/.min_vruntime.min 909405 ± 5% +13.7% 1034081 ± 6% sched_debug.cfs_rq:/.spread0.avg 1229377 ± 4% +15.2% 1416676 ± 3% sched_debug.cfs_rq:/.spread0.max 240213 +12.5% 270308 sched_debug.cpu.clock.avg 240225 +12.5% 270320 sched_debug.cpu.clock.max 240200 +12.5% 270295 sched_debug.cpu.clock.min 237580 +12.5% 267317 sched_debug.cpu.clock_task.avg 237797 +12.5% 267523 sched_debug.cpu.clock_task.max 232767 +12.8% 262508 sched_debug.cpu.clock_task.min 64022 ± 2% +31.4% 84141 sched_debug.cpu.nr_switches.avg 77073 +28.8% 99292 ± 3% sched_debug.cpu.nr_switches.max 60980 ± 2% +32.5% 80775 sched_debug.cpu.nr_switches.min 1399 ± 8% +23.7% 1731 ± 4% sched_debug.cpu.nr_uninterruptible.max 248.99 ± 6% +21.4% 302.15 ± 9% sched_debug.cpu.nr_uninterruptible.stddev 240200 +12.5% 270295 sched_debug.cpu_clk 239703 +12.6% 269797 sched_debug.ktime 240543 +12.5% 270629 sched_debug.sched_clk 1.221e+10 -1.1% 1.208e+10 perf-stat.i.branch-instructions 0.22 ± 23% -0.0 0.17 perf-stat.i.branch-miss-rate% 14793770 -6.0% 13912167 perf-stat.i.branch-misses 25953 ± 2% +14.8% 29803 perf-stat.i.context-switches 4869 +13.6% 5530 perf-stat.i.cpu-migrations 1.64e+10 -1.3% 1.62e+10 perf-stat.i.dTLB-loads 1.075e+09 -9.9% 9.682e+08 ± 2% perf-stat.i.dTLB-stores 430893 ± 2% +11.6% 480826 perf-stat.i.iTLB-loads 6.072e+10 -1.1% 6.005e+10 perf-stat.i.instructions 338.69 -1.5% 333.53 perf-stat.i.metric.M/sec 6039682 -7.8% 5565765 ± 2% perf-stat.i.node-store-misses 96902 ± 2% -4.0% 93045 ± 2% perf-stat.i.node-stores 0.12 -0.0 0.12 perf-stat.overall.branch-miss-rate% 1.219e+10 -1.0% 1.206e+10 perf-stat.ps.branch-instructions 14740466 -5.9% 13872099 perf-stat.ps.branch-misses 25873 ± 2% +14.9% 29722 perf-stat.ps.context-switches 4856 +13.6% 5517 perf-stat.ps.cpu-migrations 1.637e+10 -1.2% 1.617e+10 perf-stat.ps.dTLB-loads 1.072e+09 -9.9% 9.665e+08 ± 2% perf-stat.ps.dTLB-stores 428747 ± 2% +11.6% 478487 perf-stat.ps.iTLB-loads 6.061e+10 -1.1% 5.996e+10 perf-stat.ps.instructions 6028729 -7.8% 5557348 ± 2% perf-stat.ps.node-store-misses 96841 -4.1% 92846 ± 3% perf-stat.ps.node-stores 2.734e+13 +12.2% 3.066e+13 ± 2% perf-stat.total.instructions 1.54 ± 72% -86.8% 0.20 ±221% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 0.17 ± 19% -47.8% 0.09 ± 22% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.__ext4_journal_get_write_access.ext4_orphan_add.ext4_truncate 1.57 ± 11% +18.6% 1.86 ± 5% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc 1.45 ± 37% +126.7% 3.29 ± 34% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.jbd2__journal_start.__ext4_journal_start_sb 0.59 ± 34% +58.4% 0.93 ± 24% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate 3.16 ± 68% -80.8% 0.61 ±223% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 8.52 ± 28% +80.5% 15.38 ± 25% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__alloc_file.alloc_empty_file 162500 ± 9% +25.1% 203298 ± 10% perf-sched.total_wait_and_delay.count.ms 116.08 ± 9% -21.7% 90.91 ± 13% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty 134.71 ± 9% -24.4% 101.80 ± 7% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc 145.16 ± 21% -42.0% 84.13 ± 46% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.__mutex_lock.isra.0 153.37 ± 30% -68.2% 48.78 ±103% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.ext4_journal_check_start.__ext4_journal_start_sb.ext4_truncate 10.19 ± 5% -84.0% 1.62 ±223% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 76.08 ± 11% -23.6% 58.16 ± 12% perf-sched.wait_and_delay.avg.ms.wait_transaction_locked.add_transaction_credits.start_this_handle.jbd2__journal_start 69.33 ± 14% +36.5% 94.67 ± 13% perf-sched.wait_and_delay.count.jbd2_journal_commit_transaction.kjournald2.kthread.ret_from_fork 890.17 ± 5% -30.8% 615.83 ± 4% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty 3176 ± 15% +43.3% 4551 ± 17% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.__ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty 349.17 ± 5% -17.3% 288.83 ± 5% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc 654.17 ± 73% +108.3% 1362 ± 18% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.kmem_cache_alloc.start_this_handle.jbd2__journal_start 953.33 ± 3% -84.6% 146.67 ±223% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 35615 ± 10% +26.7% 45108 ± 10% perf-sched.wait_and_delay.count.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_add 54444 ± 15% +44.7% 78784 ± 17% perf-sched.wait_and_delay.count.wait_transaction_locked.add_transaction_credits.start_this_handle.jbd2__journal_start 749.81 ± 5% -18.6% 610.38 ± 3% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty 751.99 ± 7% -21.6% 589.37 ± 5% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc 545.93 ± 17% -31.5% 374.21 ± 24% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.__kmalloc.ext4_ext_remove_space.ext4_ext_truncate 619.18 ± 17% -45.6% 337.10 ± 49% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.__mutex_lock.isra.0 573.86 ± 20% -64.0% 206.68 ±104% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.ext4_journal_check_start.__ext4_journal_start_sb.ext4_truncate 1067 ± 12% -84.4% 166.83 ±223% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 650.19 ± 8% -13.1% 564.84 ± 3% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate 676.68 ± 9% -17.8% 556.38 ± 3% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.wait_for_completion.stop_two_cpus.migrate_swap 745.90 ± 4% -18.2% 609.91 ± 2% perf-sched.wait_and_delay.max.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2 775.20 ± 6% -18.9% 628.79 perf-sched.wait_and_delay.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_add 770.37 ± 6% -18.9% 624.81 perf-sched.wait_and_delay.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_del 114.49 ± 9% -22.1% 89.15 ± 13% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty 133.14 ± 9% -24.9% 99.94 ± 7% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc 142.91 ± 22% -37.9% 88.74 ± 26% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__mutex_lock.isra.0 120.05 ± 39% -61.7% 46.00 ± 96% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write.ext4_truncate.ext4_setattr 151.19 ± 30% -57.1% 64.88 ± 53% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.ext4_journal_check_start.__ext4_journal_start_sb.ext4_truncate 4.55 ± 4% +12.8% 5.13 ± 3% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.jbd2_journal_commit_transaction.kjournald2.kthread 74.36 ± 11% -24.3% 56.28 ± 13% perf-sched.wait_time.avg.ms.wait_transaction_locked.add_transaction_credits.start_this_handle.jbd2__journal_start 749.64 ± 5% -18.6% 610.37 ± 3% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__ext4_handle_dirty_metadata.ext4_do_update_inode.ext4_mark_iloc_dirty 751.98 ± 7% -21.6% 589.37 ± 5% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc 545.92 ± 17% -32.0% 371.40 ± 24% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__kmalloc.ext4_ext_remove_space.ext4_ext_truncate 619.17 ± 17% -38.7% 379.32 ± 23% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__mutex_lock.isra.0 571.69 ± 20% -52.1% 273.60 ± 56% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.ext4_journal_check_start.__ext4_journal_start_sb.ext4_truncate 650.17 ± 8% -13.1% 564.83 ± 3% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate 676.67 ± 9% -17.8% 556.37 ± 3% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.wait_for_completion.stop_two_cpus.migrate_swap 745.40 ± 4% -18.3% 609.29 ± 2% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2 769.37 ± 6% -19.0% 622.87 ± 2% perf-sched.wait_time.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_add 769.35 ± 6% -19.1% 622.61 ± 2% perf-sched.wait_time.max.ms.schedule_preempt_disabled.__mutex_lock.isra.0.ext4_orphan_del 0.59 ± 4% -0.2 0.35 ± 70% perf-profile.calltrace.cycles-pp.ext4_ext_truncate.ext4_truncate.ext4_setattr.notify_change.do_truncate 0.81 ± 3% -0.1 0.70 ± 4% perf-profile.children.cycles-pp.ext4_reserve_inode_write 0.89 ± 3% -0.1 0.78 ± 3% perf-profile.children.cycles-pp.__ext4_mark_inode_dirty 1.00 ± 2% -0.1 0.89 ± 3% perf-profile.children.cycles-pp.ext4_mark_iloc_dirty 0.59 ± 4% -0.1 0.52 ± 4% perf-profile.children.cycles-pp.ext4_ext_truncate 0.65 ± 2% -0.1 0.59 ± 4% perf-profile.children.cycles-pp.ext4_do_update_inode 0.32 ± 2% -0.0 0.28 ± 2% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 0.35 ± 4% -0.0 0.30 ± 4% perf-profile.children.cycles-pp.ext4_ext_remove_space 0.21 ± 3% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.__getblk_gfp 0.15 ± 13% -0.0 0.11 ± 15% perf-profile.children.cycles-pp.run_ksoftirqd 0.18 ± 3% -0.0 0.14 ± 5% perf-profile.children.cycles-pp.crc32c_pcl_intel_update 0.15 ± 11% -0.0 0.12 ± 11% perf-profile.children.cycles-pp.smpboot_thread_fn 0.18 ± 2% -0.0 0.15 ± 4% perf-profile.children.cycles-pp.__find_get_block 0.12 ± 4% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.mutex_unlock 0.15 ± 3% -0.0 0.12 ± 5% perf-profile.children.cycles-pp.ext4_inode_csum_set 0.24 ± 3% -0.0 0.21 ± 3% perf-profile.children.cycles-pp.__close 0.10 ± 9% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.osq_unlock 0.20 ± 3% -0.0 0.18 ± 3% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.14 ± 3% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.wake_up_q 0.10 ± 4% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.cap_inode_need_killpriv 0.12 ± 4% -0.0 0.11 ± 6% perf-profile.children.cycles-pp.mutex_lock 0.08 -0.0 0.06 ± 7% perf-profile.children.cycles-pp.ext4_xattr_get 0.11 -0.0 0.09 ± 5% perf-profile.children.cycles-pp.dentry_needs_remove_privs 0.10 -0.0 0.08 ± 5% perf-profile.children.cycles-pp.__vfs_getxattr 0.11 ± 3% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.alloc_empty_file 0.11 -0.0 0.10 ± 4% perf-profile.children.cycles-pp.__alloc_file 0.18 ± 6% +0.0 0.20 ± 4% perf-profile.children.cycles-pp.do_softirq_own_stack 0.19 ± 6% +0.0 0.21 ± 4% perf-profile.children.cycles-pp.irq_exit_rcu 0.03 ±100% +0.0 0.07 ± 7% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.06 ± 45% +0.0 0.10 ± 11% perf-profile.children.cycles-pp.ktime_get 0.69 +0.1 0.75 perf-profile.children.cycles-pp.mutex_spin_on_owner 0.35 ± 6% +0.1 0.42 ± 6% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.30 ± 5% +0.1 0.38 ± 8% perf-profile.children.cycles-pp.tick_sched_timer 0.47 ± 7% +0.1 0.57 ± 7% perf-profile.children.cycles-pp.hrtimer_interrupt 0.47 ± 7% +0.1 0.58 ± 7% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.68 ± 6% +0.1 0.80 ± 4% perf-profile.children.cycles-pp.asm_call_sysvec_on_stack 0.68 ± 6% +0.1 0.81 ± 4% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.72 ± 7% +0.1 0.85 ± 4% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.24 -0.0 0.19 ± 3% perf-profile.self.cycles-pp.ext4_reserve_inode_write 0.29 ± 4% -0.0 0.24 ± 2% perf-profile.self.cycles-pp.ext4_mark_iloc_dirty 0.32 ± 3% -0.0 0.27 ± 4% perf-profile.self.cycles-pp.__mutex_lock 0.07 ± 7% -0.0 0.03 ±100% perf-profile.self.cycles-pp.ext4_inode_csum_set 0.12 ± 4% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.mutex_unlock 0.12 ± 5% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.mutex_lock 0.10 ± 6% -0.0 0.08 ± 12% perf-profile.self.cycles-pp.osq_unlock 0.12 -0.0 0.10 perf-profile.self.cycles-pp._raw_spin_lock 0.09 ± 7% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.__find_get_block 0.07 ± 5% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.ext4_xattr_get 0.09 -0.0 0.08 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.05 ± 45% +0.0 0.09 ± 12% perf-profile.self.cycles-pp.ktime_get 0.02 ±141% +0.0 0.06 perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.68 +0.1 0.74 ± 2% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.02 ±141% +0.1 0.08 ± 8% perf-profile.self.cycles-pp.ext4_orphan_add 20823 ± 2% +14.2% 23775 ± 3% softirqs.CPU0.SCHED 17539 ± 2% +22.4% 21460 ± 5% softirqs.CPU1.SCHED 17122 +19.9% 20527 softirqs.CPU10.SCHED 17055 ± 2% +19.8% 20433 softirqs.CPU11.SCHED 16919 +21.4% 20538 softirqs.CPU12.SCHED 17013 ± 2% +20.6% 20510 ± 2% softirqs.CPU13.SCHED 17119 +19.8% 20501 softirqs.CPU14.SCHED 17158 +19.1% 20431 ± 2% softirqs.CPU15.SCHED 17223 +18.7% 20443 softirqs.CPU16.SCHED 16969 ± 2% +19.7% 20305 softirqs.CPU17.SCHED 16997 +20.3% 20441 softirqs.CPU18.SCHED 17126 +19.1% 20399 softirqs.CPU19.SCHED 17544 ± 3% +18.5% 20796 ± 5% softirqs.CPU2.SCHED 17207 +19.1% 20489 softirqs.CPU20.SCHED 17215 +17.4% 20219 softirqs.CPU21.SCHED 16574 +19.2% 19758 ± 3% softirqs.CPU22.SCHED 16901 ± 2% +17.7% 19893 ± 2% softirqs.CPU23.SCHED 16901 ± 3% +19.4% 20180 ± 2% softirqs.CPU24.SCHED 16818 +19.5% 20099 ± 2% softirqs.CPU25.SCHED 16892 +18.6% 20039 softirqs.CPU26.SCHED 16839 +20.0% 20205 softirqs.CPU27.SCHED 16926 +18.7% 20091 softirqs.CPU28.SCHED 16762 ± 2% +20.1% 20125 softirqs.CPU29.SCHED 17050 ± 2% +19.7% 20410 ± 2% softirqs.CPU3.SCHED 16961 +19.7% 20304 ± 2% softirqs.CPU30.SCHED 17013 +19.0% 20238 ± 2% softirqs.CPU31.SCHED 16836 +22.4% 20599 ± 3% softirqs.CPU32.SCHED 17160 +17.5% 20164 softirqs.CPU33.SCHED 16658 ± 2% +21.7% 20268 ± 2% softirqs.CPU34.SCHED 16980 ± 2% +18.2% 20069 ± 3% softirqs.CPU35.SCHED 16720 +20.0% 20067 ± 2% softirqs.CPU36.SCHED 17043 +19.3% 20338 softirqs.CPU37.SCHED 16935 ± 2% +20.0% 20321 softirqs.CPU38.SCHED 17031 +19.5% 20349 softirqs.CPU39.SCHED 17457 ± 3% +17.1% 20438 softirqs.CPU4.SCHED 16977 +20.1% 20393 ± 2% softirqs.CPU40.SCHED 16873 +20.7% 20369 softirqs.CPU41.SCHED 17216 ± 2% +17.6% 20238 ± 2% softirqs.CPU42.SCHED 17045 ± 2% +19.3% 20342 ± 2% softirqs.CPU43.SCHED 16175 ± 3% +20.4% 19473 ± 2% softirqs.CPU44.SCHED 16913 ± 2% +18.6% 20051 softirqs.CPU45.SCHED 16989 ± 2% +17.6% 19980 softirqs.CPU46.SCHED 16774 ± 2% +18.9% 19949 softirqs.CPU47.SCHED 16922 ± 2% +19.1% 20162 softirqs.CPU48.SCHED 16933 ± 3% +19.8% 20286 softirqs.CPU49.SCHED 17135 ± 3% +20.1% 20585 softirqs.CPU5.SCHED 16998 ± 2% +16.8% 19847 softirqs.CPU50.SCHED 16815 ± 2% +19.4% 20077 softirqs.CPU51.SCHED 16962 +18.6% 20111 softirqs.CPU52.SCHED 16853 ± 2% +20.0% 20224 softirqs.CPU53.SCHED 17002 +19.5% 20320 softirqs.CPU54.SCHED 17025 ± 2% +18.8% 20228 ± 2% softirqs.CPU55.SCHED 16851 +19.8% 20190 softirqs.CPU56.SCHED 17018 +18.3% 20132 softirqs.CPU57.SCHED 16949 ± 2% +19.8% 20308 ± 2% softirqs.CPU58.SCHED 17189 +18.0% 20287 softirqs.CPU59.SCHED 16891 +19.2% 20135 softirqs.CPU6.SCHED 16989 +20.0% 20387 softirqs.CPU60.SCHED 17075 +17.7% 20102 ± 2% softirqs.CPU61.SCHED 16938 ± 2% +19.8% 20285 softirqs.CPU62.SCHED 16854 ± 2% +19.2% 20094 ± 2% softirqs.CPU63.SCHED 17063 +18.5% 20216 softirqs.CPU64.SCHED 17048 +17.2% 19984 softirqs.CPU65.SCHED 16870 +19.4% 20138 softirqs.CPU66.SCHED 16817 +20.6% 20274 ± 2% softirqs.CPU67.SCHED 16868 ± 2% +20.1% 20256 softirqs.CPU68.SCHED 16890 +19.7% 20212 ± 2% softirqs.CPU69.SCHED 16842 +20.2% 20252 softirqs.CPU7.SCHED 17011 +19.8% 20375 softirqs.CPU70.SCHED 17048 +20.1% 20478 ± 2% softirqs.CPU71.SCHED 16881 ± 2% +19.6% 20194 softirqs.CPU72.SCHED 16931 +19.9% 20294 softirqs.CPU73.SCHED 17102 +18.7% 20308 softirqs.CPU74.SCHED 16926 ± 2% +19.4% 20206 softirqs.CPU75.SCHED 16757 ± 2% +21.6% 20380 ± 2% softirqs.CPU76.SCHED 17052 +19.8% 20420 softirqs.CPU77.SCHED 16998 +19.8% 20368 softirqs.CPU78.SCHED 17209 ± 2% +19.3% 20533 softirqs.CPU79.SCHED 17084 ± 2% +18.6% 20260 softirqs.CPU8.SCHED 16973 ± 2% +20.9% 20525 ± 2% softirqs.CPU80.SCHED 17198 +18.9% 20457 ± 2% softirqs.CPU81.SCHED 16907 ± 2% +20.8% 20419 ± 2% softirqs.CPU82.SCHED 17023 +19.3% 20316 ± 2% softirqs.CPU83.SCHED 16935 +18.8% 20116 ± 2% softirqs.CPU84.SCHED 17084 +18.8% 20297 softirqs.CPU85.SCHED 17086 +19.2% 20363 softirqs.CPU86.SCHED 16995 ± 3% +19.7% 20350 ± 2% softirqs.CPU87.SCHED 17100 ± 2% +19.4% 20420 softirqs.CPU9.SCHED 1498602 +19.3% 1788245 softirqs.SCHED 80372 +12.5% 90439 ± 2% softirqs.TIMER 902.67 +13.6% 1025 ± 2% interrupts.9:IO-APIC.9-fasteoi.acpi 1556804 ± 4% +35.0% 2102423 ± 3% interrupts.CAL:Function_call_interrupts 17978 ± 4% +27.2% 22867 ± 8% interrupts.CPU0.CAL:Function_call_interrupts 902210 +13.5% 1024347 ± 2% interrupts.CPU0.LOC:Local_timer_interrupts 3865 ± 3% +22.0% 4714 ± 3% interrupts.CPU0.RES:Rescheduling_interrupts 902.67 +13.6% 1025 ± 2% interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi 17763 ± 7% +27.9% 22718 ± 8% interrupts.CPU1.CAL:Function_call_interrupts 902064 +13.5% 1024088 ± 2% interrupts.CPU1.LOC:Local_timer_interrupts 3156 ± 3% +22.8% 3874 ± 5% interrupts.CPU1.RES:Rescheduling_interrupts 16929 ± 5% +27.5% 21592 ± 6% interrupts.CPU10.CAL:Function_call_interrupts 902163 +13.5% 1024115 ± 2% interrupts.CPU10.LOC:Local_timer_interrupts 3174 ± 4% +20.5% 3825 ± 3% interrupts.CPU10.RES:Rescheduling_interrupts 17099 ± 4% +25.8% 21503 ± 7% interrupts.CPU11.CAL:Function_call_interrupts 902105 +13.5% 1024101 ± 2% interrupts.CPU11.LOC:Local_timer_interrupts 3166 ± 3% +22.1% 3865 ± 3% interrupts.CPU11.RES:Rescheduling_interrupts 16935 ± 5% +27.2% 21542 ± 7% interrupts.CPU12.CAL:Function_call_interrupts 902122 +13.5% 1024089 ± 2% interrupts.CPU12.LOC:Local_timer_interrupts 3134 ± 4% +22.4% 3837 ± 5% interrupts.CPU12.RES:Rescheduling_interrupts 17107 ± 5% +25.2% 21414 ± 6% interrupts.CPU13.CAL:Function_call_interrupts 902138 +13.5% 1024071 ± 2% interrupts.CPU13.LOC:Local_timer_interrupts 3154 ± 4% +21.3% 3825 ± 5% interrupts.CPU13.RES:Rescheduling_interrupts 16949 ± 4% +28.0% 21692 ± 8% interrupts.CPU14.CAL:Function_call_interrupts 902142 +13.5% 1024014 ± 2% interrupts.CPU14.LOC:Local_timer_interrupts 3154 ± 2% +20.8% 3809 ± 5% interrupts.CPU14.RES:Rescheduling_interrupts 16971 ± 6% +26.5% 21460 ± 7% interrupts.CPU15.CAL:Function_call_interrupts 902125 +13.5% 1024008 ± 2% interrupts.CPU15.LOC:Local_timer_interrupts 3159 ± 3% +21.7% 3847 ± 5% interrupts.CPU15.RES:Rescheduling_interrupts 16914 ± 4% +26.9% 21469 ± 6% interrupts.CPU16.CAL:Function_call_interrupts 902263 +13.5% 1024072 ± 2% interrupts.CPU16.LOC:Local_timer_interrupts 3176 ± 4% +20.6% 3831 ± 3% interrupts.CPU16.RES:Rescheduling_interrupts 16959 ± 5% +27.7% 21659 ± 7% interrupts.CPU17.CAL:Function_call_interrupts 902131 +13.5% 1024076 ± 2% interrupts.CPU17.LOC:Local_timer_interrupts 3164 ± 3% +21.1% 3834 ± 5% interrupts.CPU17.RES:Rescheduling_interrupts 16921 ± 5% +27.9% 21647 ± 7% interrupts.CPU18.CAL:Function_call_interrupts 901992 +13.5% 1024059 ± 2% interrupts.CPU18.LOC:Local_timer_interrupts 3114 ± 3% +23.7% 3851 ± 3% interrupts.CPU18.RES:Rescheduling_interrupts 16953 ± 5% +27.8% 21669 ± 7% interrupts.CPU19.CAL:Function_call_interrupts 902113 +13.5% 1024021 ± 2% interrupts.CPU19.LOC:Local_timer_interrupts 3186 ± 2% +20.8% 3850 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts 17479 ± 5% +32.7% 23186 ± 3% interrupts.CPU2.CAL:Function_call_interrupts 902148 +13.5% 1024034 ± 2% interrupts.CPU2.LOC:Local_timer_interrupts 3163 ± 3% +21.2% 3835 ± 3% interrupts.CPU2.RES:Rescheduling_interrupts 16875 ± 5% +27.8% 21575 ± 7% interrupts.CPU20.CAL:Function_call_interrupts 902126 +13.5% 1024077 ± 2% interrupts.CPU20.LOC:Local_timer_interrupts 3125 ± 3% +23.9% 3874 ± 3% interrupts.CPU20.RES:Rescheduling_interrupts 16824 ± 4% +27.8% 21502 ± 7% interrupts.CPU21.CAL:Function_call_interrupts 902123 +13.5% 1024032 ± 2% interrupts.CPU21.LOC:Local_timer_interrupts 3119 ± 2% +23.8% 3861 ± 4% interrupts.CPU21.RES:Rescheduling_interrupts 18853 ± 10% +42.1% 26784 ± 7% interrupts.CPU22.CAL:Function_call_interrupts 902129 +13.5% 1024053 ± 2% interrupts.CPU22.LOC:Local_timer_interrupts 3076 ± 2% +15.9% 3564 ± 3% interrupts.CPU22.RES:Rescheduling_interrupts 18904 ± 10% +41.5% 26758 ± 7% interrupts.CPU23.CAL:Function_call_interrupts 902141 +13.5% 1024100 ± 2% interrupts.CPU23.LOC:Local_timer_interrupts 3055 ± 2% +13.8% 3478 ± 2% interrupts.CPU23.RES:Rescheduling_interrupts 18697 ± 10% +42.0% 26548 ± 7% interrupts.CPU24.CAL:Function_call_interrupts 902132 +13.5% 1024051 ± 2% interrupts.CPU24.LOC:Local_timer_interrupts 2996 ± 3% +15.3% 3454 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts 18610 ± 10% +42.4% 26498 ± 7% interrupts.CPU25.CAL:Function_call_interrupts 902099 +13.5% 1024078 ± 2% interrupts.CPU25.LOC:Local_timer_interrupts 3061 ± 3% +15.4% 3531 ± 2% interrupts.CPU25.RES:Rescheduling_interrupts 18506 ± 10% +42.8% 26417 ± 7% interrupts.CPU26.CAL:Function_call_interrupts 902116 +13.5% 1024074 ± 2% interrupts.CPU26.LOC:Local_timer_interrupts 3016 ± 3% +15.6% 3488 ± 2% interrupts.CPU26.RES:Rescheduling_interrupts 18712 ± 11% +40.1% 26212 ± 7% interrupts.CPU27.CAL:Function_call_interrupts 902131 +13.5% 1023984 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts 3032 ± 3% +16.4% 3528 ± 2% interrupts.CPU27.RES:Rescheduling_interrupts 18533 ± 10% +41.4% 26214 ± 7% interrupts.CPU28.CAL:Function_call_interrupts 902108 +13.5% 1024107 ± 2% interrupts.CPU28.LOC:Local_timer_interrupts 3022 ± 2% +17.1% 3539 ± 2% interrupts.CPU28.RES:Rescheduling_interrupts 18483 ± 10% +41.9% 26234 ± 7% interrupts.CPU29.CAL:Function_call_interrupts 902262 +13.5% 1024052 ± 2% interrupts.CPU29.LOC:Local_timer_interrupts 3034 +16.4% 3533 ± 2% interrupts.CPU29.RES:Rescheduling_interrupts 17359 ± 5% +29.3% 22454 ± 7% interrupts.CPU3.CAL:Function_call_interrupts 902230 +13.5% 1024080 ± 2% interrupts.CPU3.LOC:Local_timer_interrupts 3160 ± 4% +22.2% 3863 ± 3% interrupts.CPU3.RES:Rescheduling_interrupts 18526 ± 10% +42.2% 26340 ± 7% interrupts.CPU30.CAL:Function_call_interrupts 902146 +13.5% 1024088 ± 2% interrupts.CPU30.LOC:Local_timer_interrupts 3003 +16.6% 3503 ± 2% interrupts.CPU30.RES:Rescheduling_interrupts 18417 ± 10% +42.1% 26162 ± 8% interrupts.CPU31.CAL:Function_call_interrupts 902108 +13.5% 1024056 ± 2% interrupts.CPU31.LOC:Local_timer_interrupts 2992 ± 3% +17.2% 3506 ± 2% interrupts.CPU31.RES:Rescheduling_interrupts 18324 ± 10% +41.6% 25944 ± 7% interrupts.CPU32.CAL:Function_call_interrupts 902092 +13.5% 1024078 ± 2% interrupts.CPU32.LOC:Local_timer_interrupts 3037 ± 3% +15.2% 3498 ± 2% interrupts.CPU32.RES:Rescheduling_interrupts 18353 ± 11% +42.0% 26057 ± 7% interrupts.CPU33.CAL:Function_call_interrupts 902085 +13.5% 1024101 ± 2% interrupts.CPU33.LOC:Local_timer_interrupts 3055 ± 4% +14.6% 3502 ± 2% interrupts.CPU33.RES:Rescheduling_interrupts 18376 ± 10% +41.2% 25953 ± 7% interrupts.CPU34.CAL:Function_call_interrupts 902095 +13.5% 1024078 ± 2% interrupts.CPU34.LOC:Local_timer_interrupts 3015 ± 2% +14.4% 3448 ± 3% interrupts.CPU34.RES:Rescheduling_interrupts 18232 ± 10% +42.5% 25986 ± 7% interrupts.CPU35.CAL:Function_call_interrupts 902197 +13.5% 1024042 ± 2% interrupts.CPU35.LOC:Local_timer_interrupts 2990 ± 3% +16.6% 3487 ± 2% interrupts.CPU35.RES:Rescheduling_interrupts 18314 ± 10% +41.9% 25991 ± 7% interrupts.CPU36.CAL:Function_call_interrupts 902046 +13.5% 1024078 ± 2% interrupts.CPU36.LOC:Local_timer_interrupts 3002 ± 2% +15.9% 3479 interrupts.CPU36.RES:Rescheduling_interrupts 18369 ± 10% +40.6% 25829 ± 7% interrupts.CPU37.CAL:Function_call_interrupts 902117 +13.5% 1024026 ± 2% interrupts.CPU37.LOC:Local_timer_interrupts 2999 ± 2% +16.4% 3490 ± 2% interrupts.CPU37.RES:Rescheduling_interrupts 18331 ± 10% +41.5% 25932 ± 7% interrupts.CPU38.CAL:Function_call_interrupts 902118 +13.5% 1024085 ± 2% interrupts.CPU38.LOC:Local_timer_interrupts 2995 ± 3% +16.0% 3474 ± 4% interrupts.CPU38.RES:Rescheduling_interrupts 18313 ± 10% +41.6% 25935 ± 7% interrupts.CPU39.CAL:Function_call_interrupts 902082 +13.5% 1024103 ± 2% interrupts.CPU39.LOC:Local_timer_interrupts 2973 ± 2% +18.1% 3510 interrupts.CPU39.RES:Rescheduling_interrupts 18089 ± 10% +20.9% 21864 ± 6% interrupts.CPU4.CAL:Function_call_interrupts 902120 +13.5% 1024041 ± 2% interrupts.CPU4.LOC:Local_timer_interrupts 3126 ± 2% +22.9% 3842 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts 18303 ± 10% +42.3% 26045 ± 7% interrupts.CPU40.CAL:Function_call_interrupts 902120 +13.5% 1024097 ± 2% interrupts.CPU40.LOC:Local_timer_interrupts 3019 ± 2% +14.7% 3462 ± 3% interrupts.CPU40.RES:Rescheduling_interrupts 18288 ± 10% +42.8% 26107 ± 7% interrupts.CPU41.CAL:Function_call_interrupts 902064 +13.5% 1024075 ± 2% interrupts.CPU41.LOC:Local_timer_interrupts 2992 +16.6% 3489 ± 2% interrupts.CPU41.RES:Rescheduling_interrupts 18169 ± 10% +43.3% 26042 ± 7% interrupts.CPU42.CAL:Function_call_interrupts 902151 +13.5% 1024091 ± 2% interrupts.CPU42.LOC:Local_timer_interrupts 3010 ± 2% +16.7% 3512 ± 2% interrupts.CPU42.RES:Rescheduling_interrupts 18161 ± 11% +42.4% 25871 ± 7% interrupts.CPU43.CAL:Function_call_interrupts 902074 +13.5% 1024076 ± 2% interrupts.CPU43.LOC:Local_timer_interrupts 2946 ± 2% +18.1% 3478 ± 2% interrupts.CPU43.RES:Rescheduling_interrupts 16873 ± 4% +26.9% 21415 ± 6% interrupts.CPU44.CAL:Function_call_interrupts 902175 +13.5% 1024065 ± 2% interrupts.CPU44.LOC:Local_timer_interrupts 3067 ± 4% +21.2% 3719 ± 5% interrupts.CPU44.RES:Rescheduling_interrupts 16915 ± 5% +28.3% 21704 ± 6% interrupts.CPU45.CAL:Function_call_interrupts 902101 +13.5% 1024072 ± 2% interrupts.CPU45.LOC:Local_timer_interrupts 3056 ± 3% +22.2% 3734 ± 3% interrupts.CPU45.RES:Rescheduling_interrupts 16894 ± 4% +27.6% 21553 ± 7% interrupts.CPU46.CAL:Function_call_interrupts 902132 +13.5% 1024049 ± 2% interrupts.CPU46.LOC:Local_timer_interrupts 3069 ± 2% +22.9% 3773 ± 4% interrupts.CPU46.RES:Rescheduling_interrupts 16781 ± 5% +27.6% 21408 ± 7% interrupts.CPU47.CAL:Function_call_interrupts 902147 +13.5% 1024088 ± 2% interrupts.CPU47.LOC:Local_timer_interrupts 3086 ± 3% +20.9% 3732 ± 4% interrupts.CPU47.RES:Rescheduling_interrupts 16892 ± 5% +27.3% 21496 ± 7% interrupts.CPU48.CAL:Function_call_interrupts 902150 +13.5% 1024084 ± 2% interrupts.CPU48.LOC:Local_timer_interrupts 3123 ± 4% +19.1% 3721 ± 4% interrupts.CPU48.RES:Rescheduling_interrupts 16686 ± 4% +27.9% 21340 ± 7% interrupts.CPU49.CAL:Function_call_interrupts 902113 +13.5% 1024070 ± 2% interrupts.CPU49.LOC:Local_timer_interrupts 3039 ± 4% +22.8% 3733 ± 4% interrupts.CPU49.RES:Rescheduling_interrupts 17142 ± 5% +29.2% 22150 ± 7% interrupts.CPU5.CAL:Function_call_interrupts 902296 +13.5% 1024251 ± 2% interrupts.CPU5.LOC:Local_timer_interrupts 3178 ± 3% +21.7% 3866 ± 3% interrupts.CPU5.RES:Rescheduling_interrupts 16894 ± 4% +27.4% 21529 ± 6% interrupts.CPU50.CAL:Function_call_interrupts 902077 +13.5% 1024082 ± 2% interrupts.CPU50.LOC:Local_timer_interrupts 3106 ± 4% +21.0% 3759 ± 3% interrupts.CPU50.RES:Rescheduling_interrupts 16842 ± 4% +28.0% 21565 ± 6% interrupts.CPU51.CAL:Function_call_interrupts 902143 +13.5% 1024096 ± 2% interrupts.CPU51.LOC:Local_timer_interrupts 3106 ± 3% +20.1% 3730 ± 3% interrupts.CPU51.RES:Rescheduling_interrupts 16754 ± 5% +28.8% 21571 ± 7% interrupts.CPU52.CAL:Function_call_interrupts 902098 +13.5% 1024069 ± 2% interrupts.CPU52.LOC:Local_timer_interrupts 3050 ± 2% +21.8% 3715 ± 4% interrupts.CPU52.RES:Rescheduling_interrupts 16868 ± 5% +27.3% 21481 ± 6% interrupts.CPU53.CAL:Function_call_interrupts 902150 +13.5% 1024062 ± 2% interrupts.CPU53.LOC:Local_timer_interrupts 3054 ± 3% +22.1% 3729 ± 4% interrupts.CPU53.RES:Rescheduling_interrupts 16759 ± 5% +27.2% 21311 ± 7% interrupts.CPU54.CAL:Function_call_interrupts 902146 +13.5% 1024081 ± 2% interrupts.CPU54.LOC:Local_timer_interrupts 3061 ± 4% +23.5% 3780 ± 5% interrupts.CPU54.RES:Rescheduling_interrupts 16893 ± 4% +26.4% 21358 ± 7% interrupts.CPU55.CAL:Function_call_interrupts 902113 +13.5% 1024081 ± 2% interrupts.CPU55.LOC:Local_timer_interrupts 3073 ± 3% +21.8% 3742 ± 5% interrupts.CPU55.RES:Rescheduling_interrupts 16820 ± 4% +26.6% 21288 ± 7% interrupts.CPU56.CAL:Function_call_interrupts 902136 +13.5% 1024085 ± 2% interrupts.CPU56.LOC:Local_timer_interrupts 3052 ± 4% +22.3% 3734 ± 5% interrupts.CPU56.RES:Rescheduling_interrupts 16837 ± 4% +26.6% 21324 ± 7% interrupts.CPU57.CAL:Function_call_interrupts 902135 +13.5% 1024226 ± 2% interrupts.CPU57.LOC:Local_timer_interrupts 3075 ± 3% +21.1% 3723 ± 3% interrupts.CPU57.RES:Rescheduling_interrupts 16851 ± 5% +26.7% 21359 ± 7% interrupts.CPU58.CAL:Function_call_interrupts 902145 +13.5% 1024084 ± 2% interrupts.CPU58.LOC:Local_timer_interrupts 3067 ± 3% +22.0% 3742 ± 4% interrupts.CPU58.RES:Rescheduling_interrupts 16813 ± 4% +27.8% 21480 ± 7% interrupts.CPU59.CAL:Function_call_interrupts 902135 +13.5% 1024105 ± 2% interrupts.CPU59.LOC:Local_timer_interrupts 3075 ± 3% +23.1% 3784 ± 5% interrupts.CPU59.RES:Rescheduling_interrupts 17318 ± 4% +25.8% 21778 ± 6% interrupts.CPU6.CAL:Function_call_interrupts 902058 +13.5% 1024091 ± 2% interrupts.CPU6.LOC:Local_timer_interrupts 3167 ± 4% +22.1% 3866 ± 4% interrupts.CPU6.RES:Rescheduling_interrupts 16910 ± 4% +26.7% 21430 ± 7% interrupts.CPU60.CAL:Function_call_interrupts 902154 +13.5% 1024028 ± 2% interrupts.CPU60.LOC:Local_timer_interrupts 3097 ± 3% +21.4% 3760 ± 4% interrupts.CPU60.RES:Rescheduling_interrupts 16900 ± 4% +27.5% 21555 ± 7% interrupts.CPU61.CAL:Function_call_interrupts 902139 +13.5% 1024089 ± 2% interrupts.CPU61.LOC:Local_timer_interrupts 3063 ± 4% +22.0% 3738 ± 3% interrupts.CPU61.RES:Rescheduling_interrupts 16991 ± 4% +26.0% 21414 ± 7% interrupts.CPU62.CAL:Function_call_interrupts 902132 +13.5% 1024089 ± 2% interrupts.CPU62.LOC:Local_timer_interrupts 3125 ± 3% +19.7% 3742 ± 3% interrupts.CPU62.RES:Rescheduling_interrupts 16967 ± 5% +26.9% 21537 ± 6% interrupts.CPU63.CAL:Function_call_interrupts 902111 +13.5% 1024071 ± 2% interrupts.CPU63.LOC:Local_timer_interrupts 3110 ± 3% +22.1% 3799 ± 4% interrupts.CPU63.RES:Rescheduling_interrupts 16832 ± 4% +28.1% 21557 ± 7% interrupts.CPU64.CAL:Function_call_interrupts 902165 +13.5% 1024037 ± 2% interrupts.CPU64.LOC:Local_timer_interrupts 3095 ± 3% +22.5% 3791 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts 16887 ± 5% +27.7% 21563 ± 7% interrupts.CPU65.CAL:Function_call_interrupts 902159 +13.5% 1024083 ± 2% interrupts.CPU65.LOC:Local_timer_interrupts 3079 ± 4% +23.5% 3802 ± 4% interrupts.CPU65.RES:Rescheduling_interrupts 18236 ± 10% +43.5% 26177 ± 8% interrupts.CPU66.CAL:Function_call_interrupts 902097 +13.5% 1023991 ± 2% interrupts.CPU66.LOC:Local_timer_interrupts 2952 ± 3% +16.0% 3423 ± 3% interrupts.CPU66.RES:Rescheduling_interrupts 18440 ± 11% +41.6% 26117 ± 7% interrupts.CPU67.CAL:Function_call_interrupts 902050 +13.5% 1024090 ± 2% interrupts.CPU67.LOC:Local_timer_interrupts 2948 ± 2% +16.8% 3443 ± 3% interrupts.CPU67.RES:Rescheduling_interrupts 18284 ± 10% +43.0% 26148 ± 7% interrupts.CPU68.CAL:Function_call_interrupts 902119 +13.5% 1024096 ± 2% interrupts.CPU68.LOC:Local_timer_interrupts 2960 ± 2% +17.7% 3485 ± 3% interrupts.CPU68.RES:Rescheduling_interrupts 18357 ± 10% +42.0% 26070 ± 7% interrupts.CPU69.CAL:Function_call_interrupts 902096 +13.5% 1024089 ± 2% interrupts.CPU69.LOC:Local_timer_interrupts 2977 +16.1% 3455 ± 5% interrupts.CPU69.RES:Rescheduling_interrupts 16984 ± 5% +28.6% 21843 ± 6% interrupts.CPU7.CAL:Function_call_interrupts 902119 +13.5% 1024116 ± 2% interrupts.CPU7.LOC:Local_timer_interrupts 3122 ± 2% +22.8% 3835 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts 18176 ± 10% +43.4% 26056 ± 7% interrupts.CPU70.CAL:Function_call_interrupts 902106 +13.5% 1024063 ± 2% interrupts.CPU70.LOC:Local_timer_interrupts 2937 ± 2% +18.5% 3479 ± 2% interrupts.CPU70.RES:Rescheduling_interrupts 18320 ± 10% +42.2% 26047 ± 8% interrupts.CPU71.CAL:Function_call_interrupts 902142 +13.5% 1024123 ± 2% interrupts.CPU71.LOC:Local_timer_interrupts 2999 ± 3% +16.3% 3487 ± 3% interrupts.CPU71.RES:Rescheduling_interrupts 18410 ± 10% +42.0% 26143 ± 7% interrupts.CPU72.CAL:Function_call_interrupts 902129 +13.5% 1024094 ± 2% interrupts.CPU72.LOC:Local_timer_interrupts 2965 ± 3% +17.6% 3488 ± 4% interrupts.CPU72.RES:Rescheduling_interrupts 18305 ± 10% +42.5% 26085 ± 8% interrupts.CPU73.CAL:Function_call_interrupts 902133 +13.5% 1024041 ± 2% interrupts.CPU73.LOC:Local_timer_interrupts 2977 ± 3% +16.6% 3471 ± 3% interrupts.CPU73.RES:Rescheduling_interrupts 18344 ± 10% +42.6% 26152 ± 7% interrupts.CPU74.CAL:Function_call_interrupts 902128 +13.5% 1024065 ± 2% interrupts.CPU74.LOC:Local_timer_interrupts 3002 ± 2% +17.0% 3511 ± 3% interrupts.CPU74.RES:Rescheduling_interrupts 18310 ± 10% +41.9% 25976 ± 7% interrupts.CPU75.CAL:Function_call_interrupts 902143 +13.5% 1024072 ± 2% interrupts.CPU75.LOC:Local_timer_interrupts 2922 ± 3% +18.5% 3463 ± 3% interrupts.CPU75.RES:Rescheduling_interrupts 18275 ± 10% +41.6% 25883 ± 7% interrupts.CPU76.CAL:Function_call_interrupts 902099 +13.5% 1024098 ± 2% interrupts.CPU76.LOC:Local_timer_interrupts 2921 ± 4% +18.3% 3457 ± 2% interrupts.CPU76.RES:Rescheduling_interrupts 18330 ± 10% +42.1% 26054 ± 7% interrupts.CPU77.CAL:Function_call_interrupts 902158 +13.5% 1024103 ± 2% interrupts.CPU77.LOC:Local_timer_interrupts 2960 ± 3% +16.4% 3447 ± 3% interrupts.CPU77.RES:Rescheduling_interrupts 18199 ± 10% +43.9% 26183 ± 8% interrupts.CPU78.CAL:Function_call_interrupts 902127 +13.5% 1024113 ± 2% interrupts.CPU78.LOC:Local_timer_interrupts 2936 ± 3% +17.8% 3460 ± 2% interrupts.CPU78.RES:Rescheduling_interrupts 18302 ± 9% +42.2% 26018 ± 7% interrupts.CPU79.CAL:Function_call_interrupts 902128 +13.5% 1024093 ± 2% interrupts.CPU79.LOC:Local_timer_interrupts 2933 ± 3% +16.1% 3406 ± 2% interrupts.CPU79.RES:Rescheduling_interrupts 17257 ± 4% +26.1% 21766 ± 6% interrupts.CPU8.CAL:Function_call_interrupts 902064 +13.5% 1024249 ± 2% interrupts.CPU8.LOC:Local_timer_interrupts 3148 ± 4% +22.8% 3867 ± 4% interrupts.CPU8.RES:Rescheduling_interrupts 18351 ± 10% +41.6% 25983 ± 8% interrupts.CPU80.CAL:Function_call_interrupts 902119 +13.5% 1024104 ± 2% interrupts.CPU80.LOC:Local_timer_interrupts 2950 ± 2% +15.0% 3393 ± 2% interrupts.CPU80.RES:Rescheduling_interrupts 18351 ± 10% +42.5% 26147 ± 8% interrupts.CPU81.CAL:Function_call_interrupts 902138 +13.5% 1024087 ± 2% interrupts.CPU81.LOC:Local_timer_interrupts 2952 ± 2% +16.0% 3424 ± 2% interrupts.CPU81.RES:Rescheduling_interrupts 18259 ± 10% +42.4% 26004 ± 7% interrupts.CPU82.CAL:Function_call_interrupts 902149 +13.5% 1024272 ± 2% interrupts.CPU82.LOC:Local_timer_interrupts 2931 ± 2% +17.0% 3429 ± 2% interrupts.CPU82.RES:Rescheduling_interrupts 18210 ± 10% +42.8% 26012 ± 8% interrupts.CPU83.CAL:Function_call_interrupts 902111 +13.5% 1024099 ± 2% interrupts.CPU83.LOC:Local_timer_interrupts 2933 ± 2% +16.6% 3419 ± 2% interrupts.CPU83.RES:Rescheduling_interrupts 18085 ± 10% +44.2% 26086 ± 8% interrupts.CPU84.CAL:Function_call_interrupts 902118 +13.5% 1024109 ± 2% interrupts.CPU84.LOC:Local_timer_interrupts 2981 ± 2% +13.9% 3396 ± 2% interrupts.CPU84.RES:Rescheduling_interrupts 18294 ± 10% +43.0% 26167 ± 7% interrupts.CPU85.CAL:Function_call_interrupts 902135 +13.5% 1024098 ± 2% interrupts.CPU85.LOC:Local_timer_interrupts 2937 ± 2% +16.8% 3430 ± 3% interrupts.CPU85.RES:Rescheduling_interrupts 18253 ± 10% +42.7% 26056 ± 8% interrupts.CPU86.CAL:Function_call_interrupts 902302 +13.5% 1024102 ± 2% interrupts.CPU86.LOC:Local_timer_interrupts 2999 ± 2% +13.5% 3403 ± 3% interrupts.CPU86.RES:Rescheduling_interrupts 18248 ± 10% +41.5% 25817 ± 8% interrupts.CPU87.CAL:Function_call_interrupts 902127 +13.5% 1024064 ± 2% interrupts.CPU87.LOC:Local_timer_interrupts 2954 ± 2% +14.3% 3377 ± 2% interrupts.CPU87.RES:Rescheduling_interrupts 16991 ± 4% +26.9% 21557 ± 6% interrupts.CPU9.CAL:Function_call_interrupts 902162 +13.5% 1024110 ± 2% interrupts.CPU9.LOC:Local_timer_interrupts 3164 ± 3% +22.5% 3876 ± 5% interrupts.CPU9.RES:Rescheduling_interrupts 79387471 +13.5% 90119618 ± 2% interrupts.LOC:Local_timer_interrupts 269188 +19.2% 320766 ± 2% interrupts.RES:Rescheduling_interrupts aim7.jobs-per-min 14000 +-------------------------------------------------------------------+ | + | 13500 |-+ + + ++ + +.+ + + + .+ | | .+ + + : :: : : :: : :.+. + + : .++ + +.| |. +.+ + : : :: : :: : + ++.+. + +. + + :: .+ + | 13000 |-+ + + :: + + + + :: + | | : + : : | 12500 |-+ + :: | | :: | 12000 |-+ O O O O O O O + | | O O O O O O | | O O O O O O O O | 11500 |-+O O O O O | | O O | 11000 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-knl-f1: 256 threads Intel(R) Genuine Intel(R) CPU 0000 @ 1.30GHz with 112G memory ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/tbox_group/test/testcase/testtime/ucode: gcc-9/performance/x86_64-rhel-8.3/debian-10.4-x86_64-20200603.cgz/lkp-knl-f1/all/aim9/5s/0xffff0190 commit: 4392fbc4ba ("ext4: drop sync argument of ext4_commit_super()") 05c2c00f37 ("ext4: protect superblock modifications with a buffer lock") 4392fbc4bab57db3 05c2c00f3769abb9e323fcaca70 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :5 20% 1:6 dmesg.WARNING:missing_R10_value_at__fsnotify_parent/0x 0:5 4% 0:6 perf-profile.children.cycles-pp.error_return 2:5 8% 2:6 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 74384 +3.4% 76883 aim9.dgram_pipe.ops_per_sec 121865 +2.6% 125020 aim9.disk_cp.ops_per_sec 198997 +1.9% 202688 aim9.disk_wrt.ops_per_sec 52029 -3.9% 49990 aim9.link_test.ops_per_sec 95645 +7.5% 102865 aim9.signal_test.ops_per_sec 80028 +5.0% 84023 aim9.stream_pipe.ops_per_sec 101888 +3.9% 105872 aim9.sync_disk_rw.ops_per_sec 17794 +1.5% 18069 aim9.tcp_test.ops_per_sec 35329 +2.1% 36070 aim9.udp_test.ops_per_sec 84505 +1.3% 85562 proc-vmstat.pgactivate 1.037e+08 -6.0% 97479544 ± 6% perf-stat.i.cache-references 1.034e+08 -6.0% 97161961 ± 6% perf-stat.ps.cache-references 412.50 ± 3% +25.8% 519.03 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.max 45.99 ± 10% +31.6% 60.51 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.stddev 20873 ± 13% -16.2% 17493 ± 6% softirqs.CPU134.RCU 10054 ± 56% -81.3% 1880 ±126% softirqs.CPU14.NET_RX 0.05 ± 6% +45.0% 0.08 ± 28% perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 0.04 ± 4% +22.3% 0.05 ± 10% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.01 ±123% +153.5% 0.04 ± 24% perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all 0.09 ± 18% +82.9% 0.17 ± 52% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.01 ±123% +175.8% 0.04 ± 12% perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all 141.15 ± 21% -41.1% 83.12 ± 41% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.18 ± 40% +70.5% 0.31 ± 41% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.path_put.set_fs_pwd 0.23 ± 33% +59.8% 0.37 ± 26% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.getname_flags.__x64_sys_unlink 141.12 ± 21% -41.1% 83.09 ± 41% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 305.60 ± 29% -43.3% 173.17 ± 21% interrupts.CPU114.NMI:Non-maskable_interrupts 305.60 ± 29% -43.3% 173.17 ± 21% interrupts.CPU114.PMI:Performance_monitoring_interrupts 335.00 ± 20% -36.6% 212.33 ± 32% interrupts.CPU239.NMI:Non-maskable_interrupts 335.00 ± 20% -36.6% 212.33 ± 32% interrupts.CPU239.PMI:Performance_monitoring_interrupts 314.80 ± 32% -37.0% 198.33 ± 46% interrupts.CPU251.NMI:Non-maskable_interrupts 314.80 ± 32% -37.0% 198.33 ± 46% interrupts.CPU251.PMI:Performance_monitoring_interrupts 23.20 ± 44% +117.7% 50.50 ± 57% interrupts.CPU53.RES:Rescheduling_interrupts 1417 ± 10% +24.6% 1766 ± 13% interrupts.CPU6.CAL:Function_call_interrupts 30.40 ± 59% +198.8% 90.83 ± 23% interrupts.CPU6.RES:Rescheduling_interrupts 1260 ± 5% +9.3% 1377 ± 7% interrupts.CPU7.CAL:Function_call_interrupts 253.40 ± 29% +46.7% 371.83 ± 27% interrupts.CPU90.NMI:Non-maskable_interrupts 253.40 ± 29% +46.7% 371.83 ± 27% interrupts.CPU90.PMI:Performance_monitoring_interrupts 1.07 ± 2% -0.2 0.86 ± 24% perf-profile.calltrace.cycles-pp.irqtime_account_irq.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.75 ± 4% +0.3 1.03 ± 34% perf-profile.calltrace.cycles-pp.timerqueue_add.enqueue_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt 0.90 ± 5% +0.3 1.21 ± 33% perf-profile.calltrace.cycles-pp.enqueue_hrtimer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 2.40 ± 6% +0.4 2.76 ± 9% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt 1.42 ± 7% +0.4 1.84 ± 14% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 0.33 ± 7% -0.1 0.25 ± 11% perf-profile.children.cycles-pp.irq_work_needs_cpu 0.09 ± 13% +0.0 0.11 ± 8% perf-profile.children.cycles-pp.hrtimer_run_queues 0.09 ± 9% +0.0 0.12 ± 19% perf-profile.children.cycles-pp.__set_cpus_allowed_ptr 0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.irq_work_single 0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.asm_sysvec_irq_work 0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.sysvec_irq_work 0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.__sysvec_irq_work 0.06 ± 12% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.irq_work_run 0.11 ± 8% +0.0 0.15 ± 22% perf-profile.children.cycles-pp.__x64_sys_sched_setaffinity 0.14 ± 5% +0.1 0.19 ± 17% perf-profile.children.cycles-pp.irq_work_run_list 0.29 ± 6% +0.1 0.39 ± 28% perf-profile.children.cycles-pp.rb_insert_color 0.77 ± 4% +0.3 1.07 ± 34% perf-profile.children.cycles-pp.timerqueue_add 0.92 ± 5% +0.3 1.23 ± 32% perf-profile.children.cycles-pp.enqueue_hrtimer 2.45 ± 6% +0.4 2.82 ± 9% perf-profile.children.cycles-pp.clockevents_program_event 4.23 ± 2% +0.4 4.61 ± 7% perf-profile.children.cycles-pp.scheduler_tick 0.97 ± 2% -0.1 0.83 ± 15% perf-profile.self.cycles-pp.irqtime_account_irq 0.75 ± 4% -0.1 0.67 ± 6% perf-profile.self.cycles-pp.__sysvec_apic_timer_interrupt 0.30 ± 8% -0.1 0.23 ± 15% perf-profile.self.cycles-pp.irq_work_needs_cpu 0.19 ± 3% +0.0 0.21 ± 6% perf-profile.self.cycles-pp.trigger_load_balance 0.09 ± 13% +0.0 0.11 ± 8% perf-profile.self.cycles-pp.hrtimer_run_queues 0.02 ±122% +0.0 0.06 ± 14% perf-profile.self.cycles-pp.io_serial_in 0.14 ± 11% +0.0 0.19 ± 22% perf-profile.self.cycles-pp.tick_sched_handle 0.27 ± 4% +0.1 0.37 ± 27% perf-profile.self.cycles-pp.rb_insert_color Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

1 year

5
11
0 / 0

[drm/i915/dp] 4a8d79901d: WARNING:at_drivers/gpu/drm/i915/display/intel_display_power.c:#assert_can_disable_lcpll[i915]

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 4a8d79901d5bed0812d272c372aa40282937b50f ("drm/i915/dp: Enable Intel's HDR backlight interface (only SDR for now)") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: suspend-stress version: with following parameters: mode: freeze iterations: 10 on test machine: 4 threads Broadwell-Y with 8G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> kern :warn : [ 21.137894] ------------[ cut here ]------------ kern :warn : [ 21.137897] CPU PWM1 enabled kern :warn : [ 21.137930] WARNING: CPU: 3 PID: 575 at drivers/gpu/drm/i915/display/intel_display_power.c:4902 assert_can_disable_lcpll+0x335/0x3c0 [i915] kern :warn : [ 21.138084] Modules linked in: sd_mod t10_pi sg x86_pkg_temp_thermal intel_powerclamp i915 hid_rmi rmi_core hid_multitouch coretemp intel_ra pl_msr crct10dif_pclmul wmi_bmof crc32_pclmul intel_gtt crc32c_intel ghash_clmulni_intel rapl drm_kms_helper intel_cstate ahci libahci serio_raw mei_me proces sor_thermal_device intel_uncore processor_thermal_rfim syscopyarea sysfillrect intel_soc_dts_iosf sysimgblt intel_pch_thermal hid_sensor_custom fb_sys_fops li bata bcma processor_thermal_mbox joydev mei drm processor_thermal_rapl intel_rapl_common int340x_thermal_zone i2c_hid ideapad_laptop sparse_keymap wmi rfkill video int3400_thermal acpi_thermal_rel i2c_designware_platform dw_dmac i2c_designware_core acpi_pad ip_tables kern :warn : [ 21.138148] CPU: 3 PID: 575 Comm: kworker/u8:6 Tainted: G W I 5.11.0-rc2-00745-g4a8d79901d5b #1 kern :warn : [ 21.138153] Hardware name: LENOVO 80HE/VIUU4, BIOS A6CN38WW 09/30/2014 kern :warn : [ 21.138155] Workqueue: events_unbound async_run_entry_fn kern :warn : [ 21.138161] RIP: 0010:assert_can_disable_lcpll+0x335/0x3c0 [i915] kern :info : [ 21.138263] ahci 0000:00:1f.2: pci_pm_suspend_late+0x0/0x40 returned 0 after 0 usecs kern :warn : [ 21.138296] Code: c0 75 22 e8 8d 61 cb ff e9 f3 fd ff ff e8 fd bc 6a c1 0f 0b e9 0c fd ff ff e8 f1 bc 6a c1 0f 0b e9 05 fe ff ff e8 e5 bc 6a c1 <0f> 0b e9 cf fd ff ff e8 d9 bc 6a c1 0f 0b e9 a2 fd ff ff e8 cd bc kern :warn : [ 21.138300] RSP: 0018:ffffc900006f3d58 EFLAGS: 00010282 kern :warn : [ 21.138304] RAX: 0000000000000000 RBX: ffff8881451002e8 RCX: 0000000000000027 kern :warn : [ 21.138307] RDX: 0000000000000027 RSI: 0000000000000002 RDI: ffff888249397cf8 kern :warn : [ 21.138310] RBP: ffff888145100000 R08: ffff888249397cf0 R09: ffffc900006f3cf0 kern :warn : [ 21.138313] R10: 0000000000000001 R11: 3030302035313969 R12: ffff8881451007f0 kern :warn : [ 21.138316] R13: ffff888145106c60 R14: 0000000000000002 R15: 0000000000000000 kern :warn : [ 21.138319] FS: 0000000000000000(0000) GS:ffff888249380000(0000) knlGS:0000000000000000 kern :warn : [ 21.138322] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 kern :warn : [ 21.138325] CR2: 0000557c22ec68a0 CR3: 0000000004c0a003 CR4: 00000000003706e0 kern :warn : [ 21.138329] Call Trace: kern :warn : [ 21.138334] hsw_disable_lcpll+0x22/0x280 [i915] kern :warn : [ 21.138479] i915_drm_suspend_late+0x5e/0x100 [i915] kern :warn : [ 21.138578] ? pci_pm_poweroff_late+0x40/0x40 kern :warn : [ 21.138582] dpm_run_callback+0x4c/0x140 kern :warn : [ 21.138590] __device_suspend_late+0x98/0x1a0 kern :warn : [ 21.138596] async_suspend_late+0x1b/0xa0 kern :warn : [ 21.138602] async_run_entry_fn+0x39/0x160 kern :warn : [ 21.138608] process_one_work+0x1ed/0x3c0 kern :warn : [ 21.138612] worker_thread+0x50/0x3c0 kern :warn : [ 21.138617] ? process_one_work+0x3c0/0x3c0 kern :warn : [ 21.138620] kthread+0x116/0x160 kern :warn : [ 21.138626] ? kthread_park+0xa0/0xa0 kern :warn : [ 21.138631] ret_from_fork+0x22/0x30 kern :warn : [ 21.138640] ---[ end trace a93fe4d40a5a37bb ]--- To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml Thanks, Oliver Sang

1 year, 5 months

4
9
0 / 0

[vdpa_sim_net] 79991caf52: net/ipv4/ipmr.c:#RCU-list_traversed_in_non-reader_section

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 79991caf5202c7989928be534727805f8f68bb8d ("vdpa_sim_net: Add support for user supported devices") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git Dongli-Zhang/vhost-scsi-alloc-vhost_scsi-with-kvzalloc-to-avoid-delay/20210129-191605 in testcase: trinity version: trinity-static-x86_64-x86_64-f93256fb_2019-08-28 with following parameters: runtime: 300s test-description: Trinity is a linux system call fuzz tester. test-url: http://codemonkey.org.uk/projects/trinity/ on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-------------------------------------------------------------------------+------------+------------+ | | 39502d042a | 79991caf52 | +-------------------------------------------------------------------------+------------+------------+ | boot_successes | 0 | 0 | | boot_failures | 62 | 57 | | WARNING:suspicious_RCU_usage | 62 | 57 | | security/smack/smack_lsm.c:#RCU-list_traversed_in_non-reader_section | 62 | 57 | | security/smack/smack_access.c:#RCU-list_traversed_in_non-reader_section | 62 | 57 | | BUG:workqueue_lockup-pool | 33 | 40 | | BUG:kernel_hang_in_boot_stage | 6 | 2 | | net/mac80211/util.c:#RCU-list_traversed_in_non-reader_section | 23 | 15 | | WARNING:SOFTIRQ-safe->SOFTIRQ-unsafe_lock_order_detected | 18 | | | WARNING:inconsistent_lock_state | 5 | | | inconsistent{SOFTIRQ-ON-W}->{IN-SOFTIRQ-W}usage | 5 | | | calltrace:asm_call_irq_on_stack | 2 | | | RIP:lock_acquire | 2 | | | RIP:check_kcov_mode | 1 | | | RIP:native_safe_halt | 2 | | | INFO:rcu_sched_self-detected_stall_on_CPU | 2 | | | RIP:clear_page_rep | 1 | | | WARNING:at_drivers/gpu/drm/vkms/vkms_crtc.c:#vkms_vblank_simulate | 9 | 7 | | RIP:vkms_vblank_simulate | 9 | 7 | | RIP:__slab_alloc | 3 | 3 | | RIP:__do_softirq | 2 | | | RIP:console_unlock | 6 | 3 | | invoked_oom-killer:gfp_mask=0x | 1 | | | Mem-Info | 1 | | | RIP:vprintk_emit | 1 | | | RIP:__asan_load4 | 1 | | | kernel_BUG_at_kernel/sched/core.c | 0 | 1 | | invalid_opcode:#[##] | 0 | 1 | | RIP:sched_cpu_dying | 0 | 1 | | WARNING:possible_circular_locking_dependency_detected | 0 | 1 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 1 | | net/ipv4/ipmr.c:#RCU-list_traversed_in_non-reader_section | 0 | 8 | | RIP:arch_local_irq_restore | 0 | 1 | | RIP:idr_get_free | 0 | 1 | | net/ipv6/ip6mr.c:#RCU-list_traversed_in_non-reader_section | 0 | 2 | +-------------------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 890.196279] ============================= [ 890.212608] WARNING: suspicious RCU usage [ 890.228281] 5.11.0-rc4-00008-g79991caf5202 #1 Tainted: G W [ 890.244087] ----------------------------- [ 890.259417] net/ipv4/ipmr.c:138 RCU-list traversed in non-reader section!! [ 890.275043] [ 890.275043] other info that might help us debug this: [ 890.275043] [ 890.318497] [ 890.318497] rcu_scheduler_active = 2, debug_locks = 1 [ 890.346089] 2 locks held by trinity-c1/2476: [ 890.360897] #0: ffff888149d6f400 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xc0/0xe0 [ 890.375165] #1: ffff8881cabfd5c8 (&p->lock){+.+.}-{3:3}, at: seq_read_iter+0xa0/0x9c0 [ 890.389706] [ 890.389706] stack backtrace: [ 890.416375] CPU: 1 PID: 2476 Comm: trinity-c1 Tainted: G W 5.11.0-rc4-00008-g79991caf5202 #1 [ 890.430706] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 890.444971] Call Trace: [ 890.458554] dump_stack+0x15f/0x1bf [ 890.471996] ipmr_get_table+0x140/0x160 [ 890.485328] ipmr_vif_seq_start+0x4d/0xe0 [ 890.498620] seq_read_iter+0x1b2/0x9c0 [ 890.511469] ? kvm_sched_clock_read+0x14/0x40 [ 890.524008] ? sched_clock+0x1b/0x40 [ 890.536095] ? iov_iter_init+0x7c/0xa0 [ 890.548028] seq_read+0x2fd/0x3e0 [ 890.559948] ? seq_hlist_next_percpu+0x140/0x140 [ 890.572204] ? should_fail+0x78/0x2a0 [ 890.584189] ? write_comp_data+0x2a/0xa0 [ 890.596235] ? __sanitizer_cov_trace_pc+0x1d/0x60 [ 890.608134] ? seq_hlist_next_percpu+0x140/0x140 [ 890.620042] proc_reg_read+0x14e/0x180 [ 890.631585] do_iter_read+0x397/0x420 [ 890.642843] vfs_readv+0xf5/0x160 [ 890.653833] ? vfs_iter_read+0x80/0x80 [ 890.664229] ? __fdget_pos+0xc0/0xe0 [ 890.674236] ? pvclock_clocksource_read+0xd9/0x1a0 [ 890.684259] ? kvm_sched_clock_read+0x14/0x40 [ 890.693852] ? sched_clock+0x1b/0x40 [ 890.702898] ? sched_clock_cpu+0x18/0x120 [ 890.711648] ? write_comp_data+0x2a/0xa0 [ 890.720243] ? __sanitizer_cov_trace_pc+0x1d/0x60 [ 890.729290] do_readv+0x111/0x260 [ 890.738205] ? vfs_readv+0x160/0x160 [ 890.747154] ? lockdep_hardirqs_on+0x77/0x100 [ 890.756100] ? syscall_enter_from_user_mode+0x8a/0x100 [ 890.765126] do_syscall_64+0x34/0x80 [ 890.773795] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 890.782630] RIP: 0033:0x453b29 [ 890.791189] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 3b 84 00 00 c3 66 2e 0f 1f 84 00 00 00 00 [ 890.810866] RSP: 002b:00007ffcda44fb18 EFLAGS: 00000246 ORIG_RAX: 0000000000000013 [ 890.820764] RAX: ffffffffffffffda RBX: 0000000000000013 RCX: 0000000000453b29 [ 890.830792] RDX: 000000000000009a RSI: 0000000001de1c00 RDI: 00000000000000b9 [ 890.840626] RBP: 00007ffcda44fbc0 R08: 722c279d69ffc468 R09: 0000000000000400 [ 890.850366] R10: 0098d82a42c63c22 R11: 0000000000000246 R12: 0000000000000002 [ 890.860001] R13: 00007f042ae6f058 R14: 00000000010a2830 R15: 00007f042ae6f000 To reproduce: # build kernel cd linux cp config-5.11.0-rc4-00008-g79991caf5202 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Oliver Sang

1 year, 6 months

5
4
0 / 0

[locking/qspinlock] 0e8d8f4f12: stress-ng.zero.ops_per_sec -9.7% regression

by kernel test robot

Greeting, FYI, we noticed a -9.7% regression of stress-ng.zero.ops_per_sec due to commit: commit: 0e8d8f4f1214cfbac219d6917b5f6460f818bb7c ("[PATCH v13 3/6] locking/qspinlock: Introduce CNA into the slow path of qspinlock") url: https://github.com/0day-ci/linux/commits/Alex-Kogan/Add-NUMA-awareness-to... base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git cb262935a166bdef0ccfe6e2adffa00c0f2d038a in testcase: stress-ng on test machine: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 112G memory with following parameters: nr_threads: 100% disk: 1HDD testtime: 30s class: device cpufreq_governor: performance ucode: 0x42e In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------------------------------------+ | testcase: change | fsmark: fsmark.files_per_sec 213.9% improvement | | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cpufreq_governor=performance | | | disk=1BRD_48G | | | filesize=4M | | | fs=btrfs | | | iterations=1x | | | nr_threads=64t | | | sync_method=NoSync | | | test_size=24G | | | ucode=0x5003003 | +------------------+---------------------------------------------------------------------------+ | testcase: change | reaim: reaim.jobs_per_min 96.1% improvement | | test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cpufreq_governor=performance | | | nr_task=100% | | | runtime=300s | | | test=new_fserver | | | ucode=0x5003003 | +------------------+---------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/testcase/testtime/ucode: device/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-ivb-2ep1/stress-ng/30s/0x42e commit: cb45bab007 ("locking/qspinlock: Refactor the qspinlock slow path") 0e8d8f4f12 ("locking/qspinlock: Introduce CNA into the slow path of qspinlock") cb45bab007ff0cfc 0e8d8f4f1214cfbac219d6917b5 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 last_state.is_incomplete_run :4 25% 1:4 dmesg.Kernel_panic-not_syncing:softlockup:hung_tasks :4 25% 1:4 dmesg.RIP:smp_call_function_many_cond :4 25% 1:4 dmesg.RIP:smp_call_function_single %stddev %change %stddev \ | \ 481980 ± 9% -12.0% 424197 ± 5% stress-ng.time.involuntary_context_switches 1.198e+08 -4.0% 1.15e+08 stress-ng.time.minor_page_faults 689.06 -4.9% 655.29 stress-ng.time.user_time 10626516 -9.7% 9598124 stress-ng.zero.ops 354216 -9.7% 319937 stress-ng.zero.ops_per_sec 12871 +19.8% 15424 ± 12% meminfo.max_used_kB 29.00 +4.6% 30.33 vmstat.cpu.id 29.66 +3.7% 30.75 ± 2% iostat.cpu.idle 8.21 -7.6% 7.59 ± 2% iostat.cpu.user 67636 ± 12% -21.3% 53263 ± 14% sched_debug.cpu.nr_switches.max 70.29 ± 16% -22.1% 54.73 ± 14% sched_debug.cpu.nr_uninterruptible.stddev 7841 ± 58% +418.8% 40676 ± 2% numa-meminfo.node0.Active 6226 ± 71% +542.5% 40001 numa-meminfo.node0.Active(anon) 15043 ± 32% +271.6% 55899 numa-meminfo.node0.Shmem 77673 ± 3% -50.0% 38874 ± 9% numa-meminfo.node1.Active 77439 ± 3% -51.0% 37925 ± 8% numa-meminfo.node1.Active(anon) 233.75 ±157% +306.0% 949.00 ± 66% numa-meminfo.node1.Active(file) 575517 ± 4% -8.8% 525050 ± 3% numa-meminfo.node1.FilePages 87936 ± 4% -53.1% 41275 ± 9% numa-meminfo.node1.Shmem 2097 ± 5% +26.8% 2660 ± 21% slabinfo.dmaengine-unmap-16.active_objs 2097 ± 5% +26.8% 2660 ± 21% slabinfo.dmaengine-unmap-16.num_objs 40336 ± 6% +102.0% 81465 ± 50% slabinfo.filp.active_objs 657.00 ± 6% +98.0% 1301 ± 49% slabinfo.filp.active_slabs 42081 ± 6% +97.9% 83296 ± 49% slabinfo.filp.num_objs 657.00 ± 6% +98.0% 1301 ± 49% slabinfo.filp.num_slabs 13106 ± 3% +12.6% 14752 ± 3% slabinfo.shmem_inode_cache.active_objs 13224 ± 3% +12.5% 14873 ± 3% slabinfo.shmem_inode_cache.num_objs 1557 ± 71% +545.2% 10045 ± 2% numa-vmstat.node0.nr_active_anon 117.50 ± 57% -97.4% 3.00 ± 47% numa-vmstat.node0.nr_mlock 3761 ± 32% +273.2% 14037 numa-vmstat.node0.nr_shmem 1557 ± 71% +545.2% 10045 ± 2% numa-vmstat.node0.nr_zone_active_anon 19391 ± 3% -50.8% 9533 ± 7% numa-vmstat.node1.nr_active_anon 58.25 ±157% +301.1% 233.67 ± 67% numa-vmstat.node1.nr_active_file 143875 ± 3% -8.7% 131291 ± 3% numa-vmstat.node1.nr_file_pages 21981 ± 4% -52.9% 10346 ± 8% numa-vmstat.node1.nr_shmem 19391 ± 3% -50.8% 9533 ± 7% numa-vmstat.node1.nr_zone_active_anon 58.25 ±157% +301.1% 233.67 ± 67% numa-vmstat.node1.nr_zone_active_file 110768 ± 3% -10.6% 99008 ± 4% softirqs.CPU0.RCU 108763 ± 3% -10.1% 97774 ± 4% softirqs.CPU1.RCU 104199 ± 3% -9.5% 94314 ± 4% softirqs.CPU12.RCU 106156 ± 5% -11.6% 93873 ± 4% softirqs.CPU13.RCU 107231 ± 4% -11.3% 95088 ± 6% softirqs.CPU14.RCU 105547 ± 5% -10.5% 94461 ± 4% softirqs.CPU15.RCU 104326 ± 5% -7.4% 96653 ± 4% softirqs.CPU22.RCU 108301 ± 3% -9.9% 97571 ± 3% softirqs.CPU25.RCU 109423 ± 3% -10.9% 97448 ± 3% softirqs.CPU26.RCU 109554 ± 4% -7.2% 101691 ± 3% softirqs.CPU31.RCU 108410 ± 4% -11.2% 96233 ± 3% softirqs.CPU37.RCU 107189 ± 4% -9.8% 96642 ± 4% softirqs.CPU38.RCU 108943 ± 4% -10.9% 97052 ± 4% softirqs.CPU39.RCU 3348 ± 37% +214.8% 10541 ± 51% softirqs.NET_RX 20942 ± 2% -6.2% 19637 ± 4% proc-vmstat.nr_active_anon 67173 ± 4% -3.7% 64699 proc-vmstat.nr_anon_pages 10595 -1.7% 10418 proc-vmstat.nr_mapped 25771 ± 2% -5.1% 24461 ± 3% proc-vmstat.nr_shmem 38683 +5.7% 40878 ± 3% proc-vmstat.nr_slab_unreclaimable 20942 ± 2% -6.2% 19637 ± 4% proc-vmstat.nr_zone_active_anon 21010 ± 11% -23.8% 16020 ± 11% proc-vmstat.numa_hint_faults 16666 ± 11% -28.6% 11904 ± 7% proc-vmstat.numa_hint_faults_local 69020 ± 3% -3.7% 66499 proc-vmstat.numa_other 21276729 -9.4% 19266740 proc-vmstat.pgactivate 45217 +4.0% 47044 proc-vmstat.pgalloc_dma32 1.065e+08 -2.8% 1.035e+08 proc-vmstat.pgalloc_normal 610.50 ± 10% +83.9% 1123 ± 28% proc-vmstat.pgdeactivate 1.209e+08 -4.1% 1.159e+08 proc-vmstat.pgfault 1.064e+08 -2.7% 1.035e+08 proc-vmstat.pgfree 5.50 ± 20% +71439.4% 3934 ±141% proc-vmstat.pgmigrate_fail 3540 ± 29% +139.9% 8494 ± 7% proc-vmstat.pgrotated 10600738 -9.5% 9598142 proc-vmstat.unevictable_pgs_mlocked 10600295 -9.5% 9598142 proc-vmstat.unevictable_pgs_munlocked 0.05 ± 97% -59.4% 0.02 ± 2% perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown] 0.03 ± 6% +29749.0% 9.03 ± 71% perf-sched.sch_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 20.46 ± 3% -8.9% 18.64 ± 4% perf-sched.sch_delay.max.ms.wait_for_partner.fifo_open.do_dentry_open.path_openat 7526 ± 15% -25.7% 5592 ± 5% perf-sched.total_wait_and_delay.max.ms 7526 ± 15% -25.7% 5592 ± 5% perf-sched.total_wait_time.max.ms 9.67 ±168% -97.6% 0.24 ± 30% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 343.12 ±139% +236.4% 1154 ± 14% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter 1.39 ±173% +26107.3% 364.94 perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 5.59 ± 6% -20.2% 4.46 ± 6% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 1770 ± 6% +26.3% 2236 ± 7% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 18.50 ± 51% +272.3% 68.88 ± 22% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.remove_vma.__do_munmap.__vm_munmap 20.79 ± 26% +217.3% 65.97 ± 9% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.unmap_vmas.unmap_region.__do_munmap 242.02 ± 18% -60.5% 95.68 ± 51% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.12 ± 32% +266.9% 0.43 ± 78% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.alloc_pages_vma.do_anonymous_page 1971 ±123% -99.4% 11.34 ± 10% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.alloc_pages_vma.shmem_alloc_page 9.67 ±168% -97.6% 0.24 ± 30% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 348.74 ±136% +231.0% 1154 ± 14% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter 5.86 ± 4% +1071.7% 68.64 ± 19% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.khugepaged.kthread.ret_from_fork 38.63 ± 52% +843.9% 364.62 perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 5.57 ± 6% -20.3% 4.44 ± 6% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 1.57 ± 14% +683.9% 12.28 ±100% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.alloc_pages_vma.do_anonymous_page 2468 ±103% -99.5% 11.34 ± 10% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.alloc_pages_vma.shmem_alloc_page 3.91 ± 14% -41.4% 2.29 ± 6% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_read.__x64_sys_msync.do_syscall_64 7.68 ± 17% +1121.8% 93.81 ± 13% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.khugepaged.kthread.ret_from_fork 18.50 ± 51% +272.3% 68.88 ± 22% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.remove_vma.__do_munmap.__vm_munmap 20.79 ± 26% +217.3% 65.97 ± 9% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.unmap_vmas.unmap_region.__do_munmap 242.00 ± 18% -62.5% 90.66 ± 61% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 47358339 ± 7% -10.5% 42382902 ± 2% perf-stat.i.branch-misses 19845519 ± 2% -8.5% 18156709 ± 3% perf-stat.i.cache-misses 1.72e+08 -5.7% 1.621e+08 perf-stat.i.cache-references 725.90 ± 5% -13.2% 630.12 ± 5% perf-stat.i.cpu-migrations 3.09e+09 -4.3% 2.957e+09 perf-stat.i.dTLB-stores 19570792 -6.2% 18352176 perf-stat.i.iTLB-load-misses 19871 ± 4% +9.8% 21819 ± 3% perf-stat.i.instructions-per-iTLB-miss 382.70 -3.0% 371.36 ± 2% perf-stat.i.metric.M/sec 7111839 ± 4% -11.4% 6302363 ± 2% perf-stat.i.node-load-misses 8853002 ± 4% -9.6% 8004893 perf-stat.i.node-loads 6500622 -7.9% 5986359 ± 4% perf-stat.i.node-store-misses 8596412 -6.5% 8036645 ± 3% perf-stat.i.node-stores 5.60 ± 2% -3.9% 5.38 perf-stat.overall.MPKI 0.63 ± 7% -0.1 0.57 perf-stat.overall.branch-miss-rate% 5180 ± 2% +9.2% 5659 ± 3% perf-stat.overall.cycles-between-cache-misses 1575 +4.8% 1651 ± 3% perf-stat.overall.instructions-per-iTLB-miss 44.52 -0.5 43.98 perf-stat.overall.node-load-miss-rate% 43.02 -0.4 42.61 perf-stat.overall.node-store-miss-rate% 47096030 ± 7% -11.4% 41713545 ± 3% perf-stat.ps.branch-misses 19690067 ± 2% -9.4% 17836765 ± 3% perf-stat.ps.cache-misses 1.7e+08 ± 2% -6.9% 1.582e+08 ± 2% perf-stat.ps.cache-references 1.019e+11 -1.1% 1.008e+11 perf-stat.ps.cpu-cycles 748.16 ± 5% -11.3% 663.37 ± 5% perf-stat.ps.cpu-migrations 3.052e+09 -5.5% 2.885e+09 ± 2% perf-stat.ps.dTLB-stores 19264658 -7.6% 17802646 ± 2% perf-stat.ps.iTLB-load-misses 7041022 ± 4% -12.4% 6164936 perf-stat.ps.node-load-misses 8774703 ± 4% -10.5% 7851024 perf-stat.ps.node-loads 6428583 -9.0% 5851692 ± 4% perf-stat.ps.node-store-misses 8513354 -7.5% 7878662 ± 4% perf-stat.ps.node-stores 793.75 ± 6% -16.3% 664.33 ± 12% interrupts.22:IO-APIC.22-fasteoi.ehci_hcd:usb1 10846 ± 12% -22.7% 8387 ± 8% interrupts.CPU0.CAL:Function_call_interrupts 4222 ± 14% -49.6% 2128 ± 43% interrupts.CPU1.NMI:Non-maskable_interrupts 4222 ± 14% -49.6% 2128 ± 43% interrupts.CPU1.PMI:Performance_monitoring_interrupts 3219 ± 28% -21.0% 2544 ± 29% interrupts.CPU10.NMI:Non-maskable_interrupts 3219 ± 28% -21.0% 2544 ± 29% interrupts.CPU10.PMI:Performance_monitoring_interrupts 12387 ± 8% -10.7% 11065 ± 10% interrupts.CPU10.TLB:TLB_shootdowns 3883 ± 23% -35.3% 2511 ± 43% interrupts.CPU11.NMI:Non-maskable_interrupts 3883 ± 23% -35.3% 2511 ± 43% interrupts.CPU11.PMI:Performance_monitoring_interrupts 11453 ± 37% -34.1% 7553 ± 14% interrupts.CPU12.CAL:Function_call_interrupts 2074 ±126% -77.6% 464.00 ± 9% interrupts.CPU12.RES:Rescheduling_interrupts 13347 ± 59% -45.5% 7274 ± 10% interrupts.CPU12.TLB:TLB_shootdowns 11138 ± 23% -31.3% 7657 ± 13% interrupts.CPU13.CAL:Function_call_interrupts 3399 ± 34% -29.1% 2410 ± 50% interrupts.CPU15.NMI:Non-maskable_interrupts 3399 ± 34% -29.1% 2410 ± 50% interrupts.CPU15.PMI:Performance_monitoring_interrupts 3318 ± 29% -25.3% 2479 ± 34% interrupts.CPU16.NMI:Non-maskable_interrupts 3318 ± 29% -25.3% 2479 ± 34% interrupts.CPU16.PMI:Performance_monitoring_interrupts 4574 ± 17% -58.7% 1890 ± 29% interrupts.CPU20.NMI:Non-maskable_interrupts 4574 ± 17% -58.7% 1890 ± 29% interrupts.CPU20.PMI:Performance_monitoring_interrupts 2768 ± 37% -44.7% 1531 ± 30% interrupts.CPU22.NMI:Non-maskable_interrupts 2768 ± 37% -44.7% 1531 ± 30% interrupts.CPU22.PMI:Performance_monitoring_interrupts 516.00 ± 23% -18.9% 418.33 ± 5% interrupts.CPU22.RES:Rescheduling_interrupts 3326 ± 27% -28.2% 2387 ± 34% interrupts.CPU23.NMI:Non-maskable_interrupts 3326 ± 27% -28.2% 2387 ± 34% interrupts.CPU23.PMI:Performance_monitoring_interrupts 3850 ± 20% -39.2% 2340 ± 50% interrupts.CPU25.NMI:Non-maskable_interrupts 3850 ± 20% -39.2% 2340 ± 50% interrupts.CPU25.PMI:Performance_monitoring_interrupts 859.25 ± 48% -46.0% 464.33 ± 5% interrupts.CPU25.RES:Rescheduling_interrupts 13154 ± 35% -25.2% 9841 ± 14% interrupts.CPU25.TLB:TLB_shootdowns 3711 ± 23% -45.6% 2018 ± 45% interrupts.CPU26.NMI:Non-maskable_interrupts 3711 ± 23% -45.6% 2018 ± 45% interrupts.CPU26.PMI:Performance_monitoring_interrupts 4507 ± 4% -43.3% 2556 ± 40% interrupts.CPU27.NMI:Non-maskable_interrupts 4507 ± 4% -43.3% 2556 ± 40% interrupts.CPU27.PMI:Performance_monitoring_interrupts 11363 ± 9% -14.9% 9671 ± 8% interrupts.CPU27.TLB:TLB_shootdowns 4430 ± 5% -51.0% 2172 ± 43% interrupts.CPU28.NMI:Non-maskable_interrupts 4430 ± 5% -51.0% 2172 ± 43% interrupts.CPU28.PMI:Performance_monitoring_interrupts 9512 ± 3% -16.7% 7921 ± 8% interrupts.CPU29.CAL:Function_call_interrupts 3914 ± 14% -59.0% 1606 ± 27% interrupts.CPU29.NMI:Non-maskable_interrupts 3914 ± 14% -59.0% 1606 ± 27% interrupts.CPU29.PMI:Performance_monitoring_interrupts 998.00 ± 77% -55.5% 444.00 ± 2% interrupts.CPU29.RES:Rescheduling_interrupts 11508 ± 12% -22.8% 8881 ± 10% interrupts.CPU29.TLB:TLB_shootdowns 3287 ± 22% -36.6% 2082 ± 43% interrupts.CPU30.NMI:Non-maskable_interrupts 3287 ± 22% -36.6% 2082 ± 43% interrupts.CPU30.PMI:Performance_monitoring_interrupts 571.75 ± 12% -21.0% 451.67 ± 10% interrupts.CPU30.RES:Rescheduling_interrupts 10554 ± 20% -26.5% 7757 ± 6% interrupts.CPU31.CAL:Function_call_interrupts 3521 ± 21% -47.8% 1838 ± 9% interrupts.CPU31.NMI:Non-maskable_interrupts 3521 ± 21% -47.8% 1838 ± 9% interrupts.CPU31.PMI:Performance_monitoring_interrupts 604.00 ± 20% -26.0% 447.00 ± 14% interrupts.CPU31.RES:Rescheduling_interrupts 3946 ± 21% -51.9% 1898 ± 6% interrupts.CPU32.NMI:Non-maskable_interrupts 3946 ± 21% -51.9% 1898 ± 6% interrupts.CPU32.PMI:Performance_monitoring_interrupts 2221 ± 74% -74.2% 573.00 ± 13% interrupts.CPU32.RES:Rescheduling_interrupts 2969 ± 22% -40.2% 1774 ± 11% interrupts.CPU33.NMI:Non-maskable_interrupts 2969 ± 22% -40.2% 1774 ± 11% interrupts.CPU33.PMI:Performance_monitoring_interrupts 13653 ± 16% -40.8% 8079 ± 6% interrupts.CPU34.CAL:Function_call_interrupts 3074 ± 20% -41.7% 1791 ± 18% interrupts.CPU34.NMI:Non-maskable_interrupts 3074 ± 20% -41.7% 1791 ± 18% interrupts.CPU34.PMI:Performance_monitoring_interrupts 18328 ± 31% -48.1% 9519 ± 7% interrupts.CPU34.TLB:TLB_shootdowns 11937 ± 21% -36.1% 7623 ± 15% interrupts.CPU35.CAL:Function_call_interrupts 3503 ± 30% -36.4% 2229 ± 12% interrupts.CPU35.NMI:Non-maskable_interrupts 3503 ± 30% -36.4% 2229 ± 12% interrupts.CPU35.PMI:Performance_monitoring_interrupts 16203 ± 41% -47.9% 8444 ± 15% interrupts.CPU35.TLB:TLB_shootdowns 4311 ± 20% -43.4% 2440 ± 50% interrupts.CPU37.NMI:Non-maskable_interrupts 4311 ± 20% -43.4% 2440 ± 50% interrupts.CPU37.PMI:Performance_monitoring_interrupts 1028 ± 98% -60.6% 404.67 ± 5% interrupts.CPU37.RES:Rescheduling_interrupts 11376 ± 22% -35.6% 7324 ± 2% interrupts.CPU37.TLB:TLB_shootdowns 3390 ± 35% -35.8% 2177 ± 50% interrupts.CPU38.NMI:Non-maskable_interrupts 3390 ± 35% -35.8% 2177 ± 50% interrupts.CPU38.PMI:Performance_monitoring_interrupts 2077 ±133% -80.7% 401.33 ± 12% interrupts.CPU38.RES:Rescheduling_interrupts 3760 ± 19% -54.3% 1716 ± 66% interrupts.CPU39.NMI:Non-maskable_interrupts 3760 ± 19% -54.3% 1716 ± 66% interrupts.CPU39.PMI:Performance_monitoring_interrupts 3338 ± 68% -85.1% 496.67 ± 17% interrupts.CPU4.RES:Rescheduling_interrupts 3731 ± 27% -45.6% 2030 ± 38% interrupts.CPU40.NMI:Non-maskable_interrupts 3731 ± 27% -45.6% 2030 ± 38% interrupts.CPU40.PMI:Performance_monitoring_interrupts 4265 ± 5% -43.5% 2409 ± 45% interrupts.CPU42.NMI:Non-maskable_interrupts 4265 ± 5% -43.5% 2409 ± 45% interrupts.CPU42.PMI:Performance_monitoring_interrupts 3425 ± 18% -41.4% 2007 ± 51% interrupts.CPU43.NMI:Non-maskable_interrupts 3425 ± 18% -41.4% 2007 ± 51% interrupts.CPU43.PMI:Performance_monitoring_interrupts 627.00 ± 46% -43.4% 354.67 ± 2% interrupts.CPU43.RES:Rescheduling_interrupts 4679 ± 13% -71.6% 1331 ± 21% interrupts.CPU44.NMI:Non-maskable_interrupts 4679 ± 13% -71.6% 1331 ± 21% interrupts.CPU44.PMI:Performance_monitoring_interrupts 494.50 ± 18% -25.8% 367.00 ± 15% interrupts.CPU44.RES:Rescheduling_interrupts 3975 ± 18% -43.4% 2249 ± 46% interrupts.CPU45.NMI:Non-maskable_interrupts 3975 ± 18% -43.4% 2249 ± 46% interrupts.CPU45.PMI:Performance_monitoring_interrupts 8711 ± 9% -11.6% 7705 interrupts.CPU46.CAL:Function_call_interrupts 3650 ± 22% -44.9% 2010 ± 55% interrupts.CPU46.NMI:Non-maskable_interrupts 3650 ± 22% -44.9% 2010 ± 55% interrupts.CPU46.PMI:Performance_monitoring_interrupts 440.75 ± 9% -16.7% 367.00 ± 7% interrupts.CPU46.RES:Rescheduling_interrupts 7891 ± 10% -25.7% 5860 ± 11% interrupts.CPU47.CAL:Function_call_interrupts 793.00 ± 40% -50.6% 392.00 ± 3% interrupts.CPU47.RES:Rescheduling_interrupts 2663 ± 23% -21.6% 2087 ± 7% interrupts.CPU5.NMI:Non-maskable_interrupts 2663 ± 23% -21.6% 2087 ± 7% interrupts.CPU5.PMI:Performance_monitoring_interrupts 12129 ± 30% -32.7% 8164 ± 2% interrupts.CPU7.CAL:Function_call_interrupts 14622 ± 38% -33.7% 9701 ± 6% interrupts.CPU7.TLB:TLB_shootdowns 793.75 ± 6% -16.3% 664.33 ± 12% interrupts.CPU8.22:IO-APIC.22-fasteoi.ehci_hcd:usb1 2516 ± 7% +26.7% 3188 ± 15% interrupts.CPU8.NMI:Non-maskable_interrupts 2516 ± 7% +26.7% 3188 ± 15% interrupts.CPU8.PMI:Performance_monitoring_interrupts 1784 ± 54% -58.8% 735.00 ± 4% interrupts.CPU8.RES:Rescheduling_interrupts 656.75 ± 18% +51.0% 992.00 ± 20% interrupts.CPU9.RES:Rescheduling_interrupts 168231 ± 7% -33.4% 112034 ± 19% interrupts.NMI:Non-maskable_interrupts 168231 ± 7% -33.4% 112034 ± 19% interrupts.PMI:Performance_monitoring_interrupts 7.01 ± 14% -3.4 3.60 ± 50% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 4.21 ± 21% -1.9 2.29 ± 63% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 2.84 ± 12% -1.9 0.96 ± 39% perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.06 ± 21% -1.8 2.23 ± 65% perf-profile.calltrace.cycles-pp.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 2.64 ± 13% -1.8 0.87 ± 40% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.84 ± 22% -1.7 2.13 ± 67% perf-profile.calltrace.cycles-pp.__fput.task_work_run.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 1.49 ± 7% -0.9 0.54 ± 70% perf-profile.calltrace.cycles-pp.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2.do_sys_open 1.46 ± 8% -0.9 0.53 ± 70% perf-profile.calltrace.cycles-pp.__alloc_file.alloc_empty_file.path_openat.do_filp_open.do_sys_openat2 1.28 ± 5% -0.8 0.51 ± 71% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 1.06 ± 6% -0.6 0.42 ± 71% perf-profile.calltrace.cycles-pp.__entry_text_start 0.77 ± 13% +0.3 1.05 ± 11% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 0.82 ± 12% +0.3 1.12 ± 11% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 1.23 ± 13% +0.4 1.68 ± 18% perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock 1.23 ± 13% +0.4 1.68 ± 18% perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit 1.82 ± 13% +0.6 2.39 ± 17% perf-profile.calltrace.cycles-pp.create_basic_memory_bitmaps.snapshot_open.misc_open.chrdev_open.do_dentry_open 0.00 +0.6 0.61 ± 12% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.00 +0.6 0.62 ± 12% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 4.96 ± 7% +1.7 6.69 ± 19% perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.misc_open.chrdev_open.do_dentry_open 7.19 ± 14% -3.4 3.78 ± 45% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 3.03 ± 53% -2.8 0.24 ± 21% perf-profile.children.cycles-pp.link_path_walk 4.28 ± 21% -1.9 2.35 ± 61% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 2.85 ± 12% -1.9 0.96 ± 40% perf-profile.children.cycles-pp.ksys_mmap_pgoff 4.26 ± 21% -1.9 2.38 ± 62% perf-profile.children.cycles-pp.task_work_run 2.65 ± 13% -1.8 0.88 ± 40% perf-profile.children.cycles-pp.vm_mmap_pgoff 4.04 ± 22% -1.8 2.27 ± 64% perf-profile.children.cycles-pp.__fput 1.69 ± 54% -1.6 0.13 ± 22% perf-profile.children.cycles-pp.inode_permission 1.47 ± 51% -1.3 0.16 ± 3% perf-profile.children.cycles-pp.walk_component 1.33 ± 54% -1.2 0.11 ± 14% perf-profile.children.cycles-pp.lookup_fast 1.49 ± 27% -1.1 0.36 ± 28% perf-profile.children.cycles-pp._raw_spin_lock 1.49 ± 31% -1.1 0.40 ± 48% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 1.51 ± 21% -1.1 0.44 ± 53% perf-profile.children.cycles-pp.down_write_killable 0.91 ± 62% -0.9 0.04 ± 71% perf-profile.children.cycles-pp.hidraw_release 1.58 ± 8% -0.8 0.73 ± 24% perf-profile.children.cycles-pp.alloc_empty_file 1.55 ± 9% -0.8 0.72 ± 24% perf-profile.children.cycles-pp.__alloc_file 1.49 ± 5% -0.7 0.74 ± 28% perf-profile.children.cycles-pp.syscall_return_via_sysret 1.26 ± 6% -0.6 0.63 ± 27% perf-profile.children.cycles-pp.__entry_text_start 1.12 ± 9% -0.6 0.55 ± 24% perf-profile.children.cycles-pp.kmem_cache_alloc 0.87 ± 18% -0.6 0.31 ± 34% perf-profile.children.cycles-pp.do_mmap 0.63 ± 7% -0.4 0.22 ± 25% perf-profile.children.cycles-pp.dput 0.55 ± 17% -0.4 0.15 ± 34% perf-profile.children.cycles-pp.security_file_open 0.53 ± 19% -0.4 0.14 ± 36% perf-profile.children.cycles-pp.apparmor_file_open 0.52 ± 32% -0.4 0.14 ± 55% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.48 ± 22% -0.4 0.12 ± 13% perf-profile.children.cycles-pp.ima_file_check 0.65 ± 7% -0.4 0.29 ± 35% perf-profile.children.cycles-pp.__x64_sys_select 0.47 ± 23% -0.4 0.11 ± 14% perf-profile.children.cycles-pp.security_task_getsecid 0.64 ± 7% -0.4 0.29 ± 35% perf-profile.children.cycles-pp.kern_select 0.62 ± 13% -0.3 0.27 ± 36% perf-profile.children.cycles-pp.__x64_sys_exit_group 0.62 ± 12% -0.3 0.28 ± 33% perf-profile.children.cycles-pp.do_group_exit 0.62 ± 12% -0.3 0.28 ± 33% perf-profile.children.cycles-pp.do_exit 0.45 ± 24% -0.3 0.11 ± 19% perf-profile.children.cycles-pp.apparmor_task_getsecid 0.49 ± 65% -0.3 0.16 ± 83% perf-profile.children.cycles-pp.__x64_sys_munmap 0.49 ± 64% -0.3 0.16 ± 83% perf-profile.children.cycles-pp.__vm_munmap 0.48 ± 5% -0.3 0.16 ± 34% perf-profile.children.cycles-pp.lockref_put_or_lock 0.51 ± 9% -0.3 0.19 ± 15% perf-profile.children.cycles-pp.security_file_alloc 0.51 ± 7% -0.3 0.20 ± 38% perf-profile.children.cycles-pp.__x64_sys_close 0.59 ± 12% -0.3 0.30 ± 13% perf-profile.children.cycles-pp.__do_sys_clone 0.59 ± 12% -0.3 0.30 ± 13% perf-profile.children.cycles-pp.kernel_clone 0.41 ± 11% -0.3 0.13 ± 18% perf-profile.children.cycles-pp.apparmor_file_alloc_security 0.57 ± 12% -0.3 0.29 ± 12% perf-profile.children.cycles-pp.copy_process 0.51 ± 8% -0.3 0.23 ± 38% perf-profile.children.cycles-pp.core_sys_select 0.43 ± 14% -0.3 0.17 ± 14% perf-profile.children.cycles-pp.mmput 0.43 ± 14% -0.3 0.17 ± 14% perf-profile.children.cycles-pp.exit_mmap 0.45 ± 8% -0.3 0.19 ± 35% perf-profile.children.cycles-pp.get_unmapped_area 0.40 ± 17% -0.3 0.14 ± 20% perf-profile.children.cycles-pp.__do_sys_newfstat 0.50 ± 13% -0.2 0.26 ± 12% perf-profile.children.cycles-pp.dup_mm 0.71 ± 9% -0.2 0.48 ± 11% perf-profile.children.cycles-pp.do_softirq_own_stack 0.75 ± 10% -0.2 0.52 ± 11% perf-profile.children.cycles-pp.irq_exit_rcu 0.49 ± 12% -0.2 0.26 ± 12% perf-profile.children.cycles-pp.dup_mmap 0.41 ± 9% -0.2 0.19 ± 36% perf-profile.children.cycles-pp.__fget_files 0.28 ± 52% -0.2 0.06 ± 75% perf-profile.children.cycles-pp.mmap_region 0.35 ± 5% -0.2 0.14 ± 34% perf-profile.children.cycles-pp.security_mmap_file 0.39 ± 7% -0.2 0.18 ± 15% perf-profile.children.cycles-pp.getname_flags 0.32 ± 7% -0.2 0.12 ± 35% perf-profile.children.cycles-pp.complete_walk 0.32 ± 7% -0.2 0.12 ± 31% perf-profile.children.cycles-pp.unlazy_walk 0.29 ± 22% -0.2 0.09 ± 27% perf-profile.children.cycles-pp.vfs_fstat 0.30 ± 17% -0.2 0.10 ± 12% perf-profile.children.cycles-pp.security_file_free 0.28 ± 21% -0.2 0.08 ± 14% perf-profile.children.cycles-pp.aa_get_task_label 0.29 ± 17% -0.2 0.09 ± 14% perf-profile.children.cycles-pp.apparmor_file_free_security 0.25 ± 32% -0.2 0.06 ± 13% perf-profile.children.cycles-pp.step_into 0.36 ± 10% -0.2 0.17 ± 39% perf-profile.children.cycles-pp.do_select 0.33 ± 7% -0.2 0.14 ± 34% perf-profile.children.cycles-pp.arch_get_unmapped_area_topdown 0.29 ± 7% -0.2 0.10 ± 31% perf-profile.children.cycles-pp.__legitimize_path 0.36 ± 17% -0.2 0.17 ± 7% perf-profile.children.cycles-pp.__schedule 0.29 ± 8% -0.2 0.11 ± 40% perf-profile.children.cycles-pp.terminate_walk 0.34 ± 8% -0.2 0.17 ± 34% perf-profile.children.cycles-pp.__fget_light 0.25 ± 25% -0.2 0.08 ± 24% perf-profile.children.cycles-pp.may_open 0.25 ± 18% -0.2 0.08 ± 35% perf-profile.children.cycles-pp.mutex_lock 0.35 ± 7% -0.2 0.18 ± 50% perf-profile.children.cycles-pp.__alloc_pages_nodemask 0.39 ± 13% -0.2 0.22 ± 24% perf-profile.children.cycles-pp.ksys_lseek 0.25 ± 58% -0.2 0.09 ± 84% perf-profile.children.cycles-pp.__do_munmap 0.22 ± 8% -0.2 0.06 ± 71% perf-profile.children.cycles-pp.lockref_get_not_dead 0.21 ± 17% -0.2 0.06 ± 75% perf-profile.children.cycles-pp.cdev_put 0.25 ± 10% -0.2 0.10 ± 38% perf-profile.children.cycles-pp.__close_fd 0.24 ± 10% -0.1 0.10 ± 38% perf-profile.children.cycles-pp.pick_file 0.37 ± 10% -0.1 0.23 ± 12% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.27 ± 9% -0.1 0.13 ± 33% perf-profile.children.cycles-pp.__alloc_fd 0.21 ± 23% -0.1 0.07 ± 20% perf-profile.children.cycles-pp.free_pgtables 0.27 ± 20% -0.1 0.13 ± 9% perf-profile.children.cycles-pp.schedule 0.26 ± 7% -0.1 0.12 ± 18% perf-profile.children.cycles-pp.strncpy_from_user 0.23 ± 10% -0.1 0.09 ± 39% perf-profile.children.cycles-pp.filp_close 0.20 ± 11% -0.1 0.07 ± 71% perf-profile.children.cycles-pp.vm_unmapped_area 0.25 ± 4% -0.1 0.12 ± 29% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 0.17 ± 16% -0.1 0.04 ± 76% perf-profile.children.cycles-pp.kobject_put 0.19 ± 19% -0.1 0.06 ± 29% perf-profile.children.cycles-pp.copy_page_range 0.22 ± 11% -0.1 0.10 ± 14% perf-profile.children.cycles-pp.try_to_wake_up 0.21 ± 9% -0.1 0.08 ± 34% perf-profile.children.cycles-pp.common_file_perm 0.22 ± 10% -0.1 0.10 ± 32% perf-profile.children.cycles-pp.fput_many 0.29 ± 21% -0.1 0.17 ± 47% perf-profile.children.cycles-pp.drm_release 0.29 ± 21% -0.1 0.17 ± 47% perf-profile.children.cycles-pp.drm_client_dev_restore 0.29 ± 21% -0.1 0.17 ± 47% perf-profile.children.cycles-pp.drm_fbdev_client_restore 0.29 ± 21% -0.1 0.17 ± 47% perf-profile.children.cycles-pp.__drm_fb_helper_restore_fbdev_mode_unlocked 0.29 ± 21% -0.1 0.17 ± 47% perf-profile.children.cycles-pp.drm_client_modeset_commit 0.29 ± 21% -0.1 0.17 ± 47% perf-profile.children.cycles-pp.drm_client_modeset_commit_locked 0.29 ± 21% -0.1 0.17 ± 47% perf-profile.children.cycles-pp.drm_client_modeset_commit_atomic 0.28 ± 4% -0.1 0.16 ± 48% perf-profile.children.cycles-pp.get_page_from_freelist 0.20 ± 15% -0.1 0.08 ± 30% perf-profile.children.cycles-pp.unmap_vmas 0.16 ± 21% -0.1 0.04 ± 76% perf-profile.children.cycles-pp.copy_pte_range 0.18 ± 9% -0.1 0.07 ± 71% perf-profile.children.cycles-pp.__x64_sys_fcntl 0.22 ± 7% -0.1 0.10 ± 36% perf-profile.children.cycles-pp.__x64_sys_poll 0.22 ± 7% -0.1 0.10 ± 9% perf-profile.children.cycles-pp.__check_object_size 0.18 ± 12% -0.1 0.07 ± 25% perf-profile.children.cycles-pp.unmap_page_range 0.20 ± 8% -0.1 0.09 ± 33% perf-profile.children.cycles-pp.do_sys_poll 0.15 ± 7% -0.1 0.05 ± 72% perf-profile.children.cycles-pp.__might_sleep 0.15 ± 8% -0.1 0.05 ± 78% perf-profile.children.cycles-pp.obj_cgroup_charge 0.15 ± 9% -0.1 0.06 ± 16% perf-profile.children.cycles-pp.zap_pte_range 0.17 ± 20% -0.1 0.07 ± 72% perf-profile.children.cycles-pp.__slab_alloc 0.16 ± 22% -0.1 0.07 ± 72% perf-profile.children.cycles-pp.___slab_alloc 0.14 ± 19% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.unlink_anon_vmas 0.17 ± 38% -0.1 0.07 ± 74% perf-profile.children.cycles-pp.block_ioctl 0.16 ± 9% -0.1 0.07 ± 20% perf-profile.children.cycles-pp.anon_vma_fork 0.18 ± 5% -0.1 0.09 ± 18% perf-profile.children.cycles-pp.___might_sleep 0.14 ± 32% -0.1 0.05 ± 81% perf-profile.children.cycles-pp.drm_stub_open 0.14 ± 32% -0.1 0.05 ± 81% perf-profile.children.cycles-pp.drm_open 0.14 ± 32% -0.1 0.05 ± 81% perf-profile.children.cycles-pp.drm_master_open 0.16 ± 9% -0.1 0.07 ± 23% perf-profile.children.cycles-pp.get_obj_cgroup_from_current 0.16 ± 38% -0.1 0.07 ± 74% perf-profile.children.cycles-pp.blkdev_ioctl 0.12 ± 12% -0.1 0.04 ± 71% perf-profile.children.cycles-pp._cond_resched 0.12 ± 8% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.memset_erms 0.14 ± 25% -0.1 0.06 ± 72% perf-profile.children.cycles-pp.allocate_slab 0.12 ± 13% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.call_rcu 0.17 ± 25% -0.1 0.09 ± 50% perf-profile.children.cycles-pp.path_lookupat 0.17 ± 30% -0.1 0.10 ± 61% perf-profile.children.cycles-pp.__fxstat64 0.11 ± 12% -0.1 0.03 ± 70% perf-profile.children.cycles-pp.anon_vma_clone 0.11 ± 10% -0.1 0.03 ± 70% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.15 ± 16% -0.1 0.08 ± 10% perf-profile.children.cycles-pp.pick_next_task_fair 0.14 ± 6% -0.1 0.07 ± 11% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 0.10 ± 12% -0.1 0.03 ± 70% perf-profile.children.cycles-pp.cp_new_stat 0.13 ± 10% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.__d_lookup_rcu 0.10 ± 15% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.generic_permission 0.14 ± 9% -0.1 0.08 ± 20% perf-profile.children.cycles-pp.__x86_retpoline_rax 0.08 ± 5% -0.1 0.03 ± 70% perf-profile.children.cycles-pp.ktime_get_ts64 0.09 ± 4% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.__might_fault 0.07 ± 17% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.schedule_timeout 0.11 ± 9% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.update_load_avg 0.09 ± 8% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.newidle_balance 0.10 ± 14% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.task_tick_fair 0.06 ± 15% +0.0 0.08 ± 16% perf-profile.children.cycles-pp.tick_irq_enter 0.04 ± 57% +0.0 0.06 perf-profile.children.cycles-pp.sched_clock_cpu 0.08 ± 13% +0.0 0.11 ± 4% perf-profile.children.cycles-pp.irq_enter_rcu 0.07 ± 5% +0.0 0.12 ± 24% perf-profile.children.cycles-pp.clockevents_program_event 0.01 ±173% +0.1 0.07 ± 18% perf-profile.children.cycles-pp.rebalance_domains 0.09 ± 4% +0.1 0.15 ± 26% perf-profile.children.cycles-pp.ktime_get 0.16 ± 9% +0.1 0.23 ± 7% perf-profile.children.cycles-pp.menu_select 0.21 ± 11% +0.1 0.28 ± 16% perf-profile.children.cycles-pp.io_serial_out 0.15 ± 20% +0.1 0.23 ± 6% perf-profile.children.cycles-pp.delay_tsc 1.83 ± 14% +0.5 2.38 ± 12% perf-profile.children.cycles-pp.serial8250_console_putchar 1.85 ± 14% +0.6 2.40 ± 12% perf-profile.children.cycles-pp.wait_for_xmitr 1.99 ± 15% +0.6 2.56 ± 16% perf-profile.children.cycles-pp.__dynamic_pr_debug 1.82 ± 13% +0.6 2.39 ± 17% perf-profile.children.cycles-pp.create_basic_memory_bitmaps 2.13 ± 13% +0.6 2.74 ± 13% perf-profile.children.cycles-pp.console_unlock 2.03 ± 13% +0.6 2.65 ± 12% perf-profile.children.cycles-pp.uart_console_write 2.17 ± 15% +0.6 2.79 ± 14% perf-profile.children.cycles-pp.printk 2.17 ± 15% +0.6 2.79 ± 14% perf-profile.children.cycles-pp.vprintk_emit 2.05 ± 13% +0.6 2.67 ± 12% perf-profile.children.cycles-pp.serial8250_console_write 0.00 +3.7 3.73 ± 66% perf-profile.children.cycles-pp.__cna_queued_spin_lock_slowpath 2.92 ± 6% -1.5 1.44 ± 22% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 1.48 ± 5% -0.7 0.74 ± 28% perf-profile.self.cycles-pp.syscall_return_via_sysret 1.25 ± 6% -0.6 0.62 ± 28% perf-profile.self.cycles-pp.__entry_text_start 0.52 ± 18% -0.4 0.13 ± 34% perf-profile.self.cycles-pp.apparmor_file_open 0.50 ± 32% -0.4 0.13 ± 54% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.44 ± 5% -0.3 0.13 ± 36% perf-profile.self.cycles-pp.lockref_put_or_lock 0.38 ± 12% -0.3 0.12 ± 18% perf-profile.self.cycles-pp.apparmor_file_alloc_security 0.52 ± 4% -0.3 0.27 ± 25% perf-profile.self.cycles-pp._raw_spin_lock 0.28 ± 22% -0.2 0.05 ± 72% perf-profile.self.cycles-pp.aa_get_task_label 0.45 ± 7% -0.2 0.24 ± 20% perf-profile.self.cycles-pp.kmem_cache_alloc 0.39 ± 10% -0.2 0.19 ± 36% perf-profile.self.cycles-pp.__fget_files 0.28 ± 19% -0.2 0.08 ± 11% perf-profile.self.cycles-pp.apparmor_file_free_security 0.30 ± 8% -0.2 0.11 ± 36% perf-profile.self.cycles-pp.do_dentry_open 0.21 ± 21% -0.2 0.04 ± 76% perf-profile.self.cycles-pp.mutex_lock 0.20 ± 3% -0.1 0.06 ± 71% perf-profile.self.cycles-pp.lockref_get_not_dead 0.20 ± 11% -0.1 0.06 ± 73% perf-profile.self.cycles-pp.vm_unmapped_area 0.18 ± 8% -0.1 0.06 ± 73% perf-profile.self.cycles-pp.common_file_perm 0.17 ± 14% -0.1 0.04 ± 76% perf-profile.self.cycles-pp.kobject_put 0.27 ± 15% -0.1 0.15 ± 25% perf-profile.self.cycles-pp.__alloc_file 0.16 ± 39% -0.1 0.05 ± 78% perf-profile.self.cycles-pp.rwsem_down_write_slowpath 0.16 ± 11% -0.1 0.05 ± 77% perf-profile.self.cycles-pp.fput_many 0.14 ± 7% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.__might_sleep 0.18 ± 5% -0.1 0.09 ± 30% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.14 ± 11% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.do_sys_openat2 0.15 ± 5% -0.1 0.06 ± 73% perf-profile.self.cycles-pp.__fput 0.20 ± 5% -0.1 0.11 ± 31% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.13 ± 14% -0.1 0.04 ± 70% perf-profile.self.cycles-pp.link_path_walk 0.18 ± 8% -0.1 0.09 ± 18% perf-profile.self.cycles-pp.___might_sleep 0.13 ± 14% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.get_obj_cgroup_from_current 0.12 ± 6% -0.1 0.04 ± 73% perf-profile.self.cycles-pp.do_mmap 0.12 ± 10% -0.1 0.05 ± 71% perf-profile.self.cycles-pp.__x86_retpoline_rax 0.12 ± 24% -0.1 0.05 ± 72% perf-profile.self.cycles-pp.allocate_slab 0.12 ± 8% -0.1 0.06 ± 19% perf-profile.self.cycles-pp.__d_lookup_rcu 0.10 ± 13% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.generic_permission 0.09 ± 8% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.10 ± 4% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 0.07 ± 17% +0.0 0.11 ± 8% perf-profile.self.cycles-pp.cpuidle_enter_state 0.10 ± 9% +0.0 0.14 ± 10% perf-profile.self.cycles-pp.menu_select 0.03 ±100% +0.1 0.09 ± 36% perf-profile.self.cycles-pp.ktime_get 0.15 ± 20% +0.1 0.23 ± 6% perf-profile.self.cycles-pp.delay_tsc 0.20 ± 16% +0.1 0.28 ± 16% perf-profile.self.cycles-pp.io_serial_out 0.00 +3.7 3.73 ± 66% perf-profile.self.cycles-pp.__cna_queued_spin_lock_slowpath stress-ng.zero.ops_per_sec 400000 +------------------------------------------------------------------+ |.+.. .+.. .+..+. .+.+.. +.. | 350000 |-+ + + +.+..+.+.. .+..+ O +. O O +.+..O+O +.+..+.+..+.| | O : O : O O + O O + O O O | 300000 |-+ : : O | 250000 |-+ : : | | : : | 200000 |-+ : : | | : : | 150000 |-+ : : | 100000 |-+ : : | | :: | 50000 |-+ :: | | : | 0 +------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-csl-2ap2: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory ========================================================================================= compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase/ucode: gcc-9/performance/1BRD_48G/4M/btrfs/1x/x86_64-rhel-8.3/64t/debian-10.4-x86_64-20200603.cgz/NoSync/lkp-csl-2ap2/24G/fsmark/0x5003003 commit: cb45bab007 ("locking/qspinlock: Refactor the qspinlock slow path") 0e8d8f4f12 ("locking/qspinlock: Introduce CNA into the slow path of qspinlock") cb45bab007ff0cfc 0e8d8f4f1214cfbac219d6917b5 ---------------- --------------------------- %stddev %change %stddev \ | \ 1180 ± 2% +213.9% 3706 ± 11% fsmark.files_per_sec 773.00 ± 4% -30.2% 539.75 ± 7% fsmark.time.percent_of_cpu_this_job_got 137.19 ± 3% -28.5% 98.13 ± 8% fsmark.time.system_time 48917 ± 4% -9.0% 44537 ± 8% meminfo.AnonHugePages 4.83 ± 4% -1.0 3.83 ± 5% mpstat.cpu.all.sys% 93.48 +1.1% 94.51 iostat.cpu.idle 5.76 ± 2% -16.9% 4.79 ± 6% iostat.cpu.system 93.00 +1.1% 94.00 vmstat.cpu.id 20606 ± 5% +97.4% 40682 ± 34% vmstat.system.cs 4237881 ± 29% +321.3% 17854499 ±106% cpuidle.C1.time 44169 ± 32% +396.5% 219318 ±113% cpuidle.C1.usage 72208 ± 6% +191.8% 210719 ± 39% cpuidle.POLL.time 36708 ± 9% +306.0% 149034 ± 48% cpuidle.POLL.usage 2157 ± 2% +13.8% 2454 ± 6% slabinfo.biovec-max.active_objs 2271 ± 2% +12.8% 2561 ± 5% slabinfo.biovec-max.num_objs 6624 +13.4% 7512 ± 5% slabinfo.btrfs_delayed_node.active_objs 6768 +12.1% 7589 ± 5% slabinfo.btrfs_delayed_node.num_objs 47815 ± 86% -89.2% 5149 ± 6% sched_debug.cfs_rq:/.load.stddev 142.69 ± 15% +66.8% 237.99 ± 13% sched_debug.cfs_rq:/.load_avg.avg 1598 ± 30% +92.4% 3075 ± 20% sched_debug.cfs_rq:/.load_avg.max 383.38 ± 21% +90.4% 730.10 ± 13% sched_debug.cfs_rq:/.load_avg.stddev 37712 ± 16% -27.1% 27508 ± 16% sched_debug.cfs_rq:/.min_vruntime.avg 58.89 ± 49% +57.8% 92.96 ± 5% sched_debug.cfs_rq:/.util_est_enqueued.stddev 1901 ± 4% -8.3% 1743 ± 3% proc-vmstat.nr_active_anon 5055884 +3.2% 5219972 proc-vmstat.nr_file_pages 4788961 ± 2% +3.4% 4953360 proc-vmstat.nr_inactive_file 36576 ± 2% -4.1% 35069 proc-vmstat.nr_kernel_stack 9756 -1.6% 9601 proc-vmstat.nr_mapped 5205 ± 2% -5.1% 4939 proc-vmstat.nr_shmem 1901 ± 4% -8.3% 1743 ± 3% proc-vmstat.nr_zone_active_anon 4788961 ± 2% +3.4% 4953360 proc-vmstat.nr_zone_inactive_file 4864 ± 5% -8.2% 4465 proc-vmstat.pgactivate 2.507e+09 -8.7% 2.29e+09 perf-stat.i.branch-instructions 71942409 -5.3% 68125925 ± 2% perf-stat.i.cache-misses 4.372e+10 -18.5% 3.565e+10 ± 3% perf-stat.i.cpu-cycles 1.231e+10 -7.5% 1.138e+10 perf-stat.i.instructions 2515 ± 8% -11.4% 2228 ± 5% perf-stat.i.instructions-per-iTLB-miss 0.23 -17.3% 0.19 ± 3% perf-stat.i.metric.GHz 0.71 ± 19% +37.6% 0.97 ± 17% perf-stat.i.metric.K/sec 35.43 -6.2% 33.24 ± 2% perf-stat.i.metric.M/sec 3.58 -8.5% 3.28 perf-stat.overall.cpi 612.62 -9.0% 557.36 ± 3% perf-stat.overall.cycles-between-cache-misses 2610 ± 6% -11.9% 2299 ± 4% perf-stat.overall.instructions-per-iTLB-miss 0.28 +9.3% 0.31 perf-stat.overall.ipc 2.39e+09 ± 2% -6.3% 2.239e+09 perf-stat.ps.branch-instructions 68534728 ± 2% -4.6% 65359103 ± 2% perf-stat.ps.cache-misses 4.199e+10 ± 2% -13.3% 3.641e+10 perf-stat.ps.cpu-cycles 1.173e+10 ± 2% -5.3% 1.112e+10 perf-stat.ps.instructions 226946 ± 18% +44.8% 328685 ± 19% numa-meminfo.node0.AnonPages 249710 ± 18% +41.9% 354296 ± 20% numa-meminfo.node0.AnonPages.max 233745 ± 17% +42.0% 331984 ± 20% numa-meminfo.node0.Inactive(anon) 8680 ± 73% +135.9% 20474 ± 28% numa-meminfo.node0.PageTables 250653 ± 2% +6.8% 267784 ± 5% numa-meminfo.node0.Unevictable 2890241 ± 9% -58.3% 1206212 ± 10% numa-meminfo.node0.Writeback 338312 ± 83% +111.9% 716979 ± 64% numa-meminfo.node1.Dirty 8444 ± 82% +169.8% 22785 ± 37% numa-meminfo.node2.AnonHugePages 236489 ± 16% +142.5% 573382 ± 36% numa-meminfo.node2.Dirty 2484617 ± 13% +117.6% 5406277 ± 50% numa-meminfo.node2.FilePages 2459002 ± 19% +115.1% 5290461 ± 51% numa-meminfo.node2.Inactive 2216704 ± 16% +131.7% 5135596 ± 53% numa-meminfo.node2.Inactive(file) 871702 ± 16% +52.8% 1331684 ± 6% numa-meminfo.node2.Writeback 3888 ± 16% -24.5% 2937 ± 6% numa-meminfo.node3.Active(anon) 57796 ± 19% -49.7% 29078 ± 34% numa-meminfo.node3.KReclaimable 57796 ± 19% -49.7% 29078 ± 34% numa-meminfo.node3.SReclaimable 130240 ± 7% -30.1% 91037 ± 15% numa-meminfo.node3.Slab 56616 ± 18% +45.2% 82199 ± 19% numa-vmstat.node0.nr_anon_pages 58314 ± 17% +42.4% 83024 ± 20% numa-vmstat.node0.nr_inactive_anon 2172 ± 73% +135.7% 5122 ± 28% numa-vmstat.node0.nr_page_table_pages 62662 ± 2% +6.8% 66945 ± 5% numa-vmstat.node0.nr_unevictable 738434 ± 9% -59.3% 300391 ± 10% numa-vmstat.node0.nr_writeback 58312 ± 17% +42.4% 83023 ± 20% numa-vmstat.node0.nr_zone_inactive_anon 62662 ± 2% +6.8% 66945 ± 5% numa-vmstat.node0.nr_zone_unevictable 926442 ± 6% -53.1% 434282 ± 12% numa-vmstat.node0.nr_zone_write_pending 84324 ± 81% +112.0% 178764 ± 64% numa-vmstat.node1.nr_dirty 427046 ± 84% +74.8% 746464 ± 46% numa-vmstat.node1.nr_zone_write_pending 627042 ± 18% +129.0% 1435752 ± 55% numa-vmstat.node2.nr_dirtied 59963 ± 18% +137.8% 142585 ± 36% numa-vmstat.node2.nr_dirty 634106 ± 15% +112.1% 1345185 ± 50% numa-vmstat.node2.nr_file_pages 567086 ± 17% +125.3% 1277507 ± 53% numa-vmstat.node2.nr_inactive_file 223407 ± 17% +48.3% 331299 ± 7% numa-vmstat.node2.nr_writeback 567087 ± 17% +125.3% 1277515 ± 53% numa-vmstat.node2.nr_zone_inactive_file 283372 ± 17% +67.2% 473906 ± 8% numa-vmstat.node2.nr_zone_write_pending 922.00 ± 19% -20.0% 737.25 ± 6% numa-vmstat.node3.nr_active_anon 14591 ± 20% -50.1% 7276 ± 35% numa-vmstat.node3.nr_slab_reclaimable 922.00 ± 19% -20.0% 737.25 ± 6% numa-vmstat.node3.nr_zone_active_anon 141466 ± 17% +523.4% 881964 ± 85% numa-vmstat.node3.numa_other 282981 ± 14% +46.6% 414731 ± 12% interrupts.CAL:Function_call_interrupts 770.00 ± 51% -56.5% 335.00 ± 59% interrupts.CPU0.NMI:Non-maskable_interrupts 770.00 ± 51% -56.5% 335.00 ± 59% interrupts.CPU0.PMI:Performance_monitoring_interrupts 346.25 ± 15% -46.7% 184.50 ± 23% interrupts.CPU110.NMI:Non-maskable_interrupts 346.25 ± 15% -46.7% 184.50 ± 23% interrupts.CPU110.PMI:Performance_monitoring_interrupts 1359 ± 14% +31.7% 1789 ± 17% interrupts.CPU128.CAL:Function_call_interrupts 1378 ± 14% +2294.9% 33020 ±164% interrupts.CPU153.CAL:Function_call_interrupts 214.50 ± 34% +91.7% 411.25 ± 39% interrupts.CPU16.NMI:Non-maskable_interrupts 214.50 ± 34% +91.7% 411.25 ± 39% interrupts.CPU16.PMI:Performance_monitoring_interrupts 403.50 ± 35% -40.3% 241.00 ± 8% interrupts.CPU169.NMI:Non-maskable_interrupts 403.50 ± 35% -40.3% 241.00 ± 8% interrupts.CPU169.PMI:Performance_monitoring_interrupts 439.00 ± 50% -51.5% 212.75 ± 22% interrupts.CPU170.NMI:Non-maskable_interrupts 439.00 ± 50% -51.5% 212.75 ± 22% interrupts.CPU170.PMI:Performance_monitoring_interrupts 1062 ±116% -77.3% 240.75 ± 23% interrupts.CPU174.NMI:Non-maskable_interrupts 1062 ±116% -77.3% 240.75 ± 23% interrupts.CPU174.PMI:Performance_monitoring_interrupts 291.25 ± 31% -34.2% 191.75 ± 30% interrupts.CPU2.NMI:Non-maskable_interrupts 291.25 ± 31% -34.2% 191.75 ± 30% interrupts.CPU2.PMI:Performance_monitoring_interrupts 342.00 ± 43% -58.3% 142.75 ± 16% interrupts.CPU22.NMI:Non-maskable_interrupts 342.00 ± 43% -58.3% 142.75 ± 16% interrupts.CPU22.PMI:Performance_monitoring_interrupts 241.25 ± 22% -42.6% 138.50 ± 22% interrupts.CPU23.NMI:Non-maskable_interrupts 241.25 ± 22% -42.6% 138.50 ± 22% interrupts.CPU23.PMI:Performance_monitoring_interrupts 1365 ± 14% +49.5% 2041 ± 24% interrupts.CPU35.CAL:Function_call_interrupts 222.00 ± 94% +175.1% 610.75 ± 51% interrupts.CPU63.NMI:Non-maskable_interrupts 222.00 ± 94% +175.1% 610.75 ± 51% interrupts.CPU63.PMI:Performance_monitoring_interrupts 146.75 ± 39% +104.6% 300.25 ± 32% interrupts.CPU64.NMI:Non-maskable_interrupts 146.75 ± 39% +104.6% 300.25 ± 32% interrupts.CPU64.PMI:Performance_monitoring_interrupts 557.00 ± 88% -61.3% 215.75 ± 21% interrupts.CPU74.NMI:Non-maskable_interrupts 557.00 ± 88% -61.3% 215.75 ± 21% interrupts.CPU74.PMI:Performance_monitoring_interrupts 319.25 ± 22% -43.9% 179.25 ± 36% interrupts.CPU75.NMI:Non-maskable_interrupts 319.25 ± 22% -43.9% 179.25 ± 36% interrupts.CPU75.PMI:Performance_monitoring_interrupts 1343 ± 15% +23.0% 1652 ± 4% interrupts.CPU80.CAL:Function_call_interrupts 1095 ± 66% -76.5% 257.00 ± 17% interrupts.CPU96.NMI:Non-maskable_interrupts 1095 ± 66% -76.5% 257.00 ± 17% interrupts.CPU96.PMI:Performance_monitoring_interrupts 817.25 ± 12% +62.6% 1329 ± 17% interrupts.RES:Rescheduling_interrupts 12.39 ± 23% -8.2 4.22 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 12.35 ± 24% -8.1 4.21 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 12.12 ± 24% -8.0 4.08 ± 6% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 12.11 ± 24% -8.0 4.07 ± 6% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 12.10 ± 24% -8.0 4.06 ± 6% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 12.07 ± 24% -8.0 4.04 ± 6% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write 12.28 ± 24% -7.5 4.83 ± 5% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.07 ± 31% -6.1 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata 6.36 ± 30% -4.8 1.55 ± 11% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 6.31 ± 30% -4.8 1.51 ± 12% perf-profile.calltrace.cycles-pp.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 6.31 ± 30% -4.8 1.51 ± 12% perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 6.13 ± 31% -4.8 1.38 ± 14% perf-profile.calltrace.cycles-pp._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write 4.39 ± 18% -3.2 1.21 ± 15% perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write 4.38 ± 18% -3.2 1.20 ± 15% perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write 4.21 ± 19% -3.1 1.09 ± 20% perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter 0.13 ±173% +0.5 0.63 ± 6% perf-profile.calltrace.cycles-pp.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 4.51 ± 6% +0.5 5.04 ± 3% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 4.48 ± 5% +0.5 5.02 ± 3% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 4.37 ± 6% +0.6 4.93 ± 3% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.18 ±173% +0.6 0.75 ± 27% perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write.cold.new_sync_write 0.18 ±173% +0.6 0.76 ± 27% perf-profile.calltrace.cycles-pp.vprintk_emit.devkmsg_emit.devkmsg_write.cold.new_sync_write.vfs_write 0.18 ±173% +0.6 0.76 ± 27% perf-profile.calltrace.cycles-pp.devkmsg_write.cold.new_sync_write.vfs_write.ksys_write.do_syscall_64 0.18 ±173% +0.6 0.76 ± 27% perf-profile.calltrace.cycles-pp.devkmsg_emit.devkmsg_write.cold.new_sync_write.vfs_write.ksys_write 0.18 ±173% +0.6 0.77 ± 27% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 0.18 ±173% +0.6 0.77 ± 27% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 0.18 ±173% +0.6 0.77 ± 27% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 0.18 ±173% +0.6 0.77 ± 27% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 0.18 ±173% +0.6 0.77 ± 27% perf-profile.calltrace.cycles-pp.write 0.00 +0.6 0.62 ± 7% perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_irq_exit.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.00 +1.1 1.06 ± 21% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write 7.48 ± 5% +1.2 8.68 ± 2% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 0.00 +1.3 1.33 ± 15% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.__reserve_bytes.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata 11.79 ± 8% +2.8 14.63 ± 8% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 9.11 ± 14% +4.4 13.51 ± 18% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 10.58 ± 26% -10.6 0.00 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 12.10 ± 24% -8.0 4.06 ± 6% perf-profile.children.cycles-pp.btrfs_file_write_iter 12.07 ± 24% -8.0 4.04 ± 6% perf-profile.children.cycles-pp.btrfs_buffered_write 14.00 ± 18% -7.9 6.09 ± 7% perf-profile.children.cycles-pp._raw_spin_lock 12.64 ± 23% -7.8 4.85 ± 5% perf-profile.children.cycles-pp.ksys_write 12.62 ± 24% -7.8 4.83 ± 5% perf-profile.children.cycles-pp.new_sync_write 12.63 ± 23% -7.8 4.85 ± 5% perf-profile.children.cycles-pp.vfs_write 13.17 ± 23% -7.6 5.60 ± 6% perf-profile.children.cycles-pp.do_syscall_64 13.24 ± 23% -7.3 5.92 ± 6% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 6.38 ± 30% -4.8 1.55 ± 12% perf-profile.children.cycles-pp.__reserve_bytes 6.33 ± 30% -4.8 1.52 ± 12% perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes 6.36 ± 30% -4.8 1.55 ± 11% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 4.49 ± 18% -3.3 1.23 ± 15% perf-profile.children.cycles-pp.btrfs_block_rsv_release 4.39 ± 18% -3.2 1.21 ± 15% perf-profile.children.cycles-pp.btrfs_inode_rsv_release 0.16 ± 77% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.do_filp_open 0.16 ± 77% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.path_openat 0.16 ± 75% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.do_sys_open 0.16 ± 75% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.do_sys_openat2 0.21 ± 32% -0.1 0.13 ± 29% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.22 ± 9% -0.1 0.16 ± 6% perf-profile.children.cycles-pp.update_blocked_averages 0.10 ± 29% -0.0 0.05 ± 70% perf-profile.children.cycles-pp.update_ts_time_stats 0.07 ± 12% -0.0 0.04 ± 58% perf-profile.children.cycles-pp.__pagevec_lru_add_fn 0.16 ± 9% +0.0 0.20 ± 12% perf-profile.children.cycles-pp.brd_lookup_page 0.13 ± 11% +0.0 0.17 ± 18% perf-profile.children.cycles-pp.__radix_tree_lookup 0.14 ± 7% +0.1 0.20 ± 9% perf-profile.children.cycles-pp.__intel_pmu_enable_all 0.48 ± 8% +0.1 0.62 ± 8% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.51 ± 29% +0.3 0.77 ± 27% perf-profile.children.cycles-pp.write 0.39 ± 32% +0.3 0.65 ± 6% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.50 ± 30% +0.3 0.76 ± 27% perf-profile.children.cycles-pp.devkmsg_write.cold 0.50 ± 30% +0.3 0.76 ± 27% perf-profile.children.cycles-pp.devkmsg_emit 0.03 ±173% +0.3 0.29 ±108% perf-profile.children.cycles-pp.osq_lock 0.03 ±173% +0.3 0.29 ±107% perf-profile.children.cycles-pp.__mutex_lock 0.11 ±130% +0.3 0.45 ± 68% perf-profile.children.cycles-pp.__do_sys_finit_module 0.11 ±130% +0.3 0.45 ± 68% perf-profile.children.cycles-pp.load_module 0.11 ±130% +0.3 0.45 ± 67% perf-profile.children.cycles-pp.syscall 0.14 ± 88% +0.4 0.54 ± 53% perf-profile.children.cycles-pp.wb_workfn 0.14 ± 88% +0.4 0.54 ± 53% perf-profile.children.cycles-pp.wb_writeback 0.14 ± 88% +0.4 0.54 ± 53% perf-profile.children.cycles-pp.writeback_sb_inodes 0.12 ±105% +0.4 0.53 ± 54% perf-profile.children.cycles-pp.__writeback_single_inode 4.76 ± 5% +0.5 5.21 ± 3% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 4.65 ± 5% +0.5 5.13 ± 2% perf-profile.children.cycles-pp.hrtimer_interrupt 6.15 ± 5% +0.5 6.65 ± 2% perf-profile.children.cycles-pp.asm_call_sysvec_on_stack 2.06 ± 28% +1.0 3.05 ± 9% perf-profile.children.cycles-pp.ktime_get 7.80 ± 4% +1.1 8.94 ± 2% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 10.10 ± 6% +2.0 12.05 ± 6% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.00 +2.6 2.56 ± 19% perf-profile.children.cycles-pp.__cna_queued_spin_lock_slowpath 9.16 ± 14% +4.4 13.57 ± 18% perf-profile.children.cycles-pp.menu_select 10.50 ± 26% -10.5 0.00 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.18 ± 30% -0.1 0.11 ± 27% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.10 ± 11% +0.0 0.13 ± 10% perf-profile.self.cycles-pp.__extent_writepage 0.14 ± 7% +0.1 0.20 ± 9% perf-profile.self.cycles-pp.__intel_pmu_enable_all 0.08 ± 10% +0.1 0.16 ± 22% perf-profile.self.cycles-pp.end_page_writeback 0.03 ±173% +0.2 0.28 ±105% perf-profile.self.cycles-pp.osq_lock 1.77 ± 32% +1.0 2.75 ± 10% perf-profile.self.cycles-pp.ktime_get 0.00 +2.5 2.51 ± 19% perf-profile.self.cycles-pp.__cna_queued_spin_lock_slowpath 7.68 ± 15% +4.1 11.79 ± 21% perf-profile.self.cycles-pp.menu_select *************************************************************************************************** lkp-csl-2ap3: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2ap3/new_fserver/reaim/0x5003003 commit: cb45bab007 ("locking/qspinlock: Refactor the qspinlock slow path") 0e8d8f4f12 ("locking/qspinlock: Introduce CNA into the slow path of qspinlock") cb45bab007ff0cfc 0e8d8f4f1214cfbac219d6917b5 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 last_state.is_incomplete_run 1:4 -25% :4 last_state.wget_initrd :4 8% 0:4 perf-profile.children.cycles-pp.error_entry %stddev %change %stddev \ | \ 556.93 -64.1% 200.03 reaim.child_systime 49.95 +3.6% 51.77 reaim.child_utime 318145 +96.1% 623983 reaim.jobs_per_min 1657 +96.1% 3249 reaim.jobs_per_min_child 92.26 -7.8% 85.11 reaim.jti 327019 +100.4% 655467 reaim.max_jobs_per_min 3.55 -49.0% 1.81 reaim.parent_time 7.19 ± 2% +100.0% 14.38 ± 4% reaim.std_dev_percent 0.24 ± 2% -7.3% 0.22 ± 2% reaim.std_dev_time 462973 +17.9% 545826 reaim.time.involuntary_context_switches 38288 +98.5% 75996 ± 2% reaim.time.major_page_faults 1.041e+08 +44.8% 1.509e+08 reaim.time.minor_page_faults 10863 -39.7% 6549 reaim.time.percent_of_cpu_this_job_got 30257 -47.9% 15754 reaim.time.system_time 2714 +50.2% 4077 reaim.time.user_time 3000456 ± 2% +38.9% 4167220 ± 2% reaim.time.voluntary_context_switches 1043200 +44.9% 1512000 reaim.workload 29843 +40.0% 41767 uptime.idle 20127118 ± 12% +48.7% 29930674 ± 4% cpuidle.C1.time 284217 ± 9% +66.1% 472177 ± 2% cpuidle.C1.usage 308815 ± 13% +26.0% 389201 ± 16% cpuidle.POLL.usage 41.44 +20.8 62.28 mpstat.cpu.all.idle% 0.69 +1.3 1.98 ± 2% mpstat.cpu.all.soft% 51.88 -24.6 27.26 mpstat.cpu.all.sys% 4.60 +2.3 6.91 mpstat.cpu.all.usr% 41.33 +50.0% 62.00 vmstat.cpu.id 53.00 -43.4% 30.00 vmstat.cpu.sy 110.00 -25.9% 81.50 ± 2% vmstat.procs.r 25559 ± 2% +46.2% 37370 vmstat.system.cs 269427 ± 4% -12.8% 234969 meminfo.Active 269427 ± 4% -12.8% 234969 meminfo.Active(anon) 32953 +9.7% 36144 ± 2% meminfo.KernelStack 19202 +63.2% 31338 meminfo.PageTables 483410 +11.8% 540459 meminfo.SUnreclaim 304770 ± 4% -12.8% 265628 meminfo.Shmem 620741 +10.5% 685878 meminfo.Slab 18788579 +43.8% 27017272 ± 4% numa-numastat.node0.local_node 18827783 +43.8% 27074532 ± 4% numa-numastat.node0.numa_hit 19046024 +45.8% 27771991 ± 2% numa-numastat.node1.local_node 19126483 +45.6% 27844754 ± 2% numa-numastat.node1.numa_hit 18849870 +44.4% 27228340 ± 2% numa-numastat.node2.local_node 18926181 +44.1% 27271719 ± 2% numa-numastat.node2.numa_hit 18622314 +46.2% 27216882 ± 3% numa-numastat.node3.local_node 18686267 +46.1% 27303516 ± 3% numa-numastat.node3.numa_hit 42977 ± 81% -79.9% 8629 ± 59% numa-meminfo.node0.Active 42977 ± 81% -79.9% 8629 ± 59% numa-meminfo.node0.Active(anon) 7040 +24.4% 8759 ± 8% numa-meminfo.node1.KernelStack 2790 ± 11% +140.4% 6708 ± 26% numa-meminfo.node1.PageTables 103698 +19.3% 123717 ± 9% numa-meminfo.node1.SUnreclaim 129487 ± 2% +18.0% 152790 ± 7% numa-meminfo.node1.Slab 32541 ± 26% +46.6% 47695 ± 24% numa-meminfo.node2.KReclaimable 32541 ± 26% +46.6% 47695 ± 24% numa-meminfo.node2.SReclaimable 150472 ± 20% +21.5% 182840 ± 15% numa-meminfo.node2.Slab 167884 ± 33% +71.3% 287506 ± 45% numa-meminfo.node3.AnonPages.max 2974 ± 11% +145.0% 7288 ± 28% numa-meminfo.node3.PageTables 109568 ± 4% +13.7% 124544 ± 12% numa-meminfo.node3.SUnreclaim 10666 ± 81% -79.8% 2152 ± 59% numa-vmstat.node0.nr_active_anon 10665 ± 81% -79.8% 2152 ± 59% numa-vmstat.node0.nr_zone_active_anon 9854862 +41.8% 13975810 ± 2% numa-vmstat.node0.numa_hit 9773079 +41.9% 13870000 ± 2% numa-vmstat.node0.numa_local 7029 +25.1% 8796 ± 8% numa-vmstat.node1.nr_kernel_stack 678.67 ± 8% +152.6% 1714 ± 26% numa-vmstat.node1.nr_page_table_pages 25872 +20.1% 31062 ± 9% numa-vmstat.node1.nr_slab_unreclaimable 9970820 +42.6% 14214395 numa-vmstat.node1.numa_hit 9810384 +43.4% 14063434 numa-vmstat.node1.numa_local 8132 ± 26% +46.4% 11904 ± 24% numa-vmstat.node2.nr_slab_reclaimable 9969168 +43.4% 14294976 ± 2% numa-vmstat.node2.numa_hit 9812426 +44.6% 14186949 ± 3% numa-vmstat.node2.numa_local 726.67 ± 12% +158.7% 1879 ± 26% numa-vmstat.node3.nr_page_table_pages 27327 ± 4% +13.9% 31114 ± 12% numa-vmstat.node3.nr_slab_unreclaimable 9719726 +44.0% 13991557 ± 2% numa-vmstat.node3.numa_hit 9615461 +44.1% 13853485 ± 2% numa-vmstat.node3.numa_local 104269 ± 49% +32.4% 138078 ± 20% numa-vmstat.node3.numa_other 67226 ± 4% -12.4% 58905 proc-vmstat.nr_active_anon 316527 -3.1% 306823 proc-vmstat.nr_file_pages 92663 -2.9% 89935 proc-vmstat.nr_inactive_anon 33012 +9.3% 36075 ± 2% proc-vmstat.nr_kernel_stack 4865 +59.3% 7749 ± 4% proc-vmstat.nr_page_table_pages 76214 ± 4% -12.7% 66510 proc-vmstat.nr_shmem 34346 +5.8% 36334 proc-vmstat.nr_slab_reclaimable 120937 +11.5% 134852 proc-vmstat.nr_slab_unreclaimable 67226 ± 4% -12.4% 58905 proc-vmstat.nr_zone_active_anon 92663 -2.9% 89935 proc-vmstat.nr_zone_inactive_anon 152011 ± 9% -72.0% 42500 ± 30% proc-vmstat.numa_hint_faults 75604538 +44.9% 1.095e+08 proc-vmstat.numa_hit 75344552 +45.0% 1.093e+08 proc-vmstat.numa_local 129725 ± 13% -78.5% 27855 ± 33% proc-vmstat.numa_pages_migrated 440421 ± 8% -27.0% 321634 ± 22% proc-vmstat.numa_pte_updates 187992 ± 4% +5.4% 198156 proc-vmstat.pgactivate 83223343 +45.7% 1.213e+08 proc-vmstat.pgalloc_normal 1.056e+08 +44.3% 1.524e+08 proc-vmstat.pgfault 83115624 +45.8% 1.212e+08 proc-vmstat.pgfree 129725 ± 13% -78.5% 27855 ± 33% proc-vmstat.pgmigrate_success 10628125 +46.1% 15525117 proc-vmstat.pgreuse 13330 ± 54% +717.2% 108933 ± 77% sched_debug.cfs_rq:/.MIN_vruntime.avg 184231 ± 54% +190.7% 535537 ± 48% sched_debug.cfs_rq:/.MIN_vruntime.stddev 6040 ± 6% +338.5% 26486 ± 51% sched_debug.cfs_rq:/.load.avg 284313 ± 37% +159.7% 738429 ± 14% sched_debug.cfs_rq:/.load.max 1180 ± 35% -100.0% 0.00 sched_debug.cfs_rq:/.load.min 22088 ± 35% +398.6% 110141 ± 30% sched_debug.cfs_rq:/.load.stddev 13330 ± 54% +717.2% 108933 ± 77% sched_debug.cfs_rq:/.max_vruntime.avg 184231 ± 54% +190.7% 535537 ± 48% sched_debug.cfs_rq:/.max_vruntime.stddev 16468615 -51.1% 8045534 ± 11% sched_debug.cfs_rq:/.min_vruntime.avg 16715699 -50.1% 8341867 ± 11% sched_debug.cfs_rq:/.min_vruntime.max 15817769 -51.3% 7697369 ± 11% sched_debug.cfs_rq:/.min_vruntime.min 0.71 ± 11% -47.8% 0.37 ± 14% sched_debug.cfs_rq:/.nr_running.avg 1.17 +19.3% 1.39 ± 4% sched_debug.cfs_rq:/.nr_running.max 0.22 ± 35% -100.0% 0.00 sched_debug.cfs_rq:/.nr_running.min 0.14 ± 18% +74.2% 0.25 ± 6% sched_debug.cfs_rq:/.nr_running.stddev 694.83 ± 15% -51.9% 334.48 ± 20% sched_debug.cfs_rq:/.runnable_avg.avg 110.72 ± 51% -90.4% 10.63 ±114% sched_debug.cfs_rq:/.runnable_avg.min 155.35 ± 11% +42.5% 221.31 ± 9% sched_debug.cfs_rq:/.runnable_avg.stddev 64422 ± 94% +295.3% 254650 ± 19% sched_debug.cfs_rq:/.spread0.avg 311765 ± 14% +76.7% 550961 ± 14% sched_debug.cfs_rq:/.spread0.max -587893 -84.1% -93210 sched_debug.cfs_rq:/.spread0.min 685.90 ± 14% -56.7% 297.04 ± 23% sched_debug.cfs_rq:/.util_avg.avg 100.39 ± 59% -90.9% 9.12 ±125% sched_debug.cfs_rq:/.util_avg.min 153.19 ± 12% +28.3% 196.53 ± 12% sched_debug.cfs_rq:/.util_avg.stddev 398.32 ± 19% -79.6% 81.24 ± 25% sched_debug.cfs_rq:/.util_est_enqueued.avg 932.06 ± 8% -26.5% 684.67 ± 16% sched_debug.cfs_rq:/.util_est_enqueued.max 0.28 ± 28% -100.0% 0.00 sched_debug.cfs_rq:/.util_est_enqueued.min 186.70 ± 17% -31.0% 128.84 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.stddev 42223 ± 22% +72.6% 72869 ± 21% sched_debug.cpu.avg_idle.min 513534 ± 67% -58.1% 214959 ± 3% sched_debug.cpu.avg_idle.stddev 69440 ± 18% -53.1% 32566 ± 23% sched_debug.cpu.curr->pid.avg 465.44 ± 54% -100.0% 0.00 sched_debug.cpu.curr->pid.min 0.71 ± 11% -40.6% 0.42 ± 9% sched_debug.cpu.nr_running.avg 0.22 ± 35% -100.0% 0.00 sched_debug.cpu.nr_running.min 0.18 ± 10% +68.8% 0.31 ± 5% sched_debug.cpu.nr_running.stddev 21723 ± 2% +35.3% 29395 ± 9% sched_debug.cpu.nr_switches.avg 37415 ± 16% +23.6% 46248 ± 5% sched_debug.cpu.nr_switches.max 17608 +44.3% 25417 ± 10% sched_debug.cpu.nr_switches.min 161.39 ± 12% +47.1% 237.34 ± 18% sched_debug.cpu.nr_uninterruptible.max -54.78 +48.6% -81.42 sched_debug.cpu.nr_uninterruptible.min 24.07 ± 4% +58.5% 38.15 ± 11% sched_debug.cpu.nr_uninterruptible.stddev 11566 +11.1% 12849 slabinfo.Acpi-State.active_objs 11566 +11.1% 12849 slabinfo.Acpi-State.num_objs 240075 +13.6% 272838 slabinfo.dentry.active_objs 5793 +13.6% 6580 slabinfo.dentry.active_slabs 243365 +13.6% 276386 slabinfo.dentry.num_objs 5793 +13.6% 6580 slabinfo.dentry.num_slabs 955.00 ± 9% +16.4% 1112 ± 7% slabinfo.file_lock_cache.active_objs 955.00 ± 9% +16.4% 1112 ± 7% slabinfo.file_lock_cache.num_objs 14271 +14.7% 16366 slabinfo.files_cache.active_objs 14271 +14.7% 16366 slabinfo.files_cache.num_objs 145916 +52.7% 222770 ± 2% slabinfo.filp.active_objs 2300 +53.0% 3519 ± 2% slabinfo.filp.active_slabs 147261 +53.0% 225274 ± 2% slabinfo.filp.num_objs 2300 +53.0% 3519 ± 2% slabinfo.filp.num_slabs 1319 ± 14% +26.2% 1664 ± 8% slabinfo.khugepaged_mm_slot.active_objs 1319 ± 14% +26.2% 1664 ± 8% slabinfo.khugepaged_mm_slot.num_objs 1037 +19.6% 1240 slabinfo.kmalloc-512.active_slabs 66437 +19.5% 79424 slabinfo.kmalloc-512.num_objs 1037 +19.6% 1240 slabinfo.kmalloc-512.num_slabs 114959 +9.0% 125273 slabinfo.kmalloc-8.active_objs 4536 ± 5% +7.2% 4862 ± 3% slabinfo.kmalloc-rcl-64.active_objs 4536 ± 5% +7.2% 4862 ± 3% slabinfo.kmalloc-rcl-64.num_objs 10920 +18.9% 12978 slabinfo.mm_struct.active_objs 10920 +18.9% 12979 slabinfo.mm_struct.num_objs 1104 ± 5% -16.9% 917.00 ± 10% slabinfo.mnt_cache.active_objs 1104 ± 5% -16.9% 917.00 ± 10% slabinfo.mnt_cache.num_objs 1551 +14.3% 1773 ± 2% slabinfo.names_cache.active_objs 1551 +14.3% 1773 ± 2% slabinfo.names_cache.num_objs 35694 ± 2% +49.3% 53295 ± 4% slabinfo.pid.active_objs 557.00 ± 2% +49.4% 832.25 ± 4% slabinfo.pid.active_slabs 35694 ± 2% +49.3% 53295 ± 4% slabinfo.pid.num_objs 557.00 ± 2% +49.4% 832.25 ± 4% slabinfo.pid.num_slabs 8786 +20.2% 10558 slabinfo.sighand_cache.active_objs 590.00 +20.3% 709.75 slabinfo.sighand_cache.active_slabs 8858 +20.3% 10654 slabinfo.sighand_cache.num_objs 590.00 +20.3% 709.75 slabinfo.sighand_cache.num_slabs 8502 +11.6% 9489 ± 2% slabinfo.sock_inode_cache.active_objs 8502 +11.6% 9489 ± 2% slabinfo.sock_inode_cache.num_objs 39805 ± 2% +45.4% 57892 ± 2% slabinfo.task_delay_info.active_objs 780.00 ± 2% +45.5% 1135 ± 2% slabinfo.task_delay_info.active_slabs 39805 ± 2% +45.5% 57905 ± 2% slabinfo.task_delay_info.num_objs 780.00 ± 2% +45.5% 1135 ± 2% slabinfo.task_delay_info.num_slabs 3088 +29.3% 3992 ± 2% slabinfo.task_struct.active_objs 3098 +29.3% 4005 ± 2% slabinfo.task_struct.active_slabs 3098 +29.3% 4005 ± 2% slabinfo.task_struct.num_objs 3098 +29.3% 4005 ± 2% slabinfo.task_struct.num_slabs 1.353e+10 +11.8% 1.513e+10 perf-stat.i.branch-instructions 84012986 ± 5% +46.6% 1.232e+08 ± 6% perf-stat.i.branch-misses 31.81 ± 12% -10.8 21.01 ± 21% perf-stat.i.cache-miss-rate% 68723351 ± 2% -10.6% 61409511 ± 4% perf-stat.i.cache-misses 2.121e+08 ± 10% +57.2% 3.333e+08 ± 8% perf-stat.i.cache-references 25583 ± 2% +51.4% 38727 perf-stat.i.context-switches 5.21 ± 3% -25.2% 3.90 ± 5% perf-stat.i.cpi 192200 +7.5% 206633 perf-stat.i.cpu-clock 3.37e+11 -28.3% 2.415e+11 perf-stat.i.cpu-cycles 3432 +61.5% 5541 ± 3% perf-stat.i.cpu-migrations 4302 -29.4% 3039 ± 8% perf-stat.i.cycles-between-cache-misses 2726072 ± 23% +52.3% 4151327 ± 26% perf-stat.i.dTLB-load-misses 1.215e+10 +6.3% 1.292e+10 perf-stat.i.dTLB-loads 941111 ± 11% +46.9% 1382681 ± 11% perf-stat.i.dTLB-store-misses 2.493e+09 +48.4% 3.701e+09 perf-stat.i.dTLB-stores 71.80 ± 4% -6.7 65.13 perf-stat.i.iTLB-load-miss-rate% 8355924 ± 6% +47.0% 12280660 ± 7% perf-stat.i.iTLB-load-misses 3268044 ± 18% +65.6% 5412376 ± 8% perf-stat.i.iTLB-loads 6.464e+10 +19.2% 7.704e+10 perf-stat.i.instructions 6911 ± 3% -32.1% 4691 ± 6% perf-stat.i.instructions-per-iTLB-miss 0.23 +28.9% 0.30 perf-stat.i.ipc 124.96 +101.4% 251.73 ± 2% perf-stat.i.major-faults 1.76 -41.2% 1.03 perf-stat.i.metric.GHz 0.56 ± 8% +32.5% 0.75 ± 7% perf-stat.i.metric.K/sec 332691 +49.2% 496370 perf-stat.i.minor-faults 91.27 -4.0 87.22 perf-stat.i.node-load-miss-rate% 18748059 -13.0% 16311904 ± 3% perf-stat.i.node-load-misses 1587953 +50.0% 2382313 perf-stat.i.node-loads 86.40 -8.3 78.07 ± 2% perf-stat.i.node-store-miss-rate% 9547999 -19.9% 7644037 ± 2% perf-stat.i.node-store-misses 1064509 +92.2% 2045699 ± 2% perf-stat.i.node-stores 332816 +49.2% 496622 perf-stat.i.page-faults 192200 +7.5% 206633 perf-stat.i.task-clock 3.25 ± 9% +36.0% 4.42 ± 7% perf-stat.overall.MPKI 0.62 ± 5% +0.2 0.83 ± 5% perf-stat.overall.branch-miss-rate% 32.96 ± 10% -13.4 19.58 ± 11% perf-stat.overall.cache-miss-rate% 5.25 -42.4% 3.02 perf-stat.overall.cpi 4953 -28.7% 3533 ± 4% perf-stat.overall.cycles-between-cache-misses 0.02 ± 23% +0.0 0.03 ± 24% perf-stat.overall.dTLB-load-miss-rate% 72.34 ± 3% -3.1 69.29 perf-stat.overall.iTLB-load-miss-rate% 7868 ± 6% -20.0% 6292 ± 7% perf-stat.overall.instructions-per-iTLB-miss 0.19 +73.5% 0.33 perf-stat.overall.ipc 92.20 -4.8 87.37 perf-stat.overall.node-load-miss-rate% 90.00 -9.8 80.24 perf-stat.overall.node-store-miss-rate% 19255954 -23.1% 14811673 perf-stat.overall.path-length 1.382e+10 +5.2% 1.454e+10 perf-stat.ps.branch-instructions 85147082 ± 5% +42.1% 1.21e+08 ± 5% perf-stat.ps.branch-misses 69832184 ± 2% -9.6% 63144472 ± 4% perf-stat.ps.cache-misses 2.141e+08 ± 9% +52.0% 3.254e+08 ± 8% perf-stat.ps.cache-references 25647 ± 2% +46.2% 37498 perf-stat.ps.context-switches 3.458e+11 -35.6% 2.227e+11 perf-stat.ps.cpu-cycles 3456 +55.5% 5375 ± 4% perf-stat.ps.cpu-migrations 2731148 ± 22% +50.7% 4114660 ± 24% perf-stat.ps.dTLB-load-misses 951612 ± 11% +42.4% 1354727 ± 10% perf-stat.ps.dTLB-store-misses 2.534e+09 +43.0% 3.624e+09 perf-stat.ps.dTLB-stores 8410006 ± 6% +39.9% 11763281 ± 6% perf-stat.ps.iTLB-load-misses 3251030 ± 17% +60.5% 5218236 ± 7% perf-stat.ps.iTLB-loads 6.591e+10 +11.7% 7.364e+10 perf-stat.ps.instructions 127.07 +97.9% 251.44 ± 2% perf-stat.ps.major-faults 338752 +44.5% 489644 perf-stat.ps.minor-faults 19070733 -11.1% 16952678 ± 4% perf-stat.ps.node-load-misses 1613838 +51.7% 2448074 perf-stat.ps.node-loads 9716997 -18.4% 7933032 ± 2% perf-stat.ps.node-store-misses 1079544 +80.8% 1952245 perf-stat.ps.node-stores 338879 +44.6% 489896 perf-stat.ps.page-faults 2.009e+13 +11.5% 2.239e+13 perf-stat.total.instructions 0.04 ± 54% +278.3% 0.14 ± 41% perf-sched.sch_delay.avg.ms.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk.part 0.11 ±123% +813.3% 1.04 ±120% perf-sched.sch_delay.avg.ms.d_alloc_parallel.__lookup_slow.walk_component.path_lookupat.isra 0.03 ± 9% +210.5% 0.10 ± 21% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.17 ± 89% +210.6% 0.52 ± 59% perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 0.05 ± 34% +1883.2% 0.91 ± 48% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 0.12 ± 45% +772.8% 1.04 ± 65% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown] 0.09 ± 9% +1004.3% 0.97 ± 90% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 0.06 ± 28% +390.8% 0.29 ± 41% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 0.63 ± 7% +80.2% 1.13 ± 21% perf-sched.sch_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork.[unknown] 0.07 ± 16% +113.1% 0.14 ± 21% perf-sched.sch_delay.avg.ms.io_schedule.__lock_page_killable.filemap_fault.__do_fault 0.01 ± 7% +196.2% 0.02 ± 65% perf-sched.sch_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.04 ± 45% +742.5% 0.35 ± 83% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.alloc_pages_vma.wp_page_copy 0.07 ± 78% -84.3% 0.01 ±142% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.allocate_slab.___slab_alloc 0.04 ± 79% +2194.9% 0.89 ±164% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.pte_alloc_one.__pte_alloc 0.07 ± 19% -86.5% 0.01 ± 59% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.__anon_vma_prepare.do_fault.__handle_mm_fault 0.25 ±105% -98.2% 0.00 ±137% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.copy_page_to_iter.generic_file_buffered_read.new_sync_read 0.40 ± 6% +91.2% 0.77 ± 5% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 0.08 ±100% -92.6% 0.01 ± 94% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.down_write.__anon_vma_prepare.do_anonymous_page 0.03 ± 79% +277.4% 0.12 ± 73% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.down_write.__vma_adjust.__split_vma 0.01 ± 16% +1.5e+05% 8.25 ±103% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.down_write.do_truncate.path_openat 0.04 ± 19% +318.9% 0.16 ± 44% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.down_write.unlink_anon_vmas.free_pgtables 0.04 ± 7% +1180.0% 0.51 ±145% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.down_write.unlink_file_vma.free_pgtables 0.00 ± 10% -89.3% 0.00 ±173% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.down_write_killable.__vm_munmap.__x64_sys_munmap 0.07 ± 54% +242.3% 0.26 ± 62% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.down_write_killable.vm_mmap_pgoff.elf_map 0.03 ± 11% -87.8% 0.00 ± 13% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.down_write_killable.vm_mmap_pgoff.ksys_mmap_pgoff 0.01 ± 49% +1.3e+05% 8.04 ±152% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.dput.do_unlinkat.do_syscall_64 0.10 ±127% +16890.4% 16.82 ±163% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.dput.done_path_create.do_linkat 0.13 ± 18% +459.3% 0.71 ± 18% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.dput.path_openat.do_filp_open 0.44 ±118% +204.8% 1.33 ± 94% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.dput.scan_positives.dcache_readdir 0.01 ± 36% +3156.9% 0.43 ± 63% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.dput.step_into.path_openat 0.07 ± 21% +554.5% 0.46 ± 93% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.dput.terminate_walk.path_openat 0.00 ±141% +19625.0% 0.07 ±150% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.generic_file_buffered_read.__kernel_read.load_elf_binary 0.08 ±134% +686.6% 0.66 ± 59% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__d_alloc.d_alloc_cursor 0.04 ± 37% +648.3% 0.33 ± 28% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.security_file_alloc.__alloc_file 0.21 ± 61% -77.6% 0.05 ±139% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.vm_area_alloc.__install_special_mapping 0.30 ± 97% -99.2% 0.00 ± 34% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.vm_area_alloc.do_brk_flags 3.67 ±140% -100.0% 0.00 ±173% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc_node_trace.__get_vm_area_node.__vmalloc_node_range 0.05 ± 51% +75.6% 0.09 ± 30% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc_trace.perf_event_mmap.mmap_region 0.19 ± 85% -93.9% 0.01 ±148% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc_trace.perf_event_mmap.mprotect_fixup 0.01 ± 66% +89763.4% 8.39 ±106% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.mnt_want_write.path_openat.do_filp_open 0.01 ±108% +903.8% 0.13 ±129% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.mutex_lock.futex_exec_release.exec_mm_release 0.03 ± 53% -67.5% 0.01 ± 83% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.mutex_lock.futex_exit_release.exit_mm_release 0.11 ±125% +1883.9% 2.25 ± 40% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 0.01 ± 70% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate 0.30 ± 3% +308.2% 1.22 ± 26% perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.sched_exec.bprm_execve 0.08 ± 68% -100.0% 0.00 perf-sched.sch_delay.avg.ms.preempt_schedule_common._cond_resched.wait_for_completion.stop_two_cpus.migrate_swap 0.01 ±103% -100.0% 0.00 perf-sched.sch_delay.avg.ms.rwsem_down_read_slowpath.page_lock_anon_vma_read.rmap_walk_anon.try_to_unmap 0.01 ±106% -100.0% 0.00 perf-sched.sch_delay.avg.ms.rwsem_down_read_slowpath.rmap_walk_anon.remove_migration_ptes.migrate_pages 0.01 ± 49% +1115.0% 0.14 ±143% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.__vma_adjust.__split_vma.__do_munmap 0.03 ± 52% -89.4% 0.00 ±107% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.anon_vma_clone.anon_vma_fork.dup_mmap 0.02 ± 54% +1681.5% 0.39 ± 70% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.01 ± 80% +507.1% 0.09 ± 45% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process 0.02 ± 24% +1250.0% 0.26 ± 68% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.filename_create.do_linkat.__x64_sys_link 0.07 ± 65% -87.3% 0.01 ± 75% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.unlink_anon_vmas.free_pgtables.exit_mmap 0.02 ± 20% +232.6% 0.05 ± 33% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap 0.01 ± 27% +363.9% 0.04 ± 43% perf-sched.sch_delay.avg.ms.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap 0.19 ± 68% -95.1% 0.01 ± 13% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.04 ± 18% +5366.5% 1.93 ± 40% perf-sched.sch_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 0.04 ± 39% +464.8% 0.24 ± 60% perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.00 ±141% +48300.0% 0.97 ±170% perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all 0.20 ± 7% +1238.1% 2.63 ± 31% perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 0.12 ± 27% -74.7% 0.03 ±149% perf-sched.sch_delay.avg.ms.wait_for_partner.fifo_open.do_dentry_open.path_openat 0.01 ± 7% +9784.2% 1.25 ± 18% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork 5.01 ± 31% +324.7% 21.26 ± 36% perf-sched.sch_delay.max.ms.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk.part 3.59 ±132% +1076.4% 42.25 ± 96% perf-sched.sch_delay.max.ms.d_alloc_parallel.__lookup_slow.walk_component.path_lookupat.isra 16.19 ± 7% +1397.5% 242.51 ± 30% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 9.13 ± 27% +2518.1% 238.94 ± 62% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 10.04 ± 65% +1971.0% 207.85 ± 63% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown] 9.21 ± 61% +2000.8% 193.58 ± 63% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 11.01 ± 18% +799.2% 99.02 ± 86% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 14.73 ± 13% +265.9% 53.91 ± 26% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork.[unknown] 10.65 ± 44% +151.1% 26.76 ± 19% perf-sched.sch_delay.max.ms.io_schedule.__lock_page_killable.filemap_fault.__do_fault 1.21 ± 24% +2921.4% 36.62 ±124% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.53 ± 55% +1359.3% 7.76 ±113% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.alloc_pages_vma.wp_page_copy 0.30 ± 92% -90.7% 0.03 ±155% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.allocate_slab.___slab_alloc 0.37 ± 69% +3449.8% 13.07 ±162% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.pte_alloc_one.__pte_alloc 0.48 ± 37% -89.3% 0.05 ±102% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.__anon_vma_prepare.do_fault.__handle_mm_fault 0.55 ± 93% -95.3% 0.03 ±153% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.__kmalloc_node.memcg_alloc_page_obj_cgroups.kmem_cache_alloc_node 0.97 ± 13% +591.3% 6.72 ±119% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.change_p4d_range.change_protection.mprotect_fixup 0.31 ±106% -98.6% 0.00 ±137% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.copy_page_to_iter.generic_file_buffered_read.new_sync_read 11.49 ± 5% +87.8% 21.57 ± 31% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 0.32 ±102% -96.0% 0.01 ± 79% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.down_write.__anon_vma_prepare.do_anonymous_page 1.05 ± 98% +478.9% 6.10 ± 82% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.down_write.__vma_adjust.__split_vma 0.02 ± 5% +1.4e+06% 217.29 ±104% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.down_write.do_truncate.path_openat 1.95 ± 26% +682.5% 15.25 ± 64% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.down_write.unlink_anon_vmas.free_pgtables 0.93 ± 14% +9721.3% 91.57 ±158% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.down_write.unlink_file_vma.free_pgtables 0.00 ± 10% -78.6% 0.00 ±173% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.down_write_killable.__vm_munmap.__x64_sys_munmap 0.24 ± 43% -96.7% 0.01 ± 72% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.down_write_killable.vm_mmap_pgoff.ksys_mmap_pgoff 0.01 ± 28% +8.3e+05% 113.60 ±164% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.dput.do_unlinkat.do_syscall_64 0.21 ±108% +34647.0% 72.97 ±145% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.dput.done_path_create.do_linkat 3.03 ± 39% +1407.5% 45.73 ± 29% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.dput.path_openat.do_filp_open 1.22 ± 39% +4068.4% 50.91 ± 79% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.dput.step_into.path_openat 3.38 ± 60% +3700.6% 128.54 ±127% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.dput.terminate_walk.path_openat 0.88 ± 46% +299.6% 3.50 ± 55% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.exit_mmap.mmput.begin_new_exec 0.00 ±141% +22250.0% 0.30 ±167% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.generic_file_buffered_read.__kernel_read.load_elf_binary 0.57 ±139% +12527.6% 71.47 ± 89% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__d_alloc.d_alloc_cursor 0.00 ±141% +1525.0% 0.03 ±144% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__delayacct_tsk_init.copy_process 1.73 ± 43% +863.5% 16.71 ± 50% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.security_file_alloc.__alloc_file 3.67 ±140% -100.0% 0.00 ±173% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc_node_trace.__get_vm_area_node.__vmalloc_node_range 3.19 ± 52% +345.8% 14.24 ± 84% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc_trace.perf_event_mmap.mmap_region 1.01 ± 99% -91.8% 0.08 ±167% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc_trace.perf_event_mmap.mprotect_fixup 0.11 ± 85% +1.7e+05% 187.38 ± 84% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.mnt_want_write.path_openat.do_filp_open 0.05 ±130% +602.6% 0.36 ±142% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.mutex_lock.futex_exec_release.exec_mm_release 0.52 ±136% +10794.6% 57.09 ± 72% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 0.06 ± 92% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate 28.50 ± 12% +645.4% 212.48 ± 39% perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.sched_exec.bprm_execve 1.20 ± 76% -100.0% 0.00 perf-sched.sch_delay.max.ms.preempt_schedule_common._cond_resched.wait_for_completion.stop_two_cpus.migrate_swap 0.18 ±130% -100.0% 0.00 perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.page_lock_anon_vma_read.rmap_walk_anon.try_to_unmap 0.20 ±132% -100.0% 0.00 perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.rmap_walk_anon.remove_migration_ptes.migrate_pages 1.33 ±103% -96.9% 0.04 ±173% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.__put_anon_vma.unlink_anon_vmas.free_pgtables 1.53 ± 79% +6359.8% 98.51 ±148% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.__vma_adjust.__split_vma.__do_munmap 2.57 ± 63% +676.1% 19.92 ± 49% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.__vma_adjust.__split_vma.mprotect_fixup 0.78 ± 76% -99.2% 0.01 ±138% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.anon_vma_clone.anon_vma_fork.dup_mmap 0.19 ± 94% -97.6% 0.00 ±100% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.anon_vma_fork.dup_mmap.dup_mm 5.70 ± 45% +1878.3% 112.68 ± 89% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.68 ±111% +1530.5% 60.07 ± 26% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process 6.33 ± 17% +2353.2% 155.41 ±105% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.filename_create.do_linkat.__x64_sys_link 2.23 ± 66% -98.3% 0.04 ± 65% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.unlink_anon_vmas.free_pgtables.exit_mmap 6.50 ± 41% +643.0% 48.28 ± 78% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap 2.45 ± 43% +1688.6% 43.80 ± 88% perf-sched.sch_delay.max.ms.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap 9.31 ± 80% -99.6% 0.04 ± 54% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.52 ± 65% +15185.6% 79.99 ± 38% perf-sched.sch_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 16.34 ± 98% +385.7% 79.39 ± 79% perf-sched.sch_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.00 ±141% +72153.1% 1.93 ±171% perf-sched.sch_delay.max.ms.schedule_timeout.wait_for_completion.__flush_work.lru_add_drain_all 10.28 ± 58% +2363.0% 253.08 ± 17% perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork 8.41 ± 37% +3406.7% 295.08 ± 29% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork 0.12 ± 74% +358.2% 0.54 ± 29% perf-sched.total_sch_delay.average.ms 39.70 ± 3% -15.2% 33.67 ± 3% perf-sched.total_wait_and_delay.average.ms 157875 ± 3% +29.4% 204320 ± 3% perf-sched.total_wait_and_delay.count.ms 5585 +14.5% 6395 ± 3% perf-sched.total_wait_and_delay.max.ms 39.58 ± 3% -16.3% 33.13 ± 3% perf-sched.total_wait_time.average.ms 5585 +14.5% 6395 ± 3% perf-sched.total_wait_time.max.ms 6.67 ± 2% +90.8% 12.72 ± 10% perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 6.25 ± 22% +706.2% 50.40 ± 54% perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 6.09 ± 15% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 38.34 ± 13% +47.7% 56.61 ± 33% perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 5.09 +14.7% 5.83 ± 3% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 0.33 ± 3% +330.6% 1.42 ± 24% perf-sched.wait_and_delay.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.sched_exec.bprm_execve 1.33 ± 20% +419.5% 6.93 ± 7% perf-sched.wait_and_delay.avg.ms.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.60 ± 70% +168.1% 1.61 ± 18% perf-sched.wait_and_delay.avg.ms.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process 3.71 ± 11% +29.9% 4.82 ± 12% perf-sched.wait_and_delay.avg.ms.rwsem_down_write_slowpath.path_openat.do_filp_open.do_sys_openat2 0.04 ±141% +440.1% 0.24 ± 12% perf-sched.wait_and_delay.avg.ms.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.unmap_region 0.06 ±141% +604.1% 0.41 ± 20% perf-sched.wait_and_delay.avg.ms.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap 251.31 ± 7% -36.4% 159.76 ± 18% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 87.92 ± 14% +50.7% 132.47 ± 22% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 384.15 -32.3% 260.23 ± 10% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 142.29 -30.8% 98.47 ± 6% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork 35.67 ± 11% +58.4% 56.50 perf-sched.wait_and_delay.count.devkmsg_read.vfs_read.ksys_read.do_syscall_64 14.67 ± 8% +19.3% 17.50 ± 4% perf-sched.wait_and_delay.count.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64 35.67 ± 9% +58.4% 56.50 perf-sched.wait_and_delay.count.do_syslog.part.0.kmsg_read.vfs_read 39991 +28.3% 51295 ± 4% perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 41076 +26.8% 52088 ± 4% perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 914.67 ± 2% +68.3% 1539 ± 6% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 122.33 ±141% +401.1% 613.00 ± 22% perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 1649 ± 8% -100.0% 0.00 perf-sched.wait_and_delay.count.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 4728 ± 13% -91.3% 412.00 ±100% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.dput.scan_positives.dcache_readdir 4.67 ± 44% +1325.0% 66.50 ± 51% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 1987 ± 4% -13.4% 1720 ± 4% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 3064 ± 8% +76.6% 5412 ± 13% perf-sched.wait_and_delay.count.preempt_schedule_common._cond_resched.stop_one_cpu.sched_exec.bprm_execve 2046 ± 7% +68.3% 3443 ± 15% perf-sched.wait_and_delay.count.rwsem_down_read_slowpath.walk_component.path_lookupat.isra.0 896.33 ± 5% +81.3% 1625 ± 12% perf-sched.wait_and_delay.count.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 776.00 ± 75% +292.9% 3049 ± 44% perf-sched.wait_and_delay.count.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process 1649 ± 32% +198.7% 4926 ± 45% perf-sched.wait_and_delay.count.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap 354.67 ±141% +482.9% 2067 ± 36% perf-sched.wait_and_delay.count.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.unmap_region 350.00 ±141% +496.1% 2086 ± 33% perf-sched.wait_and_delay.count.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap 13.00 ± 21% +61.5% 21.00 perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 163.67 ± 3% -35.2% 106.00 ± 25% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 1762 ± 4% -17.5% 1454 ± 12% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 9129 ± 2% +43.1% 13062 ± 9% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork 12642 +52.2% 19242 ± 3% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork 5512 -30.9% 3811 perf-sched.wait_and_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 5512 -30.9% 3811 perf-sched.wait_and_delay.max.ms.do_syslog.part.0.kmsg_read.vfs_read 340.62 ±141% +203.4% 1033 perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi.[unknown] 5496 -100.0% 0.00 perf-sched.wait_and_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 5580 -30.1% 3899 perf-sched.wait_and_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 1495 ± 31% +114.1% 3203 ± 4% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 28.50 ± 12% +645.4% 212.48 ± 39% perf-sched.wait_and_delay.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.sched_exec.bprm_execve 20.64 ± 78% +2946.5% 628.69 ± 62% perf-sched.wait_and_delay.max.ms.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process 20.38 ± 28% +299.0% 81.32 ± 53% perf-sched.wait_and_delay.max.ms.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap 3.90 ±141% +526.7% 24.46 ± 34% perf-sched.wait_and_delay.max.ms.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.unmap_region 3.78 ±141% +1570.8% 63.11 ± 45% perf-sched.wait_and_delay.max.ms.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap 5176 -31.0% 3569 perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 5585 -30.2% 3899 perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 505.28 +31.3% 663.68 ± 9% perf-sched.wait_and_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 4817 ± 15% -31.2% 3315 ± 3% perf-sched.wait_and_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork 1791 -16.1% 1502 ± 3% perf-sched.wait_and_delay.max.ms.worker_thread.kthread.ret_from_fork 0.08 ± 39% +460.9% 0.43 ± 27% perf-sched.wait_time.avg.ms.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk.part 0.17 ± 92% +182.7% 0.48 ± 59% perf-sched.wait_time.avg.ms.d_alloc_parallel.__lookup_slow.walk_component.path_lookupat.isra 6.50 ± 3% +87.7% 12.20 ± 8% perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 0.59 ±123% +129.1% 1.35 ± 86% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault.[unknown] 6.21 ± 22% +697.5% 49.49 ± 55% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt.[unknown] 8.60 ± 95% +175.7% 23.70 ± 34% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown] 6.03 ± 15% +354.9% 27.43 ± 13% perf-sched.wait_time.avg.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 0.00 ± 88% +24178.6% 1.13 ±156% perf-sched.wait_time.avg.ms.io_schedule.__lock_page.__do_fault.do_fault 38.33 ± 13% +47.6% 56.59 ± 33% perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.53 ±120% -98.4% 0.01 ±102% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__alloc_pages_nodemask.__pmd_alloc.__handle_mm_fault 0.01 ±115% +6617.2% 0.36 ±168% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__kmalloc.load_elf_phdrs.load_elf_binary 0.42 ± 44% +593.4% 2.89 ± 57% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__kmalloc.security_task_alloc.copy_process 0.08 ± 57% +614.6% 0.60 ± 48% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables 0.27 ±140% +525.8% 1.67 ± 86% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.cgroup_can_fork.copy_process.kernel_clone 0.05 ± 25% +978.8% 0.50 ± 50% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.change_p4d_range.change_protection.mprotect_fixup 0.53 ± 73% +685.8% 4.14 ± 66% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.copy_pte_range.copy_page_range.dup_mmap 0.02 ±123% +6830.4% 1.59 ±103% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_read.path_openat.do_filp_open 0.06 ± 29% +796.1% 0.51 ± 93% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_read.walk_component.link_path_walk 0.01 ±141% +2254.2% 0.14 ±161% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_read_killable.create_elf_tables.isra 0.01 ± 97% +9626.4% 1.13 ± 62% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write.__anon_vma_prepare.do_fault 0.02 ± 87% +4118.8% 0.79 ± 55% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write.__vma_adjust.shift_arg_pages 1.50 ± 68% +1104.7% 18.05 ±100% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write.anon_vma_fork.dup_mmap 2.02 ± 13% +454.8% 11.23 ± 69% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write.do_truncate.path_openat 1.78 ± 66% +587.5% 12.24 ± 99% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write.dup_mmap.dup_mm 0.86 ± 39% +37.3% 1.18 ± 11% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write.unlink_file_vma.free_pgtables 0.01 ±141% +3608.3% 0.22 ± 85% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write_killable.__vm_munmap.elf_map 0.01 ±135% +539.5% 0.08 ± 83% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write_killable.vm_brk_flags.load_elf_interp 0.04 ± 21% +251.3% 0.14 ± 50% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.down_write_killable.vm_mmap_pgoff.elf_map 0.25 ± 70% +1512.8% 4.09 ±132% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.dput.__fput.task_work_run 0.18 ±139% +1040.9% 2.06 ± 93% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.dput.dcache_dir_close.__fput 0.32 ± 75% +3385.1% 11.30 ±101% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.dput.do_unlinkat.do_syscall_64 9.06 ± 10% +226.6% 29.60 ± 20% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.dput.scan_positives.dcache_readdir 2.19 ±119% +216.9% 6.95 ± 21% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.dput.step_into.path_openat 0.16 ± 19% +1845.7% 3.15 ± 29% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.dput.terminate_walk.path_openat 0.03 ± 70% +2914.6% 0.82 ± 63% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.exit_mmap.mmput.begin_new_exec 0.97 ± 23% +131.5% 2.24 ± 56% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.exit_mmap.mmput.do_exit 0.01 ±126% +319.3% 0.06 ± 53% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__anon_vma_prepare.do_anonymous_page 0.04 ± 15% +2327.3% 0.89 ± 54% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__d_alloc.d_alloc 0.62 ± 70% +3428.4% 21.95 ± 35% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__d_alloc.d_alloc_cursor 0.29 ± 72% +1816.0% 5.56 ± 89% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.copy_fs_struct.copy_process 0.05 ±119% +1698.6% 0.95 ± 91% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.getname_flags.do_sys_openat2 0.18 ± 72% +533.7% 1.16 ± 31% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.getname_flags.user_path_at_empty 0.41 ± 34% +890.7% 4.09 ± 49% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.prepare_creds.copy_creds 3.70 ± 18% +55.5% 5.76 ± 42% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.vm_area_dup.dup_mmap 10.64 ±105% +189.5% 30.81 ± 66% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.mnt_want_write.path_openat.do_filp_open 0.10 ± 68% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.mutex_lock_killable.begin_new_exec.load_elf_binary 0.05 ± 97% +1166.8% 0.58 ± 67% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.remove_vma.__do_munmap.__vm_munmap 5.09 +14.7% 5.83 ± 3% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.__set_cpus_allowed_ptr.sched_setaffinity 2.90 ± 20% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate 0.03 ± 5% +539.8% 0.20 ± 26% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.stop_one_cpu.sched_exec.bprm_execve 2.38 ± 18% -100.0% 0.00 perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.wait_for_completion.stop_two_cpus.migrate_swap 0.23 ± 2% +87.1% 0.44 ± 35% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.zap_pte_range.unmap_page_range.unmap_vmas 2.62 ± 44% -100.0% 0.00 perf-sched.wait_time.avg.ms.rwsem_down_read_slowpath.page_lock_anon_vma_read.rmap_walk_anon.try_to_unmap 2.41 ± 41% -100.0% 0.00 perf-sched.wait_time.avg.ms.rwsem_down_read_slowpath.rmap_walk_anon.remove_migration_ptes.migrate_pages 0.17 ± 4% +96.1% 0.34 ± 48% perf-sched.wait_time.avg.ms.rwsem_down_write_slowpath.__vma_adjust.__split_vma.mprotect_fixup 1.31 ± 21% +398.6% 6.54 ± 7% perf-sched.wait_time.avg.ms.rwsem_down_write_slowpath.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.93 ± 7% +64.0% 1.53 ± 16% perf-sched.wait_time.avg.ms.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process 0.17 ± 14% +31.3% 0.22 ± 13% perf-sched.wait_time.avg.ms.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.unmap_region 0.20 ± 15% +83.9% 0.37 ± 25% perf-sched.wait_time.avg.ms.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap 251.12 ± 7% -36.4% 159.75 ± 18% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 87.92 ± 14% +50.6% 132.37 ± 23% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 383.96 -32.9% 257.59 ± 9% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork 142.28 -31.7% 97.22 ± 6% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork 8.16 ± 46% +383.8% 39.47 ± 75% perf-sched.wait_time.max.ms.d_alloc_parallel.__lookup_slow.walk_component.link_path_walk.part 5512 -30.9% 3811 perf-sched.wait_time.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64 5512 -30.9% 3811 perf-sched.wait_time.max.ms.do_syslog.part.0.kmsg_read.vfs_read 5496 -30.6% 3813 perf-sched.wait_time.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 0.07 ±107% +29156.2% 21.16 ±159% perf-sched.wait_time.max.ms.io_schedule.__lock_page.__do_fault.do_fault 5580 -30.1% 3899 perf-sched.wait_time.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 4.62 ± 40% +343.5% 20.48 ± 14% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__kmalloc.security_task_alloc.copy_process 2.87 ± 85% +498.7% 17.16 ± 34% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables 1.03 ±140% +525.1% 6.42 ± 54% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.cgroup_can_fork.copy_process.kernel_clone 0.85 ± 22% +2220.1% 19.71 ± 82% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.change_p4d_range.change_protection.mprotect_fixup 7.98 ± 62% +586.9% 54.83 ±100% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.copy_pte_range.copy_page_range.dup_mmap 0.42 ± 73% +476.2% 2.44 ± 95% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.copy_strings.isra.0 0.18 ±131% +4103.5% 7.45 ± 88% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_read.path_openat.do_filp_open 2.68 ± 50% +1905.1% 53.65 ±123% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_read.walk_component.link_path_walk 0.04 ±141% +3772.9% 1.39 ±166% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_read_killable.create_elf_tables.isra 0.03 ±122% +22698.3% 6.76 ± 88% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_write.__anon_vma_prepare.do_fault 0.25 ±103% +3843.4% 9.87 ± 62% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_write.__vma_adjust.shift_arg_pages 23.74 ± 13% +1153.9% 297.72 ±136% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_write.anon_vma_clone.anon_vma_fork 8.46 ± 76% +3275.9% 285.49 ±146% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_write.anon_vma_fork.dup_mmap 8.55 ± 14% +2664.7% 236.39 ± 90% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_write.do_truncate.path_openat 0.02 ±141% +3120.9% 0.78 ± 90% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_write_killable.__vm_munmap.elf_map 0.63 ± 47% +456.1% 3.51 ± 76% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.down_write_killable.vm_mmap_pgoff.elf_map 0.36 ±139% +758.2% 3.10 ± 66% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.dcache_dir_close.__fput 1.91 ± 89% +6827.3% 132.20 ±137% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.do_unlinkat.do_syscall_64 336.18 ±140% +73.0% 581.60 ± 78% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.done_path_create.do_linkat 341.13 ±137% +196.8% 1012 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.step_into.path_openat 9.64 ± 52% +4704.9% 463.29 ± 72% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.terminate_walk.path_openat 0.94 ± 71% +1718.2% 17.05 ± 33% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.exit_mmap.mmput.begin_new_exec 6.94 ± 41% +683.1% 54.36 ±113% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.exit_mmap.mmput.do_exit 0.07 ±119% +888.4% 0.65 ± 80% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__anon_vma_prepare.do_anonymous_page 0.42 ± 51% +3060.9% 13.37 ± 71% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__d_alloc.d_alloc 1.61 ± 71% +29984.9% 484.07 ± 65% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.__d_alloc.d_alloc_cursor 1.01 ± 76% +779.3% 8.90 ± 68% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.copy_fs_struct.copy_process 1.25 ± 73% +327.5% 5.36 ± 28% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.getname_flags.user_path_at_empty 23.94 ± 16% +259.0% 85.92 ± 53% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc.vm_area_dup.dup_mmap 6.01 ± 27% +52.0% 9.13 ± 24% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.kmem_cache_alloc_trace.perf_event_mmap.mmap_region 0.27 ± 77% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.mutex_lock_killable.begin_new_exec.load_elf_binary 0.41 ± 97% +1470.1% 6.36 ± 97% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.remove_vma.__do_munmap.__vm_munmap 2.81 ± 79% +146.3% 6.92 ± 44% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.remove_vma.__do_munmap.mmap_region 7.51 ± 23% +226.7% 24.54 ± 98% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.remove_vma.exit_mmap.mmput 7.36 ±116% +497.6% 44.00 ± 94% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.shmem_undo_range.shmem_truncate_range.shmem_evict_inode 1495 ± 31% +114.1% 3203 ± 4% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.smpboot_thread_fn.kthread.ret_from_fork 9.21 ± 22% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.migrate_task_to.task_numa_migrate 12.64 ± 2% +300.8% 50.66 ± 43% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.stop_one_cpu.sched_exec.bprm_execve 7.61 ± 29% -100.0% 0.00 perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.wait_for_completion.stop_two_cpus.migrate_swap 20.69 ± 50% -100.0% 0.00 perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.page_lock_anon_vma_read.rmap_walk_anon.try_to_unmap 19.13 ± 36% -100.0% 0.00 perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.rmap_walk_anon.remove_migration_ptes.migrate_pages 2.03 ± 55% -96.2% 0.08 ±151% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.__put_anon_vma.unlink_anon_vmas.free_pgtables 7.53 ± 25% +397.1% 37.43 ± 76% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.__vma_adjust.__split_vma.mprotect_fixup 31.44 ± 23% +1888.2% 625.17 ± 62% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process 2.32 ± 99% -88.3% 0.27 ± 87% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.unlink_anon_vmas.free_pgtables.exit_mmap 20.37 ± 28% +184.7% 57.99 ± 80% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap 11.76 ± 17% +170.3% 31.80 ± 43% perf-sched.wait_time.max.ms.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap 5176 -31.0% 3569 perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 5585 -30.2% 3899 perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 505.14 +15.5% 583.69 ± 5% perf-sched.wait_time.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork 4816 ± 15% -31.5% 3297 ± 3% perf-sched.wait_time.max.ms.smpboot_thread_fn.kthread.ret_from_fork 1791 -20.7% 1420 ± 3% perf-sched.wait_time.max.ms.worker_thread.kthread.ret_from_fork 29455 ± 2% +33.9% 39449 ± 7% softirqs.CPU0.NET_RX 25470 +32.0% 33628 softirqs.CPU0.SCHED 26529 ± 11% +43.7% 38110 ± 9% softirqs.CPU1.NET_RX 22797 ± 3% +43.8% 32782 ± 3% softirqs.CPU1.SCHED 28371 ± 3% +54.2% 43762 ± 13% softirqs.CPU10.NET_RX 22629 ± 2% +43.4% 32439 ± 2% softirqs.CPU10.SCHED 27171 ± 4% +38.4% 37593 ± 9% softirqs.CPU100.NET_RX 23180 ± 2% +39.1% 32248 softirqs.CPU100.SCHED 28196 ± 2% +34.4% 37895 ± 5% softirqs.CPU101.NET_RX 23100 +40.7% 32511 ± 2% softirqs.CPU101.SCHED 27883 ± 3% +43.5% 40015 ± 11% softirqs.CPU102.NET_RX 22937 ± 2% +41.4% 32444 ± 2% softirqs.CPU102.SCHED 28037 +40.7% 39461 ± 7% softirqs.CPU103.NET_RX 23000 ± 2% +41.0% 32435 ± 2% softirqs.CPU103.SCHED 28345 +34.9% 38229 ± 9% softirqs.CPU104.NET_RX 22314 ± 3% +45.3% 32417 ± 2% softirqs.CPU104.SCHED 27097 ± 2% +40.7% 38114 ± 3% softirqs.CPU105.NET_RX 22615 ± 2% +42.8% 32294 ± 2% softirqs.CPU105.SCHED 26463 ± 3% +50.9% 39943 ± 12% softirqs.CPU106.NET_RX 22730 ± 2% +42.2% 32319 ± 2% softirqs.CPU106.SCHED 27957 ± 2% +29.0% 36058 ± 6% softirqs.CPU107.NET_RX 22901 ± 2% +41.3% 32362 ± 2% softirqs.CPU107.SCHED 29320 ± 8% +33.4% 39101 ± 10% softirqs.CPU108.NET_RX 22873 ± 2% +41.2% 32301 ± 2% softirqs.CPU108.SCHED 29106 ± 2% +31.5% 38266 ± 3% softirqs.CPU109.NET_RX 22591 ± 2% +43.1% 32336 softirqs.CPU109.SCHED 28741 ± 5% +34.6% 38676 ± 6% softirqs.CPU11.NET_RX 22399 ± 3% +44.5% 32364 ± 2% softirqs.CPU11.SCHED 26804 ± 5% +41.8% 38006 ± 5% softirqs.CPU110.NET_RX 22766 ± 2% +43.3% 32619 ± 2% softirqs.CPU110.SCHED 25623 ± 7% +45.9% 37393 ± 9% softirqs.CPU111.NET_RX 22586 ± 2% +43.4% 32391 ± 2% softirqs.CPU111.SCHED 28020 ± 6% +31.8% 36917 ± 7% softirqs.CPU112.NET_RX 22741 +42.2% 32349 ± 2% softirqs.CPU112.SCHED 28310 ± 7% +35.1% 38234 ± 12% softirqs.CPU113.NET_RX 22884 ± 2% +42.4% 32577 ± 2% softirqs.CPU113.SCHED 25668 ± 5% +45.9% 37457 ± 7% softirqs.CPU114.NET_RX 22706 ± 3% +41.7% 32178 softirqs.CPU114.SCHED 28637 ± 5% +37.6% 39418 ± 6% softirqs.CPU115.NET_RX 22796 +41.0% 32139 softirqs.CPU115.SCHED 26953 ± 7% +42.1% 38291 ± 7% softirqs.CPU116.NET_RX 22646 +42.1% 32171 ± 2% softirqs.CPU116.SCHED 28477 ± 8% +35.4% 38544 ± 3% softirqs.CPU117.NET_RX 22866 ± 2% +40.8% 32189 ± 2% softirqs.CPU117.SCHED 27445 ± 7% +40.9% 38675 ± 11% softirqs.CPU118.NET_RX 22902 +41.4% 32386 ± 2% softirqs.CPU118.SCHED 26884 ± 4% +38.3% 37182 ± 6% softirqs.CPU119.NET_RX 22721 ± 2% +42.0% 32265 ± 2% softirqs.CPU119.SCHED 28376 ± 9% +36.6% 38767 ± 9% softirqs.CPU12.NET_RX 22706 ± 3% +40.4% 31877 ± 2% softirqs.CPU12.SCHED 27315 ± 7% +45.1% 39644 ± 4% softirqs.CPU120.NET_RX 23157 ± 2% +38.9% 32175 softirqs.CPU120.SCHED 26387 ± 5% +51.6% 40015 ± 4% softirqs.CPU121.NET_RX 22856 +38.5% 31646 softirqs.CPU121.SCHED 25176 ± 7% +67.6% 42194 ± 4% softirqs.CPU122.NET_RX 22775 +40.3% 31962 softirqs.CPU122.SCHED 25636 ± 5% +60.8% 41225 ± 7% softirqs.CPU123.NET_RX 23322 +37.7% 32124 softirqs.CPU123.SCHED 29777 ± 2% +37.4% 40911 ± 3% softirqs.CPU124.NET_RX 23141 ± 2% +38.7% 32098 softirqs.CPU124.SCHED 25720 ± 7% +53.0% 39356 ± 4% softirqs.CPU125.NET_RX 23140 ± 3% +38.7% 32104 softirqs.CPU125.SCHED 27106 ± 2% +44.7% 39214 ± 2% softirqs.CPU126.NET_RX 33141 ± 3% +11.4% 36924 ± 7% softirqs.CPU126.RCU 23154 ± 2% +38.6% 32083 softirqs.CPU126.SCHED 25748 ± 4% +53.8% 39601 ± 8% softirqs.CPU127.NET_RX 23011 ± 2% +39.4% 32068 softirqs.CPU127.SCHED 28647 ± 11% +42.1% 40718 softirqs.CPU128.NET_RX 34186 ± 3% +11.6% 38137 ± 7% softirqs.CPU128.RCU 23173 ± 2% +38.2% 32016 softirqs.CPU128.SCHED 27101 ± 3% +48.6% 40259 ± 5% softirqs.CPU129.NET_RX 22818 +40.4% 32045 softirqs.CPU129.SCHED 31534 ± 12% +25.9% 39696 ± 10% softirqs.CPU13.NET_RX 22674 ± 2% +41.6% 32108 ± 2% softirqs.CPU13.SCHED 27452 +48.9% 40871 ± 2% softirqs.CPU130.NET_RX 23203 +37.1% 31807 ± 2% softirqs.CPU130.SCHED 26896 ± 3% +51.9% 40860 ± 4% softirqs.CPU131.NET_RX 23148 ± 2% +37.4% 31814 softirqs.CPU131.SCHED 28393 ± 3% +41.2% 40084 ± 7% softirqs.CPU132.NET_RX 23200 +36.7% 31706 softirqs.CPU132.SCHED 26482 ± 3% +54.4% 40891 ± 3% softirqs.CPU133.NET_RX 22970 +41.0% 32382 ± 2% softirqs.CPU133.SCHED 26774 ± 7% +44.7% 38735 ± 5% softirqs.CPU134.NET_RX 33646 +10.7% 37240 ± 7% softirqs.CPU134.RCU 23088 ± 2% +38.6% 32008 softirqs.CPU134.SCHED 26666 +54.1% 41081 ± 7% softirqs.CPU135.NET_RX 33456 ± 4% +14.0% 38141 ± 8% softirqs.CPU135.RCU 22804 +40.0% 31925 softirqs.CPU135.SCHED 26562 ± 5% +55.4% 41276 ± 7% softirqs.CPU136.NET_RX 23134 ± 3% +38.9% 32141 softirqs.CPU136.SCHED 25297 ± 4% +62.9% 41210 ± 6% softirqs.CPU137.NET_RX 22846 ± 2% +39.9% 31955 softirqs.CPU137.SCHED 26519 ± 6% +51.6% 40192 ± 4% softirqs.CPU138.NET_RX 22478 ± 2% +41.7% 31858 softirqs.CPU138.SCHED 26079 ± 9% +62.0% 42258 ± 2% softirqs.CPU139.NET_RX 22994 ± 3% +39.2% 32007 softirqs.CPU139.SCHED 26936 ± 2% +43.0% 38512 ± 9% softirqs.CPU14.NET_RX 22604 ± 3% +42.3% 32166 ± 3% softirqs.CPU14.SCHED 27625 ± 9% +44.2% 39847 ± 5% softirqs.CPU140.NET_RX 33066 ± 5% +14.6% 37895 ± 8% softirqs.CPU140.RCU 22609 +41.6% 32006 softirqs.CPU140.SCHED 26460 ± 5% +58.1% 41826 ± 5% softirqs.CPU141.NET_RX 34319 ± 4% +9.6% 37601 ± 8% softirqs.CPU141.RCU 22848 ± 2% +39.5% 31868 softirqs.CPU141.SCHED 27385 ± 8% +52.1% 41645 ± 6% softirqs.CPU142.NET_RX 22881 ± 2% +40.0% 32033 softirqs.CPU142.SCHED 27724 ± 3% +46.8% 40700 ± 5% softirqs.CPU143.NET_RX 34109 ± 4% +10.8% 37789 ± 8% softirqs.CPU143.RCU 23092 ± 2% +39.1% 32110 softirqs.CPU143.SCHED 28902 ± 8% +30.5% 37711 softirqs.CPU144.NET_RX 34885 ± 7% +8.2% 37745 ± 7% softirqs.CPU144.RCU 22671 +40.8% 31929 softirqs.CPU144.SCHED 29181 ± 10% +43.3% 41814 ± 7% softirqs.CPU145.NET_RX 22655 +38.5% 31371 ± 2% softirqs.CPU145.SCHED 28250 ± 7% +33.9% 37815 ± 5% softirqs.CPU146.NET_RX 22502 +42.7% 32115 softirqs.CPU146.SCHED 26352 ± 5% +44.3% 38033 ± 4% softirqs.CPU147.NET_RX 34098 +9.1% 37212 ± 5% softirqs.CPU147.RCU 22712 +39.8% 31759 softirqs.CPU147.SCHED 26457 ± 5% +47.7% 39077 ± 2% softirqs.CPU148.NET_RX 34771 +10.8% 38542 ± 5% softirqs.CPU148.RCU 22626 +41.1% 31925 softirqs.CPU148.SCHED 27813 ± 3% +38.2% 38431 ± 3% softirqs.CPU149.NET_RX 34785 ± 3% +9.4% 38071 ± 6% softirqs.CPU149.RCU 22830 +39.5% 31857 softirqs.CPU149.SCHED 27674 +35.0% 37356 ± 10% softirqs.CPU15.NET_RX 22708 ± 3% +40.8% 31984 ± 2% softirqs.CPU15.SCHED 28914 ± 4% +38.6% 40073 ± 5% softirqs.CPU150.NET_RX 22867 +40.2% 32054 softirqs.CPU150.SCHED 26654 +46.9% 39145 ± 4% softirqs.CPU151.NET_RX 35269 ± 5% +7.1% 37772 ± 5% softirqs.CPU151.RCU 22567 +41.7% 31987 softirqs.CPU151.SCHED 27340 ± 3% +35.5% 37050 ± 7% softirqs.CPU152.NET_RX 22561 +41.1% 31833 softirqs.CPU152.SCHED 25682 ± 4% +52.9% 39264 ± 3% softirqs.CPU153.NET_RX 33854 ± 3% +13.6% 38450 ± 5% softirqs.CPU153.RCU 22484 +42.2% 31972 softirqs.CPU153.SCHED 27824 ± 6% +45.0% 40343 softirqs.CPU154.NET_RX 34996 ± 2% +9.1% 38165 ± 5% softirqs.CPU154.RCU 22409 +41.5% 31717 softirqs.CPU154.SCHED 26948 ± 6% +51.9% 40940 ± 3% softirqs.CPU155.NET_RX 35069 +10.6% 38779 ± 5% softirqs.CPU155.RCU 22696 +42.2% 32266 softirqs.CPU155.SCHED 26787 ± 3% +45.7% 39032 ± 4% softirqs.CPU156.NET_RX 22583 +41.3% 31902 softirqs.CPU156.SCHED 26065 ± 2% +47.8% 38529 ± 4% softirqs.CPU157.NET_RX 22442 +42.0% 31866 softirqs.CPU157.SCHED 27310 ± 3% +42.2% 38826 ± 8% softirqs.CPU158.NET_RX 22593 +40.5% 31752 softirqs.CPU158.SCHED 26411 ± 4% +47.7% 39018 softirqs.CPU159.NET_RX 22535 +42.5% 32103 softirqs.CPU159.SCHED 28586 ± 3% +30.9% 37421 ± 8% softirqs.CPU16.NET_RX 22668 ± 3% +45.0% 32879 ± 3% softirqs.CPU16.SCHED 27341 ± 2% +39.4% 38105 ± 2% softirqs.CPU160.NET_RX 22494 +41.5% 31829 softirqs.CPU160.SCHED 26309 +47.7% 38870 ± 6% softirqs.CPU161.NET_RX 22652 +41.1% 31964 softirqs.CPU161.SCHED 28102 ± 3% +35.3% 38031 ± 3% softirqs.CPU162.NET_RX 34995 ± 4% +6.5% 37255 ± 7% softirqs.CPU162.RCU 22800 +40.4% 32000 softirqs.CPU162.SCHED 24958 ± 5% +53.9% 38412 ± 7% softirqs.CPU163.NET_RX 22582 +42.1% 32079 softirqs.CPU163.SCHED 27327 ± 2% +36.9% 37415 ± 4% softirqs.CPU164.NET_RX 22516 +43.0% 32204 softirqs.CPU164.SCHED 27456 ± 5% +46.4% 40188 ± 4% softirqs.CPU165.NET_RX 22509 +42.6% 32104 softirqs.CPU165.SCHED 26840 ± 3% +42.0% 38118 ± 5% softirqs.CPU166.NET_RX 22847 +40.1% 32002 softirqs.CPU166.SCHED 28288 ± 2% +45.7% 41208 ± 8% softirqs.CPU167.NET_RX 22656 +43.6% 32529 ± 2% softirqs.CPU167.SCHED 27845 +40.0% 38982 ± 5% softirqs.CPU168.NET_RX 22861 +39.9% 31984 softirqs.CPU168.SCHED 26590 ± 8% +55.4% 41316 ± 5% softirqs.CPU169.NET_RX 22774 +40.5% 31990 softirqs.CPU169.SCHED 27965 ± 4% +34.4% 37581 ± 6% softirqs.CPU17.NET_RX 22416 +43.0% 32064 ± 3% softirqs.CPU17.SCHED 29095 ± 2% +43.8% 41848 ± 4% softirqs.CPU170.NET_RX 22753 +41.9% 32295 softirqs.CPU170.SCHED 25289 +56.3% 39538 ± 4% softirqs.CPU171.NET_RX 33179 ± 2% +7.3% 35615 ± 7% softirqs.CPU171.RCU 22899 +40.5% 32171 softirqs.CPU171.SCHED 25293 ± 3% +60.0% 40480 ± 7% softirqs.CPU172.NET_RX 22584 +42.6% 32196 softirqs.CPU172.SCHED 27200 ± 5% +50.1% 40831 ± 4% softirqs.CPU173.NET_RX 22804 +41.1% 32183 softirqs.CPU173.SCHED 27779 ± 3% +49.3% 41464 ± 7% softirqs.CPU174.NET_RX 22569 +42.0% 32056 softirqs.CPU174.SCHED 26980 ± 6% +49.2% 40261 ± 5% softirqs.CPU175.NET_RX 22818 +41.2% 32220 softirqs.CPU175.SCHED 27496 ± 6% +48.8% 40909 ± 9% softirqs.CPU176.NET_RX 22818 +40.9% 32139 softirqs.CPU176.SCHED 27197 +42.7% 38817 ± 8% softirqs.CPU177.NET_RX 22770 +40.4% 31976 softirqs.CPU177.SCHED 27688 ± 2% +45.6% 40318 ± 8% softirqs.CPU178.NET_RX 22820 +39.9% 31937 softirqs.CPU178.SCHED 26078 ± 4% +51.3% 39444 ± 5% softirqs.CPU179.NET_RX 22716 +40.5% 31912 softirqs.CPU179.SCHED 27755 ± 2% +41.2% 39191 ± 8% softirqs.CPU18.NET_RX 22621 ± 2% +41.0% 31900 softirqs.CPU18.SCHED 28019 ± 5% +40.3% 39303 ± 9% softirqs.CPU180.NET_RX 22549 +41.3% 31869 softirqs.CPU180.SCHED 27487 ± 10% +46.9% 40381 ± 7% softirqs.CPU181.NET_RX 22776 +40.2% 31943 softirqs.CPU181.SCHED 26432 ± 2% +55.0% 40971 ± 7% softirqs.CPU182.NET_RX 22815 +40.7% 32105 softirqs.CPU182.SCHED 27997 ± 4% +46.5% 41010 ± 4% softirqs.CPU183.NET_RX 22875 +39.9% 32004 softirqs.CPU183.SCHED 24478 ± 14% +67.2% 40939 ± 10% softirqs.CPU184.NET_RX 22192 ± 3% +45.0% 32177 softirqs.CPU184.SCHED 27366 +40.4% 38417 ± 8% softirqs.CPU185.NET_RX 22879 +40.4% 32119 softirqs.CPU185.SCHED 28955 ± 4% +36.2% 39443 ± 6% softirqs.CPU186.NET_RX 22929 +39.2% 31913 softirqs.CPU186.SCHED 26140 ± 5% +50.3% 39290 ± 8% softirqs.CPU187.NET_RX 23163 +38.6% 32110 softirqs.CPU187.SCHED 26199 +56.2% 40920 ± 11% softirqs.CPU188.NET_RX 22814 +41.5% 32276 softirqs.CPU188.SCHED 27178 ± 5% +49.7% 40699 ± 3% softirqs.CPU189.NET_RX 22673 +41.5% 32083 softirqs.CPU189.SCHED 27438 ± 7% +38.7% 38069 ± 5% softirqs.CPU19.NET_RX 22583 ± 3% +42.0% 32066 ± 2% softirqs.CPU19.SCHED 26118 ± 9% +42.6% 37257 ± 8% softirqs.CPU190.NET_RX 22621 +41.6% 32022 softirqs.CPU190.SCHED 25609 ± 5% +52.5% 39055 ± 9% softirqs.CPU191.NET_RX 22510 +40.5% 31618 softirqs.CPU191.SCHED 28410 ± 5% +34.1% 38091 ± 5% softirqs.CPU2.NET_RX 22190 +43.9% 31937 ± 2% softirqs.CPU2.SCHED 28923 ± 10% +37.5% 39780 ± 9% softirqs.CPU20.NET_RX 22749 ± 4% +41.2% 32113 ± 2% softirqs.CPU20.SCHED 27078 ± 3% +38.2% 37429 ± 8% softirqs.CPU21.NET_RX 22477 ± 3% +42.5% 32029 ± 2% softirqs.CPU21.SCHED 26006 ± 9% +49.7% 38922 ± 11% softirqs.CPU22.NET_RX 22743 ± 3% +41.2% 32108 ± 2% softirqs.CPU22.SCHED 28691 ± 2% +39.6% 40046 ± 5% softirqs.CPU23.NET_RX 22668 ± 3% +41.1% 31976 ± 2% softirqs.CPU23.SCHED 32698 ± 4% +27.5% 41704 ± 3% softirqs.CPU24.NET_RX 23538 +34.9% 31764 softirqs.CPU24.SCHED 27793 +44.1% 40038 ± 9% softirqs.CPU25.NET_RX 22892 ± 2% +38.0% 31602 softirqs.CPU25.SCHED 30671 ± 3% +36.4% 41822 ± 7% softirqs.CPU26.NET_RX 23207 ± 2% +36.8% 31744 softirqs.CPU26.SCHED 26281 ± 6% +57.6% 41417 ± 3% softirqs.CPU27.NET_RX 22758 ± 2% +40.4% 31952 softirqs.CPU27.SCHED 26713 ± 5% +54.6% 41300 ± 4% softirqs.CPU28.NET_RX 33769 ± 2% +11.3% 37593 ± 7% softirqs.CPU28.RCU 22857 ± 3% +38.2% 31586 softirqs.CPU28.SCHED 27219 ± 3% +53.1% 41664 ± 8% softirqs.CPU29.NET_RX 22988 ± 3% +38.2% 31775 softirqs.CPU29.SCHED 26571 ± 5% +51.2% 40171 ± 8% softirqs.CPU3.NET_RX 23000 ± 3% +38.9% 31946 ± 3% softirqs.CPU3.SCHED 26856 ± 4% +57.9% 42396 ± 2% softirqs.CPU30.NET_RX 22915 ± 3% +38.0% 31629 softirqs.CPU30.SCHED 28747 ± 6% +42.3% 40898 ± 9% softirqs.CPU31.NET_RX 22863 ± 2% +39.4% 31864 softirqs.CPU31.SCHED 26418 ± 4% +56.0% 41211 ± 4% softirqs.CPU32.NET_RX 22861 ± 3% +38.3% 31613 softirqs.CPU32.SCHED 27379 ± 13% +52.0% 41619 ± 3% softirqs.CPU33.NET_RX 22743 +38.7% 31538 softirqs.CPU33.SCHED 27331 ± 6% +47.9% 40432 ± 7% softirqs.CPU34.NET_RX 23071 ± 2% +37.5% 31730 softirqs.CPU34.SCHED 27997 ± 9% +49.7% 41922 ± 4% softirqs.CPU35.NET_RX 34868 ± 3% +10.6% 38580 ± 7% softirqs.CPU35.RCU 23029 ± 3% +37.4% 31635 softirqs.CPU35.SCHED 26763 ± 4% +59.4% 42657 ± 6% softirqs.CPU36.NET_RX 22829 +38.7% 31657 softirqs.CPU36.SCHED 27252 +48.6% 40489 ± 3% softirqs.CPU37.NET_RX 22546 ± 2% +40.2% 31603 softirqs.CPU37.SCHED 26072 ± 9% +67.4% 43643 ± 7% softirqs.CPU38.NET_RX 22711 ± 4% +39.6% 31710 softirqs.CPU38.SCHED 27579 ± 3% +57.8% 43518 ± 5% softirqs.CPU39.NET_RX 22705 ± 2% +39.2% 31596 softirqs.CPU39.SCHED 28570 ± 3% +30.7% 37345 ± 7% softirqs.CPU4.NET_RX 22763 ± 3% +41.4% 32189 ± 3% softirqs.CPU4.SCHED 26544 ± 5% +52.0% 40348 ± 5% softirqs.CPU40.NET_RX 22746 ± 3% +39.5% 31726 softirqs.CPU40.SCHED 28165 ± 5% +47.1% 41432 ± 3% softirqs.CPU41.NET_RX 22791 ± 3% +38.8% 31624 softirqs.CPU41.SCHED 28718 ± 6% +43.4% 41176 ± 8% softirqs.CPU42.NET_RX 22603 +39.3% 31493 softirqs.CPU42.SCHED 27257 ± 4% +54.4% 42085 ± 4% softirqs.CPU43.NET_RX 22763 ± 3% +39.4% 31742 softirqs.CPU43.SCHED 26604 ± 3% +58.0% 42041 ± 5% softirqs.CPU44.NET_RX 33772 ± 5% +13.2% 38218 ± 7% softirqs.CPU44.RCU 22764 ± 2% +39.6% 31790 softirqs.CPU44.SCHED 26786 ± 4% +54.4% 41361 ± 6% softirqs.CPU45.NET_RX 34271 ± 3% +10.2% 37764 ± 8% softirqs.CPU45.RCU 22813 ± 3% +38.4% 31565 softirqs.CPU45.SCHED 28082 ± 5% +43.0% 40154 ± 7% softirqs.CPU46.NET_RX 34703 ± 4% +9.0% 37818 ± 6% softirqs.CPU46.RCU 22800 ± 4% +39.4% 31785 softirqs.CPU46.SCHED 28229 ± 2% +49.3% 42156 ± 6% softirqs.CPU47.NET_RX 35007 ± 3% +9.5% 38323 ± 7% softirqs.CPU47.RCU 22878 ± 2% +38.2% 31613 softirqs.CPU47.SCHED 29087 ± 4% +39.9% 40686 ± 3% softirqs.CPU48.NET_RX 22510 +40.5% 31621 softirqs.CPU48.SCHED 27833 ± 5% +47.3% 40991 ± 6% softirqs.CPU49.NET_RX 22119 +42.7% 31572 softirqs.CPU49.SCHED 28242 +35.9% 38374 ± 6% softirqs.CPU5.NET_RX 22636 ± 3% +42.1% 32161 ± 2% softirqs.CPU5.SCHED 28615 ± 4% +37.2% 39250 ± 5% softirqs.CPU50.NET_RX 34409 +12.2% 38623 ± 5% softirqs.CPU50.RCU 22248 +41.5% 31482 softirqs.CPU50.SCHED 28739 ± 5% +40.6% 40405 softirqs.CPU51.NET_RX 22458 ± 2% +41.1% 31679 softirqs.CPU51.SCHED 28428 ± 6% +34.7% 38284 softirqs.CPU52.NET_RX 22308 +41.7% 31601 softirqs.CPU52.SCHED 29032 ± 6% +33.6% 38786 ± 7% softirqs.CPU53.NET_RX 34502 ± 5% +10.0% 37943 ± 8% softirqs.CPU53.RCU 22408 +40.7% 31526 softirqs.CPU53.SCHED 27640 ± 7% +42.5% 39380 ± 2% softirqs.CPU54.NET_RX 22473 +40.7% 31609 softirqs.CPU54.SCHED 26943 ± 9% +42.4% 38374 ± 4% softirqs.CPU55.NET_RX 22289 ± 2% +42.7% 31800 softirqs.CPU55.SCHED 27400 ± 4% +40.2% 38409 ± 5% softirqs.CPU56.NET_RX 22354 +40.0% 31300 softirqs.CPU56.SCHED 26999 ± 2% +46.5% 39554 ± 6% softirqs.CPU57.NET_RX 34130 ± 3% +12.9% 38546 ± 7% softirqs.CPU57.RCU 22383 +41.0% 31569 softirqs.CPU57.SCHED 26684 ± 3% +50.6% 40177 ± 3% softirqs.CPU58.NET_RX 34403 ± 3% +11.9% 38510 ± 7% softirqs.CPU58.RCU 21983 +44.1% 31667 softirqs.CPU58.SCHED 28353 +41.2% 40037 ± 5% softirqs.CPU59.NET_RX 22456 +40.7% 31589 softirqs.CPU59.SCHED 27801 ± 10% +34.5% 37398 ± 5% softirqs.CPU6.NET_RX 22604 ± 3% +41.8% 32058 ± 2% softirqs.CPU6.SCHED 26348 ± 3% +55.7% 41023 softirqs.CPU60.NET_RX 22333 +41.1% 31517 softirqs.CPU60.SCHED 27592 ± 8% +43.0% 39449 ± 4% softirqs.CPU61.NET_RX 22340 +40.8% 31449 softirqs.CPU61.SCHED 27991 ± 6% +39.9% 39173 ± 6% softirqs.CPU62.NET_RX 22336 +41.0% 31505 softirqs.CPU62.SCHED 27979 ± 6% +42.4% 39848 ± 6% softirqs.CPU63.NET_RX 22455 +40.7% 31595 softirqs.CPU63.SCHED 27343 ± 4% +47.1% 40224 ± 7% softirqs.CPU64.NET_RX 22264 +42.4% 31709 softirqs.CPU64.SCHED 30930 ± 12% +24.7% 38565 ± 3% softirqs.CPU65.NET_RX 22563 +39.9% 31572 softirqs.CPU65.SCHED 27228 ± 3% +40.3% 38207 ± 5% softirqs.CPU66.NET_RX 33624 ± 2% +10.9% 37300 ± 6% softirqs.CPU66.RCU 22263 +42.0% 31615 softirqs.CPU66.SCHED 26629 ± 7% +45.8% 38826 ± 3% softirqs.CPU67.NET_RX 22267 +42.1% 31644 softirqs.CPU67.SCHED 27039 ± 7% +46.4% 39598 ± 3% softirqs.CPU68.NET_RX 22376 +41.0% 31552 softirqs.CPU68.SCHED 26835 ± 4% +41.3% 37921 ± 4% softirqs.CPU69.NET_RX 22292 +42.2% 31695 softirqs.CPU69.SCHED 28526 ± 9% +36.6% 38980 ± 9% softirqs.CPU7.NET_RX 22556 ± 3% +42.0% 32026 ± 3% softirqs.CPU7.SCHED 27716 ± 6% +44.2% 39956 ± 5% softirqs.CPU70.NET_RX 22376 +40.4% 31421 softirqs.CPU70.SCHED 25987 ± 2% +54.0% 40017 ± 5% softirqs.CPU71.NET_RX 22256 +42.9% 31803 softirqs.CPU71.SCHED 30271 ± 2% +31.6% 39836 ± 8% softirqs.CPU72.NET_RX 22864 +39.9% 31988 softirqs.CPU72.SCHED 28646 ± 7% +40.8% 40326 ± 6% softirqs.CPU73.NET_RX 22434 +41.4% 31715 softirqs.CPU73.SCHED 27673 ± 4% +44.6% 40027 ± 5% softirqs.CPU74.NET_RX 22454 +41.1% 31688 softirqs.CPU74.SCHED 26780 ± 5% +50.1% 40207 ± 5% softirqs.CPU75.NET_RX 22613 +40.8% 31840 softirqs.CPU75.SCHED 28051 +45.5% 40826 ± 5% softirqs.CPU76.NET_RX 22608 +41.0% 31879 softirqs.CPU76.SCHED 26339 ± 5% +49.7% 39425 ± 10% softirqs.CPU77.NET_RX 22329 +43.4% 32019 softirqs.CPU77.SCHED 28822 +38.8% 40011 ± 6% softirqs.CPU78.NET_RX 22588 +41.2% 31903 softirqs.CPU78.SCHED 26965 ± 4% +50.8% 40656 ± 8% softirqs.CPU79.NET_RX 22337 +41.9% 31701 softirqs.CPU79.SCHED 28854 ± 2% +36.0% 39243 ± 7% softirqs.CPU8.NET_RX 33868 ± 2% +8.2% 36644 ± 4% softirqs.CPU8.RCU 22572 ± 2% +41.6% 31966 ± 3% softirqs.CPU8.SCHED 25294 ± 3% +62.3% 41046 ± 9% softirqs.CPU80.NET_RX 22560 +41.6% 31949 softirqs.CPU80.SCHED 27226 ± 6% +44.2% 39272 ± 5% softirqs.CPU81.NET_RX 22433 +41.8% 31802 softirqs.CPU81.SCHED 28179 ± 2% +45.8% 41089 ± 6% softirqs.CPU82.NET_RX 22503 +41.1% 31742 softirqs.CPU82.SCHED 25370 ± 6% +65.5% 41993 ± 3% softirqs.CPU83.NET_RX 33224 ± 3% +9.5% 36387 ± 5% softirqs.CPU83.RCU 22482 +43.4% 32239 softirqs.CPU83.SCHED 29150 ± 4% +41.9% 41357 ± 6% softirqs.CPU84.NET_RX 22540 +40.3% 31629 softirqs.CPU84.SCHED 27566 ± 7% +45.2% 40032 ± 6% softirqs.CPU85.NET_RX 22614 +40.5% 31779 softirqs.CPU85.SCHED 26616 ± 3% +48.0% 39392 ± 8% softirqs.CPU86.NET_RX 22503 +41.8% 31899 softirqs.CPU86.SCHED 27217 +54.7% 42097 ± 4% softirqs.CPU87.NET_RX 32656 +11.1% 36286 ± 8% softirqs.CPU87.RCU 22383 +41.8% 31744 softirqs.CPU87.SCHED 29444 ± 5% +35.2% 39821 ± 6% softirqs.CPU88.NET_RX 22560 ± 2% +41.3% 31887 softirqs.CPU88.SCHED 26383 ± 3% +49.4% 39403 ± 5% softirqs.CPU89.NET_RX 22647 +41.2% 31984 softirqs.CPU89.SCHED 27413 +45.5% 39873 ± 3% softirqs.CPU9.NET_RX 22436 ± 3% +43.3% 32154 ± 2% softirqs.CPU9.SCHED 28117 ± 4% +39.1% 39110 ± 5% softirqs.CPU90.NET_RX 22524 +41.6% 31903 softirqs.CPU90.SCHED 28805 ± 3% +43.1% 41207 ± 2% softirqs.CPU91.NET_RX 22981 +38.0% 31714 softirqs.CPU91.SCHED 27920 ± 9% +46.1% 40778 ± 7% softirqs.CPU92.NET_RX 33255 ± 4% +8.9% 36199 ± 7% softirqs.CPU92.RCU 22467 +42.2% 31955 softirqs.CPU92.SCHED 26935 ± 5% +49.3% 40211 ± 6% softirqs.CPU93.NET_RX 22407 +42.4% 31898 softirqs.CPU93.SCHED 25585 ± 6% +63.7% 41880 ± 5% softirqs.CPU94.NET_RX 33095 ± 4% +10.4% 36528 ± 9% softirqs.CPU94.RCU 22417 +41.7% 31761 softirqs.CPU94.SCHED 27659 ± 6% +43.3% 39639 ± 6% softirqs.CPU95.NET_RX 21998 +42.6% 31361 softirqs.CPU95.SCHED 26582 ± 2% +43.4% 38128 ± 8% softirqs.CPU96.NET_RX 22170 ± 3% +45.7% 32297 softirqs.CPU96.SCHED 27522 ± 5% +39.4% 38353 ± 9% softirqs.CPU97.NET_RX 22468 ± 3% +43.4% 32209 ± 2% softirqs.CPU97.SCHED 27117 ± 12% +39.8% 37902 ± 10% softirqs.CPU98.NET_RX 22600 +43.2% 32361 ± 2% softirqs.CPU98.SCHED 26557 ± 2% +43.7% 38161 ± 7% softirqs.CPU99.NET_RX 23003 +41.6% 32565 softirqs.CPU99.SCHED 5270210 +44.9% 7635912 softirqs.NET_RX 4357512 +40.8% 6136169 softirqs.SCHED 80.01 -80.0 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.scan_positives.dcache_readdir.iterate_dir 81.90 -68.9 13.02 ± 5% perf-profile.calltrace.cycles-pp.dcache_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe 81.91 -68.9 13.03 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe 81.90 -68.9 13.03 ± 5% perf-profile.calltrace.cycles-pp.iterate_dir.__x64_sys_getdents64.do_syscall_64.entry_SYSCALL_64_after_hwframe 80.60 -68.8 11.80 ± 5% perf-profile.calltrace.cycles-pp.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64.do_syscall_64 80.30 -68.7 11.63 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.scan_positives.dcache_readdir.iterate_dir.__x64_sys_getdents64 85.88 -51.1 34.79 ± 13% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 85.98 -50.8 35.15 ± 13% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 0.37 ± 70% +0.4 0.80 ± 4% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 0.37 ± 70% +0.4 0.81 ± 4% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 0.37 ± 70% +0.4 0.81 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 0.39 ± 70% +0.4 0.82 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.creat64 0.39 ± 70% +0.4 0.84 ± 3% perf-profile.calltrace.cycles-pp.creat64 0.37 ± 70% +0.6 0.93 ± 10% perf-profile.calltrace.cycles-pp.div_long 0.00 +0.6 0.62 ± 10% perf-profile.calltrace.cycles-pp.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start.asm_call_sysvec_on_stack 0.00 +0.6 0.62 ± 10% perf-profile.calltrace.cycles-pp.__strcat_chk 0.39 ± 70% +0.7 1.04 ± 9% perf-profile.calltrace.cycles-pp.add_int.add_int 0.38 ± 70% +0.7 1.04 ± 8% perf-profile.calltrace.cycles-pp.add_long.add_long 0.00 +0.7 0.68 ± 9% perf-profile.calltrace.cycles-pp.rcu_do_batch.rcu_core.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack 0.00 +0.7 0.69 ± 9% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu 0.00 +0.7 0.73 ± 3% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 0.00 +0.8 0.76 ± 9% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt 0.00 +0.8 0.76 ± 9% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.00 +0.8 0.76 ± 9% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.36 ± 70% +0.8 1.12 ± 11% perf-profile.calltrace.cycles-pp.__strncat_chk 0.00 +0.8 0.77 ± 3% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 0.00 +0.8 0.77 ± 9% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.00 +0.8 0.79 ± 23% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_slowpath.__vma_adjust.__split_vma.__do_munmap 0.00 +0.8 0.81 ± 23% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.__vma_adjust.__split_vma.__do_munmap.mmap_region 0.00 +0.8 0.81 ± 22% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_slowpath.__vma_adjust.__split_vma.mprotect_fixup 0.00 +0.8 0.82 ± 3% perf-profile.calltrace.cycles-pp.walk_component.link_path_walk.path_openat.do_filp_open.do_sys_openat2 0.00 +0.8 0.84 ± 21% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.__vma_adjust.__split_vma.mprotect_fixup.do_mprotect_pkey 0.00 +0.8 0.85 ± 3% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault 0.00 +0.9 0.85 ± 3% perf-profile.calltrace.cycles-pp.link_path_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.00 +0.9 0.86 ± 3% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault 0.00 +0.9 0.86 ± 22% perf-profile.calltrace.cycles-pp.__vma_adjust.__split_vma.__do_munmap.mmap_region.do_mmap 0.18 ±141% +0.9 1.04 ± 24% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.unmap_region 0.17 ±141% +0.9 1.03 ± 22% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap 0.00 +0.9 0.88 ± 24% perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.exit_mmap.mmput.begin_new_exec 0.00 +0.9 0.88 ± 21% perf-profile.calltrace.cycles-pp.__split_vma.__do_munmap.mmap_region.do_mmap.vm_mmap_pgoff 0.18 ±141% +0.9 1.07 ± 23% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.unmap_region.__do_munmap 0.17 ±141% +0.9 1.06 ± 21% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.vma_link.mmap_region.do_mmap.vm_mmap_pgoff 0.18 ±141% +0.9 1.08 ± 23% perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.unmap_region.__do_munmap.mmap_region 0.19 ±141% +0.9 1.09 ± 23% perf-profile.calltrace.cycles-pp.free_pgtables.unmap_region.__do_munmap.mmap_region.do_mmap 0.19 ±141% +0.9 1.12 ± 23% perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.mmap_region.do_mmap.vm_mmap_pgoff 0.00 +0.9 0.93 ± 4% perf-profile.calltrace.cycles-pp.asm_exc_page_fault 0.17 ±141% +0.9 1.10 ± 19% perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.begin_new_exec.load_elf_binary 0.18 ±141% +0.9 1.12 ± 20% perf-profile.calltrace.cycles-pp.vma_link.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff 0.00 +0.9 0.94 ± 19% perf-profile.calltrace.cycles-pp.__vma_adjust.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect 0.00 +1.0 0.98 ± 18% perf-profile.calltrace.cycles-pp.__split_vma.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64 0.00 +1.0 1.02 ± 18% perf-profile.calltrace.cycles-pp.mprotect_fixup.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.20 ±141% +1.0 1.22 ± 24% perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.exit_mmap.mmput.do_exit 0.00 +1.0 1.03 ± 18% perf-profile.calltrace.cycles-pp.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.0 1.03 ± 18% perf-profile.calltrace.cycles-pp.do_mprotect_pkey.__x64_sys_mprotect.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.39 ± 72% +1.1 1.45 ± 11% perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.39 ± 72% +1.1 1.45 ± 11% perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.1 1.06 ± 26% perf-profile.calltrace.cycles-pp._raw_spin_lock.lockref_get.d_alloc_cursor.dcache_dir_open.do_dentry_open 0.00 +1.1 1.06 ± 26% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.lockref_get.d_alloc_cursor.dcache_dir_open 0.00 +1.1 1.06 ± 26% perf-profile.calltrace.cycles-pp.lockref_get.d_alloc_cursor.dcache_dir_open.do_dentry_open.path_openat 0.00 +1.1 1.06 ± 26% perf-profile.calltrace.cycles-pp.dcache_dir_open.do_dentry_open.path_openat.do_filp_open.do_sys_openat2 0.00 +1.1 1.06 ± 26% perf-profile.calltrace.cycles-pp.d_alloc_cursor.dcache_dir_open.do_dentry_open.path_openat.do_filp_open 0.20 ±141% +1.1 1.27 ± 14% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fork 0.20 ±141% +1.1 1.27 ± 14% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork 0.20 ±141% +1.1 1.27 ± 14% perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork 0.19 ±141% +1.1 1.27 ± 14% perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork 0.21 ±141% +1.1 1.33 ± 22% perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.mmput.do_exit.do_group_exit 0.00 +1.1 1.12 ± 11% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.dcache_readdir.iterate_dir.__x64_sys_getdents64 0.60 ± 6% +1.1 1.73 ± 24% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.begin_new_exec.load_elf_binary.exec_binprm 0.20 ±141% +1.1 1.33 ± 13% perf-profile.calltrace.cycles-pp.__libc_fork 0.60 ± 6% +1.1 1.73 ± 24% perf-profile.calltrace.cycles-pp.mmput.begin_new_exec.load_elf_binary.exec_binprm.bprm_execve 0.17 ±141% +1.1 1.32 ± 16% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 0.00 +1.1 1.15 ± 18% perf-profile.calltrace.cycles-pp.drain_obj_stock.refill_obj_stock.kmem_cache_free.rcu_do_batch.rcu_core 0.62 ± 6% +1.2 1.78 ± 23% perf-profile.calltrace.cycles-pp.begin_new_exec.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common 0.00 +1.2 1.18 ± 21% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 0.80 ± 20% +1.2 2.03 ± 22% perf-profile.calltrace.cycles-pp.__do_munmap.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff 0.73 ± 17% +1.2 1.98 ± 14% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group 0.73 ± 17% +1.2 1.98 ± 14% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.00 +1.3 1.28 ± 18% perf-profile.calltrace.cycles-pp._raw_spin_lock.lockref_get.do_dentry_open.path_openat.do_filp_open 0.00 +1.3 1.28 ± 18% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.lockref_get.do_dentry_open.path_openat 0.00 +1.3 1.29 ± 18% perf-profile.calltrace.cycles-pp.lockref_get.do_dentry_open.path_openat.do_filp_open.do_sys_openat2 0.00 +1.4 1.36 ± 39% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process 0.81 ± 16% +1.4 2.20 ± 13% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.81 ± 16% +1.4 2.21 ± 13% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.81 ± 16% +1.4 2.21 ± 13% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.4 1.41 ± 38% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.dup_mmap.dup_mm.copy_process.kernel_clone 0.00 +1.4 1.41 ± 17% perf-profile.calltrace.cycles-pp.refill_obj_stock.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start 0.18 ±141% +1.7 1.92 ± 26% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap 0.19 ±141% +1.8 2.01 ± 25% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.unlink_file_vma.free_pgtables.exit_mmap.mmput 0.83 ± 5% +1.8 2.65 ± 14% perf-profile.calltrace.cycles-pp.load_elf_binary.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve 0.83 ± 4% +1.8 2.67 ± 13% perf-profile.calltrace.cycles-pp.exec_binprm.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64 0.36 ±141% +2.0 2.32 ± 15% perf-profile.calltrace.cycles-pp.dup_mm.copy_process.kernel_clone.__do_sys_clone.do_syscall_64 1.24 ± 20% +2.0 3.23 ± 21% perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64 1.25 ± 20% +2.0 3.23 ± 21% perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.26 ± 20% +2.0 3.25 ± 21% perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.26 ± 20% +2.0 3.25 ± 21% perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.68 ± 14% +2.0 3.72 ± 10% perf-profile.calltrace.cycles-pp.string_rtns_1 0.19 ±141% +2.1 2.29 ± 15% perf-profile.calltrace.cycles-pp.dup_mmap.dup_mm.copy_process.kernel_clone.__do_sys_clone 0.39 ±141% +2.2 2.58 ± 13% perf-profile.calltrace.cycles-pp.copy_process.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +2.6 2.62 ± 6% perf-profile.calltrace.cycles-pp.do_dentry_open.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.40 ± 70% +3.1 3.52 ± 15% perf-profile.calltrace.cycles-pp.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd 0.61 ± 11% +3.2 3.79 ± 15% perf-profile.calltrace.cycles-pp.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn 0.61 ± 11% +3.2 3.79 ± 15% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread 0.61 ± 11% +3.2 3.79 ± 15% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 0.61 ± 11% +3.2 3.79 ± 15% perf-profile.calltrace.cycles-pp.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 0.61 ± 12% +3.2 3.82 ± 15% perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork 0.65 ± 11% +3.2 3.88 ± 15% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 0.65 ± 11% +3.2 3.89 ± 15% perf-profile.calltrace.cycles-pp.ret_from_fork 0.22 ±141% +4.1 4.27 ± 28% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.22 ±141% +4.1 4.28 ± 28% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.30 ± 10% +6.1 9.36 ± 10% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 0.00 +6.2 6.21 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.lockref_get_not_dead.__legitimize_path.unlazy_walk.complete_walk 0.00 +6.2 6.22 ± 3% perf-profile.calltrace.cycles-pp.lockref_get_not_dead.__legitimize_path.unlazy_walk.complete_walk.path_openat 0.00 +6.2 6.22 ± 3% perf-profile.calltrace.cycles-pp.unlazy_walk.complete_walk.path_openat.do_filp_open.do_sys_openat2 0.00 +6.2 6.22 ± 3% perf-profile.calltrace.cycles-pp.__legitimize_path.unlazy_walk.complete_walk.path_openat.do_filp_open 0.00 +6.2 6.22 ± 3% perf-profile.calltrace.cycles-pp.complete_walk.path_openat.do_filp_open.do_sys_openat2.do_sys_open 0.00 +6.5 6.47 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.lockref_put_or_lock.dput.step_into.path_openat 0.00 +6.5 6.47 ± 7% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.lockref_put_or_lock.dput.step_into 0.00 +6.5 6.47 ± 7% perf-profile.calltrace.cycles-pp.dput.step_into.path_openat.do_filp_open.do_open_execat 0.00 +6.5 6.47 ± 7% perf-profile.calltrace.cycles-pp.lockref_put_or_lock.dput.step_into.path_openat.do_filp_open 0.00 +7.0 6.99 ± 19% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel 0.00 +7.0 6.99 ± 19% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel 0.00 +7.0 6.99 ± 18% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__open64_nocancel 0.00 +7.0 7.00 ± 18% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__open64_nocancel 0.00 +7.0 7.00 ± 18% perf-profile.calltrace.cycles-pp.__open64_nocancel 3.79 ± 10% +7.0 10.82 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 3.86 ± 9% +7.1 10.96 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 4.17 ± 6% +7.4 11.59 ± 8% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 4.17 ± 6% +7.4 11.59 ± 8% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 4.17 ± 6% +7.4 11.59 ± 8% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 4.20 ± 6% +7.4 11.63 ± 8% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 0.00 +8.6 8.58 ± 38% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +8.6 8.59 ± 38% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +11.2 11.23 ± 5% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.scan_positives.dcache_readdir.iterate_dir 0.56 ± 7% +11.4 11.95 ± 2% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64 0.56 ± 7% +11.4 11.96 ± 2% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +24.2 24.23 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock.lockref_get_not_dead.__legitimize_path.unlazy_walk.pick_link 0.00 +24.2 24.25 ± 4% perf-profile.calltrace.cycles-pp.unlazy_walk.pick_link.step_into.path_openat.do_filp_open 0.00 +24.2 24.25 ± 4% perf-profile.calltrace.cycles-pp.__legitimize_path.unlazy_walk.pick_link.step_into.path_openat 0.00 +24.2 24.25 ± 4% perf-profile.calltrace.cycles-pp.lockref_get_not_dead.__legitimize_path.unlazy_walk.pick_link.step_into 0.00 +24.2 24.25 ± 4% perf-profile.calltrace.cycles-pp.pick_link.step_into.path_openat.do_filp_open.do_open_execat 1.02 ± 4% +24.4 25.41 ± 8% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve 1.02 ± 4% +24.4 25.42 ± 8% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve 1.02 ± 4% +24.4 25.42 ± 8% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve 1.02 ± 4% +24.4 25.42 ± 8% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve 1.03 ± 4% +24.4 25.43 ± 8% perf-profile.calltrace.cycles-pp.execve 0.00 +30.4 30.45 ± 3% perf-profile.calltrace.cycles-pp.__cna_queued_spin_lock_slowpath._raw_spin_lock.lockref_get_not_dead.__legitimize_path.unlazy_walk 0.00 +30.8 30.82 ± 3% perf-profile.calltrace.cycles-pp.step_into.path_openat.do_filp_open.do_open_execat.bprm_execve 0.00 +30.8 30.84 ± 3% perf-profile.calltrace.cycles-pp.do_filp_open.do_open_execat.bprm_execve.do_execveat_common.__x64_sys_execve 0.00 +30.8 30.84 ± 3% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_open_execat.bprm_execve.do_execveat_common 0.00 +30.8 30.84 ± 3% perf-profile.calltrace.cycles-pp.do_open_execat.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64 0.95 ± 4% +32.8 33.76 ± 3% perf-profile.calltrace.cycles-pp.bprm_execve.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe 82.33 -82.3 0.00 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 81.90 -68.9 13.02 ± 5% perf-profile.children.cycles-pp.dcache_readdir 81.91 -68.9 13.03 ± 5% perf-profile.children.cycles-pp.__x64_sys_getdents64 81.90 -68.9 13.03 ± 5% perf-profile.children.cycles-pp.iterate_dir 80.60 -68.8 11.80 ± 5% perf-profile.children.cycles-pp.scan_positives 82.71 -27.9 54.83 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 89.05 -18.0 71.08 perf-profile.children.cycles-pp.do_syscall_64 89.35 -17.5 71.82 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 0.54 ± 22% -0.2 0.32 ± 20% perf-profile.children.cycles-pp.llseek 0.62 -0.2 0.45 ± 9% perf-profile.children.cycles-pp.ksys_lseek 0.62 -0.2 0.45 ± 9% perf-profile.children.cycles-pp.dcache_dir_lseek 0.06 ± 8% +0.0 0.09 ± 21% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.06 ± 13% +0.0 0.10 ± 15% perf-profile.children.cycles-pp.update_load_avg 0.16 ± 8% +0.0 0.21 ± 17% perf-profile.children.cycles-pp.ktime_get 0.00 +0.1 0.05 perf-profile.children.cycles-pp.copy_page 0.00 +0.1 0.05 perf-profile.children.cycles-pp.copy_string_kernel 0.00 +0.1 0.05 perf-profile.children.cycles-pp.time 0.00 +0.1 0.05 perf-profile.children.cycles-pp.__x64_sys_rt_sigreturn 0.00 +0.1 0.05 perf-profile.children.cycles-pp.perf_iterate_sb 0.00 +0.1 0.05 perf-profile.children.cycles-pp._raw_spin_trylock 0.00 +0.1 0.05 perf-profile.children.cycles-pp.irqtime_account_irq 0.00 +0.1 0.05 perf-profile.children.cycles-pp.unlock_page 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.common_file_perm 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.shift_arg_pages 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.__list_del_entry_valid 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.__check_object_size 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.compar1 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.lru_add_drain_cpu 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.do_faccessat 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.call_rcu 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.rebalance_domains 0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.__pud_alloc 0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.do_task_dead 0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.new_sync_read 0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.ip_finish_output2 0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.sync_regs 0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.d_add 0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp.lru_add_drain 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp._cond_resched 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.cfree 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.update_rq_clock 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.pagevec_lru_move_fn 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.__rb_insert_augmented 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.kfree 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.security_cred_free 0.00 +0.1 0.06 ± 14% perf-profile.children.cycles-pp.worker_thread 0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.read 0.00 +0.1 0.06 perf-profile.children.cycles-pp.anon_vma_interval_tree_insert 0.00 +0.1 0.06 perf-profile.children.cycles-pp.__might_sleep 0.00 +0.1 0.06 perf-profile.children.cycles-pp.simple_lookup 0.00 +0.1 0.06 perf-profile.children.cycles-pp.create_elf_tables 0.00 +0.1 0.06 perf-profile.children.cycles-pp.setup_arg_pages 0.02 ±141% +0.1 0.08 ± 54% perf-profile.children.cycles-pp.irq_enter_rcu 0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.alloc_pages_vma 0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.page_add_file_rmap 0.00 +0.1 0.06 ± 17% perf-profile.children.cycles-pp.memcpy_erms 0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.do_linkat 0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.__x64_sys_link 0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.inode_permission 0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.ip_output 0.00 +0.1 0.07 ± 23% perf-profile.children.cycles-pp.tick_nohz_next_event 0.05 ± 70% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.__close 0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.try_charge 0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.do_truncate 0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.udp_sendmsg 0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.put_task_stack 0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.free_pages_and_swap_cache 0.02 ±141% +0.1 0.08 ± 5% perf-profile.children.cycles-pp.unlink 0.07 ± 12% +0.1 0.14 ± 15% perf-profile.children.cycles-pp.write 0.06 +0.1 0.13 perf-profile.children.cycles-pp.wake_up_new_task 0.06 ± 13% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.__d_lookup_rcu 0.00 +0.1 0.07 ± 10% perf-profile.children.cycles-pp.uncharge_batch 0.00 +0.1 0.07 perf-profile.children.cycles-pp.group_send_sig_info 0.00 +0.1 0.07 perf-profile.children.cycles-pp.__get_user_pages 0.00 +0.1 0.07 perf-profile.children.cycles-pp.strnlen_user 0.00 +0.1 0.07 perf-profile.children.cycles-pp.security_task_kill 0.00 +0.1 0.07 perf-profile.children.cycles-pp.apparmor_task_kill 0.00 +0.1 0.07 ± 10% perf-profile.children.cycles-pp.__pmd_alloc 0.05 ± 8% +0.1 0.12 ± 4% perf-profile.children.cycles-pp.apparmor_file_open 0.09 ± 24% +0.1 0.16 ± 6% perf-profile.children.cycles-pp.kill 0.05 +0.1 0.12 ± 3% perf-profile.children.cycles-pp.security_file_free 0.05 +0.1 0.12 ± 3% perf-profile.children.cycles-pp.apparmor_file_free_security 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.__anon_vma_prepare 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.find_vma 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.get_arg_page 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.__get_user_pages_remote 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.mem_cgroup_charge 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.try_to_wake_up 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.00 +0.1 0.07 ± 22% perf-profile.children.cycles-pp.file_free_rcu 0.05 ± 8% +0.1 0.13 ± 6% perf-profile.children.cycles-pp.security_file_open 0.05 ± 8% +0.1 0.13 ± 3% perf-profile.children.cycles-pp.getname_flags 0.03 ± 70% +0.1 0.11 ± 4% perf-profile.children.cycles-pp.kill_pid_info 0.07 +0.1 0.15 ± 3% perf-profile.children.cycles-pp.native_irq_return_iret 0.00 +0.1 0.08 ± 6% perf-profile.children.cycles-pp.vfs_read 0.00 +0.1 0.08 ± 6% perf-profile.children.cycles-pp.__mod_memcg_state 0.00 +0.1 0.08 ± 6% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list 0.03 ± 70% +0.1 0.11 perf-profile.children.cycles-pp.kill_something_info 0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.ksys_read 0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.rcu_cblist_dequeue 0.00 +0.1 0.08 ± 8% perf-profile.children.cycles-pp.vma_interval_tree_remove 0.04 ± 71% +0.1 0.12 ± 3% perf-profile.children.cycles-pp.__x64_sys_kill 0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.filldir64 0.00 +0.1 0.08 ± 10% perf-profile.children.cycles-pp.put_cred_rcu 0.00 +0.1 0.08 ± 26% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.00 +0.1 0.08 ± 13% perf-profile.children.cycles-pp.__slab_free 0.02 ±141% +0.1 0.10 perf-profile.children.cycles-pp.up_write 0.05 ± 8% +0.1 0.14 ± 3% perf-profile.children.cycles-pp.copy_pte_range 0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.clear_page_erms 0.06 ± 8% +0.1 0.14 ± 5% perf-profile.children.cycles-pp.down_write 0.02 ±141% +0.1 0.10 ± 10% perf-profile.children.cycles-pp.malloc 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.wait4 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.down_read 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.__get_free_pages 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.__put_anon_vma 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.arch_do_signal 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.__dentry_kill 0.02 ±141% +0.1 0.11 ± 4% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.00 +0.1 0.09 perf-profile.children.cycles-pp.strncpy_from_user 0.00 +0.1 0.09 ± 7% perf-profile.children.cycles-pp.aa_get_task_label 0.00 +0.1 0.09 ± 7% perf-profile.children.cycles-pp.schedule_idle 0.00 +0.1 0.09 ± 17% perf-profile.children.cycles-pp.rwsem_wake 0.02 ±141% +0.1 0.11 ± 4% perf-profile.children.cycles-pp.___might_sleep 0.03 ± 70% +0.1 0.12 ± 4% perf-profile.children.cycles-pp.copy_strings 0.09 ± 13% +0.1 0.19 ± 14% perf-profile.children.cycles-pp.update_cfs_group 0.08 ± 10% +0.1 0.17 ± 11% perf-profile.children.cycles-pp.new_sync_write 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.sched_exec 0.02 ±141% +0.1 0.11 perf-profile.children.cycles-pp.alloc_set_pte 0.05 ± 8% +0.1 0.15 ± 2% perf-profile.children.cycles-pp.get_page_from_freelist 0.00 +0.1 0.10 ± 5% perf-profile.children.cycles-pp.prep_new_page 0.00 +0.1 0.10 ± 9% perf-profile.children.cycles-pp.__pte_alloc 0.04 ± 71% +0.1 0.13 ± 6% perf-profile.children.cycles-pp.__mmap 0.00 +0.1 0.10 ± 8% perf-profile.children.cycles-pp.sock_sendmsg 0.00 +0.1 0.10 ± 4% perf-profile.children.cycles-pp.perf_event_mmap 0.00 +0.1 0.10 ± 8% perf-profile.children.cycles-pp.__memcg_kmem_uncharge_page 0.00 +0.1 0.10 ± 45% perf-profile.children.cycles-pp.dcache_dir_close 0.00 +0.1 0.10 ± 7% perf-profile.children.cycles-pp.sock_write_iter 0.00 +0.1 0.10 ± 7% perf-profile.children.cycles-pp.lockref_put_return 0.09 ± 10% +0.1 0.19 ± 9% perf-profile.children.cycles-pp.ksys_write 0.02 ±141% +0.1 0.12 ± 3% perf-profile.children.cycles-pp.apparmor_task_getsecid 0.08 ± 5% +0.1 0.19 ± 9% perf-profile.children.cycles-pp.vfs_write 0.00 +0.1 0.11 ± 10% perf-profile.children.cycles-pp.do_unlinkat 0.08 ± 10% +0.1 0.19 ± 2% perf-profile.children.cycles-pp.find_idlest_group 0.00 +0.1 0.11 ± 33% perf-profile.children.cycles-pp.__close_nocancel 0.00 +0.1 0.11 ± 4% perf-profile.children.cycles-pp.vm_area_alloc 0.06 ± 16% +0.1 0.16 ± 5% perf-profile.children.cycles-pp.do_anonymous_page 0.02 ±141% +0.1 0.12 ± 4% perf-profile.children.cycles-pp.security_task_getsecid 0.08 ± 14% +0.1 0.19 ± 6% perf-profile.children.cycles-pp.__xstat64 0.00 +0.1 0.11 perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.06 ± 16% +0.1 0.17 ± 4% perf-profile.children.cycles-pp.obj_cgroup_charge 0.07 +0.1 0.18 ± 5% perf-profile.children.cycles-pp.apparmor_file_alloc_security 0.00 +0.1 0.11 ± 3% perf-profile.children.cycles-pp.__clear_user 0.20 ± 12% +0.1 0.31 ± 15% perf-profile.children.cycles-pp.task_tick_fair 0.02 ±141% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.find_busiest_group 0.02 ±141% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.update_sd_lb_stats 0.00 +0.1 0.11 ± 4% perf-profile.children.cycles-pp.ima_file_check 0.08 ± 10% +0.1 0.20 ± 4% perf-profile.children.cycles-pp.vma_interval_tree_insert 0.07 ± 14% +0.1 0.18 ± 4% perf-profile.children.cycles-pp.page_remove_rmap 0.02 ±141% +0.1 0.14 ± 3% perf-profile.children.cycles-pp.wp_page_copy 0.02 ±141% +0.1 0.14 ± 6% perf-profile.children.cycles-pp.pte_alloc_one 0.03 ± 70% +0.1 0.15 ± 2% perf-profile.children.cycles-pp.schedule 0.07 ± 11% +0.1 0.19 perf-profile.children.cycles-pp._dl_addr 0.00 +0.1 0.12 ± 8% perf-profile.children.cycles-pp.finish_task_switch 0.00 +0.1 0.12 ± 13% perf-profile.children.cycles-pp.__put_task_struct 0.08 +0.1 0.20 ± 5% perf-profile.children.cycles-pp.security_file_alloc 0.09 ± 9% +0.1 0.21 ± 2% perf-profile.children.cycles-pp.select_task_rq_fair 0.04 ± 71% +0.1 0.16 ± 5% perf-profile.children.cycles-pp.load_balance 0.05 ± 8% +0.1 0.18 ± 4% perf-profile.children.cycles-pp.vm_area_dup 0.04 ± 71% +0.1 0.17 ± 2% perf-profile.children.cycles-pp.__d_alloc 0.02 ±141% +0.1 0.15 ± 7% perf-profile.children.cycles-pp.do_wait 0.02 ±141% +0.1 0.15 ± 5% perf-profile.children.cycles-pp.pick_next_task_fair 0.06 ± 16% +0.1 0.19 ± 3% perf-profile.children.cycles-pp.anon_vma_clone 0.02 ±141% +0.1 0.15 ± 8% perf-profile.children.cycles-pp.__do_sys_wait4 0.02 ±141% +0.1 0.15 ± 8% perf-profile.children.cycles-pp.kernel_wait4 0.05 ± 8% +0.1 0.19 ± 2% perf-profile.children.cycles-pp.__vm_munmap 0.28 ± 13% +0.1 0.42 ± 12% perf-profile.children.cycles-pp.scheduler_tick 0.00 +0.1 0.14 ± 3% perf-profile.children.cycles-pp.cna_order_queue 0.08 ± 12% +0.1 0.22 ± 5% perf-profile.children.cycles-pp.copy_page_range 0.06 ± 19% +0.1 0.21 ± 7% perf-profile.children.cycles-pp.__memcg_kmem_charge_page 0.00 +0.1 0.15 ± 5% perf-profile.children.cycles-pp.newidle_balance 0.05 ± 8% +0.2 0.21 ± 19% perf-profile.children.cycles-pp.__lock_parent 0.08 ± 14% +0.2 0.26 ± 4% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.09 ± 15% +0.2 0.28 ± 4% perf-profile.children.cycles-pp.release_pages 0.09 ± 19% +0.2 0.28 ± 5% perf-profile.children.cycles-pp.page_counter_try_charge 0.21 ± 21% +0.2 0.41 ± 10% perf-profile.children.cycles-pp.div_int 0.39 ± 14% +0.2 0.58 ± 10% perf-profile.children.cycles-pp.update_process_times 0.39 ± 14% +0.2 0.59 ± 10% perf-profile.children.cycles-pp.tick_sched_handle 0.09 ± 20% +0.2 0.29 ± 3% perf-profile.children.cycles-pp.__memcg_kmem_charge 0.08 ± 17% +0.2 0.28 ± 3% perf-profile.children.cycles-pp.anon_vma_fork 0.16 ± 12% +0.2 0.36 ± 10% perf-profile.children.cycles-pp.mem_rtns_1 0.25 ± 20% +0.2 0.46 ± 10% perf-profile.children.cycles-pp.div_short 0.07 ± 25% +0.2 0.28 ± 4% perf-profile.children.cycles-pp.remove_vma 0.43 ± 14% +0.2 0.64 ± 10% perf-profile.children.cycles-pp.tick_sched_timer 0.14 ± 3% +0.2 0.36 ± 5% perf-profile.children.cycles-pp.setlocale 0.09 ± 9% +0.2 0.31 perf-profile.children.cycles-pp.__schedule 0.11 ± 12% +0.2 0.33 ± 4% perf-profile.children.cycles-pp.tlb_flush_mmu 0.11 ± 16% +0.2 0.34 ± 5% perf-profile.children.cycles-pp.tlb_finish_mmu 0.10 ± 4% +0.2 0.33 ± 11% perf-profile.children.cycles-pp.__fput 0.11 ± 11% +0.2 0.35 ± 4% perf-profile.children.cycles-pp.__alloc_pages_nodemask 0.52 ± 14% +0.2 0.76 ± 12% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.22 ± 19% +0.2 0.46 ± 11% perf-profile.children.cycles-pp.add_short 0.13 ± 10% +0.3 0.38 ± 5% perf-profile.children.cycles-pp.lookup_fast 0.09 ± 15% +0.3 0.35 perf-profile.children.cycles-pp.unlink_anon_vmas 0.09 ± 19% +0.3 0.34 ± 4% perf-profile.children.cycles-pp.get_obj_cgroup_from_current 0.12 ± 3% +0.3 0.38 ± 10% perf-profile.children.cycles-pp.task_work_run 0.09 ± 20% +0.3 0.35 perf-profile.children.cycles-pp.iterate_supers 0.09 ± 20% +0.3 0.36 ± 2% perf-profile.children.cycles-pp.__x64_sys_sync 0.09 ± 20% +0.3 0.36 ± 2% perf-profile.children.cycles-pp.ksys_sync 0.12 ± 11% +0.3 0.39 ± 2% perf-profile.children.cycles-pp.d_alloc 0.09 ± 20% +0.3 0.36 ± 2% perf-profile.children.cycles-pp.sync 0.75 ± 12% +0.3 1.03 ± 11% perf-profile.children.cycles-pp.hrtimer_interrupt 0.75 ± 12% +0.3 1.04 ± 12% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.15 ± 3% +0.3 0.45 ± 8% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.09 ± 20% +0.3 0.39 ± 3% perf-profile.children.cycles-pp.load_elf_interp 0.19 ± 12% +0.3 0.49 ± 3% perf-profile.children.cycles-pp.zap_pte_range 0.20 ± 11% +0.3 0.51 ± 3% perf-profile.children.cycles-pp.unmap_page_range 0.52 ± 19% +0.3 0.84 ± 3% perf-profile.children.cycles-pp.creat64 0.21 ± 8% +0.3 0.54 ± 3% perf-profile.children.cycles-pp.unmap_vmas 0.09 ± 24% +0.3 0.42 ± 11% perf-profile.children.cycles-pp.propagate_protected_usage 0.28 ± 14% +0.3 0.62 ± 10% perf-profile.children.cycles-pp.__strcat_chk 0.25 ± 7% +0.3 0.59 ± 4% perf-profile.children.cycles-pp.path_lookupat 0.25 ± 8% +0.4 0.60 ± 4% perf-profile.children.cycles-pp.filename_lookup 0.26 ± 8% +0.4 0.62 ± 4% perf-profile.children.cycles-pp.__do_sys_newstat 0.19 ± 11% +0.4 0.56 ± 4% perf-profile.children.cycles-pp.__alloc_file 0.19 ± 9% +0.4 0.56 ± 3% perf-profile.children.cycles-pp.alloc_empty_file 0.26 ± 8% +0.4 0.63 ± 4% perf-profile.children.cycles-pp.vfs_statx 0.12 ± 15% +0.4 0.49 ± 2% perf-profile.children.cycles-pp.elf_map 0.22 ± 11% +0.4 0.60 ± 2% perf-profile.children.cycles-pp.d_alloc_parallel 0.29 ± 8% +0.4 0.68 ± 3% perf-profile.children.cycles-pp.filemap_map_pages 0.24 ± 9% +0.4 0.63 ± 2% perf-profile.children.cycles-pp.__lookup_slow 0.30 ± 32% +0.4 0.73 ± 41% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.49 ± 20% +0.4 0.93 ± 10% perf-profile.children.cycles-pp.div_long 0.32 ± 10% +0.5 0.78 ± 3% perf-profile.children.cycles-pp.do_fault 0.52 ± 17% +0.5 1.04 ± 9% perf-profile.children.cycles-pp.add_int 0.51 ± 18% +0.5 1.04 ± 8% perf-profile.children.cycles-pp.add_long 0.50 ± 14% +0.6 1.12 ± 11% perf-profile.children.cycles-pp.__strncat_chk 0.12 ± 14% +0.6 0.74 ± 10% perf-profile.children.cycles-pp.terminate_walk 0.39 ± 19% +0.6 1.02 ± 18% perf-profile.children.cycles-pp.mprotect_fixup 0.39 ± 18% +0.6 1.03 ± 18% perf-profile.children.cycles-pp.__x64_sys_mprotect 0.39 ± 18% +0.6 1.03 ± 18% perf-profile.children.cycles-pp.do_mprotect_pkey 0.21 ± 23% +0.7 0.90 ± 13% perf-profile.children.cycles-pp.__memcg_kmem_uncharge 0.46 ± 21% +0.7 1.20 ± 21% perf-profile.children.cycles-pp.unmap_region 0.30 ± 17% +0.7 1.04 ± 3% perf-profile.children.cycles-pp.kmem_cache_alloc 0.48 ± 9% +0.8 1.23 ± 3% perf-profile.children.cycles-pp.__handle_mm_fault 0.50 ± 10% +0.8 1.30 ± 3% perf-profile.children.cycles-pp.handle_mm_fault 0.53 ± 9% +0.8 1.37 ± 3% perf-profile.children.cycles-pp.do_user_addr_fault 0.47 ± 9% +0.8 1.30 ± 2% perf-profile.children.cycles-pp.link_path_walk 0.54 ± 8% +0.8 1.38 ± 3% perf-profile.children.cycles-pp.exc_page_fault 0.48 ± 9% +0.9 1.33 ± 3% perf-profile.children.cycles-pp.walk_component 0.49 ± 18% +0.9 1.34 ± 13% perf-profile.children.cycles-pp.__libc_fork 0.50 ± 20% +0.9 1.41 ± 16% perf-profile.children.cycles-pp.vma_link 0.59 ± 4% +0.9 1.51 ± 4% perf-profile.children.cycles-pp.asm_exc_page_fault 0.34 ± 22% +1.1 1.43 ± 12% perf-profile.children.cycles-pp.page_counter_cancel 0.72 ± 19% +1.2 1.88 ± 19% perf-profile.children.cycles-pp.__vma_adjust 0.36 ± 22% +1.2 1.54 ± 12% perf-profile.children.cycles-pp.page_counter_uncharge 0.74 ± 19% +1.2 1.94 ± 19% perf-profile.children.cycles-pp.__split_vma 0.37 ± 21% +1.2 1.59 ± 5% perf-profile.children.cycles-pp.irq_exit_rcu 0.36 ± 22% +1.2 1.59 ± 5% perf-profile.children.cycles-pp.do_softirq_own_stack 0.00 +1.2 1.23 ± 19% perf-profile.children.cycles-pp.dcache_dir_open 0.00 +1.2 1.23 ± 19% perf-profile.children.cycles-pp.d_alloc_cursor 0.69 ± 15% +1.3 1.96 ± 14% perf-profile.children.cycles-pp.begin_new_exec 0.85 ± 20% +1.4 2.21 ± 20% perf-profile.children.cycles-pp.__do_munmap 0.81 ± 16% +1.4 2.22 ± 13% perf-profile.children.cycles-pp.__x64_sys_exit_group 0.81 ± 16% +1.4 2.22 ± 13% perf-profile.children.cycles-pp.do_group_exit 0.81 ± 16% +1.4 2.22 ± 13% perf-profile.children.cycles-pp.do_exit 1.09 ± 16% +1.4 2.50 ± 8% perf-profile.children.cycles-pp.asm_call_sysvec_on_stack 0.84 ± 20% +1.4 2.29 ± 15% perf-profile.children.cycles-pp.dup_mmap 0.39 ± 20% +1.5 1.84 ± 12% perf-profile.children.cycles-pp.drain_obj_stock 0.85 ± 20% +1.5 2.32 ± 15% perf-profile.children.cycles-pp.dup_mm 1.18 ± 15% +1.6 2.74 ± 8% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.94 ± 18% +1.6 2.58 ± 13% perf-profile.children.cycles-pp.copy_process 1.29 ± 11% +1.6 2.94 ± 7% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 1.00 ± 18% +1.7 2.72 ± 13% perf-profile.children.cycles-pp.__do_sys_clone 1.00 ± 18% +1.7 2.72 ± 13% perf-profile.children.cycles-pp.kernel_clone 0.47 ± 18% +1.8 2.25 ± 12% perf-profile.children.cycles-pp.refill_obj_stock 0.92 ± 14% +1.9 2.78 ± 10% perf-profile.children.cycles-pp.load_elf_binary 0.93 ± 14% +1.9 2.79 ± 10% perf-profile.children.cycles-pp.exec_binprm 1.68 ± 14% +2.1 3.74 ± 10% perf-profile.children.cycles-pp.string_rtns_1 1.30 ± 20% +2.1 3.37 ± 20% perf-profile.children.cycles-pp.ksys_mmap_pgoff 1.35 ± 19% +2.2 3.51 ± 24% perf-profile.children.cycles-pp.unlink_file_vma 1.36 ± 20% +2.3 3.63 ± 18% perf-profile.children.cycles-pp.mmap_region 1.37 ± 20% +2.3 3.67 ± 18% perf-profile.children.cycles-pp.do_mmap 1.39 ± 20% +2.3 3.72 ± 18% perf-profile.children.cycles-pp.vm_mmap_pgoff 1.46 ± 19% +2.5 3.92 ± 21% perf-profile.children.cycles-pp.free_pgtables 1.40 ± 16% +2.5 3.88 ± 14% perf-profile.children.cycles-pp.exit_mmap 1.40 ± 16% +2.5 3.89 ± 14% perf-profile.children.cycles-pp.mmput 0.00 +2.7 2.65 ± 9% perf-profile.children.cycles-pp.lockref_get 0.08 ± 5% +2.7 2.80 ± 8% perf-profile.children.cycles-pp.do_dentry_open 0.61 ± 11% +3.2 3.79 ± 15% perf-profile.children.cycles-pp.run_ksoftirqd 0.61 ± 12% +3.2 3.82 ± 15% perf-profile.children.cycles-pp.smpboot_thread_fn 0.65 ± 11% +3.2 3.88 ± 15% perf-profile.children.cycles-pp.kthread 0.66 ± 11% +3.3 3.92 ± 15% perf-profile.children.cycles-pp.ret_from_fork 0.91 ± 14% +4.3 5.25 ± 12% perf-profile.children.cycles-pp.rcu_do_batch 0.92 ± 13% +4.3 5.25 ± 12% perf-profile.children.cycles-pp.rcu_core 0.97 ± 13% +4.4 5.38 ± 12% perf-profile.children.cycles-pp.__softirqentry_text_start 0.99 ± 14% +4.5 5.52 ± 10% perf-profile.children.cycles-pp.kmem_cache_free 3.02 ± 19% +4.7 7.76 ± 23% perf-profile.children.cycles-pp.osq_lock 3.15 ± 19% +5.0 8.16 ± 22% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 3.33 ± 10% +6.1 9.38 ± 10% perf-profile.children.cycles-pp.intel_idle 0.00 +6.4 6.37 ± 2% perf-profile.children.cycles-pp.complete_walk 0.13 ± 3% +7.1 7.19 ± 18% perf-profile.children.cycles-pp.__open64_nocancel 3.89 ± 9% +7.1 10.99 ± 8% perf-profile.children.cycles-pp.cpuidle_enter_state 3.89 ± 9% +7.1 11.00 ± 8% perf-profile.children.cycles-pp.cpuidle_enter 0.28 ± 3% +7.2 7.48 ± 6% perf-profile.children.cycles-pp.lockref_put_or_lock 4.17 ± 6% +7.4 11.59 ± 8% perf-profile.children.cycles-pp.start_secondary 4.20 ± 6% +7.4 11.63 ± 8% perf-profile.children.cycles-pp.secondary_startup_64_no_verify 4.20 ± 6% +7.4 11.63 ± 8% perf-profile.children.cycles-pp.cpu_startup_entry 4.20 ± 6% +7.4 11.63 ± 8% perf-profile.children.cycles-pp.do_idle 0.54 ± 4% +7.5 8.04 ± 6% perf-profile.children.cycles-pp.dput 1.12 ± 2% +11.1 12.26 ± 2% perf-profile.children.cycles-pp.do_sys_open 1.12 ± 2% +11.1 12.25 ± 2% perf-profile.children.cycles-pp.do_sys_openat2 0.06 ± 14% +24.2 24.28 ± 4% perf-profile.children.cycles-pp.pick_link 1.03 ± 4% +24.4 25.43 ± 8% perf-profile.children.cycles-pp.execve 0.17 ± 10% +30.8 30.92 ± 3% perf-profile.children.cycles-pp.lockref_get_not_dead 0.10 ± 8% +30.8 30.87 ± 3% perf-profile.children.cycles-pp.do_open_execat 0.18 ± 9% +30.8 30.95 ± 3% perf-profile.children.cycles-pp.__legitimize_path 0.18 ± 12% +30.8 30.96 ± 3% perf-profile.children.cycles-pp.unlazy_walk 0.20 ± 10% +31.0 31.15 ± 3% perf-profile.children.cycles-pp.step_into 1.05 ± 14% +32.7 33.76 ± 3% perf-profile.children.cycles-pp.bprm_execve 1.14 ± 13% +32.9 33.99 ± 3% perf-profile.children.cycles-pp.do_execveat_common 1.15 ± 13% +32.9 34.01 ± 3% perf-profile.children.cycles-pp.__x64_sys_execve 1.16 ± 3% +41.8 42.98 ± 2% perf-profile.children.cycles-pp.path_openat 1.17 ± 3% +41.8 42.98 ± 2% perf-profile.children.cycles-pp.do_filp_open 0.00 +54.2 54.20 ± 2% perf-profile.children.cycles-pp.__cna_queued_spin_lock_slowpath 81.68 -81.7 0.00 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.05 ± 8% +0.0 0.08 ± 21% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.00 +0.1 0.05 perf-profile.self.cycles-pp.link_path_walk 0.00 +0.1 0.05 perf-profile.self.cycles-pp.apparmor_task_getsecid 0.00 +0.1 0.05 perf-profile.self.cycles-pp.page_add_file_rmap 0.00 +0.1 0.05 perf-profile.self.cycles-pp.common_file_perm 0.00 +0.1 0.05 perf-profile.self.cycles-pp.__list_del_entry_valid 0.00 +0.1 0.05 perf-profile.self.cycles-pp.copy_page 0.00 +0.1 0.05 perf-profile.self.cycles-pp._raw_spin_trylock 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.__rb_insert_augmented 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.compar1 0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.__might_sleep 0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.sync_regs 0.00 +0.1 0.06 ± 14% perf-profile.self.cycles-pp.__alloc_file 0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.kfree 0.00 +0.1 0.06 perf-profile.self.cycles-pp.strnlen_user 0.00 +0.1 0.06 perf-profile.self.cycles-pp.anon_vma_interval_tree_insert 0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.memcpy_erms 0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.free_pages_and_swap_cache 0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.filldir64 0.00 +0.1 0.07 ± 7% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.06 ± 16% +0.1 0.12 ± 3% perf-profile.self.cycles-pp.__d_lookup_rcu 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.copy_pte_range 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.00 +0.1 0.07 ± 19% perf-profile.self.cycles-pp.file_free_rcu 0.00 +0.1 0.07 ± 10% perf-profile.self.cycles-pp.rwsem_down_write_slowpath 0.05 ± 8% +0.1 0.12 ± 4% perf-profile.self.cycles-pp.apparmor_file_open 0.05 +0.1 0.12 ± 3% perf-profile.self.cycles-pp.apparmor_file_free_security 0.00 +0.1 0.07 ± 5% perf-profile.self.cycles-pp.vma_interval_tree_remove 0.07 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.native_irq_return_iret 0.00 +0.1 0.08 ± 6% perf-profile.self.cycles-pp.__mod_memcg_state 0.00 +0.1 0.08 ± 11% perf-profile.self.cycles-pp.rcu_cblist_dequeue 0.00 +0.1 0.08 ± 14% perf-profile.self.cycles-pp.__handle_mm_fault 0.00 +0.1 0.08 ± 5% perf-profile.self.cycles-pp.down_read 0.02 ±141% +0.1 0.10 ± 5% perf-profile.self.cycles-pp.up_write 0.00 +0.1 0.08 ± 13% perf-profile.self.cycles-pp.__slab_free 0.06 ± 8% +0.1 0.14 ± 8% perf-profile.self.cycles-pp.lockref_get_not_dead 0.00 +0.1 0.08 ± 5% perf-profile.self.cycles-pp.clear_page_erms 0.03 ± 70% +0.1 0.12 ± 6% perf-profile.self.cycles-pp.down_write 0.00 +0.1 0.09 ± 7% perf-profile.self.cycles-pp.malloc 0.00 +0.1 0.09 ± 7% perf-profile.self.cycles-pp.aa_get_task_label 0.02 ±141% +0.1 0.11 ± 4% perf-profile.self.cycles-pp.___might_sleep 0.09 ± 13% +0.1 0.19 ± 14% perf-profile.self.cycles-pp.update_cfs_group 0.06 ± 13% +0.1 0.15 ± 5% perf-profile.self.cycles-pp.page_remove_rmap 0.07 ± 11% +0.1 0.17 ± 2% perf-profile.self.cycles-pp.find_idlest_group 0.00 +0.1 0.10 ± 4% perf-profile.self.cycles-pp.lockref_put_return 0.07 ± 7% +0.1 0.17 ± 6% perf-profile.self.cycles-pp.apparmor_file_alloc_security 0.00 +0.1 0.11 ± 7% perf-profile.self.cycles-pp.update_sd_lb_stats 0.08 ± 10% +0.1 0.19 ± 3% perf-profile.self.cycles-pp.vma_interval_tree_insert 0.04 ± 71% +0.1 0.15 ± 5% perf-profile.self.cycles-pp.release_pages 0.00 +0.1 0.12 ± 13% perf-profile.self.cycles-pp.page_counter_uncharge 0.07 ± 11% +0.1 0.19 ± 2% perf-profile.self.cycles-pp._dl_addr 0.00 +0.1 0.13 ± 3% perf-profile.self.cycles-pp.cna_order_queue 0.10 ± 8% +0.1 0.25 ± 2% perf-profile.self.cycles-pp.zap_pte_range 0.08 ± 22% +0.2 0.23 ± 4% perf-profile.self.cycles-pp.page_counter_try_charge 0.08 ± 14% +0.2 0.25 ± 2% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.21 ± 20% +0.2 0.40 ± 11% perf-profile.self.cycles-pp.div_int 0.15 ± 9% +0.2 0.35 ± 11% perf-profile.self.cycles-pp.mem_rtns_1 0.25 ± 22% +0.2 0.45 ± 11% perf-profile.self.cycles-pp.div_short 0.22 ± 19% +0.2 0.45 ± 10% perf-profile.self.cycles-pp.add_short 0.08 ± 27% +0.2 0.32 ± 4% perf-profile.self.cycles-pp.get_obj_cgroup_from_current 0.42 ± 2% +0.3 0.69 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 0.22 ± 9% +0.3 0.49 ± 2% perf-profile.self.cycles-pp.filemap_map_pages 0.11 ± 19% +0.3 0.41 ± 3% perf-profile.self.cycles-pp.kmem_cache_alloc 0.07 ± 6% +0.3 0.40 ± 10% perf-profile.self.cycles-pp.refill_obj_stock 0.28 ± 14% +0.3 0.60 ± 11% perf-profile.self.cycles-pp.__strcat_chk 0.09 ± 24% +0.3 0.42 ± 11% perf-profile.self.cycles-pp.propagate_protected_usage 0.07 ± 11% +0.4 0.46 ± 10% perf-profile.self.cycles-pp.drain_obj_stock 0.49 ± 19% +0.4 0.92 ± 10% perf-profile.self.cycles-pp.div_long 0.51 ± 17% +0.5 1.01 ± 8% perf-profile.self.cycles-pp.add_long 0.51 ± 17% +0.5 1.02 ± 9% perf-profile.self.cycles-pp.add_int 0.49 ± 14% +0.6 1.10 ± 11% perf-profile.self.cycles-pp.__strncat_chk 0.26 ± 21% +0.8 1.04 ± 12% perf-profile.self.cycles-pp.page_counter_cancel 1.66 ± 14% +2.0 3.65 ± 10% perf-profile.self.cycles-pp.string_rtns_1 0.49 ± 13% +2.6 3.11 ± 9% perf-profile.self.cycles-pp.kmem_cache_free 2.93 ± 19% +4.6 7.50 ± 22% perf-profile.self.cycles-pp.osq_lock 3.33 ± 10% +6.0 9.38 ± 10% perf-profile.self.cycles-pp.intel_idle 0.00 +53.4 53.38 ± 2% perf-profile.self.cycles-pp.__cna_queued_spin_lock_slowpath 1579 ± 53% +572.8% 10624 ± 82% interrupts.31:PCI-MSI.524289-edge.eth0-TxRx-0 1203 ±112% +61.3% 1940 ± 68% interrupts.32:PCI-MSI.524290-edge.eth0-TxRx-1 8340 ± 68% -76.0% 2003 ± 74% interrupts.34:PCI-MSI.524292-edge.eth0-TxRx-3 775419 ± 5% +21.9% 944875 ± 2% interrupts.CAL:Function_call_interrupts 5146 ± 4% -51.2% 2512 ± 34% interrupts.CPU0.NMI:Non-maskable_interrupts 5146 ± 4% -51.2% 2512 ± 34% interrupts.CPU0.PMI:Performance_monitoring_interrupts 1113 ± 8% +13.1% 1259 ± 4% interrupts.CPU0.RES:Rescheduling_interrupts 643.67 ± 4% +53.6% 988.50 ± 7% interrupts.CPU0.TLB:TLB_shootdowns 4043 ± 7% +33.6% 5400 ± 5% interrupts.CPU1.CAL:Function_call_interrupts 5177 ± 3% -41.7% 3020 ± 36% interrupts.CPU1.NMI:Non-maskable_interrupts 5177 ± 3% -41.7% 3020 ± 36% interrupts.CPU1.PMI:Performance_monitoring_interrupts 735.33 ± 5% +48.3% 1090 ± 8% interrupts.CPU1.RES:Rescheduling_interrupts 562.00 ± 12% +47.5% 829.00 ± 6% interrupts.CPU1.TLB:TLB_shootdowns 1579 ± 53% +572.8% 10624 ± 82% interrupts.CPU10.31:PCI-MSI.524289-edge.eth0-TxRx-0 3821 ± 7% +28.3% 4903 ± 6% interrupts.CPU10.CAL:Function_call_interrupts 5187 ± 4% -23.5% 3968 ± 5% interrupts.CPU10.NMI:Non-maskable_interrupts 5187 ± 4% -23.5% 3968 ± 5% interrupts.CPU10.PMI:Performance_monitoring_interrupts 738.67 ± 5% +48.7% 1098 ± 4% interrupts.CPU10.RES:Rescheduling_interrupts 522.00 ± 33% +72.4% 900.00 ± 10% interrupts.CPU10.TLB:TLB_shootdowns 3934 ± 9% +24.9% 4913 ± 6% interrupts.CPU100.CAL:Function_call_interrupts 5204 ± 3% -24.7% 3916 ± 8% interrupts.CPU100.NMI:Non-maskable_interrupts 5204 ± 3% -24.7% 3916 ± 8% interrupts.CPU100.PMI:Performance_monitoring_interrupts 743.67 ± 7% +61.4% 1200 ± 9% interrupts.CPU100.RES:Rescheduling_interrupts 474.00 ± 14% +91.8% 909.00 ± 12% interrupts.CPU100.TLB:TLB_shootdowns 3945 ± 4% +24.3% 4901 ± 4% interrupts.CPU101.CAL:Function_call_interrupts 5197 ± 3% -36.2% 3313 ± 26% interrupts.CPU101.NMI:Non-maskable_interrupts 5197 ± 3% -36.2% 3313 ± 26% interrupts.CPU101.PMI:Performance_monitoring_interrupts 720.33 ± 5% +60.8% 1158 ± 7% interrupts.CPU101.RES:Rescheduling_interrupts 496.67 ± 7% +82.6% 906.75 ± 15% interrupts.CPU101.TLB:TLB_shootdowns 3918 ± 5% +22.8% 4810 ± 6% interrupts.CPU102.CAL:Function_call_interrupts 5186 ± 4% -25.2% 3879 ± 10% interrupts.CPU102.NMI:Non-maskable_interrupts 5186 ± 4% -25.2% 3879 ± 10% interrupts.CPU102.PMI:Performance_monitoring_interrupts 699.33 ± 3% +61.8% 1131 ± 7% interrupts.CPU102.RES:Rescheduling_interrupts 406.33 ± 7% +108.2% 846.00 ± 12% interrupts.CPU102.TLB:TLB_shootdowns 3882 ± 5% +26.2% 4897 ± 4% interrupts.CPU103.CAL:Function_call_interrupts 5224 ± 3% -34.9% 3399 ± 26% interrupts.CPU103.NMI:Non-maskable_interrupts 5224 ± 3% -34.9% 3399 ± 26% interrupts.CPU103.PMI:Performance_monitoring_interrupts 658.33 ± 4% +73.4% 1141 ± 5% interrupts.CPU103.RES:Rescheduling_interrupts 449.00 ± 16% +89.6% 851.50 ± 8% interrupts.CPU103.TLB:TLB_shootdowns 3881 ± 3% +23.9% 4807 ± 3% interrupts.CPU104.CAL:Function_call_interrupts 5212 ± 3% -26.6% 3826 ± 8% interrupts.CPU104.NMI:Non-maskable_interrupts 5212 ± 3% -26.6% 3826 ± 8% interrupts.CPU104.PMI:Performance_monitoring_interrupts 637.67 ± 7% +73.6% 1106 ± 9% interrupts.CPU104.RES:Rescheduling_interrupts 423.33 ± 7% +90.1% 804.75 ± 14% interrupts.CPU104.TLB:TLB_shootdowns 3845 +20.2% 4623 interrupts.CPU105.CAL:Function_call_interrupts 661.33 ± 8% +68.5% 1114 ± 2% interrupts.CPU105.RES:Rescheduling_interrupts 467.33 ± 17% +87.6% 876.50 ± 21% interrupts.CPU105.TLB:TLB_shootdowns 3959 ± 4% +18.2% 4679 interrupts.CPU106.CAL:Function_call_interrupts 5251 ± 3% -25.6% 3905 ± 10% interrupts.CPU106.NMI:Non-maskable_interrupts 5251 ± 3% -25.6% 3905 ± 10% interrupts.CPU106.PMI:Performance_monitoring_interrupts 721.33 ± 5% +59.2% 1148 ± 8% interrupts.CPU106.RES:Rescheduling_interrupts 419.00 ± 15% +150.8% 1050 ± 30% interrupts.CPU106.TLB:TLB_shootdowns 3930 ± 5% +18.9% 4674 ± 4% interrupts.CPU107.CAL:Function_call_interrupts 5177 ± 5% -34.4% 3398 ± 26% interrupts.CPU107.NMI:Non-maskable_interrupts 5177 ± 5% -34.4% 3398 ± 26% interrupts.CPU107.PMI:Performance_monitoring_interrupts 665.33 ± 5% +75.8% 1169 ± 4% interrupts.CPU107.RES:Rescheduling_interrupts 445.00 ± 18% +83.9% 818.50 ± 13% interrupts.CPU107.TLB:TLB_shootdowns 3915 ± 8% +20.9% 4735 ± 3% interrupts.CPU108.CAL:Function_call_interrupts 5255 ± 3% -36.2% 3350 ± 25% interrupts.CPU108.NMI:Non-maskable_interrupts 5255 ± 3% -36.2% 3350 ± 25% interrupts.CPU108.PMI:Performance_monitoring_interrupts 684.33 ± 7% +67.5% 1146 ± 7% interrupts.CPU108.RES:Rescheduling_interrupts 509.00 ± 11% +66.4% 846.75 ± 6% interrupts.CPU108.TLB:TLB_shootdowns 3965 ± 3% +19.8% 4749 ± 4% interrupts.CPU109.CAL:Function_call_interrupts 5207 ± 3% -42.8% 2976 ± 38% interrupts.CPU109.NMI:Non-maskable_interrupts 5207 ± 3% -42.8% 2976 ± 38% interrupts.CPU109.PMI:Performance_monitoring_interrupts 701.00 ± 4% +55.2% 1087 ± 5% interrupts.CPU109.RES:Rescheduling_interrupts 510.33 ± 11% +71.9% 877.25 ± 6% interrupts.CPU109.TLB:TLB_shootdowns 1203 ±112% +61.3% 1940 ± 68% interrupts.CPU11.32:PCI-MSI.524290-edge.eth0-TxRx-1 3794 ± 5% +31.8% 4999 ± 4% interrupts.CPU11.CAL:Function_call_interrupts 734.33 ± 6% +51.7% 1114 ± 3% interrupts.CPU11.RES:Rescheduling_interrupts 458.67 ± 14% +100.9% 921.50 ± 10% interrupts.CPU11.TLB:TLB_shootdowns 3949 ± 5% +27.3% 5029 ± 4% interrupts.CPU110.CAL:Function_call_interrupts 5230 ± 3% -35.3% 3385 ± 24% interrupts.CPU110.NMI:Non-maskable_interrupts 5230 ± 3% -35.3% 3385 ± 24% interrupts.CPU110.PMI:Performance_monitoring_interrupts 670.67 ± 6% +74.5% 1170 ± 7% interrupts.CPU110.RES:Rescheduling_interrupts 442.00 ± 8% +88.5% 833.00 ± 18% interrupts.CPU110.TLB:TLB_shootdowns 3865 ± 4% +24.2% 4802 ± 4% interrupts.CPU111.CAL:Function_call_interrupts 5230 ± 3% -36.3% 3330 ± 25% interrupts.CPU111.NMI:Non-maskable_interrupts 5230 ± 3% -36.3% 3330 ± 25% interrupts.CPU111.PMI:Performance_monitoring_interrupts 634.00 ± 9% +85.7% 1177 ± 10% interrupts.CPU111.RES:Rescheduling_interrupts 441.33 ± 13% +96.4% 866.75 ± 14% interrupts.CPU111.TLB:TLB_shootdowns 3873 ± 5% +24.7% 4829 ± 2% interrupts.CPU112.CAL:Function_call_interrupts 5121 ± 4% -35.0% 3326 ± 24% interrupts.CPU112.NMI:Non-maskable_interrupts 5121 ± 4% -35.0% 3326 ± 24% interrupts.CPU112.PMI:Performance_monitoring_interrupts 669.00 ± 2% +72.3% 1152 ± 7% interrupts.CPU112.RES:Rescheduling_interrupts 430.00 ± 2% +111.9% 911.00 ± 15% interrupts.CPU112.TLB:TLB_shootdowns 3921 ± 6% +24.6% 4885 ± 3% interrupts.CPU113.CAL:Function_call_interrupts 672.33 ± 8% +70.0% 1142 ± 7% interrupts.CPU113.RES:Rescheduling_interrupts 431.33 ± 9% +98.2% 855.00 ± 11% interrupts.CPU113.TLB:TLB_shootdowns 3779 ± 7% +27.7% 4827 ± 4% interrupts.CPU114.CAL:Function_call_interrupts 641.00 ± 13% +79.3% 1149 ± 4% interrupts.CPU114.RES:Rescheduling_interrupts 504.00 ± 10% +77.2% 893.25 ± 20% interrupts.CPU114.TLB:TLB_shootdowns 3904 ± 6% +20.2% 4693 interrupts.CPU115.CAL:Function_call_interrupts 651.67 ± 3% +78.9% 1165 ± 2% interrupts.CPU115.RES:Rescheduling_interrupts 501.67 ± 29% +69.9% 852.50 ± 4% interrupts.CPU115.TLB:TLB_shootdowns 3818 ± 3% +23.2% 4703 interrupts.CPU116.CAL:Function_call_interrupts 630.67 ± 7% +81.6% 1145 ± 6% interrupts.CPU116.RES:Rescheduling_interrupts 439.67 ± 14% +98.7% 873.50 ± 15% interrupts.CPU116.TLB:TLB_shootdowns 4154 ± 4% +13.8% 4728 interrupts.CPU117.CAL:Function_call_interrupts 650.67 ± 8% +70.7% 1110 ± 6% interrupts.CPU117.RES:Rescheduling_interrupts 502.33 ± 12% +77.0% 889.00 ± 10% interrupts.CPU117.TLB:TLB_shootdowns 3947 ± 5% +21.6% 4798 ± 5% interrupts.CPU118.CAL:Function_call_interrupts 659.33 +71.4% 1130 ± 4% interrupts.CPU118.RES:Rescheduling_interrupts 464.67 ± 11% +84.5% 857.25 ± 9% interrupts.CPU118.TLB:TLB_shootdowns 4065 ± 4% +19.6% 4860 ± 5% interrupts.CPU119.CAL:Function_call_interrupts 614.00 ± 8% +82.9% 1123 ± 4% interrupts.CPU119.RES:Rescheduling_interrupts 480.33 ± 8% +74.6% 838.50 ± 8% interrupts.CPU119.TLB:TLB_shootdowns 4011 ± 3% +20.9% 4848 ± 7% interrupts.CPU12.CAL:Function_call_interrupts 769.33 ± 2% +44.1% 1108 ± 8% interrupts.CPU12.RES:Rescheduling_interrupts 502.00 ± 4% +96.4% 986.00 ± 12% interrupts.CPU12.TLB:TLB_shootdowns 4091 ± 4% +22.3% 5003 ± 3% interrupts.CPU120.CAL:Function_call_interrupts 717.33 ± 9% +64.2% 1177 ± 6% interrupts.CPU120.RES:Rescheduling_interrupts 459.67 ± 14% +103.1% 933.50 ± 15% interrupts.CPU120.TLB:TLB_shootdowns 3923 ± 4% +25.2% 4913 ± 3% interrupts.CPU121.CAL:Function_call_interrupts 617.33 ± 3% +89.4% 1169 ± 4% interrupts.CPU121.RES:Rescheduling_interrupts 401.67 ± 22% +195.5% 1186 ± 39% interrupts.CPU121.TLB:TLB_shootdowns 3867 ± 4% +34.9% 5218 ± 4% interrupts.CPU122.CAL:Function_call_interrupts 596.67 ± 8% +100.9% 1198 ± 5% interrupts.CPU122.RES:Rescheduling_interrupts 450.00 ± 16% +109.0% 940.50 ± 6% interrupts.CPU122.TLB:TLB_shootdowns 4031 ± 8% +23.2% 4967 ± 3% interrupts.CPU123.CAL:Function_call_interrupts 628.00 ± 4% +94.1% 1218 ± 7% interrupts.CPU123.RES:Rescheduling_interrupts 474.67 ± 13% +96.1% 931.00 ± 12% interrupts.CPU123.TLB:TLB_shootdowns 4232 ± 4% +20.3% 5090 ± 3% interrupts.CPU124.CAL:Function_call_interrupts 639.00 +82.9% 1168 ± 7% interrupts.CPU124.RES:Rescheduling_interrupts 463.33 ± 17% +110.1% 973.25 ± 11% interrupts.CPU124.TLB:TLB_shootdowns 3972 +21.5% 4826 ± 3% interrupts.CPU125.CAL:Function_call_interrupts 658.67 ± 8% +78.4% 1175 ± 4% interrupts.CPU125.RES:Rescheduling_interrupts 480.00 ± 13% +96.0% 940.75 ± 23% interrupts.CPU125.TLB:TLB_shootdowns 4093 ± 3% +20.8% 4947 ± 4% interrupts.CPU126.CAL:Function_call_interrupts 672.67 ± 5% +75.9% 1183 ± 2% interrupts.CPU126.RES:Rescheduling_interrupts 498.00 ± 10% +90.0% 946.25 ± 11% interrupts.CPU126.TLB:TLB_shootdowns 4111 ± 5% +20.8% 4965 ± 5% interrupts.CPU127.CAL:Function_call_interrupts 613.67 ± 2% +95.2% 1198 ± 5% interrupts.CPU127.RES:Rescheduling_interrupts 451.67 ± 21% +109.8% 947.50 ± 14% interrupts.CPU127.TLB:TLB_shootdowns 4066 +20.3% 4891 ± 3% interrupts.CPU128.CAL:Function_call_interrupts 5200 ± 2% -23.3% 3986 ± 9% interrupts.CPU128.NMI:Non-maskable_interrupts 5200 ± 2% -23.3% 3986 ± 9% interrupts.CPU128.PMI:Performance_monitoring_interrupts 663.33 ± 4% +74.6% 1158 ± 6% interrupts.CPU128.RES:Rescheduling_interrupts 472.33 ± 17% +102.4% 956.00 ± 8% interrupts.CPU128.TLB:TLB_shootdowns 4075 ± 8% +20.3% 4900 interrupts.CPU129.CAL:Function_call_interrupts 640.33 ± 2% +73.2% 1108 ± 8% interrupts.CPU129.RES:Rescheduling_interrupts 445.33 ± 19% +113.4% 950.25 ± 11% interrupts.CPU129.TLB:TLB_shootdowns 8340 ± 68% -76.0% 2003 ± 74% interrupts.CPU13.34:PCI-MSI.524292-edge.eth0-TxRx-3 4049 ± 7% +20.6% 4885 ± 2% interrupts.CPU13.CAL:Function_call_interrupts 5130 ± 3% -24.2% 3886 ± 5% interrupts.CPU13.NMI:Non-maskable_interrupts 5130 ± 3% -24.2% 3886 ± 5% interrupts.CPU13.PMI:Performance_monitoring_interrupts 724.00 ± 5% +57.8% 1142 ± 5% interrupts.CPU13.RES:Rescheduling_interrupts 532.67 ± 23% +70.5% 908.25 ± 12% interrupts.CPU13.TLB:TLB_shootdowns 3881 ± 6% +30.0% 5044 ± 4% interrupts.CPU130.CAL:Function_call_interrupts 621.33 ± 10% +89.4% 1176 ± 8% interrupts.CPU130.RES:Rescheduling_interrupts 516.33 ± 16% +92.9% 996.25 ± 15% interrupts.CPU130.TLB:TLB_shootdowns 4035 ± 2% +22.0% 4923 ± 3% interrupts.CPU131.CAL:Function_call_interrupts 661.67 ± 7% +69.5% 1121 ± 4% interrupts.CPU131.RES:Rescheduling_interrupts 425.67 ± 11% +116.8% 922.75 ± 15% interrupts.CPU131.TLB:TLB_shootdowns 4345 ± 3% +13.3% 4924 ± 5% interrupts.CPU132.CAL:Function_call_interrupts 694.00 +68.9% 1172 ± 5% interrupts.CPU132.RES:Rescheduling_interrupts 476.00 ± 11% +94.5% 926.00 ± 13% interrupts.CPU132.TLB:TLB_shootdowns 4091 ± 7% +22.7% 5018 ± 4% interrupts.CPU133.CAL:Function_call_interrupts 663.67 ± 6% +68.6% 1119 ± 4% interrupts.CPU133.RES:Rescheduling_interrupts 534.33 ± 9% +78.7% 954.75 ± 13% interrupts.CPU133.TLB:TLB_shootdowns 3856 ± 7% +26.7% 4886 ± 3% interrupts.CPU134.CAL:Function_call_interrupts 5199 ± 2% -34.5% 3406 ± 24% interrupts.CPU134.NMI:Non-maskable_interrupts 5199 ± 2% -34.5% 3406 ± 24% interrupts.CPU134.PMI:Performance_monitoring_interrupts 667.00 ± 13% +74.7% 1165 ± 6% interrupts.CPU134.RES:Rescheduling_interrupts 447.33 ± 9% +116.7% 969.25 ± 3% interrupts.CPU134.TLB:TLB_shootdowns 4120 ± 11% +18.6% 4886 ± 3% interrupts.CPU135.CAL:Function_call_interrupts 5233 ± 3% -24.8% 3934 ± 9% interrupts.CPU135.NMI:Non-maskable_interrupts 5233 ± 3% -24.8% 3934 ± 9% interrupts.CPU135.PMI:Performance_monitoring_interrupts 643.67 ± 5% +81.8% 1170 ± 7% interrupts.CPU135.RES:Rescheduling_interrupts 490.00 ± 17% +88.4% 923.00 ± 18% interrupts.CPU135.TLB:TLB_shootdowns 4092 ± 3% +22.8% 5026 ± 3% interrupts.CPU136.CAL:Function_call_interrupts 5225 ± 3% -34.3% 3430 ± 26% interrupts.CPU136.NMI:Non-maskable_interrupts 5225 ± 3% -34.3% 3430 ± 26% interrupts.CPU136.PMI:Performance_monitoring_interrupts 673.33 ± 4% +80.6% 1216 ± 4% interrupts.CPU136.RES:Rescheduling_interrupts 488.00 ± 8% +87.9% 916.75 ± 13% interrupts.CPU136.TLB:TLB_shootdowns 4234 ± 9% +16.3% 4925 ± 4% interrupts.CPU137.CAL:Function_call_interrupts 5217 ± 4% -34.5% 3418 ± 24% interrupts.CPU137.NMI:Non-maskable_interrupts 5217 ± 4% -34.5% 3418 ± 24% interrupts.CPU137.PMI:Performance_monitoring_interrupts 625.33 ± 14% +87.5% 1172 ± 6% interrupts.CPU137.RES:Rescheduling_interrupts 441.67 ± 10% +115.5% 951.75 ± 13% interrupts.CPU137.TLB:TLB_shootdowns 3987 ± 6% +23.1% 4907 ± 2% interrupts.CPU138.CAL:Function_call_interrupts 5218 ± 3% -42.7% 2992 ± 38% interrupts.CPU138.NMI:Non-maskable_interrupts 5218 ± 3% -42.7% 2992 ± 38% interrupts.CPU138.PMI:Performance_monitoring_interrupts 672.33 ± 3% +76.5% 1186 ± 4% interrupts.CPU138.RES:Rescheduling_interrupts 439.67 ± 8% +113.4% 938.25 ± 11% interrupts.CPU138.TLB:TLB_shootdowns 4040 ± 7% +23.1% 4973 ± 4% interrupts.CPU139.CAL:Function_call_interrupts 635.33 ± 3% +85.3% 1177 ± 7% interrupts.CPU139.RES:Rescheduling_interrupts 425.33 ± 17% +110.7% 896.00 ± 10% interrupts.CPU139.TLB:TLB_shootdowns 3920 ± 2% +21.7% 4771 ± 2% interrupts.CPU14.CAL:Function_call_interrupts 5123 ± 4% -43.3% 2903 ± 30% interrupts.CPU14.NMI:Non-maskable_interrupts 5123 ± 4% -43.3% 2903 ± 30% interrupts.CPU14.PMI:Performance_monitoring_interrupts 725.00 ± 12% +56.2% 1132 ± 6% interrupts.CPU14.RES:Rescheduling_interrupts 402.00 ± 6% +110.0% 844.25 ± 10% interrupts.CPU14.TLB:TLB_shootdowns 4006 ± 11% +19.8% 4800 ± 3% interrupts.CPU140.CAL:Function_call_interrupts 5230 ± 3% -24.8% 3932 ± 10% interrupts.CPU140.NMI:Non-maskable_interrupts 5230 ± 3% -24.8% 3932 ± 10% interrupts.CPU140.PMI:Performance_monitoring_interrupts 578.33 ± 3% +102.5% 1171 ± 5% interrupts.CPU140.RES:Rescheduling_interrupts 506.00 ± 5% +90.3% 963.00 ± 11% interrupts.CPU140.TLB:TLB_shootdowns 4080 ± 6% +25.8% 5131 ± 7% interrupts.CPU141.CAL:Function_call_interrupts 5148 ± 4% -33.6% 3418 ± 25% interrupts.CPU141.NMI:Non-maskable_interrupts 5148 ± 4% -33.6% 3418 ± 25% interrupts.CPU141.PMI:Performance_monitoring_interrupts 695.00 ± 3% +74.7% 1214 ± 7% interrupts.CPU141.RES:Rescheduling_interrupts 503.33 ± 23% +150.9% 1263 ± 44% interrupts.CPU141.TLB:TLB_shootdowns 4155 ± 8% +16.6% 4843 ± 4% interrupts.CPU142.CAL:Function_call_interrupts 638.00 ± 6% +82.1% 1161 ± 6% interrupts.CPU142.RES:Rescheduling_interrupts 509.33 ± 8% +89.0% 962.50 ± 14% interrupts.CPU142.TLB:TLB_shootdowns 4018 ± 6% +24.3% 4993 interrupts.CPU143.CAL:Function_call_interrupts 643.67 ± 2% +86.6% 1201 ± 8% interrupts.CPU143.RES:Rescheduling_interrupts 553.67 ± 7% +83.0% 1013 ± 10% interrupts.CPU143.TLB:TLB_shootdowns 3990 ± 5% +25.0% 4989 ± 4% interrupts.CPU144.CAL:Function_call_interrupts 654.00 ± 10% +78.0% 1164 ± 4% interrupts.CPU144.RES:Rescheduling_interrupts 415.33 ± 4% +124.9% 934.25 ± 9% interrupts.CPU144.TLB:TLB_shootdowns 3875 ± 3% +30.5% 5056 ± 6% interrupts.CPU145.CAL:Function_call_interrupts 647.00 ± 13% +79.9% 1163 ± 4% interrupts.CPU145.RES:Rescheduling_interrupts 445.67 ± 16% +112.7% 947.75 ± 12% interrupts.CPU145.TLB:TLB_shootdowns 3891 ± 7% +24.8% 4857 ± 5% interrupts.CPU146.CAL:Function_call_interrupts 635.00 ± 6% +80.7% 1147 ± 5% interrupts.CPU146.RES:Rescheduling_interrupts 438.67 ± 11% +127.1% 996.25 ± 10% interrupts.CPU146.TLB:TLB_shootdowns 3899 ± 7% +28.0% 4990 ± 4% interrupts.CPU147.CAL:Function_call_interrupts 5189 ± 3% -24.6% 3915 ± 10% interrupts.CPU147.NMI:Non-maskable_interrupts 5189 ± 3% -24.6% 3915 ± 10% interrupts.CPU147.PMI:Performance_monitoring_interrupts 643.33 +80.3% 1160 ± 2% interrupts.CPU147.RES:Rescheduling_interrupts 430.00 ± 16% +116.9% 932.50 ± 17% interrupts.CPU147.TLB:TLB_shootdowns 4129 ± 12% +18.4% 4889 interrupts.CPU148.CAL:Function_call_interrupts 5219 ± 3% -24.7% 3932 ± 9% interrupts.CPU148.NMI:Non-maskable_interrupts 5219 ± 3% -24.7% 3932 ± 9% interrupts.CPU148.PMI:Performance_monitoring_interrupts 588.33 ± 7% +93.6% 1139 ± 5% interrupts.CPU148.RES:Rescheduling_interrupts 501.00 ± 16% +78.7% 895.25 ± 8% interrupts.CPU148.TLB:TLB_shootdowns 4186 ± 9% +18.3% 4953 ± 3% interrupts.CPU149.CAL:Function_call_interrupts 5185 ± 4% -32.3% 3509 ± 31% interrupts.CPU149.NMI:Non-maskable_interrupts 5185 ± 4% -32.3% 3509 ± 31% interrupts.CPU149.PMI:Performance_monitoring_interrupts 638.67 +70.9% 1091 ± 2% interrupts.CPU149.RES:Rescheduling_interrupts 558.33 ± 15% +71.8% 959.25 ± 7% interrupts.CPU149.TLB:TLB_shootdowns 3912 ± 6% +21.4% 4750 ± 4% interrupts.CPU15.CAL:Function_call_interrupts 5197 ± 3% -22.8% 4013 ± 4% interrupts.CPU15.NMI:Non-maskable_interrupts 5197 ± 3% -22.8% 4013 ± 4% interrupts.CPU15.PMI:Performance_monitoring_interrupts 776.33 ± 7% +39.1% 1080 ± 6% interrupts.CPU15.RES:Rescheduling_interrupts 473.00 ± 12% +98.5% 938.75 ± 11% interrupts.CPU15.TLB:TLB_shootdowns 4178 ± 9% +18.1% 4936 ± 4% interrupts.CPU150.CAL:Function_call_interrupts 5192 ± 4% -25.4% 3873 ± 10% interrupts.CPU150.NMI:Non-maskable_interrupts 5192 ± 4% -25.4% 3873 ± 10% interrupts.CPU150.PMI:Performance_monitoring_interrupts 678.67 ± 8% +64.3% 1115 ± 5% interrupts.CPU150.RES:Rescheduling_interrupts 513.33 ± 28% +73.6% 891.00 ± 21% interrupts.CPU150.TLB:TLB_shootdowns 4052 ± 6% +21.9% 4939 ± 3% interrupts.CPU151.CAL:Function_call_interrupts 5260 ± 3% -25.8% 3903 ± 10% interrupts.CPU151.NMI:Non-maskable_interrupts 5260 ± 3% -25.8% 3903 ± 10% interrupts.CPU151.PMI:Performance_monitoring_interrupts 623.33 ± 3% +86.9% 1164 ± 4% interrupts.CPU151.RES:Rescheduling_interrupts 484.67 ± 11% +98.0% 959.50 ± 8% interrupts.CPU151.TLB:TLB_shootdowns 3982 ± 8% +25.1% 4982 ± 4% interrupts.CPU152.CAL:Function_call_interrupts 5183 ± 2% -32.9% 3475 ± 26% interrupts.CPU152.NMI:Non-maskable_interrupts 5183 ± 2% -32.9% 3475 ± 26% interrupts.CPU152.PMI:Performance_monitoring_interrupts 623.33 ± 5% +79.5% 1119 ± 2% interrupts.CPU152.RES:Rescheduling_interrupts 494.00 ± 14% +89.7% 937.25 ± 9% interrupts.CPU152.TLB:TLB_shootdowns 4109 ± 6% +24.9% 5131 ± 10% interrupts.CPU153.CAL:Function_call_interrupts 5179 ± 4% -31.7% 3536 ± 31% interrupts.CPU153.NMI:Non-maskable_interrupts 5179 ± 4% -31.7% 3536 ± 31% interrupts.CPU153.PMI:Performance_monitoring_interrupts 614.67 ± 3% +94.8% 1197 ± 9% interrupts.CPU153.RES:Rescheduling_interrupts 486.00 ± 19% +83.9% 893.75 ± 13% interrupts.CPU153.TLB:TLB_shootdowns 4161 ± 5% +16.5% 4846 ± 2% interrupts.CPU154.CAL:Function_call_interrupts 635.33 ± 3% +74.6% 1109 ± 3% interrupts.CPU154.RES:Rescheduling_interrupts 490.67 ± 21% +87.4% 919.50 ± 8% interrupts.CPU154.TLB:TLB_shootdowns 3946 ± 8% +28.2% 5057 ± 2% interrupts.CPU155.CAL:Function_call_interrupts 631.33 ± 9% +78.6% 1127 ± 3% interrupts.CPU155.RES:Rescheduling_interrupts 489.33 ± 17% +107.9% 1017 ± 11% interrupts.CPU155.TLB:TLB_shootdowns 4028 ± 5% +21.3% 4887 ± 4% interrupts.CPU156.CAL:Function_call_interrupts 627.33 ± 2% +82.2% 1142 ± 2% interrupts.CPU156.RES:Rescheduling_interrupts 452.33 ± 14% +113.0% 963.25 ± 15% interrupts.CPU156.TLB:TLB_shootdowns 3893 ± 8% +28.2% 4993 ± 4% interrupts.CPU157.CAL:Function_call_interrupts 637.33 +80.4% 1149 ± 4% interrupts.CPU157.RES:Rescheduling_interrupts 451.67 ± 20% +105.6% 928.50 ± 11% interrupts.CPU157.TLB:TLB_shootdowns 4155 ± 9% +19.6% 4971 ± 4% interrupts.CPU158.CAL:Function_call_interrupts 628.33 ± 6% +75.8% 1104 ± 2% interrupts.CPU158.RES:Rescheduling_interrupts 468.33 ± 11% +103.4% 952.50 ± 11% interrupts.CPU158.TLB:TLB_shootdowns 3965 ± 8% +23.8% 4910 ± 5% interrupts.CPU159.CAL:Function_call_interrupts 597.33 ± 4% +95.2% 1166 ± 2% interrupts.CPU159.RES:Rescheduling_interrupts 436.00 ± 12% +121.7% 966.50 ± 11% interrupts.CPU159.TLB:TLB_shootdowns 3811 ± 3% +28.3% 4890 ± 6% interrupts.CPU16.CAL:Function_call_interrupts 5211 ± 3% -24.1% 3953 ± 5% interrupts.CPU16.NMI:Non-maskable_interrupts 5211 ± 3% -24.1% 3953 ± 5% interrupts.CPU16.PMI:Performance_monitoring_interrupts 726.67 ± 6% +55.9% 1132 ± 7% interrupts.CPU16.RES:Rescheduling_interrupts 505.67 ± 13% +75.1% 885.50 ± 12% interrupts.CPU16.TLB:TLB_shootdowns 3965 ± 8% +24.8% 4948 ± 6% interrupts.CPU160.CAL:Function_call_interrupts 636.67 ± 4% +74.8% 1113 ± 2% interrupts.CPU160.RES:Rescheduling_interrupts 516.67 ± 17% +88.3% 973.00 ± 14% interrupts.CPU160.TLB:TLB_shootdowns 4054 ± 8% +24.2% 5036 ± 4% interrupts.CPU161.CAL:Function_call_interrupts 601.00 ± 6% +89.6% 1139 ± 3% interrupts.CPU161.RES:Rescheduling_interrupts 514.33 ± 13% +81.9% 935.50 ± 12% interrupts.CPU161.TLB:TLB_shootdowns 4039 ± 11% +19.7% 4836 ± 3% interrupts.CPU162.CAL:Function_call_interrupts 643.33 ± 7% +67.2% 1075 interrupts.CPU162.RES:Rescheduling_interrupts 469.33 ± 18% +107.1% 972.00 ± 8% interrupts.CPU162.TLB:TLB_shootdowns 4022 ± 7% +20.9% 4864 ± 3% interrupts.CPU163.CAL:Function_call_interrupts 609.33 ± 3% +92.5% 1172 interrupts.CPU163.RES:Rescheduling_interrupts 431.00 ± 18% +146.3% 1061 ± 6% interrupts.CPU163.TLB:TLB_shootdowns 4036 +20.7% 4869 ± 4% interrupts.CPU164.CAL:Function_call_interrupts 613.00 ± 4% +86.4% 1142 ± 6% interrupts.CPU164.RES:Rescheduling_interrupts 492.33 ± 19% +87.9% 925.25 ± 10% interrupts.CPU164.TLB:TLB_shootdowns 3957 ± 3% +23.4% 4883 ± 4% interrupts.CPU165.CAL:Function_call_interrupts 4307 ± 27% -42.1% 2492 ± 41% interrupts.CPU165.NMI:Non-maskable_interrupts 4307 ± 27% -42.1% 2492 ± 41% interrupts.CPU165.PMI:Performance_monitoring_interrupts 583.00 ± 10% +88.9% 1101 ± 3% interrupts.CPU165.RES:Rescheduling_interrupts 449.00 ± 22% +108.7% 937.25 ± 13% interrupts.CPU165.TLB:TLB_shootdowns 4219 ± 5% +16.9% 4932 ± 4% interrupts.CPU166.CAL:Function_call_interrupts 624.33 ± 4% +76.9% 1104 ± 5% interrupts.CPU166.RES:Rescheduling_interrupts 484.33 ± 17% +100.6% 971.50 ± 17% interrupts.CPU166.TLB:TLB_shootdowns 4064 ± 10% +21.0% 4918 ± 3% interrupts.CPU167.CAL:Function_call_interrupts 616.67 ± 6% +83.5% 1131 ± 4% interrupts.CPU167.RES:Rescheduling_interrupts 452.00 ± 12% +95.7% 884.50 ± 8% interrupts.CPU167.TLB:TLB_shootdowns 4087 ± 6% +19.4% 4879 ± 6% interrupts.CPU168.CAL:Function_call_interrupts 649.67 +69.1% 1098 ± 4% interrupts.CPU168.RES:Rescheduling_interrupts 478.33 ± 6% +92.4% 920.50 ± 12% interrupts.CPU168.TLB:TLB_shootdowns 3753 ± 7% +20.6% 4524 ± 4% interrupts.CPU169.CAL:Function_call_interrupts 585.67 +81.9% 1065 ± 2% interrupts.CPU169.RES:Rescheduling_interrupts 459.00 ± 4% +104.2% 937.50 ± 3% interrupts.CPU169.TLB:TLB_shootdowns 3764 ± 5% +28.6% 4841 ± 3% interrupts.CPU17.CAL:Function_call_interrupts 5198 ± 3% -43.0% 2964 ± 35% interrupts.CPU17.NMI:Non-maskable_interrupts 5198 ± 3% -43.0% 2964 ± 35% interrupts.CPU17.PMI:Performance_monitoring_interrupts 729.33 ± 5% +52.1% 1109 ± 7% interrupts.CPU17.RES:Rescheduling_interrupts 450.67 ± 10% +104.4% 921.25 ± 7% interrupts.CPU17.TLB:TLB_shootdowns 4099 ± 8% +24.8% 5118 ± 5% interrupts.CPU170.CAL:Function_call_interrupts 590.67 ± 7% +91.4% 1130 interrupts.CPU170.RES:Rescheduling_interrupts 552.33 ± 17% +75.9% 971.75 ± 10% interrupts.CPU170.TLB:TLB_shootdowns 4023 ± 8% +19.5% 4808 interrupts.CPU171.CAL:Function_call_interrupts 594.67 ± 7% +82.6% 1085 ± 5% interrupts.CPU171.RES:Rescheduling_interrupts 468.33 ± 13% +94.0% 908.75 ± 7% interrupts.CPU171.TLB:TLB_shootdowns 3917 ± 9% +25.0% 4895 ± 5% interrupts.CPU172.CAL:Function_call_interrupts 5248 ± 3% -25.4% 3915 ± 10% interrupts.CPU172.NMI:Non-maskable_interrupts 5248 ± 3% -25.4% 3915 ± 10% interrupts.CPU172.PMI:Performance_monitoring_interrupts 543.00 ± 3% +103.6% 1105 ± 5% interrupts.CPU172.RES:Rescheduling_interrupts 468.67 ± 15% +94.0% 909.25 ± 17% interrupts.CPU172.TLB:TLB_shootdowns 4057 ± 6% +22.6% 4975 ± 2% interrupts.CPU173.CAL:Function_call_interrupts 5158 ± 5% -31.7% 3521 ± 31% interrupts.CPU173.NMI:Non-maskable_interrupts 5158 ± 5% -31.7% 3521 ± 31% interrupts.CPU173.PMI:Performance_monitoring_interrupts 597.67 ± 5% +87.9% 1123 ± 3% interrupts.CPU173.RES:Rescheduling_interrupts 498.67 ± 19% +133.2% 1163 ± 26% interrupts.CPU173.TLB:TLB_shootdowns 3833 ± 8% +26.8% 4859 ± 3% interrupts.CPU174.CAL:Function_call_interrupts 5208 ± 4% -24.7% 3920 ± 11% interrupts.CPU174.NMI:Non-maskable_interrupts 5208 ± 4% -24.7% 3920 ± 11% interrupts.CPU174.PMI:Performance_monitoring_interrupts 570.67 ± 2% +91.1% 1090 ± 7% interrupts.CPU174.RES:Rescheduling_interrupts 470.00 ± 19% +108.3% 979.00 ± 12% interrupts.CPU174.TLB:TLB_shootdowns 4117 ± 7% +17.7% 4845 ± 3% interrupts.CPU175.CAL:Function_call_interrupts 5202 ± 4% -25.1% 3896 ± 8% interrupts.CPU175.NMI:Non-maskable_interrupts 5202 ± 4% -25.1% 3896 ± 8% interrupts.CPU175.PMI:Performance_monitoring_interrupts 638.33 ± 3% +70.8% 1090 ± 5% interrupts.CPU175.RES:Rescheduling_interrupts 463.00 ± 17% +104.4% 946.50 ± 8% interrupts.CPU175.TLB:TLB_shootdowns 4021 ± 4% +16.9% 4699 ± 2% interrupts.CPU176.CAL:Function_call_interrupts 608.67 ± 6% +79.4% 1091 ± 6% interrupts.CPU176.RES:Rescheduling_interrupts 477.33 ± 10% +97.2% 941.50 ± 7% interrupts.CPU176.TLB:TLB_shootdowns 4071 ± 7% +15.5% 4702 interrupts.CPU177.CAL:Function_call_interrupts 596.33 ± 4% +82.2% 1086 interrupts.CPU177.RES:Rescheduling_interrupts 435.00 ± 5% +114.7% 934.00 ± 14% interrupts.CPU177.TLB:TLB_shootdowns 4130 ± 7% +16.6% 4815 ± 4% interrupts.CPU178.CAL:Function_call_interrupts 633.33 ± 4% +64.1% 1039 ± 4% interrupts.CPU178.RES:Rescheduling_interrupts 478.67 ± 15% +95.2% 934.50 ± 19% interrupts.CPU178.TLB:TLB_shootdowns 3821 ± 4% +26.8% 4846 ± 3% interrupts.CPU179.CAL:Function_call_interrupts 594.67 +76.6% 1050 ± 4% interrupts.CPU179.RES:Rescheduling_interrupts 479.00 ± 7% +96.5% 941.00 ± 6% interrupts.CPU179.TLB:TLB_shootdowns 4196 ± 4% +16.5% 4890 ± 5% interrupts.CPU18.CAL:Function_call_interrupts 728.33 +51.1% 1100 ± 3% interrupts.CPU18.RES:Rescheduling_interrupts 506.00 ± 11% +71.7% 868.75 ± 6% interrupts.CPU18.TLB:TLB_shootdowns 3950 ± 6% +23.3% 4870 ± 2% interrupts.CPU180.CAL:Function_call_interrupts 606.33 ± 8% +72.5% 1045 ± 2% interrupts.CPU180.RES:Rescheduling_interrupts 498.00 ± 5% +72.3% 858.00 ± 17% interrupts.CPU180.TLB:TLB_shootdowns 4113 ± 11% +18.3% 4864 ± 5% interrupts.CPU181.CAL:Function_call_interrupts 620.33 ± 4% +73.7% 1077 ± 5% interrupts.CPU181.RES:Rescheduling_interrupts 483.33 ± 5% +90.3% 919.75 ± 11% interrupts.CPU181.TLB:TLB_shootdowns 4070 ± 8% +17.0% 4761 ± 2% interrupts.CPU182.CAL:Function_call_interrupts 588.00 ± 7% +82.8% 1075 ± 3% interrupts.CPU182.RES:Rescheduling_interrupts 480.00 ± 17% +124.7% 1078 ± 10% interrupts.CPU182.TLB:TLB_shootdowns 4198 ± 7% +15.9% 4868 ± 2% interrupts.CPU183.CAL:Function_call_interrupts 681.33 ± 6% +59.5% 1087 ± 6% interrupts.CPU183.RES:Rescheduling_interrupts 509.33 ± 16% +78.5% 909.00 ± 15% interrupts.CPU183.TLB:TLB_shootdowns 3955 ± 2% +25.7% 4970 ± 3% interrupts.CPU184.CAL:Function_call_interrupts 565.00 ± 6% +93.8% 1095 ± 6% interrupts.CPU184.RES:Rescheduling_interrupts 443.67 ± 14% +107.0% 918.25 ± 9% interrupts.CPU184.TLB:TLB_shootdowns 4105 ± 6% +16.5% 4781 ± 5% interrupts.CPU185.CAL:Function_call_interrupts 546.67 ± 5% +105.8% 1125 ± 7% interrupts.CPU185.RES:Rescheduling_interrupts 505.00 ± 18% +86.8% 943.50 ± 8% interrupts.CPU185.TLB:TLB_shootdowns 4067 ± 6% +19.9% 4878 ± 3% interrupts.CPU186.CAL:Function_call_interrupts 604.67 ± 6% +71.2% 1035 ± 7% interrupts.CPU186.RES:Rescheduling_interrupts 486.67 ± 12% +89.7% 923.25 ± 8% interrupts.CPU186.TLB:TLB_shootdowns 588.00 ± 6% +77.9% 1046 ± 5% interrupts.CPU187.RES:Rescheduling_interrupts 499.67 ± 15% +86.2% 930.50 ± 15% interrupts.CPU187.TLB:TLB_shootdowns 4186 ± 5% +17.1% 4901 ± 5% interrupts.CPU188.CAL:Function_call_interrupts 616.33 ± 2% +77.1% 1091 ± 4% interrupts.CPU188.RES:Rescheduling_interrupts 461.33 ± 9% +102.1% 932.25 ± 15% interrupts.CPU188.TLB:TLB_shootdowns 3980 ± 6% +21.8% 4849 ± 2% interrupts.CPU189.CAL:Function_call_interrupts 594.33 +87.8% 1116 ± 3% interrupts.CPU189.RES:Rescheduling_interrupts 431.33 ± 10% +126.0% 974.75 ± 5% interrupts.CPU189.TLB:TLB_shootdowns 3937 ± 5% +21.2% 4772 ± 5% interrupts.CPU19.CAL:Function_call_interrupts 5173 ± 3% -31.9% 3525 ± 25% interrupts.CPU19.NMI:Non-maskable_interrupts 5173 ± 3% -31.9% 3525 ± 25% interrupts.CPU19.PMI:Performance_monitoring_interrupts 722.67 ± 9% +48.1% 1070 ± 2% interrupts.CPU19.RES:Rescheduling_interrupts 479.67 ± 18% +95.3% 937.00 ± 9% interrupts.CPU19.TLB:TLB_shootdowns 3905 ± 5% +25.2% 4890 ± 5% interrupts.CPU190.CAL:Function_call_interrupts 5230 ± 2% -24.7% 3937 ± 10% interrupts.CPU190.NMI:Non-maskable_interrupts 5230 ± 2% -24.7% 3937 ± 10% interrupts.CPU190.PMI:Performance_monitoring_interrupts 580.33 ± 3% +84.6% 1071 ± 2% interrupts.CPU190.RES:Rescheduling_interrupts 454.33 ± 12% +111.7% 961.75 ± 14% interrupts.CPU190.TLB:TLB_shootdowns 3985 ± 4% +23.4% 4918 ± 4% interrupts.CPU191.CAL:Function_call_interrupts 5184 ± 4% -34.0% 3422 ± 24% interrupts.CPU191.NMI:Non-maskable_interrupts 5184 ± 4% -34.0% 3422 ± 24% interrupts.CPU191.PMI:Performance_monitoring_interrupts 589.67 ± 2% +91.5% 1129 ± 7% interrupts.CPU191.RES:Rescheduling_interrupts 498.67 ± 25% +102.2% 1008 ± 14% interrupts.CPU191.TLB:TLB_shootdowns 3847 ± 6% +29.1% 4965 ± 8% interrupts.CPU2.CAL:Function_call_interrupts 5162 ± 3% -23.5% 3948 ± 5% interrupts.CPU2.NMI:Non-maskable_interrupts 5162 ± 3% -23.5% 3948 ± 5% interrupts.CPU2.PMI:Performance_monitoring_interrupts 691.33 ± 3% +59.0% 1099 ± 7% interrupts.CPU2.RES:Rescheduling_interrupts 486.67 ± 6% +83.7% 894.25 ± 13% interrupts.CPU2.TLB:TLB_shootdowns 4000 ± 3% +19.6% 4786 ± 5% interrupts.CPU20.CAL:Function_call_interrupts 5181 ± 2% -42.1% 2999 ± 32% interrupts.CPU20.NMI:Non-maskable_interrupts 5181 ± 2% -42.1% 2999 ± 32% interrupts.CPU20.PMI:Performance_monitoring_interrupts 774.00 ± 11% +45.8% 1128 ± 5% interrupts.CPU20.RES:Rescheduling_interrupts 458.33 ± 11% +111.7% 970.25 ± 12% interrupts.CPU20.TLB:TLB_shootdowns 4143 ± 4% +13.1% 4684 ± 3% interrupts.CPU21.CAL:Function_call_interrupts 814.67 ± 19% +35.1% 1100 ± 6% interrupts.CPU21.RES:Rescheduling_interrupts 453.67 ± 12% +97.1% 894.25 ± 13% interrupts.CPU21.TLB:TLB_shootdowns 3936 +26.2% 4968 ± 4% interrupts.CPU22.CAL:Function_call_interrupts 5194 ± 2% -22.7% 4017 ± 5% interrupts.CPU22.NMI:Non-maskable_interrupts 5194 ± 2% -22.7% 4017 ± 5% interrupts.CPU22.PMI:Performance_monitoring_interrupts 754.67 ± 11% +45.8% 1100 ± 7% interrupts.CPU22.RES:Rescheduling_interrupts 467.67 ± 14% +100.0% 935.25 ± 17% interrupts.CPU22.TLB:TLB_shootdowns 4046 ± 4% +17.6% 4758 ± 2% interrupts.CPU23.CAL:Function_call_interrupts 716.67 ± 2% +45.6% 1043 ± 5% interrupts.CPU23.RES:Rescheduling_interrupts 416.33 ± 7% +108.4% 867.50 ± 12% interrupts.CPU23.TLB:TLB_shootdowns 4323 ± 6% +17.2% 5066 ± 5% interrupts.CPU24.CAL:Function_call_interrupts 5170 ± 3% -22.1% 4027 ± 6% interrupts.CPU24.NMI:Non-maskable_interrupts 5170 ± 3% -22.1% 4027 ± 6% interrupts.CPU24.PMI:Performance_monitoring_interrupts 876.33 ± 4% +33.1% 1166 ± 6% interrupts.CPU24.RES:Rescheduling_interrupts 678.33 ± 16% +50.4% 1020 ± 9% interrupts.CPU24.TLB:TLB_shootdowns 4259 ± 7% +18.1% 5027 ± 2% interrupts.CPU25.CAL:Function_call_interrupts 5240 ± 3% -23.8% 3993 ± 8% interrupts.CPU25.NMI:Non-maskable_interrupts 5240 ± 3% -23.8% 3993 ± 8% interrupts.CPU25.PMI:Performance_monitoring_interrupts 791.67 ± 3% +44.3% 1142 ± 7% interrupts.CPU25.RES:Rescheduling_interrupts 594.00 ± 22% +62.4% 964.75 ± 15% interrupts.CPU25.TLB:TLB_shootdowns 4234 ± 13% +20.7% 5111 ± 3% interrupts.CPU26.CAL:Function_call_interrupts 821.33 ± 3% +41.4% 1161 ± 6% interrupts.CPU26.RES:Rescheduling_interrupts 528.00 ± 5% +96.6% 1038 ± 18% interrupts.CPU26.TLB:TLB_shootdowns 4202 ± 4% +21.4% 5101 ± 4% interrupts.CPU27.CAL:Function_call_interrupts 5218 ± 2% -22.5% 4045 ± 8% interrupts.CPU27.NMI:Non-maskable_interrupts 5218 ± 2% -22.5% 4045 ± 8% interrupts.CPU27.PMI:Performance_monitoring_interrupts 720.00 ± 5% +58.6% 1142 ± 5% interrupts.CPU27.RES:Rescheduling_interrupts 502.67 ± 16% +89.7% 953.50 ± 12% interrupts.CPU27.TLB:TLB_shootdowns 4162 ± 14% +18.2% 4921 ± 5% interrupts.CPU28.CAL:Function_call_interrupts 5157 ± 2% -23.4% 3951 ± 7% interrupts.CPU28.NMI:Non-maskable_interrupts 5157 ± 2% -23.4% 3951 ± 7% interrupts.CPU28.PMI:Performance_monitoring_interrupts 755.33 ± 3% +47.4% 1113 ± 5% interrupts.CPU28.RES:Rescheduling_interrupts 497.00 ± 10% +91.9% 953.50 ± 3% interrupts.CPU28.TLB:TLB_shootdowns 4088 ± 5% +24.1% 5072 ± 6% interrupts.CPU29.CAL:Function_call_interrupts 5188 ± 3% -22.5% 4021 ± 6% interrupts.CPU29.NMI:Non-maskable_interrupts 5188 ± 3% -22.5% 4021 ± 6% interrupts.CPU29.PMI:Performance_monitoring_interrupts 719.33 ± 6% +57.0% 1129 ± 6% interrupts.CPU29.RES:Rescheduling_interrupts 464.33 ± 2% +131.1% 1073 ± 26% interrupts.CPU29.TLB:TLB_shootdowns 3770 +25.7% 4739 ± 5% interrupts.CPU3.CAL:Function_call_interrupts 746.33 ± 6% +50.2% 1120 ± 11% interrupts.CPU3.RES:Rescheduling_interrupts 511.67 ± 14% +74.5% 892.75 ± 6% interrupts.CPU3.TLB:TLB_shootdowns 4203 ± 7% +18.9% 4998 ± 3% interrupts.CPU30.CAL:Function_call_interrupts 5183 ± 3% -41.1% 3055 ± 37% interrupts.CPU30.NMI:Non-maskable_interrupts 5183 ± 3% -41.1% 3055 ± 37% interrupts.CPU30.PMI:Performance_monitoring_interrupts 749.33 ± 5% +52.9% 1146 ± 6% interrupts.CPU30.RES:Rescheduling_interrupts 469.33 ± 17% +108.5% 978.75 ± 8% interrupts.CPU30.TLB:TLB_shootdowns 4133 ± 5% +20.3% 4971 ± 2% interrupts.CPU31.CAL:Function_call_interrupts 5174 ± 3% -31.7% 3536 ± 26% interrupts.CPU31.NMI:Non-maskable_interrupts 5174 ± 3% -31.7% 3536 ± 26% interrupts.CPU31.PMI:Performance_monitoring_interrupts 777.33 ± 5% +53.8% 1195 ± 5% interrupts.CPU31.RES:Rescheduling_interrupts 496.00 ± 9% +96.7% 975.75 ± 16% interrupts.CPU31.TLB:TLB_shootdowns 4107 ± 7% +18.1% 4849 ± 2% interrupts.CPU32.CAL:Function_call_interrupts 5163 ± 3% -21.8% 4038 ± 7% interrupts.CPU32.NMI:Non-maskable_interrupts 5163 ± 3% -21.8% 4038 ± 7% interrupts.CPU32.PMI:Performance_monitoring_interrupts 711.67 +52.9% 1088 ± 5% interrupts.CPU32.RES:Rescheduling_interrupts 489.33 ± 18% +115.7% 1055 ± 11% interrupts.CPU32.TLB:TLB_shootdowns 4170 ± 4% +17.4% 4897 ± 2% interrupts.CPU33.CAL:Function_call_interrupts 5215 ± 3% -33.3% 3480 ± 25% interrupts.CPU33.NMI:Non-maskable_interrupts 5215 ± 3% -33.3% 3480 ± 25% interrupts.CPU33.PMI:Performance_monitoring_interrupts 785.33 ± 7% +46.2% 1148 ± 4% interrupts.CPU33.RES:Rescheduling_interrupts 547.67 ± 13% +71.6% 939.75 ± 11% interrupts.CPU33.TLB:TLB_shootdowns 4302 ± 6% +16.0% 4992 ± 4% interrupts.CPU34.CAL:Function_call_interrupts 5214 ± 3% -22.7% 4031 ± 6% interrupts.CPU34.NMI:Non-maskable_interrupts 5214 ± 3% -22.7% 4031 ± 6% interrupts.CPU34.PMI:Performance_monitoring_interrupts 787.33 ± 4% +41.4% 1113 ± 4% interrupts.CPU34.RES:Rescheduling_interrupts 590.33 ± 2% +60.2% 945.50 ± 11% interrupts.CPU34.TLB:TLB_shootdowns 4044 ± 11% +22.9% 4969 ± 4% interrupts.CPU35.CAL:Function_call_interrupts 5220 ± 3% -32.6% 3520 ± 25% interrupts.CPU35.NMI:Non-maskable_interrupts 5220 ± 3% -32.6% 3520 ± 25% interrupts.CPU35.PMI:Performance_monitoring_interrupts 735.67 ± 4% +56.9% 1154 ± 9% interrupts.CPU35.RES:Rescheduling_interrupts 485.00 ± 12% +101.3% 976.50 ± 11% interrupts.CPU35.TLB:TLB_shootdowns 4291 ± 8% +14.9% 4930 ± 5% interrupts.CPU36.CAL:Function_call_interrupts 782.67 ± 4% +51.0% 1182 ± 4% interrupts.CPU36.RES:Rescheduling_interrupts 491.00 ± 16% +84.3% 904.75 ± 16% interrupts.CPU36.TLB:TLB_shootdowns 3949 ± 5% +24.1% 4898 ± 4% interrupts.CPU37.CAL:Function_call_interrupts 723.67 ± 3% +54.0% 1114 ± 7% interrupts.CPU37.RES:Rescheduling_interrupts 490.00 ± 9% +111.4% 1036 ± 8% interrupts.CPU37.TLB:TLB_shootdowns 5218 ± 2% -23.3% 4000 ± 7% interrupts.CPU38.NMI:Non-maskable_interrupts 5218 ± 2% -23.3% 4000 ± 7% interrupts.CPU38.PMI:Performance_monitoring_interrupts 797.67 ± 10% +40.3% 1119 ± 6% interrupts.CPU38.RES:Rescheduling_interrupts 500.33 ± 16% +89.6% 948.50 ± 12% interrupts.CPU38.TLB:TLB_shootdowns 3987 ± 5% +26.0% 5023 ± 5% interrupts.CPU39.CAL:Function_call_interrupts 772.00 +47.0% 1134 ± 6% interrupts.CPU39.RES:Rescheduling_interrupts 483.00 ± 10% +102.2% 976.75 ± 17% interrupts.CPU39.TLB:TLB_shootdowns 3933 ± 6% +38.0% 5428 ± 14% interrupts.CPU4.CAL:Function_call_interrupts 5183 ± 4% -43.3% 2939 ± 31% interrupts.CPU4.NMI:Non-maskable_interrupts 5183 ± 4% -43.3% 2939 ± 31% interrupts.CPU4.PMI:Performance_monitoring_interrupts 799.33 ± 5% +35.5% 1083 ± 4% interrupts.CPU4.RES:Rescheduling_interrupts 489.67 ± 14% +100.9% 983.50 ± 11% interrupts.CPU4.TLB:TLB_shootdowns 4034 ± 6% +23.7% 4991 interrupts.CPU40.CAL:Function_call_interrupts 5212 ± 2% -24.0% 3963 ± 7% interrupts.CPU40.NMI:Non-maskable_interrupts 5212 ± 2% -24.0% 3963 ± 7% interrupts.CPU40.PMI:Performance_monitoring_interrupts 745.67 ± 5% +53.6% 1145 ± 6% interrupts.CPU40.RES:Rescheduling_interrupts 407.67 ± 11% +140.6% 981.00 ± 13% interrupts.CPU40.TLB:TLB_shootdowns 4029 ± 3% +25.1% 5040 ± 4% interrupts.CPU41.CAL:Function_call_interrupts 5237 ± 2% -24.0% 3980 ± 7% interrupts.CPU41.NMI:Non-maskable_interrupts 5237 ± 2% -24.0% 3980 ± 7% interrupts.CPU41.PMI:Performance_monitoring_interrupts 764.00 ± 11% +49.6% 1143 ± 10% interrupts.CPU41.RES:Rescheduling_interrupts 480.33 ± 15% +102.8% 974.00 ± 14% interrupts.CPU41.TLB:TLB_shootdowns 4158 ± 9% +19.1% 4950 ± 5% interrupts.CPU42.CAL:Function_call_interrupts 5205 ± 2% -23.9% 3963 ± 7% interrupts.CPU42.NMI:Non-maskable_interrupts 5205 ± 2% -23.9% 3963 ± 7% interrupts.CPU42.PMI:Performance_monitoring_interrupts 777.00 +48.0% 1150 ± 7% interrupts.CPU42.RES:Rescheduling_interrupts 494.67 ± 7% +85.4% 917.25 ± 6% interrupts.CPU42.TLB:TLB_shootdowns 4145 ± 5% +20.0% 4974 ± 2% interrupts.CPU43.CAL:Function_call_interrupts 5199 ± 3% -23.6% 3971 ± 6% interrupts.CPU43.NMI:Non-maskable_interrupts 5199 ± 3% -23.6% 3971 ± 6% interrupts.CPU43.PMI:Performance_monitoring_interrupts 734.00 ± 4% +54.8% 1136 ± 3% interrupts.CPU43.RES:Rescheduling_interrupts 479.33 ± 21% +116.7% 1038 ± 12% interrupts.CPU43.TLB:TLB_shootdowns 4038 ± 6% +20.5% 4867 ± 2% interrupts.CPU44.CAL:Function_call_interrupts 5236 ± 3% -34.3% 3438 ± 24% interrupts.CPU44.NMI:Non-maskable_interrupts 5236 ± 3% -34.3% 3438 ± 24% interrupts.CPU44.PMI:Performance_monitoring_interrupts 721.67 +57.3% 1135 ± 6% interrupts.CPU44.RES:Rescheduling_interrupts 486.67 ± 12% +94.9% 948.50 ± 11% interrupts.CPU44.TLB:TLB_shootdowns 4089 ± 7% +19.7% 4895 ± 4% interrupts.CPU45.CAL:Function_call_interrupts 744.67 ± 4% +45.4% 1083 ± 2% interrupts.CPU45.RES:Rescheduling_interrupts 450.33 ± 17% +114.5% 966.00 ± 19% interrupts.CPU45.TLB:TLB_shootdowns 3816 ± 5% +30.8% 4991 ± 5% interrupts.CPU46.CAL:Function_call_interrupts 5219 ± 3% -32.9% 3502 ± 23% interrupts.CPU46.NMI:Non-maskable_interrupts 5219 ± 3% -32.9% 3502 ± 23% interrupts.CPU46.PMI:Performance_monitoring_interrupts 740.33 ± 8% +61.8% 1197 ± 5% interrupts.CPU46.RES:Rescheduling_interrupts 455.00 ± 6% +107.6% 944.75 ± 15% interrupts.CPU46.TLB:TLB_shootdowns 3989 ± 2% +24.0% 4947 ± 2% interrupts.CPU47.CAL:Function_call_interrupts 5204 ± 3% -32.6% 3506 ± 24% interrupts.CPU47.NMI:Non-maskable_interrupts 5204 ± 3% -32.6% 3506 ± 24% interrupts.CPU47.PMI:Performance_monitoring_interrupts 741.67 ± 4% +53.1% 1135 ± 12% interrupts.CPU47.RES:Rescheduling_interrupts 437.33 ± 17% +120.8% 965.75 ± 16% interrupts.CPU47.TLB:TLB_shootdowns 4165 ± 2% +22.3% 5095 ± 3% interrupts.CPU48.CAL:Function_call_interrupts 5176 ± 4% -21.7% 4053 ± 8% interrupts.CPU48.NMI:Non-maskable_interrupts 5176 ± 4% -21.7% 4053 ± 8% interrupts.CPU48.PMI:Performance_monitoring_interrupts 835.00 ± 2% +29.0% 1077 ± 2% interrupts.CPU48.RES:Rescheduling_interrupts 594.67 ± 17% +62.2% 964.50 ± 5% interrupts.CPU48.TLB:TLB_shootdowns 4171 ± 9% +18.7% 4950 ± 6% interrupts.CPU49.CAL:Function_call_interrupts 5186 ± 4% -33.4% 3452 ± 24% interrupts.CPU49.NMI:Non-maskable_interrupts 5186 ± 4% -33.4% 3452 ± 24% interrupts.CPU49.PMI:Performance_monitoring_interrupts 710.67 ± 4% +55.6% 1105 ± 4% interrupts.CPU49.RES:Rescheduling_interrupts 553.67 ± 5% +74.0% 963.25 ± 10% interrupts.CPU49.TLB:TLB_shootdowns 4064 ± 6% +21.6% 4944 ± 5% interrupts.CPU5.CAL:Function_call_interrupts 772.33 ± 2% +45.7% 1125 ± 4% interrupts.CPU5.RES:Rescheduling_interrupts 491.00 ± 14% +95.2% 958.50 ± 10% interrupts.CPU5.TLB:TLB_shootdowns 3929 ± 4% +26.4% 4966 ± 2% interrupts.CPU50.CAL:Function_call_interrupts 5201 ± 4% -22.9% 4011 ± 7% interrupts.CPU50.NMI:Non-maskable_interrupts 5201 ± 4% -22.9% 4011 ± 7% interrupts.CPU50.PMI:Performance_monitoring_interrupts 747.00 ± 4% +40.6% 1050 ± 3% interrupts.CPU50.RES:Rescheduling_interrupts 482.33 ± 17% +103.1% 979.75 ± 12% interrupts.CPU50.TLB:TLB_shootdowns 4192 ± 4% +20.3% 5043 ± 4% interrupts.CPU51.CAL:Function_call_interrupts 5249 ± 3% -24.0% 3987 ± 7% interrupts.CPU51.NMI:Non-maskable_interrupts 5249 ± 3% -24.0% 3987 ± 7% interrupts.CPU51.PMI:Performance_monitoring_interrupts 780.33 ± 7% +54.9% 1209 ± 9% interrupts.CPU51.RES:Rescheduling_interrupts 504.00 ± 11% +95.9% 987.50 ± 9% interrupts.CPU51.TLB:TLB_shootdowns 4084 ± 4% +21.8% 4972 ± 4% interrupts.CPU52.CAL:Function_call_interrupts 5208 ± 4% -23.1% 4006 ± 11% interrupts.CPU52.NMI:Non-maskable_interrupts 5208 ± 4% -23.1% 4006 ± 11% interrupts.CPU52.PMI:Performance_monitoring_interrupts 724.33 ± 3% +48.6% 1076 ± 4% interrupts.CPU52.RES:Rescheduling_interrupts 536.00 ± 13% +91.7% 1027 ± 7% interrupts.CPU52.TLB:TLB_shootdowns 5184 ± 3% -24.1% 3933 ± 10% interrupts.CPU53.NMI:Non-maskable_interrupts 5184 ± 3% -24.1% 3933 ± 10% interrupts.CPU53.PMI:Performance_monitoring_interrupts 722.67 ± 4% +49.1% 1077 ± 5% interrupts.CPU53.RES:Rescheduling_interrupts 447.33 ± 20% +118.6% 978.00 ± 11% interrupts.CPU53.TLB:TLB_shootdowns 4057 ± 3% +25.0% 5073 ± 5% interrupts.CPU54.CAL:Function_call_interrupts 5172 ± 4% -32.6% 3487 ± 24% interrupts.CPU54.NMI:Non-maskable_interrupts 5172 ± 4% -32.6% 3487 ± 24% interrupts.CPU54.PMI:Performance_monitoring_interrupts 722.67 +50.7% 1088 ± 7% interrupts.CPU54.RES:Rescheduling_interrupts 491.33 ± 14% +382.4% 2370 ±102% interrupts.CPU54.TLB:TLB_shootdowns 4092 ± 5% +17.9% 4825 interrupts.CPU55.CAL:Function_call_interrupts 755.67 ± 10% +49.2% 1127 ± 4% interrupts.CPU55.RES:Rescheduling_interrupts 445.67 +117.2% 968.00 ± 11% interrupts.CPU55.TLB:TLB_shootdowns 4116 ± 4% +20.0% 4940 ± 5% interrupts.CPU56.CAL:Function_call_interrupts 5144 ± 5% -33.4% 3427 ± 23% interrupts.CPU56.NMI:Non-maskable_interrupts 5144 ± 5% -33.4% 3427 ± 23% interrupts.CPU56.PMI:Performance_monitoring_interrupts 707.67 +55.5% 1100 interrupts.CPU56.RES:Rescheduling_interrupts 448.00 ± 16% +120.1% 986.00 ± 13% interrupts.CPU56.TLB:TLB_shootdowns 3861 +26.2% 4871 ± 3% interrupts.CPU57.CAL:Function_call_interrupts 5187 ± 3% -33.0% 3474 ± 25% interrupts.CPU57.NMI:Non-maskable_interrupts 5187 ± 3% -33.0% 3474 ± 25% interrupts.CPU57.PMI:Performance_monitoring_interrupts 703.00 ± 3% +55.6% 1094 ± 4% interrupts.CPU57.RES:Rescheduling_interrupts 495.67 ± 19% +106.5% 1023 ± 8% interrupts.CPU57.TLB:TLB_shootdowns 4003 ± 6% +25.3% 5016 ± 4% interrupts.CPU58.CAL:Function_call_interrupts 5267 -32.7% 3543 ± 30% interrupts.CPU58.NMI:Non-maskable_interrupts 5267 -32.7% 3543 ± 30% interrupts.CPU58.PMI:Performance_monitoring_interrupts 692.00 +61.0% 1114 interrupts.CPU58.RES:Rescheduling_interrupts 467.67 ± 16% +109.8% 981.25 ± 9% interrupts.CPU58.TLB:TLB_shootdowns 4147 ± 7% +21.2% 5024 ± 4% interrupts.CPU59.CAL:Function_call_interrupts 5175 ± 3% -33.9% 3419 ± 24% interrupts.CPU59.NMI:Non-maskable_interrupts 5175 ± 3% -33.9% 3419 ± 24% interrupts.CPU59.PMI:Performance_monitoring_interrupts 765.00 ± 2% +40.8% 1077 ± 5% interrupts.CPU59.RES:Rescheduling_interrupts 477.67 ± 23% +97.7% 944.25 ± 9% interrupts.CPU59.TLB:TLB_shootdowns 4121 ± 7% +19.7% 4932 ± 4% interrupts.CPU6.CAL:Function_call_interrupts 5158 ± 4% -31.8% 3518 ± 25% interrupts.CPU6.NMI:Non-maskable_interrupts 5158 ± 4% -31.8% 3518 ± 25% interrupts.CPU6.PMI:Performance_monitoring_interrupts 735.33 ± 10% +48.5% 1091 ± 5% interrupts.CPU6.RES:Rescheduling_interrupts 535.33 ± 13% +72.5% 923.25 ± 14% interrupts.CPU6.TLB:TLB_shootdowns 4122 ± 4% +32.7% 5471 ± 18% interrupts.CPU60.CAL:Function_call_interrupts 746.67 ± 4% +48.7% 1110 ± 4% interrupts.CPU60.RES:Rescheduling_interrupts 496.00 ± 23% +92.2% 953.25 ± 13% interrupts.CPU60.TLB:TLB_shootdowns 4022 ± 4% +33.7% 5376 ± 14% interrupts.CPU61.CAL:Function_call_interrupts 5190 ± 4% -34.9% 3378 ± 24% interrupts.CPU61.NMI:Non-maskable_interrupts 5190 ± 4% -34.9% 3378 ± 24% interrupts.CPU61.PMI:Performance_monitoring_interrupts 734.33 ± 4% +48.2% 1088 ± 2% interrupts.CPU61.RES:Rescheduling_interrupts 484.67 ± 15% +109.0% 1013 ± 9% interrupts.CPU61.TLB:TLB_shootdowns 3917 ± 5% +26.4% 4953 ± 3% interrupts.CPU62.CAL:Function_call_interrupts 5189 ± 4% -32.8% 3489 ± 25% interrupts.CPU62.NMI:Non-maskable_interrupts 5189 ± 4% -32.8% 3489 ± 25% interrupts.CPU62.PMI:Performance_monitoring_interrupts 778.00 ± 3% +42.6% 1109 ± 2% interrupts.CPU62.RES:Rescheduling_interrupts 486.33 ± 23% +110.0% 1021 ± 17% interrupts.CPU62.TLB:TLB_shootdowns 4163 ± 8% +20.8% 5029 interrupts.CPU63.CAL:Function_call_interrupts 5231 ± 2% -34.6% 3419 ± 24% interrupts.CPU63.NMI:Non-maskable_interrupts 5231 ± 2% -34.6% 3419 ± 24% interrupts.CPU63.PMI:Performance_monitoring_interrupts 775.67 ± 6% +39.5% 1082 interrupts.CPU63.RES:Rescheduling_interrupts 491.00 ± 22% +95.8% 961.25 ± 16% interrupts.CPU63.TLB:TLB_shootdowns 4016 ± 2% +24.2% 4990 ± 3% interrupts.CPU64.CAL:Function_call_interrupts 5129 ± 5% -42.0% 2973 ± 36% interrupts.CPU64.NMI:Non-maskable_interrupts 5129 ± 5% -42.0% 2973 ± 36% interrupts.CPU64.PMI:Performance_monitoring_interrupts 757.67 ± 4% +47.7% 1119 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts 453.67 ± 11% +119.8% 997.00 ± 11% interrupts.CPU64.TLB:TLB_shootdowns 3854 ± 2% +26.3% 4868 ± 7% interrupts.CPU65.CAL:Function_call_interrupts 767.33 ± 7% +37.7% 1056 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts 519.67 ± 14% +85.3% 962.75 ± 13% interrupts.CPU65.TLB:TLB_shootdowns 3915 ± 8% +24.8% 4888 ± 3% interrupts.CPU66.CAL:Function_call_interrupts 5199 ± 3% -34.4% 3409 ± 25% interrupts.CPU66.NMI:Non-maskable_interrupts 5199 ± 3% -34.4% 3409 ± 25% interrupts.CPU66.PMI:Performance_monitoring_interrupts 712.67 ± 3% +56.9% 1118 ± 6% interrupts.CPU66.RES:Rescheduling_interrupts 431.67 ± 30% +138.8% 1030 ± 14% interrupts.CPU66.TLB:TLB_shootdowns 4026 ± 9% +21.0% 4870 ± 3% interrupts.CPU67.CAL:Function_call_interrupts 5157 ± 5% -23.4% 3950 ± 9% interrupts.CPU67.NMI:Non-maskable_interrupts 5157 ± 5% -23.4% 3950 ± 9% interrupts.CPU67.PMI:Performance_monitoring_interrupts 671.00 +56.9% 1053 ± 2% interrupts.CPU67.RES:Rescheduling_interrupts 489.67 ± 16% +99.6% 977.50 ± 12% interrupts.CPU67.TLB:TLB_shootdowns 4034 ± 4% +22.0% 4921 ± 2% interrupts.CPU68.CAL:Function_call_interrupts 5175 ± 4% -31.2% 3561 ± 30% interrupts.CPU68.NMI:Non-maskable_interrupts 5175 ± 4% -31.2% 3561 ± 30% interrupts.CPU68.PMI:Performance_monitoring_interrupts 709.00 ± 4% +56.2% 1107 ± 3% interrupts.CPU68.RES:Rescheduling_interrupts 493.00 ± 15% +95.8% 965.25 ± 12% interrupts.CPU68.TLB:TLB_shootdowns 4014 ± 10% +24.6% 5001 ± 2% interrupts.CPU69.CAL:Function_call_interrupts 5152 ± 4% -32.5% 3476 ± 25% interrupts.CPU69.NMI:Non-maskable_interrupts 5152 ± 4% -32.5% 3476 ± 25% interrupts.CPU69.PMI:Performance_monitoring_interrupts 717.33 ± 3% +50.3% 1078 ± 7% interrupts.CPU69.RES:Rescheduling_interrupts 434.67 ± 26% +123.0% 969.25 ± 8% interrupts.CPU69.TLB:TLB_shootdowns 3972 ± 8% +23.1% 4892 interrupts.CPU7.CAL:Function_call_interrupts 776.67 ± 5% +40.6% 1091 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts 467.33 ± 11% +102.8% 947.75 ± 16% interrupts.CPU7.TLB:TLB_shootdowns 4068 ± 11% +26.2% 5134 interrupts.CPU70.CAL:Function_call_interrupts 5188 ± 4% -42.6% 2975 ± 37% interrupts.CPU70.NMI:Non-maskable_interrupts 5188 ± 4% -42.6% 2975 ± 37% interrupts.CPU70.PMI:Performance_monitoring_interrupts 718.00 ± 4% +54.3% 1107 ± 2% interrupts.CPU70.RES:Rescheduling_interrupts 494.00 ± 11% +104.5% 1010 ± 6% interrupts.CPU70.TLB:TLB_shootdowns 3920 ± 4% +29.5% 5078 ± 4% interrupts.CPU71.CAL:Function_call_interrupts 5112 ± 4% -23.3% 3920 ± 11% interrupts.CPU71.NMI:Non-maskable_interrupts 5112 ± 4% -23.3% 3920 ± 11% interrupts.CPU71.PMI:Performance_monitoring_interrupts 667.67 +64.9% 1101 ± 5% interrupts.CPU71.RES:Rescheduling_interrupts 473.00 ± 13% +103.8% 963.75 ± 13% interrupts.CPU71.TLB:TLB_shootdowns 3913 ± 3% +26.8% 4963 ± 2% interrupts.CPU72.CAL:Function_call_interrupts 5206 ± 3% -24.5% 3932 ± 10% interrupts.CPU72.NMI:Non-maskable_interrupts 5206 ± 3% -24.5% 3932 ± 10% interrupts.CPU72.PMI:Performance_monitoring_interrupts 811.67 ± 3% +33.3% 1081 ± 4% interrupts.CPU72.RES:Rescheduling_interrupts 3939 ± 3% +26.0% 4962 ± 6% interrupts.CPU73.CAL:Function_call_interrupts 5184 ± 2% -34.3% 3408 ± 25% interrupts.CPU73.NMI:Non-maskable_interrupts 5184 ± 2% -34.3% 3408 ± 25% interrupts.CPU73.PMI:Performance_monitoring_interrupts 763.00 ± 4% +40.6% 1072 ± 4% interrupts.CPU73.RES:Rescheduling_interrupts 570.00 ± 5% +97.1% 1123 ± 25% interrupts.CPU73.TLB:TLB_shootdowns 4069 ± 6% +21.7% 4954 ± 3% interrupts.CPU74.CAL:Function_call_interrupts 5223 ± 3% -23.9% 3972 ± 10% interrupts.CPU74.NMI:Non-maskable_interrupts 5223 ± 3% -23.9% 3972 ± 10% interrupts.CPU74.PMI:Performance_monitoring_interrupts 715.33 +53.9% 1101 ± 5% interrupts.CPU74.RES:Rescheduling_interrupts 526.00 ± 14% +90.9% 1004 ± 11% interrupts.CPU74.TLB:TLB_shootdowns 4034 ± 10% +21.5% 4900 ± 4% interrupts.CPU75.CAL:Function_call_interrupts 5241 ± 2% -42.9% 2995 ± 37% interrupts.CPU75.NMI:Non-maskable_interrupts 5241 ± 2% -42.9% 2995 ± 37% interrupts.CPU75.PMI:Performance_monitoring_interrupts 712.00 ± 2% +49.6% 1065 ± 6% interrupts.CPU75.RES:Rescheduling_interrupts 4203 ± 8% +18.8% 4995 ± 5% interrupts.CPU76.CAL:Function_call_interrupts 5216 ± 2% -32.4% 3525 ± 31% interrupts.CPU76.NMI:Non-maskable_interrupts 5216 ± 2% -32.4% 3525 ± 31% interrupts.CPU76.PMI:Performance_monitoring_interrupts 719.00 ± 2% +44.5% 1039 ± 5% interrupts.CPU76.RES:Rescheduling_interrupts 488.00 ± 10% +87.1% 913.25 ± 9% interrupts.CPU76.TLB:TLB_shootdowns 3955 ± 7% +23.9% 4901 ± 3% interrupts.CPU77.CAL:Function_call_interrupts 5224 ± 3% -24.5% 3944 ± 11% interrupts.CPU77.NMI:Non-maskable_interrupts 5224 ± 3% -24.5% 3944 ± 11% interrupts.CPU77.PMI:Performance_monitoring_interrupts 703.67 ± 8% +50.9% 1062 ± 3% interrupts.CPU77.RES:Rescheduling_interrupts 466.67 ± 7% +125.8% 1053 ± 13% interrupts.CPU77.TLB:TLB_shootdowns 4189 ± 6% +15.8% 4852 ± 6% interrupts.CPU78.CAL:Function_call_interrupts 5190 ± 4% -33.4% 3455 ± 24% interrupts.CPU78.NMI:Non-maskable_interrupts 5190 ± 4% -33.4% 3455 ± 24% interrupts.CPU78.PMI:Performance_monitoring_interrupts 706.00 +49.5% 1055 ± 4% interrupts.CPU78.RES:Rescheduling_interrupts 442.67 ± 8% +135.9% 1044 ± 15% interrupts.CPU78.TLB:TLB_shootdowns 3902 ± 3% +24.7% 4865 ± 3% interrupts.CPU79.CAL:Function_call_interrupts 5225 ± 3% -35.1% 3390 ± 24% interrupts.CPU79.NMI:Non-maskable_interrupts 5225 ± 3% -35.1% 3390 ± 24% interrupts.CPU79.PMI:Performance_monitoring_interrupts 704.67 ± 4% +58.3% 1115 ± 7% interrupts.CPU79.RES:Rescheduling_interrupts 444.00 ± 13% +111.1% 937.25 ± 11% interrupts.CPU79.TLB:TLB_shootdowns 4089 ± 4% +19.8% 4898 interrupts.CPU8.CAL:Function_call_interrupts 5181 ± 4% -22.3% 4024 ± 5% interrupts.CPU8.NMI:Non-maskable_interrupts 5181 ± 4% -22.3% 4024 ± 5% interrupts.CPU8.PMI:Performance_monitoring_interrupts 785.33 ± 4% +41.3% 1110 ± 6% interrupts.CPU8.RES:Rescheduling_interrupts 480.00 ± 9% +94.0% 931.00 ± 17% interrupts.CPU8.TLB:TLB_shootdowns 4093 ± 6% +22.6% 5018 ± 4% interrupts.CPU80.CAL:Function_call_interrupts 721.33 ± 5% +44.9% 1045 ± 2% interrupts.CPU80.RES:Rescheduling_interrupts 485.33 ± 10% +101.9% 980.00 ± 6% interrupts.CPU80.TLB:TLB_shootdowns 4059 ± 9% +19.3% 4843 ± 4% interrupts.CPU81.CAL:Function_call_interrupts 5243 ± 3% -25.7% 3894 ± 10% interrupts.CPU81.NMI:Non-maskable_interrupts 5243 ± 3% -25.7% 3894 ± 10% interrupts.CPU81.PMI:Performance_monitoring_interrupts 689.67 ± 7% +53.0% 1055 ± 7% interrupts.CPU81.RES:Rescheduling_interrupts 553.67 ± 16% +87.9% 1040 ± 13% interrupts.CPU81.TLB:TLB_shootdowns 3950 ± 8% +23.1% 4863 ± 6% interrupts.CPU82.CAL:Function_call_interrupts 5207 ± 3% -33.8% 3447 ± 25% interrupts.CPU82.NMI:Non-maskable_interrupts 5207 ± 3% -33.8% 3447 ± 25% interrupts.CPU82.PMI:Performance_monitoring_interrupts 723.33 +45.0% 1048 ± 2% interrupts.CPU82.RES:Rescheduling_interrupts 424.67 ± 12% +143.4% 1033 ± 11% interrupts.CPU82.TLB:TLB_shootdowns 4227 ± 9% +20.3% 5085 ± 6% interrupts.CPU83.CAL:Function_call_interrupts 5224 ± 3% -42.6% 2998 ± 38% interrupts.CPU83.NMI:Non-maskable_interrupts 5224 ± 3% -42.6% 2998 ± 38% interrupts.CPU83.PMI:Performance_monitoring_interrupts 690.00 ± 7% +57.2% 1084 ± 3% interrupts.CPU83.RES:Rescheduling_interrupts 437.33 ± 9% +170.6% 1183 ± 19% interrupts.CPU83.TLB:TLB_shootdowns 4122 ± 10% +19.3% 4918 ± 3% interrupts.CPU84.CAL:Function_call_interrupts 5236 ± 3% -44.5% 2906 ± 29% interrupts.CPU84.NMI:Non-maskable_interrupts 5236 ± 3% -44.5% 2906 ± 29% interrupts.CPU84.PMI:Performance_monitoring_interrupts 714.00 ± 2% +49.4% 1066 ± 4% interrupts.CPU84.RES:Rescheduling_interrupts 460.00 ± 17% +124.9% 1034 ± 11% interrupts.CPU84.TLB:TLB_shootdowns 4072 ± 5% +19.5% 4864 ± 4% interrupts.CPU85.CAL:Function_call_interrupts 5256 ± 3% -24.5% 3967 ± 9% interrupts.CPU85.NMI:Non-maskable_interrupts 5256 ± 3% -24.5% 3967 ± 9% interrupts.CPU85.PMI:Performance_monitoring_interrupts 696.67 ± 5% +53.2% 1067 ± 5% interrupts.CPU85.RES:Rescheduling_interrupts 445.67 ± 13% +112.0% 945.00 ± 2% interrupts.CPU85.TLB:TLB_shootdowns 4091 ± 3% +21.3% 4964 ± 4% interrupts.CPU86.CAL:Function_call_interrupts 699.33 ± 5% +56.4% 1093 ± 4% interrupts.CPU86.RES:Rescheduling_interrupts 466.00 ± 14% +126.0% 1053 ± 4% interrupts.CPU86.TLB:TLB_shootdowns 4003 ± 8% +25.3% 5015 ± 4% interrupts.CPU87.CAL:Function_call_interrupts 687.00 ± 3% +54.1% 1058 ± 5% interrupts.CPU87.RES:Rescheduling_interrupts 474.33 ± 18% +116.1% 1025 ± 9% interrupts.CPU87.TLB:TLB_shootdowns 4087 ± 8% +21.9% 4982 ± 3% interrupts.CPU88.CAL:Function_call_interrupts 695.67 +48.5% 1033 ± 2% interrupts.CPU88.RES:Rescheduling_interrupts 482.00 ± 12% +114.2% 1032 ± 7% interrupts.CPU88.TLB:TLB_shootdowns 3940 ± 8% +25.5% 4944 ± 5% interrupts.CPU89.CAL:Function_call_interrupts 5209 ± 3% -25.2% 3895 ± 10% interrupts.CPU89.NMI:Non-maskable_interrupts 5209 ± 3% -25.2% 3895 ± 10% interrupts.CPU89.PMI:Performance_monitoring_interrupts 695.67 ± 4% +51.6% 1054 ± 4% interrupts.CPU89.RES:Rescheduling_interrupts 434.00 ± 15% +131.6% 1005 ± 9% interrupts.CPU89.TLB:TLB_shootdowns 3991 ± 2% +18.5% 4728 ± 3% interrupts.CPU9.CAL:Function_call_interrupts 5149 ± 5% -33.6% 3418 ± 25% interrupts.CPU9.NMI:Non-maskable_interrupts 5149 ± 5% -33.6% 3418 ± 25% interrupts.CPU9.PMI:Performance_monitoring_interrupts 704.67 ± 7% +56.2% 1100 ± 4% interrupts.CPU9.RES:Rescheduling_interrupts 433.67 ± 8% +107.6% 900.25 ± 12% interrupts.CPU9.TLB:TLB_shootdowns 4145 ± 6% +17.4% 4868 ± 6% interrupts.CPU90.CAL:Function_call_interrupts 5217 ± 3% -42.9% 2977 ± 40% interrupts.CPU90.NMI:Non-maskable_interrupts 5217 ± 3% -42.9% 2977 ± 40% interrupts.CPU90.PMI:Performance_monitoring_interrupts 672.33 ± 4% +61.4% 1085 ± 2% interrupts.CPU90.RES:Rescheduling_interrupts 491.33 ± 14% +90.1% 934.00 ± 9% interrupts.CPU90.TLB:TLB_shootdowns 4304 ± 7% +17.4% 5051 ± 3% interrupts.CPU91.CAL:Function_call_interrupts 799.00 ± 4% +34.7% 1076 ± 3% interrupts.CPU91.RES:Rescheduling_interrupts 458.33 ± 13% +112.6% 974.50 ± 3% interrupts.CPU91.TLB:TLB_shootdowns 3978 ± 4% +21.1% 4816 ± 3% interrupts.CPU92.CAL:Function_call_interrupts 5190 ± 4% -54.3% 2371 ± 21% interrupts.CPU92.NMI:Non-maskable_interrupts 5190 ± 4% -54.3% 2371 ± 21% interrupts.CPU92.PMI:Performance_monitoring_interrupts 648.00 +66.6% 1079 ± 4% interrupts.CPU92.RES:Rescheduling_interrupts 470.67 ± 14% +99.7% 940.00 ± 17% interrupts.CPU92.TLB:TLB_shootdowns 3937 ± 6% +23.5% 4863 ± 5% interrupts.CPU93.CAL:Function_call_interrupts 5221 ± 3% -35.6% 3365 ± 23% interrupts.CPU93.NMI:Non-maskable_interrupts 5221 ± 3% -35.6% 3365 ± 23% interrupts.CPU93.PMI:Performance_monitoring_interrupts 677.67 ± 6% +53.6% 1041 ± 5% interrupts.CPU93.RES:Rescheduling_interrupts 481.00 ± 4% +123.9% 1077 ± 4% interrupts.CPU93.TLB:TLB_shootdowns 4005 ± 5% +22.8% 4919 ± 6% interrupts.CPU94.CAL:Function_call_interrupts 648.00 ± 2% +61.7% 1047 ± 6% interrupts.CPU94.RES:Rescheduling_interrupts 469.67 ± 9% +103.2% 954.25 ± 17% interrupts.CPU94.TLB:TLB_shootdowns 4039 ± 2% +26.0% 5088 ± 5% interrupts.CPU95.CAL:Function_call_interrupts 664.00 ± 8% +65.4% 1098 ± 8% interrupts.CPU95.RES:Rescheduling_interrupts 458.00 ± 18% +119.0% 1003 ± 8% interrupts.CPU95.TLB:TLB_shootdowns 3941 ± 2% +20.5% 4750 ± 3% interrupts.CPU96.CAL:Function_call_interrupts 5213 ± 5% -26.6% 3824 ± 9% interrupts.CPU96.NMI:Non-maskable_interrupts 5213 ± 5% -26.6% 3824 ± 9% interrupts.CPU96.PMI:Performance_monitoring_interrupts 707.67 ± 4% +69.0% 1196 ± 4% interrupts.CPU96.RES:Rescheduling_interrupts 453.00 ± 5% +111.2% 956.75 ± 9% interrupts.CPU96.TLB:TLB_shootdowns 3892 ± 3% +29.5% 5041 ± 8% interrupts.CPU97.CAL:Function_call_interrupts 5166 ± 4% -25.1% 3867 ± 9% interrupts.CPU97.NMI:Non-maskable_interrupts 5166 ± 4% -25.1% 3867 ± 9% interrupts.CPU97.PMI:Performance_monitoring_interrupts 755.67 ± 8% +55.6% 1175 ± 5% interrupts.CPU97.RES:Rescheduling_interrupts 487.00 ± 10% +86.1% 906.25 ± 8% interrupts.CPU97.TLB:TLB_shootdowns 3869 ± 3% +23.6% 4782 interrupts.CPU98.CAL:Function_call_interrupts 5178 ± 5% -34.9% 3369 ± 24% interrupts.CPU98.NMI:Non-maskable_interrupts 5178 ± 5% -34.9% 3369 ± 24% interrupts.CPU98.PMI:Performance_monitoring_interrupts 687.33 ± 5% +68.1% 1155 ± 4% interrupts.CPU98.RES:Rescheduling_interrupts 389.00 ± 13% +121.1% 860.25 ± 8% interrupts.CPU98.TLB:TLB_shootdowns 4000 +21.9% 4876 ± 6% interrupts.CPU99.CAL:Function_call_interrupts 719.67 +67.0% 1202 ± 2% interrupts.CPU99.RES:Rescheduling_interrupts 463.00 ± 8% +76.1% 815.50 ± 10% interrupts.CPU99.TLB:TLB_shootdowns 913220 ± 2% -24.1% 693121 ± 9% interrupts.NMI:Non-maskable_interrupts 913220 ± 2% -24.1% 693121 ± 9% interrupts.PMI:Performance_monitoring_interrupts 132379 +62.4% 215045 interrupts.RES:Rescheduling_interrupts 94465 ± 13% +95.5% 184704 ± 9% interrupts.TLB:TLB_shootdowns Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

1 year, 6 months

2
2
0 / 0

d28296d248: stress-ng.sigsegv.ops_per_sec -82.7% regression

by kernel test robot

Greeting, FYI, we noticed a -82.7% regression of stress-ng.sigsegv.ops_per_sec due to commit: commit: d28296d2484fa11e94dff65e93eb25802a443d47 ("[PATCH v7 5/7] Reimplement RLIMIT_SIGPENDING on top of ucounts") url: https://github.com/0day-ci/linux/commits/Alexey-Gladkov/Count-rlimits-in-... base: https://git.kernel.org/cgit/linux/kernel/git/shuah/linux-kselftest.git next in testcase: stress-ng on test machine: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 112G memory with following parameters: nr_threads: 100% disk: 1HDD testtime: 60s class: interrupt test: sigsegv cpufreq_governor: performance ucode: 0x42e In addition to that, the commit also has significant impact on the following tests: +------------------+-----------------------------------------------------------------------+ | testcase: change | stress-ng: stress-ng.sigq.ops_per_sec -56.1% regression | | test machine | 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 112G memory | | test parameters | class=interrupt | | | cpufreq_governor=performance | | | disk=1HDD | | | nr_threads=100% | | | test=sigq | | | testtime=60s | | | ucode=0x42e | +------------------+-----------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml bin/lkp run compatible-job.yaml ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode: interrupt/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-ivb-2ep1/sigsegv/stress-ng/60s/0x42e commit: 4660d663b4 ("Reimplement RLIMIT_MSGQUEUE on top of ucounts") d28296d248 ("Reimplement RLIMIT_SIGPENDING on top of ucounts") 4660d663b4207ce6 d28296d2484fa11e94dff65e93e ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 14:6 -217% 1:6 perf-profile.children.cycles-pp.error_entry 12:6 -179% 1:6 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 4.766e+08 -82.7% 82358308 stress-ng.sigsegv.ops 7942807 -82.7% 1372639 stress-ng.sigsegv.ops_per_sec 29408 -77.5% 6621 ± 61% stress-ng.time.file_system_inputs 1566 +69.4% 2653 stress-ng.time.system_time 1274 -84.8% 193.22 ± 8% stress-ng.time.user_time 12458 ± 5% +37.2% 17097 ± 5% numa-meminfo.node1.Active(anon) 51.41 +66.5% 85.59 iostat.cpu.system 41.17 -84.2% 6.50 ± 7% iostat.cpu.user 3040 ± 4% +37.9% 4193 ± 4% numa-vmstat.node1.nr_active_anon 3040 ± 4% +37.9% 4193 ± 4% numa-vmstat.node1.nr_zone_active_anon 50.83 +67.2% 85.00 vmstat.cpu.sy 40.50 -85.6% 5.83 ± 11% vmstat.cpu.us 225.33 -77.7% 50.33 ± 62% vmstat.io.bi 7.00 -100.0% 0.00 vmstat.memory.buff 20735 ± 2% -14.1% 17812 ± 5% meminfo.Active 13506 ± 3% +31.9% 17812 ± 5% meminfo.Active(anon) 7228 -100.0% 0.00 meminfo.Active(file) 29308 +18.4% 34687 ± 2% meminfo.Shmem 202067 -9.5% 182899 meminfo.VmallocUsed 0.01 ± 17% -0.0 0.00 ± 10% mpstat.cpu.all.iowait% 1.04 -0.1 0.92 mpstat.cpu.all.irq% 0.03 ± 8% -0.0 0.02 ± 4% mpstat.cpu.all.soft% 51.54 +35.6 87.17 mpstat.cpu.all.sys% 42.22 -35.6 6.66 ± 8% mpstat.cpu.all.usr% 0.00 ± 70% +191.7% 0.01 ± 26% perf-sched.sch_delay.avg.ms.__sched_text_start.__sched_text_start.worker_thread.kthread.ret_from_fork 0.00 ± 47% +158.8% 0.01 ± 43% perf-sched.sch_delay.max.ms.__sched_text_start.__sched_text_start.worker_thread.kthread.ret_from_fork 0.27 ± 25% -55.4% 0.12 ± 37% perf-sched.total_wait_and_delay.average.ms 202.17 ± 23% -29.5% 142.50 ± 13% perf-sched.total_wait_and_delay.count.ms 0.21 ± 18% -69.3% 0.06 ± 58% perf-sched.total_wait_time.average.ms 0.00 ± 70% +191.7% 0.01 ± 26% perf-sched.wait_and_delay.avg.ms.__sched_text_start.__sched_text_start.worker_thread.kthread.ret_from_fork 8.17 ± 29% -85.7% 1.17 ±125% perf-sched.wait_and_delay.count.__sched_text_start.__sched_text_start.schedule_timeout.rcu_gp_kthread.kthread 0.00 ± 47% +158.8% 0.01 ± 43% perf-sched.wait_and_delay.max.ms.__sched_text_start.__sched_text_start.worker_thread.kthread.ret_from_fork 3.11 ± 11% -76.1% 0.74 ±142% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.schedule_timeout.rcu_gp_kthread.kthread 1790 ± 15% -30.1% 1250 ± 13% slabinfo.dmaengine-unmap-16.active_objs 1790 ± 15% -30.1% 1250 ± 13% slabinfo.dmaengine-unmap-16.num_objs 123.00 -100.0% 0.00 slabinfo.ext4_pending_reservation.active_objs 123.00 -100.0% 0.00 slabinfo.ext4_pending_reservation.num_objs 3619 -100.0% 0.00 slabinfo.f2fs_free_nid.active_objs 3619 -100.0% 0.00 slabinfo.f2fs_free_nid.num_objs 62865 -44.1% 35155 slabinfo.kmalloc-64.active_objs 984.33 -43.9% 552.50 slabinfo.kmalloc-64.active_slabs 63031 -43.9% 35389 slabinfo.kmalloc-64.num_objs 984.33 -43.9% 552.50 slabinfo.kmalloc-64.num_slabs 161.00 ± 9% +67.1% 269.00 ± 7% slabinfo.xfs_buf.active_objs 161.00 ± 9% +67.1% 269.00 ± 7% slabinfo.xfs_buf.num_objs 3399 ± 3% +32.9% 4519 ± 4% proc-vmstat.nr_active_anon 1806 -100.0% 0.00 proc-vmstat.nr_active_file 9333 +3.0% 9610 proc-vmstat.nr_mapped 7344 +18.4% 8698 ± 2% proc-vmstat.nr_shmem 16319 -0.9% 16176 proc-vmstat.nr_slab_reclaimable 24399 -6.2% 22882 proc-vmstat.nr_slab_unreclaimable 3399 ± 3% +32.9% 4519 ± 4% proc-vmstat.nr_zone_active_anon 1806 -100.0% 0.00 proc-vmstat.nr_zone_active_file 3693 ± 80% -80.1% 736.17 ± 61% proc-vmstat.numa_hint_faults 293002 -2.7% 284991 proc-vmstat.numa_hit 249530 -3.3% 241180 proc-vmstat.numa_local 5007 ±111% -90.5% 478.00 ± 81% proc-vmstat.numa_pages_migrated 11443 ± 2% -7.0% 10636 ± 4% proc-vmstat.pgactivate 332528 -3.9% 319693 proc-vmstat.pgalloc_normal 249148 ± 2% -4.1% 239053 ± 2% proc-vmstat.pgfree 5007 ±111% -90.5% 478.00 ± 81% proc-vmstat.pgmigrate_success 14704 -77.5% 3310 ± 61% proc-vmstat.pgpgin 0.00 +2.1e+105% 2095 proc-vmstat.pgpgout 13870 ± 10% -55.1% 6227 ± 28% softirqs.CPU0.RCU 9989 ± 3% -62.2% 3775 ± 23% softirqs.CPU1.RCU 8625 ± 13% -76.1% 2061 ± 10% softirqs.CPU10.RCU 7954 ± 15% -65.9% 2709 ± 18% softirqs.CPU14.RCU 9075 ± 14% -78.7% 1929 ± 11% softirqs.CPU17.RCU 8522 ± 13% -76.7% 1985 ± 22% softirqs.CPU18.RCU 9595 ± 7% -63.3% 3522 ± 22% softirqs.CPU2.RCU 8455 ± 11% -74.5% 2152 ± 45% softirqs.CPU20.RCU 8320 ± 12% -76.7% 1939 ± 14% softirqs.CPU21.RCU 8338 ± 13% -71.7% 2359 ± 32% softirqs.CPU23.RCU 8541 ± 12% -75.5% 2089 ± 32% softirqs.CPU26.RCU 9639 ± 20% -79.5% 1976 ± 17% softirqs.CPU28.RCU 9232 ± 13% -78.0% 2026 ± 6% softirqs.CPU30.RCU 7857 ± 17% -68.9% 2446 ± 27% softirqs.CPU34.RCU 8619 ± 11% -75.8% 2081 ± 30% softirqs.CPU36.RCU 9614 ± 3% -74.3% 2469 ± 15% softirqs.CPU4.RCU 8962 ± 10% -77.9% 1981 ± 12% softirqs.CPU41.RCU 9027 ± 12% -78.6% 1932 ± 8% softirqs.CPU42.RCU 9364 ± 12% -76.5% 2197 ± 8% softirqs.CPU44.RCU 8774 ± 13% -75.5% 2147 ± 21% softirqs.CPU47.RCU 8783 ± 12% -76.0% 2105 ± 12% softirqs.CPU5.RCU 9007 ± 9% -75.8% 2177 ± 8% softirqs.CPU6.RCU 417664 ± 7% -72.8% 113621 ± 8% softirqs.RCU 12708 ± 4% +13.0% 14362 ± 2% softirqs.TIMER 60500 -27.7% 43751 interrupts.CAL:Function_call_interrupts 1121 -26.9% 819.17 ± 3% interrupts.CPU10.CAL:Function_call_interrupts 1561 ± 43% -48.8% 800.00 ± 5% interrupts.CPU11.CAL:Function_call_interrupts 1425 ± 6% -25.3% 1065 ± 6% interrupts.CPU12.CAL:Function_call_interrupts 166.17 ± 13% -26.4% 122.33 ± 21% interrupts.CPU13.RES:Rescheduling_interrupts 1402 ± 18% -25.6% 1043 ± 22% interrupts.CPU15.CAL:Function_call_interrupts 129.17 ± 50% -42.5% 74.33 ± 4% interrupts.CPU17.RES:Rescheduling_interrupts 1182 ± 9% -31.0% 815.00 ± 2% interrupts.CPU20.CAL:Function_call_interrupts 1120 -29.7% 787.17 ± 4% interrupts.CPU21.CAL:Function_call_interrupts 1115 ± 3% -28.2% 801.17 interrupts.CPU23.CAL:Function_call_interrupts 1169 ± 7% -27.2% 851.33 ± 5% interrupts.CPU24.CAL:Function_call_interrupts 177.33 ± 98% -55.9% 78.17 ± 6% interrupts.CPU25.RES:Rescheduling_interrupts 1142 ± 16% -28.8% 813.00 ± 3% interrupts.CPU27.CAL:Function_call_interrupts 1229 ± 18% -33.3% 820.33 ± 4% interrupts.CPU28.CAL:Function_call_interrupts 1124 ± 4% -28.3% 806.17 interrupts.CPU29.CAL:Function_call_interrupts 1123 ± 3% -28.5% 803.00 interrupts.CPU30.CAL:Function_call_interrupts 1127 ± 2% -32.0% 766.67 ± 19% interrupts.CPU31.CAL:Function_call_interrupts 1066 ± 8% -22.3% 829.33 ± 7% interrupts.CPU32.CAL:Function_call_interrupts 1109 -26.0% 820.50 ± 4% interrupts.CPU34.CAL:Function_call_interrupts 1315 ± 22% -37.7% 818.83 ± 2% interrupts.CPU38.CAL:Function_call_interrupts 1164 ± 4% -29.0% 827.00 ± 3% interrupts.CPU39.CAL:Function_call_interrupts 5513 ± 35% +13.1% 6237 ± 33% interrupts.CPU39.NMI:Non-maskable_interrupts 5513 ± 35% +13.1% 6237 ± 33% interrupts.CPU39.PMI:Performance_monitoring_interrupts 1277 ± 23% -36.1% 815.83 interrupts.CPU40.CAL:Function_call_interrupts 97.33 ± 28% -25.7% 72.33 ± 6% interrupts.CPU40.RES:Rescheduling_interrupts 1116 -24.8% 839.00 ± 11% interrupts.CPU42.CAL:Function_call_interrupts 1130 ± 3% -28.5% 808.67 ± 3% interrupts.CPU43.CAL:Function_call_interrupts 1121 -29.8% 787.50 ± 4% interrupts.CPU45.CAL:Function_call_interrupts 1119 -27.3% 813.83 interrupts.CPU46.CAL:Function_call_interrupts 1167 ± 6% -28.0% 840.67 interrupts.CPU47.CAL:Function_call_interrupts 1667 ± 41% -44.3% 928.67 ± 24% interrupts.CPU5.CAL:Function_call_interrupts 1369 ± 24% -39.5% 827.67 ± 3% interrupts.CPU6.CAL:Function_call_interrupts 96.83 ± 25% -23.1% 74.50 ± 2% interrupts.CPU7.RES:Rescheduling_interrupts 1123 -28.1% 807.00 interrupts.CPU9.CAL:Function_call_interrupts 0.72 ± 5% +107.4% 1.50 ± 2% perf-stat.i.MPKI 8.023e+09 -13.5% 6.943e+09 perf-stat.i.branch-instructions 1.08 ± 2% -0.5 0.55 ± 3% perf-stat.i.branch-miss-rate% 71073978 ± 2% -67.4% 23149119 ± 3% perf-stat.i.branch-misses 29.33 ± 2% +5.9 35.23 perf-stat.i.cache-miss-rate% 6189596 +120.2% 13628525 perf-stat.i.cache-misses 21228048 +82.4% 38714786 perf-stat.i.cache-references 3.36 +34.8% 4.53 perf-stat.i.cpi 109.52 ± 2% -22.5% 84.92 perf-stat.i.cpu-migrations 22695 -56.5% 9882 perf-stat.i.cycles-between-cache-misses 1.15 ± 7% -0.9 0.30 ± 4% perf-stat.i.dTLB-load-miss-rate% 1.398e+08 ± 7% -83.4% 23247225 ± 4% perf-stat.i.dTLB-load-misses 1.154e+10 -34.4% 7.564e+09 perf-stat.i.dTLB-loads 1.17 -0.0 1.13 perf-stat.i.dTLB-store-miss-rate% 1.321e+08 -82.1% 23679743 perf-stat.i.dTLB-store-misses 1.071e+10 -81.3% 2.005e+09 perf-stat.i.dTLB-stores 41869658 -74.5% 10693569 ± 56% perf-stat.i.iTLB-load-misses 19932113 ± 38% -88.3% 2325708 ± 64% perf-stat.i.iTLB-loads 3.945e+10 -25.9% 2.924e+10 perf-stat.i.instructions 1199 ± 4% +182.7% 3389 ± 26% perf-stat.i.instructions-per-iTLB-miss 0.31 -23.7% 0.24 perf-stat.i.ipc 634.71 -45.5% 345.81 perf-stat.i.metric.M/sec 166710 ± 10% +3369.3% 5783625 perf-stat.i.node-load-misses 227268 ± 6% +3063.1% 7188743 perf-stat.i.node-loads 48.41 -4.5 43.91 perf-stat.i.node-store-miss-rate% 5425859 -11.8% 4783945 perf-stat.i.node-store-misses 5599687 +6.1% 5943407 perf-stat.i.node-stores 7532204 -82.7% 1305118 perf-stat.i.page-faults 0.54 +146.0% 1.32 perf-stat.overall.MPKI 0.89 ± 2% -0.6 0.33 ± 3% perf-stat.overall.branch-miss-rate% 29.17 ± 2% +6.0 35.20 perf-stat.overall.cache-miss-rate% 3.45 +35.0% 4.65 perf-stat.overall.cpi 21953 -54.5% 9979 perf-stat.overall.cycles-between-cache-misses 1.20 ± 7% -0.9 0.31 ± 4% perf-stat.overall.dTLB-load-miss-rate% 1.22 -0.1 1.17 perf-stat.overall.dTLB-store-miss-rate% 942.39 +245.5% 3255 ± 28% perf-stat.overall.instructions-per-iTLB-miss 0.29 -25.9% 0.21 perf-stat.overall.ipc 42.24 ± 2% +2.3 44.58 perf-stat.overall.node-load-miss-rate% 49.21 -4.6 44.59 perf-stat.overall.node-store-miss-rate% 7.894e+09 -13.5% 6.83e+09 perf-stat.ps.branch-instructions 69952381 ± 2% -67.4% 22781972 ± 3% perf-stat.ps.branch-misses 6093197 +120.1% 13409962 perf-stat.ps.cache-misses 20897937 +82.3% 38097787 perf-stat.ps.cache-references 107.78 ± 2% -22.4% 83.62 perf-stat.ps.cpu-migrations 1.375e+08 ± 7% -83.4% 22871450 ± 4% perf-stat.ps.dTLB-load-misses 1.135e+10 -34.4% 7.442e+09 perf-stat.ps.dTLB-loads 1.3e+08 -82.1% 23295591 perf-stat.ps.dTLB-store-misses 1.054e+10 -81.3% 1.973e+09 perf-stat.ps.dTLB-stores 41193894 -74.5% 10519305 ± 56% perf-stat.ps.iTLB-load-misses 19610606 ± 38% -88.3% 2288293 ± 64% perf-stat.ps.iTLB-loads 3.882e+10 -25.9% 2.876e+10 perf-stat.ps.instructions 164152 ± 10% +3366.2% 5689843 perf-stat.ps.node-load-misses 223940 ± 6% +3058.2% 7072454 perf-stat.ps.node-loads 5338769 -11.8% 4706549 perf-stat.ps.node-store-misses 5510338 +6.1% 5847491 perf-stat.ps.node-stores 7410609 -82.7% 1283937 perf-stat.ps.page-faults 2.454e+12 -25.9% 1.817e+12 perf-stat.total.instructions 33.68 -29.6 4.04 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 18.94 -16.5 2.46 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 13.67 -12.3 1.42 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 12.27 -10.9 1.36 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 8.04 -7.0 1.05 ± 2% perf-profile.calltrace.cycles-pp.__entry_text_start 6.06 -6.1 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_safe_stack 6.02 ± 2% -5.4 0.66 ± 2% perf-profile.calltrace.cycles-pp.__x64_sys_rt_sigaction.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.51 -4.8 0.70 perf-profile.calltrace.cycles-pp.__setup_rt_frame.arch_do_signal_or_restart.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault 0.00 +0.6 0.65 perf-profile.calltrace.cycles-pp.inc_rlimit_ucounts_and_test.__sigqueue_alloc.__send_signal.force_sig_info_to_task.force_sig_fault 18.91 +27.0 45.89 perf-profile.calltrace.cycles-pp.irqentry_exit_to_user_mode.asm_exc_page_fault 15.00 +30.4 45.42 perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault 14.66 +30.7 45.38 perf-profile.calltrace.cycles-pp.arch_do_signal_or_restart.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault 11.73 +34.2 45.97 perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault 8.85 +36.7 45.54 perf-profile.calltrace.cycles-pp.__bad_area_nosemaphore.exc_page_fault.asm_exc_page_fault 8.16 +37.2 45.40 perf-profile.calltrace.cycles-pp.force_sig_fault.__bad_area_nosemaphore.exc_page_fault.asm_exc_page_fault 8.04 +37.3 45.39 perf-profile.calltrace.cycles-pp.force_sig_info_to_task.force_sig_fault.__bad_area_nosemaphore.exc_page_fault.asm_exc_page_fault 6.53 +37.8 44.36 perf-profile.calltrace.cycles-pp.get_signal.arch_do_signal_or_restart.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault 7.06 +38.0 45.05 perf-profile.calltrace.cycles-pp.__send_signal.force_sig_info_to_task.force_sig_fault.__bad_area_nosemaphore.exc_page_fault 4.28 ± 2% +39.2 43.46 perf-profile.calltrace.cycles-pp.__sigqueue_free.get_signal.arch_do_signal_or_restart.exit_to_user_mode_prepare.irqentry_exit_to_user_mode 4.87 ± 2% +39.9 44.81 perf-profile.calltrace.cycles-pp.__sigqueue_alloc.__send_signal.force_sig_info_to_task.force_sig_fault.__bad_area_nosemaphore 0.00 +42.7 42.72 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.put_ucounts.__sigqueue_free.get_signal 0.00 +43.0 42.99 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.put_ucounts.__sigqueue_free.get_signal.arch_do_signal_or_restart 0.00 +43.1 43.08 perf-profile.calltrace.cycles-pp.put_ucounts.__sigqueue_free.get_signal.arch_do_signal_or_restart.exit_to_user_mode_prepare 0.00 +43.5 43.52 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.get_ucounts.__sigqueue_alloc.__send_signal 0.00 +44.0 43.97 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.get_ucounts.__sigqueue_alloc.__send_signal.force_sig_info_to_task 0.00 +44.1 44.05 perf-profile.calltrace.cycles-pp.get_ucounts.__sigqueue_alloc.__send_signal.force_sig_info_to_task.force_sig_fault 31.89 +60.0 91.94 perf-profile.calltrace.cycles-pp.asm_exc_page_fault 33.84 -29.7 4.10 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 19.01 -16.5 2.48 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 14.93 -13.3 1.59 perf-profile.children.cycles-pp.syscall_return_via_sysret 13.14 -11.7 1.43 ± 2% perf-profile.children.cycles-pp.do_syscall_64 10.60 -9.3 1.27 perf-profile.children.cycles-pp.__entry_text_start 6.17 ± 2% -5.5 0.68 ± 2% perf-profile.children.cycles-pp.__x64_sys_rt_sigaction 5.55 -4.8 0.71 ± 2% perf-profile.children.cycles-pp.__setup_rt_frame 3.63 ± 2% -3.2 0.40 ± 2% perf-profile.children.cycles-pp.__x64_sys_rt_sigprocmask 3.54 -3.2 0.32 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 3.11 ± 3% -2.8 0.35 ± 3% perf-profile.children.cycles-pp._copy_from_user 2.91 -2.5 0.37 ± 2% perf-profile.children.cycles-pp.copy_fpstate_to_sigframe 2.38 -2.1 0.28 perf-profile.children.cycles-pp.__irqentry_text_end 2.40 ± 4% -2.1 0.30 ± 2% perf-profile.children.cycles-pp.__might_fault 2.23 -1.9 0.32 perf-profile.children.cycles-pp.native_irq_return_iret 2.26 ± 2% -1.9 0.35 ± 5% perf-profile.children.cycles-pp.do_user_addr_fault 1.86 -1.6 0.27 ± 3% perf-profile.children.cycles-pp.do_sigaction 1.75 -1.5 0.21 ± 3% perf-profile.children.cycles-pp.copy_user_generic_unrolled 1.67 ± 3% -1.5 0.19 ± 3% perf-profile.children.cycles-pp.__set_current_blocked 1.51 ± 2% -1.3 0.18 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irq 1.33 ± 3% -1.1 0.19 perf-profile.children.cycles-pp.copy_siginfo_to_user 1.24 ± 3% -1.1 0.13 ± 3% perf-profile.children.cycles-pp.recalc_sigpending 1.30 ± 4% -1.1 0.18 ± 3% perf-profile.children.cycles-pp._copy_to_user 1.23 ± 3% -1.1 0.15 ± 3% perf-profile.children.cycles-pp.___might_sleep 1.07 ± 4% -0.9 0.12 ± 3% perf-profile.children.cycles-pp.signal_setup_done 0.98 ± 4% -0.9 0.11 ± 4% perf-profile.children.cycles-pp.fpu__clear 0.93 ± 4% -0.8 0.12 ± 8% perf-profile.children.cycles-pp.__might_sleep 0.89 -0.8 0.14 ± 3% perf-profile.children.cycles-pp.__clear_user 0.85 ± 4% -0.7 0.10 ± 3% perf-profile.children.cycles-pp.__set_task_blocked 0.81 ± 2% -0.7 0.10 ± 8% perf-profile.children.cycles-pp.sigprocmask 0.76 ± 11% -0.7 0.07 ± 10% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 0.70 ± 2% -0.6 0.06 ± 9% perf-profile.children.cycles-pp.signal_wake_up_state 0.73 -0.6 0.13 ± 7% perf-profile.children.cycles-pp.__perf_sw_event 0.50 ± 2% -0.5 0.04 ± 44% perf-profile.children.cycles-pp.complete_signal 0.50 ± 4% -0.4 0.08 ± 11% perf-profile.children.cycles-pp.___perf_sw_event 0.45 -0.4 0.06 ± 6% perf-profile.children.cycles-pp.sync_regs 0.44 -0.4 0.07 ± 10% perf-profile.children.cycles-pp.fixup_vdso_exception 0.35 ± 2% -0.3 0.05 perf-profile.children.cycles-pp.prepare_signal 0.35 ± 3% -0.3 0.08 ± 5% perf-profile.children.cycles-pp.is_prefetch 0.33 ± 11% -0.3 0.07 ± 6% perf-profile.children.cycles-pp.kmem_cache_alloc 0.19 ± 4% -0.1 0.05 ± 45% perf-profile.children.cycles-pp.copy_from_kernel_nofault 0.33 +0.0 0.37 ± 3% perf-profile.children.cycles-pp.kmem_cache_free 0.00 +0.4 0.36 ± 4% perf-profile.children.cycles-pp.dec_rlimit_ucounts 0.00 +0.6 0.65 perf-profile.children.cycles-pp.inc_rlimit_ucounts_and_test 19.11 +26.8 45.91 perf-profile.children.cycles-pp.irqentry_exit_to_user_mode 15.69 +29.8 45.51 perf-profile.children.cycles-pp.exit_to_user_mode_prepare 14.70 +30.7 45.38 perf-profile.children.cycles-pp.arch_do_signal_or_restart 11.77 +34.2 45.99 perf-profile.children.cycles-pp.exc_page_fault 8.90 +36.7 45.55 perf-profile.children.cycles-pp.__bad_area_nosemaphore 8.17 +37.2 45.40 perf-profile.children.cycles-pp.force_sig_fault 8.10 +37.3 45.39 perf-profile.children.cycles-pp.force_sig_info_to_task 6.57 +37.8 44.38 perf-profile.children.cycles-pp.get_signal 7.11 +38.0 45.06 perf-profile.children.cycles-pp.__send_signal 4.28 ± 2% +39.2 43.46 perf-profile.children.cycles-pp.__sigqueue_free 4.89 ± 2% +39.9 44.82 perf-profile.children.cycles-pp.__sigqueue_alloc 0.00 +43.1 43.09 perf-profile.children.cycles-pp.put_ucounts 0.00 +44.1 44.05 perf-profile.children.cycles-pp.get_ucounts 31.95 +60.0 91.95 perf-profile.children.cycles-pp.asm_exc_page_fault 0.00 +86.2 86.24 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.32 ± 4% +86.7 87.01 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 18.17 -15.8 2.36 perf-profile.self.cycles-pp.syscall_exit_to_user_mode 14.90 -13.3 1.58 perf-profile.self.cycles-pp.syscall_return_via_sysret 10.60 -9.3 1.27 perf-profile.self.cycles-pp.__entry_text_start 4.55 ± 2% -4.5 0.04 ± 72% perf-profile.self.cycles-pp.__sigqueue_alloc 4.09 ± 2% -3.6 0.49 ± 2% perf-profile.self.cycles-pp.irqentry_exit_to_user_mode 2.30 ± 5% -2.1 0.19 ± 4% perf-profile.self.cycles-pp.do_syscall_64 2.38 -2.1 0.28 perf-profile.self.cycles-pp.__irqentry_text_end 2.22 -1.9 0.32 perf-profile.self.cycles-pp.native_irq_return_iret 1.92 ± 8% -1.8 0.16 ± 2% perf-profile.self.cycles-pp.__x64_sys_rt_sigaction 1.72 ± 3% -1.5 0.20 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 1.71 -1.5 0.21 ± 4% perf-profile.self.cycles-pp.copy_fpstate_to_sigframe 1.68 -1.5 0.20 ± 3% perf-profile.self.cycles-pp.copy_user_generic_unrolled 1.46 ± 4% -1.3 0.12 ± 4% perf-profile.self.cycles-pp.__x64_sys_rt_sigprocmask 1.46 ± 2% -1.3 0.17 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irq 1.21 ± 6% -1.1 0.14 ± 5% perf-profile.self.cycles-pp.__setup_rt_frame 1.21 ± 2% -1.1 0.14 ± 3% perf-profile.self.cycles-pp.___might_sleep 1.06 ± 4% -1.0 0.09 ± 5% perf-profile.self.cycles-pp.recalc_sigpending 1.01 ± 6% -1.0 0.05 perf-profile.self.cycles-pp.asm_exc_page_fault 0.97 ± 4% -0.9 0.10 ± 7% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.95 ± 2% -0.8 0.12 ± 4% perf-profile.self.cycles-pp.exit_to_user_mode_prepare 0.98 -0.8 0.15 ± 4% perf-profile.self.cycles-pp.do_sigaction 0.84 ± 4% -0.7 0.10 ± 6% perf-profile.self.cycles-pp.__might_sleep 0.72 ± 8% -0.6 0.09 ± 4% perf-profile.self.cycles-pp._copy_from_user 0.71 ± 2% -0.6 0.10 ± 7% perf-profile.self.cycles-pp.do_user_addr_fault 0.65 ± 14% -0.6 0.06 ± 11% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 0.68 ± 6% -0.6 0.10 ± 3% perf-profile.self.cycles-pp.__might_fault 0.80 -0.6 0.25 ± 3% perf-profile.self.cycles-pp.get_signal 0.64 ± 4% -0.6 0.08 ± 5% perf-profile.self.cycles-pp.fpu__clear 0.57 ± 3% -0.5 0.07 ± 7% perf-profile.self.cycles-pp.__send_signal 0.53 ± 2% -0.5 0.08 ± 6% perf-profile.self.cycles-pp.__clear_user 0.49 ± 5% -0.4 0.04 ± 45% perf-profile.self.cycles-pp.arch_do_signal_or_restart 0.43 -0.4 0.06 ± 7% perf-profile.self.cycles-pp.fixup_vdso_exception 0.42 ± 4% -0.4 0.06 ± 7% perf-profile.self.cycles-pp.___perf_sw_event 0.41 -0.4 0.05 ± 9% perf-profile.self.cycles-pp.sync_regs 0.33 ± 3% -0.3 0.04 ± 44% perf-profile.self.cycles-pp.prepare_signal 0.31 ± 12% -0.2 0.07 ± 7% perf-profile.self.cycles-pp.kmem_cache_alloc 0.22 ± 3% -0.1 0.10 ± 7% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.33 +0.0 0.36 ± 3% perf-profile.self.cycles-pp.kmem_cache_free 0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.get_ucounts 0.00 +0.1 0.06 ± 6% perf-profile.self.cycles-pp.put_ucounts 0.00 +0.4 0.36 ± 4% perf-profile.self.cycles-pp.dec_rlimit_ucounts 0.27 ± 4% +0.5 0.77 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.00 +0.6 0.65 perf-profile.self.cycles-pp.inc_rlimit_ucounts_and_test 0.00 +86.2 86.24 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath stress-ng.time.user_time 1400 +--------------------------------------------------------------------+ |.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.++.+.+.++.+.++.+.+. +.+.+.++.| 1200 |-+ + | | | 1000 |-+ | | | 800 |-+ | | | 600 |-+ | | | 400 |-+ O | | | 200 |-OO O O O O O OO O OO O O OO O O OO O OO O O OO O OO | | | 0 +--------------------------------------------------------------------+ stress-ng.time.system_time 2800 +--------------------------------------------------------------------+ | O O O O OO O O O O O | 2600 |-OO O O O O OO O OO OO O O O O O O | | | 2400 |-+ O | | | 2200 |-+ | | | 2000 |-+ | | | 1800 |-+ | | | 1600 |.++.+.+.++.+. .+.++.+.+.++.+.+.++.+.++.+.+.++.+.+ .+.+.++.+.+.++.| | +.++ + | 1400 +--------------------------------------------------------------------+ stress-ng.sigsegv.ops 5e+08 +-----------------------------------------------------------------+ |.++.+.++.+.++.+.++.++.+.++.+.++.+.++.+.++.+.++.+.++.++.+.++.+.++.| 4.5e+08 |-+ | 4e+08 |-+ | | | 3.5e+08 |-+ | 3e+08 |-+ | | | 2.5e+08 |-+ | 2e+08 |-+ | | | 1.5e+08 |-+ | 1e+08 |-+ | | OO O OO O OO O OO OO O OO O OO O OO O OO O OO O OO | 5e+07 +-----------------------------------------------------------------+ stress-ng.sigsegv.ops_per_sec 8e+06 +-------------------------------------------------------------------+ | | 7e+06 |-+ | | | 6e+06 |-+ | | | 5e+06 |-+ | | | 4e+06 |-+ | | | 3e+06 |-+ | | | 2e+06 |-+ | | OO O O OO O OO O OO O O OO O OO O OO O O OO O OO O O | 1e+06 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-ivb-2ep1: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 112G memory ========================================================================================= class/compiler/cpufreq_governor/disk/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode: interrupt/gcc-9/performance/1HDD/x86_64-rhel-8.3/100%/debian-10.4-x86_64-20200603.cgz/lkp-ivb-2ep1/sigq/stress-ng/60s/0x42e commit: 4660d663b4 ("Reimplement RLIMIT_MSGQUEUE on top of ucounts") d28296d248 ("Reimplement RLIMIT_SIGPENDING on top of ucounts") 4660d663b4207ce6 d28296d2484fa11e94dff65e93e ---------------- --------------------------- %stddev %change %stddev \ | \ 4.176e+08 -56.1% 1.831e+08 ± 21% stress-ng.sigq.ops 6959423 -56.1% 3051857 ± 21% stress-ng.sigq.ops_per_sec 2.157e+08 ± 8% -66.1% 73100172 ± 9% stress-ng.time.involuntary_context_switches 14467 +1.4% 14674 stress-ng.time.minor_page_faults 4513 +1.2% 4569 stress-ng.time.percent_of_cpu_this_job_got 2150 +17.9% 2535 ± 3% stress-ng.time.system_time 660.96 -53.1% 309.96 ± 30% stress-ng.time.user_time 2.478e+08 ± 5% -70.5% 73168836 ± 9% stress-ng.time.voluntary_context_switches 7.70 ± 8% -6.8% 7.18 iostat.cpu.idle 70.60 +16.6% 82.32 ± 3% iostat.cpu.system 21.70 -51.6% 10.49 ± 29% iostat.cpu.user 15660 ± 3% +13.7% 17806 ± 3% meminfo.Active 15660 ± 3% +13.7% 17806 ± 3% meminfo.Active(anon) 31878 ± 2% +8.8% 34688 ± 2% meminfo.Shmem 123900 ± 66% -66.9% 41021 ± 30% cpuidle.C1.usage 1.507e+08 ± 6% +9.5% 1.65e+08 ± 4% cpuidle.C6.time 681599 ± 38% -72.8% 185137 ±113% cpuidle.POLL.time 645871 ± 39% -98.5% 9580 ± 75% cpuidle.POLL.usage 1.78 ± 3% -0.8 0.95 mpstat.cpu.all.irq% 0.03 ± 4% -0.0 0.02 ± 27% mpstat.cpu.all.soft% 70.90 +12.5 83.42 ± 3% mpstat.cpu.all.sys% 22.35 -11.6 10.75 ± 29% mpstat.cpu.all.usr% 825.33 ± 2% -11.6% 729.67 ± 5% slabinfo.file_lock_cache.active_objs 825.33 ± 2% -11.6% 729.67 ± 5% slabinfo.file_lock_cache.num_objs 1455 ± 11% -32.4% 983.00 ± 13% slabinfo.khugepaged_mm_slot.active_objs 1455 ± 11% -32.4% 983.00 ± 13% slabinfo.khugepaged_mm_slot.num_objs 69.67 +17.2% 81.67 ± 3% vmstat.cpu.sy 21.00 -52.4% 10.00 ± 29% vmstat.cpu.us 7139282 ± 7% -68.5% 2251603 ± 9% vmstat.system.cs 518916 ± 6% -80.5% 101139 ± 4% vmstat.system.in 8082 ± 51% -68.3% 2565 ± 17% softirqs.CPU10.SCHED 6261 ± 72% -56.4% 2729 ± 14% softirqs.CPU17.SCHED 6147 ± 66% -64.3% 2195 ± 3% softirqs.CPU25.SCHED 16334 ± 41% -82.6% 2846 ± 24% softirqs.CPU27.SCHED 6280 ± 56% -61.9% 2394 ± 9% softirqs.CPU39.SCHED 8248 ± 50% -73.2% 2209 ± 8% softirqs.CPU40.SCHED 228327 ± 9% -46.3% 122665 ± 2% softirqs.SCHED 3851 ± 4% +13.8% 4381 ± 4% proc-vmstat.nr_active_anon 9587 +2.8% 9855 proc-vmstat.nr_mapped 7943 ± 2% +8.7% 8630 ± 2% proc-vmstat.nr_shmem 3851 ± 4% +13.8% 4381 ± 4% proc-vmstat.nr_zone_active_anon 438.33 ±122% +1463.6% 6853 ± 69% proc-vmstat.numa_pages_migrated 8816 ± 4% +11.5% 9827 ± 6% proc-vmstat.pgactivate 291848 +1.7% 296689 proc-vmstat.pgalloc_normal 438.33 ±122% +1463.6% 6853 ± 69% proc-vmstat.pgmigrate_success 252.00 ± 19% +30.8% 329.67 ± 7% numa-vmstat.node0.nr_active_anon 44373 ± 23% -64.0% 15967 ± 49% numa-vmstat.node0.nr_anon_pages 46179 ± 21% -61.8% 17619 ± 38% numa-vmstat.node0.nr_inactive_anon 252.00 ± 19% +30.8% 329.67 ± 7% numa-vmstat.node0.nr_zone_active_anon 46179 ± 21% -61.8% 17619 ± 38% numa-vmstat.node0.nr_zone_inactive_anon 3678 ± 4% +11.5% 4100 ± 4% numa-vmstat.node1.nr_active_anon 13880 ± 74% +206.7% 42565 ± 17% numa-vmstat.node1.nr_anon_pages 16004 ± 59% +181.1% 44988 ± 14% numa-vmstat.node1.nr_inactive_anon 3678 ± 4% +11.5% 4100 ± 4% numa-vmstat.node1.nr_zone_active_anon 16004 ± 59% +181.1% 44988 ± 14% numa-vmstat.node1.nr_zone_inactive_anon 1007 ± 19% +31.3% 1322 ± 7% numa-meminfo.node0.Active 1007 ± 19% +31.3% 1322 ± 7% numa-meminfo.node0.Active(anon) 39280 ± 35% -66.1% 13314 ± 74% numa-meminfo.node0.AnonHugePages 177224 ± 24% -63.9% 63933 ± 49% numa-meminfo.node0.AnonPages 182948 ± 22% -52.3% 87286 ± 59% numa-meminfo.node0.AnonPages.max 184457 ± 21% -61.8% 70532 ± 38% numa-meminfo.node0.Inactive 184457 ± 21% -61.8% 70532 ± 38% numa-meminfo.node0.Inactive(anon) 1080630 ± 5% -12.3% 947332 ± 3% numa-meminfo.node0.MemUsed 14267 ± 4% +17.0% 16695 ± 5% numa-meminfo.node1.Active 14267 ± 4% +17.0% 16695 ± 5% numa-meminfo.node1.Active(anon) 56124 ± 74% +202.8% 169963 ± 17% numa-meminfo.node1.AnonPages 75025 ± 56% +139.7% 179816 ± 14% numa-meminfo.node1.AnonPages.max 64951 ± 59% +176.3% 179452 ± 14% numa-meminfo.node1.Inactive 64951 ± 59% +176.3% 179452 ± 14% numa-meminfo.node1.Inactive(anon) 941178 ± 5% +14.6% 1078500 ± 3% numa-meminfo.node1.MemUsed 51.14 ± 15% +47.4% 75.37 ± 13% sched_debug.cfs_rq:/.load_avg.avg 114.91 ± 14% +38.2% 158.79 ± 13% sched_debug.cfs_rq:/.load_avg.stddev 6.99 ± 70% +352.8% 31.65 ± 27% sched_debug.cfs_rq:/.removed.load_avg.avg 47.92 ± 70% +153.1% 121.29 ± 12% sched_debug.cfs_rq:/.removed.load_avg.stddev 2.38 ± 97% +182.8% 6.74 ± 36% sched_debug.cfs_rq:/.removed.runnable_avg.avg 2.38 ± 97% +182.7% 6.73 ± 36% sched_debug.cfs_rq:/.removed.util_avg.avg 356.50 ± 3% -24.1% 270.50 ± 31% sched_debug.cfs_rq:/.util_avg.min 290.54 ± 2% +10.6% 321.30 ± 5% sched_debug.cfs_rq:/.util_est_enqueued.avg 446696 ± 4% +50.1% 670473 ± 7% sched_debug.cpu.avg_idle.avg 1120651 ± 11% +20.5% 1350114 ± 10% sched_debug.cpu.avg_idle.max 2124 ± 10% +43.0% 3038 ± 15% sched_debug.cpu.avg_idle.min 708.23 ± 3% -14.4% 606.42 ± 4% sched_debug.cpu.clock_task.stddev 989.50 ± 8% +12.0% 1108 sched_debug.cpu.curr->pid.min 21503 ± 71% +114.0% 46014 ± 39% sched_debug.cpu.max_idle_balance_cost.stddev 4583665 ± 7% -68.5% 1443748 ± 9% sched_debug.cpu.nr_switches.avg 6508139 ± 3% -71.5% 1854831 sched_debug.cpu.nr_switches.max 2183394 ± 28% -83.2% 367194 ± 87% sched_debug.cpu.nr_switches.min 1135976 ± 11% -69.9% 342489 ± 34% sched_debug.cpu.nr_switches.stddev 0.03 ± 59% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__sched_text_start.__sched_text_start.do_syslog.part.0 0.01 ± 12% -36.4% 0.00 ± 26% perf-sched.sch_delay.avg.ms.__sched_text_start.__sched_text_start.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 0.05 ± 57% -100.0% 0.00 perf-sched.sch_delay.max.ms.__sched_text_start.__sched_text_start.do_syslog.part.0 0.19 ± 77% -61.0% 0.07 ± 5% perf-sched.sch_delay.max.ms.__sched_text_start.__sched_text_start.pipe_read.new_sync_read.vfs_read 7.83 ±140% -99.8% 0.01 ± 38% perf-sched.sch_delay.max.ms.__sched_text_start.__sched_text_start.smpboot_thread_fn.kthread.ret_from_fork 0.36 ± 18% -64.4% 0.13 ± 75% perf-sched.total_wait_and_delay.average.ms 0.24 ± 5% -72.2% 0.07 ± 66% perf-sched.total_wait_time.average.ms 1.38 ± 6% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__sched_text_start.__sched_text_start.do_syslog.part.0 0.03 ± 41% -61.6% 0.01 ± 80% perf-sched.wait_and_delay.avg.ms.__sched_text_start.__sched_text_start.do_task_dead.do_exit.do_group_exit 0.01 ± 12% -36.4% 0.00 ± 26% perf-sched.wait_and_delay.avg.ms.__sched_text_start.__sched_text_start.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 0.08 ± 17% -30.3% 0.05 ± 3% perf-sched.wait_and_delay.avg.ms.__sched_text_start.__sched_text_start.pipe_read.new_sync_read.vfs_read 2.00 -100.0% 0.00 perf-sched.wait_and_delay.count.__sched_text_start.__sched_text_start.do_syslog.part.0 9.00 -81.5% 1.67 ± 28% perf-sched.wait_and_delay.count.__sched_text_start.__sched_text_start.schedule_timeout.rcu_gp_kthread.kthread 2.72 ± 8% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__sched_text_start.__sched_text_start.do_syslog.part.0 0.80 ± 45% -89.3% 0.09 ± 10% perf-sched.wait_and_delay.max.ms.__sched_text_start.__sched_text_start.pipe_read.new_sync_read.vfs_read 7.83 ±140% -99.8% 0.01 ± 38% perf-sched.wait_and_delay.max.ms.__sched_text_start.__sched_text_start.smpboot_thread_fn.kthread.ret_from_fork 1.35 ± 8% -100.0% 0.00 perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.do_syslog.part.0 0.03 ± 42% -66.3% 0.01 ± 82% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.do_task_dead.do_exit.do_group_exit 0.07 ± 24% -27.6% 0.05 ± 3% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.pipe_read.new_sync_read.vfs_read 3.21 ± 2% -58.6% 1.33 ± 70% perf-sched.wait_time.avg.ms.__sched_text_start.__sched_text_start.schedule_timeout.rcu_gp_kthread.kthread 2.69 ± 8% -100.0% 0.00 perf-sched.wait_time.max.ms.__sched_text_start.__sched_text_start.do_syslog.part.0 0.77 ± 50% -89.7% 0.08 ± 10% perf-sched.wait_time.max.ms.__sched_text_start.__sched_text_start.pipe_read.new_sync_read.vfs_read 6.12 ± 3% -27.9% 4.42 ± 3% perf-stat.i.MPKI 1.008e+10 ± 4% -20.8% 7.989e+09 ± 2% perf-stat.i.branch-instructions 1.55 ± 2% -0.7 0.89 perf-stat.i.branch-miss-rate% 1.372e+08 ± 2% -60.4% 54290819 perf-stat.i.branch-misses 5.24 ± 7% +6.4 11.66 ± 5% perf-stat.i.cache-miss-rate% 12610881 ± 8% +30.8% 16498692 ± 4% perf-stat.i.cache-misses 2.994e+08 -50.1% 1.495e+08 ± 2% perf-stat.i.cache-references 7370989 ± 7% -68.6% 2316288 ± 9% perf-stat.i.context-switches 2.75 ± 4% +37.7% 3.78 ± 2% perf-stat.i.cpi 37935 ± 20% -99.1% 344.33 ± 39% perf-stat.i.cpu-migrations 10857 ± 9% -24.6% 8184 ± 4% perf-stat.i.cycles-between-cache-misses 1.33 ± 4% -0.5 0.81 ± 15% perf-stat.i.dTLB-load-miss-rate% 1.921e+08 ± 3% -59.4% 78058438 ± 14% perf-stat.i.dTLB-load-misses 1.375e+10 ± 4% -32.6% 9.26e+09 perf-stat.i.dTLB-loads 0.51 ± 5% +0.1 0.61 ± 9% perf-stat.i.dTLB-store-miss-rate% 52874010 -55.6% 23456497 ± 13% perf-stat.i.dTLB-store-misses 9.955e+09 ± 3% -63.0% 3.687e+09 ± 4% perf-stat.i.dTLB-stores 62946854 ± 4% -59.7% 25392611 ± 3% perf-stat.i.iTLB-load-misses 4.88e+10 ± 4% -28.0% 3.514e+10 perf-stat.i.instructions 1071 +54.5% 1655 ± 10% perf-stat.i.instructions-per-iTLB-miss 0.38 ± 3% -26.8% 0.28 ± 3% perf-stat.i.ipc 0.52 ± 7% -70.6% 0.15 ± 11% perf-stat.i.metric.K/sec 711.88 ± 3% -38.1% 440.48 perf-stat.i.metric.M/sec 47.42 -1.7 45.75 perf-stat.i.node-load-miss-rate% 3468825 ± 20% +118.3% 7571866 ± 5% perf-stat.i.node-load-misses 3747717 ± 20% +133.9% 8764955 ± 4% perf-stat.i.node-loads 47.67 -2.6 45.11 perf-stat.i.node-store-miss-rate% 8290895 ± 4% -28.2% 5954593 ± 6% perf-stat.i.node-store-misses 8800840 -19.9% 7053271 ± 5% perf-stat.i.node-stores 6.14 ± 3% -30.8% 4.25 perf-stat.overall.MPKI 1.36 ± 2% -0.7 0.68 perf-stat.overall.branch-miss-rate% 4.22 ± 10% +6.8 11.05 ± 6% perf-stat.overall.cache-miss-rate% 2.79 ± 4% +38.7% 3.87 perf-stat.overall.cpi 10868 ± 8% -23.9% 8267 ± 4% perf-stat.overall.cycles-between-cache-misses 1.38 ± 4% -0.5 0.84 ± 15% perf-stat.overall.dTLB-load-miss-rate% 0.53 ± 5% +0.1 0.63 ± 8% perf-stat.overall.dTLB-store-miss-rate% 775.35 +78.9% 1386 ± 5% perf-stat.overall.instructions-per-iTLB-miss 0.36 ± 4% -28.0% 0.26 perf-stat.overall.ipc 48.07 -1.7 46.34 perf-stat.overall.node-load-miss-rate% 48.49 -2.7 45.76 perf-stat.overall.node-store-miss-rate% 9.922e+09 ± 4% -20.8% 7.859e+09 ± 2% perf-stat.ps.branch-instructions 1.351e+08 ± 2% -60.4% 53438494 perf-stat.ps.branch-misses 12410866 ± 8% +30.8% 16234283 ± 4% perf-stat.ps.cache-misses 2.946e+08 -50.1% 1.47e+08 ± 2% perf-stat.ps.cache-references 7252050 ± 7% -68.6% 2278311 ± 9% perf-stat.ps.context-switches 37323 ± 20% -99.1% 338.80 ± 39% perf-stat.ps.cpu-migrations 1.89e+08 ± 3% -59.4% 76787292 ± 14% perf-stat.ps.dTLB-load-misses 1.352e+10 ± 4% -32.6% 9.11e+09 perf-stat.ps.dTLB-loads 52020930 -55.6% 23075080 ± 13% perf-stat.ps.dTLB-store-misses 9.794e+09 ± 3% -63.0% 3.627e+09 ± 4% perf-stat.ps.dTLB-stores 61931461 ± 4% -59.7% 24978175 ± 3% perf-stat.ps.iTLB-load-misses 4.802e+10 ± 4% -28.0% 3.457e+10 perf-stat.ps.instructions 3412948 ± 20% +118.2% 7448345 ± 5% perf-stat.ps.node-load-misses 3687448 ± 20% +133.8% 8622075 ± 4% perf-stat.ps.node-loads 8157655 ± 4% -28.2% 5857930 ± 6% perf-stat.ps.node-store-misses 8659916 -19.9% 6939197 ± 5% perf-stat.ps.node-stores 3.035e+12 ± 4% -27.9% 2.189e+12 perf-stat.total.instructions 82.33 ± 64% +1322.3% 1171 ±127% interrupts.36:PCI-MSI.2621442-edge.eth0-TxRx-1 17959753 ± 6% -99.7% 45889 ± 2% interrupts.CAL:Function_call_interrupts 103593 ±125% -99.5% 569.33 ± 9% interrupts.CPU0.CAL:Function_call_interrupts 45339 ±104% -94.7% 2414 ± 30% interrupts.CPU0.RES:Rescheduling_interrupts 1078794 ±112% -99.9% 1002 ± 12% interrupts.CPU1.CAL:Function_call_interrupts 6507 ± 26% +21.7% 7922 ± 6% interrupts.CPU1.NMI:Non-maskable_interrupts 6507 ± 26% +21.7% 7922 ± 6% interrupts.CPU1.PMI:Performance_monitoring_interrupts 505526 ±101% -99.8% 1223 ± 46% interrupts.CPU1.RES:Rescheduling_interrupts 791738 ± 80% -99.9% 898.67 ± 13% interrupts.CPU10.CAL:Function_call_interrupts 406363 ± 71% -99.4% 2378 ± 80% interrupts.CPU10.RES:Rescheduling_interrupts 46401 ±115% -98.3% 789.00 ± 21% interrupts.CPU11.CAL:Function_call_interrupts 5509 ± 35% +25.8% 6931 ± 28% interrupts.CPU11.NMI:Non-maskable_interrupts 5509 ± 35% +25.8% 6931 ± 28% interrupts.CPU11.PMI:Performance_monitoring_interrupts 27829 ±104% -87.3% 3526 ± 77% interrupts.CPU11.RES:Rescheduling_interrupts 290182 ±130% -99.6% 1299 ± 24% interrupts.CPU12.CAL:Function_call_interrupts 170757 ±136% -98.3% 2886 ± 40% interrupts.CPU12.RES:Rescheduling_interrupts 131940 ± 43% -99.1% 1206 ± 37% interrupts.CPU13.CAL:Function_call_interrupts 68303 ± 32% -97.9% 1445 ±126% interrupts.CPU13.RES:Rescheduling_interrupts 241043 ±116% -99.5% 1319 ± 32% interrupts.CPU14.CAL:Function_call_interrupts 98573 ± 93% -97.0% 3005 ± 47% interrupts.CPU14.RES:Rescheduling_interrupts 874074 ± 72% -99.9% 875.33 ± 6% interrupts.CPU15.CAL:Function_call_interrupts 350120 ± 49% -99.9% 412.67 ± 52% interrupts.CPU15.RES:Rescheduling_interrupts 577696 ± 23% -99.9% 855.33 ± 3% interrupts.CPU16.CAL:Function_call_interrupts 8263 -24.9% 6202 ± 23% interrupts.CPU16.NMI:Non-maskable_interrupts 8263 -24.9% 6202 ± 23% interrupts.CPU16.PMI:Performance_monitoring_interrupts 237815 ± 15% -99.5% 1220 ± 53% interrupts.CPU16.RES:Rescheduling_interrupts 686797 ±112% -99.9% 933.67 ± 13% interrupts.CPU17.CAL:Function_call_interrupts 317108 ±105% -99.8% 536.00 ± 49% interrupts.CPU17.RES:Rescheduling_interrupts 748163 ±122% -99.9% 863.00 ± 5% interrupts.CPU18.CAL:Function_call_interrupts 320920 ±112% -99.5% 1469 ± 95% interrupts.CPU18.RES:Rescheduling_interrupts 286835 ±121% -99.7% 943.33 ± 15% interrupts.CPU19.CAL:Function_call_interrupts 174455 ±113% -98.7% 2326 ± 69% interrupts.CPU19.RES:Rescheduling_interrupts 443377 ± 83% -99.8% 944.33 ± 12% interrupts.CPU2.CAL:Function_call_interrupts 57297 ± 99% -98.5% 855.00 ± 6% interrupts.CPU20.CAL:Function_call_interrupts 54090 ± 67% -99.0% 535.33 ± 74% interrupts.CPU20.RES:Rescheduling_interrupts 27584 ± 80% -96.8% 890.33 ± 6% interrupts.CPU21.CAL:Function_call_interrupts 27052 ± 82% -94.3% 1539 ± 58% interrupts.CPU21.RES:Rescheduling_interrupts 62804 ±104% -97.8% 1362 ± 47% interrupts.CPU22.CAL:Function_call_interrupts 27230 ± 84% -92.6% 2002 ± 86% interrupts.CPU22.RES:Rescheduling_interrupts 351930 ± 72% -99.7% 966.00 ± 16% interrupts.CPU23.CAL:Function_call_interrupts 136149 ± 62% -98.1% 2565 ± 93% interrupts.CPU23.RES:Rescheduling_interrupts 366644 ±138% -99.8% 877.67 ± 5% interrupts.CPU24.CAL:Function_call_interrupts 499203 ± 82% -99.8% 862.33 ± 3% interrupts.CPU25.CAL:Function_call_interrupts 298172 ± 70% -99.2% 2317 ± 19% interrupts.CPU25.RES:Rescheduling_interrupts 11844 ± 61% -92.9% 838.33 ± 2% interrupts.CPU26.CAL:Function_call_interrupts 82.33 ± 64% +1322.3% 1171 ±127% interrupts.CPU27.36:PCI-MSI.2621442-edge.eth0-TxRx-1 1841110 ± 49% -100.0% 746.33 ± 26% interrupts.CPU27.CAL:Function_call_interrupts 5501 ± 35% +25.6% 6909 ± 28% interrupts.CPU27.NMI:Non-maskable_interrupts 5501 ± 35% +25.6% 6909 ± 28% interrupts.CPU27.PMI:Performance_monitoring_interrupts 1089819 ± 55% -99.8% 1727 ± 80% interrupts.CPU27.RES:Rescheduling_interrupts 74718 ±118% -98.7% 942.00 interrupts.CPU28.CAL:Function_call_interrupts 5504 ± 35% +25.6% 6912 ± 28% interrupts.CPU28.NMI:Non-maskable_interrupts 5504 ± 35% +25.6% 6912 ± 28% interrupts.CPU28.PMI:Performance_monitoring_interrupts 67380 ±125% -98.5% 1034 ± 37% interrupts.CPU28.RES:Rescheduling_interrupts 367428 ± 77% -99.7% 1054 ± 22% interrupts.CPU29.CAL:Function_call_interrupts 239038 ± 82% -99.3% 1767 ± 84% interrupts.CPU29.RES:Rescheduling_interrupts 220744 ±117% -99.5% 1046 ± 21% interrupts.CPU3.CAL:Function_call_interrupts 191269 ±119% -98.4% 3122 ± 15% interrupts.CPU3.RES:Rescheduling_interrupts 247241 ± 81% -99.6% 874.33 ± 5% interrupts.CPU30.CAL:Function_call_interrupts 161878 ± 84% -98.1% 3063 ± 37% interrupts.CPU30.RES:Rescheduling_interrupts 144713 ±110% -99.3% 1077 ± 23% interrupts.CPU31.CAL:Function_call_interrupts 5502 ± 35% +25.6% 6911 ± 28% interrupts.CPU31.NMI:Non-maskable_interrupts 5502 ± 35% +25.6% 6911 ± 28% interrupts.CPU31.PMI:Performance_monitoring_interrupts 84227 ±100% -96.4% 3013 ± 72% interrupts.CPU31.RES:Rescheduling_interrupts 104510 ±109% -99.2% 848.00 ± 4% interrupts.CPU32.CAL:Function_call_interrupts 5515 ± 35% +31.4% 7249 ± 20% interrupts.CPU32.NMI:Non-maskable_interrupts 5515 ± 35% +31.4% 7249 ± 20% interrupts.CPU32.PMI:Performance_monitoring_interrupts 70068 ±101% -96.7% 2320 ±126% interrupts.CPU32.RES:Rescheduling_interrupts 87671 ±131% -99.0% 857.33 ± 2% interrupts.CPU33.CAL:Function_call_interrupts 35054 ±127% -96.5% 1242 ± 80% interrupts.CPU33.RES:Rescheduling_interrupts 77336 ±124% -98.7% 988.33 ± 21% interrupts.CPU34.CAL:Function_call_interrupts 66106 ±123% -96.0% 2630 ± 52% interrupts.CPU34.RES:Rescheduling_interrupts 136901 ±122% -99.1% 1278 ± 30% interrupts.CPU35.CAL:Function_call_interrupts 731054 ± 69% -99.9% 1046 ± 15% interrupts.CPU36.CAL:Function_call_interrupts 376136 ± 70% -99.6% 1429 ± 37% interrupts.CPU36.RES:Rescheduling_interrupts 941442 ±105% -99.9% 851.33 ± 7% interrupts.CPU37.CAL:Function_call_interrupts 300885 ± 80% -99.2% 2379 ± 57% interrupts.CPU37.RES:Rescheduling_interrupts 242878 ±117% -99.6% 995.00 ± 16% interrupts.CPU38.CAL:Function_call_interrupts 702836 ± 69% -99.9% 855.00 ± 3% interrupts.CPU39.CAL:Function_call_interrupts 286450 ± 74% -99.5% 1448 ± 70% interrupts.CPU39.RES:Rescheduling_interrupts 37564 ± 55% -96.0% 1505 ± 62% interrupts.CPU4.CAL:Function_call_interrupts 6346 ± 26% +24.0% 7870 ± 6% interrupts.CPU4.NMI:Non-maskable_interrupts 6346 ± 26% +24.0% 7870 ± 6% interrupts.CPU4.PMI:Performance_monitoring_interrupts 24154 ± 65% -94.0% 1444 ± 83% interrupts.CPU4.RES:Rescheduling_interrupts 977103 ± 53% -99.9% 837.33 ± 3% interrupts.CPU40.CAL:Function_call_interrupts 512103 ± 71% -99.7% 1579 ± 66% interrupts.CPU40.RES:Rescheduling_interrupts 190023 ±117% -99.6% 818.67 ± 2% interrupts.CPU41.CAL:Function_call_interrupts 346179 ± 54% -99.8% 829.00 ± 2% interrupts.CPU42.CAL:Function_call_interrupts 183103 ± 48% -99.5% 850.00 ± 54% interrupts.CPU42.RES:Rescheduling_interrupts 371814 ±131% -99.8% 863.33 ± 5% interrupts.CPU43.CAL:Function_call_interrupts 158123 ±124% -99.2% 1224 ± 71% interrupts.CPU43.RES:Rescheduling_interrupts 578077 ± 61% -99.9% 850.67 ± 3% interrupts.CPU44.CAL:Function_call_interrupts 331471 ± 53% -99.7% 1120 ± 91% interrupts.CPU44.RES:Rescheduling_interrupts 172794 ±114% -99.5% 838.33 ± 4% interrupts.CPU45.CAL:Function_call_interrupts 89408 ±111% -99.6% 326.00 ± 23% interrupts.CPU45.RES:Rescheduling_interrupts 34858 ± 59% -97.3% 947.00 ± 13% interrupts.CPU46.CAL:Function_call_interrupts 22612 ± 53% -87.6% 2794 ± 27% interrupts.CPU46.RES:Rescheduling_interrupts 41349 ± 57% -97.6% 996.67 ± 10% interrupts.CPU47.CAL:Function_call_interrupts 27498 ± 54% -93.7% 1725 ± 89% interrupts.CPU47.RES:Rescheduling_interrupts 45388 ± 61% -97.5% 1141 ± 57% interrupts.CPU5.CAL:Function_call_interrupts 32500 ± 63% -96.7% 1063 ± 95% interrupts.CPU5.RES:Rescheduling_interrupts 534772 ±105% -99.8% 843.33 ± 3% interrupts.CPU6.CAL:Function_call_interrupts 259978 ± 94% -98.5% 4008 ± 4% interrupts.CPU6.RES:Rescheduling_interrupts 589336 ± 71% -99.8% 1042 ± 25% interrupts.CPU7.CAL:Function_call_interrupts 311886 ± 69% -99.4% 1966 ± 66% interrupts.CPU7.RES:Rescheduling_interrupts 363153 ± 95% -99.8% 858.00 ± 5% interrupts.CPU8.CAL:Function_call_interrupts 190915 ± 83% -98.1% 3681 ± 94% interrupts.CPU8.RES:Rescheduling_interrupts 78806 ± 46% -98.9% 905.33 ± 7% interrupts.CPU9.CAL:Function_call_interrupts 47384 ± 29% -95.9% 1963 ± 58% interrupts.CPU9.RES:Rescheduling_interrupts 9209745 ± 15% -99.0% 95188 ± 9% interrupts.RES:Rescheduling_interrupts 20.15 ± 4% -14.2 5.98 ± 15% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 15.92 ± 4% -11.9 3.98 ± 7% perf-profile.calltrace.cycles-pp.signal_wake_up_state.__send_signal.do_send_sig_info.kill_pid_info.do_rt_sigqueueinfo 15.71 ± 4% -11.8 3.95 ± 7% perf-profile.calltrace.cycles-pp.try_to_wake_up.signal_wake_up_state.__send_signal.do_send_sig_info.kill_pid_info 13.68 ± 6% -10.8 2.93 ± 10% perf-profile.calltrace.cycles-pp.schedule_hrtimeout_range_clock.do_sigtimedwait.__x64_sys_rt_sigtimedwait.do_syscall_64.entry_SYSCALL_64_after_hwframe 12.23 ± 26% -10.4 1.80 ± 38% perf-profile.calltrace.cycles-pp.group_send_sig_info.kill_pid_info.do_rt_sigqueueinfo.__x64_sys_rt_sigqueueinfo.do_syscall_64 11.55 ± 27% -10.0 1.56 ± 39% perf-profile.calltrace.cycles-pp.security_task_kill.group_send_sig_info.kill_pid_info.do_rt_sigqueueinfo.__x64_sys_rt_sigqueueinfo 11.33 ± 27% -9.8 1.49 ± 40% perf-profile.calltrace.cycles-pp.apparmor_task_kill.security_task_kill.group_send_sig_info.kill_pid_info.do_rt_sigqueueinfo 11.78 ± 5% -9.3 2.52 ± 9% perf-profile.calltrace.cycles-pp.schedule.schedule_hrtimeout_range_clock.do_sigtimedwait.__x64_sys_rt_sigtimedwait.do_syscall_64 11.53 ± 5% -9.1 2.47 ± 9% perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.schedule_hrtimeout_range_clock.do_sigtimedwait.__x64_sys_rt_sigtimedwait 8.27 ± 13% -6.3 1.99 ± 7% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 7.32 ± 2% -4.9 2.43 ± 25% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 6.20 ± 13% -4.7 1.53 ± 10% perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 5.94 ± 13% -4.5 1.46 ± 10% perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 6.32 ± 8% -4.3 2.07 ± 6% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.signal_wake_up_state.__send_signal.do_send_sig_info 4.57 ± 30% -4.1 0.44 ± 74% perf-profile.calltrace.cycles-pp.aa_get_task_label.apparmor_task_kill.security_task_kill.group_send_sig_info.kill_pid_info 5.55 ± 8% -3.7 1.90 ± 6% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.signal_wake_up_state.__send_signal 4.43 ± 7% -3.4 0.99 ± 6% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.signal_wake_up_state.__send_signal.do_send_sig_info 4.29 ± 7% -3.3 0.95 ± 6% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.signal_wake_up_state.__send_signal 4.19 ± 6% -3.3 0.92 ± 8% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__sched_text_start.schedule.schedule_hrtimeout_range_clock.do_sigtimedwait 4.60 -2.9 1.69 ± 28% perf-profile.calltrace.cycles-pp.__entry_text_start 1.93 ± 16% -1.6 0.37 ± 70% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__sched_text_start.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode 2.40 ± 6% -1.5 0.86 ± 6% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.signal_wake_up_state 0.00 +0.7 0.67 perf-profile.calltrace.cycles-pp.inc_rlimit_ucounts_and_test.__sigqueue_alloc.__send_signal.do_send_sig_info.kill_pid_info 0.67 ± 16% +2.4 3.11 ± 53% perf-profile.calltrace.cycles-pp.__lock_task_sighand.do_send_sig_info.kill_pid_info.do_rt_sigqueueinfo.__x64_sys_rt_sigqueueinfo 0.42 ± 71% +2.7 3.08 ± 53% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__lock_task_sighand.do_send_sig_info.kill_pid_info.do_rt_sigqueueinfo 0.00 +3.0 2.98 ± 54% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__lock_task_sighand.do_send_sig_info.kill_pid_info 0.00 +3.2 3.23 ± 51% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.do_sigtimedwait.__x64_sys_rt_sigtimedwait.do_syscall_64 0.00 +3.3 3.29 ± 51% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.do_sigtimedwait.__x64_sys_rt_sigtimedwait.do_syscall_64.entry_SYSCALL_64_after_hwframe 35.47 ± 7% +9.5 45.02 perf-profile.calltrace.cycles-pp.__x64_sys_rt_sigqueueinfo.do_syscall_64.entry_SYSCALL_64_after_hwframe 34.17 ± 7% +10.4 44.54 perf-profile.calltrace.cycles-pp.do_rt_sigqueueinfo.__x64_sys_rt_sigqueueinfo.do_syscall_64.entry_SYSCALL_64_after_hwframe 33.33 ± 7% +10.9 44.22 perf-profile.calltrace.cycles-pp.kill_pid_info.do_rt_sigqueueinfo.__x64_sys_rt_sigqueueinfo.do_syscall_64.entry_SYSCALL_64_after_hwframe 81.67 +12.6 94.25 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 19.16 ± 4% +19.5 38.64 ± 8% perf-profile.calltrace.cycles-pp.__send_signal.do_send_sig_info.kill_pid_info.do_rt_sigqueueinfo.__x64_sys_rt_sigqueueinfo 21.31 ± 6% +20.5 41.77 ± 4% perf-profile.calltrace.cycles-pp.__x64_sys_rt_sigtimedwait.do_syscall_64.entry_SYSCALL_64_after_hwframe 20.83 ± 3% +21.5 42.33 ± 3% perf-profile.calltrace.cycles-pp.do_send_sig_info.kill_pid_info.do_rt_sigqueueinfo.__x64_sys_rt_sigqueueinfo.do_syscall_64 19.46 ± 5% +21.9 41.31 ± 4% perf-profile.calltrace.cycles-pp.do_sigtimedwait.__x64_sys_rt_sigtimedwait.do_syscall_64.entry_SYSCALL_64_after_hwframe 59.72 ± 2% +28.1 87.78 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.78 ± 8% +30.8 34.58 ± 9% perf-profile.calltrace.cycles-pp.dequeue_signal.do_sigtimedwait.__x64_sys_rt_sigtimedwait.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.07 ± 8% +31.3 34.40 ± 9% perf-profile.calltrace.cycles-pp.__dequeue_signal.dequeue_signal.do_sigtimedwait.__x64_sys_rt_sigtimedwait.do_syscall_64 1.74 ± 9% +32.1 33.86 ± 9% perf-profile.calltrace.cycles-pp.__sigqueue_free.__dequeue_signal.dequeue_signal.do_sigtimedwait.__x64_sys_rt_sigtimedwait 1.94 ± 8% +32.5 34.40 ± 8% perf-profile.calltrace.cycles-pp.__sigqueue_alloc.__send_signal.do_send_sig_info.kill_pid_info.do_rt_sigqueueinfo 0.00 +32.8 32.81 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.put_ucounts.__sigqueue_free.__dequeue_signal 0.00 +33.0 33.02 ± 8% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.get_ucounts.__sigqueue_alloc.__send_signal 0.00 +33.2 33.22 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.put_ucounts.__sigqueue_free.__dequeue_signal.dequeue_signal 0.00 +33.3 33.32 ± 9% perf-profile.calltrace.cycles-pp.put_ucounts.__sigqueue_free.__dequeue_signal.dequeue_signal.do_sigtimedwait 0.00 +33.5 33.48 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.get_ucounts.__sigqueue_alloc.__send_signal.do_send_sig_info 0.00 +33.6 33.57 ± 8% perf-profile.calltrace.cycles-pp.get_ucounts.__sigqueue_alloc.__send_signal.do_send_sig_info.kill_pid_info 20.22 ± 4% -14.2 6.00 ± 15% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 18.14 ± 8% -14.1 4.06 ± 10% perf-profile.children.cycles-pp.schedule 17.92 ± 7% -14.0 3.96 ± 9% perf-profile.children.cycles-pp.__sched_text_start 15.92 ± 4% -11.9 3.98 ± 7% perf-profile.children.cycles-pp.signal_wake_up_state 15.73 ± 4% -11.8 3.96 ± 7% perf-profile.children.cycles-pp.try_to_wake_up 13.71 ± 6% -10.8 2.93 ± 10% perf-profile.children.cycles-pp.schedule_hrtimeout_range_clock 12.24 ± 26% -10.4 1.80 ± 38% perf-profile.children.cycles-pp.group_send_sig_info 11.56 ± 27% -10.0 1.56 ± 39% perf-profile.children.cycles-pp.security_task_kill 11.35 ± 27% -9.9 1.49 ± 40% perf-profile.children.cycles-pp.apparmor_task_kill 8.45 ± 12% -6.5 2.00 ± 7% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 8.04 -5.3 2.71 ± 25% perf-profile.children.cycles-pp.syscall_return_via_sysret 6.34 ± 8% -4.3 2.07 ± 6% perf-profile.children.cycles-pp.select_task_rq_fair 4.58 ± 30% -4.0 0.55 ± 36% perf-profile.children.cycles-pp.aa_get_task_label 6.03 -4.0 2.05 ± 28% perf-profile.children.cycles-pp.__entry_text_start 4.70 ± 6% -3.7 0.99 ± 6% perf-profile.children.cycles-pp.ttwu_do_activate 5.63 ± 8% -3.7 1.93 ± 6% perf-profile.children.cycles-pp.select_idle_sibling 4.57 ± 6% -3.6 0.96 ± 6% perf-profile.children.cycles-pp.enqueue_task_fair 4.23 ± 7% -3.3 0.93 ± 8% perf-profile.children.cycles-pp.dequeue_task_fair 3.78 ± 12% -2.9 0.91 ± 11% perf-profile.children.cycles-pp.pick_next_task_fair 3.44 ± 9% -2.7 0.78 ± 9% perf-profile.children.cycles-pp.switch_mm_irqs_off 3.07 ± 10% -2.4 0.68 ± 9% perf-profile.children.cycles-pp.update_load_avg 2.96 ± 3% -2.3 0.69 ± 9% perf-profile.children.cycles-pp.update_curr 2.35 ± 3% -1.9 0.48 ± 6% perf-profile.children.cycles-pp.enqueue_entity 2.26 ± 13% -1.7 0.56 ± 10% perf-profile.children.cycles-pp.load_new_mm_cr3 2.44 ± 6% -1.6 0.86 ± 5% perf-profile.children.cycles-pp.available_idle_cpu 1.82 ± 4% -1.4 0.41 ± 7% perf-profile.children.cycles-pp.dequeue_entity 1.93 ± 3% -1.4 0.52 ± 32% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 1.63 ± 10% -1.3 0.28 ± 8% perf-profile.children.cycles-pp._raw_spin_lock 1.59 ± 13% -1.3 0.30 ± 10% perf-profile.children.cycles-pp.switch_fpu_return 1.63 ± 11% -1.2 0.39 ± 10% perf-profile.children.cycles-pp.reweight_entity 1.47 ± 7% -1.2 0.31 ± 11% perf-profile.children.cycles-pp.__switch_to 1.25 ± 12% -1.0 0.27 ± 13% perf-profile.children.cycles-pp.__switch_to_asm 1.21 ± 10% -0.9 0.27 ± 13% perf-profile.children.cycles-pp.set_next_entity 1.07 ± 7% -0.9 0.13 ± 25% perf-profile.children.cycles-pp.finish_task_switch 1.30 ± 4% -0.9 0.41 ± 10% perf-profile.children.cycles-pp._copy_from_user 1.11 ± 7% -0.8 0.27 ± 12% perf-profile.children.cycles-pp.update_rq_clock 1.04 ± 4% -0.8 0.24 ± 15% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.99 ± 13% -0.8 0.21 ± 14% perf-profile.children.cycles-pp.hrtimer_start_range_ns 0.98 ± 4% -0.8 0.22 ± 13% perf-profile.children.cycles-pp.check_preempt_curr 1.03 -0.8 0.28 ± 4% perf-profile.children.cycles-pp.recalc_sigpending 0.97 ± 12% -0.7 0.22 ± 10% perf-profile.children.cycles-pp.__update_load_avg_se 0.90 ± 8% -0.7 0.18 ± 12% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template 0.95 ± 10% -0.7 0.23 ± 10% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.94 ± 33% -0.7 0.23 ± 40% perf-profile.children.cycles-pp.aa_may_signal 0.94 ± 6% -0.7 0.23 ± 2% perf-profile.children.cycles-pp.copy_siginfo_to_user 0.95 ± 6% -0.7 0.27 ± 9% perf-profile.children.cycles-pp.__might_fault 0.85 ± 5% -0.7 0.19 ± 11% perf-profile.children.cycles-pp.check_preempt_wakeup 0.83 ± 9% -0.6 0.19 ± 7% perf-profile.children.cycles-pp.___perf_sw_event 1.00 ± 2% -0.6 0.36 ± 20% perf-profile.children.cycles-pp.__copy_siginfo_from_user 0.79 ± 17% -0.6 0.19 ± 11% perf-profile.children.cycles-pp.put_prev_entity 0.80 ± 12% -0.6 0.20 ± 8% perf-profile.children.cycles-pp.pick_next_entity 0.82 ± 2% -0.5 0.29 ± 24% perf-profile.children.cycles-pp.__x64_sys_getpid 0.68 ± 2% -0.5 0.16 ± 3% perf-profile.children.cycles-pp.update_cfs_group 0.87 ± 17% -0.5 0.38 perf-profile.children.cycles-pp.asm_call_sysvec_on_stack 0.53 ± 6% -0.5 0.06 ± 16% perf-profile.children.cycles-pp.complete_signal 0.58 ± 6% -0.5 0.12 ± 10% perf-profile.children.cycles-pp.sched_clock_cpu 0.55 ± 5% -0.4 0.10 ± 12% perf-profile.children.cycles-pp.hrtimer_try_to_cancel 0.79 ± 12% -0.4 0.35 ± 7% perf-profile.children.cycles-pp.cpumask_next_wrap 0.62 ± 18% -0.4 0.19 ± 10% perf-profile.children.cycles-pp._find_next_bit 0.71 ± 2% -0.4 0.28 ± 34% perf-profile.children.cycles-pp.__task_pid_nr_ns 0.62 ± 3% -0.4 0.21 ± 30% perf-profile.children.cycles-pp.__x64_sys_getuid 0.53 ± 14% -0.4 0.14 ± 17% perf-profile.children.cycles-pp.__calc_delta 0.59 ± 8% -0.4 0.20 ± 32% perf-profile.children.cycles-pp.check_kill_permission 0.50 ± 6% -0.4 0.11 ± 11% perf-profile.children.cycles-pp.sched_clock 0.50 ± 13% -0.4 0.12 ± 10% perf-profile.children.cycles-pp.copy_fpregs_to_fpstate 0.47 ± 6% -0.4 0.09 ± 18% perf-profile.children.cycles-pp.perf_tp_event 0.48 ± 6% -0.4 0.10 ± 9% perf-profile.children.cycles-pp.native_sched_clock 0.48 ± 3% -0.4 0.11 ± 4% perf-profile.children.cycles-pp.___might_sleep 0.74 ± 12% -0.4 0.39 perf-profile.children.cycles-pp.kmem_cache_free 0.45 ± 8% -0.3 0.13 ± 13% perf-profile.children.cycles-pp._copy_to_user 0.36 ± 5% -0.3 0.06 ± 8% perf-profile.children.cycles-pp.__set_task_blocked 0.35 ± 22% -0.3 0.07 ± 11% perf-profile.children.cycles-pp.cpumask_next 0.38 ± 7% -0.3 0.10 ± 14% perf-profile.children.cycles-pp.ktime_get 0.46 ± 10% -0.3 0.19 ± 36% perf-profile.children.cycles-pp.__radix_tree_lookup 0.64 ± 4% -0.3 0.37 ± 6% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.33 ± 16% -0.3 0.06 ± 14% perf-profile.children.cycles-pp.cpuacct_charge 0.53 -0.3 0.26 ± 33% perf-profile.children.cycles-pp.send_signal 0.33 ± 16% -0.3 0.07 ± 12% perf-profile.children.cycles-pp.__wrgsbase_inactive 0.34 ± 8% -0.2 0.09 ± 5% perf-profile.children.cycles-pp.update_min_vruntime 0.39 ± 3% -0.2 0.15 ± 12% perf-profile.children.cycles-pp.copy_user_generic_unrolled 0.32 ± 6% -0.2 0.07 ± 6% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime 0.28 ± 2% -0.2 0.04 ± 71% perf-profile.children.cycles-pp.lock_hrtimer_base 0.30 ± 9% -0.2 0.06 ± 7% perf-profile.children.cycles-pp.set_next_buddy 0.35 ± 5% -0.2 0.11 ± 36% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 0.30 ± 5% -0.2 0.07 ± 17% perf-profile.children.cycles-pp.rb_erase 0.32 ± 2% -0.2 0.10 ± 9% perf-profile.children.cycles-pp.__x86_retpoline_rax 0.30 ± 5% -0.2 0.08 ± 14% perf-profile.children.cycles-pp.__might_sleep 0.29 ± 7% -0.2 0.07 ± 12% perf-profile.children.cycles-pp.__clear_user 0.31 ± 14% -0.2 0.09 ± 9% perf-profile.children.cycles-pp.clear_buddies 0.25 ± 3% -0.2 0.05 perf-profile.children.cycles-pp.recalc_sigpending_tsk 0.29 ± 12% -0.2 0.09 ± 22% perf-profile.children.cycles-pp.prepare_signal 0.30 ± 4% -0.2 0.11 ± 34% perf-profile.children.cycles-pp.from_kuid_munged 0.27 ± 4% -0.2 0.09 ± 36% perf-profile.children.cycles-pp.map_id_up 0.21 ± 21% -0.2 0.03 ± 70% perf-profile.children.cycles-pp.enqueue_hrtimer 0.22 ± 13% -0.2 0.06 ± 8% perf-profile.children.cycles-pp.get_timespec64 0.22 ± 17% -0.2 0.06 ± 86% perf-profile.children.cycles-pp.__cgroup_account_cputime 0.20 ± 4% -0.2 0.04 ± 76% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.20 ± 6% -0.2 0.05 perf-profile.children.cycles-pp.__list_del_entry_valid 0.19 ± 7% -0.2 0.04 ± 71% perf-profile.children.cycles-pp.perf_trace_sched_switch 0.23 ± 8% -0.1 0.08 ± 31% perf-profile.children.cycles-pp.audit_signal_info 0.22 ± 5% -0.1 0.08 ± 16% perf-profile.children.cycles-pp.rcu_read_unlock_strict 0.17 ± 9% -0.1 0.03 ± 70% perf-profile.children.cycles-pp.read_tsc 0.18 ± 7% -0.1 0.05 ± 72% perf-profile.children.cycles-pp.audit_signal_info_syscall 0.17 ± 5% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc 0.00 +0.5 0.51 ± 3% perf-profile.children.cycles-pp.dec_rlimit_ucounts 0.00 +0.7 0.67 ± 2% perf-profile.children.cycles-pp.inc_rlimit_ucounts_and_test 0.68 ± 16% +2.4 3.11 ± 53% perf-profile.children.cycles-pp.__lock_task_sighand 0.42 ± 6% +2.9 3.30 ± 51% perf-profile.children.cycles-pp._raw_spin_lock_irq 35.50 ± 7% +9.5 45.03 perf-profile.children.cycles-pp.__x64_sys_rt_sigqueueinfo 34.19 ± 7% +10.4 44.55 perf-profile.children.cycles-pp.do_rt_sigqueueinfo 33.35 ± 7% +10.9 44.22 perf-profile.children.cycles-pp.kill_pid_info 81.79 +12.5 94.33 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 19.21 ± 4% +19.4 38.65 ± 8% perf-profile.children.cycles-pp.__send_signal 21.36 ± 6% +20.4 41.78 ± 4% perf-profile.children.cycles-pp.__x64_sys_rt_sigtimedwait 20.86 ± 3% +21.5 42.34 ± 3% perf-profile.children.cycles-pp.do_send_sig_info 19.50 ± 5% +21.8 41.33 ± 4% perf-profile.children.cycles-pp.do_sigtimedwait 59.88 ± 2% +28.0 87.85 ± 2% perf-profile.children.cycles-pp.do_syscall_64 3.81 ± 8% +30.9 34.66 ± 9% perf-profile.children.cycles-pp.dequeue_signal 3.09 ± 8% +31.3 34.41 ± 9% perf-profile.children.cycles-pp.__dequeue_signal 1.74 ± 9% +32.1 33.87 ± 9% perf-profile.children.cycles-pp.__sigqueue_free 1.95 ± 8% +32.5 34.40 ± 8% perf-profile.children.cycles-pp.__sigqueue_alloc 0.00 +33.3 33.32 ± 9% perf-profile.children.cycles-pp.put_ucounts 0.00 +33.6 33.58 ± 8% perf-profile.children.cycles-pp.get_ucounts 1.07 ± 11% +68.8 69.87 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.74 ± 39% +71.3 72.04 ± 4% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 11.75 ± 2% -7.8 3.95 ± 26% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 8.02 -5.3 2.70 ± 25% perf-profile.self.cycles-pp.syscall_return_via_sysret 5.78 ± 25% -5.1 0.71 ± 43% perf-profile.self.cycles-pp.apparmor_task_kill 4.55 ± 30% -4.0 0.53 ± 34% perf-profile.self.cycles-pp.aa_get_task_label 6.02 -4.0 2.05 ± 28% perf-profile.self.cycles-pp.__entry_text_start 1.79 ± 9% -1.7 0.06 ± 13% perf-profile.self.cycles-pp.__sigqueue_alloc 2.22 ± 9% -1.7 0.50 ± 9% perf-profile.self.cycles-pp.__sched_text_start 2.25 ± 13% -1.7 0.55 ± 11% perf-profile.self.cycles-pp.load_new_mm_cr3 2.40 ± 6% -1.5 0.85 ± 6% perf-profile.self.cycles-pp.available_idle_cpu 1.57 ± 12% -1.3 0.30 ± 10% perf-profile.self.cycles-pp.switch_fpu_return 1.74 ± 6% -1.2 0.50 ± 11% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 1.79 ± 8% -1.2 0.60 ± 6% perf-profile.self.cycles-pp.select_idle_sibling 1.41 ± 7% -1.1 0.30 ± 11% perf-profile.self.cycles-pp.__switch_to 1.25 ± 3% -1.0 0.27 ± 8% perf-profile.self.cycles-pp.update_curr 1.23 ± 12% -1.0 0.27 ± 13% perf-profile.self.cycles-pp.__switch_to_asm 1.17 ± 7% -0.9 0.23 ± 9% perf-profile.self.cycles-pp.update_load_avg 1.15 ± 2% -0.9 0.21 ± 5% perf-profile.self.cycles-pp.switch_mm_irqs_off 1.13 ± 3% -0.8 0.28 ± 8% perf-profile.self.cycles-pp._raw_spin_lock 0.94 ± 13% -0.8 0.14 ± 9% perf-profile.self.cycles-pp.try_to_wake_up 1.06 -0.7 0.32 ± 28% perf-profile.self.cycles-pp.do_syscall_64 0.93 ± 12% -0.7 0.22 ± 12% perf-profile.self.cycles-pp.__update_load_avg_se 0.93 ± 33% -0.7 0.23 ± 40% perf-profile.self.cycles-pp.aa_may_signal 0.90 ± 10% -0.7 0.22 ± 9% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.77 -0.6 0.15 ± 5% perf-profile.self.cycles-pp.recalc_sigpending 0.77 ± 13% -0.6 0.20 ± 7% perf-profile.self.cycles-pp.reweight_entity 0.74 ± 7% -0.6 0.19 ± 11% perf-profile.self.cycles-pp.update_rq_clock 0.67 ± 7% -0.5 0.14 ± 6% perf-profile.self.cycles-pp.___perf_sw_event 0.63 ± 7% -0.5 0.11 ± 11% perf-profile.self.cycles-pp.enqueue_entity 0.67 ± 3% -0.5 0.15 ± 3% perf-profile.self.cycles-pp.update_cfs_group 0.66 ± 9% -0.5 0.16 ± 12% perf-profile.self.cycles-pp.pick_next_task_fair 0.61 ± 13% -0.5 0.12 ± 4% perf-profile.self.cycles-pp.enqueue_task_fair 0.60 ± 5% -0.5 0.11 ± 14% perf-profile.self.cycles-pp.dequeue_task_fair 0.62 ± 8% -0.5 0.14 ± 5% perf-profile.self.cycles-pp.select_task_rq_fair 0.63 ± 6% -0.5 0.16 ± 10% perf-profile.self.cycles-pp.do_sigtimedwait 0.61 ± 18% -0.4 0.19 ± 10% perf-profile.self.cycles-pp._find_next_bit 0.68 ± 3% -0.4 0.27 ± 33% perf-profile.self.cycles-pp.__task_pid_nr_ns 0.54 ± 3% -0.4 0.14 ± 6% perf-profile.self.cycles-pp.__dequeue_signal 0.51 -0.4 0.12 ± 20% perf-profile.self.cycles-pp.__send_signal 0.53 ± 14% -0.4 0.14 ± 15% perf-profile.self.cycles-pp.__calc_delta 0.49 ± 13% -0.4 0.12 ± 10% perf-profile.self.cycles-pp.copy_fpregs_to_fpstate 0.46 ± 7% -0.4 0.09 ± 10% perf-profile.self.cycles-pp.native_sched_clock 0.46 ± 3% -0.4 0.10 ± 4% perf-profile.self.cycles-pp.___might_sleep 0.50 ± 7% -0.3 0.16 ± 23% perf-profile.self.cycles-pp.exit_to_user_mode_prepare 0.50 ± 3% -0.3 0.17 ± 29% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.44 ± 14% -0.3 0.10 ± 16% perf-profile.self.cycles-pp.schedule 0.42 ± 5% -0.3 0.09 ± 9% perf-profile.self.cycles-pp.finish_task_switch 0.43 ± 11% -0.3 0.10 ± 9% perf-profile.self.cycles-pp.pick_next_entity 0.41 ± 10% -0.3 0.08 ± 5% perf-profile.self.cycles-pp.__x64_sys_rt_sigtimedwait 0.44 ± 6% -0.3 0.13 ± 6% perf-profile.self.cycles-pp._copy_from_user 0.50 ± 3% -0.3 0.20 ± 11% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.63 ± 20% -0.3 0.35 ± 6% perf-profile.self.cycles-pp.kmem_cache_free 0.37 ± 3% -0.3 0.10 ± 12% perf-profile.self.cycles-pp.check_preempt_wakeup 0.37 ± 7% -0.3 0.10 ± 4% perf-profile.self.cycles-pp.dequeue_entity 0.33 ± 15% -0.3 0.06 ± 13% perf-profile.self.cycles-pp.cpuacct_charge 0.46 ± 11% -0.3 0.19 ± 36% perf-profile.self.cycles-pp.__radix_tree_lookup 0.34 ± 3% -0.3 0.07 ± 11% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.29 ± 7% -0.3 0.04 ± 71% perf-profile.self.cycles-pp.perf_tp_event 0.36 ± 11% -0.3 0.10 ± 4% perf-profile.self.cycles-pp.__might_fault 0.32 ± 7% -0.2 0.08 ± 5% perf-profile.self.cycles-pp.update_min_vruntime 0.31 ± 17% -0.2 0.07 ± 12% perf-profile.self.cycles-pp.__wrgsbase_inactive 0.37 ± 2% -0.2 0.14 ± 13% perf-profile.self.cycles-pp.copy_user_generic_unrolled 0.30 ± 4% -0.2 0.07 ± 7% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime 0.26 ± 8% -0.2 0.04 ± 71% perf-profile.self.cycles-pp.set_next_buddy 0.29 ± 5% -0.2 0.07 ± 17% perf-profile.self.cycles-pp.rb_erase 0.28 ± 11% -0.2 0.06 ± 14% perf-profile.self.cycles-pp.set_next_entity 0.30 ± 4% -0.2 0.09 ± 30% perf-profile.self.cycles-pp.__x64_sys_getuid 0.39 ± 13% -0.2 0.18 ± 9% perf-profile.self.cycles-pp.cpumask_next_wrap 0.27 ± 5% -0.2 0.07 ± 17% perf-profile.self.cycles-pp.__might_sleep 0.31 ± 3% -0.2 0.12 ± 28% perf-profile.self.cycles-pp.__x64_sys_rt_sigqueueinfo 0.24 -0.2 0.05 perf-profile.self.cycles-pp.recalc_sigpending_tsk 0.28 ± 5% -0.2 0.09 ± 36% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 0.27 ± 12% -0.2 0.08 ± 24% perf-profile.self.cycles-pp.prepare_signal 0.27 ± 13% -0.2 0.08 ± 10% perf-profile.self.cycles-pp.clear_buddies 0.26 ± 3% -0.2 0.08 ± 12% perf-profile.self.cycles-pp.__x86_retpoline_rax 0.21 ± 17% -0.2 0.04 ± 71% perf-profile.self.cycles-pp.put_prev_entity 0.26 ± 4% -0.2 0.09 ± 36% perf-profile.self.cycles-pp.map_id_up 0.23 ± 13% -0.2 0.07 ± 11% perf-profile.self.cycles-pp.schedule_hrtimeout_range_clock 0.19 ± 8% -0.2 0.04 ± 70% perf-profile.self.cycles-pp.ktime_get 0.19 ± 9% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.perf_trace_sched_switch 0.18 ± 4% -0.1 0.03 ± 70% perf-profile.self.cycles-pp.__list_del_entry_valid 0.19 ± 8% -0.1 0.05 ± 78% perf-profile.self.cycles-pp.check_kill_permission 0.26 -0.1 0.13 ± 26% perf-profile.self.cycles-pp.send_signal 0.21 ± 4% -0.1 0.08 ± 26% perf-profile.self.cycles-pp.__x64_sys_getpid 0.20 ± 2% -0.1 0.06 ± 19% perf-profile.self.cycles-pp.security_task_kill 0.17 ± 4% -0.1 0.05 ± 78% perf-profile.self.cycles-pp.kill_pid_info 0.16 ± 8% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.rcu_read_unlock_strict 0.16 ± 10% -0.1 0.05 ± 72% perf-profile.self.cycles-pp.audit_signal_info_syscall 0.24 ± 8% -0.1 0.14 ± 3% perf-profile.self.cycles-pp.dequeue_signal 0.12 ± 4% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.do_rt_sigqueueinfo 0.12 ± 11% -0.1 0.04 ± 70% perf-profile.self.cycles-pp.do_send_sig_info 0.15 ± 6% -0.0 0.10 perf-profile.self.cycles-pp.kmem_cache_alloc 0.00 +0.1 0.07 ± 11% perf-profile.self.cycles-pp.get_ucounts 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.put_ucounts 0.88 ± 7% +0.2 1.04 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.00 +0.5 0.51 ± 3% perf-profile.self.cycles-pp.dec_rlimit_ucounts 0.00 +0.7 0.67 perf-profile.self.cycles-pp.inc_rlimit_ucounts_and_test 0.74 ± 39% +71.3 72.04 ± 4% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

1 year, 6 months

4
6
0 / 0

[sched/fair] b360fb5e59: stress-ng.vm-segv.ops_per_sec -13.9% regression

by kernel test robot

Greeting, FYI, we noticed a -13.9% regression of stress-ng.vm-segv.ops_per_sec due to commit: commit: b360fb5e5954a8a440ef95bf11257e2e7ea90340 ("[PATCH v2 1/7] sched/fair: Ignore percpu threads for imbalance pulls") url: https://github.com/0day-ci/linux/commits/Valentin-Schneider/sched-fair-mi... base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git c5e6fc08feb2b88dc5dac2f3c817e1c2a4cafda4 in testcase: stress-ng on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory with following parameters: nr_threads: 10% disk: 1HDD testtime: 60s fs: ext4 class: vm test: vm-segv cpufreq_governor: performance ucode: 0x5003003 If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml bin/lkp run compatible-job.yaml ========================================================================================= class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode: vm/gcc-9/performance/1HDD/ext4/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp7/vm-segv/stress-ng/60s/0x5003003 commit: c5e6fc08fe ("sched,x86: Allow !PREEMPT_DYNAMIC") b360fb5e59 ("sched/fair: Ignore percpu threads for imbalance pulls") c5e6fc08feb2b88d b360fb5e5954a8a440ef95bf112 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:6 -3% 1:6 perf-profile.children.cycles-pp.error_entry 0:6 -1% 0:6 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 11324 ± 3% -28.1% 8140 ± 3% stress-ng.time.involuntary_context_switches 6818 ± 15% +315.2% 28311 ± 12% stress-ng.time.major_page_faults 30952041 -12.8% 26988502 stress-ng.time.minor_page_faults 378.82 +5.3% 398.75 stress-ng.time.system_time 215.82 -10.0% 194.24 stress-ng.time.user_time 62102177 -13.9% 53448474 stress-ng.time.voluntary_context_switches 810348 -13.9% 698034 stress-ng.vm-segv.ops 13505 -13.9% 11633 stress-ng.vm-segv.ops_per_sec 9.76 -1.3% 9.63 iostat.cpu.system 311.06 +3.9% 323.18 pmeter.Average_Active_Power 1937515 -13.9% 1667746 vmstat.system.cs 2.33e+08 +15.2% 2.685e+08 cpuidle.C1.time 14107313 -9.8% 12722308 ± 2% cpuidle.C1.usage 51960611 -15.6% 43858596 cpuidle.POLL.usage 1.11 -0.3 0.85 mpstat.cpu.all.irq% 0.18 -0.0 0.16 mpstat.cpu.all.soft% 0.40 -0.1 0.35 mpstat.cpu.all.usr% 52529 ± 6% -35.8% 33706 ± 44% sched_debug.cfs_rq:/.min_vruntime.avg 38574 ± 10% -41.7% 22479 ± 41% sched_debug.cfs_rq:/.min_vruntime.min 624583 -56.9% 269004 ± 99% sched_debug.cpu.nr_switches.avg 421467 ± 6% -65.0% 147633 ±102% sched_debug.cpu.nr_switches.min 33017 ± 41% +71.9% 56752 ± 4% numa-vmstat.node0.nr_anon_pages 33861 ± 40% +68.8% 57163 ± 4% numa-vmstat.node0.nr_inactive_anon 33861 ± 40% +68.8% 57163 ± 4% numa-vmstat.node0.nr_zone_inactive_anon 31421 ± 44% -75.4% 7730 ± 31% numa-vmstat.node1.nr_anon_pages 32588 ± 42% -71.2% 9386 ± 25% numa-vmstat.node1.nr_inactive_anon 1949 ± 5% -15.9% 1638 ± 9% numa-vmstat.node1.nr_page_table_pages 32588 ± 42% -71.2% 9386 ± 25% numa-vmstat.node1.nr_zone_inactive_anon 7935635 ± 13% -19.3% 6407159 ± 6% numa-vmstat.node1.numa_hit 7732506 ± 14% -19.7% 6206498 ± 6% numa-vmstat.node1.numa_local 131518 ± 41% +72.1% 226297 ± 4% numa-meminfo.node0.AnonPages 140036 ± 38% +66.7% 233420 ± 4% numa-meminfo.node0.AnonPages.max 136203 ± 40% +68.2% 229070 ± 4% numa-meminfo.node0.Inactive 135279 ± 40% +68.9% 228531 ± 4% numa-meminfo.node0.Inactive(anon) 7950 ± 8% +15.4% 9178 ± 2% numa-meminfo.node0.PageTables 40320 ± 68% -79.8% 8138 ± 45% numa-meminfo.node1.AnonHugePages 125324 ± 44% -75.9% 30215 ± 32% numa-meminfo.node1.AnonPages 208760 ± 26% -48.1% 108370 ± 8% numa-meminfo.node1.AnonPages.max 129998 ± 42% -71.1% 37570 ± 25% numa-meminfo.node1.Inactive 129996 ± 42% -71.4% 37183 ± 25% numa-meminfo.node1.Inactive(anon) 7433 ± 6% -18.1% 6091 ± 9% numa-meminfo.node1.PageTables 9921 +2.0% 10121 proc-vmstat.nr_mapped 23344 -3.6% 22514 proc-vmstat.nr_slab_reclaimable 30383698 ± 2% -13.8% 26186115 proc-vmstat.numa_hit 30297093 ± 2% -13.9% 26099520 proc-vmstat.numa_local 1600 ±149% -100.0% 0.00 proc-vmstat.numa_pages_migrated 9337 ± 2% -12.5% 8174 ± 3% proc-vmstat.pgactivate 34255062 ± 2% -14.7% 29225155 proc-vmstat.pgalloc_normal 31199740 -12.6% 27277537 proc-vmstat.pgfault 34113059 ± 2% -14.7% 29083007 proc-vmstat.pgfree 1600 ±149% -100.0% 0.00 proc-vmstat.pgmigrate_success 345591 -8.1% 317680 proc-vmstat.pgreuse 11501 ± 5% +9.6% 12610 ± 6% softirqs.CPU12.RCU 10678 ± 5% +16.0% 12383 ± 2% softirqs.CPU16.RCU 10871 ± 4% +13.1% 12294 ± 2% softirqs.CPU17.RCU 10724 ± 2% +13.8% 12205 ± 3% softirqs.CPU18.RCU 10810 ± 4% +16.2% 12560 ± 3% softirqs.CPU19.RCU 10647 ± 6% +16.2% 12372 ± 6% softirqs.CPU20.RCU 10863 ± 3% +14.7% 12461 ± 3% softirqs.CPU21.RCU 11231 ± 5% +14.6% 12873 ± 6% softirqs.CPU22.RCU 11141 ± 6% +21.0% 13480 ± 8% softirqs.CPU64.RCU 11209 ± 6% +20.8% 13545 ± 2% softirqs.CPU65.RCU 11108 ± 3% +20.0% 13334 ± 6% softirqs.CPU66.RCU 11414 ± 9% +16.9% 13345 ± 6% softirqs.CPU67.RCU 11162 ± 4% +16.2% 12968 ± 9% softirqs.CPU68.RCU 11035 ± 5% +13.6% 12533 ± 4% softirqs.CPU69.RCU 11003 ± 5% +18.9% 13078 ± 8% softirqs.CPU70.RCU 11097 ± 4% +14.9% 12756 ± 5% softirqs.CPU71.RCU 76929 +14.7% 88227 slabinfo.anon_vma_chain.active_objs 1205 +14.4% 1379 slabinfo.anon_vma_chain.active_slabs 77192 +14.4% 88314 slabinfo.anon_vma_chain.num_objs 1205 +14.4% 1379 slabinfo.anon_vma_chain.num_slabs 23274 +12.2% 26121 slabinfo.kmalloc-512.active_objs 9748 ± 2% -10.0% 8776 ± 3% slabinfo.pid.active_objs 9748 ± 2% -10.0% 8776 ± 3% slabinfo.pid.num_objs 28777 -11.3% 25539 ± 2% slabinfo.proc_inode_cache.active_objs 28919 -11.3% 25647 ± 2% slabinfo.proc_inode_cache.num_objs 11256 ± 5% -23.8% 8581 ± 3% slabinfo.task_delay_info.active_objs 11256 ± 5% -23.8% 8581 ± 3% slabinfo.task_delay_info.num_objs 53293 +12.8% 60111 slabinfo.vm_area_struct.active_objs 1335 +12.6% 1503 slabinfo.vm_area_struct.active_slabs 53426 +12.6% 60156 slabinfo.vm_area_struct.num_objs 1335 +12.6% 1503 slabinfo.vm_area_struct.num_slabs 20536 ± 2% +25.3% 25732 ± 12% slabinfo.vmap_area.active_objs 24731 +23.5% 30551 ± 3% slabinfo.vmap_area.num_objs 4.799e+09 -12.2% 4.215e+09 perf-stat.i.branch-instructions 1.09 +0.1 1.14 perf-stat.i.branch-miss-rate% 50816825 -7.1% 47191966 perf-stat.i.branch-misses 4.52 ± 3% +5.6 10.16 ± 3% perf-stat.i.cache-miss-rate% 18266157 ± 2% +125.2% 41128630 ± 3% perf-stat.i.cache-misses 4.817e+08 -11.6% 4.258e+08 perf-stat.i.cache-references 1998506 -14.0% 1718841 perf-stat.i.context-switches 1.89 +12.4% 2.12 perf-stat.i.cpi 4.232e+10 -1.4% 4.175e+10 perf-stat.i.cpu-cycles 836.21 ± 4% +14.8% 959.82 ± 7% perf-stat.i.cpu-migrations 2507 -54.0% 1153 ± 5% perf-stat.i.cycles-between-cache-misses 4212845 ± 3% -10.4% 3775244 ± 2% perf-stat.i.dTLB-load-misses 6.137e+09 -12.7% 5.359e+09 perf-stat.i.dTLB-loads 0.03 ± 3% +0.0 0.04 ± 2% perf-stat.i.dTLB-store-miss-rate% 1127230 ± 3% -6.7% 1051240 ± 2% perf-stat.i.dTLB-store-misses 3.192e+09 -13.4% 2.766e+09 perf-stat.i.dTLB-stores 67.76 -0.7 67.08 perf-stat.i.iTLB-load-miss-rate% 17023777 -10.2% 15291627 perf-stat.i.iTLB-load-misses 7885583 -7.2% 7319880 perf-stat.i.iTLB-loads 2.237e+10 -12.5% 1.956e+10 perf-stat.i.instructions 0.53 -10.8% 0.48 perf-stat.i.ipc 111.04 ± 13% +310.1% 455.43 ± 11% perf-stat.i.major-faults 0.44 -1.4% 0.43 perf-stat.i.metric.GHz 0.06 ± 12% +21.9% 0.07 ± 14% perf-stat.i.metric.K/sec 152.39 -12.5% 133.27 perf-stat.i.metric.M/sec 494626 -12.8% 431405 perf-stat.i.minor-faults 77.46 +12.6 90.07 perf-stat.i.node-load-miss-rate% 3962536 ± 4% +195.0% 11688629 ± 3% perf-stat.i.node-load-misses 1000403 +7.6% 1076834 ± 2% perf-stat.i.node-loads 41.34 ± 2% +31.6 72.93 perf-stat.i.node-store-miss-rate% 1494039 ± 5% +198.5% 4459287 ± 3% perf-stat.i.node-store-misses 1975747 ± 3% -25.8% 1466690 ± 2% perf-stat.i.node-stores 520459 -12.8% 453999 perf-stat.i.page-faults 1.06 +0.1 1.12 perf-stat.overall.branch-miss-rate% 3.78 ± 2% +5.8 9.60 ± 3% perf-stat.overall.cache-miss-rate% 1.89 +12.7% 2.13 perf-stat.overall.cpi 2324 ± 2% -56.1% 1021 ± 3% perf-stat.overall.cycles-between-cache-misses 0.04 ± 3% +0.0 0.04 ± 2% perf-stat.overall.dTLB-store-miss-rate% 68.35 -0.7 67.65 perf-stat.overall.iTLB-load-miss-rate% 0.53 -11.2% 0.47 perf-stat.overall.ipc 79.66 +11.8 91.48 perf-stat.overall.node-load-miss-rate% 42.93 ± 3% +32.1 75.07 perf-stat.overall.node-store-miss-rate% 4.727e+09 -12.1% 4.154e+09 perf-stat.ps.branch-instructions 50025626 -7.1% 46471888 perf-stat.ps.branch-misses 17939323 ± 2% +124.6% 40288410 ± 3% perf-stat.ps.cache-misses 4.743e+08 -11.5% 4.196e+08 perf-stat.ps.cache-references 1968071 -13.9% 1694380 perf-stat.ps.context-switches 4.167e+10 -1.4% 4.11e+10 perf-stat.ps.cpu-cycles 821.15 ± 4% +14.8% 942.91 ± 7% perf-stat.ps.cpu-migrations 4147982 ± 3% -10.3% 3720585 ± 2% perf-stat.ps.dTLB-load-misses 6.043e+09 -12.6% 5.282e+09 perf-stat.ps.dTLB-loads 1109704 ± 3% -6.6% 1036060 ± 2% perf-stat.ps.dTLB-store-misses 3.144e+09 -13.3% 2.726e+09 perf-stat.ps.dTLB-stores 16765267 -10.1% 15068821 perf-stat.ps.iTLB-load-misses 7761893 -7.1% 7207514 perf-stat.ps.iTLB-loads 2.203e+10 -12.5% 1.928e+10 perf-stat.ps.instructions 108.63 ± 13% +309.6% 444.94 ± 12% perf-stat.ps.major-faults 487068 -12.7% 425183 perf-stat.ps.minor-faults 3881430 ± 4% +194.6% 11435790 ± 3% perf-stat.ps.node-load-misses 989805 +7.5% 1064249 ± 2% perf-stat.ps.node-loads 1464554 ± 5% +198.0% 4363838 ± 3% perf-stat.ps.node-store-misses 1945831 ± 3% -25.6% 1448410 ± 2% perf-stat.ps.node-stores 512507 -12.7% 447452 perf-stat.ps.page-faults 1.389e+12 -12.5% 1.215e+12 perf-stat.total.instructions 239831 -33.6% 159201 interrupts.CAL:Function_call_interrupts 2858 ± 21% -36.1% 1825 ± 17% interrupts.CPU0.CAL:Function_call_interrupts 76.83 ± 14% -73.1% 20.67 ± 24% interrupts.CPU0.TLB:TLB_shootdowns 3447 ± 10% -38.8% 2108 ± 18% interrupts.CPU1.CAL:Function_call_interrupts 2457 ± 16% -30.3% 1712 ± 15% interrupts.CPU10.CAL:Function_call_interrupts 2252 ± 16% -29.6% 1585 ± 22% interrupts.CPU11.CAL:Function_call_interrupts 2156 ± 23% -27.9% 1555 ± 13% interrupts.CPU12.CAL:Function_call_interrupts 378.33 ± 11% +42.1% 537.50 ± 19% interrupts.CPU13.RES:Rescheduling_interrupts 2358 ± 15% -40.2% 1409 ± 8% interrupts.CPU14.CAL:Function_call_interrupts 2216 ± 20% -36.0% 1418 ± 10% interrupts.CPU15.CAL:Function_call_interrupts 2387 ± 15% -24.2% 1811 ± 5% interrupts.CPU16.CAL:Function_call_interrupts 2329 ± 14% -36.0% 1491 ± 14% interrupts.CPU17.CAL:Function_call_interrupts 2509 ± 13% -39.2% 1525 ± 15% interrupts.CPU18.CAL:Function_call_interrupts 2333 ± 19% -32.9% 1566 ± 18% interrupts.CPU19.CAL:Function_call_interrupts 3082 ± 15% -36.0% 1972 ± 23% interrupts.CPU2.CAL:Function_call_interrupts 2019 ± 6% -24.8% 1518 ± 17% interrupts.CPU2.NMI:Non-maskable_interrupts 2019 ± 6% -24.8% 1518 ± 17% interrupts.CPU2.PMI:Performance_monitoring_interrupts 2456 ± 11% -36.1% 1569 ± 11% interrupts.CPU21.CAL:Function_call_interrupts 2668 ± 21% -37.7% 1662 ± 11% interrupts.CPU22.CAL:Function_call_interrupts 2361 ± 15% -34.6% 1543 ± 15% interrupts.CPU23.CAL:Function_call_interrupts 2651 ± 8% -22.4% 2057 ± 17% interrupts.CPU24.CAL:Function_call_interrupts 2936 ± 10% -42.9% 1675 ± 20% interrupts.CPU25.CAL:Function_call_interrupts 2734 ± 22% -37.4% 1712 ± 24% interrupts.CPU26.CAL:Function_call_interrupts 2614 ± 19% -34.1% 1723 ± 18% interrupts.CPU27.CAL:Function_call_interrupts 2774 ± 13% -46.3% 1490 ± 21% interrupts.CPU28.CAL:Function_call_interrupts 2454 ± 18% -44.0% 1373 ± 18% interrupts.CPU29.CAL:Function_call_interrupts 433.83 ± 16% -30.7% 300.67 ± 13% interrupts.CPU29.RES:Rescheduling_interrupts 2969 ± 19% -40.4% 1769 ± 15% interrupts.CPU3.CAL:Function_call_interrupts 2566 ± 10% -32.4% 1734 ± 13% interrupts.CPU30.CAL:Function_call_interrupts 2614 ± 16% -43.6% 1475 ± 26% interrupts.CPU31.CAL:Function_call_interrupts 2212 ± 11% -34.4% 1452 ± 11% interrupts.CPU32.CAL:Function_call_interrupts 2273 ± 16% -39.6% 1372 ± 23% interrupts.CPU33.CAL:Function_call_interrupts 2313 ± 20% -34.2% 1521 ± 18% interrupts.CPU36.CAL:Function_call_interrupts 2258 ± 19% -31.3% 1550 ± 25% interrupts.CPU38.CAL:Function_call_interrupts 2482 ± 16% -29.8% 1743 ± 14% interrupts.CPU4.CAL:Function_call_interrupts 2466 ± 16% -40.9% 1458 ± 23% interrupts.CPU40.CAL:Function_call_interrupts 2488 ± 21% -42.9% 1420 ± 21% interrupts.CPU41.CAL:Function_call_interrupts 2327 ± 16% -39.7% 1403 ± 19% interrupts.CPU42.CAL:Function_call_interrupts 2352 ± 16% -40.0% 1412 ± 20% interrupts.CPU44.CAL:Function_call_interrupts 2252 ± 15% -41.0% 1329 ± 26% interrupts.CPU46.CAL:Function_call_interrupts 2490 ± 13% -40.1% 1491 ± 23% interrupts.CPU47.CAL:Function_call_interrupts 2628 ± 13% -32.4% 1777 ± 17% interrupts.CPU48.CAL:Function_call_interrupts 2752 ± 14% -35.0% 1788 ± 12% interrupts.CPU49.CAL:Function_call_interrupts 2668 ± 15% -39.8% 1605 ± 20% interrupts.CPU5.CAL:Function_call_interrupts 2693 ± 12% -35.1% 1749 ± 15% interrupts.CPU50.CAL:Function_call_interrupts 2515 ± 17% -35.9% 1613 ± 22% interrupts.CPU51.CAL:Function_call_interrupts 2809 ± 16% -45.4% 1534 ± 14% interrupts.CPU52.CAL:Function_call_interrupts 2748 ± 11% -35.2% 1780 ± 22% interrupts.CPU53.CAL:Function_call_interrupts 2634 ± 12% -34.1% 1736 ± 9% interrupts.CPU54.CAL:Function_call_interrupts 2607 ± 11% -38.0% 1616 ± 9% interrupts.CPU55.CAL:Function_call_interrupts 2527 ± 19% -38.3% 1558 ± 13% interrupts.CPU57.CAL:Function_call_interrupts 2620 ± 12% -22.3% 2037 ± 15% interrupts.CPU58.CAL:Function_call_interrupts 2569 ± 10% -30.1% 1795 ± 16% interrupts.CPU61.CAL:Function_call_interrupts 2514 ± 8% -36.3% 1601 ± 25% interrupts.CPU62.CAL:Function_call_interrupts 2525 ± 16% -31.9% 1719 ± 25% interrupts.CPU63.CAL:Function_call_interrupts 2494 ± 8% -24.6% 1880 ± 11% interrupts.CPU64.CAL:Function_call_interrupts 506.50 ± 15% +26.9% 642.67 ± 9% interrupts.CPU65.RES:Rescheduling_interrupts 486.67 ± 9% +29.5% 630.33 ± 15% interrupts.CPU66.RES:Rescheduling_interrupts 120.00 ±126% -92.4% 9.17 ± 36% interrupts.CPU66.TLB:TLB_shootdowns 2444 ± 12% -21.9% 1908 ± 18% interrupts.CPU67.CAL:Function_call_interrupts 2403 ± 16% -32.6% 1620 ± 15% interrupts.CPU69.CAL:Function_call_interrupts 2673 ± 11% -37.2% 1679 ± 24% interrupts.CPU7.CAL:Function_call_interrupts 2360 ± 12% -27.0% 1723 ± 14% interrupts.CPU70.CAL:Function_call_interrupts 2408 ± 11% -31.9% 1639 ± 16% interrupts.CPU71.CAL:Function_call_interrupts 2591 ± 13% -36.9% 1635 ± 24% interrupts.CPU75.CAL:Function_call_interrupts 2793 ± 13% -39.7% 1685 ± 17% interrupts.CPU76.CAL:Function_call_interrupts 2662 ± 14% -37.3% 1668 ± 20% interrupts.CPU77.CAL:Function_call_interrupts 1899 ± 8% -25.9% 1408 ± 33% interrupts.CPU77.NMI:Non-maskable_interrupts 1899 ± 8% -25.9% 1408 ± 33% interrupts.CPU77.PMI:Performance_monitoring_interrupts 2624 ± 10% -35.4% 1694 ± 27% interrupts.CPU78.CAL:Function_call_interrupts 2816 ± 12% -35.0% 1829 ± 25% interrupts.CPU79.CAL:Function_call_interrupts 2393 ± 10% -36.2% 1525 ± 11% interrupts.CPU8.CAL:Function_call_interrupts 2465 ± 16% -33.6% 1637 ± 18% interrupts.CPU80.CAL:Function_call_interrupts 2430 ± 16% -41.1% 1432 ± 9% interrupts.CPU81.CAL:Function_call_interrupts 2369 ± 7% -25.5% 1765 ± 14% interrupts.CPU83.CAL:Function_call_interrupts 2509 ± 20% -40.1% 1504 ± 19% interrupts.CPU84.CAL:Function_call_interrupts 2624 ± 20% -34.7% 1713 ± 26% interrupts.CPU85.CAL:Function_call_interrupts 2329 ± 15% -29.0% 1654 ± 21% interrupts.CPU86.CAL:Function_call_interrupts 2599 ± 14% -39.2% 1580 ± 17% interrupts.CPU88.CAL:Function_call_interrupts 2413 ± 13% -39.0% 1471 ± 12% interrupts.CPU9.CAL:Function_call_interrupts 2492 ± 19% -36.1% 1592 ± 30% interrupts.CPU90.CAL:Function_call_interrupts 2325 ± 21% -32.6% 1567 ± 19% interrupts.CPU91.CAL:Function_call_interrupts 2557 ± 13% -42.3% 1474 ± 19% interrupts.CPU92.CAL:Function_call_interrupts 2576 ± 19% -45.2% 1412 ± 19% interrupts.CPU94.CAL:Function_call_interrupts 2571 ± 10% -46.4% 1377 ± 20% interrupts.CPU95.CAL:Function_call_interrupts 475.17 ± 15% -22.4% 368.50 ± 13% interrupts.CPU95.RES:Rescheduling_interrupts 5869 ± 2% -83.2% 988.50 ± 12% interrupts.TLB:TLB_shootdowns 2.74 ± 6% -0.3 2.40 ± 3% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 2.66 ± 6% -0.3 2.33 ± 3% perf-profile.calltrace.cycles-pp.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry.start_secondary 1.88 ± 4% -0.2 1.64 ± 4% perf-profile.calltrace.cycles-pp.ptrace_request.arch_ptrace.__x64_sys_ptrace.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.89 ± 4% -0.2 1.65 ± 4% perf-profile.calltrace.cycles-pp.arch_ptrace.__x64_sys_ptrace.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.02 ± 5% -0.2 1.79 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.do_notify_parent_cldstop.ptrace_stop.ptrace_do_notify.ptrace_notify 1.87 ± 4% -0.2 1.64 ± 4% perf-profile.calltrace.cycles-pp.ptrace_resume.ptrace_request.arch_ptrace.__x64_sys_ptrace.do_syscall_64 1.90 ± 6% -0.2 1.67 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.do_notify_parent_cldstop.ptrace_stop.ptrace_do_notify 1.73 ± 4% -0.2 1.50 ± 4% perf-profile.calltrace.cycles-pp.try_to_wake_up.ptrace_resume.ptrace_request.arch_ptrace.__x64_sys_ptrace 1.46 ± 6% -0.2 1.25 ± 4% perf-profile.calltrace.cycles-pp.schedule.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 1.42 ± 6% -0.2 1.22 ± 4% perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.do_wait.kernel_wait4.__do_sys_wait4 0.63 ± 5% -0.2 0.46 ± 44% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.ptrace_resume 1.25 ± 6% -0.2 1.09 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry 1.19 ± 5% -0.1 1.05 ± 4% perf-profile.calltrace.cycles-pp.do_notify_parent_cldstop.ptrace_stop.ptrace_do_notify.ptrace_notify.syscall_trace_enter 0.75 ± 7% -0.1 0.64 ± 3% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__sched_text_start.schedule.do_wait.kernel_wait4 0.83 ± 4% -0.1 0.73 ± 5% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.ptrace_resume.ptrace_request 0.67 ± 7% -0.1 0.57 ± 4% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule.do_wait 0.86 ± 4% -0.1 0.76 ± 4% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.ptrace_resume.ptrace_request.arch_ptrace 0.61 ± 6% +0.1 0.69 ± 4% perf-profile.calltrace.cycles-pp.kmem_cache_free.remove_vma.__do_munmap.__vm_munmap.__x64_sys_munmap 0.78 ± 6% +0.1 0.88 ± 3% perf-profile.calltrace.cycles-pp.remove_vma.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 0.67 ± 8% +0.1 0.78 ± 4% perf-profile.calltrace.cycles-pp.anon_vma_interval_tree_insert.anon_vma_clone.anon_vma_fork.dup_mmap.dup_mm 0.58 ± 4% +0.1 0.69 ± 6% perf-profile.calltrace.cycles-pp.queued_read_lock_slowpath.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 1.09 ± 5% +0.1 1.22 ± 4% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap 1.35 ± 5% +0.2 1.53 ± 4% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap 1.36 ± 5% +0.2 1.54 ± 4% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 0.79 ± 5% +0.2 0.99 ± 4% perf-profile.calltrace.cycles-pp.ptrace_check_attach.__x64_sys_ptrace.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.00 ± 6% +0.2 2.24 ± 3% perf-profile.calltrace.cycles-pp.anon_vma_clone.anon_vma_fork.dup_mmap.dup_mm.copy_process 0.68 ± 7% +0.3 0.94 ± 6% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.67 ± 7% +0.3 0.93 ± 6% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.69 ± 7% +0.3 0.94 ± 6% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 2.31 ± 6% +0.3 2.57 ± 3% perf-profile.calltrace.cycles-pp.filemap_map_pages.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault 2.42 ± 6% +0.3 2.68 ± 3% perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault 1.24 ± 6% +0.3 1.58 ± 7% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 0.18 ±141% +0.4 0.55 ± 6% perf-profile.calltrace.cycles-pp.__vmalloc_node_range.dup_task_struct.copy_process.kernel_clone.__do_sys_clone 1.36 ± 6% +0.4 1.74 ± 6% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 1.94 ± 6% +0.4 2.36 ± 4% perf-profile.calltrace.cycles-pp.unlink_anon_vmas.free_pgtables.unmap_region.__do_munmap.__vm_munmap 0.26 ±100% +0.5 0.73 ± 9% perf-profile.calltrace.cycles-pp.unlink_file_vma.free_pgtables.unmap_region.__do_munmap.__vm_munmap 0.00 +0.6 0.58 ± 4% perf-profile.calltrace.cycles-pp.wait_task_inactive.ptrace_check_attach.__x64_sys_ptrace.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.69 ± 5% +0.6 3.33 ± 3% perf-profile.calltrace.cycles-pp.free_pgtables.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 5.68 ± 5% -0.6 5.06 ± 3% perf-profile.children.cycles-pp.__sched_text_start 3.69 ± 5% -0.5 3.22 ± 3% perf-profile.children.cycles-pp.try_to_wake_up 2.79 ± 6% -0.4 2.42 ± 3% perf-profile.children.cycles-pp.schedule_idle 2.49 ± 5% -0.3 2.21 ± 4% perf-profile.children.cycles-pp.do_notify_parent_cldstop 2.23 ± 5% -0.3 1.96 ± 3% perf-profile.children.cycles-pp.__wake_up_common_lock 2.08 ± 5% -0.3 1.83 ± 3% perf-profile.children.cycles-pp.__wake_up_common 1.89 ± 4% -0.2 1.65 ± 4% perf-profile.children.cycles-pp.arch_ptrace 1.89 ± 4% -0.2 1.64 ± 4% perf-profile.children.cycles-pp.ptrace_request 1.87 ± 4% -0.2 1.64 ± 4% perf-profile.children.cycles-pp.ptrace_resume 1.68 ± 5% -0.2 1.47 ± 4% perf-profile.children.cycles-pp.enqueue_task_fair 1.70 ± 5% -0.2 1.50 ± 4% perf-profile.children.cycles-pp.ttwu_do_activate 1.33 ± 5% -0.2 1.14 ± 4% perf-profile.children.cycles-pp.enqueue_entity 1.45 ± 5% -0.2 1.26 ± 3% perf-profile.children.cycles-pp.pick_next_task_fair 1.47 ± 5% -0.2 1.30 ± 3% perf-profile.children.cycles-pp.dequeue_task_fair 1.17 ± 5% -0.2 1.02 ± 3% perf-profile.children.cycles-pp.update_load_avg 1.33 ± 5% -0.2 1.18 ± 2% perf-profile.children.cycles-pp.dequeue_entity 0.84 ± 7% -0.1 0.73 ± 4% perf-profile.children.cycles-pp.get_page_from_freelist 0.52 ± 8% -0.1 0.43 ± 3% perf-profile.children.cycles-pp.page_add_file_rmap 0.75 ± 4% -0.1 0.66 ± 3% perf-profile.children.cycles-pp.select_task_rq_fair 0.65 ± 4% -0.1 0.57 ± 3% perf-profile.children.cycles-pp.update_curr 0.52 ± 6% -0.1 0.44 ± 6% perf-profile.children.cycles-pp.___might_sleep 0.39 ± 5% -0.1 0.33 ± 5% perf-profile.children.cycles-pp.__switch_to_asm 0.50 ± 5% -0.1 0.44 ± 3% perf-profile.children.cycles-pp.wake_up_new_task 0.46 ± 5% -0.1 0.41 ± 5% perf-profile.children.cycles-pp.wp_page_copy 0.41 ± 4% -0.0 0.36 ± 2% perf-profile.children.cycles-pp.find_idlest_group 0.38 ± 6% -0.0 0.33 ± 3% perf-profile.children.cycles-pp.tick_nohz_idle_exit 0.39 ± 6% -0.0 0.34 ± 5% perf-profile.children.cycles-pp.sched_clock 0.37 ± 6% -0.0 0.33 ± 6% perf-profile.children.cycles-pp.native_sched_clock 0.28 ± 7% -0.0 0.24 ± 6% perf-profile.children.cycles-pp.switch_mm_irqs_off 0.31 ± 3% -0.0 0.28 ± 3% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template 0.23 ± 6% -0.0 0.19 ± 6% perf-profile.children.cycles-pp.rmqueue 0.18 ± 5% -0.0 0.14 ± 5% perf-profile.children.cycles-pp.tick_nohz_idle_enter 0.16 ± 4% -0.0 0.13 ± 5% perf-profile.children.cycles-pp.__cond_resched 0.23 ± 5% -0.0 0.20 ± 6% perf-profile.children.cycles-pp.reweight_entity 0.07 ± 11% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.perf_iterate_sb 0.19 ± 5% -0.0 0.16 ± 4% perf-profile.children.cycles-pp.allocate_slab 0.26 ± 5% -0.0 0.23 ± 6% perf-profile.children.cycles-pp.update_ts_time_stats 0.14 ± 7% -0.0 0.11 ± 9% perf-profile.children.cycles-pp.free_pcppages_bulk 0.17 ± 4% -0.0 0.14 ± 4% perf-profile.children.cycles-pp.___perf_sw_event 0.20 ± 11% -0.0 0.17 ± 7% perf-profile.children.cycles-pp.sync_regs 0.15 ± 5% -0.0 0.12 ± 7% perf-profile.children.cycles-pp.free_unref_page_list 0.19 ± 9% -0.0 0.17 ± 6% perf-profile.children.cycles-pp.__put_user_nocheck_4 0.09 ± 7% -0.0 0.07 ± 11% perf-profile.children.cycles-pp.memcpy_erms 0.24 ± 4% -0.0 0.21 ± 4% perf-profile.children.cycles-pp.__task_pid_nr_ns 0.07 ± 6% -0.0 0.05 perf-profile.children.cycles-pp.rcu_eqs_enter 0.15 ± 4% -0.0 0.13 ± 5% perf-profile.children.cycles-pp.place_entity 0.07 ± 10% -0.0 0.05 ± 45% perf-profile.children.cycles-pp.perf_event_task 0.12 ± 5% -0.0 0.10 ± 6% perf-profile.children.cycles-pp.ttwu_queue_wakelist 0.08 ± 6% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.arch_dup_task_struct 0.09 ± 8% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.call_cpuidle 0.07 ± 7% -0.0 0.05 ± 45% perf-profile.children.cycles-pp.lru_add_drain 0.14 ± 5% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.copy_page 0.07 ± 6% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.rcu_all_qs 0.07 ± 7% -0.0 0.05 ± 7% perf-profile.children.cycles-pp.unfreeze_partials 0.09 ± 8% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.__calc_delta 0.08 ± 14% +0.0 0.11 ± 11% perf-profile.children.cycles-pp.userfaultfd_unmap_prep 0.06 ± 11% +0.0 0.09 ± 10% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.14 ± 11% +0.0 0.18 ± 9% perf-profile.children.cycles-pp.__memcg_kmem_uncharge 0.33 ± 5% +0.0 0.37 ± 6% perf-profile.children.cycles-pp.__put_anon_vma 0.25 ± 5% +0.0 0.29 ± 6% perf-profile.children.cycles-pp.free_pages_and_swap_cache 0.18 ± 7% +0.0 0.22 ± 11% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.12 ± 10% +0.1 0.17 ± 14% perf-profile.children.cycles-pp.tick_nohz_next_event 0.02 ±141% +0.1 0.07 ± 13% perf-profile.children.cycles-pp.lock_page_lruvec_irqsave 0.16 ± 9% +0.1 0.21 ± 8% perf-profile.children.cycles-pp.__rb_erase_color 0.19 ± 11% +0.1 0.24 ± 9% perf-profile.children.cycles-pp.page_counter_uncharge 0.18 ± 9% +0.1 0.24 ± 9% perf-profile.children.cycles-pp.page_counter_cancel 0.26 ± 10% +0.1 0.32 ± 3% perf-profile.children.cycles-pp.find_get_task_by_vpid 0.49 ± 8% +0.1 0.55 ± 6% perf-profile.children.cycles-pp.__vmalloc_node_range 0.00 +0.1 0.06 ± 14% perf-profile.children.cycles-pp.calc_global_load_tick 0.21 ± 6% +0.1 0.28 ± 8% perf-profile.children.cycles-pp.drain_obj_stock 0.58 ± 4% +0.1 0.65 ± 3% perf-profile.children.cycles-pp.irq_exit_rcu 0.52 ± 5% +0.1 0.59 ± 2% perf-profile.children.cycles-pp.do_softirq_own_stack 0.16 ± 7% +0.1 0.23 ± 7% perf-profile.children.cycles-pp.scheduler_tick 0.00 +0.1 0.07 ± 18% perf-profile.children.cycles-pp.timekeeping_max_deferment 0.44 ± 5% +0.1 0.51 ± 9% perf-profile.children.cycles-pp.page_counter_try_charge 0.59 ± 5% +0.1 0.67 perf-profile.children.cycles-pp.__softirqentry_text_start 0.38 ± 5% +0.1 0.46 ± 4% perf-profile.children.cycles-pp._raw_read_lock 0.21 ± 13% +0.1 0.28 ± 12% perf-profile.children.cycles-pp.queued_write_lock_slowpath 0.28 ± 5% +0.1 0.37 ± 12% perf-profile.children.cycles-pp.vma_interval_tree_remove 0.34 ± 5% +0.1 0.43 ± 4% perf-profile.children.cycles-pp.anon_vma_interval_tree_remove 0.79 ± 6% +0.1 0.88 ± 3% perf-profile.children.cycles-pp.remove_vma 0.28 ± 13% +0.1 0.38 ± 7% perf-profile.children.cycles-pp.alloc_vmap_area 0.51 ± 6% +0.1 0.60 ± 6% perf-profile.children.cycles-pp.refill_obj_stock 0.29 ± 14% +0.1 0.39 ± 6% perf-profile.children.cycles-pp.__get_vm_area_node 0.28 ± 7% +0.1 0.38 ± 8% perf-profile.children.cycles-pp.tick_sched_handle 0.27 ± 7% +0.1 0.37 ± 8% perf-profile.children.cycles-pp.update_process_times 0.33 ± 7% +0.1 0.45 ± 7% perf-profile.children.cycles-pp.tick_sched_timer 0.67 ± 7% +0.1 0.79 ± 5% perf-profile.children.cycles-pp.anon_vma_interval_tree_insert 0.62 ± 6% +0.1 0.74 ± 4% perf-profile.children.cycles-pp.down_write 0.46 ± 6% +0.1 0.58 ± 4% perf-profile.children.cycles-pp.wait_task_inactive 0.48 ± 5% +0.1 0.61 ± 6% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.28 ± 6% +0.1 0.41 ± 7% perf-profile.children.cycles-pp.clockevents_program_event 1.11 ± 5% +0.1 1.25 ± 4% perf-profile.children.cycles-pp.release_pages 0.62 ± 4% +0.1 0.77 ± 4% perf-profile.children.cycles-pp.ktime_get 0.09 ± 24% +0.2 0.26 ± 11% perf-profile.children.cycles-pp.osq_lock 1.37 ± 5% +0.2 1.54 ± 4% perf-profile.children.cycles-pp.tlb_finish_mmu 1.36 ± 5% +0.2 1.54 ± 4% perf-profile.children.cycles-pp.tlb_flush_mmu 0.79 ± 5% +0.2 0.99 ± 4% perf-profile.children.cycles-pp.ptrace_check_attach 0.51 ± 4% +0.2 0.73 ± 9% perf-profile.children.cycles-pp.unlink_file_vma 2.00 ± 6% +0.2 2.24 ± 3% perf-profile.children.cycles-pp.anon_vma_clone 2.33 ± 6% +0.3 2.59 ± 3% perf-profile.children.cycles-pp.filemap_map_pages 2.43 ± 6% +0.3 2.69 ± 3% perf-profile.children.cycles-pp.do_fault 1.00 ± 5% +0.3 1.27 ± 6% perf-profile.children.cycles-pp.queued_read_lock_slowpath 0.22 ± 12% +0.3 0.49 ± 5% perf-profile.children.cycles-pp.cgroup_enter_frozen 0.90 ± 4% +0.3 1.18 ± 4% perf-profile.children.cycles-pp.hrtimer_interrupt 0.91 ± 4% +0.3 1.19 ± 4% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.26 ± 5% +0.3 0.57 ± 5% perf-profile.children.cycles-pp.cgroup_leave_frozen 0.19 ± 12% +0.3 0.51 ± 7% perf-profile.children.cycles-pp.rwsem_spin_on_owner 1.47 ± 3% +0.3 1.82 ± 3% perf-profile.children.cycles-pp.asm_call_sysvec_on_stack 1.73 ± 3% +0.4 2.12 ± 4% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 1.95 ± 6% +0.4 2.38 ± 3% perf-profile.children.cycles-pp.unlink_anon_vmas 1.88 ± 3% +0.4 2.32 ± 3% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.83 ± 6% +0.6 1.42 ± 4% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.35 ± 13% +0.6 0.96 ± 6% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 2.69 ± 5% +0.6 3.34 ± 3% perf-profile.children.cycles-pp.free_pgtables 1.23 ± 8% +0.8 2.06 ± 5% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 1.15 ± 5% -0.1 1.03 ± 4% perf-profile.self.cycles-pp.__sched_text_start 0.48 ± 8% -0.1 0.38 ± 4% perf-profile.self.cycles-pp.page_add_file_rmap 0.55 ± 5% -0.1 0.45 ± 5% perf-profile.self.cycles-pp.update_load_avg 0.71 ± 6% -0.1 0.62 ± 4% perf-profile.self.cycles-pp.update_rq_clock 0.51 ± 6% -0.1 0.43 ± 6% perf-profile.self.cycles-pp.___might_sleep 0.54 ± 3% -0.1 0.48 ± 5% perf-profile.self.cycles-pp.do_idle 0.39 ± 6% -0.1 0.33 ± 5% perf-profile.self.cycles-pp.__switch_to_asm 0.31 ± 5% -0.1 0.26 ± 4% perf-profile.self.cycles-pp.update_curr 0.24 ± 7% -0.0 0.20 ± 7% perf-profile.self.cycles-pp.obj_cgroup_charge 0.36 ± 4% -0.0 0.32 ± 3% perf-profile.self.cycles-pp.find_idlest_group 0.23 ± 5% -0.0 0.20 ± 6% perf-profile.self.cycles-pp.reweight_entity 0.22 ± 4% -0.0 0.19 ± 4% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.20 ± 10% -0.0 0.17 ± 7% perf-profile.self.cycles-pp.sync_regs 0.12 ± 11% -0.0 0.09 ± 11% perf-profile.self.cycles-pp.try_charge 0.09 ± 7% -0.0 0.06 ± 11% perf-profile.self.cycles-pp.memcpy_erms 0.36 ± 2% -0.0 0.34 ± 3% perf-profile.self.cycles-pp.native_irq_return_iret 0.23 ± 3% -0.0 0.20 ± 4% perf-profile.self.cycles-pp.__task_pid_nr_ns 0.12 ± 6% -0.0 0.09 ± 11% perf-profile.self.cycles-pp.ttwu_queue_wakelist 0.13 ± 5% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.___perf_sw_event 0.16 ± 7% -0.0 0.14 ± 5% perf-profile.self.cycles-pp.dequeue_entity 0.14 ± 4% -0.0 0.12 ± 6% perf-profile.self.cycles-pp.__wake_up_common 0.14 ± 6% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.copy_page 0.08 ± 5% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.call_cpuidle 0.09 ± 8% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.__calc_delta 0.07 ± 7% +0.0 0.08 perf-profile.self.cycles-pp.alloc_vmap_area 0.06 ± 7% +0.0 0.08 ± 10% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.07 ± 14% +0.0 0.11 ± 8% perf-profile.self.cycles-pp.userfaultfd_unmap_prep 0.08 ± 9% +0.0 0.11 ± 5% perf-profile.self.cycles-pp.drain_obj_stock 0.15 ± 8% +0.0 0.19 ± 5% perf-profile.self.cycles-pp.unlink_anon_vmas 0.16 ± 9% +0.0 0.21 ± 9% perf-profile.self.cycles-pp.page_counter_cancel 0.12 ± 11% +0.1 0.18 ± 11% perf-profile.self.cycles-pp.__rb_erase_color 0.27 ± 5% +0.1 0.33 ± 6% perf-profile.self.cycles-pp.cpuidle_enter_state 0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.cgroup_enter_frozen 0.15 ± 8% +0.1 0.21 ± 6% perf-profile.self.cycles-pp.find_get_task_by_vpid 0.00 +0.1 0.06 ± 14% perf-profile.self.cycles-pp.calc_global_load_tick 0.00 +0.1 0.07 ± 18% perf-profile.self.cycles-pp.timekeeping_max_deferment 0.38 ± 6% +0.1 0.45 ± 3% perf-profile.self.cycles-pp._raw_read_lock 0.06 ± 11% +0.1 0.15 ± 7% perf-profile.self.cycles-pp.rwsem_down_write_slowpath 0.33 ± 5% +0.1 0.43 ± 4% perf-profile.self.cycles-pp.anon_vma_interval_tree_remove 0.27 ± 4% +0.1 0.37 ± 12% perf-profile.self.cycles-pp.vma_interval_tree_remove 0.81 ± 4% +0.1 0.92 ± 4% perf-profile.self.cycles-pp.kmem_cache_free 0.67 ± 8% +0.1 0.78 ± 4% perf-profile.self.cycles-pp.anon_vma_interval_tree_insert 0.31 ± 8% +0.1 0.43 ± 5% perf-profile.self.cycles-pp.ptrace_stop 0.85 ± 6% +0.1 0.97 ± 4% perf-profile.self.cycles-pp.release_pages 0.49 ± 6% +0.1 0.62 ± 5% perf-profile.self.cycles-pp.down_write 0.63 ± 6% +0.1 0.76 ± 4% perf-profile.self.cycles-pp.do_wait 0.17 ± 9% +0.2 0.32 ± 3% perf-profile.self.cycles-pp.wait_task_inactive 0.56 ± 6% +0.2 0.72 ± 4% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.43 ± 6% +0.2 0.59 ± 5% perf-profile.self.cycles-pp.ktime_get 0.09 ± 23% +0.2 0.26 ± 11% perf-profile.self.cycles-pp.osq_lock 0.18 ± 12% +0.3 0.50 ± 7% perf-profile.self.cycles-pp.rwsem_spin_on_owner 1.33 ± 6% +0.3 1.67 ± 3% perf-profile.self.cycles-pp.filemap_map_pages 1.22 ± 7% +0.8 2.04 ± 5% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath pmeter.Average_Active_Power 328 +---------------------------------------------------------------------+ 326 |-+ O O | | O O O O O O O O O O O O O O O O | 324 |-O O O O O O O O O O O O OO | 322 |-+ O O O O O O O | 320 |-+ | 318 |-+ | | | 316 |-+ + | 314 |-+ :: + | 312 |-+ +. : +. +. +. +. ++ + : .+ | 310 |-+ + + .++. : +. : + : +. + + .+ + : +.+ +.+ .+ | |.++ + + + +.+.+ ++.++ +.+ + : .+ + | 308 |-+ + | 306 +---------------------------------------------------------------------+ stress-ng.time.user_time 220 +---------------------------------------------------------------------+ |.+ + .++.+. +.+ + .+ ++. | 215 |-++ ++.+ .++. + + + +.+ + ++.+.+ + :.+ : + ++.+.+ | | + + +.+ + :+ + + + + +.+ | 210 |-+ + + + | | | 205 |-+ | | | 200 |-+ | | | 195 |-+O O OO O O OO O O O OO O O O O O O O O OO O | | O O O O O O O O O O O O O O O O | 190 |-+ O O | | | 185 +---------------------------------------------------------------------+ stress-ng.time.system_time 405 +---------------------------------------------------------------------+ | O O OO OO | 400 |-+O O O O | | O O O O O O OO O O OO OO OO O OO | | O OO O O O O O O OO O | 395 |-+ O | | | 390 |-+ | | | 385 |-+ | | | | .+. +. .+. +. | 380 |-++ ++.++.+.++.+. +. .+ +.+ + ++.+.++.+ +.++.+. +.+ .+.+ | |.+ + ++.+ +.+ + + | 375 +---------------------------------------------------------------------+ stress-ng.time.major_page_faults 55000 +-------------------------------------------------------------------+ 50000 |-+ O | | | 45000 |-O | 40000 |-+O O O | 35000 |-+ O O O O O O O O | 30000 |-+ O O O O O O O O | | O O OO OO O O O O O O O O O | 25000 |-+ O O O O O | 20000 |-+ O | 15000 |-+ | 10000 |-+ | |.+ .++.++.+.++.++.+.++.++. +.+.++.+ .+.++. +.+ .+. +. +.+.++.++.+ | 5000 |-++ + + + + + + | 0 +-------------------------------------------------------------------+ stress-ng.time.voluntary_context_switches 6.4e+07 +-----------------------------------------------------------------+ | +. + .+ +.+ | 6.2e+07 |.++ .+ ++. +.+ +.++ .++.++ .+ .+. :: + : : +.+ .+ | | + + +. + + + + + + + + : : +.+ + | 6e+07 |-+ + ++ + + + | | | 5.8e+07 |-+ | | | 5.6e+07 |-+ | | | 5.4e+07 |-+ O OO O O O O O O O O O O OO O O | | OO OO O O O O O O OO O OO O O OO O O | 5.2e+07 |-+ OO OO | | | 5e+07 +-----------------------------------------------------------------+ stress-ng.vm-segv.ops 820000 +------------------------------------------------------------------+ |.++ .+ +.+ ++ +.++ +.++.+ .+ .+ :: + + : + .+ | 800000 |-+ + + +. + :+ + .+ :.+ + :: :+ +.: + | 780000 |-+ + ++ + + + + + + | | | 760000 |-+ | | | 740000 |-+ | | | 720000 |-+ | 700000 |-+ O O OO O O O O O | | O OO O O O OO O OO OO O O OO OO O O OO O O O | 680000 |-O O OO O O | | O | 660000 +------------------------------------------------------------------+ stress-ng.vm-segv.ops_per_sec 14000 +-------------------------------------------------------------------+ | | 13500 |.++ .+ .+ +.++.++.+ .++.++ + + + ++ +.++.+ | | : + +. + :+ : + : + + :+ :+ + + + +.+ | | : : +.+ + :+ : + + + + ++ | 13000 |-+ + + + | | | 12500 |-+ | | | 12000 |-+ | | O O O O | | O OO O O O O O OO O O OO O O O O O OO O | 11500 |-O O OO O O O O O O O O O O O | | O | 11000 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

1 year, 7 months

2
2
0 / 0

[btrfs] 196d59ab9c: fxmark.hdd_btrfs_MWCL_9_bufferedio.works/sec -16.4% regression

by kernel test robot

Greeting, FYI, we noticed a -16.4% regression of fxmark.hdd_btrfs_MWCL_9_bufferedio.works/sec due to commit: please be noted there are also other regressions in fxmark tests. e.g. 56097 -29.1% 39797 fxmark.hdd_btrfs_MWCL_18_bufferedio.works/sec 53599 -13.7% 46230 ± 3% fxmark.hdd_btrfs_MWCL_72_bufferedio.works/sec commit: 196d59ab9ccc975d8d29292845d227cdf4423ef8 ("btrfs: switch extent buffer tree lock to rw_semaphore") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: fxmark on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory with following parameters: disk: 1HDD media: hdd test: MWCL fstype: btrfs directio: bufferedio cpufreq_governor: performance ucode: 0x11 If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/directio/disk/fstype/kconfig/media/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/bufferedio/1HDD/btrfs/x86_64-rhel-8.3/hdd/debian-10.4-x86_64-20200603.cgz/lkp-knm01/MWCL/fxmark/0x11 commit: ecdcf3c259 ("btrfs: open code insert_orphan_item") 196d59ab9c ("btrfs: switch extent buffer tree lock to rw_semaphore") ecdcf3c259e4c36e 196d59ab9ccc975d8d29292845d ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 8% 0:4 perf-profile.children.cycles-pp.error_return 0:4 12% 0:4 perf-profile.children.cycles-pp.error_entry 0:4 11% 0:4 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 123.68 +88.8% 233.54 fxmark.hdd_btrfs_MWCL_18_bufferedio.idle_sec 24.30 +89.1% 45.95 fxmark.hdd_btrfs_MWCL_18_bufferedio.idle_util 0.51 ± 13% +46.3% 0.75 ± 4% fxmark.hdd_btrfs_MWCL_18_bufferedio.iowait_sec 0.10 ± 13% +46.6% 0.15 ± 5% fxmark.hdd_btrfs_MWCL_18_bufferedio.iowait_util 4.79 ± 3% -14.1% 4.11 ± 2% fxmark.hdd_btrfs_MWCL_18_bufferedio.softirq_sec 0.94 ± 3% -14.0% 0.81 ± 2% fxmark.hdd_btrfs_MWCL_18_bufferedio.softirq_util 338.07 -31.2% 232.52 fxmark.hdd_btrfs_MWCL_18_bufferedio.sys_sec 66.42 -31.1% 45.75 fxmark.hdd_btrfs_MWCL_18_bufferedio.sys_util 10.98 -21.0% 8.66 fxmark.hdd_btrfs_MWCL_18_bufferedio.user_sec 2.16 -20.9% 1.70 fxmark.hdd_btrfs_MWCL_18_bufferedio.user_util 1683695 -29.1% 1194164 fxmark.hdd_btrfs_MWCL_18_bufferedio.works 56097 -29.1% 39797 fxmark.hdd_btrfs_MWCL_18_bufferedio.works/sec 241.75 ± 3% +71.4% 414.43 fxmark.hdd_btrfs_MWCL_27_bufferedio.idle_sec 31.25 ± 3% +71.3% 53.52 fxmark.hdd_btrfs_MWCL_27_bufferedio.idle_util 478.99 ± 2% -35.8% 307.43 fxmark.hdd_btrfs_MWCL_27_bufferedio.sys_sec 61.90 -35.9% 39.71 fxmark.hdd_btrfs_MWCL_27_bufferedio.sys_util 11.25 -12.8% 9.81 ± 3% fxmark.hdd_btrfs_MWCL_27_bufferedio.user_sec 1.45 -12.8% 1.27 ± 3% fxmark.hdd_btrfs_MWCL_27_bufferedio.user_util 1707056 -22.6% 1321210 fxmark.hdd_btrfs_MWCL_27_bufferedio.works 56877 -22.6% 44009 fxmark.hdd_btrfs_MWCL_27_bufferedio.works/sec 1.37 ± 5% -77.9% 0.30 ± 30% fxmark.hdd_btrfs_MWCL_2_bufferedio.idle_sec 2.29 ± 5% -78.1% 0.50 ± 30% fxmark.hdd_btrfs_MWCL_2_bufferedio.idle_util 1.11 ± 11% -13.9% 0.96 ± 9% fxmark.hdd_btrfs_MWCL_2_bufferedio.softirq_sec 1.86 ± 11% -14.9% 1.58 ± 9% fxmark.hdd_btrfs_MWCL_2_bufferedio.softirq_util 341.20 ± 2% +75.6% 598.98 fxmark.hdd_btrfs_MWCL_36_bufferedio.idle_sec 32.89 ± 2% +75.5% 57.71 fxmark.hdd_btrfs_MWCL_36_bufferedio.idle_util 0.46 ± 27% +138.5% 1.08 ± 19% fxmark.hdd_btrfs_MWCL_36_bufferedio.iowait_sec 0.04 ± 27% +138.0% 0.10 ± 19% fxmark.hdd_btrfs_MWCL_36_bufferedio.iowait_util 635.07 -40.5% 377.89 fxmark.hdd_btrfs_MWCL_36_bufferedio.sys_sec 61.22 -40.5% 36.41 fxmark.hdd_btrfs_MWCL_36_bufferedio.sys_util 1747939 -15.5% 1476154 fxmark.hdd_btrfs_MWCL_36_bufferedio.works 58241 -15.6% 49175 fxmark.hdd_btrfs_MWCL_36_bufferedio.works/sec 487.29 +62.8% 793.11 ± 2% fxmark.hdd_btrfs_MWCL_45_bufferedio.idle_sec 37.52 +62.7% 61.04 ± 2% fxmark.hdd_btrfs_MWCL_45_bufferedio.idle_util 744.34 -41.0% 439.31 ± 4% fxmark.hdd_btrfs_MWCL_45_bufferedio.sys_sec 57.31 -41.0% 33.81 ± 4% fxmark.hdd_btrfs_MWCL_45_bufferedio.sys_util 1759079 -10.8% 1568784 ± 3% fxmark.hdd_btrfs_MWCL_45_bufferedio.works 58631 -10.9% 52267 ± 3% fxmark.hdd_btrfs_MWCL_45_bufferedio.works/sec 6.38 -18.0% 5.23 ± 7% fxmark.hdd_btrfs_MWCL_4_bufferedio.idle_sec 5.47 -20.5% 4.35 ± 7% fxmark.hdd_btrfs_MWCL_4_bufferedio.idle_util 1.75 ± 5% -33.2% 1.17 ± 8% fxmark.hdd_btrfs_MWCL_4_bufferedio.softirq_sec 1.50 ± 5% -35.3% 0.97 ± 8% fxmark.hdd_btrfs_MWCL_4_bufferedio.softirq_util 659.89 +52.6% 1007 fxmark.hdd_btrfs_MWCL_54_bufferedio.idle_sec 42.16 +53.0% 64.49 fxmark.hdd_btrfs_MWCL_54_bufferedio.idle_util 8.08 +13.2% 9.14 ± 2% fxmark.hdd_btrfs_MWCL_54_bufferedio.softirq_sec 0.52 +13.5% 0.59 ± 2% fxmark.hdd_btrfs_MWCL_54_bufferedio.softirq_util 833.14 -42.2% 481.74 fxmark.hdd_btrfs_MWCL_54_bufferedio.sys_sec 53.22 -42.0% 30.85 fxmark.hdd_btrfs_MWCL_54_bufferedio.sys_util 827.45 +47.9% 1223 fxmark.hdd_btrfs_MWCL_63_bufferedio.idle_sec 45.34 +47.9% 67.04 fxmark.hdd_btrfs_MWCL_63_bufferedio.idle_util 8.54 +16.8% 9.98 fxmark.hdd_btrfs_MWCL_63_bufferedio.softirq_sec 0.47 +16.8% 0.55 fxmark.hdd_btrfs_MWCL_63_bufferedio.softirq_util 919.60 -43.1% 522.80 ± 2% fxmark.hdd_btrfs_MWCL_63_bufferedio.sys_sec 50.38 -43.2% 28.64 ± 2% fxmark.hdd_btrfs_MWCL_63_bufferedio.sys_util 1017 +54.2% 1569 fxmark.hdd_btrfs_MWCL_72_bufferedio.idle_sec 48.62 +53.9% 74.84 fxmark.hdd_btrfs_MWCL_72_bufferedio.idle_util 983.26 -55.2% 440.30 ± 4% fxmark.hdd_btrfs_MWCL_72_bufferedio.sys_sec 46.99 -55.3% 21.00 ± 3% fxmark.hdd_btrfs_MWCL_72_bufferedio.sys_util 1610685 -13.9% 1387293 ± 3% fxmark.hdd_btrfs_MWCL_72_bufferedio.works 53599 -13.7% 46230 ± 3% fxmark.hdd_btrfs_MWCL_72_bufferedio.works/sec 29.75 +89.7% 56.45 ± 2% fxmark.hdd_btrfs_MWCL_9_bufferedio.idle_sec 11.62 +86.9% 21.72 ± 2% fxmark.hdd_btrfs_MWCL_9_bufferedio.idle_util 7.64 -10.8% 6.82 fxmark.hdd_btrfs_MWCL_9_bufferedio.irq_util 3.10 ± 6% -15.1% 2.64 ± 3% fxmark.hdd_btrfs_MWCL_9_bufferedio.softirq_sec 1.21 ± 6% -16.4% 1.01 ± 3% fxmark.hdd_btrfs_MWCL_9_bufferedio.softirq_util 194.38 -10.0% 174.90 fxmark.hdd_btrfs_MWCL_9_bufferedio.sys_sec 75.94 -11.4% 67.29 fxmark.hdd_btrfs_MWCL_9_bufferedio.sys_util 8.95 -11.7% 7.91 fxmark.hdd_btrfs_MWCL_9_bufferedio.user_sec 3.50 -13.1% 3.04 fxmark.hdd_btrfs_MWCL_9_bufferedio.user_util 1359770 -16.4% 1136680 fxmark.hdd_btrfs_MWCL_9_bufferedio.works 45317 -16.4% 37885 fxmark.hdd_btrfs_MWCL_9_bufferedio.works/sec 549.29 -3.3% 531.11 fxmark.time.elapsed_time 549.29 -3.3% 531.11 fxmark.time.elapsed_time.max 1096944 -15.4% 927632 ± 2% fxmark.time.file_system_outputs 67.00 -22.4% 52.00 fxmark.time.percent_of_cpu_this_job_got 356.30 -25.2% 266.46 fxmark.time.system_time 13.97 -7.7% 12.90 fxmark.time.user_time 687910 ± 2% -14.9% 585190 fxmark.time.voluntary_context_switches 92480 +57.6% 145791 cpuidle.POLL.usage 45.12 +17.7% 53.12 iostat.cpu.idle 47.26 -17.5% 39.01 iostat.cpu.system 4465896 -11.0% 3976872 numa-numastat.node0.local_node 4465847 -10.9% 3976837 numa-numastat.node0.numa_hit 48.04 +8.2 56.24 mpstat.cpu.all.idle% 1.00 ± 5% -0.1 0.94 ± 4% mpstat.cpu.all.soft% 43.86 -8.1 35.72 mpstat.cpu.all.sys% 44.75 +17.9% 52.75 vmstat.cpu.id 51.00 -16.7% 42.50 vmstat.cpu.sy 23992 -12.0% 21119 vmstat.io.bo 2338252 -9.9% 2106851 vmstat.memory.cache 11.50 ± 4% -63.0% 4.25 ± 10% vmstat.procs.r 92744 +6.5% 98768 vmstat.system.cs 175350 -18.3% 143184 meminfo.Active 174192 -18.4% 142148 meminfo.Active(file) 214134 -21.0% 169130 ± 3% meminfo.Dirty 152957 -23.9% 116365 ± 4% meminfo.Inactive(file) 914555 -16.8% 760508 meminfo.KReclaimable 914555 -16.8% 760508 meminfo.SReclaimable 441392 -9.9% 397727 meminfo.SUnreclaim 28606 ± 3% -19.0% 23181 ± 3% meminfo.Shmem 1355948 -14.6% 1158236 meminfo.Slab 1683 ± 4% -8.9% 1532 ± 4% meminfo.Writeback 176261 -18.5% 143572 numa-meminfo.node0.Active 175123 -18.6% 142537 numa-meminfo.node0.Active(file) 214487 -21.0% 169509 ± 4% numa-meminfo.node0.Dirty 152258 -23.1% 117060 ± 4% numa-meminfo.node0.Inactive(file) 897715 -17.0% 744913 numa-meminfo.node0.KReclaimable 897715 -17.0% 744913 numa-meminfo.node0.SReclaimable 429290 -9.9% 386596 numa-meminfo.node0.SUnreclaim 28605 ± 3% -19.0% 23175 ± 3% numa-meminfo.node0.Shmem 1327006 -14.7% 1131510 numa-meminfo.node0.Slab 1752 ± 5% -11.4% 1552 numa-meminfo.node0.Writeback 43694 -18.5% 35593 numa-vmstat.node0.nr_active_file 916940 -14.6% 783445 numa-vmstat.node0.nr_dirtied 53663 -21.0% 42371 ± 3% numa-vmstat.node0.nr_dirty 38292 -23.2% 29417 ± 4% numa-vmstat.node0.nr_inactive_file 7144 ± 2% -19.0% 5789 ± 3% numa-vmstat.node0.nr_shmem 224840 -17.1% 186437 numa-vmstat.node0.nr_slab_reclaimable 107449 -10.0% 96654 numa-vmstat.node0.nr_slab_unreclaimable 449.50 ± 4% -14.6% 384.00 ± 4% numa-vmstat.node0.nr_writeback 862673 -14.1% 740618 numa-vmstat.node0.nr_written 43695 -18.5% 35593 numa-vmstat.node0.nr_zone_active_file 38291 -23.2% 29417 ± 4% numa-vmstat.node0.nr_zone_inactive_file 54266 -21.1% 42827 ± 3% numa-vmstat.node0.nr_zone_write_pending 642101 -18.1% 526175 slabinfo.btrfs_delayed_node.active_objs 12361 -18.0% 10131 slabinfo.btrfs_delayed_node.active_slabs 642824 -18.0% 526856 slabinfo.btrfs_delayed_node.num_objs 12361 -18.0% 10131 slabinfo.btrfs_delayed_node.num_slabs 641642 -18.1% 525750 slabinfo.btrfs_inode.active_objs 22930 -18.1% 18790 slabinfo.btrfs_inode.active_slabs 642066 -18.1% 526149 slabinfo.btrfs_inode.num_objs 22930 -18.1% 18790 slabinfo.btrfs_inode.num_slabs 614342 -15.8% 517367 slabinfo.dentry.active_objs 14843 -15.7% 12518 slabinfo.dentry.active_slabs 623419 -15.7% 525802 slabinfo.dentry.num_objs 14843 -15.7% 12518 slabinfo.dentry.num_slabs 26554 -33.0% 17795 slabinfo.kmalloc-192.active_objs 788.50 -26.9% 576.50 ± 2% slabinfo.kmalloc-192.active_slabs 33135 -26.8% 24238 ± 2% slabinfo.kmalloc-192.num_objs 788.50 -26.9% 576.50 ± 2% slabinfo.kmalloc-192.num_slabs 35021 -17.4% 28941 slabinfo.kmalloc-256.active_objs 39465 -13.8% 34015 slabinfo.kmalloc-256.num_objs 38834 -11.0% 34576 slabinfo.kmalloc-512.active_objs 3822 ± 4% -19.0% 3096 ± 9% slabinfo.mnt_cache.active_objs 4085 ± 4% -16.9% 3395 ± 8% slabinfo.mnt_cache.num_objs 22177 -98.4% 345.50 ± 2% slabinfo.numa_policy.active_objs 370.25 -98.2% 6.75 ± 6% slabinfo.numa_policy.active_slabs 22974 -98.2% 418.50 ± 6% slabinfo.numa_policy.num_objs 370.25 -98.2% 6.75 ± 6% slabinfo.numa_policy.num_slabs 30778 -11.4% 27258 slabinfo.radix_tree_node.active_objs 287.50 -10.2% 258.25 ± 2% proc-vmstat.nr_active_anon 44012 -18.2% 36011 proc-vmstat.nr_active_file 1641867 -14.6% 1402146 proc-vmstat.nr_dirtied 54266 -21.0% 42844 ± 3% proc-vmstat.nr_dirty 354597 -5.3% 335913 proc-vmstat.nr_file_pages 68032 -1.7% 66880 proc-vmstat.nr_inactive_anon 38874 -23.7% 29657 ± 4% proc-vmstat.nr_inactive_file 7185 ± 3% -19.0% 5817 ± 3% proc-vmstat.nr_shmem 231482 -16.9% 192443 proc-vmstat.nr_slab_reclaimable 110865 -10.0% 99807 proc-vmstat.nr_slab_unreclaimable 447.00 -11.0% 398.00 proc-vmstat.nr_writeback 1641866 -14.6% 1402143 proc-vmstat.nr_written 287.50 -10.2% 258.25 ± 2% proc-vmstat.nr_zone_active_anon 44012 -18.2% 36011 proc-vmstat.nr_zone_active_file 68032 -1.7% 66880 proc-vmstat.nr_zone_inactive_anon 38874 -23.7% 29657 ± 4% proc-vmstat.nr_zone_inactive_file 54890 -21.1% 43319 ± 3% proc-vmstat.nr_zone_write_pending 4487469 -10.8% 4001195 proc-vmstat.numa_hit 4487468 -10.8% 4001193 proc-vmstat.numa_local 645936 +8.8% 702580 proc-vmstat.pgactivate 9776528 -12.1% 8593562 proc-vmstat.pgalloc_normal 1323917 -14.8% 1127396 ± 3% proc-vmstat.pgdeactivate 1852505 -3.3% 1792262 proc-vmstat.pgfault 9778333 -12.1% 8596857 proc-vmstat.pgfree 13064555 -14.7% 11146070 proc-vmstat.pgpgout 1355099 -15.6% 1143590 ± 3% proc-vmstat.pgrotated 29192165 -12.6% 25511450 proc-vmstat.slabs_scanned 2682624 -2.3% 2622144 proc-vmstat.unevictable_pgs_scanned 11918547 ± 5% +7.8% 12843186 perf-stat.i.branch-instructions 0.08 ± 6% +0.0 0.09 ± 2% perf-stat.i.branch-miss-rate% 924883 ± 5% +8.7% 1004974 ± 3% perf-stat.i.branch-misses 0.16 ± 7% +0.0 0.17 perf-stat.i.cache-miss-rate% 194655 ± 8% +10.9% 215884 perf-stat.i.cache-misses 1307774 ± 8% +8.8% 1423352 perf-stat.i.cache-references 91889 +7.0% 98361 perf-stat.i.context-switches 0.09 ± 9% +8.5% 0.10 perf-stat.i.cpi 39978 +0.8% 40304 perf-stat.i.cpu-clock 4.778e+08 ± 7% +8.6% 5.187e+08 perf-stat.i.cpu-cycles 900.48 +26.4% 1138 perf-stat.i.cpu-migrations 630084 ± 7% +8.2% 681691 perf-stat.i.iTLB-load-misses 56704882 ± 5% +7.9% 61157553 perf-stat.i.iTLB-loads 56801859 ± 5% +7.9% 61262751 perf-stat.i.instructions 0.96 ± 5% +7.4% 1.03 perf-stat.i.instructions-per-iTLB-miss 0.00 ± 6% +7.0% 0.00 perf-stat.i.ipc 1.78 ± 2% +8.1% 1.93 ± 2% perf-stat.i.major-faults 0.00 ± 7% +8.5% 0.00 perf-stat.i.metric.GHz 0.33 +3.5% 0.34 perf-stat.i.metric.K/sec 0.25 ± 5% +7.7% 0.27 perf-stat.i.metric.M/sec 39978 +0.8% 40304 perf-stat.i.task-clock 12147873 ± 5% +8.2% 13143736 perf-stat.ps.branch-instructions 942554 ± 5% +9.1% 1028476 ± 3% perf-stat.ps.branch-misses 198339 ± 8% +11.4% 220964 perf-stat.ps.cache-misses 1332481 ± 8% +9.3% 1456578 perf-stat.ps.cache-references 91580 +6.7% 97730 perf-stat.ps.context-switches 4.868e+08 ± 7% +9.0% 5.308e+08 perf-stat.ps.cpu-cycles 896.06 +26.2% 1131 perf-stat.ps.cpu-migrations 641967 ± 7% +8.7% 697566 perf-stat.ps.iTLB-load-misses 57794217 ± 5% +8.3% 62587387 perf-stat.ps.iTLB-loads 57894864 ± 5% +8.3% 62696692 perf-stat.ps.instructions 1.78 ± 2% +8.4% 1.93 ± 2% perf-stat.ps.major-faults 76331 -14.9% 64991 sched_debug.cfs_rq:/.exec_clock.avg 131652 -45.5% 71764 sched_debug.cfs_rq:/.exec_clock.max 19235 -88.8% 2152 ± 10% sched_debug.cfs_rq:/.exec_clock.stddev 29025 ± 23% +115.9% 62654 ± 25% sched_debug.cfs_rq:/.load.min 228411 ± 11% -38.6% 140254 ± 44% sched_debug.cfs_rq:/.load.stddev 43.60 ± 29% +86.5% 81.33 ± 17% sched_debug.cfs_rq:/.load_avg.min 1523223 ± 3% -19.2% 1230395 ± 6% sched_debug.cfs_rq:/.min_vruntime.avg 2877263 ± 3% -54.9% 1297137 ± 5% sched_debug.cfs_rq:/.min_vruntime.max 821701 ± 6% +33.4% 1096067 ± 8% sched_debug.cfs_rq:/.min_vruntime.min 639729 ± 3% -93.7% 40423 ± 26% sched_debug.cfs_rq:/.min_vruntime.stddev 0.43 ± 2% -26.0% 0.32 ± 13% sched_debug.cfs_rq:/.nr_running.avg 0.12 ± 34% +33.3% 0.17 ± 33% sched_debug.cfs_rq:/.nr_running.min 21.91 ± 7% +83.3% 40.17 ± 8% sched_debug.cfs_rq:/.nr_spread_over.avg 6.25 ± 33% +379.6% 29.97 ± 6% sched_debug.cfs_rq:/.nr_spread_over.min 224.08 ± 12% +74.5% 390.97 ± 12% sched_debug.cfs_rq:/.runnable_avg.min 222.26 ± 12% -22.6% 171.99 ± 13% sched_debug.cfs_rq:/.runnable_avg.stddev -1308743 -104.8% 62240 ±123% sched_debug.cfs_rq:/.spread0.avg -2010183 -96.4% -72085 sched_debug.cfs_rq:/.spread0.min 639697 ± 3% -93.7% 40422 ± 26% sched_debug.cfs_rq:/.spread0.stddev 408.48 ± 5% -13.0% 355.56 ± 3% sched_debug.cfs_rq:/.util_avg.avg 1055 ± 3% -10.6% 943.19 ± 3% sched_debug.cfs_rq:/.util_avg.max 162.68 ± 12% +53.0% 248.97 ± 15% sched_debug.cfs_rq:/.util_avg.min 205.13 ± 7% -25.5% 152.83 ± 13% sched_debug.cfs_rq:/.util_avg.stddev 244.43 ± 5% -66.9% 80.95 ± 39% sched_debug.cfs_rq:/.util_est_enqueued.avg 709.90 ± 13% -39.9% 426.36 ± 11% sched_debug.cfs_rq:/.util_est_enqueued.max 203.06 ± 15% -58.6% 84.00 ± 15% sched_debug.cfs_rq:/.util_est_enqueued.stddev 848045 ± 9% +34.7% 1142214 ± 10% sched_debug.cpu.avg_idle.avg 188030 ± 23% +140.0% 451192 ± 14% sched_debug.cpu.avg_idle.min 345409 ± 8% -25.4% 257601 ± 12% sched_debug.cpu.avg_idle.stddev 347803 -9.9% 313357 sched_debug.cpu.clock.avg 347858 -9.9% 313400 sched_debug.cpu.clock.max 347747 -9.9% 313310 sched_debug.cpu.clock.min 31.60 -18.3% 25.83 ± 3% sched_debug.cpu.clock.stddev 338985 -10.5% 303239 sched_debug.cpu.clock_task.avg 340608 -10.8% 303972 sched_debug.cpu.clock_task.max 329665 -9.8% 297262 sched_debug.cpu.clock_task.min 2108 ± 4% -39.3% 1280 ± 5% sched_debug.cpu.clock_task.stddev 939.85 ± 81% +165.3% 2493 ± 27% sched_debug.cpu.curr->pid.min 3549 ± 10% -29.0% 2520 ± 15% sched_debug.cpu.curr->pid.stddev 0.00 ± 97% -98.3% 0.00 ± 4% sched_debug.cpu.next_balance.stddev 0.12 ± 34% +144.4% 0.31 ± 15% sched_debug.cpu.nr_running.min 654832 +18.8% 777815 sched_debug.cpu.nr_switches.avg 904162 ± 2% -9.8% 815631 ± 2% sched_debug.cpu.nr_switches.max 574858 +31.5% 755656 sched_debug.cpu.nr_switches.min 82240 ± 3% -81.8% 14938 ± 13% sched_debug.cpu.nr_switches.stddev 263.38 ± 26% -67.9% 84.44 ± 16% sched_debug.cpu.nr_uninterruptible.max -218.23 -78.1% -47.78 sched_debug.cpu.nr_uninterruptible.min 128.55 ± 24% -77.9% 28.46 ± 22% sched_debug.cpu.nr_uninterruptible.stddev 653092 +18.6% 774865 sched_debug.cpu.sched_count.avg 913676 ± 2% -11.5% 808187 ± 2% sched_debug.cpu.sched_count.max 573366 +31.5% 753744 sched_debug.cpu.sched_count.min 83073 ± 3% -83.4% 13777 ± 12% sched_debug.cpu.sched_count.stddev 285707 +18.5% 338620 sched_debug.cpu.sched_goidle.avg 397157 -11.6% 351276 ± 2% sched_debug.cpu.sched_goidle.max 243238 +36.3% 331500 sched_debug.cpu.sched_goidle.min 43632 ± 3% -88.0% 5215 ± 22% sched_debug.cpu.sched_goidle.stddev 276633 +183.4% 783865 ± 3% sched_debug.cpu.ttwu_count.avg 615804 ± 2% +249.5% 2152323 ± 6% sched_debug.cpu.ttwu_count.max 144528 ± 2% +324.1% 612922 ± 4% sched_debug.cpu.ttwu_count.stddev 7838 ± 4% +62.0% 12696 ± 6% sched_debug.cpu.ttwu_local.avg 23472 ± 2% -36.2% 14981 ± 4% sched_debug.cpu.ttwu_local.max 6517 ± 3% +77.0% 11535 ± 6% sched_debug.cpu.ttwu_local.min 1978 ± 8% -47.1% 1047 ± 21% sched_debug.cpu.ttwu_local.stddev 347742 -9.9% 313303 sched_debug.cpu_clk 347083 -9.9% 312643 sched_debug.ktime 348036 -9.9% 313599 sched_debug.sched_clk 116128 ± 2% -38.5% 71410 softirqs.CPU0.SCHED 91878 -27.9% 66231 softirqs.CPU1.SCHED 70366 ± 2% -21.3% 55400 softirqs.CPU10.SCHED 71066 ± 2% -22.0% 55446 softirqs.CPU11.SCHED 70819 ± 2% -21.8% 55357 softirqs.CPU12.SCHED 72824 ± 2% -22.9% 56169 ± 3% softirqs.CPU13.SCHED 72347 ± 2% -23.0% 55736 ± 3% softirqs.CPU14.SCHED 71040 -22.6% 54996 softirqs.CPU15.SCHED 73832 ± 2% -25.7% 54822 softirqs.CPU16.SCHED 73531 -24.7% 55359 softirqs.CPU17.SCHED 61336 -23.2% 47097 softirqs.CPU18.SCHED 61263 -23.3% 46993 softirqs.CPU19.SCHED 90645 ± 2% -24.7% 68276 softirqs.CPU2.SCHED 61313 ± 2% -23.3% 47013 softirqs.CPU20.SCHED 62197 -24.2% 47163 softirqs.CPU21.SCHED 64716 ± 2% -27.6% 46834 softirqs.CPU22.SCHED 63504 -26.3% 46803 softirqs.CPU23.SCHED 64003 ± 2% -26.5% 47071 softirqs.CPU24.SCHED 64911 -27.8% 46890 softirqs.CPU25.SCHED 55809 -27.1% 40707 softirqs.CPU26.SCHED 54152 -26.6% 39754 softirqs.CPU27.SCHED 54386 ± 2% -27.4% 39499 softirqs.CPU28.SCHED 54236 -27.4% 39401 softirqs.CPU29.SCHED 87173 -21.4% 68485 softirqs.CPU3.SCHED 55503 ± 2% -29.2% 39293 softirqs.CPU30.SCHED 56079 -29.9% 39303 softirqs.CPU31.SCHED 57664 -31.4% 39583 softirqs.CPU32.SCHED 55860 -29.6% 39347 softirqs.CPU33.SCHED 54670 -29.8% 38364 softirqs.CPU34.SCHED 57222 ± 2% -31.7% 39072 softirqs.CPU35.SCHED 45803 -30.9% 31647 softirqs.CPU36.SCHED 45558 ± 2% -30.5% 31645 softirqs.CPU37.SCHED 46190 ± 2% -32.1% 31368 softirqs.CPU38.SCHED 43936 ± 7% -28.9% 31240 softirqs.CPU39.SCHED 79246 -17.9% 65045 softirqs.CPU4.SCHED 46593 ± 2% -30.6% 32330 softirqs.CPU40.SCHED 46935 -31.7% 32080 softirqs.CPU41.SCHED 46761 -32.1% 31771 softirqs.CPU42.SCHED 44507 ± 2% -29.3% 31454 ± 2% softirqs.CPU43.SCHED 36826 ± 3% -32.9% 24725 softirqs.CPU44.SCHED 34982 ± 3% -30.7% 24230 softirqs.CPU45.SCHED 35262 ± 4% -30.9% 24363 softirqs.CPU46.SCHED 35716 -31.7% 24388 softirqs.CPU47.SCHED 34721 ± 3% -30.5% 24144 ± 2% softirqs.CPU48.SCHED 36083 ± 4% -32.6% 24324 softirqs.CPU49.SCHED 81693 ± 2% -20.4% 65005 softirqs.CPU5.SCHED 35183 -29.8% 24699 softirqs.CPU50.SCHED 35820 ± 2% -32.2% 24290 softirqs.CPU51.SCHED 33069 -33.1% 22117 ± 4% softirqs.CPU52.SCHED 34535 -29.6% 24296 softirqs.CPU53.SCHED 25349 -34.8% 16522 ± 6% softirqs.CPU54.SCHED 23599 ± 3% -27.7% 17051 ± 2% softirqs.CPU55.SCHED 24760 -30.9% 17107 ± 2% softirqs.CPU56.SCHED 24178 ± 4% -27.7% 17470 ± 3% softirqs.CPU57.SCHED 24307 ± 2% -30.2% 16973 softirqs.CPU58.SCHED 23460 ± 4% -27.2% 17090 softirqs.CPU59.SCHED 79700 -19.3% 64305 softirqs.CPU6.SCHED 24681 -33.2% 16485 ± 3% softirqs.CPU60.SCHED 24529 ± 3% -31.0% 16925 ± 2% softirqs.CPU61.SCHED 14304 ± 5% -29.3% 10109 softirqs.CPU62.SCHED 13347 ± 3% -29.5% 9408 ± 2% softirqs.CPU63.SCHED 10558 ± 2% -9.0% 9612 ± 3% softirqs.CPU64.RCU 14104 ± 4% -31.8% 9623 ± 2% softirqs.CPU64.SCHED 13844 ± 5% -28.8% 9862 softirqs.CPU65.SCHED 12902 ± 3% -25.5% 9610 softirqs.CPU66.SCHED 10891 ± 7% -9.6% 9842 ± 3% softirqs.CPU67.RCU 13872 ± 7% -31.3% 9526 ± 3% softirqs.CPU67.SCHED 13213 ± 8% -26.1% 9771 ± 2% softirqs.CPU68.SCHED 14232 ± 5% -32.4% 9625 ± 2% softirqs.CPU69.SCHED 81684 -21.4% 64189 softirqs.CPU7.SCHED 10183 ± 15% -25.0% 7633 ± 6% softirqs.CPU70.SCHED 13274 ± 6% -28.9% 9434 ± 3% softirqs.CPU71.SCHED 75679 -22.7% 58521 softirqs.CPU8.SCHED 68854 -20.6% 54638 softirqs.CPU9.SCHED 4101011 -22.9% 3160637 softirqs.SCHED 12836 -11.9% 11304 interrupts.180:IR-PCI-MSI.512000-edge.ahci[0000:00:1f.2] 4431149 -1.7% 4357713 interrupts.CAL:Function_call_interrupts 1266 ± 45% -46.8% 674.00 ± 19% interrupts.CPU0.NMI:Non-maskable_interrupts 1266 ± 45% -46.8% 674.00 ± 19% interrupts.CPU0.PMI:Performance_monitoring_interrupts 217128 ± 7% -30.5% 150952 ± 2% interrupts.CPU1.CAL:Function_call_interrupts 30529 ± 6% -38.9% 18640 ± 3% interrupts.CPU1.RES:Rescheduling_interrupts 3405 ± 8% -69.7% 1033 ± 7% interrupts.CPU10.RES:Rescheduling_interrupts 703.75 ± 21% -54.3% 321.75 ± 41% interrupts.CPU11.NMI:Non-maskable_interrupts 703.75 ± 21% -54.3% 321.75 ± 41% interrupts.CPU11.PMI:Performance_monitoring_interrupts 3372 ± 5% -67.9% 1081 ± 9% interrupts.CPU11.RES:Rescheduling_interrupts 3289 ± 10% -71.6% 934.50 ± 4% interrupts.CPU12.RES:Rescheduling_interrupts 102804 ± 6% +15.4% 118611 ± 8% interrupts.CPU13.CAL:Function_call_interrupts 3233 ± 8% -69.3% 994.25 ± 5% interrupts.CPU13.RES:Rescheduling_interrupts 1071 ± 57% -69.6% 325.25 ± 42% interrupts.CPU14.NMI:Non-maskable_interrupts 1071 ± 57% -69.6% 325.25 ± 42% interrupts.CPU14.PMI:Performance_monitoring_interrupts 3502 ± 8% -76.1% 838.50 ± 4% interrupts.CPU14.RES:Rescheduling_interrupts 820.25 ± 26% -58.6% 339.25 ± 36% interrupts.CPU15.NMI:Non-maskable_interrupts 820.25 ± 26% -58.6% 339.25 ± 36% interrupts.CPU15.PMI:Performance_monitoring_interrupts 3377 ± 8% -73.3% 901.25 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts 9914 -57.3% 4237 ±100% interrupts.CPU16.180:IR-PCI-MSI.512000-edge.ahci[0000:00:1f.2] 850.25 ± 26% -56.5% 369.75 ± 55% interrupts.CPU16.NMI:Non-maskable_interrupts 850.25 ± 26% -56.5% 369.75 ± 55% interrupts.CPU16.PMI:Performance_monitoring_interrupts 3510 ± 7% -75.0% 876.00 ± 10% interrupts.CPU16.RES:Rescheduling_interrupts 1201 ± 40% -76.7% 280.00 ± 22% interrupts.CPU17.NMI:Non-maskable_interrupts 1201 ± 40% -76.7% 280.00 ± 22% interrupts.CPU17.PMI:Performance_monitoring_interrupts 3545 ± 9% -75.3% 875.50 ± 6% interrupts.CPU17.RES:Rescheduling_interrupts 683.00 ± 52% -49.2% 347.25 ± 29% interrupts.CPU18.NMI:Non-maskable_interrupts 683.00 ± 52% -49.2% 347.25 ± 29% interrupts.CPU18.PMI:Performance_monitoring_interrupts 2080 ± 12% -80.9% 398.00 ± 8% interrupts.CPU18.RES:Rescheduling_interrupts 488.75 ± 23% -47.6% 256.00 ± 12% interrupts.CPU19.NMI:Non-maskable_interrupts 488.75 ± 23% -47.6% 256.00 ± 12% interrupts.CPU19.PMI:Performance_monitoring_interrupts 2009 ± 7% -77.6% 450.50 ± 12% interrupts.CPU19.RES:Rescheduling_interrupts 191815 ± 8% -22.4% 148909 ± 4% interrupts.CPU2.CAL:Function_call_interrupts 611.25 ± 29% -58.6% 253.25 ± 26% interrupts.CPU2.NMI:Non-maskable_interrupts 611.25 ± 29% -58.6% 253.25 ± 26% interrupts.CPU2.PMI:Performance_monitoring_interrupts 774.75 ± 38% -67.3% 253.25 ± 32% interrupts.CPU20.NMI:Non-maskable_interrupts 774.75 ± 38% -67.3% 253.25 ± 32% interrupts.CPU20.PMI:Performance_monitoring_interrupts 2025 ± 3% -81.8% 369.50 ± 15% interrupts.CPU20.RES:Rescheduling_interrupts 2088 ± 3% -83.5% 345.00 ± 9% interrupts.CPU21.RES:Rescheduling_interrupts 2212 ± 11% -85.3% 325.25 ± 3% interrupts.CPU22.RES:Rescheduling_interrupts 892.50 ± 44% -60.7% 351.00 ± 59% interrupts.CPU23.NMI:Non-maskable_interrupts 892.50 ± 44% -60.7% 351.00 ± 59% interrupts.CPU23.PMI:Performance_monitoring_interrupts 2264 ± 10% -85.1% 337.25 ± 10% interrupts.CPU23.RES:Rescheduling_interrupts 65733 ± 5% +18.9% 78175 ± 5% interrupts.CPU24.CAL:Function_call_interrupts 1410 ± 26% -73.1% 379.50 ± 42% interrupts.CPU24.NMI:Non-maskable_interrupts 1410 ± 26% -73.1% 379.50 ± 42% interrupts.CPU24.PMI:Performance_monitoring_interrupts 2309 ± 2% -86.3% 315.25 ± 3% interrupts.CPU24.RES:Rescheduling_interrupts 63928 ± 2% +21.4% 77636 ± 5% interrupts.CPU25.CAL:Function_call_interrupts 870.50 ± 32% -58.4% 362.00 ± 31% interrupts.CPU25.NMI:Non-maskable_interrupts 870.50 ± 32% -58.4% 362.00 ± 31% interrupts.CPU25.PMI:Performance_monitoring_interrupts 2490 ± 10% -86.4% 338.50 ± 9% interrupts.CPU25.RES:Rescheduling_interrupts 808.75 ± 16% -44.3% 450.50 ± 63% interrupts.CPU26.NMI:Non-maskable_interrupts 808.75 ± 16% -44.3% 450.50 ± 63% interrupts.CPU26.PMI:Performance_monitoring_interrupts 1312 ± 9% -84.1% 208.25 ± 4% interrupts.CPU26.RES:Rescheduling_interrupts 1458 ± 44% -78.3% 316.75 ± 46% interrupts.CPU27.NMI:Non-maskable_interrupts 1458 ± 44% -78.3% 316.75 ± 46% interrupts.CPU27.PMI:Performance_monitoring_interrupts 1337 ± 5% -84.5% 207.25 ± 11% interrupts.CPU27.RES:Rescheduling_interrupts 1270 ± 4% -5.8% 1196 interrupts.CPU275.CAL:Function_call_interrupts 1118 ± 33% -73.0% 302.25 ± 17% interrupts.CPU28.NMI:Non-maskable_interrupts 1118 ± 33% -73.0% 302.25 ± 17% interrupts.CPU28.PMI:Performance_monitoring_interrupts 1195 ± 9% -82.0% 215.00 ± 4% interrupts.CPU28.RES:Rescheduling_interrupts 854.00 ± 19% -60.5% 337.25 ± 10% interrupts.CPU29.NMI:Non-maskable_interrupts 854.00 ± 19% -60.5% 337.25 ± 10% interrupts.CPU29.PMI:Performance_monitoring_interrupts 1190 ± 9% -80.2% 235.75 ± 14% interrupts.CPU29.RES:Rescheduling_interrupts 198710 ± 10% -22.2% 154622 ± 2% interrupts.CPU3.CAL:Function_call_interrupts 1122 ± 43% -51.0% 550.25 ± 23% interrupts.CPU30.NMI:Non-maskable_interrupts 1122 ± 43% -51.0% 550.25 ± 23% interrupts.CPU30.PMI:Performance_monitoring_interrupts 1369 ± 11% -85.7% 195.25 ± 12% interrupts.CPU30.RES:Rescheduling_interrupts 1307 ± 38% -54.7% 591.75 ± 65% interrupts.CPU31.NMI:Non-maskable_interrupts 1307 ± 38% -54.7% 591.75 ± 65% interrupts.CPU31.PMI:Performance_monitoring_interrupts 1316 ± 9% -84.4% 205.25 ± 10% interrupts.CPU31.RES:Rescheduling_interrupts 1303 ± 29% -79.1% 272.25 ± 20% interrupts.CPU32.NMI:Non-maskable_interrupts 1303 ± 29% -79.1% 272.25 ± 20% interrupts.CPU32.PMI:Performance_monitoring_interrupts 1361 ± 4% -84.5% 210.75 ± 8% interrupts.CPU32.RES:Rescheduling_interrupts 863.25 ± 16% -63.1% 318.25 ± 34% interrupts.CPU33.NMI:Non-maskable_interrupts 863.25 ± 16% -63.1% 318.25 ± 34% interrupts.CPU33.PMI:Performance_monitoring_interrupts 1424 ± 7% -81.6% 261.50 ± 35% interrupts.CPU33.RES:Rescheduling_interrupts 40521 ± 8% +34.8% 54611 ± 13% interrupts.CPU34.CAL:Function_call_interrupts 1060 ± 49% -50.9% 520.75 ± 6% interrupts.CPU34.NMI:Non-maskable_interrupts 1060 ± 49% -50.9% 520.75 ± 6% interrupts.CPU34.PMI:Performance_monitoring_interrupts 1460 ± 8% -83.4% 242.50 ± 26% interrupts.CPU34.RES:Rescheduling_interrupts 42549 ± 6% +27.2% 54118 ± 15% interrupts.CPU35.CAL:Function_call_interrupts 1108 ± 21% -45.5% 604.25 ± 38% interrupts.CPU35.NMI:Non-maskable_interrupts 1108 ± 21% -45.5% 604.25 ± 38% interrupts.CPU35.PMI:Performance_monitoring_interrupts 1490 ± 11% -84.3% 234.25 ± 21% interrupts.CPU35.RES:Rescheduling_interrupts 857.25 ± 17% -48.6% 441.00 ± 39% interrupts.CPU36.NMI:Non-maskable_interrupts 857.25 ± 17% -48.6% 441.00 ± 39% interrupts.CPU36.PMI:Performance_monitoring_interrupts 921.50 ± 2% -85.5% 133.75 ± 2% interrupts.CPU36.RES:Rescheduling_interrupts 1107 ± 44% -55.3% 494.75 ± 49% interrupts.CPU37.NMI:Non-maskable_interrupts 1107 ± 44% -55.3% 494.75 ± 49% interrupts.CPU37.PMI:Performance_monitoring_interrupts 918.50 ± 3% -83.8% 149.00 ± 15% interrupts.CPU37.RES:Rescheduling_interrupts 982.00 ± 3% -85.0% 147.25 ± 19% interrupts.CPU38.RES:Rescheduling_interrupts 990.50 ± 4% -86.0% 139.00 ± 17% interrupts.CPU39.RES:Rescheduling_interrupts 170783 ± 10% -14.4% 146206 ± 6% interrupts.CPU4.CAL:Function_call_interrupts 1628 ± 35% -79.3% 337.75 ± 47% interrupts.CPU40.NMI:Non-maskable_interrupts 1628 ± 35% -79.3% 337.75 ± 47% interrupts.CPU40.PMI:Performance_monitoring_interrupts 984.00 ± 6% -86.0% 138.25 ± 11% interrupts.CPU40.RES:Rescheduling_interrupts 1080 ± 36% -63.9% 389.75 ± 41% interrupts.CPU41.NMI:Non-maskable_interrupts 1080 ± 36% -63.9% 389.75 ± 41% interrupts.CPU41.PMI:Performance_monitoring_interrupts 952.75 ± 7% -84.2% 150.25 ± 8% interrupts.CPU41.RES:Rescheduling_interrupts 1045 ± 35% -56.9% 450.25 ± 44% interrupts.CPU42.NMI:Non-maskable_interrupts 1045 ± 35% -56.9% 450.25 ± 44% interrupts.CPU42.PMI:Performance_monitoring_interrupts 983.75 ± 2% -86.7% 131.00 ± 6% interrupts.CPU42.RES:Rescheduling_interrupts 939.50 ± 29% -53.6% 435.75 ± 44% interrupts.CPU43.NMI:Non-maskable_interrupts 939.50 ± 29% -53.6% 435.75 ± 44% interrupts.CPU43.PMI:Performance_monitoring_interrupts 963.50 ± 3% -86.8% 126.75 ± 15% interrupts.CPU43.RES:Rescheduling_interrupts 27228 -33.3% 18149 ± 6% interrupts.CPU44.CAL:Function_call_interrupts 1334 ± 32% -67.4% 434.25 ± 40% interrupts.CPU44.NMI:Non-maskable_interrupts 1334 ± 32% -67.4% 434.25 ± 40% interrupts.CPU44.PMI:Performance_monitoring_interrupts 668.25 ± 8% -87.2% 85.75 ± 6% interrupts.CPU44.RES:Rescheduling_interrupts 20504 ± 6% -25.8% 15212 ± 2% interrupts.CPU45.CAL:Function_call_interrupts 869.75 ± 41% -62.5% 326.25 ± 43% interrupts.CPU45.NMI:Non-maskable_interrupts 869.75 ± 41% -62.5% 326.25 ± 43% interrupts.CPU45.PMI:Performance_monitoring_interrupts 654.75 ± 2% -89.5% 68.75 ± 12% interrupts.CPU45.RES:Rescheduling_interrupts 803.25 ± 15% -50.6% 397.00 ± 54% interrupts.CPU46.NMI:Non-maskable_interrupts 803.25 ± 15% -50.6% 397.00 ± 54% interrupts.CPU46.PMI:Performance_monitoring_interrupts 687.25 ± 10% -87.3% 87.25 ± 16% interrupts.CPU46.RES:Rescheduling_interrupts 1364 ± 15% -72.6% 374.50 ± 48% interrupts.CPU47.NMI:Non-maskable_interrupts 1364 ± 15% -72.6% 374.50 ± 48% interrupts.CPU47.PMI:Performance_monitoring_interrupts 688.75 ± 4% -84.4% 107.25 ± 16% interrupts.CPU47.RES:Rescheduling_interrupts 1290 ± 40% -70.4% 382.25 ± 51% interrupts.CPU48.NMI:Non-maskable_interrupts 1290 ± 40% -70.4% 382.25 ± 51% interrupts.CPU48.PMI:Performance_monitoring_interrupts 701.75 ± 5% -88.3% 82.25 ± 18% interrupts.CPU48.RES:Rescheduling_interrupts 1163 ± 31% -76.9% 268.50 ± 31% interrupts.CPU49.NMI:Non-maskable_interrupts 1163 ± 31% -76.9% 268.50 ± 31% interrupts.CPU49.PMI:Performance_monitoring_interrupts 704.75 ± 5% -88.6% 80.25 ± 6% interrupts.CPU49.RES:Rescheduling_interrupts 166242 ± 6% -13.7% 143496 ± 6% interrupts.CPU5.CAL:Function_call_interrupts 7461 ± 8% +25.9% 9396 ± 11% interrupts.CPU5.RES:Rescheduling_interrupts 706.25 ± 2% -88.1% 83.75 ± 15% interrupts.CPU50.RES:Rescheduling_interrupts 21177 ± 4% +11.3% 23562 ± 4% interrupts.CPU51.CAL:Function_call_interrupts 1075 ± 32% -63.1% 397.25 ± 46% interrupts.CPU51.NMI:Non-maskable_interrupts 1075 ± 32% -63.1% 397.25 ± 46% interrupts.CPU51.PMI:Performance_monitoring_interrupts 718.00 ± 5% -86.4% 97.50 ± 20% interrupts.CPU51.RES:Rescheduling_interrupts 680.75 ± 4% -86.7% 90.75 ± 22% interrupts.CPU52.RES:Rescheduling_interrupts 702.00 ± 5% -88.2% 83.00 ± 10% interrupts.CPU53.RES:Rescheduling_interrupts 1638 ± 39% -72.0% 458.25 ± 55% interrupts.CPU54.NMI:Non-maskable_interrupts 1638 ± 39% -72.0% 458.25 ± 55% interrupts.CPU54.PMI:Performance_monitoring_interrupts 388.75 ± 7% -86.9% 50.75 ± 7% interrupts.CPU54.RES:Rescheduling_interrupts 993.50 ± 26% -47.5% 521.50 ± 24% interrupts.CPU55.NMI:Non-maskable_interrupts 993.50 ± 26% -47.5% 521.50 ± 24% interrupts.CPU55.PMI:Performance_monitoring_interrupts 417.50 ± 7% -86.9% 54.75 ± 22% interrupts.CPU55.RES:Rescheduling_interrupts 1356 ± 22% -70.2% 404.00 ± 52% interrupts.CPU56.NMI:Non-maskable_interrupts 1356 ± 22% -70.2% 404.00 ± 52% interrupts.CPU56.PMI:Performance_monitoring_interrupts 394.50 ± 9% -87.8% 48.25 ± 11% interrupts.CPU56.RES:Rescheduling_interrupts 1213 ± 40% -64.4% 432.00 ± 43% interrupts.CPU57.NMI:Non-maskable_interrupts 1213 ± 40% -64.4% 432.00 ± 43% interrupts.CPU57.PMI:Performance_monitoring_interrupts 399.25 -85.0% 59.75 ± 20% interrupts.CPU57.RES:Rescheduling_interrupts 1067 ± 65% -72.4% 295.00 ± 57% interrupts.CPU58.NMI:Non-maskable_interrupts 1067 ± 65% -72.4% 295.00 ± 57% interrupts.CPU58.PMI:Performance_monitoring_interrupts 479.25 ± 2% -87.6% 59.25 ± 17% interrupts.CPU58.RES:Rescheduling_interrupts 1127 ± 41% -69.8% 341.00 ± 42% interrupts.CPU59.NMI:Non-maskable_interrupts 1127 ± 41% -69.8% 341.00 ± 42% interrupts.CPU59.PMI:Performance_monitoring_interrupts 437.25 ± 2% -88.3% 51.25 ± 32% interrupts.CPU59.RES:Rescheduling_interrupts 165868 ± 4% -11.1% 147461 ± 3% interrupts.CPU6.CAL:Function_call_interrupts 1195 ± 23% -67.7% 386.50 ± 61% interrupts.CPU6.NMI:Non-maskable_interrupts 1195 ± 23% -67.7% 386.50 ± 61% interrupts.CPU6.PMI:Performance_monitoring_interrupts 7147 ± 7% -30.6% 4959 ± 15% interrupts.CPU6.RES:Rescheduling_interrupts 111.75 ± 4% -28.6% 79.75 ± 27% interrupts.CPU6.TLB:TLB_shootdowns 1426 ± 21% -58.7% 589.00 ± 44% interrupts.CPU60.NMI:Non-maskable_interrupts 1426 ± 21% -58.7% 589.00 ± 44% interrupts.CPU60.PMI:Performance_monitoring_interrupts 410.25 ± 9% -88.0% 49.25 ± 10% interrupts.CPU60.RES:Rescheduling_interrupts 1022 ± 29% -68.2% 325.50 ± 43% interrupts.CPU61.NMI:Non-maskable_interrupts 1022 ± 29% -68.2% 325.50 ± 43% interrupts.CPU61.PMI:Performance_monitoring_interrupts 385.75 ± 8% -88.0% 46.25 ± 8% interrupts.CPU61.RES:Rescheduling_interrupts 12518 ± 11% -40.4% 7455 ± 29% interrupts.CPU62.CAL:Function_call_interrupts 1018 ± 20% -71.2% 293.00 ± 46% interrupts.CPU62.NMI:Non-maskable_interrupts 1018 ± 20% -71.2% 293.00 ± 46% interrupts.CPU62.PMI:Performance_monitoring_interrupts 207.25 ± 6% -82.8% 35.75 ± 13% interrupts.CPU62.RES:Rescheduling_interrupts 186.75 ± 3% -77.6% 41.75 ± 49% interrupts.CPU63.RES:Rescheduling_interrupts 7639 ± 3% -16.4% 6387 ± 11% interrupts.CPU64.CAL:Function_call_interrupts 194.00 ± 11% -86.1% 27.00 ± 4% interrupts.CPU64.RES:Rescheduling_interrupts 7523 ± 2% -16.6% 6277 ± 8% interrupts.CPU65.CAL:Function_call_interrupts 198.25 ± 12% -89.7% 20.50 ± 24% interrupts.CPU65.RES:Rescheduling_interrupts 7440 ± 11% -39.0% 4540 ± 26% interrupts.CPU66.CAL:Function_call_interrupts 201.50 ± 12% -90.8% 18.50 ± 9% interrupts.CPU66.RES:Rescheduling_interrupts 8240 ± 17% -43.8% 4631 ± 26% interrupts.CPU67.CAL:Function_call_interrupts 197.25 ± 14% -85.4% 28.75 ± 18% interrupts.CPU67.RES:Rescheduling_interrupts 181.75 ± 17% -84.2% 28.75 ± 27% interrupts.CPU68.RES:Rescheduling_interrupts 168.25 ± 19% -83.1% 28.50 ± 14% interrupts.CPU69.RES:Rescheduling_interrupts 161290 ± 3% -8.3% 147840 ± 5% interrupts.CPU7.CAL:Function_call_interrupts 705.75 ± 16% -54.2% 323.25 ± 9% interrupts.CPU7.NMI:Non-maskable_interrupts 705.75 ± 16% -54.2% 323.25 ± 9% interrupts.CPU7.PMI:Performance_monitoring_interrupts 7207 ± 8% -37.0% 4541 ± 10% interrupts.CPU7.RES:Rescheduling_interrupts 155.25 ± 24% -78.4% 33.50 ± 20% interrupts.CPU70.RES:Rescheduling_interrupts 132.00 ± 20% -77.5% 29.75 ± 23% interrupts.CPU71.RES:Rescheduling_interrupts 18308 ± 4% +12.9% 20662 ± 5% interrupts.CPU73.LOC:Local_timer_interrupts 3295 ± 17% -59.6% 1332 ± 14% interrupts.CPU8.RES:Rescheduling_interrupts 27403 ± 11% -14.1% 23549 ± 3% interrupts.CPU80.LOC:Local_timer_interrupts 26422 ± 2% +11.4% 29433 ± 7% interrupts.CPU89.LOC:Local_timer_interrupts 3387 ± 16% -64.7% 1194 ± 12% interrupts.CPU9.RES:Rescheduling_interrupts 63112 ± 9% -57.3% 26977 ± 21% interrupts.NMI:Non-maskable_interrupts 63112 ± 9% -57.3% 26977 ± 21% interrupts.PMI:Performance_monitoring_interrupts 196263 -39.9% 117875 ± 2% interrupts.RES:Rescheduling_interrupts 58.40 ± 4% -29.2 29.23 ± 35% perf-profile.calltrace.cycles-pp.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 58.38 ± 4% -29.2 29.22 ± 35% perf-profile.calltrace.cycles-pp.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 58.04 ± 4% -29.0 29.02 ± 35% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 58.01 ± 4% -29.0 29.00 ± 35% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64 59.31 ± 4% -29.0 30.33 ± 35% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 58.76 ± 4% -28.9 29.88 ± 34% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 54.36 ± 4% -28.1 26.21 ± 34% perf-profile.calltrace.cycles-pp.btrfs_create.path_openat.do_filp_open.do_sys_openat2.do_sys_open 39.06 ± 4% -23.6 15.48 ± 39% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.btrfs_new_inode.btrfs_create.path_openat.do_filp_open 38.60 ± 4% -23.3 15.30 ± 39% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.btrfs_new_inode.btrfs_create.path_openat 40.56 ± 4% -22.0 18.57 ± 30% perf-profile.calltrace.cycles-pp.btrfs_new_inode.btrfs_create.path_openat.do_filp_open.do_sys_openat2 12.21 ± 4% -12.2 0.00 perf-profile.calltrace.cycles-pp.btrfs_try_tree_write_lock.btrfs_search_slot.btrfs_insert_empty_items.btrfs_new_inode.btrfs_create 12.14 ± 4% -12.1 0.00 perf-profile.calltrace.cycles-pp.queued_write_lock_slowpath.btrfs_try_tree_write_lock.btrfs_search_slot.btrfs_insert_empty_items.btrfs_new_inode 11.54 ± 4% -11.5 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath.queued_write_lock_slowpath.btrfs_try_tree_write_lock.btrfs_search_slot.btrfs_insert_empty_items 11.54 ± 8% -11.5 0.00 perf-profile.calltrace.cycles-pp.queued_write_lock_slowpath.__btrfs_tree_lock.btrfs_search_slot.btrfs_insert_empty_items.btrfs_new_inode 10.81 ± 8% -10.8 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath.queued_write_lock_slowpath.__btrfs_tree_lock.btrfs_search_slot.btrfs_insert_empty_items 12.88 ± 8% -9.9 3.00 ± 22% perf-profile.calltrace.cycles-pp.__btrfs_commit_inode_delayed_items.btrfs_async_run_delayed_root.btrfs_work_helper.process_one_work.worker_thread 23.95 ± 5% -9.8 14.11 ± 38% perf-profile.calltrace.cycles-pp.__btrfs_tree_lock.btrfs_search_slot.btrfs_insert_empty_items.btrfs_new_inode.btrfs_create 12.14 ± 8% -9.8 2.36 ± 25% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_inode.__btrfs_update_delayed_inode.__btrfs_commit_inode_delayed_items.btrfs_async_run_delayed_root 12.14 ± 8% -9.8 2.38 ± 25% perf-profile.calltrace.cycles-pp.btrfs_lookup_inode.__btrfs_update_delayed_inode.__btrfs_commit_inode_delayed_items.btrfs_async_run_delayed_root.btrfs_work_helper 12.55 ± 8% -9.6 2.91 ± 23% perf-profile.calltrace.cycles-pp.__btrfs_update_delayed_inode.__btrfs_commit_inode_delayed_items.btrfs_async_run_delayed_root.btrfs_work_helper.process_one_work 13.85 ± 8% -9.4 4.47 ± 25% perf-profile.calltrace.cycles-pp.btrfs_work_helper.process_one_work.worker_thread.kthread.ret_from_fork 13.84 ± 8% -9.4 4.47 ± 26% perf-profile.calltrace.cycles-pp.btrfs_async_run_delayed_root.btrfs_work_helper.process_one_work.worker_thread.kthread 14.60 ± 8% -8.7 5.94 ± 19% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 14.62 ± 8% -8.6 5.99 ± 19% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 10.52 ± 3% -7.9 2.60 ± 46% perf-profile.calltrace.cycles-pp.insert_with_overflow.btrfs_insert_dir_item.btrfs_add_link.btrfs_create.path_openat 10.49 ± 3% -7.9 2.58 ± 46% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.insert_with_overflow.btrfs_insert_dir_item.btrfs_add_link.btrfs_create 9.61 ± 3% -7.7 1.92 ± 45% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.insert_with_overflow.btrfs_insert_dir_item.btrfs_add_link 7.57 ± 5% -7.6 0.00 perf-profile.calltrace.cycles-pp.prepare_to_wait_event.__btrfs_tree_lock.btrfs_search_slot.btrfs_insert_empty_items.btrfs_new_inode 11.34 ± 3% -7.2 4.13 ± 49% perf-profile.calltrace.cycles-pp.btrfs_insert_dir_item.btrfs_add_link.btrfs_create.path_openat.do_filp_open 7.18 ± 5% -7.2 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.prepare_to_wait_event.__btrfs_tree_lock.btrfs_search_slot.btrfs_insert_empty_items 7.11 ± 5% -7.1 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.prepare_to_wait_event.__btrfs_tree_lock.btrfs_search_slot 7.01 ± 2% -7.0 0.00 perf-profile.calltrace.cycles-pp.__btrfs_tree_lock.btrfs_lock_root_node.btrfs_search_slot.btrfs_insert_empty_items.insert_with_overflow 7.01 ± 2% -7.0 0.00 perf-profile.calltrace.cycles-pp.btrfs_lock_root_node.btrfs_search_slot.btrfs_insert_empty_items.insert_with_overflow.btrfs_insert_dir_item 11.72 ± 3% -6.2 5.53 ± 55% perf-profile.calltrace.cycles-pp.btrfs_add_link.btrfs_create.path_openat.do_filp_open.do_sys_openat2 5.93 ± 9% -4.9 0.98 ± 70% perf-profile.calltrace.cycles-pp.__btrfs_tree_read_lock.btrfs_search_slot.btrfs_lookup_inode.__btrfs_update_delayed_inode.__btrfs_commit_inode_delayed_items 17.35 ± 8% -3.5 13.82 ± 6% perf-profile.calltrace.cycles-pp.ret_from_fork 17.35 ± 8% -3.5 13.82 ± 6% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 1.27 ± 18% -0.8 0.42 ±101% perf-profile.calltrace.cycles-pp.__btrfs_read_lock_root_node.btrfs_search_slot.btrfs_insert_empty_items.insert_with_overflow.btrfs_insert_dir_item 1.23 ± 20% -0.8 0.40 ±101% perf-profile.calltrace.cycles-pp.__btrfs_tree_read_lock.__btrfs_read_lock_root_node.btrfs_search_slot.btrfs_insert_empty_items.insert_with_overflow 0.00 +0.6 0.62 ± 13% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.rebalance_domains.__softirqentry_text_start.asm_call_sysvec_on_stack 0.00 +0.6 0.63 ± 16% perf-profile.calltrace.cycles-pp._raw_spin_trylock.shrink_lock_dentry.shrink_dentry_list.prune_dcache_sb.super_cache_scan 0.47 ± 58% +0.8 1.31 ± 44% perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.00 +0.9 0.85 ± 24% perf-profile.calltrace.cycles-pp.list_lru_add.inode_lru_list_add.iput.__dentry_kill.shrink_dentry_list 0.00 +0.9 0.87 ± 24% perf-profile.calltrace.cycles-pp.inode_lru_list_add.iput.__dentry_kill.shrink_dentry_list.prune_dcache_sb 0.00 +0.9 0.90 ± 13% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack 0.00 +0.9 0.90 ± 16% perf-profile.calltrace.cycles-pp.dentry_lru_isolate.__list_lru_walk_one.list_lru_walk_one.prune_dcache_sb.super_cache_scan 0.28 ±100% +0.9 1.19 ± 12% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.00 +0.9 0.94 ± 17% perf-profile.calltrace.cycles-pp.__list_lru_walk_one.list_lru_walk_one.prune_dcache_sb.super_cache_scan.do_shrink_slab 0.00 +0.9 0.94 ± 17% perf-profile.calltrace.cycles-pp.list_lru_walk_one.prune_dcache_sb.super_cache_scan.do_shrink_slab.shrink_slab 0.00 +0.9 0.94 ± 19% perf-profile.calltrace.cycles-pp.___d_drop.__d_drop.__dentry_kill.shrink_dentry_list.prune_dcache_sb 0.00 +0.9 0.95 ± 19% perf-profile.calltrace.cycles-pp.shrink_lock_dentry.shrink_dentry_list.prune_dcache_sb.super_cache_scan.do_shrink_slab 0.14 ±173% +0.9 1.08 ± 13% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu 0.00 +0.9 0.95 ± 19% perf-profile.calltrace.cycles-pp.__d_drop.__dentry_kill.shrink_dentry_list.prune_dcache_sb.super_cache_scan 0.22 ±173% +1.0 1.20 ± 33% perf-profile.calltrace.cycles-pp.btrfs_get_32.check_leaf.btree_csum_one_bio.btrfs_submit_metadata_bio.submit_one_bio 0.00 +1.0 1.02 ± 23% perf-profile.calltrace.cycles-pp.btrfs_drop_extent_cache.btrfs_destroy_inode.destroy_inode.dispose_list.prune_icache_sb 0.80 ± 9% +1.1 1.91 ± 13% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry.start_secondary 0.00 +1.1 1.14 ± 25% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 0.26 ±173% +1.1 1.40 ± 33% perf-profile.calltrace.cycles-pp.check_dir_item.check_leaf.btree_csum_one_bio.btrfs_submit_metadata_bio.submit_one_bio 0.94 ± 9% +1.2 2.12 ± 13% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt 0.40 ± 58% +1.2 1.59 ± 25% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt 0.94 ± 9% +1.2 2.13 ± 13% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.95 ± 9% +1.2 2.17 ± 13% perf-profile.calltrace.cycles-pp.do_softirq_own_stack.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 0.00 +1.3 1.26 ± 26% perf-profile.calltrace.cycles-pp.__btrfs_release_delayed_node.btrfs_evict_inode.evict.dispose_list.prune_icache_sb 0.00 +1.3 1.33 ± 26% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 0.28 ±100% +1.3 1.62 ± 23% perf-profile.calltrace.cycles-pp.kmem_cache_free.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd 0.46 ± 58% +1.5 1.92 ± 21% perf-profile.calltrace.cycles-pp.iput.__dentry_kill.shrink_dentry_list.prune_dcache_sb.super_cache_scan 1.12 ± 9% +1.5 2.65 ± 16% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 0.56 ± 58% +1.7 2.23 ± 21% perf-profile.calltrace.cycles-pp.rcu_do_batch.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn 0.56 ± 58% +1.7 2.25 ± 21% perf-profile.calltrace.cycles-pp.rcu_core.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread 0.78 ± 21% +1.7 2.50 ± 20% perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork 0.94 ± 5% +1.7 2.68 ± 24% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 0.62 ± 59% +1.8 2.44 ± 21% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 0.62 ± 58% +1.8 2.44 ± 21% perf-profile.calltrace.cycles-pp.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 1.02 ± 5% +1.9 2.92 ± 23% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt 0.96 ± 26% +2.1 3.03 ± 25% perf-profile.calltrace.cycles-pp.btrfs_destroy_inode.destroy_inode.dispose_list.prune_icache_sb.super_cache_scan 0.81 ± 22% +2.1 2.93 ± 26% perf-profile.calltrace.cycles-pp.btrfs_evict_inode.evict.dispose_list.prune_icache_sb.super_cache_scan 1.81 ± 2% +2.3 4.13 ± 12% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 1.33 ± 9% +2.4 3.74 ± 25% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack 1.10 ± 25% +2.4 3.54 ± 26% perf-profile.calltrace.cycles-pp.destroy_inode.dispose_list.prune_icache_sb.super_cache_scan.do_shrink_slab 1.51 ± 63% +2.7 4.24 ± 32% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range 1.35 ± 14% +2.7 4.10 ± 20% perf-profile.calltrace.cycles-pp.__dentry_kill.shrink_dentry_list.prune_dcache_sb.super_cache_scan.do_shrink_slab 1.25 ± 80% +2.8 4.01 ± 28% perf-profile.calltrace.cycles-pp.check_leaf.btree_csum_one_bio.btrfs_submit_metadata_bio.submit_one_bio.submit_extent_page 1.58 ± 63% +2.8 4.43 ± 32% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents 1.30 ± 80% +2.9 4.16 ± 28% perf-profile.calltrace.cycles-pp.btree_csum_one_bio.btrfs_submit_metadata_bio.submit_one_bio.submit_extent_page.write_one_eb 1.15 ± 21% +2.9 4.04 ± 26% perf-profile.calltrace.cycles-pp.evict.dispose_list.prune_icache_sb.super_cache_scan.do_shrink_slab 1.64 ± 63% +3.0 4.60 ± 32% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.transaction_kthread 1.64 ± 63% +3.0 4.60 ± 32% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction 1.64 ± 63% +3.0 4.60 ± 32% perf-profile.calltrace.cycles-pp.btrfs_write_and_wait_transaction.btrfs_commit_transaction.transaction_kthread.kthread.ret_from_fork 1.64 ± 63% +3.0 4.60 ± 32% perf-profile.calltrace.cycles-pp.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.transaction_kthread.kthread 1.64 ± 63% +3.0 4.60 ± 32% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction 1.36 ± 80% +3.0 4.39 ± 27% perf-profile.calltrace.cycles-pp.btrfs_submit_metadata_bio.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages 1.36 ± 80% +3.0 4.39 ± 27% perf-profile.calltrace.cycles-pp.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages 1.90 ± 60% +3.3 5.20 ± 29% perf-profile.calltrace.cycles-pp.transaction_kthread.kthread.ret_from_fork 1.90 ± 60% +3.3 5.20 ± 29% perf-profile.calltrace.cycles-pp.btrfs_commit_transaction.transaction_kthread.kthread.ret_from_fork 2.09 ± 8% +3.8 5.86 ± 25% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt 1.88 ± 15% +3.8 5.71 ± 20% perf-profile.calltrace.cycles-pp.shrink_dentry_list.prune_dcache_sb.super_cache_scan.do_shrink_slab.shrink_slab 2.18 ± 15% +4.5 6.66 ± 19% perf-profile.calltrace.cycles-pp.prune_dcache_sb.super_cache_scan.do_shrink_slab.shrink_slab.drop_slab_node 2.97 ± 10% +5.5 8.45 ± 20% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 2.37 ± 23% +5.7 8.06 ± 26% perf-profile.calltrace.cycles-pp.dispose_list.prune_icache_sb.super_cache_scan.do_shrink_slab.shrink_slab 3.33 ± 10% +6.0 9.29 ± 25% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 2.56 ± 23% +6.1 8.66 ± 25% perf-profile.calltrace.cycles-pp.prune_icache_sb.super_cache_scan.do_shrink_slab.shrink_slab.drop_slab_node 3.50 ± 10% +6.2 9.73 ± 25% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 3.53 ± 10% +6.3 9.84 ± 25% perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 5.45 ± 11% +9.1 14.58 ± 23% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 5.72 ± 11% +9.9 15.63 ± 24% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 4.74 ± 19% +10.6 15.33 ± 23% perf-profile.calltrace.cycles-pp.super_cache_scan.do_shrink_slab.shrink_slab.drop_slab_node.drop_slab 4.74 ± 19% +10.6 15.34 ± 23% perf-profile.calltrace.cycles-pp.drop_slab.drop_caches_sysctl_handler.proc_sys_call_handler.new_sync_write.vfs_write 4.74 ± 19% +10.6 15.34 ± 23% perf-profile.calltrace.cycles-pp.drop_slab_node.drop_slab.drop_caches_sysctl_handler.proc_sys_call_handler.new_sync_write 4.74 ± 19% +10.6 15.34 ± 23% perf-profile.calltrace.cycles-pp.shrink_slab.drop_slab_node.drop_slab.drop_caches_sysctl_handler.proc_sys_call_handler 4.74 ± 19% +10.6 15.34 ± 23% perf-profile.calltrace.cycles-pp.do_shrink_slab.shrink_slab.drop_slab_node.drop_slab.drop_caches_sysctl_handler 5.67 ± 21% +12.2 17.87 ± 24% perf-profile.calltrace.cycles-pp.proc_sys_call_handler.new_sync_write.vfs_write.ksys_write.do_syscall_64 5.67 ± 21% +12.2 17.87 ± 24% perf-profile.calltrace.cycles-pp.drop_caches_sysctl_handler.proc_sys_call_handler.new_sync_write.vfs_write.ksys_write 5.72 ± 21% +12.3 17.99 ± 24% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.72 ± 21% +12.3 18.00 ± 24% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 5.72 ± 21% +12.3 18.00 ± 24% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 5.72 ± 21% +12.3 18.00 ± 24% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 5.72 ± 21% +12.3 18.00 ± 24% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 5.73 ± 21% +12.3 18.02 ± 24% perf-profile.calltrace.cycles-pp.write 0.00 +13.3 13.27 ± 37% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_search_slot 0.00 +14.0 13.98 ± 38% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_search_slot.btrfs_insert_empty_items 0.00 +14.1 14.08 ± 38% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.__btrfs_tree_lock.btrfs_search_slot.btrfs_insert_empty_items.btrfs_new_inode 9.40 ± 10% +15.7 25.15 ± 21% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 9.62 ± 10% +16.1 25.75 ± 21% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 15.73 ± 5% +18.0 33.75 ± 15% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 15.76 ± 5% +18.1 33.82 ± 15% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 15.76 ± 5% +18.1 33.82 ± 15% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 15.99 ± 5% +18.4 34.38 ± 15% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 53.31 ± 3% -46.2 7.08 ± 42% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 62.27 ± 4% -41.1 21.16 ± 36% perf-profile.children.cycles-pp.btrfs_search_slot 50.06 ± 4% -31.6 18.44 ± 39% perf-profile.children.cycles-pp.btrfs_insert_empty_items 58.46 ± 4% -29.1 29.38 ± 35% perf-profile.children.cycles-pp.do_sys_open 58.45 ± 4% -29.1 29.36 ± 35% perf-profile.children.cycles-pp.do_sys_openat2 58.10 ± 4% -28.9 29.18 ± 35% perf-profile.children.cycles-pp.do_filp_open 58.07 ± 4% -28.9 29.16 ± 35% perf-profile.children.cycles-pp.path_openat 54.36 ± 4% -28.1 26.22 ± 34% perf-profile.children.cycles-pp.btrfs_create 27.83 ± 3% -27.8 0.00 perf-profile.children.cycles-pp.queued_write_lock_slowpath 40.57 ± 4% -22.0 18.57 ± 30% perf-profile.children.cycles-pp.btrfs_new_inode 31.53 ± 4% -16.9 14.66 ± 37% perf-profile.children.cycles-pp.__btrfs_tree_lock 17.10 ± 5% -16.3 0.81 ± 13% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 65.59 ± 3% -15.9 49.73 ± 12% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 65.01 ± 2% -15.8 49.22 ± 11% perf-profile.children.cycles-pp.do_syscall_64 12.58 ± 4% -12.5 0.06 ± 34% perf-profile.children.cycles-pp.btrfs_try_tree_write_lock 11.61 ± 5% -11.6 0.00 perf-profile.children.cycles-pp.prepare_to_wait_event 13.37 ± 8% -10.0 3.40 ± 23% perf-profile.children.cycles-pp.__btrfs_commit_inode_delayed_items 12.15 ± 8% -9.8 2.38 ± 25% perf-profile.children.cycles-pp.btrfs_lookup_inode 9.67 ± 7% -9.7 0.00 perf-profile.children.cycles-pp.queued_read_lock_slowpath 12.56 ± 8% -9.6 2.92 ± 22% perf-profile.children.cycles-pp.__btrfs_update_delayed_inode 13.85 ± 8% -9.4 4.47 ± 26% perf-profile.children.cycles-pp.btrfs_async_run_delayed_root 13.85 ± 8% -9.4 4.47 ± 25% perf-profile.children.cycles-pp.btrfs_work_helper 14.60 ± 8% -8.7 5.94 ± 19% perf-profile.children.cycles-pp.process_one_work 14.62 ± 8% -8.6 5.99 ± 19% perf-profile.children.cycles-pp.worker_thread 10.52 ± 3% -7.9 2.60 ± 46% perf-profile.children.cycles-pp.insert_with_overflow 7.55 ± 2% -7.4 0.12 ± 47% perf-profile.children.cycles-pp.btrfs_lock_root_node 11.34 ± 3% -7.2 4.13 ± 49% perf-profile.children.cycles-pp.btrfs_insert_dir_item 11.72 ± 3% -6.2 5.53 ± 55% perf-profile.children.cycles-pp.btrfs_add_link 8.69 ± 7% -5.8 2.89 ± 38% perf-profile.children.cycles-pp.__btrfs_tree_read_lock 5.59 ± 6% -5.6 0.00 perf-profile.children.cycles-pp.finish_wait 17.35 ± 8% -3.5 13.82 ± 6% perf-profile.children.cycles-pp.kthread 17.36 ± 8% -3.5 13.83 ± 6% perf-profile.children.cycles-pp.ret_from_fork 0.80 ± 5% -0.7 0.12 ± 27% perf-profile.children.cycles-pp.__wake_up_common_lock 0.69 ± 5% -0.6 0.10 ± 33% perf-profile.children.cycles-pp.__wake_up_common 0.63 ± 6% -0.5 0.08 ± 40% perf-profile.children.cycles-pp.autoremove_wake_function 0.45 ± 13% -0.3 0.19 ± 73% perf-profile.children.cycles-pp.unwind_get_return_address 0.40 ± 13% -0.2 0.16 ± 72% perf-profile.children.cycles-pp.__kernel_text_address 0.37 ± 14% -0.2 0.14 ± 74% perf-profile.children.cycles-pp.kernel_text_address 1.13 ± 7% -0.2 0.93 ± 7% perf-profile.children.cycles-pp.pick_next_task_fair 0.69 ± 7% -0.2 0.49 ± 21% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.49 ± 12% -0.2 0.30 ± 52% perf-profile.children.cycles-pp.btrfs_get_token_32 0.38 ± 9% -0.2 0.21 ± 54% perf-profile.children.cycles-pp.__sysvec_call_function_single 0.25 ± 26% -0.1 0.11 ± 85% perf-profile.children.cycles-pp.__module_address 0.19 ± 20% -0.1 0.07 ±116% perf-profile.children.cycles-pp.__module_text_address 0.14 ± 7% -0.1 0.04 ±107% perf-profile.children.cycles-pp.btrfs_tree_read_unlock 0.24 ± 6% -0.1 0.17 ± 22% perf-profile.children.cycles-pp.__list_add_valid 0.13 ± 9% -0.1 0.07 ± 70% perf-profile.children.cycles-pp.inode_permission 0.10 ± 11% -0.1 0.04 ±101% perf-profile.children.cycles-pp.btrfs_alloc_delayed_item 0.15 ± 7% -0.1 0.10 ± 40% perf-profile.children.cycles-pp.push_leaf_left 0.10 ± 8% -0.1 0.04 ±100% perf-profile.children.cycles-pp.send_call_function_single_ipi 0.09 ± 13% -0.1 0.04 ±102% perf-profile.children.cycles-pp.flush_smp_call_function_queue 0.08 -0.1 0.03 ±100% perf-profile.children.cycles-pp.btrfs_init_acl 0.08 ± 5% -0.0 0.04 ±100% perf-profile.children.cycles-pp.btrfs_init_inode_security 0.11 ± 4% -0.0 0.06 ± 62% perf-profile.children.cycles-pp.copy_pages 0.06 ± 6% +0.0 0.09 ± 7% perf-profile.children.cycles-pp.rcu_all_qs 0.10 ± 8% +0.0 0.14 ± 11% perf-profile.children.cycles-pp.__alloc_pages_nodemask 0.04 ± 57% +0.0 0.07 ± 17% perf-profile.children.cycles-pp.select_task_rq_fair 0.06 ± 15% +0.0 0.09 ± 13% perf-profile.children.cycles-pp.__mod_memcg_state 0.04 ± 59% +0.0 0.08 ± 13% perf-profile.children.cycles-pp.note_gp_changes 0.07 ± 10% +0.0 0.11 ± 26% perf-profile.children.cycles-pp.update_dl_rq_load_avg 0.14 ± 11% +0.0 0.18 ± 16% perf-profile.children.cycles-pp.cpumask_next_and 0.06 ± 20% +0.0 0.11 ± 17% perf-profile.children.cycles-pp.rcu_needs_cpu 0.03 ±100% +0.1 0.08 ± 26% perf-profile.children.cycles-pp.pm_qos_read_value 0.06 ± 13% +0.1 0.12 ± 24% perf-profile.children.cycles-pp.update_ts_time_stats 0.06 +0.1 0.11 ± 17% perf-profile.children.cycles-pp.get_cpu_device 0.03 ±100% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.kfree 0.09 ± 13% +0.1 0.15 ± 17% perf-profile.children.cycles-pp.rb_next 0.12 ± 8% +0.1 0.18 ± 9% perf-profile.children.cycles-pp._cond_resched 0.06 ± 20% +0.1 0.12 ± 15% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.03 ±100% +0.1 0.09 ± 16% perf-profile.children.cycles-pp.rcu_dynticks_eqs_enter 0.00 +0.1 0.06 ± 20% perf-profile.children.cycles-pp.blk_flush_plug_list 0.00 +0.1 0.06 ± 20% perf-profile.children.cycles-pp.blk_mq_flush_plug_list 0.00 +0.1 0.06 ± 20% perf-profile.children.cycles-pp.blk_mq_sched_insert_requests 0.00 +0.1 0.06 ± 11% perf-profile.children.cycles-pp.perf_read 0.03 ±100% +0.1 0.09 ± 24% perf-profile.children.cycles-pp.x86_pmu_disable 0.00 +0.1 0.06 ± 16% perf-profile.children.cycles-pp.__x86_retpoline_rsi 0.01 ±173% +0.1 0.07 ± 26% perf-profile.children.cycles-pp.run_timer_softirq 0.06 ± 60% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.update_rt_rq_load_avg 0.04 ± 58% +0.1 0.10 ± 19% perf-profile.children.cycles-pp.hrtimer_forward 0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.pagevec_move_tail 0.06 ± 26% +0.1 0.13 ± 15% perf-profile.children.cycles-pp.__libc_read 0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.__free_slab 0.00 +0.1 0.07 ± 39% perf-profile.children.cycles-pp.propagate_protected_usage 0.00 +0.1 0.07 ± 31% perf-profile.children.cycles-pp.free_one_page 0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.sched_clock_idle_wakeup_event 0.00 +0.1 0.07 ± 25% perf-profile.children.cycles-pp.call_timer_fn 0.06 ± 14% +0.1 0.12 ± 20% perf-profile.children.cycles-pp.can_stop_idle_tick 0.01 ±173% +0.1 0.08 ± 43% perf-profile.children.cycles-pp.btrfs_get_8 0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.dup_mmap 0.04 ± 58% +0.1 0.11 ± 45% perf-profile.children.cycles-pp.pipe_write 0.06 ± 7% +0.1 0.13 ± 20% perf-profile.children.cycles-pp.rcu_eqs_exit 0.04 ± 60% +0.1 0.11 ± 13% perf-profile.children.cycles-pp.seq_read_iter 0.01 ±173% +0.1 0.08 ± 17% perf-profile.children.cycles-pp.local_touch_nmi 0.15 ± 4% +0.1 0.22 ± 5% perf-profile.children.cycles-pp.rb_insert_color 0.00 +0.1 0.07 ± 11% perf-profile.children.cycles-pp.__lookup_extent_mapping 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.dup_mm 0.00 +0.1 0.07 ± 17% perf-profile.children.cycles-pp.__dput_to_list 0.01 ±173% +0.1 0.09 ± 21% perf-profile.children.cycles-pp.rotate_reclaimable_page 0.00 +0.1 0.07 ± 20% perf-profile.children.cycles-pp.profile_tick 0.02 ±173% +0.1 0.10 ± 18% perf-profile.children.cycles-pp.perf_evsel__read 0.14 ± 5% +0.1 0.22 ± 15% perf-profile.children.cycles-pp.__might_sleep 0.00 +0.1 0.08 ± 15% perf-profile.children.cycles-pp.call_cpuidle 0.07 ± 22% +0.1 0.15 ± 14% perf-profile.children.cycles-pp.run_local_timers 0.00 +0.1 0.08 ± 26% perf-profile.children.cycles-pp.tlb_finish_mmu 0.00 +0.1 0.08 ± 27% perf-profile.children.cycles-pp.irq_work_needs_cpu 0.06 ± 14% +0.1 0.14 ± 12% perf-profile.children.cycles-pp.nr_iowait_cpu 0.06 ± 11% +0.1 0.14 ± 20% perf-profile.children.cycles-pp.rcu_nmi_enter 0.06 ± 14% +0.1 0.14 ± 31% perf-profile.children.cycles-pp.tick_check_broadcast_expired 0.03 ±100% +0.1 0.11 ± 28% perf-profile.children.cycles-pp.tick_check_oneshot_broadcast_this_cpu 0.03 ±100% +0.1 0.11 ± 12% perf-profile.children.cycles-pp.cpuidle_not_available 0.07 ± 11% +0.1 0.16 ± 22% perf-profile.children.cycles-pp.mmap_region 0.00 +0.1 0.08 ± 27% perf-profile.children.cycles-pp.up_read 0.01 ±173% +0.1 0.10 ± 28% perf-profile.children.cycles-pp.sched_setaffinity 0.01 ±173% +0.1 0.10 ± 28% perf-profile.children.cycles-pp.tick_nohz_idle_got_tick 0.06 ± 20% +0.1 0.14 ± 41% perf-profile.children.cycles-pp.setlocale 0.00 +0.1 0.08 ± 26% perf-profile.children.cycles-pp.__radix_tree_delete 0.05 +0.1 0.14 ± 33% perf-profile.children.cycles-pp.begin_new_exec 0.00 +0.1 0.09 ± 26% perf-profile.children.cycles-pp.tlb_flush_mmu 0.12 ± 5% +0.1 0.21 ± 5% perf-profile.children.cycles-pp.memset_erms 0.03 ±100% +0.1 0.11 ± 18% perf-profile.children.cycles-pp.tick_nohz_tick_stopped 0.03 ±100% +0.1 0.11 ± 25% perf-profile.children.cycles-pp.__etree_search 0.00 +0.1 0.09 ± 32% perf-profile.children.cycles-pp.__intel_pmu_enable_all 0.00 +0.1 0.09 ± 27% perf-profile.children.cycles-pp.free_extent_map 0.02 ±173% +0.1 0.11 ± 41% perf-profile.children.cycles-pp.test_clear_page_writeback 0.03 ±100% +0.1 0.11 ± 37% perf-profile.children.cycles-pp.intel_pmu_disable_all 0.00 +0.1 0.09 ± 23% perf-profile.children.cycles-pp.__do_munmap 0.01 ±173% +0.1 0.10 ± 18% perf-profile.children.cycles-pp.rcu_segcblist_enqueue 0.03 ±100% +0.1 0.12 ± 22% perf-profile.children.cycles-pp.rcu_gp_kthread 0.08 ± 8% +0.1 0.17 ± 22% perf-profile.children.cycles-pp.ksys_mmap_pgoff 0.03 ±100% +0.1 0.12 ± 14% perf-profile.children.cycles-pp._atomic_dec_and_lock 0.07 ± 11% +0.1 0.17 ± 23% perf-profile.children.cycles-pp.perf_event_task_tick 0.03 ±102% +0.1 0.13 ± 14% perf-profile.children.cycles-pp.rcu_nocb_try_bypass 0.03 ±100% +0.1 0.12 ± 26% perf-profile.children.cycles-pp.zap_pte_range 0.04 ± 58% +0.1 0.14 ± 18% perf-profile.children.cycles-pp.copy_process 0.07 ± 5% +0.1 0.17 ± 35% perf-profile.children.cycles-pp.do_fault 0.08 ± 6% +0.1 0.18 ± 21% perf-profile.children.cycles-pp.trigger_load_balance 0.09 ± 12% +0.1 0.19 ± 20% perf-profile.children.cycles-pp.do_mmap 0.03 ±100% +0.1 0.13 ± 20% perf-profile.children.cycles-pp.__free_pages_ok 0.11 ± 6% +0.1 0.21 ± 21% perf-profile.children.cycles-pp.rcu_eqs_enter 0.07 ± 16% +0.1 0.17 ± 18% perf-profile.children.cycles-pp.run_posix_cpu_timers 0.10 ± 21% +0.1 0.21 ± 18% perf-profile.children.cycles-pp.cmd_stat 0.10 ± 21% +0.1 0.21 ± 18% perf-profile.children.cycles-pp.__run_perf_stat 0.10 ± 21% +0.1 0.21 ± 18% perf-profile.children.cycles-pp.dispatch_events 0.10 ± 18% +0.1 0.21 ± 19% perf-profile.children.cycles-pp.process_interval 0.05 ± 62% +0.1 0.15 ± 18% perf-profile.children.cycles-pp.new_sync_read 0.07 ± 7% +0.1 0.17 ± 31% perf-profile.children.cycles-pp.__wake_up_bit 0.02 ±173% +0.1 0.12 ± 36% perf-profile.children.cycles-pp.__test_set_page_writeback 0.10 ± 22% +0.1 0.21 ± 19% perf-profile.children.cycles-pp.read_counters 0.03 ±100% +0.1 0.13 ± 20% perf-profile.children.cycles-pp.__memcg_kmem_uncharge 0.00 +0.1 0.11 ± 42% perf-profile.children.cycles-pp.rwsem_mark_wake 0.11 ± 22% +0.1 0.21 ± 19% perf-profile.children.cycles-pp.__libc_start_main 0.06 ± 11% +0.1 0.17 ± 17% perf-profile.children.cycles-pp.kernel_clone 0.00 +0.1 0.11 ± 30% perf-profile.children.cycles-pp.down_read 0.10 ± 9% +0.1 0.20 ± 22% perf-profile.children.cycles-pp.vm_mmap_pgoff 0.03 ±100% +0.1 0.14 ± 27% perf-profile.children.cycles-pp.unmap_page_range 0.00 +0.1 0.11 ± 26% perf-profile.children.cycles-pp.tick_sched_do_timer 0.10 ± 21% +0.1 0.21 ± 19% perf-profile.children.cycles-pp.main 0.10 ± 21% +0.1 0.21 ± 19% perf-profile.children.cycles-pp.run_builtin 0.22 ± 6% +0.1 0.33 ± 29% perf-profile.children.cycles-pp.idle_cpu 0.07 ± 10% +0.1 0.18 ± 10% perf-profile.children.cycles-pp.irqentry_exit 0.06 ± 7% +0.1 0.17 ± 17% perf-profile.children.cycles-pp.__do_sys_clone 0.06 ± 66% +0.1 0.17 ± 21% perf-profile.children.cycles-pp.release_pages 0.01 ±173% +0.1 0.12 ± 37% perf-profile.children.cycles-pp.clear_inode 0.04 ± 59% +0.1 0.16 ± 23% perf-profile.children.cycles-pp.truncate_inode_pages_final 0.01 ±173% +0.1 0.13 ± 15% perf-profile.children.cycles-pp.rcu_nmi_exit 0.02 ±173% +0.1 0.13 ± 18% perf-profile.children.cycles-pp.blk_mq_submit_bio 0.10 ± 7% +0.1 0.22 ± 16% perf-profile.children.cycles-pp.irqentry_enter 0.07 ± 60% +0.1 0.18 ± 21% perf-profile.children.cycles-pp.btrfs_lookup_first_ordered_extent 0.03 ±100% +0.1 0.15 ± 25% perf-profile.children.cycles-pp.unmap_vmas 0.04 ± 59% +0.1 0.17 ± 23% perf-profile.children.cycles-pp.page_counter_cancel 0.06 ± 16% +0.1 0.18 ± 20% perf-profile.children.cycles-pp.page_counter_uncharge 0.12 ± 12% +0.1 0.24 ± 22% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.11 ± 13% +0.1 0.23 ± 22% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.05 ± 63% +0.1 0.18 ± 36% perf-profile.children.cycles-pp.deactivate_file_page 0.08 ± 10% +0.1 0.21 ± 21% perf-profile.children.cycles-pp.__libc_fork 0.02 ±173% +0.1 0.15 ± 21% perf-profile.children.cycles-pp.submit_bio_noacct 0.02 ±173% +0.1 0.15 ± 23% perf-profile.children.cycles-pp.submit_bio 0.10 ± 11% +0.1 0.23 ± 29% perf-profile.children.cycles-pp.hrtimer_get_next_event 0.07 ± 26% +0.1 0.21 ± 18% perf-profile.children.cycles-pp.irqtime_account_process_tick 0.10 ± 28% +0.1 0.23 ± 9% perf-profile.children.cycles-pp.list_lru_isolate_move 0.00 +0.1 0.14 ± 24% perf-profile.children.cycles-pp.inode_wait_for_writeback 0.14 ± 8% +0.1 0.28 ± 12% perf-profile.children.cycles-pp.__x86_retpoline_rax 0.05 ± 59% +0.1 0.20 ± 48% perf-profile.children.cycles-pp.timekeeping_max_deferment 0.12 ± 15% +0.2 0.27 ± 20% perf-profile.children.cycles-pp.read 0.06 ± 80% +0.2 0.21 ± 27% perf-profile.children.cycles-pp.end_page_writeback 0.07 ± 72% +0.2 0.22 ± 27% perf-profile.children.cycles-pp.btrfs_get_16 0.07 ± 14% +0.2 0.23 ± 20% perf-profile.children.cycles-pp.drain_obj_stock 0.11 ± 4% +0.2 0.28 ± 7% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust 0.12 ± 12% +0.2 0.28 ± 16% perf-profile.children.cycles-pp.rcu_idle_exit 0.10 ± 14% +0.2 0.27 ± 26% perf-profile.children.cycles-pp.load_elf_binary 0.17 ± 13% +0.2 0.33 ± 15% perf-profile.children.cycles-pp.cpuidle_governor_latency_req 0.12 ± 5% +0.2 0.29 ± 18% perf-profile.children.cycles-pp.native_irq_return_iret 0.09 ± 13% +0.2 0.26 ± 28% perf-profile.children.cycles-pp.menu_reflect 0.01 ±173% +0.2 0.18 ± 25% perf-profile.children.cycles-pp.d_shrink_del 0.10 ± 17% +0.2 0.28 ± 25% perf-profile.children.cycles-pp.exec_binprm 0.06 ± 74% +0.2 0.23 ± 15% perf-profile.children.cycles-pp.btrfs_map_bio 0.04 ± 58% +0.2 0.22 ± 39% perf-profile.children.cycles-pp.fsnotify_grab_connector 0.14 ± 7% +0.2 0.32 ± 27% perf-profile.children.cycles-pp.__hrtimer_get_next_event 0.14 ± 19% +0.2 0.32 ± 11% perf-profile.children.cycles-pp.serial8250_console_putchar 0.15 ± 7% +0.2 0.33 ± 13% perf-profile.children.cycles-pp.timerqueue_add 0.07 ± 76% +0.2 0.25 ± 28% perf-profile.children.cycles-pp.end_bio_extent_buffer_writepage 0.10 ± 5% +0.2 0.28 ± 27% perf-profile.children.cycles-pp.__x64_sys_exit_group 0.10 ± 5% +0.2 0.28 ± 27% perf-profile.children.cycles-pp.do_group_exit 0.10 ± 5% +0.2 0.28 ± 27% perf-profile.children.cycles-pp.do_exit 0.10 ± 18% +0.2 0.28 ± 18% perf-profile.children.cycles-pp.radix_tree_delete_item 0.14 ± 19% +0.2 0.32 ± 11% perf-profile.children.cycles-pp.wait_for_xmitr 0.17 ± 9% +0.2 0.35 ± 27% perf-profile.children.cycles-pp.lapic_next_deadline 0.14 ± 18% +0.2 0.33 ± 11% perf-profile.children.cycles-pp.uart_console_write 0.12 ± 22% +0.2 0.31 ± 25% perf-profile.children.cycles-pp.pagevec_lru_move_fn 0.15 ± 16% +0.2 0.34 ± 19% perf-profile.children.cycles-pp.vfs_read 0.17 ± 16% +0.2 0.36 ± 19% perf-profile.children.cycles-pp.ksys_read 0.23 ± 10% +0.2 0.42 ± 13% perf-profile.children.cycles-pp.mutex_lock 0.14 ± 18% +0.2 0.34 ± 11% perf-profile.children.cycles-pp.serial8250_console_write 0.07 ± 76% +0.2 0.27 ± 27% perf-profile.children.cycles-pp.btrfs_end_bio 0.12 ± 15% +0.2 0.32 ± 26% perf-profile.children.cycles-pp.bprm_execve 0.23 ± 7% +0.2 0.43 ± 23% perf-profile.children.cycles-pp.update_irq_load_avg 0.14 ± 7% +0.2 0.34 ± 12% perf-profile.children.cycles-pp.arch_cpu_idle_enter 0.04 ±107% +0.2 0.24 ± 50% perf-profile.children.cycles-pp.calc_global_load_tick 0.06 ± 58% +0.2 0.26 ± 34% perf-profile.children.cycles-pp.fsnotify_destroy_marks 0.15 ± 12% +0.2 0.35 ± 19% perf-profile.children.cycles-pp.io_serial_in 0.11 ± 4% +0.2 0.32 ± 23% perf-profile.children.cycles-pp.exit_mmap 0.08 ± 71% +0.2 0.29 ± 28% perf-profile.children.cycles-pp.blk_update_request 0.13 ± 16% +0.2 0.34 ± 28% perf-profile.children.cycles-pp.dentry_unlink_inode 0.16 ± 4% +0.2 0.38 ± 34% perf-profile.children.cycles-pp.__handle_mm_fault 0.08 ± 75% +0.2 0.31 ± 29% perf-profile.children.cycles-pp.scsi_io_completion 0.08 ± 75% +0.2 0.31 ± 29% perf-profile.children.cycles-pp.scsi_end_request 0.11 ± 4% +0.2 0.33 ± 27% perf-profile.children.cycles-pp.mmput 0.09 ± 72% +0.2 0.32 ± 29% perf-profile.children.cycles-pp.blk_done_softirq 0.27 ± 5% +0.2 0.50 ± 9% perf-profile.children.cycles-pp.___might_sleep 0.20 ± 5% +0.2 0.42 ± 14% perf-profile.children.cycles-pp.enqueue_hrtimer 0.17 ± 4% +0.2 0.40 ± 32% perf-profile.children.cycles-pp.handle_mm_fault 0.11 ± 25% +0.2 0.34 ± 26% perf-profile.children.cycles-pp.__remove_inode_hash 0.35 ± 8% +0.2 0.59 ± 12% perf-profile.children.cycles-pp.update_blocked_averages 0.14 ± 16% +0.2 0.38 ± 26% perf-profile.children.cycles-pp.execve 0.17 ± 13% +0.2 0.42 ± 22% perf-profile.children.cycles-pp.timerqueue_del 0.18 ± 15% +0.2 0.43 ± 24% perf-profile.children.cycles-pp.memcpy_toio 0.18 ± 15% +0.2 0.43 ± 24% perf-profile.children.cycles-pp.drm_fb_memcpy_dstclip 0.15 ± 18% +0.2 0.40 ± 26% perf-profile.children.cycles-pp.__x64_sys_execve 0.20 ± 14% +0.2 0.45 ± 15% perf-profile.children.cycles-pp.__hrtimer_next_event_base 0.15 ± 18% +0.2 0.40 ± 27% perf-profile.children.cycles-pp.do_execveat_common 0.18 ± 15% +0.2 0.43 ± 24% perf-profile.children.cycles-pp.drm_atomic_helper_commit 0.18 ± 15% +0.2 0.43 ± 24% perf-profile.children.cycles-pp.commit_tail 0.18 ± 15% +0.2 0.43 ± 24% perf-profile.children.cycles-pp.drm_atomic_helper_commit_tail 0.18 ± 15% +0.2 0.43 ± 24% perf-profile.children.cycles-pp.drm_atomic_helper_commit_planes 0.18 ± 15% +0.2 0.43 ± 24% perf-profile.children.cycles-pp.mgag200_simple_display_pipe_update 0.18 ± 15% +0.2 0.43 ± 24% perf-profile.children.cycles-pp.mgag200_handle_damage 0.18 ± 15% +0.3 0.44 ± 24% perf-profile.children.cycles-pp.drm_atomic_helper_dirtyfb 0.20 ± 5% +0.3 0.46 ± 27% perf-profile.children.cycles-pp.asm_exc_page_fault 0.10 ± 22% +0.3 0.35 ± 16% perf-profile.children.cycles-pp.tick_nohz_irq_exit 0.19 ± 5% +0.3 0.45 ± 29% perf-profile.children.cycles-pp.exc_page_fault 0.20 ± 13% +0.3 0.46 ± 23% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.19 ± 4% +0.3 0.45 ± 29% perf-profile.children.cycles-pp.do_user_addr_fault 0.16 ± 17% +0.3 0.42 ± 22% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.34 ± 7% +0.3 0.61 ± 13% perf-profile.children.cycles-pp.run_rebalance_domains 0.16 ± 21% +0.3 0.43 ± 28% perf-profile.children.cycles-pp.__slab_free 0.21 ± 12% +0.3 0.49 ± 17% perf-profile.children.cycles-pp.console_unlock 0.22 ± 13% +0.3 0.50 ± 17% perf-profile.children.cycles-pp.asm_sysvec_irq_work 0.22 ± 13% +0.3 0.50 ± 17% perf-profile.children.cycles-pp.sysvec_irq_work 0.22 ± 13% +0.3 0.50 ± 17% perf-profile.children.cycles-pp.__sysvec_irq_work 0.22 ± 13% +0.3 0.50 ± 17% perf-profile.children.cycles-pp.irq_work_run 0.22 ± 13% +0.3 0.50 ± 17% perf-profile.children.cycles-pp.irq_work_single 0.22 ± 13% +0.3 0.50 ± 17% perf-profile.children.cycles-pp.printk 0.22 ± 13% +0.3 0.50 ± 17% perf-profile.children.cycles-pp.vprintk_emit 0.17 ± 37% +0.3 0.45 ± 13% perf-profile.children.cycles-pp.btrfs_run_delayed_refs_for_head 0.13 ± 31% +0.3 0.42 ± 24% perf-profile.children.cycles-pp.alloc_extent_state 0.22 ± 5% +0.3 0.51 ± 13% perf-profile.children.cycles-pp.read_tsc 0.89 ± 2% +0.3 1.19 ± 10% perf-profile.children.cycles-pp.update_sd_lb_stats 0.24 ± 12% +0.3 0.54 ± 16% perf-profile.children.cycles-pp.irq_work_run_list 0.11 ± 21% +0.3 0.41 ± 32% perf-profile.children.cycles-pp.__destroy_inode 0.20 ± 18% +0.3 0.51 ± 28% perf-profile.children.cycles-pp.drm_fb_helper_dirty_work 0.19 ± 62% +0.3 0.51 ± 23% perf-profile.children.cycles-pp.check_inode_key 0.23 ± 11% +0.3 0.54 ± 24% perf-profile.children.cycles-pp.__remove_hrtimer 0.24 ± 8% +0.3 0.56 ± 16% perf-profile.children.cycles-pp.irqtime_account_irq 0.20 ± 15% +0.3 0.52 ± 19% perf-profile.children.cycles-pp.rb_erase 0.47 ± 5% +0.3 0.79 ± 15% perf-profile.children.cycles-pp.update_rq_clock 0.13 ± 21% +0.3 0.46 ± 16% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.92 ± 2% +0.3 1.25 ± 10% perf-profile.children.cycles-pp.find_busiest_group 0.23 ± 38% +0.3 0.56 ± 32% perf-profile.children.cycles-pp.start_kernel 0.00 +0.3 0.34 ± 45% perf-profile.children.cycles-pp.wake_up_q 0.26 ± 7% +0.3 0.59 ± 20% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.24 ± 36% +0.4 0.59 ± 14% perf-profile.children.cycles-pp.btrfs_run_delayed_refs 0.23 ± 35% +0.4 0.59 ± 14% perf-profile.children.cycles-pp.__btrfs_run_delayed_refs 0.18 ± 23% +0.4 0.55 ± 16% perf-profile.children.cycles-pp.rcu_cblist_dequeue 0.18 ± 21% +0.4 0.54 ± 16% perf-profile.children.cycles-pp.refill_obj_stock 0.32 ± 7% +0.4 0.69 ± 11% perf-profile.children.cycles-pp.native_sched_clock 0.12 ± 10% +0.4 0.49 ± 26% perf-profile.children.cycles-pp.mem_cgroup_from_obj 0.33 ± 7% +0.4 0.71 ± 11% perf-profile.children.cycles-pp.sched_clock 0.20 ± 31% +0.4 0.58 ± 25% perf-profile.children.cycles-pp.clear_record_extent_bits 0.17 ± 15% +0.4 0.55 ± 17% perf-profile.children.cycles-pp.inode_lru_isolate 0.18 ± 28% +0.4 0.58 ± 24% perf-profile.children.cycles-pp.alloc_extent_map 0.00 +0.4 0.41 ± 44% perf-profile.children.cycles-pp.rwsem_wake 0.16 ± 21% +0.4 0.57 ± 28% perf-profile.children.cycles-pp.clear_extent_bit 0.27 ± 14% +0.4 0.68 ± 16% perf-profile.children.cycles-pp.call_rcu 0.17 ± 20% +0.4 0.61 ± 27% perf-profile.children.cycles-pp.btrfs_inode_clear_file_extent_range 0.36 ± 5% +0.5 0.82 ± 18% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 1.14 ± 3% +0.5 1.60 ± 11% perf-profile.children.cycles-pp.load_balance 0.23 ± 29% +0.5 0.70 ± 27% perf-profile.children.cycles-pp.btrfs_qgroup_check_reserved_leak 0.41 ± 7% +0.5 0.88 ± 16% perf-profile.children.cycles-pp.sched_clock_cpu 0.21 ± 16% +0.5 0.70 ± 19% perf-profile.children.cycles-pp.btrfs_drop_inode 0.46 ± 9% +0.5 0.96 ± 8% perf-profile.children.cycles-pp.__list_del_entry_valid 1.12 ± 3% +0.5 1.62 ± 4% perf-profile.children.cycles-pp.kmem_cache_alloc 0.36 ± 16% +0.6 0.94 ± 19% perf-profile.children.cycles-pp.___d_drop 0.36 ± 17% +0.6 0.96 ± 19% perf-profile.children.cycles-pp.__d_drop 0.53 ± 13% +0.6 1.14 ± 12% perf-profile.children.cycles-pp.rebalance_domains 0.29 ± 14% +0.6 0.91 ± 16% perf-profile.children.cycles-pp.dentry_lru_isolate 0.33 ± 6% +0.6 0.97 ± 15% perf-profile.children.cycles-pp.list_lru_add 0.23 ± 14% +0.6 0.87 ± 24% perf-profile.children.cycles-pp.inode_lru_list_add 0.30 ± 19% +0.7 0.95 ± 19% perf-profile.children.cycles-pp.shrink_lock_dentry 0.52 ± 42% +0.7 1.20 ± 19% perf-profile.children.cycles-pp.read_extent_buffer 0.32 ± 28% +0.7 1.03 ± 24% perf-profile.children.cycles-pp.btrfs_drop_extent_cache 0.53 ± 8% +0.8 1.29 ± 13% perf-profile.children.cycles-pp.tick_nohz_next_event 0.34 ± 24% +0.8 1.11 ± 26% perf-profile.children.cycles-pp.__clear_extent_bit 0.62 ± 25% +0.8 1.42 ± 40% perf-profile.children.cycles-pp.irq_enter_rcu 0.38 ± 13% +0.8 1.23 ± 16% perf-profile.children.cycles-pp._raw_spin_trylock 0.00 +0.9 0.85 ± 50% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.83 ± 2% +0.9 1.77 ± 20% perf-profile.children.cycles-pp.scheduler_tick 0.71 ± 38% +1.0 1.67 ± 22% perf-profile.children.cycles-pp.btrfs_get_32 0.54 ± 57% +1.0 1.54 ± 28% perf-profile.children.cycles-pp.check_dir_item 0.48 ± 13% +1.1 1.53 ± 17% perf-profile.children.cycles-pp.__list_lru_walk_one 0.48 ± 13% +1.1 1.54 ± 17% perf-profile.children.cycles-pp.list_lru_walk_one 0.63 ± 3% +1.1 1.69 ± 24% perf-profile.children.cycles-pp.clockevents_program_event 0.81 ± 9% +1.1 1.96 ± 13% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 0.57 ± 14% +1.3 1.92 ± 21% perf-profile.children.cycles-pp.iput 1.56 ± 9% +1.5 3.10 ± 14% perf-profile.children.cycles-pp.do_softirq_own_stack 1.11 ± 18% +1.7 2.79 ± 31% perf-profile.children.cycles-pp.ktime_get 0.74 ± 22% +1.7 2.44 ± 21% perf-profile.children.cycles-pp.run_ksoftirqd 0.78 ± 21% +1.7 2.50 ± 20% perf-profile.children.cycles-pp.smpboot_thread_fn 1.57 ± 3% +1.8 3.38 ± 19% perf-profile.children.cycles-pp.update_process_times 1.81 ± 7% +1.9 3.68 ± 15% perf-profile.children.cycles-pp.irq_exit_rcu 1.68 ± 3% +2.0 3.65 ± 18% perf-profile.children.cycles-pp.tick_sched_handle 0.99 ± 17% +2.0 2.98 ± 20% perf-profile.children.cycles-pp.rcu_do_batch 1.13 ± 14% +2.0 3.14 ± 19% perf-profile.children.cycles-pp.rcu_core 0.96 ± 25% +2.1 3.04 ± 25% perf-profile.children.cycles-pp.btrfs_destroy_inode 1.11 ± 16% +2.1 3.21 ± 20% perf-profile.children.cycles-pp.kmem_cache_free 0.81 ± 22% +2.1 2.94 ± 26% perf-profile.children.cycles-pp.btrfs_evict_inode 1.85 ± 2% +2.4 4.25 ± 12% perf-profile.children.cycles-pp.menu_select 1.10 ± 25% +2.4 3.54 ± 25% perf-profile.children.cycles-pp.destroy_inode 2.08 ± 6% +2.5 4.58 ± 21% perf-profile.children.cycles-pp.tick_sched_timer 1.47 ± 57% +2.7 4.17 ± 25% perf-profile.children.cycles-pp.check_leaf 1.49 ± 57% +2.7 4.23 ± 25% perf-profile.children.cycles-pp.btree_csum_one_bio 1.37 ± 14% +2.8 4.12 ± 20% perf-profile.children.cycles-pp.__dentry_kill 0.00 +2.8 2.77 ± 39% perf-profile.children.cycles-pp.rwsem_down_read_slowpath 1.56 ± 57% +2.9 4.46 ± 25% perf-profile.children.cycles-pp.btrfs_submit_metadata_bio 1.56 ± 57% +2.9 4.47 ± 25% perf-profile.children.cycles-pp.submit_one_bio 1.15 ± 21% +2.9 4.06 ± 26% perf-profile.children.cycles-pp.evict 1.64 ± 63% +3.0 4.60 ± 32% perf-profile.children.cycles-pp.btrfs_write_and_wait_transaction 1.64 ± 63% +3.0 4.60 ± 32% perf-profile.children.cycles-pp.btrfs_write_marked_extents 1.64 ± 63% +3.0 4.60 ± 32% perf-profile.children.cycles-pp.__filemap_fdatawrite_range 1.59 ± 57% +3.0 4.57 ± 24% perf-profile.children.cycles-pp.submit_extent_page 1.67 ± 57% +3.1 4.79 ± 24% perf-profile.children.cycles-pp.write_one_eb 2.28 ± 13% +3.2 5.49 ± 16% perf-profile.children.cycles-pp.__softirqentry_text_start 1.73 ± 56% +3.2 4.96 ± 24% perf-profile.children.cycles-pp.btree_write_cache_pages 1.73 ± 56% +3.2 4.96 ± 24% perf-profile.children.cycles-pp.do_writepages 1.90 ± 60% +3.3 5.20 ± 29% perf-profile.children.cycles-pp.transaction_kthread 1.90 ± 60% +3.3 5.20 ± 29% perf-profile.children.cycles-pp.btrfs_commit_transaction 1.19 ± 7% +3.6 4.75 ± 49% perf-profile.children.cycles-pp.__btrfs_release_delayed_node 1.88 ± 15% +3.8 5.72 ± 20% perf-profile.children.cycles-pp.shrink_dentry_list 3.18 ± 6% +3.9 7.07 ± 21% perf-profile.children.cycles-pp.__hrtimer_run_queues 2.18 ± 15% +4.5 6.66 ± 19% perf-profile.children.cycles-pp.prune_dcache_sb 2.99 ± 10% +5.5 8.51 ± 20% perf-profile.children.cycles-pp.intel_idle 2.37 ± 23% +5.7 8.07 ± 26% perf-profile.children.cycles-pp.dispose_list 4.75 ± 7% +6.1 10.82 ± 23% perf-profile.children.cycles-pp.hrtimer_interrupt 2.56 ± 23% +6.1 8.66 ± 25% perf-profile.children.cycles-pp.prune_icache_sb 4.96 ± 7% +6.3 11.31 ± 22% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 2.52 ± 11% +6.6 9.11 ± 31% perf-profile.children.cycles-pp._raw_spin_lock 7.09 ± 7% +8.0 15.13 ± 19% perf-profile.children.cycles-pp.asm_call_sysvec_on_stack 7.68 ± 8% +9.5 17.19 ± 21% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 8.17 ± 9% +10.2 18.41 ± 20% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 4.74 ± 19% +10.6 15.33 ± 23% perf-profile.children.cycles-pp.super_cache_scan 4.74 ± 19% +10.6 15.34 ± 23% perf-profile.children.cycles-pp.drop_slab 4.74 ± 19% +10.6 15.34 ± 23% perf-profile.children.cycles-pp.drop_slab_node 4.74 ± 19% +10.6 15.34 ± 23% perf-profile.children.cycles-pp.shrink_slab 4.74 ± 19% +10.6 15.34 ± 23% perf-profile.children.cycles-pp.do_shrink_slab 5.67 ± 21% +12.2 17.87 ± 24% perf-profile.children.cycles-pp.proc_sys_call_handler 5.67 ± 21% +12.2 17.87 ± 24% perf-profile.children.cycles-pp.drop_caches_sysctl_handler 5.72 ± 21% +12.3 17.99 ± 24% perf-profile.children.cycles-pp.new_sync_write 5.73 ± 21% +12.3 18.00 ± 24% perf-profile.children.cycles-pp.vfs_write 5.73 ± 21% +12.3 18.00 ± 24% perf-profile.children.cycles-pp.ksys_write 5.73 ± 21% +12.3 18.02 ± 24% perf-profile.children.cycles-pp.write 0.01 ±173% +14.6 14.59 ± 35% perf-profile.children.cycles-pp.osq_lock 0.00 +14.6 14.60 ± 37% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 0.00 +15.5 15.53 ± 36% perf-profile.children.cycles-pp.rwsem_optimistic_spin 9.72 ± 10% +16.4 26.11 ± 21% perf-profile.children.cycles-pp.cpuidle_enter_state 9.75 ± 10% +16.4 26.18 ± 21% perf-profile.children.cycles-pp.cpuidle_enter 15.76 ± 5% +18.1 33.82 ± 15% perf-profile.children.cycles-pp.start_secondary 15.99 ± 5% +18.4 34.38 ± 15% perf-profile.children.cycles-pp.secondary_startup_64_no_verify 15.99 ± 5% +18.4 34.38 ± 15% perf-profile.children.cycles-pp.cpu_startup_entry 15.98 ± 5% +18.4 34.38 ± 15% perf-profile.children.cycles-pp.do_idle 52.28 ± 4% -45.4 6.86 ± 42% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.42 ± 12% -0.2 0.25 ± 50% perf-profile.self.cycles-pp.btrfs_get_token_32 0.25 ± 25% -0.1 0.11 ± 86% perf-profile.self.cycles-pp.__module_address 0.26 ± 15% -0.1 0.15 ± 56% perf-profile.self.cycles-pp.__account_scheduler_latency 0.14 ± 5% -0.1 0.03 ±105% perf-profile.self.cycles-pp.btrfs_tree_read_unlock 0.28 ± 4% -0.1 0.18 ± 47% perf-profile.self.cycles-pp.memmove 0.25 ± 7% -0.1 0.16 ± 18% perf-profile.self.cycles-pp.generic_bin_search 0.23 ± 6% -0.1 0.17 ± 24% perf-profile.self.cycles-pp.__list_add_valid 0.10 ± 8% -0.1 0.04 ±100% perf-profile.self.cycles-pp.send_call_function_single_ipi 0.07 ± 14% -0.0 0.03 ±100% perf-profile.self.cycles-pp.newidle_balance 0.08 ± 19% -0.0 0.03 ±105% perf-profile.self.cycles-pp.___perf_sw_event 0.12 ± 5% -0.0 0.08 ± 58% perf-profile.self.cycles-pp.__d_lookup_rcu 0.05 ± 8% +0.0 0.08 ± 14% perf-profile.self.cycles-pp.rcu_all_qs 0.07 ± 17% +0.0 0.11 ± 19% perf-profile.self.cycles-pp.__softirqentry_text_start 0.05 ± 8% +0.0 0.09 ± 13% perf-profile.self.cycles-pp.__mod_memcg_state 0.07 ± 10% +0.0 0.11 ± 26% perf-profile.self.cycles-pp.update_dl_rq_load_avg 0.05 ± 8% +0.0 0.10 ± 24% perf-profile.self.cycles-pp.cpuidle_governor_latency_req 0.06 ± 14% +0.0 0.10 ± 21% perf-profile.self.cycles-pp.rcu_needs_cpu 0.08 ± 13% +0.1 0.14 ± 20% perf-profile.self.cycles-pp.rb_next 0.06 ± 7% +0.1 0.11 ± 14% perf-profile.self.cycles-pp.get_cpu_device 0.04 ± 57% +0.1 0.10 ± 27% perf-profile.self.cycles-pp.enqueue_hrtimer 0.00 +0.1 0.06 ± 14% perf-profile.self.cycles-pp.__free_pages_ok 0.03 ±100% +0.1 0.08 ± 23% perf-profile.self.cycles-pp.x86_pmu_disable 0.07 ± 17% +0.1 0.12 ± 26% perf-profile.self.cycles-pp.rcu_eqs_enter 0.04 ± 58% +0.1 0.10 ± 18% perf-profile.self.cycles-pp.hrtimer_forward 0.03 ±100% +0.1 0.09 ± 16% perf-profile.self.cycles-pp.rcu_dynticks_eqs_enter 0.01 ±173% +0.1 0.07 ± 22% perf-profile.self.cycles-pp.pm_qos_read_value 0.08 ± 5% +0.1 0.14 ± 13% perf-profile.self.cycles-pp.update_blocked_averages 0.06 ± 60% +0.1 0.12 ± 10% perf-profile.self.cycles-pp.update_rt_rq_load_avg 0.00 +0.1 0.06 ± 17% perf-profile.self.cycles-pp.cpuidle_enter 0.00 +0.1 0.06 ± 17% perf-profile.self.cycles-pp.__dput_to_list 0.00 +0.1 0.06 ± 34% perf-profile.self.cycles-pp.propagate_protected_usage 0.01 ±173% +0.1 0.08 ± 10% perf-profile.self.cycles-pp.irqentry_enter 0.04 ± 57% +0.1 0.10 ± 8% perf-profile.self.cycles-pp.asm_call_sysvec_on_stack 0.00 +0.1 0.07 ± 21% perf-profile.self.cycles-pp.release_pages 0.00 +0.1 0.07 ± 19% perf-profile.self.cycles-pp.tick_nohz_irq_exit 0.14 ± 5% +0.1 0.21 ± 6% perf-profile.self.cycles-pp.rb_insert_color 0.13 ± 6% +0.1 0.20 ± 18% perf-profile.self.cycles-pp.__might_sleep 0.05 ± 9% +0.1 0.12 ± 20% perf-profile.self.cycles-pp.can_stop_idle_tick 0.00 +0.1 0.07 ± 17% perf-profile.self.cycles-pp.call_cpuidle 0.00 +0.1 0.07 ± 17% perf-profile.self.cycles-pp.hrtimer_next_event_without 0.00 +0.1 0.07 ± 11% perf-profile.self.cycles-pp.__lookup_extent_mapping 0.11 ± 6% +0.1 0.18 ± 8% perf-profile.self.cycles-pp.memset_erms 0.00 +0.1 0.07 ± 20% perf-profile.self.cycles-pp.profile_tick 0.06 ± 14% +0.1 0.13 ± 11% perf-profile.self.cycles-pp.nr_iowait_cpu 0.03 ±100% +0.1 0.10 ± 31% perf-profile.self.cycles-pp.tick_check_oneshot_broadcast_this_cpu 0.00 +0.1 0.08 ± 27% perf-profile.self.cycles-pp.arch_cpu_idle_enter 0.00 +0.1 0.08 ± 24% perf-profile.self.cycles-pp.local_touch_nmi 0.00 +0.1 0.08 ± 30% perf-profile.self.cycles-pp.up_read 0.00 +0.1 0.08 ± 34% perf-profile.self.cycles-pp.free_extent_map 0.01 ±173% +0.1 0.09 ± 32% perf-profile.self.cycles-pp.tick_nohz_idle_got_tick 0.00 +0.1 0.08 ± 19% perf-profile.self.cycles-pp.sched_clock 0.03 ±100% +0.1 0.10 ± 14% perf-profile.self.cycles-pp.cpuidle_not_available 0.05 ± 8% +0.1 0.13 ± 28% perf-profile.self.cycles-pp.trigger_load_balance 0.01 ±173% +0.1 0.09 ± 23% perf-profile.self.cycles-pp.btrfs_drop_extent_cache 0.04 ± 58% +0.1 0.12 ± 21% perf-profile.self.cycles-pp.rcu_nmi_enter 0.05 ± 8% +0.1 0.14 ± 13% perf-profile.self.cycles-pp.tick_irq_enter 0.01 ±173% +0.1 0.10 ± 15% perf-profile.self.cycles-pp.rcu_segcblist_enqueue 0.06 ± 11% +0.1 0.14 ± 14% perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt 0.08 ± 24% +0.1 0.16 ± 24% perf-profile.self.cycles-pp.irq_exit_rcu 0.04 ± 59% +0.1 0.13 ± 36% perf-profile.self.cycles-pp.tick_check_broadcast_expired 0.01 ±173% +0.1 0.10 ± 25% perf-profile.self.cycles-pp.__etree_search 0.07 ± 21% +0.1 0.15 ± 33% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state 0.00 +0.1 0.09 ± 16% perf-profile.self.cycles-pp.tick_sched_handle 0.00 +0.1 0.09 ± 16% perf-profile.self.cycles-pp.rcu_nocb_try_bypass 0.00 +0.1 0.09 ± 32% perf-profile.self.cycles-pp.__intel_pmu_enable_all 0.08 ± 16% +0.1 0.17 ± 10% perf-profile.self.cycles-pp.timerqueue_add 0.06 ± 11% +0.1 0.15 ± 20% perf-profile.self.cycles-pp.rcu_idle_exit 0.03 ±100% +0.1 0.11 ± 21% perf-profile.self.cycles-pp.page_counter_cancel 0.10 ± 15% +0.1 0.19 ± 12% perf-profile.self.cycles-pp.rebalance_domains 0.07 ± 7% +0.1 0.15 ± 31% perf-profile.self.cycles-pp.scheduler_tick 0.07 ± 17% +0.1 0.15 ± 32% perf-profile.self.cycles-pp.timerqueue_del 0.00 +0.1 0.09 ± 15% perf-profile.self.cycles-pp.tick_nohz_tick_stopped 0.05 ± 62% +0.1 0.14 ± 36% perf-profile.self.cycles-pp.__remove_hrtimer 0.07 ± 15% +0.1 0.17 ± 30% perf-profile.self.cycles-pp.sched_clock_cpu 0.08 ± 6% +0.1 0.17 ± 19% perf-profile.self.cycles-pp.clockevents_program_event 0.07 ± 11% +0.1 0.17 ± 25% perf-profile.self.cycles-pp.perf_event_task_tick 0.00 +0.1 0.10 ± 15% perf-profile.self.cycles-pp.rcu_nmi_exit 0.06 ± 6% +0.1 0.16 ± 34% perf-profile.self.cycles-pp.__wake_up_bit 0.06 ± 16% +0.1 0.16 ± 15% perf-profile.self.cycles-pp.irq_enter_rcu 0.17 ± 6% +0.1 0.28 ± 12% perf-profile.self.cycles-pp.update_rq_clock 0.08 ± 19% +0.1 0.18 ± 22% perf-profile.self.cycles-pp.get_next_timer_interrupt 0.08 ± 16% +0.1 0.18 ± 32% perf-profile.self.cycles-pp.__hrtimer_get_next_event 0.00 +0.1 0.10 ± 25% perf-profile.self.cycles-pp.__dentry_kill 0.12 ± 11% +0.1 0.23 ± 14% perf-profile.self.cycles-pp.load_balance 0.07 ± 17% +0.1 0.17 ± 18% perf-profile.self.cycles-pp.run_posix_cpu_timers 0.06 ± 11% +0.1 0.17 ± 27% perf-profile.self.cycles-pp.menu_reflect 0.10 ± 10% +0.1 0.21 ± 19% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler 0.10 ± 12% +0.1 0.21 ± 12% perf-profile.self.cycles-pp.irqtime_account_irq 0.04 ± 60% +0.1 0.15 ± 26% perf-profile.self.cycles-pp.truncate_inode_pages_final 0.08 ± 19% +0.1 0.19 ± 19% perf-profile.self.cycles-pp.sysvec_apic_timer_interrupt 0.08 ± 13% +0.1 0.20 ± 24% perf-profile.self.cycles-pp.iput 0.00 +0.1 0.12 ± 63% perf-profile.self.cycles-pp.rwsem_down_read_slowpath 0.09 ± 25% +0.1 0.20 ± 22% perf-profile.self.cycles-pp.tick_sched_timer 0.11 ± 13% +0.1 0.23 ± 20% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.07 ± 21% +0.1 0.20 ± 17% perf-profile.self.cycles-pp.irqtime_account_process_tick 0.09 ± 4% +0.1 0.22 ± 13% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust 0.09 ± 8% +0.1 0.22 ± 15% perf-profile.self.cycles-pp.__x86_retpoline_rax 0.04 ±106% +0.1 0.17 ± 22% perf-profile.self.cycles-pp.btrfs_get_16 0.00 +0.1 0.13 ± 31% perf-profile.self.cycles-pp.fsnotify_grab_connector 0.10 ± 15% +0.1 0.23 ± 10% perf-profile.self.cycles-pp.__sysvec_apic_timer_interrupt 0.05 ±124% +0.1 0.19 ± 24% perf-profile.self.cycles-pp.check_inode_key 0.13 ± 9% +0.1 0.27 ± 17% perf-profile.self.cycles-pp.mutex_lock 0.07 ± 20% +0.1 0.22 ± 21% perf-profile.self.cycles-pp.inode_lru_isolate 0.12 ± 22% +0.1 0.27 ± 25% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.06 ± 58% +0.1 0.21 ± 26% perf-profile.self.cycles-pp.btrfs_destroy_inode 0.12 ± 6% +0.2 0.28 ± 32% perf-profile.self.cycles-pp.update_process_times 0.07 ± 16% +0.2 0.23 ± 29% perf-profile.self.cycles-pp.evict 0.03 ±100% +0.2 0.19 ± 26% perf-profile.self.cycles-pp.__clear_extent_bit 0.12 ± 5% +0.2 0.29 ± 18% perf-profile.self.cycles-pp.native_irq_return_iret 0.09 ± 15% +0.2 0.27 ± 27% perf-profile.self.cycles-pp.__remove_inode_hash 0.14 ± 11% +0.2 0.32 ± 16% perf-profile.self.cycles-pp.list_lru_add 0.17 ± 9% +0.2 0.35 ± 27% perf-profile.self.cycles-pp.lapic_next_deadline 0.10 ± 27% +0.2 0.29 ± 16% perf-profile.self.cycles-pp.refill_obj_stock 0.23 ± 5% +0.2 0.42 ± 24% perf-profile.self.cycles-pp.update_irq_load_avg 0.11 ± 51% +0.2 0.30 ± 36% perf-profile.self.cycles-pp.check_dir_item 0.11 ± 15% +0.2 0.31 ± 29% perf-profile.self.cycles-pp.dentry_unlink_inode 0.04 ±107% +0.2 0.24 ± 50% perf-profile.self.cycles-pp.calc_global_load_tick 0.10 ± 27% +0.2 0.30 ± 25% perf-profile.self.cycles-pp.shrink_lock_dentry 0.15 ± 9% +0.2 0.36 ± 23% perf-profile.self.cycles-pp.hrtimer_interrupt 0.15 ± 12% +0.2 0.35 ± 19% perf-profile.self.cycles-pp.io_serial_in 0.18 ± 12% +0.2 0.39 ± 14% perf-profile.self.cycles-pp.__hrtimer_next_event_base 0.25 ± 4% +0.2 0.46 ± 9% perf-profile.self.cycles-pp.___might_sleep 0.23 ± 5% +0.2 0.45 ± 21% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.16 ± 18% +0.2 0.40 ± 29% perf-profile.self.cycles-pp.memcpy_toio 0.59 ± 5% +0.2 0.83 ± 10% perf-profile.self.cycles-pp.update_sd_lb_stats 0.14 ± 20% +0.2 0.39 ± 28% perf-profile.self.cycles-pp.__slab_free 0.16 ± 15% +0.3 0.42 ± 21% perf-profile.self.cycles-pp.call_rcu 0.20 ± 9% +0.3 0.46 ± 25% perf-profile.self.cycles-pp.__hrtimer_run_queues 0.40 ± 6% +0.3 0.67 ± 7% perf-profile.self.cycles-pp.kmem_cache_alloc 0.10 ± 13% +0.3 0.36 ± 18% perf-profile.self.cycles-pp.dentry_lru_isolate 0.21 ± 3% +0.3 0.48 ± 11% perf-profile.self.cycles-pp.read_tsc 0.11 ± 35% +0.3 0.39 ± 27% perf-profile.self.cycles-pp.btrfs_evict_inode 0.19 ± 11% +0.3 0.48 ± 15% perf-profile.self.cycles-pp.tick_nohz_next_event 0.18 ± 17% +0.3 0.47 ± 19% perf-profile.self.cycles-pp.rb_erase 0.23 ± 11% +0.3 0.52 ± 28% perf-profile.self.cycles-pp.do_idle 0.19 ± 4% +0.3 0.48 ± 17% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.18 ± 54% +0.3 0.51 ± 15% perf-profile.self.cycles-pp.check_leaf 0.30 ± 7% +0.3 0.63 ± 11% perf-profile.self.cycles-pp.native_sched_clock 0.11 ± 6% +0.3 0.45 ± 29% perf-profile.self.cycles-pp.mem_cgroup_from_obj 0.18 ± 23% +0.4 0.53 ± 16% perf-profile.self.cycles-pp.rcu_cblist_dequeue 0.26 ± 8% +0.4 0.69 ± 21% perf-profile.self.cycles-pp.__btrfs_release_delayed_node 0.18 ± 17% +0.5 0.64 ± 19% perf-profile.self.cycles-pp.btrfs_drop_inode 0.43 ± 8% +0.5 0.89 ± 8% perf-profile.self.cycles-pp.__list_del_entry_valid 0.37 ± 17% +0.5 0.92 ± 18% perf-profile.self.cycles-pp.cpuidle_enter_state 0.32 ± 18% +0.6 0.88 ± 18% perf-profile.self.cycles-pp.___d_drop 0.50 ± 42% +0.6 1.14 ± 18% perf-profile.self.cycles-pp.read_extent_buffer 0.54 ± 36% +0.7 1.25 ± 24% perf-profile.self.cycles-pp.btrfs_get_32 0.35 ± 13% +0.8 1.14 ± 17% perf-profile.self.cycles-pp._raw_spin_trylock 0.00 +0.8 0.82 ± 50% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.81 ± 8% +1.0 1.80 ± 16% perf-profile.self.cycles-pp.menu_select 1.25 ± 10% +1.1 2.38 ± 13% perf-profile.self.cycles-pp._raw_spin_lock 0.65 ± 14% +1.2 1.84 ± 19% perf-profile.self.cycles-pp.kmem_cache_free 0.93 ± 20% +1.4 2.32 ± 35% perf-profile.self.cycles-pp.ktime_get 2.99 ± 10% +5.5 8.51 ± 20% perf-profile.self.cycles-pp.intel_idle 0.01 ±173% +14.0 14.04 ± 36% perf-profile.self.cycles-pp.osq_lock fxmark.hdd_btrfs_MWCL_72_bufferedio.sys_sec 1200 +--------------------------------------------------------------------+ | | 1000 |++ .+. .+.+. .+. .+.+. +.+.+. .+.+.+.+. .+. .+.| | +.+ +.+.++ +.+ + +.+.+ + +.+.+.++ +.+ | | | 800 |-+ | | | 600 |-+ | | O O O | 400 |-+ O O O O O O O O O O O OO O O O O O O O O O O O O | | O O O O O | | | 200 |-+ | | | 0 +--------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_72_bufferedio.idle_sec 1800 +--------------------------------------------------------------------+ | O O O O O O | 1600 |-+ O O O O O O O O O O OO O O O O O O O O O O OO O O O | 1400 |-+ | | | 1200 |-+ | 1000 |.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.| | | 800 |-+ | 600 |-+ | | | 400 |-+ | 200 |-+ | | | 0 +--------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_72_bufferedio.sys_util 60 +----------------------------------------------------------------------+ | | 50 |.+ | | +.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+. .+.+.+.+.| | + | 40 |-+ | | | 30 |-+ | | | 20 |-+ O O O O O OO O O O O O O O O O O O O O O O | | O O O O O O O O O O O | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_72_bufferedio.idle_util 80 +----------------------------------------------------------------------+ | O O O O O O O O O O OO O O O O O O O O O O O O O O O | 70 |-+ | 60 |-+ | | | 50 |.+.+. .+.+.+. .+.+. .+. .+.+.+ .+. .+.+.+.+.+. .+.+. | | +.+ +.+ + +.+ +.+.+ +.+.+.+ + +.| 40 |-+ | | | 30 |-+ | 20 |-+ | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_63_bufferedio.sys_sec 1000 +--------------------------------------------------------------------+ 900 |-+.+.+.+.+.++. .+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.| | + | 800 |-+ | 700 |-+ | | | 600 |-+ O O | 500 |-+ O O O O O O O O O O O O O O | 400 |-+ O O O | | O O O O OO O O O O O O O O | 300 |-+ | 200 |-+ | | | 100 |-+ | 0 +--------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_63_bufferedio.idle_sec 1400 +--------------------------------------------------------------------+ | O O O | 1200 |-+ O O O O O O O O O OO O O O | | | 1000 |-+ | | .+. .+. +. | 800 |.+.+.+.+.+.++ +.+.+.+.+.+ +.+.+ +.+.+.+.+.+.+.+.+.+.++.+.+.+.+.| | | 600 |-+ | | | 400 |-+ | | | 200 |-+ | | | 0 +--------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_63_bufferedio.sys_util 60 +----------------------------------------------------------------------+ | | 50 |.+.+.+.+.+.+.+. .+.+.+.+.+.+. .+.+.+ .+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.| | + + + | | | 40 |-+ | | | 30 |-+ O O O O O O O O O O O O O O | | O O | 20 |-O O O O O O O O O O O O O O O O O | | | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_63_bufferedio.idle_util 80 +----------------------------------------------------------------------+ | O O O O O O O O O O O O O O O O O | 70 |-+ O O O O O O O O O O O O O O O O | 60 |-+ | | | 50 |-+ | |.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.| 40 |-+ | | | 30 |-+ | 20 |-+ | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_54_bufferedio.sys_sec 1000 +--------------------------------------------------------------------+ 900 |.+ | | +. .+.+. .++. .+.+.+.+.+.+.+.+.+.++.+.+. .+.+.+.+.+. .+. .+.+.+.| 800 |-+ + + + + + ++.+ | 700 |-+ | | | 600 |-+ | 500 |-+ O O O O O O O O O OO O O O | 400 |-+ O O | | O O O O O OO O O O O O O O O O O | 300 |-+ | 200 |-+ | | | 100 |-+ | 0 +--------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_54_bufferedio.idle_sec 1200 +--------------------------------------------------------------------+ | O O OO O O O O O | 1000 |-+ O O O O O O O O O O OO O O O | | O | | | 800 |-+ | | .+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+. .+. .+. .++.+.+.+.+.| 600 |.+ + + + | | | 400 |-+ | | | | | 200 |-+ | | | 0 +--------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_54_bufferedio.sys_util 60 +----------------------------------------------------------------------+ |.+. .+. .+. .+. .+. .+.+.+. .+ .+. .+.+.+.+.+. .+. .+.| 50 |-+ + +.+.+ +.+ + + + + +.+ + +.+.+.+.+ | | | | | 40 |-+ | | O | 30 |-+ O O O O O O O O O O O O O O O | | | 20 |-O O O O O O O O O O O O O O O O O | | | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_54_bufferedio.idle_util 80 +----------------------------------------------------------------------+ | O O O O O O O O O O O O O O O O O | 70 |-+ O | 60 |-+ O O O O O O O O O O O O O O O | | | 50 |-+ | | .+. .+. .+. .+. .+.+.+. | 40 |.+.+.+.+ +.+ +.+.+.+.+.+.+.+.+.++.+.+ +.+.+.+.+ + +.+.+.| | | 30 |-+ | 20 |-+ | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_45_bufferedio.sys_sec 800 +---------------------------------------------------------------------+ | +.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+. .+.+.+.++.+.+.+.+.+.+.+.| 700 |-+ + | 600 |-+ | | | 500 |-+ | | O O O O O | 400 |-+ O O O O O O O OO O O | | | 300 |-O O O O O O O O O O O O O O O O O | 200 |-+ | | | 100 |-+ | | | 0 +---------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_45_bufferedio.idle_sec 1000 +--------------------------------------------------------------------+ 900 |-O O O O O O O O O | | O O O O O O | 800 |-+ O O O O O OO O O O | 700 |-+ | | | 600 |-+ | 500 |.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+. .++.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.| 400 |-+ + | | | 300 |-+ | 200 |-+ | | | 100 |-+ | 0 +--------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_45_bufferedio.sys_util 70 +----------------------------------------------------------------------+ | | 60 |.+ .+.+. .+. .+.+. .+. .+.+.+. .+.+. .+. .+. .+. .+. | | +.+.+ + +.+ + + ++.+ +.+.+.+ + + + +.+.| 50 |-+ | | | 40 |-+ | | O O O O O O O O | 30 |-+ O O O O O O O O | | | 20 |-O O O O O O O O O O O O O O O O O | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_45_bufferedio.idle_util 80 +----------------------------------------------------------------------+ | O O O O O O O O O O O O O O O O O | 70 |-+ | 60 |-+ O O O O O O O O O O O O O | | O O O | 50 |-+ | | | 40 |.+.+.+. .+. .+.+. .+. .+. .++.+. .+.+.+.+. .+. .+.+.+. .+.+.| | +.+ + +.+ + +.+.+ +.+ + + + | 30 |-+ | 20 |-+ | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_36_bufferedio.sys_sec 700 +---------------------------------------------------------------------+ | +.+. .+. .+. .+.++.+. .+. .+. .+.+. .+. .+.++.+.+. .+.+.+.+.| 600 |-+ + + + + +.+.+ +.+ +.+ + + | | | 500 |-+ | | | 400 |-+ O O O O | | O O O O O O O OO O O O | 300 |-+ | | O O O O O O O O O O O O | 200 |-+ O O O O O | | | 100 |-+ | | | 0 +---------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_36_bufferedio.idle_sec 800 +---------------------------------------------------------------------+ | O O O O O O O O O O O O O O O O O | 700 |-+ | 600 |-+ O O O O O O O O O O | | O OO O O O | 500 |-+ | | | 400 |-+ | |.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.| 300 |-+ | 200 |-+ | | | 100 |-+ | | | 0 +---------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_36_bufferedio.sys_util 70 +----------------------------------------------------------------------+ |. .+. | 60 |-+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+ +.+.| | | 50 |-+ | | | 40 |-+ | | O O O O O O O O O O O O O O O O | 30 |-+ | | O O O | 20 |-O O O O O O O O O O O O O O | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_36_bufferedio.idle_util 80 +----------------------------------------------------------------------+ | O O O O O O O O O O O O O O O O O | 70 |-+ | 60 |-+ O O | | O O O O O O O O O O O O O O | 50 |-+ | | | 40 |-+ | |.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.| 30 |-+ | 20 |-+ | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_27_bufferedio.works 2e+06 +-----------------------------------------------------------------+ 1.8e+06 |.+ | | +.+.++.+.+.+.+.++.+.+.+.+.++.+.+.+.+.++.+.+.+.+.++.+.+.+.+.++.+.| 1.6e+06 |-+ | 1.4e+06 |-+ | | O O OO O O O O OO O O O O OO | 1.2e+06 |-+ | 1e+06 |-O O OO O O O O O O O O O OO O O | 800000 |-+ | | | 600000 |-+ | 400000 |-+ | | | 200000 |-+ | 0 +-----------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_27_bufferedio.works_sec 70000 +-------------------------------------------------------------------+ | | 60000 |.+ .+. .+. | | +.+.+.++.+.+.+.+.+.+.+.++.+ +.+ +.+.++.+.+.+.+.+.+.+.++.+.+.+.| 50000 |-+ | | O O O O O O O OO O O | 40000 |-+ O O O O O | | O O O OO O O O O O O OO O O O O | 30000 |-+ | | | 20000 |-+ | | | 10000 |-+ | | | 0 +-------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_27_bufferedio.sys_sec 600 +---------------------------------------------------------------------+ | | 500 |++ .+. .+. .+. .+ .+. .+.| | +.+.+ +.+.+.+.++.+.+.+.+.+ +.+.+.+ +.+.+.+.+ + +.+.+.+.+ | | | 400 |-+ | | | 300 |-+ O O O O O O O O OO O O O O O O | | | 200 |-O O O O O O O O O O O O O O O O O | | | | | 100 |-+ | | | 0 +---------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_27_bufferedio.idle_sec 600 +---------------------------------------------------------------------+ | | 500 |-O O O O O O O O O O O O O O O O O | | | | O O O O O O O O O O O O O O | 400 |-+ O O | | | 300 |-+ | |.+. .+.+.+. .+ .+. .+. .+. .+. .+. .+. +. .+. .+.+. .| 200 |-+ +.+.+ + + + +.+.+ + +.+ + +.+.+ + +.+ + | | | | | 100 |-+ | | | 0 +---------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_27_bufferedio.sys_util 70 +----------------------------------------------------------------------+ |+ .+.+.+. .+. .+. .+. .+.+.+. .+ .+.+. .+. .+.+.+. .+. .+.+. .+.| 60 |-+ +.+.+ + + + + + + + + + +.+ | | | 50 |-+ | | | 40 |-+ O O O O O O O O O O O O | | O O O O | 30 |-+ | | O O O O O O O O O O O O O O O O O | 20 |-+ | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_27_bufferedio.idle_util 70 +----------------------------------------------------------------------+ | O O O | 60 |-+ | | O O O O O O O O O O O O O O | 50 |-+ O O | | | 40 |-+ | | .+. | 30 |.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+ +.+.| | | 20 |-+ | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_18_bufferedio.works 2e+06 +-----------------------------------------------------------------+ 1.8e+06 |.+ | | +.+.++.+.+.+.+.++.+.+.+.+.++.+.+.+.+.++.+.+.+.+.++.+.+.+.+.++.+.| 1.6e+06 |-+ | 1.4e+06 |-+ | | O O | 1.2e+06 |-+ O O OO O O O OO O O O OO | 1e+06 |-O O OO O O O O O O O O O OO O O | 800000 |-+ | | | 600000 |-+ | 400000 |-+ | | | 200000 |-+ | 0 +-----------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_18_bufferedio.works_sec 70000 +-------------------------------------------------------------------+ | | 60000 |.+ | | +.+.+.++.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.++.+.+.+.| 50000 |-+ | | | 40000 |-+ O O O OO O O O O O O O OO O O | | O O O O O | 30000 |-+ O O OO O O O OO O O O | | | 20000 |-+ | | | 10000 |-+ | | | 0 +-------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_18_bufferedio.user_sec 14 +----------------------------------------------------------------------+ | | 12 |.+ .+. .+. +. | | +.+.+ +.+.+.+.+.+.+.+.+.+.+.+ + +.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.| 10 |-+ | | O O O O O O O O O O O O O O O O | 8 |-+ O O O O | | O O O O O O O O O O O O O | 6 |-+ | | | 4 |-+ | | | 2 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_18_bufferedio.sys_sec 400 +---------------------------------------------------------------------+ |. | 350 |-+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.| 300 |-+ | | | 250 |-+ | | O O O O O O O O OO O O O O O O | 200 |-+ | | O O O O O O O O O O O O O O O O O | 150 |-+ | 100 |-+ | | | 50 |-+ | | | 0 +---------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_18_bufferedio.idle_sec 350 +---------------------------------------------------------------------+ | | 300 |-O O O O O O O O O O O O O O O O O | | | 250 |-+ O O O O O | | O O O O O O O O O O O | 200 |-+ | | | 150 |-+ | |.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.| 100 |-+ | | | 50 |-+ | | | 0 +---------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_18_bufferedio.user_util 2.5 +---------------------------------------------------------------------+ |. .+. .+. .+. .+. .+. | | +.+.+ +.+.+.+.++ + +.+.+ + +.+.+.+.+.+.+.++.+.+.+.+.+.+.+.| 2 |-+ | | O O O O O O O | | O O O O O O O O O O | 1.5 |-O O O O O O O O O O O O O O O O | | | 1 |-+ | | | | | 0.5 |-+ | | | | | 0 +---------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_18_bufferedio.sys_util 80 +----------------------------------------------------------------------+ | | 70 |.+.+.+.+.+.+. .+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.| 60 |-+ + | | | 50 |-+ | | O O O O O O O O O O O O O O O O | 40 |-+ | | O O O O O O O O O O O O O O O O O | 30 |-+ | 20 |-+ | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.hdd_btrfs_MWCL_18_bufferedio.idle_util 60 +----------------------------------------------------------------------+ | O O O O | 50 |-+ | | O O O O O O O O O O O O O O O | | O | 40 |-+ | | | 30 |-+ | |.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.| 20 |-+ | | | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ fxmark.time.system_time 400 +---------------------------------------------------------------------+ |. .+.| 350 |-+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+ | 300 |-+ | | | 250 |-+ O O O O O O O O OO O O O O O O | | O O O O O O O O O O O O O O O O O | 200 |-+ | | | 150 |-+ | 100 |-+ | | | 50 |-+ | | | 0 +---------------------------------------------------------------------+ fxmark.time.percent_of_cpu_this_job_got 70 +----------------------------------------------------------------------+ |.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.++.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.| 60 |-+ | | | 50 |-+ O O O O O O O O O O O O O O O O | | O O O O O O O O O O O O O O O O O | 40 |-+ | | | 30 |-+ | | | 20 |-+ | | | 10 |-+ | | | 0 +----------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

1 year, 7 months

2
1
0 / 0

[drm/i915/gt] 8c3b1ba0e7: perf-sanity-tests.Parse_event_definition_strings.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813 ("drm/i915/gt: Track the overall awake/busy time") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: perf-sanity-tests version: with following parameters: perf_compiler: gcc ucode: 0xe2 on test machine: 4 threads Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz with 32G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> please be noted this was reported on last December https://lists.01.org/hyperkitty/list/[email protected]/thread/ZPLGWG47V7CI... since already merged into mainline and the issue still exist on current master HEAD, report again. 2020-12-24 12:50:16 make ARCH= EXTRA_CFLAGS=-fno-omit-frame-pointer -fsanitize=undefined -fsanitize=address -C /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf' BUILD: Doing 'make [33m-j4[m' parallel build HOSTCC fixdep.o HOSTLD fixdep-in.o LINK fixdep diff -u tools/include/uapi/drm/i915_drm.h include/uapi/drm/i915_drm.h diff -u tools/arch/x86/lib/memcpy_64.S arch/x86/lib/memcpy_64.S diff -u tools/arch/x86/lib/memset_64.S arch/x86/lib/memset_64.S Auto-detecting system features: ... dwarf: [ [32mon[m ] ... dwarf_getlocations: [ [32mon[m ] ... glibc: [ [32mon[m ] ... libbfd: [ [32mon[m ] ... libbfd-buildid: [ [32mon[m ] ... libcap: [ [32mon[m ] ... libelf: [ [32mon[m ] ... libnuma: [ [32mon[m ] ... numa_num_possible_cpus: [ [32mon[m ] ... libperl: [ [32mon[m ] ... libpython: [ [32mon[m ] ... libcrypto: [ [32mon[m ] ... libunwind: [ [32mon[m ] ... libdw-dwarf-unwind: [ [32mon[m ] ... zlib: [ [32mon[m ] ... lzma: [ [32mon[m ] ... get_cpuid: [ [32mon[m ] ... bpf: [ [32mon[m ] ... libaio: [ [32mon[m ] ... libzstd: [ [32mon[m ] ... disassembler-four-args: [ [32mon[m ] CC fd/array.o CC exec-cmd.o CC event-parse.o CC core.o LD fd/libapi-in.o CC fs/fs.o CC cpumap.o CC help.o CC fs/tracing_path.o CC fs/cgroup.o LD fs/libapi-in.o CC cpu.o CC threadmap.o CC debug.o CC pager.o CC str_error_r.o LD libapi-in.o AR libapi.a CC parse-options.o CC event-plugin.o CC evsel.o CC trace-seq.o CC parse-filter.o CC evlist.o CC mmap.o CC parse-utils.o CC kbuffer-parse.o CC zalloc.o CC tep_strerror.o CC event-parse-api.o CC xyarray.o LD libtraceevent-in.o LINK libtraceevent.a CC lib.o GEN bpf_helper_defs.h MKDIR staticobjs/ CC staticobjs/libbpf.o LD libperf-in.o AR libperf.a GEN common-cmds.h CC run-command.o CC sigchain.o CC subcmd-config.o LD libsubcmd-in.o MKDIR staticobjs/ AR libsubcmd.a CC staticobjs/bpf.o CC staticobjs/nlattr.o CC staticobjs/btf.o HOSTCC pmu-events/json.o HOSTCC pmu-events/jsmn.o HOSTCC pmu-events/jevents.o HOSTLD pmu-events/jevents-in.o CC plugin_jbd2.o LD plugin_jbd2-in.o CC plugin_hrtimer.o LD plugin_hrtimer-in.o CC plugin_kmem.o LD plugin_kmem-in.o CC plugin_kvm.o LD plugin_kvm-in.o CC staticobjs/libbpf_errno.o CC plugin_mac80211.o CC staticobjs/str_error.o CC staticobjs/netlink.o LD plugin_mac80211-in.o CC plugin_sched_switch.o LD plugin_sched_switch-in.o CC plugin_function.o CC staticobjs/bpf_prog_linfo.o LD plugin_function-in.o CC plugin_futex.o CC staticobjs/libbpf_probes.o LD plugin_futex-in.o CC plugin_xen.o CC staticobjs/xsk.o LD plugin_xen-in.o CC plugin_scsi.o CC staticobjs/hashmap.o LD plugin_scsi-in.o CC plugin_tlb.o CC plugin_cfg80211.o CC jvmti/libjvmti.o LD plugin_cfg80211-in.o GEN perf-archive LD plugin_tlb-in.o LINK plugin_jbd2.so GEN perf-with-kcore CC jvmti/jvmti_agent.o CC staticobjs/btf_dump.o LINK plugin_hrtimer.so LINK plugin_kmem.so LINK plugin_kvm.so LINK plugin_mac80211.so CC staticobjs/ringbuf.o LINK plugin_sched_switch.so LINK plugin_function.so LINK plugin_futex.so LINK plugin_xen.so LD staticobjs/libbpf-in.o LINK libbpf.a LINK plugin_scsi.so CC jvmti/libstring.o LINK plugin_cfg80211.so CC builtin-bench.o LINK plugin_tlb.so GEN libtraceevent-dynamic-list CC jvmti/libctype.o CC builtin-annotate.o CC builtin-config.o LD jvmti/jvmti-in.o CC builtin-diff.o LINK pmu-events/jevents CC builtin-evlist.o CC builtin-ftrace.o LINK libperf-jvmti.so CC builtin-help.o CC builtin-sched.o CC builtin-buildid-list.o CC builtin-buildid-cache.o CC builtin-kallsyms.o CC builtin-list.o CC builtin-record.o CC builtin-report.o CC builtin-stat.o CC builtin-timechart.o CC builtin-top.o CC builtin-script.o CC builtin-kmem.o CC builtin-lock.o CC builtin-kvm.o CC builtin-inject.o CC builtin-mem.o CC builtin-data.o CC builtin-version.o CC builtin-c2c.o CC builtin-trace.o CC builtin-probe.o CC bench/sched-messaging.o CC bench/sched-pipe.o CC bench/syscall.o CC bench/mem-functions.o CC bench/futex-hash.o CC bench/futex-wake.o CC bench/futex-wake-parallel.o CC bench/futex-requeue.o CC bench/futex-lock-pi.o CC tests/builtin-test.o CC bench/epoll-wait.o CC bench/epoll-ctl.o CC tests/parse-events.o CC util/annotate.o CC bench/synthesize.o CC bench/kallsyms-parse.o CC bench/find-bit-bench.o CC bench/inject-buildid.o CC bench/mem-memcpy-x86-64-asm.o CC bench/mem-memset-x86-64-asm.o CC bench/numa.o CC tests/dso-data.o CC tests/attr.o CC tests/vmlinux-kallsyms.o LD bench/perf-in.o CC arch/common.o CC tests/openat-syscall.o CC arch/x86/util/header.o CC tests/openat-syscall-all-cpus.o CC arch/x86/util/tsc.o CC arch/x86/util/pmu.o CC tests/openat-syscall-tp-fields.o CC arch/x86/util/kvm-stat.o CC tests/mmap-basic.o CC arch/x86/util/perf_regs.o CC arch/x86/util/topdown.o CC tests/perf-record.o CC arch/x86/util/machine.o CC arch/x86/tests/regs_load.o CC arch/x86/tests/dwarf-unwind.o CC util/block-info.o CC arch/x86/tests/arch-tests.o CC arch/x86/util/event.o CC arch/x86/tests/rdpmc.o CC tests/evsel-roundtrip-name.o CC arch/x86/util/dwarf-regs.o CC arch/x86/util/unwind-libunwind.o CC arch/x86/tests/perf-time-to-tsc.o CC arch/x86/util/auxtrace.o CC tests/evsel-tp-sched.o CC arch/x86/util/archinsn.o CC tests/fdarray.o CC arch/x86/tests/insn-x86.o CC arch/x86/util/intel-pt.o CC arch/x86/tests/intel-pt-pkt-decoder-test.o CC tests/pmu.o CC arch/x86/tests/bp-modify.o CC util/block-range.o CC tests/pmu-events.o LD arch/x86/tests/perf-in.o CC tests/hists_common.o CC tests/hists_link.o CC util/build-id.o CC ui/setup.o CC tests/hists_filter.o CC arch/x86/util/intel-bts.o CC ui/helpline.o CC ui/progress.o CC ui/util.o CC ui/hist.o CC tests/hists_output.o CC util/cacheline.o CC util/config.o LD arch/x86/util/perf-in.o LD arch/x86/perf-in.o LD arch/perf-in.o CC tests/hists_cumulate.o CC util/copyfile.o CC util/ctype.o CC util/db-export.o CC tests/python-use.o CC tests/bp_signal.o CC tests/bp_signal_overflow.o CC tests/bp_account.o CC tests/wp.o CC tests/task-exit.o CC tests/sw-clock.o CC tests/mmap-thread-lookup.o CC util/env.o CC scripts/perl/Perf-Trace-Util/Context.o CC tests/thread-maps-share.o CC util/event.o CC tests/switch-tracking.o LD scripts/perl/Perf-Trace-Util/perf-in.o CC scripts/python/Perf-Trace-Util/Context.o LD scripts/python/Perf-Trace-Util/perf-in.o LD scripts/perf-in.o CC trace/beauty/clone.o CC tests/keep-tracking.o CC trace/beauty/fcntl.o CC tests/code-reading.o CC trace/beauty/flock.o CC util/evlist.o CC trace/beauty/fsmount.o CC trace/beauty/fspick.o CC trace/beauty/ioctl.o CC trace/beauty/kcmp.o CC trace/beauty/mount_flags.o CC trace/beauty/move_mount.o CC trace/beauty/pkey_alloc.o CC tests/sample-parsing.o CC trace/beauty/arch_prctl.o CC trace/beauty/prctl.o CC trace/beauty/renameat.o CC trace/beauty/sockaddr.o CC trace/beauty/socket.o CC trace/beauty/statx.o CC tests/parse-no-sample-id-all.o CC trace/beauty/sync_file_range.o CC tests/kmod-path.o CC trace/beauty/tracepoints/x86_irq_vectors.o CC trace/beauty/tracepoints/x86_msr.o LD trace/beauty/tracepoints/perf-in.o LD trace/beauty/perf-in.o CC perf.o CC tests/thread-map.o CC tests/llvm.o CC tests/bpf.o CC util/sideband_evlist.o CC util/evsel.o CC util/evsel_fprintf.o CC tests/topology.o CC tests/mem.o CC tests/cpumap.o CC tests/stat.o CC tests/event_update.o CC ui/stdio/hist.o CC util/perf_event_attr_fprintf.o CC tests/event-times.o CC tests/expr.o CC util/evswitch.o CC tests/backward-ring-buffer.o CC util/find_bit.o CC tests/sdt.o CC util/get_current_dir_name.o CC util/kallsyms.o CC tests/is_printable_array.o CC tests/bitmap.o CC util/levenshtein.o CC ui/browser.o CC tests/perf-hooks.o CC tests/clang.o CC util/llvm-utils.o CC tests/unit_number__scnprintf.o CC tests/mem2node.o CC tests/maps.o CC util/mmap.o CC tests/time-utils-test.o CC util/memswap.o CC ui/browsers/annotate.o BISON util/parse-events-bison.c CC util/perf_regs.o CC util/path.o CC tests/genelf.o CC ui/browsers/hists.o CC tests/api-io.o CC tests/demangle-java-test.o CC tests/pfm.o CC tests/parse-metric.o CC util/print_binary.o CC util/rlimit.o CC tests/pe-file-parsing.o CC util/argv_split.o CC tests/expand-cgroup.o CC ui/browsers/map.o CC util/rbtree.o CC ui/browsers/scripts.o CC tests/dwarf-unwind.o CC ui/browsers/header.o CC tests/llvm-src-base.o CC tests/llvm-src-kbuild.o CC tests/llvm-src-prologue.o CC tests/llvm-src-relocation.o CC ui/browsers/res_sample.o LD tests/perf-in.o CC util/libstring.o CC util/bitmap.o CC util/hweight.o CC util/smt.o CC ui/tui/setup.o CC util/strbuf.o CC util/string.o CC ui/tui/util.o CC util/strlist.o CC util/strfilter.o CC ui/tui/helpline.o CC util/top.o CC ui/tui/progress.o LD ui/tui/perf-in.o CC util/usage.o CC util/dso.o CC util/dsos.o CC util/symbol.o CC util/symbol_fprintf.o CC util/color.o CC util/color_config.o CC util/metricgroup.o CC util/header.o CC util/callchain.o CC util/values.o CC util/debug.o CC util/fncache.o CC util/machine.o CC util/map.o CC util/pstack.o CC util/session.o CC util/sample-raw.o CC util/s390-sample-raw.o CC util/syscalltbl.o CC util/ordered-events.o CC util/namespaces.o LD ui/browsers/perf-in.o LD ui/perf-in.o CC util/comm.o CC util/thread.o CC util/thread_map.o CC util/trace-event-parse.o CC util/parse-events-bison.o BISON util/pmu-bison.c CC util/trace-event-read.o CC util/trace-event-info.o GEN python/perf.so CC util/trace-event-scripting.o CC util/trace-event.o CC util/svghelper.o CC util/sort.o CC util/hist.o CC util/util.o CC util/cpumap.o CC util/affinity.o CC util/cputopo.o CC util/cgroup.o CC util/target.o CC util/rblist.o CC util/intlist.o CC util/vdso.o CC util/counts.o CC util/stat.o CC util/stat-shadow.o CC util/stat-display.o CC util/perf_api_probe.o CC util/record.o CC util/srcline.o CC util/srccode.o CC util/synthetic-events.o CC util/data.o CC util/tsc.o CC util/cloexec.o CC util/call-path.o CC util/rwsem.o CC util/thread-stack.o CC util/spark.o CC util/topdown.o CC util/stream.o CC util/auxtrace.o CC util/intel-pt-decoder/intel-pt-pkt-decoder.o GEN util/intel-pt-decoder/inat-tables.c CC util/intel-pt-decoder/intel-pt-log.o CC util/intel-pt-decoder/intel-pt-decoder.o CC util/arm-spe-decoder/arm-spe-pkt-decoder.o CC util/arm-spe-decoder/arm-spe-decoder.o LD util/arm-spe-decoder/perf-in.o CC util/scripting-engines/trace-event-perl.o GEN pmu-events/pmu-events.c CC pmu-events/pmu-events.o CC util/scripting-engines/trace-event-python.o CC util/intel-pt.o LD pmu-events/pmu-events-in.o CC util/intel-bts.o CC util/arm-spe.o CC util/intel-pt-decoder/intel-pt-insn-decoder.o LD util/scripting-engines/perf-in.o CC util/s390-cpumsf.o CC util/parse-branch-options.o CC util/dump-insn.o CC util/parse-regs-options.o CC util/parse-sublevel-options.o CC util/term.o CC util/help-unknown-cmd.o CC util/mem-events.o CC util/vsprintf.o LD util/intel-pt-decoder/perf-in.o CC util/units.o CC util/time-utils.o BISON util/expr-bison.c CC util/branch.o CC util/mem2node.o CC util/clockid.o CC util/bpf-loader.o CC util/bpf_map.o CC util/bpf-prologue.o CC util/symbol-elf.o CC util/probe-file.o CC util/probe-event.o CC util/probe-finder.o CC util/dwarf-aux.o CC util/dwarf-regs.o CC util/unwind-libunwind-local.o CC util/unwind-libunwind.o CC util/data-convert-bt.o CC util/zlib.o CC util/lzma.o CC util/zstd.o CC util/cap.o CC util/demangle-java.o CC util/demangle-rust.o CC util/jitdump.o CC util/genelf.o CC util/genelf_debug.o CC util/perf-hooks.o CC util/bpf-event.o FLEX util/parse-events-flex.c FLEX util/pmu-flex.c CC util/pmu-bison.o FLEX util/expr-flex.c CC util/expr-bison.o CC util/expr.o CC util/parse-events.o CC util/parse-events-flex.o CC util/pmu.o CC util/pmu-flex.o CC util/expr-flex.o LD util/perf-in.o LD perf-in.o LINK perf make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf' 2020-12-24 12:51:30 cd /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf 2020-12-24 12:51:30 mkdir -p /pkg 2020-12-24 12:51:30 mkdir -p /kbuild/obj/consumer/x86_64-rhel-8.3 2020-12-24 12:51:30 cp /pkg/linux/x86_64-rhel-8.3/gcc-9/8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/vmlinux.xz /tmp 2020-12-24 12:51:31 unxz -k /tmp/vmlinux.xz 2020-12-24 12:51:35 cp /tmp/vmlinux /kbuild/obj/consumer/x86_64-rhel-8.3 ignored_by_lkp: BPF filter ignored_by_lkp: LLVM search and compile ignored_by_lkp: Add vfs_getname probe to get syscall args filenames ignored_by_lkp: Use vfs_getname probe to get syscall args filenames ignored_by_lkp: Check open filename arg using perf trace + vfs_getname ignored_by_lkp: builtin clang support 2020-12-24 12:51:35 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 1 1: vmlinux symtab matches kallsyms : Ok 2020-12-24 12:51:36 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 2 2: Detect openat syscall event : Ok 2020-12-24 12:51:36 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 3 3: Detect openat syscall event on all cpus : Ok 2020-12-24 12:51:36 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 4 4: Read samples using the mmap interface : FAILED! 2020-12-24 12:51:36 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 5 5: Test data source output : Ok 2020-12-24 12:51:36 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 6 6: Parse event definition strings : FAILED! 2020-12-24 12:51:37 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 7 7: Simple expression parser : Ok 2020-12-24 12:51:38 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 8 8: PERF_RECORD_* events & perf_sample fields : Ok 2020-12-24 12:51:40 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 9 9: Parse perf pmu format : Ok 2020-12-24 12:51:40 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 10 10: PMU events : 10.1: PMU event table sanity : Ok 10.2: PMU event map aliases : Ok 10.3: Parsing of PMU event table metrics : Skip (some metrics failed) 10.4: Parsing of PMU event table metrics with fake PMUs : Ok 2020-12-24 12:51:42 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 11 11: DSO data read : Ok 2020-12-24 12:51:42 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 12 12: DSO data cache : Ok 2020-12-24 12:51:42 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 13 13: DSO data reopen : Ok 2020-12-24 12:51:42 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 14 14: Roundtrip evsel->name : Ok 2020-12-24 12:51:42 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 15 15: Parse sched tracepoints fields : Ok 2020-12-24 12:51:42 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 16 16: syscalls:sys_enter_openat event fields : Ok 2020-12-24 12:51:42 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 17 17: Setup struct perf_event_attr : Ok 2020-12-24 12:51:56 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 18 18: Match and link multiple hists : Ok 2020-12-24 12:51:56 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 19 19: 'import perf' in python : FAILED! 2020-12-24 12:51:56 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 20 20: Breakpoint overflow signal handler : Ok 2020-12-24 12:51:56 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 21 21: Breakpoint overflow sampling : Ok 2020-12-24 12:51:57 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 22 22: Breakpoint accounting : Ok 2020-12-24 12:51:57 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 23 23: Watchpoint : 23.1: Read Only Watchpoint : Skip 23.2: Write Only Watchpoint : Ok 23.3: Read / Write Watchpoint : Ok 23.4: Modify Watchpoint : Ok 2020-12-24 12:51:57 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 24 24: Number of exit events of a simple workload : FAILED! 2020-12-24 12:51:57 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 25 25: Software clock events period values : FAILED! 2020-12-24 12:51:57 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 26 26: Object code reading : FAILED! 2020-12-24 12:52:00 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 27 27: Sample parsing : Ok 2020-12-24 12:52:00 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 28 28: Use a dummy software event to keep tracking : FAILED! 2020-12-24 12:52:00 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 29 29: Parse with no sample_id_all bit set : Ok 2020-12-24 12:52:00 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 30 30: Filter hist entries : Ok 2020-12-24 12:52:00 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 31 31: Lookup mmap thread : FAILED! 2020-12-24 12:52:01 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 32 32: Share thread maps : Ok 2020-12-24 12:52:01 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 33 33: Sort output of hist entries : Ok 2020-12-24 12:52:01 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 34 34: Cumulate child hist entries : Ok 2020-12-24 12:52:01 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 35 35: Track with sched_switch : FAILED! 2020-12-24 12:52:01 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 36 36: Filter fds with revents mask in a fdarray : Ok 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 37 37: Add fd to a fdarray, making it autogrow : Ok 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 38 38: kmod_path__parse : Ok 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 39 39: Thread map : Ok 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 41 41: Session topology : FAILED! 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 43 43: Synthesize thread map : FAILED! 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 44 44: Remove thread map : FAILED! 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 45 45: Synthesize cpu map : Ok 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 46 46: Synthesize stat config : Ok 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 47 47: Synthesize stat : Ok 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 48 48: Synthesize stat round : Ok 2020-12-24 12:52:02 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 49 49: Synthesize attr update : FAILED! 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 50 50: Event times : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 51 51: Read backward ring buffer : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 52 52: Print cpu map : FAILED! 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 53 53: Merge cpu map : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 54 54: Probe SDT events : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 55 55: is_printable_array : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 56 56: Print bitmap : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 57 57: perf hooks : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 59 59: unit_number__scnprintf : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 60 60: mem2node : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 61 61: time utils : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 62 62: Test jit_write_elf : Ok 2020-12-24 12:52:03 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 63 63: Test libpfm4 support : Skip (not compiled in) 2020-12-24 12:52:04 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 64 64: Test api io : Ok 2020-12-24 12:52:04 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 65 65: maps__merge_in : FAILED! 2020-12-24 12:52:04 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 66 66: Demangle Java : Ok 2020-12-24 12:52:04 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 67 67: Parse and process metrics : Ok 2020-12-24 12:52:04 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 68 68: PE file support : FAILED! 2020-12-24 12:52:04 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 69 69: Event expansion for cgroups : Ok 2020-12-24 12:52:04 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 70 70: x86 rdpmc : Ok 2020-12-24 12:52:04 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 71 71: Convert perf time to TSC : FAILED! 2020-12-24 12:52:04 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 72 72: DWARF unwind : FAILED! 2020-12-24 12:52:05 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 73 73: x86 instruction decoder - new instructions : Ok 2020-12-24 12:52:05 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 74 74: Intel PT packet decoder : Ok 2020-12-24 12:52:05 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 75 75: x86 bp modify : Ok 2020-12-24 12:52:05 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 76 76: probe libc's inet_pton & backtrace it with ping : Ok 2020-12-24 12:52:07 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 77 77: Check Arm CoreSight trace data recording and synthesized samples: Skip 2020-12-24 12:52:07 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 78 78: build id cache operations : FAILED! 2020-12-24 12:52:08 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-8c3b1ba0e7ea9a80b0ee4b4445ea59c806787813/tools/perf/perf test 81 81: Zstd perf.data compression/decompression : FAILED! To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml bin/lkp run compatible-job.yaml Thanks, Oliver Sang

1 year, 7 months

3
9
0 / 0

[sched] 4112549ee5: WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_fwd_prog_nr[rcutorture]

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 4112549ee56c230415ebc5bbfa15533185ceb2e6 ("[PATCH] sched: Optimize __calc_delta.") url: https://github.com/0day-ci/linux/commits/Josh-Don/sched-Optimize-__calc_d... base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git c5e6fc08feb2b88dc5dac2f3c817e1c2a4cafda4 in testcase: rcutorture version: with following parameters: runtime: 300s test: cpuhotplug torture_type: rcu test-description: rcutorture is rcutorture kernel module load/unload test. test-url: https://www.kernel.org/doc/Documentation/RCU/torture.txt on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +-------------------------------------------------------------------------+------------+------------+ | | c5e6fc08fe | 4112549ee5 | +-------------------------------------------------------------------------+------------+------------+ | INFO:rcu_sched_self-detected_stall_on_CPU | 0 | 18 | | RIP:write_comp_data | 0 | 4 | | RIP:rcu_scale_wait_shutdown[rcuscale] | 0 | 6 | | RIP:___might_sleep | 0 | 8 | | RIP:rcu_scale_reader[rcuscale] | 0 | 18 | | BUG:kernel_hang_in_test_stage | 0 | 20 | | RIP:to_kthread | 0 | 7 | | RIP:__sanitizer_cov_trace_pc | 0 | 8 | | RIP:rcu_all_qs | 0 | 3 | | RIP:check_kcov_mode | 0 | 11 | | RIP:test_bit | 0 | 4 | | WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_fwd_prog_nr[rcutorture] | 0 | 8 | | RIP:rcu_torture_fwd_prog_nr[rcutorture] | 0 | 8 | | WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_fwd_prog_cr[rcutorture] | 0 | 9 | | RIP:rcu_torture_fwd_prog_cr[rcutorture] | 0 | 9 | | RIP:__kasan_check_read | 0 | 6 | | INFO:rcu_sched_detected_stalls_on_CPUs/tasks | 0 | 5 | | RIP:torture_must_stop[torture] | 0 | 2 | | RIP:check_memory_region | 0 | 9 | | RIP:__sanitizer_cov_trace_const_cmp4 | 0 | 1 | | RIP:kthread_should_stop | 0 | 1 | +-------------------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 487.587615] ------------[ cut here ]------------ [ 487.588208] WARNING: CPU: 0 PID: 3527 at kernel/rcu/rcutorture.c:1969 rcu_torture_fwd_prog_nr+0x467/0x533 [rcutorture] [ 487.589407] Modules linked in: rcutorture torture [ 487.589953] CPU: 0 PID: 3527 Comm: rcu_torture_fwd Tainted: G W 5.11.0-00053-g4112549ee56c #6 [ 487.591092] RIP: 0010:rcu_torture_fwd_prog_nr+0x467/0x533 [rcutorture] [ 487.591823] Code: 80 3c 02 00 74 05 e8 13 6d 4c e1 ff 53 30 48 8b 74 24 20 48 89 c7 e8 97 d7 ff ff 4d 85 e4 49 89 c0 75 0a 48 83 f8 01 77 04 90 <0f> 0b 90 48 8b 54 24 40 4c 89 e1 48 c7 c6 00 20 02 a0 48 c7 c7 a0 [ 487.593721] RSP: 0018:ffff88812db27d80 EFLAGS: 00010297 [ 487.594328] RAX: 0000000000000000 RBX: ffffffffa0025820 RCX: ffff88814393a8c0 [ 487.595140] RDX: ffff88814393a8c0 RSI: 000000000001480d RDI: 000000000001480d [ 487.595942] RBP: ffff88812db27e60 R08: 0000000000000000 R09: 0000000000000000 [ 487.596741] R10: ffffed10297198b1 R11: ffff88814b8cc587 R12: 0000000000000000 [ 487.597488] R13: dffffc0000000000 R14: ffff88814393a8c0 R15: 0000000000000000 [ 487.598286] FS: 0000000000000000(0000) GS:ffff8881e8800000(0000) knlGS:0000000000000000 [ 487.599203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 487.599860] CR2: 000055b0091b8f68 CR3: 000000014ce44000 CR4: 00000000000406f0 [ 487.600665] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 487.601458] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 487.602258] Call Trace: [ 487.602560] ? __sanitizer_cov_trace_pc+0x1e/0x43 [ 487.603118] ? tick_nohz_full_enabled+0x8e/0x8e [rcutorture] [ 487.603777] ? __next_timer_interrupt+0x1e2/0x1e2 [ 487.604314] ? rcu_torture_boost_cb+0x3f/0x3f [rcutorture] [ 487.604941] ? __do_set_cpus_allowed+0x1e0/0x1e0 [ 487.605476] ? __kasan_check_write+0x14/0x16 [ 487.605965] rcu_torture_fwd_prog+0x106/0x198 [rcutorture] [ 487.606619] ? rcu_torture_fwd_prog_cr+0x5c4/0x5c4 [rcutorture] [ 487.607325] ? __kasan_check_read+0x11/0x13 [ 487.607765] ? write_comp_data+0x24/0x6f [ 487.608204] ? __sanitizer_cov_trace_pc+0x1e/0x43 [ 487.608738] ? __kthread_parkme+0x101/0x16d [ 487.609226] kthread+0x327/0x33b [ 487.609576] ? _raw_spin_unlock_irq+0x9/0x13 [ 487.610021] ? rcu_torture_fwd_prog_cr+0x5c4/0x5c4 [rcutorture] [ 487.610708] ? kthread_queue_delayed_work+0xcd/0xcd [ 487.611282] ret_from_fork+0x1f/0x30 [ 487.611713] ---[ end trace 2946c7758b8349f5 ]--- To reproduce: # build kernel cd linux cp config-5.11.0-00053-g4112549ee56c .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Oliver Sang

1 year, 7 months

1
0
0 / 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP February 2021