( this report provides more results comparing to previous "[sched/fair] b08e888d97: hackbench.throughput 46.6% improvement" https://lists.01.org/hyperkitty/list/[email protected]/thread/52PNXKW4JSWC... ) Greeting, FYI, we noticed a -30.0% regression of aim7.jobs-per-min due to commit: commit: b08e888d9715f01546da7a40ca3ff72b9d1a78cc ("sched/fair: Force progress on min_vruntime") https://git.kernel.org/cgit/linux/kernel/git/peterz/queue.git sched/wip.migrate in testcase: aim7 on test machine: 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory with following parameters: disk: 1BRD_48G fs: xfs test: sync_disk_rw load: 600 cpufreq_governor: performance ucode: 0xd000280 test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system. test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/ In addition to that, the commit also has significant impact on the following tests: +------------------+----------------------------------------------------------------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput 73.0% improvement | | test machine | 96 threads 2 sockets Ice Lake with 256G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | iterations=4 | | | mode=threads | | | nr_threads=100% | | | ucode=0xb000280 | +------------------+----------------------------------------------------------------------------------------------------------------------------+ | testcase: change | phoronix-test-suite: phoronix-test-suite.fio.SequentialRead.IO_uring.No.No.4KB.DefaultTestDirectory.mb_s 17.6% improvement | | test machine | 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 128G memory | | test parameters | cpufreq_governor=performance | | | option_a=Sequential Read | | | option_b=IO_uring | | | option_c=No | | | option_d=No | | | option_e=4KB | | | option_f=Default Test Directory | | | test=fio-1.14.1 | | | ucode=0x5003006 | +------------------+----------------------------------------------------------------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput 55.5% improvement | | test machine | 104 threads 2 sockets Skylake with 192G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | iterations=4 | | | mode=threads | | | nr_threads=100% | | | ucode=0x2006a0a | +------------------+----------------------------------------------------------------------------------------------------------------------------+ | testcase: change | phoronix-test-suite: phoronix-test-suite.schbench.32.32.usec,_99.9th_latency_percentile -60.6% improvement | | test machine | 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 128G memory | | test parameters | cpufreq_governor=performance | | | option_a=32 | | | option_b=32 | | | test=schbench-1.0.0 | | | ucode=0x5003006 | +------------------+----------------------------------------------------------------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput 46.6% improvement | | test machine | 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | iterations=4 | | | mode=threads | | | nr_threads=100% | | | ucode=0xd000280 | +------------------+----------------------------------------------------------------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput 82.2% improvement | | test machine | 88 threads 2 sockets Intel(R) Xeon(R) Gold 6238M CPU @ 2.10GHz with 128G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | iterations=8 | | | mode=process | | | nr_threads=50% | | | ucode=0x5003006 | +------------------+----------------------------------------------------------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com&gt; Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/1BRD_48G/xfs/x86_64-rhel-8.3/600/debian-10.4-x86_64-20200603.cgz/lkp-icl-2sp2/sync_disk_rw/aim7/0xd000280 commit: v5.16-rc6 b08e888d97 ("sched/fair: Force progress on min_vruntime") v5.16-rc6 b08e888d9715f01546da7a40ca3 ---------------- --------------------------- %stddev %change %stddev \ | \ 31678 -30.0% 22162 aim7.jobs-per-min 32519866 ± 17% +37.1% 44574624 ± 4% cpuidle..usage 1.23 ± 16% -49.2% 0.62 ± 3% iostat.cpu.iowait 1.24 ± 16% -0.6 0.63 ± 3% mpstat.cpu.all.iowait% 82252 ± 69% +85.2% 152366 ± 12% numa-meminfo.node0.AnonHugePages 299360 ± 35% +27.6% 381979 ± 4% numa-meminfo.node0.Inactive(anon) 74872 ± 35% +27.3% 95302 ± 4% numa-vmstat.node0.nr_inactive_anon 74872 ± 35% +27.3% 95303 ± 4% numa-vmstat.node0.nr_zone_inactive_anon 32393864 ± 17% +37.1% 44411366 ± 4% turbostat.C1 67556778 ± 8% +37.1% 92646376 ± 2% turbostat.IRQ 4.67 ± 20% +60.7% 7.50 ± 10% vmstat.procs.b 177.33 ± 19% -47.0% 94.00 ± 5% vmstat.procs.r 341160 ± 9% -13.6% 294615 ± 4% proc-vmstat.numa_hint_faults 14324 ± 27% +76.0% 25206 ± 33% proc-vmstat.numa_pages_migrated 14324 ± 27% +76.0% 25206 ± 33% proc-vmstat.pgmigrate_success 33413 ± 10% +30.3% 43523 ± 4% proc-vmstat.pgreuse 4.07 +4.7% 4.26 perf-stat.i.cpi 0.26 ± 2% -6.4% 0.24 ± 2% perf-stat.i.ipc 7276 ± 5% -19.8% 5838 ± 3% perf-stat.i.minor-faults 85.21 ± 3% +2.2 87.43 perf-stat.i.node-load-miss-rate% 7282 ± 5% -19.8% 5843 ± 3% perf-stat.i.page-faults 0.44 ± 2% -0.0 0.42 perf-stat.overall.branch-miss-rate% 21.77 -2.4 19.33 perf-stat.overall.cache-miss-rate% 4.05 +5.3% 4.26 perf-stat.overall.cpi 2060 +17.2% 2415 perf-stat.overall.cycles-between-cache-misses 0.25 -5.1% 0.23 perf-stat.overall.ipc 86.69 +1.0 87.68 perf-stat.overall.node-load-miss-rate% 48.06 -0.7 47.41 perf-stat.overall.node-store-miss-rate% 7222 ± 5% -19.6% 5804 ± 3% perf-stat.ps.minor-faults 7228 ± 5% -19.6% 5809 ± 3% perf-stat.ps.page-faults 6.548e+12 +34.7% 8.819e+12 perf-stat.total.instructions 64.94 -4.7 60.21 perf-profile.calltrace.cycles-pp.__mutex_lock.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq 67.22 -4.7 62.57 perf-profile.calltrace.cycles-pp.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync 63.73 -4.2 59.54 perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq 72.42 -2.8 69.62 perf-profile.calltrace.cycles-pp.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write 3.57 ± 5% -2.1 1.49 ± 4% perf-profile.calltrace.cycles-pp.file_write_and_wait_range.xfs_file_fsync.xfs_file_buffered_write.new_sync_write.vfs_write 3.44 ± 5% -2.0 1.39 ± 4% perf-profile.calltrace.cycles-pp.filemap_fdatawrite_wbc.file_write_and_wait_range.xfs_file_fsync.xfs_file_buffered_write.new_sync_write 3.43 ± 5% -2.0 1.38 ± 5% perf-profile.calltrace.cycles-pp.do_writepages.filemap_fdatawrite_wbc.file_write_and_wait_range.xfs_file_fsync.xfs_file_buffered_write 3.41 ± 5% -2.0 1.38 ± 5% perf-profile.calltrace.cycles-pp.xfs_vm_writepages.do_writepages.filemap_fdatawrite_wbc.file_write_and_wait_range.xfs_file_fsync 1.54 ± 6% -1.2 0.36 ± 70% perf-profile.calltrace.cycles-pp.iomap_writepage_map.write_cache_pages.iomap_writepages.xfs_vm_writepages.do_writepages 2.73 ± 8% -1.1 1.66 ± 19% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xlog_cil_insert_items.xlog_cil_commit.__xfs_trans_commit 1.69 ± 6% -1.1 0.62 ± 5% perf-profile.calltrace.cycles-pp.iomap_writepages.xfs_vm_writepages.do_writepages.filemap_fdatawrite_wbc.file_write_and_wait_range 1.68 ± 6% -1.1 0.62 ± 5% perf-profile.calltrace.cycles-pp.write_cache_pages.iomap_writepages.xfs_vm_writepages.do_writepages.filemap_fdatawrite_wbc 1.70 ± 4% -1.0 0.74 ± 5% perf-profile.calltrace.cycles-pp.submit_bio_noacct.iomap_submit_ioend.xfs_vm_writepages.do_writepages.filemap_fdatawrite_wbc 1.71 ± 5% -1.0 0.74 ± 4% perf-profile.calltrace.cycles-pp.iomap_submit_ioend.xfs_vm_writepages.do_writepages.filemap_fdatawrite_wbc.file_write_and_wait_range 1.70 ± 5% -1.0 0.74 ± 4% perf-profile.calltrace.cycles-pp.__submit_bio.submit_bio_noacct.iomap_submit_ioend.xfs_vm_writepages.do_writepages 2.42 ± 7% -0.7 1.72 ± 6% perf-profile.calltrace.cycles-pp.xlog_cil_insert_items.xlog_cil_commit.__xfs_trans_commit.xfs_vn_update_time.file_update_time 93.45 -0.7 92.78 perf-profile.calltrace.cycles-pp.write 2.18 ± 7% -0.7 1.52 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.xlog_cil_insert_items.xlog_cil_commit.__xfs_trans_commit.xfs_vn_update_time 93.30 -0.6 92.66 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 93.28 -0.6 92.65 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 4.96 ± 6% -0.6 4.36 ± 6% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_vn_update_time.file_update_time.xfs_file_write_checks.xfs_file_buffered_write 4.94 ± 6% -0.6 4.35 ± 6% perf-profile.calltrace.cycles-pp.xlog_cil_commit.__xfs_trans_commit.xfs_vn_update_time.file_update_time.xfs_file_write_checks 2.90 ± 2% -0.5 2.37 ± 2% perf-profile.calltrace.cycles-pp.xlog_wait_on_iclog.xlog_force_lsn.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write 2.71 ± 2% -0.5 2.21 ± 2% perf-profile.calltrace.cycles-pp.remove_wait_queue.xlog_wait_on_iclog.xlog_force_lsn.xfs_log_force_seq.xfs_file_fsync 3.08 ± 2% -0.5 2.59 ± 2% perf-profile.calltrace.cycles-pp.xlog_force_lsn.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write.new_sync_write 2.70 ± 2% -0.5 2.20 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.remove_wait_queue.xlog_wait_on_iclog.xlog_force_lsn.xfs_log_force_seq 2.69 ± 2% -0.5 2.20 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.remove_wait_queue.xlog_wait_on_iclog.xlog_force_lsn 0.61 ± 8% -0.3 0.27 ±100% perf-profile.calltrace.cycles-pp._raw_spin_lock.xlog_cil_insert_items.xlog_cil_commit.__xfs_trans_commit.xfs_iomap_write_unwritten 0.81 ± 4% -0.2 0.57 perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq 0.69 ± 7% -0.1 0.55 ± 7% perf-profile.calltrace.cycles-pp.xlog_cil_insert_items.xlog_cil_commit.__xfs_trans_commit.xfs_iomap_write_unwritten.xfs_end_ioend 2.00 ± 7% +0.3 2.29 ± 6% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_iomap_write_unwritten.xfs_end_ioend.xfs_end_io.process_one_work 1.99 ± 7% +0.3 2.28 ± 6% perf-profile.calltrace.cycles-pp.xlog_cil_commit.__xfs_trans_commit.xfs_iomap_write_unwritten.xfs_end_ioend.xfs_end_io 0.73 ± 18% +0.4 1.08 ± 13% perf-profile.calltrace.cycles-pp.load_balance.newidle_balance.pick_next_task_fair.__schedule.schedule 1.27 ± 12% +0.4 1.62 ± 9% perf-profile.calltrace.cycles-pp.wait_for_completion.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq 0.83 ± 16% +0.4 1.19 ± 13% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.wait_for_completion 0.80 ± 17% +0.4 1.16 ± 13% perf-profile.calltrace.cycles-pp.newidle_balance.pick_next_task_fair.__schedule.schedule.schedule_timeout 1.22 ± 12% +0.4 1.58 ± 9% perf-profile.calltrace.cycles-pp.schedule_timeout.wait_for_completion.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq 1.19 ± 12% +0.4 1.56 ± 9% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_for_completion.flush_workqueue.xlog_cil_push_now 1.18 ± 13% +0.4 1.54 ± 10% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.wait_for_completion.flush_workqueue 0.30 ±100% +0.5 0.76 ± 14% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair 0.30 ±100% +0.5 0.79 ± 14% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair.__schedule 1.11 ± 13% +0.5 1.59 ± 10% perf-profile.calltrace.cycles-pp.xfs_log_ticket_ungrant.xlog_cil_commit.__xfs_trans_commit.xfs_iomap_write_unwritten.xfs_end_ioend 1.08 ± 13% +0.5 1.58 ± 11% perf-profile.calltrace.cycles-pp.xfs_log_space_wake.xfs_log_ticket_ungrant.xlog_cil_commit.__xfs_trans_commit.xfs_iomap_write_unwritten 4.65 ± 6% +0.8 5.41 ± 7% perf-profile.calltrace.cycles-pp.ret_from_fork 4.65 ± 6% +0.8 5.41 ± 7% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 4.61 ± 6% +0.8 5.39 ± 7% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 4.53 ± 6% +0.8 5.33 ± 7% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 3.73 ± 8% +0.9 4.60 ± 8% perf-profile.calltrace.cycles-pp.xfs_end_io.process_one_work.worker_thread.kthread.ret_from_fork 3.73 ± 8% +0.9 4.60 ± 8% perf-profile.calltrace.cycles-pp.xfs_end_ioend.xfs_end_io.process_one_work.worker_thread.kthread 0.99 ± 19% +0.9 1.92 ± 12% perf-profile.calltrace.cycles-pp.xfs_trans_alloc_inode.xfs_iomap_write_unwritten.xfs_end_ioend.xfs_end_io.process_one_work 0.98 ± 19% +0.9 1.92 ± 12% perf-profile.calltrace.cycles-pp.xfs_trans_alloc.xfs_trans_alloc_inode.xfs_iomap_write_unwritten.xfs_end_ioend.xfs_end_io 0.97 ± 19% +0.9 1.91 ± 12% perf-profile.calltrace.cycles-pp.xfs_trans_reserve.xfs_trans_alloc.xfs_trans_alloc_inode.xfs_iomap_write_unwritten.xfs_end_ioend 0.96 ± 20% +1.0 1.91 ± 12% perf-profile.calltrace.cycles-pp.xfs_log_reserve.xfs_trans_reserve.xfs_trans_alloc.xfs_trans_alloc_inode.xfs_iomap_write_unwritten 0.92 ± 20% +1.0 1.88 ± 12% perf-profile.calltrace.cycles-pp.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve.xfs_trans_alloc.xfs_trans_alloc_inode 2.38 ± 16% +1.0 3.42 ± 14% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xfs_log_space_wake.xfs_log_ticket_ungrant.xlog_cil_commit 2.38 ± 16% +1.0 3.42 ± 14% perf-profile.calltrace.cycles-pp._raw_spin_lock.xfs_log_space_wake.xfs_log_ticket_ungrant.xlog_cil_commit.__xfs_trans_commit 1.03 ± 37% +1.1 2.10 ± 14% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve 1.03 ± 37% +1.1 2.10 ± 14% perf-profile.calltrace.cycles-pp._raw_spin_lock.xlog_grant_head_check.xfs_log_reserve.xfs_trans_reserve.xfs_trans_alloc 3.28 ± 10% +1.1 4.41 ± 8% perf-profile.calltrace.cycles-pp.xfs_iomap_write_unwritten.xfs_end_ioend.xfs_end_io.process_one_work.worker_thread 5.05 ± 6% +1.9 6.91 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq 5.12 ± 6% +1.9 6.98 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync 4.86 ± 8% +2.1 7.00 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write 4.82 ± 8% +2.1 6.95 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync 78.02 +2.3 80.34 perf-profile.calltrace.cycles-pp.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write.new_sync_write 0.59 ± 5% +2.6 3.20 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.remove_wait_queue.xlog_cil_force_seq.xfs_log_force_seq 0.59 ± 5% +2.6 3.22 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.remove_wait_queue.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync 0.60 ± 5% +2.6 3.23 ± 3% perf-profile.calltrace.cycles-pp.remove_wait_queue.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write 64.94 -4.7 60.21 perf-profile.children.cycles-pp.__mutex_lock 67.22 -4.7 62.58 perf-profile.children.cycles-pp.flush_workqueue 63.78 -4.2 59.59 perf-profile.children.cycles-pp.osq_lock 72.42 -2.8 69.62 perf-profile.children.cycles-pp.xlog_cil_push_now 3.58 ± 5% -2.1 1.49 ± 4% perf-profile.children.cycles-pp.file_write_and_wait_range 3.44 ± 5% -2.0 1.39 ± 4% perf-profile.children.cycles-pp.filemap_fdatawrite_wbc 3.43 ± 5% -2.0 1.38 ± 5% perf-profile.children.cycles-pp.do_writepages 3.41 ± 5% -2.0 1.38 ± 5% perf-profile.children.cycles-pp.xfs_vm_writepages 1.70 ± 6% -1.1 0.61 ± 6% perf-profile.children.cycles-pp.iomap_finish_ioend 1.67 ± 6% -1.1 0.59 ± 7% perf-profile.children.cycles-pp.folio_end_writeback 1.69 ± 6% -1.1 0.62 ± 5% perf-profile.children.cycles-pp.iomap_writepages 1.69 ± 6% -1.1 0.62 ± 5% perf-profile.children.cycles-pp.write_cache_pages 1.54 ± 6% -1.0 0.52 ± 7% perf-profile.children.cycles-pp.iomap_writepage_map 1.48 ± 7% -1.0 0.46 ± 9% perf-profile.children.cycles-pp.__folio_end_writeback 1.78 ± 4% -1.0 0.79 ± 4% perf-profile.children.cycles-pp.submit_bio_noacct 1.77 ± 4% -1.0 0.78 ± 4% perf-profile.children.cycles-pp.__submit_bio 1.38 ± 7% -1.0 0.40 ± 9% perf-profile.children.cycles-pp.__folio_start_writeback 1.71 ± 4% -1.0 0.74 ± 4% perf-profile.children.cycles-pp.iomap_submit_ioend 1.31 ± 7% -1.0 0.36 ± 11% perf-profile.children.cycles-pp.sb_clear_inode_writeback 1.28 ± 7% -0.9 0.34 ± 11% perf-profile.children.cycles-pp.sb_mark_inode_writeback 3.14 ± 7% -0.8 2.29 ± 6% perf-profile.children.cycles-pp.xlog_cil_insert_items 93.48 -0.7 92.81 perf-profile.children.cycles-pp.write 93.52 -0.7 92.85 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 93.50 -0.7 92.84 perf-profile.children.cycles-pp.do_syscall_64 93.23 -0.6 92.60 perf-profile.children.cycles-pp.ksys_write 2.91 ± 2% -0.5 2.40 ± 2% perf-profile.children.cycles-pp.xlog_wait_on_iclog 3.09 ± 2% -0.4 2.64 ± 2% perf-profile.children.cycles-pp.xlog_force_lsn 0.79 ± 5% -0.3 0.44 ± 3% perf-profile.children.cycles-pp.rwsem_down_read_slowpath 0.44 ± 7% -0.3 0.19 ± 3% perf-profile.children.cycles-pp.iomap_finish_ioends 0.81 ± 4% -0.2 0.57 perf-profile.children.cycles-pp.mutex_spin_on_owner 1.16 ± 2% -0.2 0.95 perf-profile.children.cycles-pp.try_to_wake_up 0.70 ± 3% -0.2 0.49 ± 3% perf-profile.children.cycles-pp.iomap_file_buffered_write 0.30 ± 6% -0.2 0.09 ± 4% perf-profile.children.cycles-pp.wake_up_q 0.42 -0.2 0.24 ± 4% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.28 ± 3% -0.1 0.18 ± 2% perf-profile.children.cycles-pp.xfs_bmapi_write 0.32 ± 3% -0.1 0.24 ± 2% perf-profile.children.cycles-pp.brd_submit_bio 0.24 ± 5% -0.1 0.16 ± 4% perf-profile.children.cycles-pp.xfs_bmapi_convert_unwritten 0.31 ± 3% -0.1 0.23 ± 2% perf-profile.children.cycles-pp.brd_do_bvec 0.23 ± 3% -0.1 0.15 ± 4% perf-profile.children.cycles-pp.xfs_bmap_add_extent_unwritten_real 0.28 ± 2% -0.1 0.21 perf-profile.children.cycles-pp.xlog_state_do_callback 0.29 -0.1 0.22 ± 2% perf-profile.children.cycles-pp.xlog_ioend_work 0.23 ± 3% -0.1 0.16 ± 5% perf-profile.children.cycles-pp.iomap_write_end 0.20 ± 3% -0.1 0.14 ± 5% perf-profile.children.cycles-pp.iomap_write_begin 0.14 ± 2% -0.0 0.10 ± 5% perf-profile.children.cycles-pp.xlog_cil_process_committed 0.14 ± 2% -0.0 0.10 ± 5% perf-profile.children.cycles-pp.xlog_cil_committed 0.14 ± 4% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.iomap_set_range_uptodate 0.15 ± 4% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.__filemap_get_folio 0.14 ± 2% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.iomap_iter 0.15 ± 4% -0.0 0.11 ± 6% perf-profile.children.cycles-pp.pagecache_get_page 0.13 ± 4% -0.0 0.09 ± 4% perf-profile.children.cycles-pp.folio_wake_bit 0.40 -0.0 0.36 ± 2% perf-profile.children.cycles-pp.select_task_rq_fair 0.43 ± 4% -0.0 0.39 perf-profile.children.cycles-pp.update_load_avg 0.10 ± 5% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.xfs_end_bio 0.12 ± 3% -0.0 0.08 ± 4% perf-profile.children.cycles-pp.rwsem_wake 0.12 ± 5% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.wake_page_function 0.12 ± 3% -0.0 0.09 ± 6% perf-profile.children.cycles-pp.down_read 0.11 ± 4% -0.0 0.08 ± 7% perf-profile.children.cycles-pp.xfs_buffered_write_iomap_begin 0.42 ± 3% -0.0 0.39 perf-profile.children.cycles-pp.ttwu_do_activate 0.09 ± 6% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.queue_work_on 0.06 ± 7% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.xfs_map_blocks 0.10 ± 4% -0.0 0.07 ± 5% perf-profile.children.cycles-pp.xfs_btree_lookup 0.10 ± 6% -0.0 0.07 ± 5% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers 0.47 ± 3% -0.0 0.44 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair 0.30 ± 2% -0.0 0.27 ± 2% perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi 0.08 ± 5% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.__queue_work 0.11 ± 4% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.__filemap_fdatawait_range 0.10 ± 7% -0.0 0.07 ± 10% perf-profile.children.cycles-pp.xlog_grant_add_space 0.29 -0.0 0.26 perf-profile.children.cycles-pp.select_idle_sibling 0.66 -0.0 0.63 perf-profile.children.cycles-pp.asm_sysvec_call_function_single 0.11 ± 4% -0.0 0.08 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.10 ± 5% -0.0 0.07 ± 5% perf-profile.children.cycles-pp.xfs_trans_committed_bulk 0.10 -0.0 0.08 ± 6% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.09 ± 6% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.filemap_dirty_folio 0.08 -0.0 0.06 ± 9% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.10 ± 3% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.brd_lookup_page 0.35 ± 4% -0.0 0.32 ± 2% perf-profile.children.cycles-pp.enqueue_entity 0.14 ± 2% -0.0 0.12 ± 4% perf-profile.children.cycles-pp.xlog_state_clean_iclog 0.08 ± 6% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.kmem_cache_alloc 0.06 ± 7% -0.0 0.04 ± 44% perf-profile.children.cycles-pp.xfs_btree_lookup_get_block 0.16 ± 3% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.xfs_inode_item_format 0.07 ± 6% -0.0 0.05 ± 7% perf-profile.children.cycles-pp.blkdev_issue_flush 0.06 ± 7% -0.0 0.04 ± 44% perf-profile.children.cycles-pp.__folio_mark_dirty 0.26 -0.0 0.24 ± 2% perf-profile.children.cycles-pp.select_idle_cpu 0.17 -0.0 0.15 ± 6% perf-profile.children.cycles-pp.irq_exit_rcu 0.07 ± 5% -0.0 0.05 perf-profile.children.cycles-pp.memcpy_erms 0.14 ± 2% -0.0 0.12 ± 5% perf-profile.children.cycles-pp.__softirqentry_text_start 0.08 ± 4% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.llseek 0.12 ± 3% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.available_idle_cpu 0.09 ± 4% -0.0 0.07 perf-profile.children.cycles-pp.folio_wait_writeback 0.08 ± 5% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.folio_wait_bit 0.41 -0.0 0.39 perf-profile.children.cycles-pp.sysvec_call_function_single 0.12 ± 4% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.__list_add_valid 0.37 -0.0 0.36 perf-profile.children.cycles-pp.__sysvec_call_function_single 0.11 ± 3% -0.0 0.09 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid 0.07 ± 7% -0.0 0.05 perf-profile.children.cycles-pp.__might_resched 0.07 ± 7% -0.0 0.05 perf-profile.children.cycles-pp.xfs_trans_read_buf_map 0.07 -0.0 0.06 ± 9% perf-profile.children.cycles-pp.io_schedule 0.12 ± 6% -0.0 0.11 perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.16 ± 4% -0.0 0.14 ± 2% perf-profile.children.cycles-pp.update_cfs_group 0.08 ± 6% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.poll_idle 0.07 ± 7% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.__switch_to_asm 0.30 -0.0 0.28 perf-profile.children.cycles-pp.sched_ttwu_pending 0.07 -0.0 0.06 ± 6% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.07 ± 5% -0.0 0.06 perf-profile.children.cycles-pp.check_preempt_curr 0.06 ± 6% -0.0 0.05 perf-profile.children.cycles-pp.__switch_to 0.08 -0.0 0.07 perf-profile.children.cycles-pp.set_next_entity 0.24 ± 2% +0.0 0.26 ± 3% perf-profile.children.cycles-pp.xlog_write 0.22 ± 4% +0.0 0.26 ± 3% perf-profile.children.cycles-pp.update_curr 0.20 ± 2% +0.0 0.24 perf-profile.children.cycles-pp.__wake_up_common_lock 0.10 ± 17% +0.0 0.15 ± 14% perf-profile.children.cycles-pp.idle_cpu 0.12 ± 17% +0.0 0.16 ± 14% perf-profile.children.cycles-pp.cpumask_next_and 0.00 +0.1 0.06 ± 6% perf-profile.children.cycles-pp.cpuacct_charge 0.06 ± 9% +0.1 0.13 ± 4% perf-profile.children.cycles-pp.xlog_cil_set_ctx_write_state 0.11 ± 21% +0.1 0.19 ± 20% perf-profile.children.cycles-pp.raw_spin_rq_lock_nested 0.82 ± 18% +0.3 1.16 ± 13% perf-profile.children.cycles-pp.update_sd_lb_stats 0.83 ± 18% +0.3 1.17 ± 13% perf-profile.children.cycles-pp.find_busiest_group 1.28 ± 12% +0.4 1.63 ± 9% perf-profile.children.cycles-pp.wait_for_completion 1.22 ± 12% +0.4 1.59 ± 10% perf-profile.children.cycles-pp.schedule_timeout 1.44 ± 15% +0.4 1.88 ± 11% perf-profile.children.cycles-pp.pick_next_task_fair 1.22 ± 17% +0.5 1.68 ± 13% perf-profile.children.cycles-pp.load_balance 1.31 ± 16% +0.5 1.77 ± 12% perf-profile.children.cycles-pp.newidle_balance 4.65 ± 6% +0.8 5.41 ± 7% perf-profile.children.cycles-pp.ret_from_fork 4.65 ± 6% +0.8 5.41 ± 7% perf-profile.children.cycles-pp.kthread 4.61 ± 6% +0.8 5.39 ± 7% perf-profile.children.cycles-pp.worker_thread 4.53 ± 6% +0.8 5.33 ± 7% perf-profile.children.cycles-pp.process_one_work 3.73 ± 8% +0.9 4.60 ± 8% perf-profile.children.cycles-pp.xfs_end_io 3.73 ± 8% +0.9 4.60 ± 8% perf-profile.children.cycles-pp.xfs_end_ioend 0.99 ± 19% +0.9 1.93 ± 12% perf-profile.children.cycles-pp.xfs_trans_alloc_inode 2.60 ± 15% +1.0 3.62 ± 13% perf-profile.children.cycles-pp.xfs_log_ticket_ungrant 2.46 ± 16% +1.1 3.52 ± 14% perf-profile.children.cycles-pp.xfs_log_space_wake 3.28 ± 10% +1.1 4.41 ± 8% perf-profile.children.cycles-pp.xfs_iomap_write_unwritten 3.35 ± 20% +1.5 4.82 ± 13% perf-profile.children.cycles-pp.xfs_trans_alloc 3.30 ± 20% +1.5 4.79 ± 13% perf-profile.children.cycles-pp.xfs_trans_reserve 3.29 ± 21% +1.5 4.78 ± 13% perf-profile.children.cycles-pp.xfs_log_reserve 3.10 ± 22% +1.5 4.65 ± 14% perf-profile.children.cycles-pp.xlog_grant_head_check 3.31 +2.2 5.47 perf-profile.children.cycles-pp.remove_wait_queue 78.02 +2.3 80.35 perf-profile.children.cycles-pp.xlog_cil_force_seq 19.06 ± 6% +5.7 24.79 ± 3% perf-profile.children.cycles-pp._raw_spin_lock 24.70 ± 4% +5.9 30.59 ± 3% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 63.17 -4.2 59.00 perf-profile.self.cycles-pp.osq_lock 0.80 ± 4% -0.2 0.56 perf-profile.self.cycles-pp.mutex_spin_on_owner 0.10 ± 4% -0.1 0.02 ± 99% perf-profile.self.cycles-pp.__mutex_lock 0.20 ± 2% -0.1 0.13 ± 5% perf-profile.self.cycles-pp.brd_do_bvec 0.50 ± 3% -0.1 0.44 perf-profile.self.cycles-pp._raw_spin_lock 0.32 ± 3% -0.1 0.26 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.19 ± 3% -0.0 0.15 ± 10% perf-profile.self.cycles-pp.xlog_cil_commit 0.14 ± 4% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.iomap_set_range_uptodate 0.14 ± 4% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.xfs_log_ticket_ungrant 0.10 ± 7% -0.0 0.07 ± 11% perf-profile.self.cycles-pp.xlog_grant_add_space 0.11 ± 3% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.down_read 0.09 ± 4% -0.0 0.07 ± 7% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.09 ± 5% -0.0 0.07 ± 11% perf-profile.self.cycles-pp.xlog_cil_insert_items 0.07 ± 5% -0.0 0.05 perf-profile.self.cycles-pp.memcpy_erms 0.12 ± 4% -0.0 0.10 ± 3% perf-profile.self.cycles-pp.available_idle_cpu 0.21 ± 3% -0.0 0.19 ± 3% perf-profile.self.cycles-pp.update_load_avg 0.07 ± 7% -0.0 0.05 perf-profile.self.cycles-pp.__filemap_get_folio 0.07 ± 6% -0.0 0.06 perf-profile.self.cycles-pp.poll_idle 0.07 ± 7% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.__switch_to_asm 0.06 ± 7% -0.0 0.05 perf-profile.self.cycles-pp.xfs_inode_item_format 0.09 ± 5% +0.0 0.12 ± 4% perf-profile.self.cycles-pp.update_curr 0.10 ± 20% +0.0 0.14 ± 13% perf-profile.self.cycles-pp.idle_cpu 0.00 +0.1 0.06 ± 8% perf-profile.self.cycles-pp.cpuacct_charge 0.63 ± 19% +0.3 0.88 ± 13% perf-profile.self.cycles-pp.update_sd_lb_stats 24.59 ± 4% +5.9 30.46 ± 3% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 26680 ± 6% +26.2% 33670 ± 6% softirqs.CPU0.RCU 27129 ± 4% +25.4% 34028 ± 5% softirqs.CPU1.RCU 35819 ± 7% +19.2% 42711 ± 2% softirqs.CPU1.SCHED 26877 ± 4% +25.2% 33638 ± 5% softirqs.CPU10.RCU 33406 ± 8% +21.7% 40666 ± 3% softirqs.CPU10.SCHED 27439 ± 5% +22.5% 33625 ± 5% softirqs.CPU100.RCU 33435 ± 3% +25.5% 41976 ± 3% softirqs.CPU100.SCHED 27108 ± 4% +25.0% 33881 ± 6% softirqs.CPU101.RCU 33637 ± 7% +24.1% 41730 ± 2% softirqs.CPU101.SCHED 27254 ± 4% +23.7% 33710 ± 6% softirqs.CPU102.RCU 33563 ± 7% +24.0% 41608 ± 2% softirqs.CPU102.SCHED 27318 ± 5% +24.6% 34029 ± 6% softirqs.CPU103.RCU 33908 ± 7% +23.5% 41867 ± 2% softirqs.CPU103.SCHED 27236 ± 4% +25.3% 34129 ± 6% softirqs.CPU104.RCU 33617 ± 6% +25.4% 42146 ± 2% softirqs.CPU104.SCHED 27103 ± 4% +24.3% 33677 ± 5% softirqs.CPU105.RCU 33701 ± 7% +22.8% 41388 ± 3% softirqs.CPU105.SCHED 27311 ± 4% +24.3% 33946 ± 6% softirqs.CPU106.RCU 33592 ± 6% +24.3% 41754 ± 2% softirqs.CPU106.SCHED 27264 ± 4% +24.8% 34020 ± 8% softirqs.CPU107.RCU 33607 ± 7% +24.1% 41719 ± 2% softirqs.CPU107.SCHED 27382 ± 5% +24.3% 34041 ± 6% softirqs.CPU108.RCU 33398 ± 6% +24.8% 41674 ± 2% softirqs.CPU108.SCHED 27163 ± 5% +25.3% 34027 ± 6% softirqs.CPU109.RCU 33579 ± 7% +24.5% 41797 ± 2% softirqs.CPU109.SCHED 26938 ± 4% +26.2% 34005 ± 8% softirqs.CPU11.RCU 33579 ± 8% +22.6% 41160 ± 2% softirqs.CPU11.SCHED 27290 ± 4% +24.5% 33980 ± 7% softirqs.CPU110.RCU 33652 ± 6% +24.2% 41793 ± 2% softirqs.CPU110.SCHED 27276 ± 4% +25.0% 34105 ± 7% softirqs.CPU111.RCU 33704 ± 7% +24.1% 41842 ± 2% softirqs.CPU111.SCHED 26845 ± 5% +23.4% 33139 ± 6% softirqs.CPU112.RCU 33669 ± 6% +24.2% 41809 ± 2% softirqs.CPU112.SCHED 26442 ± 4% +24.6% 32952 ± 6% softirqs.CPU113.RCU 33488 ± 6% +25.0% 41877 ± 2% softirqs.CPU113.SCHED 26758 ± 4% +24.3% 33270 ± 6% softirqs.CPU114.RCU 33519 ± 7% +24.8% 41841 ± 2% softirqs.CPU114.SCHED 26694 ± 4% +23.9% 33068 ± 6% softirqs.CPU115.RCU 33670 ± 7% +24.3% 41848 ± 2% softirqs.CPU115.SCHED 26987 ± 6% +22.4% 33045 ± 6% softirqs.CPU116.RCU 33729 ± 7% +23.5% 41653 ± 2% softirqs.CPU116.SCHED 26677 ± 5% +24.8% 33283 ± 6% softirqs.CPU117.RCU 33848 ± 7% +23.5% 41818 ± 2% softirqs.CPU117.SCHED 26832 ± 4% +24.3% 33364 ± 6% softirqs.CPU118.RCU 33507 ± 7% +24.9% 41848 ± 2% softirqs.CPU118.SCHED 26968 ± 4% +22.8% 33116 ± 6% softirqs.CPU119.RCU 33572 ± 6% +24.5% 41781 ± 3% softirqs.CPU119.SCHED 26940 ± 4% +23.2% 33199 ± 5% softirqs.CPU12.RCU 33332 ± 8% +22.7% 40903 ± 3% softirqs.CPU12.SCHED 26827 ± 5% +24.1% 33293 ± 6% softirqs.CPU120.RCU 33700 ± 7% +24.3% 41900 ± 2% softirqs.CPU120.SCHED 26967 ± 5% +23.9% 33400 ± 6% softirqs.CPU121.RCU 33626 ± 7% +24.5% 41872 ± 2% softirqs.CPU121.SCHED 26889 ± 5% +23.2% 33136 ± 6% softirqs.CPU122.RCU 33833 ± 8% +23.0% 41604 ± 2% softirqs.CPU122.SCHED 26871 ± 4% +23.7% 33230 ± 6% softirqs.CPU123.RCU 33495 ± 7% +24.7% 41780 ± 2% softirqs.CPU123.SCHED 26754 ± 5% +24.5% 33322 ± 6% softirqs.CPU124.RCU 33641 ± 7% +24.7% 41947 ± 2% softirqs.CPU124.SCHED 26996 ± 4% +23.6% 33375 ± 6% softirqs.CPU125.RCU 34073 ± 7% +22.7% 41792 ± 2% softirqs.CPU125.SCHED 26799 ± 4% +24.4% 33326 ± 6% softirqs.CPU126.RCU 33749 ± 7% +22.7% 41415 ± 3% softirqs.CPU126.SCHED 27041 ± 4% +25.4% 33915 ± 7% softirqs.CPU127.RCU 32331 ± 8% +23.9% 40067 ± 3% softirqs.CPU127.SCHED 26741 ± 3% +25.1% 33453 ± 5% softirqs.CPU13.RCU 33223 ± 8% +23.3% 40953 ± 3% softirqs.CPU13.SCHED 27021 ± 4% +23.1% 33256 ± 5% softirqs.CPU14.RCU 33487 ± 8% +22.9% 41146 ± 3% softirqs.CPU14.SCHED 26982 ± 4% +24.8% 33664 ± 5% softirqs.CPU15.RCU 33299 ± 8% +23.0% 40945 ± 3% softirqs.CPU15.SCHED 27050 ± 3% +21.7% 32912 ± 5% softirqs.CPU16.RCU 33353 ± 8% +22.8% 40972 ± 3% softirqs.CPU16.SCHED 27350 ± 3% +21.0% 33088 ± 5% softirqs.CPU17.RCU 33419 ± 8% +22.4% 40895 ± 3% softirqs.CPU17.SCHED 27123 ± 4% +24.2% 33685 ± 5% softirqs.CPU18.RCU 33425 ± 8% +22.0% 40769 ± 3% softirqs.CPU18.SCHED 27547 ± 4% +21.8% 33547 ± 5% softirqs.CPU19.RCU 33626 ± 8% +21.8% 40941 ± 3% softirqs.CPU19.SCHED 27753 ± 5% +20.1% 33342 ± 7% softirqs.CPU2.RCU 34434 ± 9% +20.9% 41620 ± 2% softirqs.CPU2.SCHED 27140 ± 4% +24.3% 33721 ± 6% softirqs.CPU20.RCU 33283 ± 8% +22.7% 40824 ± 3% softirqs.CPU20.SCHED 26675 ± 3% +24.1% 33115 ± 4% softirqs.CPU21.RCU 33134 ± 8% +23.1% 40785 ± 3% softirqs.CPU21.SCHED 26826 ± 3% +23.9% 33242 ± 5% softirqs.CPU22.RCU 33259 ± 8% +22.7% 40821 ± 3% softirqs.CPU22.SCHED 26985 ± 3% +21.3% 32732 ± 6% softirqs.CPU23.RCU 33325 ± 8% +21.9% 40619 ± 3% softirqs.CPU23.SCHED 26884 ± 4% +24.9% 33583 ± 6% softirqs.CPU24.RCU 33258 ± 8% +22.5% 40725 ± 4% softirqs.CPU24.SCHED 27039 ± 3% +25.1% 33827 ± 5% softirqs.CPU25.RCU 33382 ± 8% +21.5% 40557 ± 3% softirqs.CPU25.SCHED 26982 ± 4% +25.5% 33875 ± 5% softirqs.CPU26.RCU 33406 ± 8% +21.6% 40623 ± 4% softirqs.CPU26.SCHED 26721 ± 4% +25.1% 33425 ± 5% softirqs.CPU27.RCU 33062 ± 8% +23.3% 40770 ± 3% softirqs.CPU27.SCHED 26905 ± 4% +25.9% 33864 ± 4% softirqs.CPU28.RCU 33338 ± 8% +24.1% 41371 ± 6% softirqs.CPU28.SCHED 26455 ± 4% +26.8% 33542 ± 5% softirqs.CPU29.RCU 33165 ± 8% +22.9% 40764 ± 3% softirqs.CPU29.SCHED 26612 ± 3% +27.1% 33813 ± 6% softirqs.CPU3.RCU 33684 ± 8% +23.0% 41433 ± 3% softirqs.CPU3.SCHED 26858 ± 4% +26.5% 33981 ± 5% softirqs.CPU30.RCU 33247 ± 8% +22.4% 40687 ± 3% softirqs.CPU30.SCHED 26836 ± 4% +26.5% 33945 ± 5% softirqs.CPU31.RCU 33307 ± 8% +22.4% 40782 ± 3% softirqs.CPU31.SCHED 27515 ± 6% +22.6% 33724 ± 6% softirqs.CPU32.RCU 27293 ± 4% +24.0% 33842 ± 7% softirqs.CPU33.RCU 33439 ± 7% +23.5% 41312 ± 3% softirqs.CPU33.SCHED 26977 ± 6% +24.3% 33522 ± 6% softirqs.CPU34.RCU 33293 ± 8% +22.5% 40783 ± 3% softirqs.CPU34.SCHED 26953 ± 4% +24.2% 33473 ± 6% softirqs.CPU35.RCU 33435 ± 9% +22.2% 40858 ± 3% softirqs.CPU35.SCHED 27085 ± 4% +23.4% 33416 ± 5% softirqs.CPU36.RCU 33279 ± 8% +22.7% 40848 ± 3% softirqs.CPU36.SCHED 27083 ± 5% +23.6% 33469 ± 6% softirqs.CPU37.RCU 33373 ± 8% +22.3% 40808 ± 3% softirqs.CPU37.SCHED 27124 ± 5% +22.8% 33303 ± 6% softirqs.CPU38.RCU 33272 ± 9% +22.3% 40687 ± 3% softirqs.CPU38.SCHED 27180 ± 5% +22.5% 33302 ± 6% softirqs.CPU39.RCU 33457 ± 9% +21.6% 40675 ± 3% softirqs.CPU39.SCHED 26744 ± 4% +28.2% 34276 ± 8% softirqs.CPU4.RCU 33221 ± 9% +25.6% 41728 ± 4% softirqs.CPU4.SCHED 27039 ± 5% +23.6% 33434 ± 6% softirqs.CPU40.RCU 33345 ± 8% +22.1% 40698 ± 3% softirqs.CPU40.SCHED 26958 ± 5% +24.0% 33422 ± 6% softirqs.CPU41.RCU 33267 ± 9% +21.0% 40248 ± 3% softirqs.CPU41.SCHED 27310 ± 4% +22.0% 33306 ± 5% softirqs.CPU42.RCU 33752 ± 9% +20.3% 40603 ± 3% softirqs.CPU42.SCHED 27008 ± 4% +23.1% 33254 ± 6% softirqs.CPU43.RCU 33485 ± 9% +21.3% 40633 ± 3% softirqs.CPU43.SCHED 26947 ± 5% +23.8% 33373 ± 5% softirqs.CPU44.RCU 33200 ± 8% +22.4% 40624 ± 3% softirqs.CPU44.SCHED 27144 ± 6% +22.7% 33317 ± 6% softirqs.CPU45.RCU 33461 ± 9% +21.3% 40587 ± 3% softirqs.CPU45.SCHED 27729 ± 6% +20.8% 33504 ± 6% softirqs.CPU46.RCU 33300 ± 8% +22.2% 40686 ± 3% softirqs.CPU46.SCHED 27047 ± 5% +23.8% 33495 ± 6% softirqs.CPU47.RCU 33278 ± 9% +22.6% 40810 ± 3% softirqs.CPU47.SCHED 26587 ± 5% +23.7% 32899 ± 6% softirqs.CPU48.RCU 33298 ± 8% +21.7% 40511 ± 3% softirqs.CPU48.SCHED 26504 ± 4% +23.9% 32825 ± 6% softirqs.CPU49.RCU 33566 ± 8% +21.1% 40637 ± 3% softirqs.CPU49.SCHED 27220 ± 2% +24.8% 33967 ± 7% softirqs.CPU5.RCU 33627 ± 8% +23.8% 41615 ± 3% softirqs.CPU5.SCHED 26621 ± 5% +23.3% 32822 ± 6% softirqs.CPU50.RCU 33270 ± 9% +22.3% 40696 ± 3% softirqs.CPU50.SCHED 26611 ± 5% +23.3% 32824 ± 6% softirqs.CPU51.RCU 33294 ± 8% +22.3% 40712 ± 3% softirqs.CPU51.SCHED 27039 ± 4% +22.0% 32990 ± 5% softirqs.CPU52.RCU 32827 ± 5% +23.6% 40560 ± 3% softirqs.CPU52.SCHED 26747 ± 4% +21.1% 32380 ± 4% softirqs.CPU53.RCU 33282 ± 8% +21.6% 40464 ± 3% softirqs.CPU53.SCHED 26806 ± 6% +23.6% 33135 ± 5% softirqs.CPU54.RCU 33258 ± 9% +21.9% 40554 ± 3% softirqs.CPU54.SCHED 26704 ± 5% +22.7% 32758 ± 5% softirqs.CPU55.RCU 33309 ± 9% +21.4% 40444 ± 3% softirqs.CPU55.SCHED 26738 ± 5% +24.5% 33296 ± 6% softirqs.CPU56.RCU 33092 ± 9% +22.6% 40575 ± 3% softirqs.CPU56.SCHED 26748 ± 4% +23.2% 32957 ± 5% softirqs.CPU57.RCU 33230 ± 8% +22.2% 40597 ± 3% softirqs.CPU57.SCHED 26436 ± 3% +24.1% 32798 ± 5% softirqs.CPU58.RCU 33250 ± 8% +22.4% 40700 ± 3% softirqs.CPU58.SCHED 26684 ± 4% +24.3% 33167 ± 4% softirqs.CPU59.RCU 33366 ± 8% +21.4% 40494 ± 3% softirqs.CPU59.SCHED 27453 ± 4% +24.4% 34155 ± 7% softirqs.CPU6.RCU 33840 ± 7% +23.4% 41763 ± 3% softirqs.CPU6.SCHED 26809 ± 4% +24.0% 33251 ± 5% softirqs.CPU60.RCU 33416 ± 8% +22.8% 41025 ± 3% softirqs.CPU60.SCHED 27491 ± 6% +20.1% 33026 ± 5% softirqs.CPU61.RCU 33166 ± 8% +22.4% 40606 ± 3% softirqs.CPU61.SCHED 26631 ± 7% +23.6% 32929 ± 5% softirqs.CPU62.RCU 33401 ± 8% +21.6% 40623 ± 4% softirqs.CPU62.SCHED 26954 ± 4% +22.8% 33102 ± 6% softirqs.CPU63.RCU 32772 ± 9% +22.5% 40153 ± 3% softirqs.CPU63.SCHED 26905 ± 5% +25.4% 33730 ± 5% softirqs.CPU64.RCU 32909 ± 8% +26.3% 41552 ± 3% softirqs.CPU64.SCHED 27208 ± 4% +25.9% 34254 ± 6% softirqs.CPU65.RCU 27580 ± 6% +22.1% 33676 ± 7% softirqs.CPU66.RCU 33484 ± 6% +28.3% 42949 ± 3% softirqs.CPU66.SCHED 27788 ± 8% +22.4% 34000 ± 5% softirqs.CPU67.RCU 33520 ± 7% +26.2% 42292 ± 3% softirqs.CPU67.SCHED 27641 ± 4% +23.0% 33986 ± 6% softirqs.CPU68.RCU 33216 ± 7% +26.4% 41979 ± 3% softirqs.CPU68.SCHED 26533 ± 4% +28.0% 33952 ± 6% softirqs.CPU69.RCU 33682 ± 8% +25.0% 42094 ± 2% softirqs.CPU69.SCHED 26854 ± 4% +22.8% 32975 ± 5% softirqs.CPU7.RCU 33394 ± 8% +22.3% 40854 ± 4% softirqs.CPU7.SCHED 26723 ± 5% +26.2% 33724 ± 5% softirqs.CPU70.RCU 33953 ± 7% +23.8% 42019 ± 2% softirqs.CPU70.SCHED 27263 ± 4% +23.7% 33712 ± 6% softirqs.CPU71.RCU 33677 ± 6% +25.1% 42139 ± 2% softirqs.CPU71.SCHED 27296 ± 3% +23.6% 33744 ± 5% softirqs.CPU72.RCU 34080 ± 5% +24.2% 42330 ± 3% softirqs.CPU72.SCHED 27288 ± 4% +22.9% 33548 ± 5% softirqs.CPU73.RCU 33891 ± 5% +24.8% 42306 ± 3% softirqs.CPU73.SCHED 27282 ± 4% +24.2% 33878 ± 6% softirqs.CPU74.RCU 33691 ± 6% +24.9% 42087 ± 2% softirqs.CPU74.SCHED 26775 ± 3% +25.7% 33662 ± 5% softirqs.CPU75.RCU 33827 ± 7% +23.0% 41608 ± 2% softirqs.CPU75.SCHED 27230 ± 3% +24.2% 33807 ± 4% softirqs.CPU76.RCU 34217 ± 7% +23.0% 42076 ± 3% softirqs.CPU76.SCHED 27218 ± 3% +24.5% 33886 ± 5% softirqs.CPU77.RCU 33603 ± 6% +25.0% 41999 ± 2% softirqs.CPU77.SCHED 27066 ± 3% +23.7% 33481 ± 6% softirqs.CPU78.RCU 34156 ± 7% +23.1% 42034 ± 2% softirqs.CPU78.SCHED 27061 ± 3% +24.7% 33749 ± 5% softirqs.CPU79.RCU 33721 ± 6% +24.8% 42075 ± 2% softirqs.CPU79.SCHED 27706 ± 5% +21.4% 33630 ± 6% softirqs.CPU8.RCU 33378 ± 8% +21.9% 40685 ± 2% softirqs.CPU8.SCHED 27020 ± 3% +20.7% 32620 ± 7% softirqs.CPU80.RCU 33184 ± 6% +26.4% 41948 ± 3% softirqs.CPU80.SCHED 27174 ± 4% +23.3% 33500 ± 5% softirqs.CPU81.RCU 33937 ± 7% +24.5% 42246 ± 2% softirqs.CPU81.SCHED 27680 ± 8% +22.5% 33914 ± 7% softirqs.CPU82.RCU 33620 ± 6% +23.5% 41514 ± 2% softirqs.CPU82.SCHED 27380 ± 4% +22.4% 33502 ± 6% softirqs.CPU83.RCU 34232 ± 8% +22.6% 41975 ± 2% softirqs.CPU83.SCHED 27096 ± 4% +24.1% 33618 ± 6% softirqs.CPU84.RCU 33663 ± 6% +25.1% 42127 ± 2% softirqs.CPU84.SCHED 26788 ± 4% +25.1% 33499 ± 5% softirqs.CPU85.RCU 33446 ± 5% +25.1% 41827 ± 2% softirqs.CPU85.SCHED 26692 ± 4% +24.9% 33349 ± 5% softirqs.CPU86.RCU 33968 ± 7% +23.4% 41922 ± 2% softirqs.CPU86.SCHED 26816 ± 4% +24.4% 33365 ± 5% softirqs.CPU87.RCU 33705 ± 7% +23.7% 41694 ± 2% softirqs.CPU87.SCHED 27090 ± 5% +24.8% 33795 ± 6% softirqs.CPU88.RCU 33641 ± 6% +25.6% 42246 ± 2% softirqs.CPU88.SCHED 26812 ± 4% +24.9% 33485 ± 5% softirqs.CPU89.RCU 33711 ± 7% +23.8% 41737 ± 2% softirqs.CPU89.SCHED 26758 ± 4% +25.3% 33539 ± 5% softirqs.CPU9.RCU 33274 ± 8% +22.8% 40868 ± 3% softirqs.CPU9.SCHED 26767 ± 4% +27.5% 34136 ± 5% softirqs.CPU90.RCU 33442 ± 8% +25.6% 42015 softirqs.CPU90.SCHED 26961 ± 4% +25.4% 33811 ± 5% softirqs.CPU91.RCU 33452 ± 6% +25.6% 42003 ± 3% softirqs.CPU91.SCHED 26796 ± 8% +25.3% 33568 ± 5% softirqs.CPU92.RCU 33727 ± 7% +24.6% 42014 ± 2% softirqs.CPU92.SCHED 26980 ± 4% +25.4% 33845 ± 6% softirqs.CPU93.RCU 33730 ± 7% +25.8% 42430 ± 2% softirqs.CPU93.SCHED 26995 ± 4% +24.0% 33466 ± 5% softirqs.CPU94.RCU 33769 ± 7% +23.2% 41590 ± 2% softirqs.CPU94.SCHED 27260 ± 5% +24.0% 33811 ± 5% softirqs.CPU95.RCU 33769 ± 6% +23.9% 41824 ± 2% softirqs.CPU95.SCHED 27560 ± 5% +25.7% 34647 ± 8% softirqs.CPU96.RCU 33904 ± 7% +25.8% 42662 ± 3% softirqs.CPU96.SCHED 27381 ± 4% +23.8% 33896 ± 6% softirqs.CPU97.RCU 33677 ± 7% +23.0% 41423 ± 2% softirqs.CPU97.SCHED 27409 ± 4% +25.0% 34268 ± 6% softirqs.CPU98.RCU 33808 ± 7% +24.0% 41924 ± 2% softirqs.CPU98.SCHED 27215 ± 4% +23.8% 33694 ± 6% softirqs.CPU99.RCU 33286 ± 7% +25.1% 41647 softirqs.CPU99.SCHED 3459594 ± 4% +24.0% 4291075 ± 6% softirqs.RCU 4308078 ± 7% +23.3% 5311352 ± 2% softirqs.SCHED 37815 ± 6% +19.2% 45093 ± 3% softirqs.TIMER *************************************************************************************************** lkp-icl-2sp1: 96 threads 2 sockets Ice Lake with 256G memory ========================================================================================= compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-9/performance/pipe/4/x86_64-rhel-8.3/threads/100%/debian-10.4-x86_64-20200603.cgz/lkp-icl-2sp1/hackbench/0xb000280 commit: v5.16-rc6 b08e888d97 ("sched/fair: Force progress on min_vruntime") v5.16-rc6 b08e888d9715f01546da7a40ca3 ---------------- --------------------------- %stddev %change %stddev \ | \ 473007 +73.0% 818434 hackbench.throughput 17897953 ± 3% -56.2% 7832807 ± 5% cpuidle..usage 146.49 -28.8% 104.23 ± 2% uptime.boot 113175 -29.9% 79387 meminfo.AnonHugePages 5268 ± 11% +27.7% 6728 meminfo.PageTables 91947 ± 4% -53.6% 42661 ± 59% numa-meminfo.node0.AnonHugePages 63263 -32.2% 42868 ± 56% numa-meminfo.node0.KernelStack 7760 ± 79% -67.2% 2544 ±109% numa-meminfo.node1.Mapped 3.41 ± 25% +3.0 6.45 ± 13% mpstat.cpu.all.idle% 0.04 ± 2% -0.0 0.04 ± 2% mpstat.cpu.all.soft% 84.65 -7.7 76.95 mpstat.cpu.all.sys% 10.96 +4.8 15.71 mpstat.cpu.all.usr% 21693147 +160.0% 56408989 ± 2% numa-numastat.node0.local_node 21728287 +159.3% 56340737 ± 2% numa-numastat.node0.numa_hit 17675019 ± 4% +199.6% 52961688 ± 2% numa-numastat.node1.local_node 17672611 ± 3% +199.6% 52946454 ± 2% numa-numastat.node1.numa_hit 13005297 +124.2% 29160989 ± 2% numa-vmstat.node0.numa_hit 12970256 +125.0% 29178686 ± 2% numa-vmstat.node0.numa_local 1866 ± 70% -65.8% 637.33 ±109% numa-vmstat.node1.nr_mapped 9935088 ± 3% +171.5% 26969825 ± 5% numa-vmstat.node1.numa_hit 9849174 ± 3% +173.3% 26917736 ± 5% numa-vmstat.node1.numa_local 4.67 ± 20% +92.9% 9.00 ± 9% vmstat.cpu.id 10.00 +50.0% 15.00 vmstat.cpu.us 941.33 +106.2% 1940 vmstat.procs.r 20378307 -95.0% 1008729 vmstat.system.cs 1149369 -81.7% 210349 vmstat.system.in 2928 +2.8% 3012 turbostat.Bzy_MHz 13632710 ± 2% -66.7% 4542661 ± 7% turbostat.C1 0.50 ± 3% -0.1 0.42 ± 4% turbostat.C1% 2611317 ± 13% +22.3% 3194550 ± 2% turbostat.C1E 2.77 ± 13% +3.1 5.83 ± 15% turbostat.C1E% 53365 ± 48% -40.4% 31832 ± 70% turbostat.C6 1.213e+08 -89.6% 12672793 turbostat.IRQ 76371 +4.3% 79645 proc-vmstat.nr_anon_pages 71944 -5.7% 67865 ± 2% proc-vmstat.nr_kernel_stack 1315 ± 11% +26.8% 1667 ± 5% proc-vmstat.nr_page_table_pages 71694 -1.9% 70365 proc-vmstat.nr_slab_unreclaimable 50662 ± 36% -99.6% 213.33 ±107% proc-vmstat.numa_hint_faults 27057 ± 34% -100.0% 1.67 ± 28% proc-vmstat.numa_hint_faults_local 39349867 +177.8% 1.093e+08 proc-vmstat.numa_hit 39278411 +178.5% 1.094e+08 proc-vmstat.numa_local 13176 ± 10% -89.3% 1403 ±112% proc-vmstat.numa_pages_migrated 83800 ± 11% -97.5% 2132 ± 91% proc-vmstat.numa_pte_updates 39343762 +177.5% 1.092e+08 proc-vmstat.pgalloc_normal 532749 ± 13% -41.6% 311104 proc-vmstat.pgfault 38985963 +179.9% 1.091e+08 proc-vmstat.pgfree 13176 ± 10% -89.3% 1403 ±112% proc-vmstat.pgmigrate_success 22598 ± 2% -32.7% 15216 proc-vmstat.pgreuse 11.47 -68.6% 3.60 perf-stat.i.MPKI 0.86 -0.5 0.40 ± 4% perf-stat.i.branch-miss-rate% 3.498e+08 -59.4% 1.42e+08 ± 2% perf-stat.i.branch-misses 4.99 ± 3% +21.4 26.36 ± 2% perf-stat.i.cache-miss-rate% 1.055e+08 +49.5% 1.578e+08 ± 2% perf-stat.i.cache-misses 2.338e+09 -70.8% 6.832e+08 perf-stat.i.cache-references 20946038 -95.3% 988301 perf-stat.i.context-switches 1.34 -2.7% 1.31 perf-stat.i.cpi 1283187 -90.1% 126605 ± 2% perf-stat.i.cpu-migrations 2734 -36.4% 1739 perf-stat.i.cycles-between-cache-misses 0.00 ± 3% -0.0 0.00 ± 14% perf-stat.i.dTLB-load-miss-rate% 2387946 ± 7% -63.6% 869224 ± 8% perf-stat.i.dTLB-load-misses 6.249e+10 +3.6% 6.477e+10 perf-stat.i.dTLB-loads 464565 ± 23% -51.7% 224519 ± 3% perf-stat.i.dTLB-store-misses 588.32 -2.7% 572.63 perf-stat.i.metric.K/sec 86.12 -19.6 66.55 perf-stat.i.node-load-miss-rate% 13026858 +22.7% 15981909 ± 2% perf-stat.i.node-load-misses 2062979 ± 5% +290.7% 8059176 ± 3% perf-stat.i.node-loads 58.45 +2.6 61.08 perf-stat.i.node-store-miss-rate% 10655145 +64.1% 17486258 perf-stat.i.node-store-misses 7676075 ± 2% +40.4% 10776422 ± 2% perf-stat.i.node-stores 11.19 -71.0% 3.24 ± 2% perf-stat.overall.MPKI 0.83 -0.5 0.34 ± 2% perf-stat.overall.branch-miss-rate% 4.53 +18.7 23.22 ± 3% perf-stat.overall.cache-miss-rate% 2589 -33.0% 1734 perf-stat.overall.cycles-between-cache-misses 0.00 ± 7% -0.0 0.00 ± 8% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 24% -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate% 86.27 -19.9 66.40 perf-stat.overall.node-load-miss-rate% 58.22 +3.7 61.88 perf-stat.overall.node-store-miss-rate% 3.461e+08 -59.7% 1.396e+08 ± 2% perf-stat.ps.branch-misses 1.045e+08 +48.6% 1.553e+08 ± 2% perf-stat.ps.cache-misses 2.309e+09 -71.0% 6.692e+08 perf-stat.ps.cache-references 20735335 -95.4% 962887 perf-stat.ps.context-switches 1258324 -90.3% 122191 ± 2% perf-stat.ps.cpu-migrations 2353383 ± 7% -64.1% 845637 ± 8% perf-stat.ps.dTLB-load-misses 6.195e+10 +3.0% 6.382e+10 perf-stat.ps.dTLB-loads 458770 ± 23% -52.2% 219278 ± 2% perf-stat.ps.dTLB-store-misses 12871051 +22.1% 15713487 ± 2% perf-stat.ps.node-load-misses 2047905 ± 5% +288.2% 7950325 ± 3% perf-stat.ps.node-loads 10536805 +63.2% 17197621 perf-stat.ps.node-store-misses 7563252 ± 2% +40.1% 10595385 ± 2% perf-stat.ps.node-stores 2.095e+13 -42.6% 1.204e+13 perf-stat.total.instructions 60.58 ± 70% -60.6 0.00 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 59.97 ± 70% -60.0 0.00 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 59.97 ± 70% -60.0 0.00 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 59.92 ± 70% -59.9 0.00 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 55.24 ± 70% -55.2 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 54.77 ± 70% -54.8 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 29.75 ± 71% -29.7 0.00 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 23.41 ± 71% -23.4 0.00 perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 19.07 ± 72% -19.1 0.00 perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 13.27 ± 75% -13.3 0.00 perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 11.95 ± 75% -12.0 0.00 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 6.09 ± 75% -6.1 0.00 perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.00 +5.6 5.59 ± 15% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.sched_setaffinity.__evlist__disable.cmd_record.run_builtin 0.00 +5.6 5.59 ± 15% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.sched_setaffinity.__evlist__disable.cmd_record 0.00 +5.6 5.59 ± 15% perf-profile.calltrace.cycles-pp.cmd_record.run_builtin.main.__libc_start_main 0.00 +5.6 5.59 ± 15% perf-profile.calltrace.cycles-pp.__evlist__disable.cmd_record.run_builtin.main.__libc_start_main 0.00 +5.6 5.59 ± 15% perf-profile.calltrace.cycles-pp.sched_setaffinity.__evlist__disable.cmd_record.run_builtin.main 0.45 ±141% +8.2 8.62 ± 41% perf-profile.calltrace.cycles-pp.__libc_start_main 0.45 ±141% +8.2 8.62 ± 41% perf-profile.calltrace.cycles-pp.main.__libc_start_main 0.45 ±141% +8.2 8.62 ± 41% perf-profile.calltrace.cycles-pp.run_builtin.main.__libc_start_main 0.00 +11.1 11.15 ± 70% perf-profile.calltrace.cycles-pp.fault_in_iov_iter_readable.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write 0.00 +11.1 11.15 ± 70% perf-profile.calltrace.cycles-pp.fault_in_readable.fault_in_iov_iter_readable.generic_perform_write.__generic_file_write_iter.generic_file_write_iter 0.00 +12.7 12.66 ± 53% perf-profile.calltrace.cycles-pp.__get_user_nocheck_1.fault_in_readable.fault_in_iov_iter_readable.generic_perform_write.__generic_file_write_iter 60.58 ± 70% -60.6 0.00 perf-profile.children.cycles-pp.secondary_startup_64_no_verify 60.58 ± 70% -60.6 0.00 perf-profile.children.cycles-pp.cpu_startup_entry 60.58 ± 70% -60.6 0.00 perf-profile.children.cycles-pp.do_idle 59.97 ± 70% -60.0 0.00 perf-profile.children.cycles-pp.start_secondary 56.08 ± 70% -56.1 0.00 perf-profile.children.cycles-pp.cpuidle_enter 55.96 ± 70% -56.0 0.00 perf-profile.children.cycles-pp.cpuidle_enter_state 29.86 ± 71% -29.9 0.00 perf-profile.children.cycles-pp.intel_idle 21.90 ± 71% -21.9 0.00 perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 19.49 ± 72% -19.5 0.00 perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 13.69 ± 75% -13.7 0.00 perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 12.27 ± 75% -12.3 0.00 perf-profile.children.cycles-pp.hrtimer_interrupt 6.46 ± 75% -6.5 0.00 perf-profile.children.cycles-pp.__hrtimer_run_queues 5.06 ± 70% -5.1 0.00 perf-profile.children.cycles-pp.ktime_get 0.05 ±141% +5.5 5.59 ± 15% perf-profile.children.cycles-pp.__evlist__disable 3.11 ±104% +7.7 10.81 ± 39% perf-profile.children.cycles-pp.do_sys_open 3.11 ±104% +7.7 10.81 ± 39% perf-profile.children.cycles-pp.do_sys_openat2 0.50 ±122% +8.1 8.62 ± 41% perf-profile.children.cycles-pp.__libc_start_main 0.50 ±122% +8.1 8.62 ± 41% perf-profile.children.cycles-pp.main 0.50 ±122% +8.1 8.62 ± 41% perf-profile.children.cycles-pp.run_builtin 0.05 ±141% +8.6 8.62 ± 41% perf-profile.children.cycles-pp.cmd_record 0.65 ± 86% +10.9 11.52 ± 44% perf-profile.children.cycles-pp.asm_exc_page_fault 0.00 +11.1 11.15 ± 70% perf-profile.children.cycles-pp.fault_in_iov_iter_readable 0.00 +11.1 11.15 ± 70% perf-profile.children.cycles-pp.fault_in_readable 0.00 +11.1 11.15 ± 70% perf-profile.children.cycles-pp.__get_user_nocheck_1 0.91 ±141% +14.1 15.02 ± 58% perf-profile.children.cycles-pp.sched_setaffinity 29.86 ± 71% -29.9 0.00 perf-profile.self.cycles-pp.intel_idle 3.88 ± 71% -3.9 0.00 perf-profile.self.cycles-pp.ktime_get 19595 ± 5% -45.5% 10675 ± 4% softirqs.CPU0.RCU 10568 ± 5% -16.9% 8779 ± 11% softirqs.CPU0.SCHED 19063 ± 2% -48.4% 9839 ± 6% softirqs.CPU1.RCU 18467 ± 7% -47.8% 9639 ± 5% softirqs.CPU10.RCU 18139 ± 4% -48.4% 9359 ± 5% softirqs.CPU11.RCU 8381 ± 13% -37.6% 5230 ± 14% softirqs.CPU11.SCHED 18196 ± 4% -48.7% 9333 ± 3% softirqs.CPU12.RCU 18919 ± 7% -48.2% 9806 ± 5% softirqs.CPU13.RCU 17668 ± 4% -47.1% 9345 ± 5% softirqs.CPU14.RCU 18473 ± 4% -48.9% 9431 ± 4% softirqs.CPU15.RCU 19178 ± 4% -48.6% 9852 ± 4% softirqs.CPU16.RCU 19449 ± 5% -48.5% 10016 ± 4% softirqs.CPU17.RCU 19394 ± 4% -49.3% 9832 ± 5% softirqs.CPU18.RCU 20317 ± 5% -50.6% 10044 ± 5% softirqs.CPU19.RCU 7704 ± 29% -54.3% 3519 ± 16% softirqs.CPU19.SCHED 18679 ± 3% -49.3% 9472 ± 6% softirqs.CPU2.RCU 8663 ± 18% -45.7% 4700 ± 12% softirqs.CPU2.SCHED 22698 ± 7% -54.9% 10233 ± 6% softirqs.CPU20.RCU 19652 ± 5% -38.3% 12135 ± 17% softirqs.CPU21.RCU 19819 ± 6% -49.0% 10107 ± 5% softirqs.CPU22.RCU 19786 ± 5% -47.2% 10443 ± 5% softirqs.CPU23.RCU 19436 ± 10% -49.1% 9886 ± 7% softirqs.CPU24.RCU 19458 ± 10% -48.6% 10001 ± 8% softirqs.CPU25.RCU 19003 ± 10% -48.8% 9728 ± 7% softirqs.CPU26.RCU 19271 ± 10% -48.4% 9944 ± 6% softirqs.CPU27.RCU 19392 ± 11% -49.1% 9869 ± 6% softirqs.CPU28.RCU 19078 ± 11% -50.9% 9366 ± 10% softirqs.CPU29.RCU 18580 ± 5% -47.4% 9776 ± 4% softirqs.CPU3.RCU 19226 ± 10% -49.4% 9736 ± 8% softirqs.CPU30.RCU 19141 ± 11% -48.0% 9955 ± 9% softirqs.CPU31.RCU 17632 ± 15% -47.2% 9308 ± 4% softirqs.CPU32.RCU 17815 ± 8% -49.1% 9063 ± 5% softirqs.CPU33.RCU 18371 ± 8% -49.2% 9336 ± 5% softirqs.CPU34.RCU 18134 ± 8% -48.9% 9266 ± 3% softirqs.CPU35.RCU 17821 ± 9% -50.8% 8768 ± 7% softirqs.CPU36.RCU 17941 ± 8% -48.3% 9267 ± 5% softirqs.CPU37.RCU 18045 ± 9% -47.5% 9472 ± 9% softirqs.CPU38.RCU 17753 ± 7% -49.1% 9042 ± 5% softirqs.CPU39.RCU 18564 ± 6% -48.1% 9627 ± 5% softirqs.CPU4.RCU 17696 ± 8% -48.4% 9136 ± 4% softirqs.CPU40.RCU 17910 ± 8% -48.2% 9281 ± 3% softirqs.CPU41.RCU 17891 ± 7% -49.1% 9101 ± 3% softirqs.CPU42.RCU 17650 ± 8% -48.2% 9151 ± 3% softirqs.CPU43.RCU 17670 ± 8% -48.9% 9021 ± 4% softirqs.CPU44.RCU 17950 ± 8% -49.5% 9061 ± 4% softirqs.CPU45.RCU 17297 ± 11% -45.5% 9429 ± 4% softirqs.CPU46.RCU 17974 ± 8% -45.9% 9726 ± 6% softirqs.CPU47.RCU 18337 ± 5% -48.6% 9418 ± 5% softirqs.CPU48.RCU 18280 ± 5% -48.1% 9484 ± 4% softirqs.CPU49.RCU 17985 ± 5% -48.3% 9298 ± 4% softirqs.CPU5.RCU 18142 ± 5% -48.5% 9340 ± 5% softirqs.CPU50.RCU 18265 ± 5% -47.6% 9573 ± 2% softirqs.CPU51.RCU 18258 ± 6% -48.7% 9365 ± 3% softirqs.CPU52.RCU 17934 ± 5% -48.5% 9240 ± 3% softirqs.CPU53.RCU 17871 ± 6% -47.1% 9453 ± 2% softirqs.CPU54.RCU 18220 ± 5% -48.3% 9423 ± 4% softirqs.CPU55.RCU 17961 ± 5% -48.7% 9208 ± 6% softirqs.CPU56.RCU 17752 ± 5% -51.1% 8688 ± 5% softirqs.CPU57.RCU 18358 ± 4% -48.4% 9466 ± 5% softirqs.CPU58.RCU 17603 ± 5% -48.1% 9138 ± 4% softirqs.CPU59.RCU 17838 ± 6% -48.6% 9168 ± 3% softirqs.CPU6.RCU 19581 ± 18% -53.7% 9074 ± 6% softirqs.CPU60.RCU 17896 ± 5% -47.4% 9418 ± 5% softirqs.CPU61.RCU 16595 ± 8% -43.5% 9370 ± 7% softirqs.CPU62.RCU 18176 -48.7% 9318 ± 5% softirqs.CPU63.RCU 19405 ± 5% -48.0% 10088 ± 5% softirqs.CPU64.RCU 19307 ± 3% -50.2% 9616 ± 6% softirqs.CPU65.RCU 19248 ± 5% -48.5% 9911 ± 6% softirqs.CPU66.RCU 19507 ± 5% -49.9% 9773 ± 4% softirqs.CPU67.RCU 20475 ± 6% -51.6% 9913 ± 5% softirqs.CPU68.RCU 19852 ± 4% -37.8% 12353 ± 25% softirqs.CPU69.RCU 18247 ± 5% -48.6% 9373 ± 3% softirqs.CPU7.RCU 19627 ± 4% -48.5% 10109 ± 5% softirqs.CPU70.RCU 20240 ± 6% -39.5% 12240 ± 23% softirqs.CPU71.RCU 19200 ± 10% -47.7% 10046 ± 8% softirqs.CPU72.RCU 19267 ± 10% -50.0% 9631 ± 6% softirqs.CPU73.RCU 19258 ± 10% -48.0% 10017 ± 13% softirqs.CPU74.RCU 19359 ± 10% -50.6% 9558 ± 7% softirqs.CPU75.RCU 19419 ± 11% -49.2% 9874 ± 8% softirqs.CPU76.RCU 19053 ± 10% -49.3% 9660 ± 7% softirqs.CPU77.RCU 19200 ± 10% -50.6% 9493 ± 7% softirqs.CPU78.RCU 19387 ± 10% -49.9% 9710 ± 10% softirqs.CPU79.RCU 18383 ± 5% -47.7% 9608 ± 6% softirqs.CPU8.RCU 17093 ± 15% -45.6% 9307 ± 5% softirqs.CPU80.RCU 17643 ± 7% -46.7% 9410 ± 8% softirqs.CPU81.RCU 18138 ± 9% -49.7% 9116 ± 5% softirqs.CPU82.RCU 17943 ± 9% -49.4% 9083 ± 6% softirqs.CPU83.RCU 17899 ± 10% -52.4% 8519 ± 10% softirqs.CPU84.RCU 17664 ± 8% -47.1% 9352 ± 5% softirqs.CPU85.RCU 17767 ± 9% -49.0% 9069 ± 7% softirqs.CPU86.RCU 17548 ± 9% -49.1% 8925 ± 6% softirqs.CPU87.RCU 17412 ± 8% -49.4% 8816 ± 4% softirqs.CPU88.RCU 17769 ± 10% -50.0% 8886 ± 2% softirqs.CPU89.RCU 17825 ± 6% -49.5% 9007 ± 3% softirqs.CPU9.RCU 17319 ± 10% -49.3% 8773 ± 3% softirqs.CPU90.RCU 17563 ± 10% -49.9% 8803 ± 3% softirqs.CPU91.RCU 17260 ± 9% -47.9% 8984 ± 4% softirqs.CPU92.RCU 17682 ± 7% -49.6% 8903 ± 6% softirqs.CPU93.RCU 17757 ± 16% -48.5% 9151 ± 5% softirqs.CPU94.RCU 17566 ± 10% -48.0% 9139 ± 2% softirqs.CPU95.RCU 10681 ± 6% -25.5% 7953 ± 9% softirqs.NET_RX 1777579 ± 6% -48.4% 916589 ± 3% softirqs.RCU 650400 ± 4% -43.2% 369606 ± 12% softirqs.SCHED 28944 -22.0% 22563 softirqs.TIMER *************************************************************************************************** lkp-csl-2sp8: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 128G memory ========================================================================================= compiler/cpufreq_governor/kconfig/option_a/option_b/option_c/option_d/option_e/option_f/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/Sequential Read/IO_uring/No/No/4KB/Default Test Directory/debian-x86_64-phoronix/lkp-csl-2sp8/fio-1.14.1/phoronix-test-suite/0x5003006 commit: v5.16-rc6 b08e888d97 ("sched/fair: Force progress on min_vruntime") v5.16-rc6 b08e888d9715f01546da7a40ca3 ---------------- --------------------------- %stddev %change %stddev \ | \ 376365 +17.6% 442631 phoronix-test-suite.fio.SequentialRead.IO_uring.No.No.4KB.DefaultTestDirectory.iops 1470 +17.6% 1729 phoronix-test-suite.fio.SequentialRead.IO_uring.No.No.4KB.DefaultTestDirectory.mb_s 1.025e+09 ± 8% +27.1% 1.303e+09 ± 3% phoronix-test-suite.time.file_system_inputs 230278 ± 2% +2.5% 236078 phoronix-test-suite.time.minor_page_faults 205.20 +30.9% 268.67 phoronix-test-suite.time.percent_of_cpu_this_job_got 852.47 ± 10% +46.6% 1249 ± 2% phoronix-test-suite.time.system_time 1.708e+08 ± 8% +17.1% 2e+08 ± 5% cpuidle..usage 1.87 ± 13% -0.5 1.40 mpstat.cpu.all.irq% 0.44 ± 4% -0.1 0.38 mpstat.cpu.all.soft% 2.12 ± 2% +0.8 2.90 mpstat.cpu.all.sys% 49503 ± 17% +84.9% 91516 ± 16% numa-meminfo.node1.Active 22948 ± 60% +239.7% 77953 ± 5% numa-meminfo.node1.Active(anon) 35605 ±104% -67.0% 11758 ± 18% numa-meminfo.node1.AnonPages 5739 ± 61% +239.6% 19492 ± 6% numa-vmstat.node1.nr_active_anon 8889 ±104% -67.0% 2934 ± 18% numa-vmstat.node1.nr_anon_pages 5739 ± 61% +239.6% 19492 ± 6% numa-vmstat.node1.nr_zone_active_anon 1053346 +18.2% 1245049 vmstat.io.bi 3.00 +33.3% 4.00 vmstat.procs.r 958801 -22.0% 747861 vmstat.system.cs 467562 ± 5% -12.3% 410115 vmstat.system.in 263481 +2.9% 271242 proc-vmstat.nr_active_anon 102192 -1.0% 101167 proc-vmstat.nr_anon_pages 366673 -4.0% 351942 proc-vmstat.nr_inactive_anon 27458 ± 2% -9.7% 24795 ± 2% proc-vmstat.nr_mapped 263481 +2.9% 271242 proc-vmstat.nr_zone_active_anon 366673 -4.0% 351942 proc-vmstat.nr_zone_inactive_anon 63827 ± 12% +26.0% 80425 ± 2% proc-vmstat.numa_hint_faults 51593 ± 43% -84.7% 7882 ± 2% proc-vmstat.numa_hint_faults_local 872033 ± 3% -60.5% 344383 proc-vmstat.pgactivate 5.126e+08 ± 8% +27.1% 6.517e+08 ± 3% proc-vmstat.pgpgin 2007 ± 4% +13.8% 2285 turbostat.Bzy_MHz 38566878 ± 37% +93.4% 74573651 ± 2% turbostat.C1 0.77 ± 20% +1.1 1.89 turbostat.C1% 47178399 ± 30% +101.3% 94964964 ± 9% turbostat.C1E 23.77 ± 59% +58.7 82.46 ± 14% turbostat.C1E% 43166471 ± 6% -72.5% 11880364 ±134% turbostat.C6 69.31 ± 20% -59.9 9.37 ±125% turbostat.C6% 41833970 ± 33% -55.7% 18521335 ± 6% turbostat.POLL 0.26 ± 7% -0.1 0.16 ± 3% turbostat.POLL% 127.23 +3.0% 130.99 turbostat.PkgWatt 1.65e+09 +7.8% 1.779e+09 perf-stat.i.branch-instructions 20358047 +12.2% 22837712 ± 2% perf-stat.i.cache-misses 1.713e+08 ± 9% +16.1% 1.989e+08 perf-stat.i.cache-references 966764 -22.1% 753134 perf-stat.i.context-switches 3.10 ± 13% -22.5% 2.40 ± 3% perf-stat.i.cpi 1.343e+10 ± 3% +8.9% 1.463e+10 perf-stat.i.cpu-cycles 489.18 ± 7% +3478.5% 17505 ± 3% perf-stat.i.cpu-migrations 1147 ± 4% +81.9% 2086 ± 32% perf-stat.i.cycles-between-cache-misses 0.26 ± 27% -0.2 0.04 ±124% perf-stat.i.dTLB-load-miss-rate% 1442652 ± 22% -78.3% 313493 ± 93% perf-stat.i.dTLB-load-misses 0.06 ± 25% -0.0 0.01 ± 98% perf-stat.i.dTLB-store-miss-rate% 213418 ± 16% -70.8% 62308 ± 69% perf-stat.i.dTLB-store-misses 1.256e+09 ± 2% +8.3% 1.361e+09 ± 3% perf-stat.i.dTLB-stores 68.35 ± 5% -10.5 57.82 ± 2% perf-stat.i.iTLB-load-miss-rate% 1664243 ± 15% +52.7% 2540790 perf-stat.i.iTLB-loads 7.841e+09 ± 2% +10.6% 8.672e+09 perf-stat.i.instructions 139818 ± 3% +8.9% 152314 perf-stat.i.metric.GHz 56897181 ± 2% +10.3% 62736780 ± 2% perf-stat.i.metric.M/sec 7434466 ± 10% +11.6% 8293419 perf-stat.i.node-loads 40.59 ± 5% +20.5 61.09 ± 18% perf-stat.i.node-store-miss-rate% 89739 ± 10% +85.0% 166001 ± 14% perf-stat.i.node-store-misses 178412 ± 5% -37.2% 112025 ± 24% perf-stat.i.node-stores 0.06 ± 24% -0.1 0.01 ± 87% perf-stat.overall.dTLB-load-miss-rate% 0.02 ± 15% -0.0 0.00 ± 73% perf-stat.overall.dTLB-store-miss-rate% 73.61 ± 2% -8.8 64.85 perf-stat.overall.iTLB-load-miss-rate% 33.44 ± 8% +26.4 59.82 ± 15% perf-stat.overall.node-store-miss-rate% 1.646e+09 +7.8% 1.775e+09 perf-stat.ps.branch-instructions 20305229 +12.2% 22783903 ± 2% perf-stat.ps.cache-misses 1.708e+08 ± 9% +16.1% 1.984e+08 perf-stat.ps.cache-references 964170 -22.1% 751337 perf-stat.ps.context-switches 1.339e+10 ± 3% +9.0% 1.46e+10 perf-stat.ps.cpu-cycles 487.88 ± 7% +3479.5% 17463 ± 3% perf-stat.ps.cpu-migrations 1439904 ± 22% -78.3% 312915 ± 93% perf-stat.ps.dTLB-load-misses 212992 ± 16% -70.8% 62193 ± 69% perf-stat.ps.dTLB-store-misses 1.253e+09 ± 2% +8.4% 1.358e+09 ± 3% perf-stat.ps.dTLB-stores 1660326 ± 15% +52.7% 2535381 perf-stat.ps.iTLB-loads 7.821e+09 ± 2% +10.6% 8.652e+09 perf-stat.ps.instructions 7414336 ± 10% +11.6% 8273682 perf-stat.ps.node-loads 89545 ± 10% +85.0% 165628 ± 14% perf-stat.ps.node-store-misses 177973 ± 5% -37.2% 111790 ± 24% perf-stat.ps.node-stores 3.782e+12 ± 8% +19.2% 4.507e+12 ± 3% perf-stat.total.instructions 59536380 ± 7% -42.5% 34257639 ± 3% softirqs.BLOCK 1054716 ± 12% -36.9% 665282 ± 26% softirqs.CPU0.BLOCK 81062 ± 9% -24.4% 61259 ± 2% softirqs.CPU0.RCU 1036406 ± 10% -38.3% 639408 ± 30% softirqs.CPU1.BLOCK 79360 ± 8% -23.7% 60570 ± 2% softirqs.CPU1.RCU 1253755 ± 21% -39.3% 760509 ± 7% softirqs.CPU10.BLOCK 80309 ± 9% -23.9% 61075 softirqs.CPU10.RCU 1245927 ± 17% -36.0% 797804 ± 6% softirqs.CPU11.BLOCK 82012 ± 9% -26.2% 60543 softirqs.CPU11.RCU 1285058 ± 13% -38.3% 793232 ± 20% softirqs.CPU12.BLOCK 82271 ± 8% -26.6% 60384 softirqs.CPU12.RCU 1283187 ± 9% -45.7% 696888 ± 14% softirqs.CPU13.BLOCK 78945 ± 7% -24.5% 59623 ± 2% softirqs.CPU13.RCU 1373427 ± 13% -52.7% 649978 ± 11% softirqs.CPU14.BLOCK 1.00 +1.4e+06% 14357 ±141% softirqs.CPU14.NET_RX 79448 ± 8% -24.3% 60178 ± 3% softirqs.CPU14.RCU 1383327 ± 9% -50.9% 679618 ± 7% softirqs.CPU15.BLOCK 79982 ± 9% -23.7% 61005 ± 3% softirqs.CPU15.RCU 1386098 ± 16% -42.6% 795873 ± 10% softirqs.CPU16.BLOCK 1295085 ± 12% -34.9% 842505 ± 11% softirqs.CPU17.BLOCK 70277 ± 10% -20.4% 55961 softirqs.CPU17.RCU 1353234 ± 11% -38.8% 828325 ± 8% softirqs.CPU18.BLOCK 70744 ± 10% -20.0% 56594 softirqs.CPU18.RCU 1216182 ± 12% -39.7% 733842 ± 3% softirqs.CPU19.BLOCK 68584 ± 10% -18.9% 55611 softirqs.CPU19.RCU 977365 ± 13% -27.2% 711491 ± 5% softirqs.CPU2.BLOCK 81559 ± 8% -26.1% 60267 softirqs.CPU2.RCU 1393849 ± 23% -41.2% 819133 ± 3% softirqs.CPU20.BLOCK 70580 ± 10% -20.3% 56284 ± 2% softirqs.CPU20.RCU 1231720 ± 10% -31.6% 842435 ± 4% softirqs.CPU21.BLOCK 70460 ± 9% -19.7% 56548 softirqs.CPU21.RCU 1117879 ± 10% -30.2% 780585 ± 5% softirqs.CPU22.BLOCK 70077 ± 10% -20.0% 56048 softirqs.CPU22.RCU 1081738 ± 15% -30.3% 753762 ± 16% softirqs.CPU23.BLOCK 70734 ± 10% -19.1% 57234 softirqs.CPU23.RCU 69070 ± 9% -20.9% 54643 softirqs.CPU24.RCU 72214 ± 7% -25.2% 54044 softirqs.CPU26.RCU 76007 ± 12% -27.0% 55489 softirqs.CPU27.RCU 72355 ± 15% -26.0% 53577 ± 2% softirqs.CPU28.RCU 74771 ± 15% -27.7% 54088 ± 4% softirqs.CPU29.RCU 1190927 ± 11% -38.8% 728634 ± 16% softirqs.CPU3.BLOCK 81431 ± 8% -26.3% 60000 softirqs.CPU3.RCU 75434 ± 9% -29.1% 53446 ± 3% softirqs.CPU30.RCU 67359 ± 8% -24.0% 51162 ± 3% softirqs.CPU31.RCU 85242 ± 11% -27.7% 61636 ± 6% softirqs.CPU32.RCU 95777 ± 13% -34.2% 62981 ± 4% softirqs.CPU33.RCU 90274 ± 10% -28.7% 64354 ± 4% softirqs.CPU34.RCU 97748 ± 11% -33.8% 64709 ± 5% softirqs.CPU35.RCU 92039 ± 12% -30.7% 63826 ± 4% softirqs.CPU36.RCU 84095 ± 10% -24.6% 63425 ± 3% softirqs.CPU38.RCU 78728 ± 11% -19.4% 63422 ± 3% softirqs.CPU39.RCU 1196133 ± 16% -39.8% 720416 ± 7% softirqs.CPU4.BLOCK 81523 ± 7% -25.7% 60573 ± 2% softirqs.CPU4.RCU 90519 ± 12% -28.4% 64803 ± 3% softirqs.CPU40.RCU 98051 ± 14% -33.9% 64811 ± 4% softirqs.CPU41.RCU 98731 ± 14% -35.3% 63879 ± 3% softirqs.CPU42.RCU 90545 ± 18% -31.8% 61738 ± 3% softirqs.CPU43.RCU 94571 ± 13% -32.8% 63590 ± 7% softirqs.CPU44.RCU 93471 ± 14% -31.9% 63681 ± 3% softirqs.CPU45.RCU 96331 ± 15% -34.7% 62900 ± 7% softirqs.CPU46.RCU 89977 ± 14% -30.0% 63014 ± 3% softirqs.CPU47.RCU 980374 ± 16% -40.5% 583777 ± 26% softirqs.CPU48.BLOCK 80946 ± 11% -26.2% 59718 ± 10% softirqs.CPU48.RCU 911068 ± 10% -40.5% 541667 ± 31% softirqs.CPU49.BLOCK 76174 ± 10% -21.6% 59700 ± 4% softirqs.CPU49.RCU 1074881 ± 18% -32.5% 725037 ± 15% softirqs.CPU5.BLOCK 82459 ± 7% -26.6% 60519 softirqs.CPU5.RCU 957147 ± 11% -31.4% 656173 ± 10% softirqs.CPU50.BLOCK 84198 ± 9% -27.6% 60930 softirqs.CPU50.RCU 1124759 ± 14% -41.7% 655333 ± 17% softirqs.CPU51.BLOCK 82209 ± 9% -25.1% 61544 softirqs.CPU51.RCU 1101818 ± 16% -38.8% 674827 ± 11% softirqs.CPU52.BLOCK 82512 ± 9% -25.0% 61899 softirqs.CPU52.RCU 989601 ± 20% -36.3% 630829 ± 10% softirqs.CPU53.BLOCK 84347 ± 8% -25.9% 62524 ± 2% softirqs.CPU53.RCU 85255 ± 8% -27.7% 61649 softirqs.CPU54.RCU 1104742 ± 10% -39.9% 663938 ± 4% softirqs.CPU55.BLOCK 83970 ± 8% -25.7% 62425 softirqs.CPU55.RCU 82692 ± 8% -23.3% 63464 ± 2% softirqs.CPU56.RCU 1235581 ± 18% -38.8% 756556 ± 10% softirqs.CPU57.BLOCK 81825 ± 9% -24.8% 61505 softirqs.CPU57.RCU 1159094 ± 22% -40.2% 693531 ± 4% softirqs.CPU58.BLOCK 81383 ± 9% -23.0% 62656 ± 2% softirqs.CPU58.RCU 1185793 ± 17% -43.8% 666626 ± 9% softirqs.CPU59.BLOCK 83557 ± 9% -24.5% 63065 ± 2% softirqs.CPU59.RCU 82050 ± 8% -25.0% 61501 softirqs.CPU6.RCU 1186248 ± 12% -37.9% 737065 ± 19% softirqs.CPU60.BLOCK 83555 ± 8% -24.5% 63109 softirqs.CPU60.RCU 1183126 ± 7% -48.4% 610813 ± 12% softirqs.CPU61.BLOCK 81162 ± 8% -21.9% 63365 ± 3% softirqs.CPU61.RCU 1258897 ± 6% -51.7% 608431 ± 14% softirqs.CPU62.BLOCK 80412 ± 9% -21.7% 62934 ± 2% softirqs.CPU62.RCU 1298250 ± 8% -54.3% 593774 softirqs.CPU63.BLOCK 81865 ± 9% -24.4% 61867 softirqs.CPU63.RCU 1267325 ± 16% -43.7% 713505 ± 8% softirqs.CPU64.BLOCK 69509 ± 11% -20.2% 55456 ± 2% softirqs.CPU64.RCU 1205294 ± 9% -39.6% 727622 ± 10% softirqs.CPU65.BLOCK 71149 ± 9% -23.3% 54607 ± 4% softirqs.CPU65.RCU 63147 ± 8% +9.8% 69320 ± 2% softirqs.CPU65.SCHED 1265229 ± 9% -40.7% 750819 ± 10% softirqs.CPU66.BLOCK 70916 ± 9% -19.5% 57064 softirqs.CPU66.RCU 1109606 ± 11% -42.7% 635867 ± 11% softirqs.CPU67.BLOCK 1251227 ± 19% -41.7% 729624 ± 3% softirqs.CPU68.BLOCK 63430 ± 9% +9.7% 69614 ± 2% softirqs.CPU68.SCHED 1120943 ± 12% -28.6% 800259 ± 7% softirqs.CPU69.BLOCK 1168642 ± 10% -39.5% 707200 ± 10% softirqs.CPU7.BLOCK 80444 ± 8% -26.7% 58966 softirqs.CPU7.RCU 1040410 ± 8% -32.2% 704979 ± 10% softirqs.CPU70.BLOCK 63006 ± 10% +13.4% 71424 ± 5% softirqs.CPU70.SCHED 1002134 ± 14% -33.0% 671645 ± 14% softirqs.CPU71.BLOCK 71269 ± 10% -18.4% 58126 softirqs.CPU71.RCU 43717 ±199% -100.0% 3.67 ±105% softirqs.CPU72.BLOCK 70931 ± 7% -22.1% 55283 ± 3% softirqs.CPU72.RCU 74362 ± 10% -25.2% 55647 ± 2% softirqs.CPU74.RCU 77602 ± 13% -29.4% 54793 ± 3% softirqs.CPU75.RCU 73523 ± 15% -26.0% 54434 ± 2% softirqs.CPU76.RCU 75483 ± 16% -28.1% 54242 softirqs.CPU77.RCU 74803 ± 10% -28.2% 53724 ± 4% softirqs.CPU78.RCU 68209 ± 9% -19.6% 54841 ± 2% softirqs.CPU79.RCU 1209453 ± 19% -40.7% 717410 ± 16% softirqs.CPU8.BLOCK 80402 ± 9% -26.9% 58734 softirqs.CPU8.RCU 84733 ± 10% -29.6% 59691 ± 4% softirqs.CPU80.RCU 93194 ± 12% -33.5% 62016 ± 3% softirqs.CPU81.RCU 89513 ± 10% -30.5% 62186 ± 3% softirqs.CPU82.RCU 95911 ± 12% -34.0% 63311 ± 4% softirqs.CPU83.RCU 90266 ± 12% -28.8% 64279 ± 3% softirqs.CPU84.RCU 85421 ± 11% -29.1% 60544 ± 3% softirqs.CPU86.RCU 78500 ± 10% -22.9% 60525 ± 6% softirqs.CPU87.RCU 89451 ± 12% -29.6% 62930 ± 3% softirqs.CPU88.RCU 97789 ± 14% -35.0% 63573 ± 4% softirqs.CPU89.RCU 1273783 ± 13% -35.3% 823995 ± 12% softirqs.CPU9.BLOCK 80463 ± 8% -24.6% 60684 softirqs.CPU9.RCU 96366 ± 15% -33.7% 63900 ± 4% softirqs.CPU90.RCU 89292 ± 17% -30.5% 62086 ± 2% softirqs.CPU91.RCU 74019 ±198% -100.0% 0.67 ±141% softirqs.CPU92.BLOCK 93591 ± 13% -33.3% 62444 ± 4% softirqs.CPU92.RCU 93721 ± 13% -33.4% 62412 ± 3% softirqs.CPU93.RCU 93243 ± 12% -32.4% 63053 ± 3% softirqs.CPU94.RCU 89195 ± 11% -29.4% 63007 ± 3% softirqs.CPU95.RCU 7767504 ± 10% -26.1% 5739783 ± 2% softirqs.RCU 15.21 ± 8% -6.8 8.39 ± 2% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event 14.33 ± 8% -6.4 7.92 ± 2% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow 14.27 ± 8% -6.4 7.88 ± 2% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow 6.70 ± 6% -3.2 3.51 perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule 6.67 ± 7% -3.2 3.49 perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch 13.40 ± 4% -3.1 10.26 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.io_wqe_worker.ret_from_fork 8.02 ± 5% -3.1 4.93 ± 2% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward 7.82 ± 6% -2.9 4.91 ± 7% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr 7.79 ± 6% -2.9 4.89 ± 7% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime 5.24 ± 23% -2.5 2.71 ± 4% perf-profile.calltrace.cycles-pp.perf_callchain_user.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward 4.72 ± 6% -2.4 2.31 ± 2% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule 4.62 ± 7% -2.4 2.25 perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair 4.54 ± 7% -2.3 2.20 ± 2% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity 4.28 ± 11% -2.3 1.96 ± 2% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule 4.93 ± 6% -2.3 2.66 ± 7% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule 15.17 ± 4% -2.0 13.19 ± 5% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.io_wqe_worker.ret_from_fork 6.05 ± 5% -2.0 4.10 ± 4% perf-profile.calltrace.cycles-pp.loop_process_work.process_one_work.worker_thread.kthread.ret_from_fork 15.79 ± 4% -1.9 13.84 ± 5% perf-profile.calltrace.cycles-pp.schedule_timeout.io_wqe_worker.ret_from_fork 6.12 ± 5% -1.9 4.19 ± 5% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 3.42 ± 16% -1.8 1.60 ± 10% perf-profile.calltrace.cycles-pp.newidle_balance.pick_next_task_fair.__schedule.schedule.schedule_timeout 10.44 ± 4% -1.8 8.65 ± 6% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 5.88 ± 7% -1.6 4.30 ± 8% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.io_wqe_worker 3.59 ± 14% -1.6 2.01 ± 2% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.schedule_timeout 4.34 ± 9% -1.5 2.82 ± 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.schedule_timeout 2.79 ± 17% -1.3 1.44 ± 24% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair.__schedule 2.73 ± 17% -1.3 1.40 ± 25% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair 2.51 ± 6% -1.3 1.21 ± 3% perf-profile.calltrace.cycles-pp.raise_softirq.blk_mq_complete_request_remote.blk_mq_complete_request.loop_process_work.process_one_work 2.64 ± 6% -1.3 1.34 ± 2% perf-profile.calltrace.cycles-pp.blk_mq_complete_request.loop_process_work.process_one_work.worker_thread.kthread 2.63 ± 6% -1.3 1.33 ± 2% perf-profile.calltrace.cycles-pp.blk_mq_complete_request_remote.blk_mq_complete_request.loop_process_work.process_one_work.worker_thread 2.47 ± 6% -1.3 1.19 ± 3% perf-profile.calltrace.cycles-pp.try_to_wake_up.raise_softirq.blk_mq_complete_request_remote.blk_mq_complete_request.loop_process_work 3.97 ± 3% -0.9 3.04 ± 7% perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork 1.60 ± 14% -0.9 0.70 ± 34% perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample 2.93 ± 8% -0.9 2.05 perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.schedule_timeout.io_wqe_worker 6.41 ± 5% -0.9 5.56 ± 6% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 1.09 ± 9% -0.7 0.36 ± 70% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity 3.58 ± 9% -0.7 2.90 ± 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.io_wqe_worker 1.28 ± 7% -0.6 0.65 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.raise_softirq.blk_mq_complete_request_remote 1.29 ± 8% -0.6 0.66 ± 3% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.raise_softirq.blk_mq_complete_request_remote.blk_mq_complete_request 2.20 ± 4% -0.6 1.58 ± 8% perf-profile.calltrace.cycles-pp.schedule.smpboot_thread_fn.kthread.ret_from_fork 1.24 ± 8% -0.6 0.63 ± 3% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.raise_softirq 2.18 ± 4% -0.6 1.57 ± 8% perf-profile.calltrace.cycles-pp.__schedule.schedule.smpboot_thread_fn.kthread.ret_from_fork 1.18 ± 9% -0.6 0.58 ± 5% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up 1.15 ± 9% -0.6 0.55 ± 3% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate 3.53 ± 5% -0.6 2.95 ± 10% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 1.12 ± 8% -0.6 0.54 ± 4% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair 3.18 ± 4% -0.5 2.69 ± 10% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 3.70 ± 8% -0.5 3.22 ± 13% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.ttwu_do_wakeup 1.70 ± 3% -0.3 1.42 ± 5% perf-profile.calltrace.cycles-pp.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 1.99 ± 9% -0.3 1.71 ± 4% perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry 1.69 ± 2% -0.3 1.41 ± 5% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread.ret_from_fork 1.18 ± 9% -0.3 0.93 ± 6% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 1.62 ± 3% -0.2 1.38 ± 5% perf-profile.calltrace.cycles-pp.blk_complete_reqs.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn.kthread 1.46 ± 3% -0.2 1.24 ± 5% perf-profile.calltrace.cycles-pp.blk_mq_end_request.blk_complete_reqs.__softirqentry_text_start.run_ksoftirqd.smpboot_thread_fn 1.38 ± 2% -0.2 1.17 ± 5% perf-profile.calltrace.cycles-pp.blk_update_request.blk_mq_end_request.blk_complete_reqs.__softirqentry_text_start.run_ksoftirqd 0.98 ± 2% -0.2 0.78 ± 6% perf-profile.calltrace.cycles-pp.iomap_dio_bio_end_io.blk_update_request.blk_mq_end_request.blk_complete_reqs.__softirqentry_text_start 1.19 ± 7% -0.2 1.01 ± 14% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 1.65 ± 4% -0.1 1.51 ± 3% perf-profile.calltrace.cycles-pp.iomap_dio_bio_iter.__iomap_dio_rw.iomap_dio_rw.ext4_file_read_iter.io_read 0.63 ± 3% -0.1 0.57 ± 5% perf-profile.calltrace.cycles-pp.iov_iter_get_pages.bio_iov_iter_get_pages.iomap_dio_bio_iter.__iomap_dio_rw.iomap_dio_rw 0.87 ± 8% +0.2 1.02 ± 4% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule_idle.do_idle 0.89 ± 8% +0.2 1.04 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule_idle.do_idle.cpu_startup_entry 0.68 ± 9% +0.2 0.92 ± 7% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter_atomic.generic_perform_write.__generic_file_write_iter 0.71 ± 9% +0.2 0.96 ± 8% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter_atomic.generic_perform_write.__generic_file_write_iter.generic_file_write_iter 0.75 ± 9% +0.3 1.04 ± 8% perf-profile.calltrace.cycles-pp.copy_page_from_iter_atomic.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write 0.36 ± 83% +0.4 0.73 ± 17% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 2.73 ± 9% +0.4 3.16 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.io_worker_handle_work.io_wqe_worker.ret_from_fork 2.86 ± 9% +0.5 3.31 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.io_worker_handle_work.io_wqe_worker.ret_from_fork 0.23 ±122% +0.5 0.74 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock.io_wqe_enqueue.io_queue_async_work.io_submit_sqes.__x64_sys_io_uring_enter 0.10 ±200% +0.6 0.66 ± 10% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.io_wqe_enqueue.io_queue_async_work.io_submit_sqes 0.00 +0.6 0.56 ± 6% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.worker_thread.kthread 0.00 +0.6 0.61 ± 6% perf-profile.calltrace.cycles-pp.shmem_write_end.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write 0.00 +0.6 0.65 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.loop_queue_rq.__blk_mq_try_issue_directly.blk_mq_request_issue_directly 0.00 +0.7 0.66 ± 6% perf-profile.calltrace.cycles-pp.__get_user_nocheck_1.fault_in_readable.fault_in_iov_iter_readable.generic_perform_write.__generic_file_write_iter 0.00 +0.7 0.71 ± 6% perf-profile.calltrace.cycles-pp.fault_in_readable.fault_in_iov_iter_readable.generic_perform_write.__generic_file_write_iter.generic_file_write_iter 0.00 +0.7 0.73 ± 6% perf-profile.calltrace.cycles-pp.fault_in_iov_iter_readable.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write 0.00 +0.8 0.78 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.loop_queue_rq.__blk_mq_try_issue_directly.blk_mq_request_issue_directly.blk_mq_flush_plug_list 2.26 ± 6% +1.0 3.23 ± 7% perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write 2.31 ± 6% +1.0 3.33 ± 7% perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.new_sync_write.vfs_write.ksys_write 0.00 +1.1 1.06 ± 9% perf-profile.calltrace.cycles-pp.try_to_wake_up.__queue_work.queue_work_on.loop_queue_rq.__blk_mq_try_issue_directly 2.40 ± 6% +1.1 3.51 ± 7% perf-profile.calltrace.cycles-pp.generic_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64 1.50 ± 18% +1.1 2.62 ± 13% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.io_wq_worker_sleeping.schedule.schedule_timeout 0.00 +1.1 1.12 ± 9% perf-profile.calltrace.cycles-pp.__queue_work.queue_work_on.loop_queue_rq.__blk_mq_try_issue_directly.blk_mq_request_issue_directly 1.60 ± 17% +1.1 2.73 ± 13% perf-profile.calltrace.cycles-pp._raw_spin_lock.io_wq_worker_sleeping.schedule.schedule_timeout.io_wqe_worker 2.41 ± 6% +1.1 3.56 ± 7% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.18 ± 7% +1.2 4.34 ± 6% perf-profile.calltrace.cycles-pp.__libc_start_main 1.75 ± 16% +1.2 2.91 ± 13% perf-profile.calltrace.cycles-pp.io_wq_worker_sleeping.schedule.schedule_timeout.io_wqe_worker.ret_from_fork 2.68 ± 7% +1.3 3.94 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.__libc_start_main 2.51 ± 6% +1.3 3.78 ± 7% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 0.00 +1.3 1.28 ± 7% perf-profile.calltrace.cycles-pp.queue_work_on.loop_queue_rq.__blk_mq_try_issue_directly.blk_mq_request_issue_directly.blk_mq_flush_plug_list 2.69 ± 7% +1.3 3.98 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write.__libc_start_main 0.00 +1.3 1.29 ± 8% perf-profile.calltrace.cycles-pp.__schedule.schedule.worker_thread.kthread.ret_from_fork 0.00 +1.3 1.30 ± 9% perf-profile.calltrace.cycles-pp.schedule.worker_thread.kthread.ret_from_fork 2.56 ± 6% +1.3 3.88 ± 7% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.__libc_start_main 2.80 ± 7% +1.4 4.18 ± 6% perf-profile.calltrace.cycles-pp.__libc_write.__libc_start_main 3.28 ± 2% +1.4 4.69 ± 4% perf-profile.calltrace.cycles-pp.iomap_dio_rw.ext4_file_read_iter.io_read.io_issue_sqe.io_wq_submit_work 3.27 ± 3% +1.4 4.69 ± 4% perf-profile.calltrace.cycles-pp.__iomap_dio_rw.iomap_dio_rw.ext4_file_read_iter.io_read.io_issue_sqe 3.53 ± 3% +1.4 4.97 ± 4% perf-profile.calltrace.cycles-pp.ext4_file_read_iter.io_read.io_issue_sqe.io_wq_submit_work.io_worker_handle_work 3.72 ± 3% +1.4 5.17 ± 4% perf-profile.calltrace.cycles-pp.io_read.io_issue_sqe.io_wq_submit_work.io_worker_handle_work.io_wqe_worker 3.74 ± 3% +1.5 5.19 ± 4% perf-profile.calltrace.cycles-pp.io_issue_sqe.io_wq_submit_work.io_worker_handle_work.io_wqe_worker.ret_from_fork 3.78 ± 3% +1.5 5.24 ± 4% perf-profile.calltrace.cycles-pp.io_wq_submit_work.io_worker_handle_work.io_wqe_worker.ret_from_fork 0.81 ± 2% +1.5 2.31 ± 6% perf-profile.calltrace.cycles-pp.loop_queue_rq.__blk_mq_try_issue_directly.blk_mq_request_issue_directly.blk_mq_flush_plug_list.blk_flush_plug 0.83 ± 2% +1.5 2.35 ± 5% perf-profile.calltrace.cycles-pp.__blk_mq_try_issue_directly.blk_mq_request_issue_directly.blk_mq_flush_plug_list.blk_flush_plug.blk_finish_plug 0.89 ± 2% +1.5 2.40 ± 5% perf-profile.calltrace.cycles-pp.blk_finish_plug.__iomap_dio_rw.iomap_dio_rw.ext4_file_read_iter.io_read 0.88 ± 2% +1.5 2.39 ± 6% perf-profile.calltrace.cycles-pp.blk_mq_flush_plug_list.blk_flush_plug.blk_finish_plug.__iomap_dio_rw.iomap_dio_rw 0.84 ± 2% +1.5 2.36 ± 6% perf-profile.calltrace.cycles-pp.blk_mq_request_issue_directly.blk_mq_flush_plug_list.blk_flush_plug.blk_finish_plug.__iomap_dio_rw 0.88 ± 2% +1.5 2.40 ± 5% perf-profile.calltrace.cycles-pp.blk_flush_plug.blk_finish_plug.__iomap_dio_rw.iomap_dio_rw.ext4_file_read_iter 4.23 ± 13% +1.6 5.82 ± 11% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.io_wqe_worker.ret_from_fork 4.34 ± 13% +1.6 5.96 ± 11% perf-profile.calltrace.cycles-pp._raw_spin_lock.io_wqe_worker.ret_from_fork 7.48 ± 4% +2.1 9.54 ± 7% perf-profile.calltrace.cycles-pp.io_worker_handle_work.io_wqe_worker.ret_from_fork 24.63 ± 3% -5.7 18.94 ± 4% perf-profile.children.cycles-pp.__schedule 15.86 ± 5% -4.8 11.02 ± 3% perf-profile.children.cycles-pp.perf_prepare_sample 15.14 ± 5% -4.8 10.38 ± 3% perf-profile.children.cycles-pp.perf_callchain 15.08 ± 5% -4.8 10.33 ± 3% perf-profile.children.cycles-pp.get_perf_callchain 19.36 ± 4% -4.4 14.99 ± 4% perf-profile.children.cycles-pp.perf_tp_event 18.86 ± 4% -4.3 14.57 ± 4% perf-profile.children.cycles-pp.perf_swevent_overflow 18.83 ± 4% -4.3 14.54 ± 4% perf-profile.children.cycles-pp.__perf_event_overflow 18.75 ± 4% -4.3 14.48 ± 4% perf-profile.children.cycles-pp.perf_event_output_forward 17.60 ± 4% -3.7 13.88 ± 5% perf-profile.children.cycles-pp.schedule_timeout 19.52 ± 3% -3.4 16.13 ± 6% perf-profile.children.cycles-pp.schedule 5.64 ± 14% -2.7 2.89 ± 4% perf-profile.children.cycles-pp.perf_callchain_user 7.22 ± 3% -2.6 4.61 ± 2% perf-profile.children.cycles-pp.perf_trace_sched_switch 4.97 ± 13% -2.2 2.73 ± 4% perf-profile.children.cycles-pp.__get_user_nocheck_8 8.40 ± 5% -2.1 6.25 ± 4% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime 8.80 ± 5% -2.1 6.71 ± 4% perf-profile.children.cycles-pp.update_curr 6.06 ± 5% -2.0 4.11 ± 5% perf-profile.children.cycles-pp.loop_process_work 9.11 ± 2% -1.9 7.18 ± 3% perf-profile.children.cycles-pp.perf_callchain_kernel 6.13 ± 5% -1.9 4.19 ± 5% perf-profile.children.cycles-pp.process_one_work 10.44 ± 4% -1.8 8.65 ± 6% perf-profile.children.cycles-pp.kthread 5.43 ± 6% -1.7 3.72 ± 2% perf-profile.children.cycles-pp.dequeue_entity 5.50 ± 6% -1.7 3.80 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair 7.44 ± 2% -1.6 5.80 ± 3% perf-profile.children.cycles-pp.unwind_next_frame 2.52 ± 6% -1.3 1.21 ± 2% perf-profile.children.cycles-pp.raise_softirq 2.65 ± 6% -1.3 1.34 ± 2% perf-profile.children.cycles-pp.blk_mq_complete_request 2.63 ± 6% -1.3 1.33 ± 2% perf-profile.children.cycles-pp.blk_mq_complete_request_remote 6.85 ± 6% -1.1 5.78 ± 8% perf-profile.children.cycles-pp.pick_next_task_fair 4.56 ± 2% -1.0 3.58 ± 2% perf-profile.children.cycles-pp.__unwind_start 3.98 ± 3% -0.9 3.04 ± 7% perf-profile.children.cycles-pp.smpboot_thread_fn 6.41 ± 5% -0.9 5.56 ± 6% perf-profile.children.cycles-pp.worker_thread 1.23 ± 5% -0.8 0.40 ± 6% perf-profile.children.cycles-pp.preempt_schedule_common 1.33 ± 5% -0.8 0.50 ± 5% perf-profile.children.cycles-pp.__cond_resched 1.85 ± 5% -0.7 1.20 perf-profile.children.cycles-pp.ttwu_do_activate 1.82 ± 5% -0.7 1.17 perf-profile.children.cycles-pp.enqueue_task_fair 1.66 ± 5% -0.6 1.05 perf-profile.children.cycles-pp.enqueue_entity 2.54 -0.6 1.93 ± 2% perf-profile.children.cycles-pp.__orc_find 4.14 ± 4% -0.6 3.58 ± 9% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 3.64 ± 4% -0.5 3.17 ± 9% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 2.21 ± 3% -0.5 1.74 ± 6% perf-profile.children.cycles-pp.__softirqentry_text_start 2.51 ± 8% -0.3 2.18 ± 4% perf-profile.children.cycles-pp.finish_task_switch 1.41 ± 2% -0.3 1.09 ± 2% perf-profile.children.cycles-pp.memcpy_erms 1.20 ± 5% -0.3 0.90 ± 6% perf-profile.children.cycles-pp.orc_find 1.70 ± 3% -0.3 1.42 ± 5% perf-profile.children.cycles-pp.run_ksoftirqd 1.20 ± 9% -0.3 0.94 ± 5% perf-profile.children.cycles-pp.menu_select 1.63 ± 2% -0.2 1.39 ± 5% perf-profile.children.cycles-pp.blk_complete_reqs 1.47 ± 2% -0.2 1.25 ± 5% perf-profile.children.cycles-pp.blk_mq_end_request 1.38 ± 2% -0.2 1.18 ± 5% perf-profile.children.cycles-pp.blk_update_request 0.72 ± 8% -0.2 0.52 ± 7% perf-profile.children.cycles-pp.irq_exit_rcu 0.98 ± 2% -0.2 0.79 ± 7% perf-profile.children.cycles-pp.iomap_dio_bio_end_io 1.49 ± 5% -0.2 1.30 ± 12% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.75 ± 6% -0.2 0.57 ± 4% perf-profile.children.cycles-pp.stack_access_ok 0.49 ± 5% -0.2 0.33 ± 7% perf-profile.children.cycles-pp.io_req_task_work_add 0.28 ± 14% -0.2 0.12 ± 27% perf-profile.children.cycles-pp.irq_enter_rcu 0.27 ± 14% -0.2 0.11 ± 29% perf-profile.children.cycles-pp.tick_irq_enter 0.76 ± 4% -0.2 0.61 ± 8% perf-profile.children.cycles-pp.unwind_get_return_address 0.29 ± 20% -0.1 0.16 ± 6% perf-profile.children.cycles-pp.raw_spin_rq_lock_nested 1.65 ± 5% -0.1 1.52 ± 3% perf-profile.children.cycles-pp.iomap_dio_bio_iter 0.64 ± 6% -0.1 0.51 ± 10% perf-profile.children.cycles-pp.__kernel_text_address 0.48 ± 5% -0.1 0.37 ± 10% perf-profile.children.cycles-pp.kernel_text_address 0.27 ± 9% -0.1 0.15 ± 3% perf-profile.children.cycles-pp.task_work_add 0.56 ± 3% -0.1 0.45 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.33 ± 9% -0.1 0.22 ± 5% perf-profile.children.cycles-pp._find_next_bit 0.49 ± 4% -0.1 0.38 ± 3% perf-profile.children.cycles-pp.native_sched_clock 0.82 ± 6% -0.1 0.72 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.13 ± 10% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.mutex_lock 0.25 ± 6% -0.1 0.17 ± 4% perf-profile.children.cycles-pp.update_blocked_averages 0.27 ± 7% -0.1 0.20 ± 7% perf-profile.children.cycles-pp.__switch_to 0.11 ± 24% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.__hrtimer_next_event_base 0.18 ± 8% -0.1 0.11 ± 11% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.18 ± 10% -0.1 0.12 ± 14% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler 0.32 ± 5% -0.1 0.26 ± 18% perf-profile.children.cycles-pp.set_next_entity 0.12 ± 13% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.rcu_idle_exit 0.13 ± 12% -0.1 0.08 ± 17% perf-profile.children.cycles-pp.__module_address 0.63 ± 3% -0.1 0.58 ± 5% perf-profile.children.cycles-pp.iov_iter_get_pages 0.14 ± 15% -0.1 0.09 ± 15% perf-profile.children.cycles-pp.check_preempt_curr 0.08 ± 9% -0.1 0.03 ± 70% perf-profile.children.cycles-pp.rcu_eqs_exit 0.21 ± 6% -0.1 0.16 ± 5% perf-profile.children.cycles-pp.irqtime_account_irq 0.13 ± 15% -0.1 0.08 ± 10% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.24 ± 6% -0.0 0.19 ± 7% perf-profile.children.cycles-pp.read_tsc 0.19 ± 2% -0.0 0.15 ± 6% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.11 ± 9% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.run_rebalance_domains 0.27 ± 6% -0.0 0.23 ± 2% perf-profile.children.cycles-pp.__switch_to_asm 0.15 ± 13% -0.0 0.10 ± 12% perf-profile.children.cycles-pp.shmem_add_to_page_cache 0.22 ± 4% -0.0 0.18 ± 5% perf-profile.children.cycles-pp.ftrace_graph_ret_addr 0.11 ± 18% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.29 ± 7% -0.0 0.25 ± 3% perf-profile.children.cycles-pp.try_grab_compound_head 0.24 ± 9% -0.0 0.20 ± 10% perf-profile.children.cycles-pp.iomap_dio_complete 0.14 ± 7% -0.0 0.10 ± 19% perf-profile.children.cycles-pp.vfs_read 0.22 ± 5% -0.0 0.18 ± 13% perf-profile.children.cycles-pp.rebalance_domains 0.44 ± 5% -0.0 0.40 ± 4% perf-profile.children.cycles-pp.__perf_event_header__init_id 0.10 ± 13% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.update_irq_load_avg 0.14 ± 7% -0.0 0.10 ± 16% perf-profile.children.cycles-pp.ksys_read 0.15 ± 4% -0.0 0.12 ± 11% perf-profile.children.cycles-pp.core_kernel_text 0.07 ± 7% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.proc_reg_read 0.10 ± 7% -0.0 0.07 ± 12% perf-profile.children.cycles-pp.read 0.06 ± 6% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.get_page_from_freelist 0.09 ± 14% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.reweight_entity 0.11 ± 12% -0.0 0.08 perf-profile.children.cycles-pp.shmem_alloc_page 0.08 ± 12% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.perf_output_put_handle 0.09 ± 4% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.__wrgsbase_inactive 0.07 ± 10% -0.0 0.05 perf-profile.children.cycles-pp.arch_scale_freq_tick 0.09 ± 5% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.seq_read_iter 0.08 ± 14% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.tracing_gen_ctx_irq_test 0.07 ± 6% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.seq_read 0.08 ± 5% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.perf_trace_run_bpf_submit 0.15 ± 6% +0.0 0.17 ± 4% perf-profile.children.cycles-pp.ext4_map_blocks 0.10 ± 7% +0.0 0.12 ± 10% perf-profile.children.cycles-pp.ext4_es_lookup_extent 0.04 ± 51% +0.0 0.07 ± 11% perf-profile.children.cycles-pp.current_time 0.03 ± 81% +0.0 0.07 ± 7% perf-profile.children.cycles-pp.__fsnotify_parent 0.12 ± 8% +0.0 0.15 ± 6% perf-profile.children.cycles-pp.__list_del_entry_valid 0.16 ± 11% +0.0 0.20 ± 10% perf-profile.children.cycles-pp._raw_read_lock 0.14 ± 10% +0.0 0.18 ± 4% perf-profile.children.cycles-pp.io_wqe_dec_running 0.07 ± 13% +0.0 0.11 ± 11% perf-profile.children.cycles-pp.folio_mark_accessed 0.12 ± 12% +0.0 0.16 ± 5% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.29 ± 9% +0.0 0.34 ± 7% perf-profile.children.cycles-pp.lock_timer_base 0.07 ± 26% +0.1 0.13 ± 16% perf-profile.children.cycles-pp.calc_global_load_tick 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.io_clean_op 0.08 ± 7% +0.1 0.14 ± 5% perf-profile.children.cycles-pp.__entry_text_start 0.01 ±200% +0.1 0.07 ± 6% perf-profile.children.cycles-pp.perf_swevent_event 0.11 ± 29% +0.1 0.18 ± 7% perf-profile.children.cycles-pp.io_worker_release 0.00 +0.1 0.07 perf-profile.children.cycles-pp.__fdget_pos 0.01 ±200% +0.1 0.09 ± 14% perf-profile.children.cycles-pp.down_write 0.04 ± 50% +0.1 0.13 ± 6% perf-profile.children.cycles-pp.__calc_delta 0.37 ± 8% +0.1 0.47 ± 5% perf-profile.children.cycles-pp.__filemap_get_folio 0.38 ± 5% +0.1 0.48 ± 3% perf-profile.children.cycles-pp.tick_nohz_next_event 0.38 ± 9% +0.1 0.48 ± 4% perf-profile.children.cycles-pp.pagecache_get_page 0.05 +0.1 0.20 ± 15% perf-profile.children.cycles-pp.perf_trace_sched_migrate_task 0.45 ± 7% +0.2 0.61 ± 5% perf-profile.children.cycles-pp.shmem_write_end 0.05 ± 7% +0.2 0.22 ± 16% perf-profile.children.cycles-pp.set_task_cpu 0.07 ± 58% +0.2 0.24 ± 5% perf-profile.children.cycles-pp.timekeeping_max_deferment 0.56 ± 23% +0.2 0.79 ± 16% perf-profile.children.cycles-pp.clockevents_program_event 0.69 ± 10% +0.2 0.93 ± 7% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.71 ± 10% +0.2 0.96 ± 8% perf-profile.children.cycles-pp.copyin 0.76 ± 10% +0.3 1.04 ± 8% perf-profile.children.cycles-pp.copy_page_from_iter_atomic 0.33 ± 18% +0.4 0.69 ± 6% perf-profile.children.cycles-pp.__get_user_nocheck_1 0.34 ± 18% +0.4 0.71 ± 6% perf-profile.children.cycles-pp.fault_in_readable 0.35 ± 17% +0.4 0.73 ± 6% perf-profile.children.cycles-pp.fault_in_iov_iter_readable 0.36 ± 19% +0.7 1.02 ± 8% perf-profile.children.cycles-pp.perf_output_begin_forward 0.60 ± 7% +0.7 1.33 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.22 ± 5% +0.9 1.12 ± 9% perf-profile.children.cycles-pp.__queue_work 0.38 ± 7% +0.9 1.28 ± 8% perf-profile.children.cycles-pp.queue_work_on 2.29 ± 7% +1.0 3.25 ± 7% perf-profile.children.cycles-pp.generic_perform_write 2.34 ± 7% +1.0 3.34 ± 7% perf-profile.children.cycles-pp.__generic_file_write_iter 2.43 ± 7% +1.1 3.52 ± 7% perf-profile.children.cycles-pp.generic_file_write_iter 2.45 ± 7% +1.1 3.57 ± 7% perf-profile.children.cycles-pp.new_sync_write 3.18 ± 7% +1.2 4.34 ± 6% perf-profile.children.cycles-pp.__libc_start_main 1.75 ± 16% +1.2 2.92 ± 13% perf-profile.children.cycles-pp.io_wq_worker_sleeping 2.55 ± 7% +1.2 3.79 ± 7% perf-profile.children.cycles-pp.vfs_write 2.59 ± 7% +1.3 3.88 ± 7% perf-profile.children.cycles-pp.ksys_write 2.87 ± 7% +1.4 4.27 ± 6% perf-profile.children.cycles-pp.__libc_write 3.28 ± 2% +1.4 4.69 ± 4% perf-profile.children.cycles-pp.__iomap_dio_rw 3.28 ± 2% +1.4 4.70 ± 4% perf-profile.children.cycles-pp.iomap_dio_rw 3.54 ± 3% +1.4 4.98 ± 4% perf-profile.children.cycles-pp.ext4_file_read_iter 3.72 ± 3% +1.4 5.17 ± 4% perf-profile.children.cycles-pp.io_read 3.74 ± 3% +1.4 5.19 ± 4% perf-profile.children.cycles-pp.io_issue_sqe 3.78 ± 3% +1.5 5.25 ± 4% perf-profile.children.cycles-pp.io_wq_submit_work 0.81 ± 3% +1.5 2.31 ± 6% perf-profile.children.cycles-pp.loop_queue_rq 0.84 ± 2% +1.5 2.35 ± 5% perf-profile.children.cycles-pp.__blk_mq_try_issue_directly 0.89 ± 2% +1.5 2.40 ± 5% perf-profile.children.cycles-pp.blk_finish_plug 0.89 ± 2% +1.5 2.40 ± 5% perf-profile.children.cycles-pp.blk_flush_plug 0.88 ± 2% +1.5 2.39 ± 6% perf-profile.children.cycles-pp.blk_mq_flush_plug_list 0.84 ± 2% +1.5 2.36 ± 6% perf-profile.children.cycles-pp.blk_mq_request_issue_directly 7.49 ± 4% +2.1 9.55 ± 7% perf-profile.children.cycles-pp.io_worker_handle_work 11.19 ± 10% +3.6 14.83 ± 11% perf-profile.children.cycles-pp._raw_spin_lock 10.16 ± 11% +4.1 14.28 ± 10% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 3.96 ± 16% -2.3 1.64 ± 4% perf-profile.self.cycles-pp.__get_user_nocheck_8 2.52 -0.6 1.92 ± 2% perf-profile.self.cycles-pp.__orc_find 2.90 ± 3% -0.6 2.33 ± 3% perf-profile.self.cycles-pp.unwind_next_frame 0.64 ± 24% -0.5 0.13 ± 6% perf-profile.self.cycles-pp.perf_callchain_user 1.35 ± 2% -0.3 1.05 ± 2% perf-profile.self.cycles-pp.memcpy_erms 0.57 ± 15% -0.2 0.32 ± 18% perf-profile.self.cycles-pp.menu_select 1.04 ± 5% -0.2 0.82 ± 6% perf-profile.self.cycles-pp.orc_find 0.50 ± 9% -0.2 0.34 ± 8% perf-profile.self.cycles-pp.cpuidle_enter_state 0.67 ± 6% -0.2 0.52 ± 4% perf-profile.self.cycles-pp.stack_access_ok 0.49 ± 11% -0.1 0.37 ± 2% perf-profile.self.cycles-pp.finish_task_switch 0.47 ± 4% -0.1 0.37 ± 2% perf-profile.self.cycles-pp.native_sched_clock 0.81 ± 5% -0.1 0.71 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.57 ± 4% -0.1 0.48 perf-profile.self.cycles-pp.perf_callchain_kernel 0.29 ± 10% -0.1 0.20 ± 8% perf-profile.self.cycles-pp._find_next_bit 0.36 ± 9% -0.1 0.27 ± 6% perf-profile.self.cycles-pp.__unwind_start 0.27 ± 8% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.__switch_to 0.36 ± 4% -0.1 0.29 ± 4% perf-profile.self.cycles-pp.perf_output_sample 0.10 ± 31% -0.1 0.03 ± 70% perf-profile.self.cycles-pp.__hrtimer_next_event_base 0.20 ± 5% -0.1 0.14 ± 15% perf-profile.self.cycles-pp.do_idle 0.24 ± 7% -0.1 0.18 perf-profile.self.cycles-pp.get_perf_callchain 0.18 ± 2% -0.1 0.13 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.14 ± 13% -0.0 0.09 ± 14% perf-profile.self.cycles-pp.load_balance 0.13 ± 10% -0.0 0.08 ± 17% perf-profile.self.cycles-pp.__module_address 0.27 ± 6% -0.0 0.23 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 0.23 ± 5% -0.0 0.19 ± 6% perf-profile.self.cycles-pp.read_tsc 0.19 ± 7% -0.0 0.14 ± 8% perf-profile.self.cycles-pp.ftrace_graph_ret_addr 0.29 ± 7% -0.0 0.25 ± 3% perf-profile.self.cycles-pp.try_grab_compound_head 0.19 ± 7% -0.0 0.15 ± 9% perf-profile.self.cycles-pp.kernel_text_address 0.07 ± 10% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.update_blocked_averages 0.10 ± 14% -0.0 0.06 perf-profile.self.cycles-pp.perf_trace_sched_switch 0.17 ± 8% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.perf_prepare_sample 0.10 ± 13% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.09 ± 14% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.reweight_entity 0.09 ± 12% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.update_irq_load_avg 0.10 ± 11% -0.0 0.08 ± 12% perf-profile.self.cycles-pp.task_work_add 0.13 ± 5% -0.0 0.10 ± 12% perf-profile.self.cycles-pp.core_kernel_text 0.09 ± 4% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.__wrgsbase_inactive 0.07 ± 10% -0.0 0.05 perf-profile.self.cycles-pp.arch_scale_freq_tick 0.08 ± 14% -0.0 0.06 ± 13% perf-profile.self.cycles-pp.tracing_gen_ctx_irq_test 0.11 ± 6% +0.0 0.13 ± 7% perf-profile.self.cycles-pp.__list_del_entry_valid 0.08 ± 26% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.__fget_light 0.03 ± 81% +0.0 0.06 ± 7% perf-profile.self.cycles-pp.__fsnotify_parent 0.03 ± 82% +0.0 0.07 ± 7% perf-profile.self.cycles-pp.__entry_text_start 0.14 ± 8% +0.0 0.17 ± 7% perf-profile.self.cycles-pp.io_wqe_dec_running 0.16 ± 10% +0.0 0.20 ± 9% perf-profile.self.cycles-pp._raw_read_lock 0.02 ±123% +0.0 0.07 ± 7% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.11 ± 10% +0.0 0.16 ± 5% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.01 ±200% +0.0 0.06 perf-profile.self.cycles-pp.generic_perform_write 0.07 ± 26% +0.1 0.13 ± 16% perf-profile.self.cycles-pp.calc_global_load_tick 0.18 ± 4% +0.1 0.23 ± 5% perf-profile.self.cycles-pp.__filemap_get_folio 0.20 ± 6% +0.1 0.25 ± 10% perf-profile.self.cycles-pp.loop_process_work 0.11 ± 27% +0.1 0.17 ± 7% perf-profile.self.cycles-pp.io_worker_release 0.04 ± 50% +0.1 0.11 ± 4% perf-profile.self.cycles-pp.__libc_write 0.01 ±200% +0.1 0.08 ± 12% perf-profile.self.cycles-pp.copy_page_from_iter_atomic 0.01 ±200% +0.1 0.08 ± 6% perf-profile.self.cycles-pp.vfs_write 0.04 ± 50% +0.1 0.12 ± 7% perf-profile.self.cycles-pp.__calc_delta 0.08 ± 18% +0.1 0.16 ± 18% perf-profile.self.cycles-pp.update_process_times 0.39 ± 4% +0.1 0.47 ± 9% perf-profile.self.cycles-pp.io_worker_handle_work 0.37 ± 6% +0.1 0.50 ± 8% perf-profile.self.cycles-pp.shmem_write_end 0.07 ± 61% +0.2 0.24 ± 5% perf-profile.self.cycles-pp.timekeeping_max_deferment 0.33 ± 17% +0.2 0.51 ± 6% perf-profile.self.cycles-pp.try_to_wake_up 0.69 ± 10% +0.2 0.93 ± 7% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.57 ± 10% +0.3 0.89 ± 7% perf-profile.self.cycles-pp.perf_output_copy 0.28 ± 19% +0.4 0.64 ± 7% perf-profile.self.cycles-pp.__get_user_nocheck_1 0.36 ± 19% +0.7 1.02 ± 8% perf-profile.self.cycles-pp.perf_output_begin_forward 10.04 ± 11% +4.1 14.17 ± 10% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath *************************************************************************************************** lkp-skl-fpga01: 104 threads 2 sockets Skylake with 192G memory ========================================================================================= compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-9/performance/pipe/4/x86_64-rhel-8.3/threads/100%/debian-10.4-x86_64-20200603.cgz/lkp-skl-fpga01/hackbench/0x2006a0a commit: v5.16-rc6 b08e888d97 ("sched/fair: Force progress on min_vruntime") v5.16-rc6 b08e888d9715f01546da7a40ca3 ---------------- --------------------------- %stddev %change %stddev \ | \ 243924 +55.5% 379330 hackbench.throughput 15540713 ± 8% -60.4% 6160950 ± 3% cpuidle..usage 261.57 -29.5% 184.44 uptime.boot 1.18 -0.1 1.05 mpstat.cpu.all.irq% 0.04 ± 5% -0.0 0.04 ± 2% mpstat.cpu.all.soft% 73.85 -11.8 62.02 mpstat.cpu.all.sys% 22.89 +11.7 34.62 mpstat.cpu.all.usr% 20206732 +171.9% 54945573 numa-numastat.node0.local_node 20253752 +171.5% 54982565 numa-numastat.node0.numa_hit 21045850 ± 2% +183.3% 59621383 numa-numastat.node1.local_node 21058134 +183.2% 59642825 numa-numastat.node1.numa_hit 7043843 ± 12% -37.3% 4419834 ± 4% turbostat.C1 0.24 ± 12% +0.1 0.30 ± 4% turbostat.C1% 2.04e+08 ± 2% -84.8% 30989257 turbostat.IRQ 115.28 +14.7% 132.18 turbostat.RAMWatt 22.17 +53.4% 34.00 vmstat.cpu.us 5220877 ± 14% -41.6% 3050905 ± 2% vmstat.memory.cache 1204 +77.7% 2139 vmstat.procs.r 14664745 -95.2% 704900 ± 2% vmstat.system.cs 911202 -74.5% 232488 vmstat.system.in 2250 ± 15% -27.9% 1621 ± 10% numa-meminfo.node0.Active 2095 ± 16% -31.0% 1445 ± 11% numa-meminfo.node0.Active(anon) 188542 ± 50% -58.5% 78333 ± 13% numa-meminfo.node0.Mapped 67992 ± 7% -72.8% 18495 ± 50% numa-meminfo.node1.Active 67992 ± 7% -72.8% 18495 ± 50% numa-meminfo.node1.Active(anon) 72982 ± 46% -58.8% 30071 ± 88% numa-meminfo.node1.AnonHugePages 130528 ± 30% -47.3% 68777 ± 54% numa-meminfo.node1.AnonPages 2539845 ± 29% -81.1% 479406 ± 14% numa-meminfo.node1.Inactive 2539845 ± 29% -81.1% 479406 ± 14% numa-meminfo.node1.Inactive(anon) 502608 ± 21% -79.8% 101561 ± 18% numa-meminfo.node1.Mapped 2492 ± 10% -42.0% 1446 ± 12% numa-meminfo.node1.PageTables 2477433 ± 30% -82.7% 429019 ± 23% numa-meminfo.node1.Shmem 70018 ± 6% -72.8% 19048 ± 42% meminfo.Active 69863 ± 6% -73.0% 18872 ± 42% meminfo.Active(anon) 192446 -26.1% 142292 ± 2% meminfo.AnonHugePages 341625 -10.9% 304256 meminfo.AnonPages 5121644 ± 15% -42.7% 2935312 ± 3% meminfo.Cached 3116939 ± 25% -71.1% 899660 ± 10% meminfo.Committed_AS 2901201 ± 27% -74.9% 728894 ± 12% meminfo.Inactive 2901053 ± 27% -74.9% 728746 ± 12% meminfo.Inactive(anon) 687779 ± 14% -73.8% 180190 ± 11% meminfo.Mapped 7146592 ± 11% -30.2% 4990358 meminfo.Memused 5611 ± 3% -16.2% 4703 meminfo.PageTables 2629586 ± 29% -83.2% 442853 ± 20% meminfo.Shmem 7270248 ± 10% -30.2% 5074455 meminfo.max_used_kB 522.33 ± 16% -30.7% 362.17 ± 11% numa-vmstat.node0.nr_active_anon 46914 ± 50% -56.3% 20487 ± 15% numa-vmstat.node0.nr_mapped 522.33 ± 16% -30.7% 362.17 ± 11% numa-vmstat.node0.nr_zone_active_anon 12264878 +142.5% 29744174 ± 2% numa-vmstat.node0.numa_hit 12214256 +143.1% 29692277 ± 2% numa-vmstat.node0.numa_local 16894 ± 6% -72.1% 4714 ± 49% numa-vmstat.node1.nr_active_anon 32556 ± 30% -47.0% 17259 ± 55% numa-vmstat.node1.nr_anon_pages 631923 ± 30% -80.1% 125630 ± 15% numa-vmstat.node1.nr_inactive_anon 124390 ± 22% -78.5% 26774 ± 17% numa-vmstat.node1.nr_mapped 620.00 ± 10% -41.0% 366.00 ± 11% numa-vmstat.node1.nr_page_table_pages 616293 ± 31% -81.7% 113060 ± 21% numa-vmstat.node1.nr_shmem 16894 ± 6% -72.1% 4714 ± 49% numa-vmstat.node1.nr_zone_active_anon 631922 ± 30% -80.1% 125629 ± 15% numa-vmstat.node1.nr_zone_inactive_anon 11597593 ± 2% +170.4% 31363549 ± 2% numa-vmstat.node1.numa_hit 11533651 ± 2% +171.4% 31297047 ± 2% numa-vmstat.node1.numa_local 17516 ± 6% -73.4% 4662 ± 40% proc-vmstat.nr_active_anon 85408 -11.0% 76023 proc-vmstat.nr_anon_pages 4710721 +1.1% 4764703 proc-vmstat.nr_dirty_background_threshold 9432961 +1.1% 9541057 proc-vmstat.nr_dirty_threshold 1280589 ± 15% -42.8% 732711 ± 2% proc-vmstat.nr_file_pages 47394242 +1.1% 47934851 proc-vmstat.nr_free_pages 725395 ± 26% -75.0% 181084 ± 11% proc-vmstat.nr_inactive_anon 80621 -4.0% 77417 proc-vmstat.nr_kernel_stack 171686 ± 13% -73.8% 44975 ± 11% proc-vmstat.nr_mapped 1402 ± 3% -16.3% 1173 proc-vmstat.nr_page_table_pages 657573 ± 29% -83.3% 109596 ± 19% proc-vmstat.nr_shmem 28926 -5.7% 27281 proc-vmstat.nr_slab_reclaimable 67226 -1.6% 66140 proc-vmstat.nr_slab_unreclaimable 17516 ± 6% -73.4% 4662 ± 40% proc-vmstat.nr_zone_active_anon 725395 ± 26% -75.0% 181084 ± 11% proc-vmstat.nr_zone_inactive_anon 390080 ± 4% -62.6% 145788 ± 20% proc-vmstat.numa_hint_faults 284764 ± 6% -71.6% 80809 ± 21% proc-vmstat.numa_hint_faults_local 41317354 +177.4% 1.146e+08 proc-vmstat.numa_hit 41258052 +177.7% 1.146e+08 proc-vmstat.numa_local 36260 ± 42% -80.8% 6978 ± 47% proc-vmstat.numa_pages_migrated 627669 -50.8% 308691 ± 10% proc-vmstat.numa_pte_updates 41321465 +177.4% 1.146e+08 proc-vmstat.pgalloc_normal 1221322 ± 2% -38.8% 747607 ± 4% proc-vmstat.pgfault 39928898 +185.4% 1.14e+08 proc-vmstat.pgfree 36260 ± 42% -80.8% 6978 ± 47% proc-vmstat.pgmigrate_success 37167 ± 2% -36.7% 23533 ± 2% proc-vmstat.pgreuse 10.86 ± 2% -60.4% 4.30 perf-stat.i.MPKI 2.773e+10 -24.3% 2.098e+10 perf-stat.i.branch-instructions 1.46 -0.2 1.25 perf-stat.i.branch-miss-rate% 3.945e+08 -33.9% 2.609e+08 perf-stat.i.branch-misses 4.19 ± 5% +28.0 32.20 perf-stat.i.cache-miss-rate% 53454898 ± 5% +150.0% 1.336e+08 perf-stat.i.cache-misses 1.412e+09 -68.8% 4.407e+08 perf-stat.i.cache-references 14863174 -95.3% 698605 ± 2% perf-stat.i.context-switches 2.08 +26.7% 2.64 perf-stat.i.cpi 918790 -93.6% 58630 ± 3% perf-stat.i.cpu-migrations 5445 ± 5% -61.5% 2098 perf-stat.i.cycles-between-cache-misses 0.12 +0.1 0.24 perf-stat.i.dTLB-load-miss-rate% 48798801 +56.3% 76261572 perf-stat.i.dTLB-load-misses 3.971e+10 -19.4% 3.201e+10 perf-stat.i.dTLB-loads 0.00 ± 10% -0.0 0.00 ± 3% perf-stat.i.dTLB-store-miss-rate% 303926 ± 3% -46.0% 164007 ± 2% perf-stat.i.dTLB-store-misses 2.368e+10 -21.7% 1.855e+10 perf-stat.i.dTLB-stores 64515637 ± 5% +35.0% 87084500 ± 7% perf-stat.i.iTLB-load-misses 1.352e+11 -22.9% 1.043e+11 perf-stat.i.instructions 2402 ± 12% -48.6% 1235 ± 5% perf-stat.i.instructions-per-iTLB-miss 0.49 -22.8% 0.38 perf-stat.i.ipc 3.42 ± 32% -53.5% 1.59 perf-stat.i.major-faults 361.63 ± 3% +14.5% 413.95 ± 4% perf-stat.i.metric.K/sec 889.15 -22.1% 692.26 perf-stat.i.metric.M/sec 5580 ± 2% -9.5% 5049 ± 5% perf-stat.i.minor-faults 80.85 +2.0 82.88 perf-stat.i.node-load-miss-rate% 9339361 ± 4% +92.4% 17965649 ± 4% perf-stat.i.node-load-misses 2176060 ± 3% +71.5% 3731319 ± 2% perf-stat.i.node-loads 57.54 ± 2% +17.6 75.11 perf-stat.i.node-store-miss-rate% 3921170 ± 4% +231.4% 12995856 ± 3% perf-stat.i.node-store-misses 2780324 +55.0% 4310621 ± 2% perf-stat.i.node-stores 5583 ± 2% -9.5% 5051 ± 5% perf-stat.i.page-faults 10.40 -59.6% 4.20 perf-stat.overall.MPKI 1.42 -0.2 1.24 perf-stat.overall.branch-miss-rate% 3.80 ± 5% +26.8 30.57 perf-stat.overall.cache-miss-rate% 2.03 +30.1% 2.64 perf-stat.overall.cpi 5143 ± 5% -60.0% 2055 perf-stat.overall.cycles-between-cache-misses 0.12 +0.1 0.24 perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 3% -0.0 0.00 ± 2% perf-stat.overall.dTLB-store-miss-rate% 95.23 +1.6 96.82 perf-stat.overall.iTLB-load-miss-rate% 2096 ± 5% -42.6% 1202 ± 7% perf-stat.overall.instructions-per-iTLB-miss 0.49 -23.2% 0.38 perf-stat.overall.ipc 81.04 +1.7 82.71 perf-stat.overall.node-load-miss-rate% 58.43 ± 2% +16.7 75.14 perf-stat.overall.node-store-miss-rate% 2.757e+10 -24.4% 2.085e+10 perf-stat.ps.branch-instructions 3.923e+08 -34.0% 2.59e+08 perf-stat.ps.branch-misses 53174568 ± 5% +150.3% 1.331e+08 perf-stat.ps.cache-misses 1.399e+09 -68.9% 4.354e+08 perf-stat.ps.cache-references 14787641 -95.4% 683155 ± 2% perf-stat.ps.context-switches 901548 -93.8% 55778 ± 3% perf-stat.ps.cpu-migrations 48660610 +55.8% 75815767 perf-stat.ps.dTLB-load-misses 3.951e+10 -19.5% 3.181e+10 perf-stat.ps.dTLB-loads 299532 ± 3% -46.0% 161819 ± 2% perf-stat.ps.dTLB-store-misses 2.356e+10 -21.8% 1.843e+10 perf-stat.ps.dTLB-stores 64330425 ± 5% +34.7% 86669158 ± 7% perf-stat.ps.iTLB-load-misses 1.345e+11 -22.9% 1.036e+11 perf-stat.ps.instructions 3.34 ± 32% -52.4% 1.59 perf-stat.ps.major-faults 5385 -8.5% 4929 ± 5% perf-stat.ps.minor-faults 9247666 ± 4% +93.1% 17857896 ± 4% perf-stat.ps.node-load-misses 2160605 ± 3% +72.6% 3728183 ± 2% perf-stat.ps.node-loads 3886969 ± 4% +232.3% 12914952 ± 3% perf-stat.ps.node-store-misses 2761577 +54.7% 4270821 ± 2% perf-stat.ps.node-stores 5388 -8.5% 4931 ± 5% perf-stat.ps.page-faults 2.83e+13 -51.6% 1.371e+13 perf-stat.total.instructions 35304 ± 9% -35.1% 22902 ± 4% softirqs.CPU0.RCU 34522 ± 10% -34.0% 22784 ± 9% softirqs.CPU1.RCU 7675 ± 19% -34.3% 5038 ± 19% softirqs.CPU1.SCHED 34716 ± 9% -36.7% 21979 ± 6% softirqs.CPU10.RCU 33713 ± 9% -37.0% 21228 ± 4% softirqs.CPU100.RCU 33959 ± 9% -37.6% 21176 ± 5% softirqs.CPU101.RCU 34338 ± 9% -38.6% 21077 ± 4% softirqs.CPU102.RCU 34548 ± 9% -38.1% 21387 ± 5% softirqs.CPU103.RCU 8452 ± 17% -44.1% 4728 ± 18% softirqs.CPU103.SCHED 34720 ± 8% -37.5% 21684 ± 4% softirqs.CPU11.RCU 35190 ± 9% -38.2% 21733 ± 3% softirqs.CPU12.RCU 35034 ± 7% -36.4% 22294 ± 4% softirqs.CPU13.RCU 34598 ± 9% -36.6% 21921 ± 3% softirqs.CPU14.RCU 37841 ± 8% -38.5% 23284 ± 5% softirqs.CPU15.RCU 37752 ± 8% -38.4% 23246 ± 4% softirqs.CPU16.RCU 37597 ± 8% -37.7% 23428 ± 2% softirqs.CPU17.RCU 37341 ± 8% -36.5% 23696 ± 4% softirqs.CPU18.RCU 36883 ± 8% -37.1% 23210 ± 4% softirqs.CPU19.RCU 34873 ± 8% -38.8% 21355 ± 6% softirqs.CPU2.RCU 37958 ± 9% -38.7% 23259 ± 4% softirqs.CPU20.RCU 38116 ± 8% -39.2% 23170 ± 4% softirqs.CPU21.RCU 37496 ± 8% -38.2% 23187 ± 4% softirqs.CPU22.RCU 37683 ± 7% -38.0% 23378 ± 4% softirqs.CPU23.RCU 37514 ± 7% -37.1% 23592 ± 4% softirqs.CPU24.RCU 37848 ± 8% -37.3% 23748 ± 6% softirqs.CPU25.RCU 33410 ± 9% -40.1% 20017 ± 3% softirqs.CPU26.RCU 33936 ± 9% -39.0% 20699 ± 6% softirqs.CPU27.RCU 10205 ± 11% -36.7% 6462 ± 28% softirqs.CPU27.SCHED 33054 ± 9% -40.4% 19714 ± 4% softirqs.CPU28.RCU 33007 ± 10% -39.9% 19836 ± 4% softirqs.CPU29.RCU 35600 ± 11% -40.0% 21365 ± 4% softirqs.CPU3.RCU 35544 ± 10% -36.8% 22467 ± 5% softirqs.CPU30.RCU 34897 ± 8% -36.9% 22011 ± 6% softirqs.CPU31.RCU 34213 ± 6% -36.5% 21719 ± 4% softirqs.CPU32.RCU 33298 ± 5% -36.0% 21323 ± 5% softirqs.CPU33.RCU 7987 ± 17% -42.5% 4595 ± 8% softirqs.CPU33.SCHED 34005 ± 9% -37.8% 21150 ± 4% softirqs.CPU34.RCU 34250 ± 9% -38.2% 21163 ± 4% softirqs.CPU35.RCU 34218 ± 9% -37.9% 21256 ± 4% softirqs.CPU36.RCU 34474 ± 9% -36.2% 22010 ± 5% softirqs.CPU37.RCU 34626 ± 9% -38.0% 21485 ± 5% softirqs.CPU38.RCU 34276 ± 9% -38.1% 21203 ± 5% softirqs.CPU39.RCU 34472 ± 9% -36.9% 21742 ± 4% softirqs.CPU4.RCU 33896 ± 9% -38.1% 20986 ± 5% softirqs.CPU40.RCU 34236 ± 9% -38.2% 21159 ± 4% softirqs.CPU41.RCU 34549 ± 9% -37.9% 21445 ± 5% softirqs.CPU42.RCU 34676 ± 9% -37.7% 21617 ± 6% softirqs.CPU43.RCU 34667 ± 9% -38.1% 21469 ± 4% softirqs.CPU44.RCU 34607 ± 9% -40.1% 20718 ± 4% softirqs.CPU45.RCU 34814 ± 9% -38.9% 21266 ± 6% softirqs.CPU46.RCU 34743 ± 10% -39.6% 20991 ± 4% softirqs.CPU47.RCU 34411 ± 10% -39.1% 20945 ± 5% softirqs.CPU48.RCU 34696 ± 10% -40.0% 20819 ± 4% softirqs.CPU49.RCU 35847 ± 8% -38.3% 22125 ± 4% softirqs.CPU5.RCU 34536 ± 9% -38.7% 21160 ± 6% softirqs.CPU50.RCU 34457 ± 9% -38.6% 21150 ± 4% softirqs.CPU51.RCU 36863 ± 8% -40.1% 22070 ± 3% softirqs.CPU52.RCU 35183 ± 14% -34.2% 23157 ± 5% softirqs.CPU53.RCU 37456 ± 8% -35.6% 24126 ± 8% softirqs.CPU54.RCU 36481 ± 12% -37.5% 22814 ± 5% softirqs.CPU55.RCU 37552 ± 8% -39.1% 22880 ± 5% softirqs.CPU56.RCU 38204 ± 12% -39.5% 23108 ± 4% softirqs.CPU57.RCU 37570 ± 8% -40.6% 22333 ± 5% softirqs.CPU58.RCU 37972 ± 7% -39.2% 23087 ± 5% softirqs.CPU59.RCU 35010 ± 9% -38.0% 21717 ± 5% softirqs.CPU6.RCU 34777 ± 9% -37.3% 21821 ± 4% softirqs.CPU60.RCU 34756 ± 8% -35.2% 22536 ± 6% softirqs.CPU61.RCU 35005 ± 7% -37.6% 21829 ± 5% softirqs.CPU62.RCU 35414 ± 9% -37.9% 21979 ± 4% softirqs.CPU63.RCU 35050 ± 8% -37.2% 22015 ± 4% softirqs.CPU64.RCU 35234 ± 9% -37.6% 21997 ± 4% softirqs.CPU65.RCU 34721 ± 9% -37.5% 21699 ± 3% softirqs.CPU66.RCU 35608 ± 7% -38.9% 21772 ± 5% softirqs.CPU67.RCU 34949 ± 8% -39.0% 21317 ± 4% softirqs.CPU68.RCU 35032 ± 9% -37.8% 21778 ± 4% softirqs.CPU69.RCU 34514 ± 9% -36.3% 21983 ± 9% softirqs.CPU7.RCU 35351 ± 9% -38.1% 21883 ± 2% softirqs.CPU70.RCU 33661 ± 9% -35.2% 21816 ± 4% softirqs.CPU71.RCU 34803 ± 8% -39.5% 21054 ± 7% softirqs.CPU72.RCU 34299 ± 8% -37.4% 21463 ± 2% softirqs.CPU73.RCU 34667 ± 9% -38.3% 21385 ± 3% softirqs.CPU74.RCU 32951 ± 8% -38.2% 20359 ± 7% softirqs.CPU75.RCU 33407 ± 8% -39.0% 20370 ± 4% softirqs.CPU76.RCU 33587 ± 7% -40.7% 19914 ± 4% softirqs.CPU77.RCU 34722 ± 8% -39.0% 21194 ± 7% softirqs.CPU78.RCU 33933 ± 10% -37.8% 21112 ± 4% softirqs.CPU79.RCU 34812 ± 7% -38.0% 21592 ± 5% softirqs.CPU8.RCU 35145 ± 9% -39.1% 21396 ± 4% softirqs.CPU80.RCU 34879 ± 8% -38.8% 21332 ± 5% softirqs.CPU81.RCU 34627 ± 8% -39.3% 21007 ± 4% softirqs.CPU82.RCU 34760 ± 8% -39.0% 21215 ± 4% softirqs.CPU83.RCU 34952 ± 8% -39.0% 21312 ± 4% softirqs.CPU84.RCU 34522 ± 8% -39.1% 21024 ± 5% softirqs.CPU85.RCU 34406 ± 9% -38.6% 21137 ± 5% softirqs.CPU86.RCU 34419 ± 8% -39.0% 20987 ± 4% softirqs.CPU87.RCU 34403 ± 8% -37.9% 21358 ± 3% softirqs.CPU88.RCU 35015 ± 9% -38.0% 21701 ± 4% softirqs.CPU89.RCU 35199 ± 9% -38.3% 21704 ± 6% softirqs.CPU9.RCU 34512 ± 9% -37.9% 21441 ± 4% softirqs.CPU90.RCU 34052 ± 9% -38.0% 21098 ± 4% softirqs.CPU91.RCU 34048 ± 9% -37.6% 21252 ± 4% softirqs.CPU92.RCU 34155 ± 8% -37.3% 21398 ± 5% softirqs.CPU93.RCU 34243 ± 8% -37.7% 21347 ± 4% softirqs.CPU94.RCU 34387 ± 9% -37.4% 21536 ± 5% softirqs.CPU95.RCU 34242 ± 8% -37.3% 21472 ± 4% softirqs.CPU96.RCU 34569 ± 8% -38.6% 21224 ± 4% softirqs.CPU97.RCU 34086 ± 9% -37.5% 21317 ± 4% softirqs.CPU98.RCU 34399 ± 9% -37.8% 21380 ± 5% softirqs.CPU99.RCU 3645591 ± 8% -38.0% 2259598 ± 4% softirqs.RCU 713024 ± 7% -37.9% 442950 ± 11% softirqs.SCHED 45382 ± 4% -22.4% 35207 ± 3% softirqs.TIMER 21.02 ± 10% -20.0 1.07 ± 50% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 20.54 ± 10% -19.5 1.05 ± 50% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 20.34 ± 10% -19.3 1.04 ± 51% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 20.02 ± 11% -19.0 1.02 ± 52% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 29.44 ± 5% -14.3 15.18 ± 3% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 14.88 ± 15% -14.1 0.78 ± 88% perf-profile.calltrace.cycles-pp.schedule.pipe_read.new_sync_read.vfs_read.ksys_read 14.70 ± 15% -14.0 0.70 ±107% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.new_sync_read.vfs_read 30.12 ± 5% -13.9 16.20 ± 3% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 31.81 ± 4% -12.3 19.49 ± 2% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 11.62 ± 55% -11.2 0.40 ±223% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event 33.67 ± 4% -10.9 22.79 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 10.54 ± 54% -10.3 0.29 ±223% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow 44.47 ± 2% -10.1 34.37 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 10.32 ± 54% -10.0 0.29 ±223% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow 44.84 ± 2% -10.0 34.89 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write 9.40 ± 53% -9.2 0.19 ±223% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward 8.80 ± 14% -8.5 0.35 ±107% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 8.71 ± 15% -8.4 0.27 ±146% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 24.75 ± 6% -7.1 17.70 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 7.28 ± 14% -7.0 0.26 ±146% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.new_sync_read 25.37 ± 6% -6.9 18.47 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.92 ± 54% -5.7 0.24 ±223% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr 5.88 ± 54% -5.7 0.23 ±223% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime 5.85 ± 28% -5.6 0.23 ±148% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 27.30 ± 5% -5.3 22.01 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 52.39 -4.7 47.68 perf-profile.calltrace.cycles-pp.__libc_write 28.82 ± 4% -4.2 24.66 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 0.71 ± 9% +0.3 0.96 perf-profile.calltrace.cycles-pp.__pthread_disable_asynccancel 0.68 ± 9% +0.3 0.96 ± 2% perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.new_sync_write.vfs_write.ksys_write 10.15 ± 2% +0.4 10.53 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 0.66 ± 11% +0.4 1.10 ± 2% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write 0.96 ± 9% +0.4 1.41 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +0.5 0.52 ± 2% perf-profile.calltrace.cycles-pp.current_time.atime_needs_update.touch_atime.pipe_read.new_sync_read 0.00 +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.__x64_sys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 0.00 +0.6 0.55 ± 2% perf-profile.calltrace.cycles-pp.fput_many.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 0.00 +0.6 0.59 ± 2% perf-profile.calltrace.cycles-pp.fput_many.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 0.82 ± 8% +0.6 1.42 ± 11% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.03 ± 9% +0.6 1.65 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read 0.82 ± 7% +0.6 1.47 ± 11% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 0.39 ± 70% +0.7 1.05 ± 2% perf-profile.calltrace.cycles-pp.__pthread_enable_asynccancel 0.86 ± 9% +0.7 1.56 perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write 1.10 ± 7% +0.7 1.80 perf-profile.calltrace.cycles-pp.atime_needs_update.touch_atime.pipe_read.new_sync_read.vfs_read 1.39 ± 6% +0.7 2.10 ± 2% perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +0.7 0.71 ± 4% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read 1.29 ± 8% +0.8 2.04 ± 7% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.8 0.76 ± 6% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_charge.obj_cgroup_charge_pages.__memcg_kmem_charge_page.__alloc_pages 1.16 ± 9% +0.8 1.93 perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 0.83 ± 6% +0.8 1.61 ± 7% perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64 1.30 ± 8% +0.8 2.09 ± 7% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 0.86 ± 7% +0.8 1.70 ± 7% perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64 1.02 ± 9% +0.9 1.90 ± 2% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 1.66 ± 9% +1.0 2.67 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 1.11 ± 8% +1.2 2.36 ± 6% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.35 ± 8% +1.5 2.82 ± 5% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.22 ± 9% +1.5 3.72 ± 2% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 2.26 ± 9% +1.7 3.97 perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +1.9 1.94 ± 4% perf-profile.calltrace.cycles-pp.page_counter_charge.obj_cgroup_charge_pages.__memcg_kmem_charge_page.__alloc_pages.pipe_write 2.78 ± 9% +2.1 4.86 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_write 2.86 ± 9% +2.1 4.95 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_read 0.00 +2.1 2.12 ± 4% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.__memcg_kmem_charge_page.__alloc_pages.pipe_write.new_sync_write 0.00 +2.2 2.16 ± 4% perf-profile.calltrace.cycles-pp.__memcg_kmem_charge_page.__alloc_pages.pipe_write.new_sync_write.vfs_write 1.26 ± 31% +2.3 3.54 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_safe_stack.__libc_write 0.00 +2.3 2.30 ± 4% perf-profile.calltrace.cycles-pp.__alloc_pages.pipe_write.new_sync_write.vfs_write.ksys_write 2.50 ± 9% +2.4 4.91 perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_read 2.51 ± 8% +2.5 5.03 perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_write 0.00 +2.7 2.70 ± 2% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.uncharge_batch.__mem_cgroup_uncharge.__put_page 1.61 ± 11% +2.8 4.41 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_safe_stack.__libc_read 0.00 +3.0 3.04 ± 4% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_uncharge.uncharge_batch.__mem_cgroup_uncharge.__put_page 7.16 ± 7% +3.4 10.51 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 45.21 +4.7 49.92 perf-profile.calltrace.cycles-pp.__libc_read 0.00 +6.0 6.05 ± 3% perf-profile.calltrace.cycles-pp.page_counter_uncharge.uncharge_batch.__mem_cgroup_uncharge.__put_page.pipe_read 0.00 +6.2 6.20 ± 3% perf-profile.calltrace.cycles-pp.uncharge_batch.__mem_cgroup_uncharge.__put_page.pipe_read.new_sync_read 0.00 +6.3 6.25 ± 3% perf-profile.calltrace.cycles-pp.__mem_cgroup_uncharge.__put_page.pipe_read.new_sync_read.vfs_read 0.00 +6.3 6.26 ± 3% perf-profile.calltrace.cycles-pp.__put_page.pipe_read.new_sync_read.vfs_read.ksys_read 22.50 ± 10% -21.3 1.16 ± 49% perf-profile.children.cycles-pp.__wake_up_common_lock 21.96 ± 11% -20.8 1.11 ± 50% perf-profile.children.cycles-pp.__wake_up_common 21.76 ± 11% -20.7 1.10 ± 51% perf-profile.children.cycles-pp.autoremove_wake_function 21.51 ± 11% -20.2 1.28 ± 47% perf-profile.children.cycles-pp.try_to_wake_up 20.25 ± 14% -19.1 1.12 ± 54% perf-profile.children.cycles-pp.schedule 20.08 ± 15% -19.0 1.13 ± 55% perf-profile.children.cycles-pp.__schedule 29.50 ± 5% -14.2 15.25 ± 3% perf-profile.children.cycles-pp.pipe_write 30.15 ± 5% -13.9 16.24 ± 3% perf-profile.children.cycles-pp.new_sync_write 14.55 ± 49% -13.5 1.08 ±117% perf-profile.children.cycles-pp.perf_tp_event 13.74 ± 51% -12.7 1.03 ±117% perf-profile.children.cycles-pp.perf_swevent_overflow 13.71 ± 51% -12.7 1.03 ±117% perf-profile.children.cycles-pp.__perf_event_overflow 13.62 ± 51% -12.6 1.02 ±117% perf-profile.children.cycles-pp.perf_event_output_forward 31.84 ± 4% -12.3 19.52 ± 2% perf-profile.children.cycles-pp.vfs_write 13.19 ± 51% -12.3 0.94 ±118% perf-profile.children.cycles-pp.perf_prepare_sample 12.04 ± 51% -11.2 0.86 ±118% perf-profile.children.cycles-pp.perf_callchain 11.96 ± 51% -11.1 0.86 ±118% perf-profile.children.cycles-pp.get_perf_callchain 33.70 ± 4% -10.9 22.84 perf-profile.children.cycles-pp.ksys_write 81.19 ± 2% -10.5 70.66 perf-profile.children.cycles-pp.do_syscall_64 11.08 ± 51% -10.3 0.80 ±118% perf-profile.children.cycles-pp.perf_callchain_kernel 81.93 ± 2% -10.3 71.67 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 9.70 ± 33% -8.9 0.81 ± 86% perf-profile.children.cycles-pp.update_curr 8.94 ± 13% -8.4 0.56 ± 53% perf-profile.children.cycles-pp.ttwu_do_activate 8.85 ± 14% -8.3 0.55 ± 54% perf-profile.children.cycles-pp.enqueue_task_fair 8.32 ± 51% -7.7 0.59 ±118% perf-profile.children.cycles-pp.unwind_next_frame 7.88 ± 14% -7.4 0.52 ± 60% perf-profile.children.cycles-pp.dequeue_task_fair 24.85 ± 6% -7.1 17.77 perf-profile.children.cycles-pp.pipe_read 25.41 ± 6% -6.9 18.52 perf-profile.children.cycles-pp.new_sync_read 7.05 ± 49% -6.4 0.60 ±117% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime 6.18 ± 24% -5.8 0.43 ± 70% perf-profile.children.cycles-pp.enqueue_entity 27.33 ± 5% -5.3 22.05 perf-profile.children.cycles-pp.vfs_read 5.83 ± 5% -5.2 0.62 ± 2% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 52.87 -5.0 47.86 perf-profile.children.cycles-pp.__libc_write 5.14 ± 28% -4.8 0.38 ± 82% perf-profile.children.cycles-pp.dequeue_entity 4.87 ± 50% -4.5 0.34 ±117% perf-profile.children.cycles-pp.__unwind_start 28.87 ± 4% -4.2 24.71 perf-profile.children.cycles-pp.ksys_read 4.38 ± 50% -4.1 0.24 ±118% perf-profile.children.cycles-pp.perf_trace_sched_switch 4.28 ± 13% -4.1 0.14 ± 33% perf-profile.children.cycles-pp._raw_spin_lock 3.81 ± 38% -3.5 0.30 ± 91% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template 3.63 ± 11% -3.5 0.16 ± 3% perf-profile.children.cycles-pp.select_task_rq_fair 3.45 ± 19% -3.4 0.05 ±111% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 3.07 ± 11% -2.9 0.19 ± 3% perf-profile.children.cycles-pp.update_load_avg 2.80 ± 11% -2.6 0.16 ± 5% perf-profile.children.cycles-pp.pick_next_task_fair 2.67 ± 11% -2.5 0.12 ± 3% perf-profile.children.cycles-pp.select_idle_sibling 2.08 ± 51% -1.9 0.15 ±116% perf-profile.children.cycles-pp.__orc_find 1.99 ± 12% -1.8 0.16 ± 13% perf-profile.children.cycles-pp.update_cfs_group 1.80 ± 11% -1.7 0.08 ± 6% perf-profile.children.cycles-pp.select_idle_cpu 1.50 ± 51% -1.4 0.11 ±118% perf-profile.children.cycles-pp.unwind_get_return_address 1.30 ± 11% -1.2 0.10 ± 3% perf-profile.children.cycles-pp.reweight_entity 1.25 ± 51% -1.2 0.09 ±118% perf-profile.children.cycles-pp.orc_find 1.21 ± 51% -1.1 0.09 ±118% perf-profile.children.cycles-pp.__kernel_text_address 1.17 ± 11% -1.1 0.10 ± 6% perf-profile.children.cycles-pp.switch_fpu_return 1.12 ± 12% -1.0 0.08 ± 6% perf-profile.children.cycles-pp.set_next_entity 1.01 ± 52% -0.9 0.07 ±117% perf-profile.children.cycles-pp.kernel_text_address 0.98 ± 12% -0.9 0.09 ± 5% perf-profile.children.cycles-pp.prepare_to_wait_event 0.95 ± 51% -0.9 0.06 ±118% perf-profile.children.cycles-pp.stack_access_ok 0.94 ± 11% -0.9 0.07 ± 8% perf-profile.children.cycles-pp.__switch_to_asm 1.05 ± 11% -0.9 0.18 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.85 ± 9% -0.8 0.07 perf-profile.children.cycles-pp.__update_load_avg_se 0.84 ± 12% -0.8 0.08 ± 4% perf-profile.children.cycles-pp.restore_fpregs_from_fpstate 0.80 ± 11% -0.7 0.06 ± 7% perf-profile.children.cycles-pp.__switch_to 0.78 ± 11% -0.7 0.04 ± 44% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.76 ± 12% -0.7 0.05 perf-profile.children.cycles-pp.update_rq_clock 0.56 ± 5% -0.5 0.03 ±105% perf-profile.children.cycles-pp.sched_clock_cpu 0.42 ± 12% -0.3 0.09 ± 4% perf-profile.children.cycles-pp.__calc_delta 0.39 ± 10% -0.3 0.07 ± 7% perf-profile.children.cycles-pp.__list_del_entry_valid 0.28 ± 11% -0.2 0.09 perf-profile.children.cycles-pp._raw_spin_lock_irq 0.27 ± 12% -0.2 0.09 ± 4% perf-profile.children.cycles-pp.anon_pipe_buf_release 0.11 ± 4% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.generic_update_time 0.09 ± 4% -0.0 0.04 ± 45% perf-profile.children.cycles-pp.__mark_inode_dirty 0.10 ± 7% +0.0 0.13 ± 5% perf-profile.children.cycles-pp.kill_fasync 0.00 +0.1 0.05 ± 7% perf-profile.children.cycles-pp.uncharge_folio 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp.get_mem_cgroup_from_objcg 0.00 +0.1 0.06 ± 8% perf-profile.children.cycles-pp.__count_memcg_events 0.05 ± 46% +0.1 0.11 ± 5% perf-profile.children.cycles-pp.wake_q_add 0.16 ± 14% +0.1 0.22 ± 6% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.rmqueue 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.free_unref_page 0.17 ± 5% +0.1 0.25 ± 2% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.16 ± 25% +0.1 0.24 ± 3% perf-profile.children.cycles-pp.start_thread 0.10 ± 12% +0.1 0.18 ± 2% perf-profile.children.cycles-pp.iov_iter_init 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.get_page_from_freelist 0.07 ± 10% +0.1 0.16 ± 4% perf-profile.children.cycles-pp.rw_verify_area 0.00 +0.1 0.10 ± 9% perf-profile.children.cycles-pp.page_counter_try_charge 0.22 ± 10% +0.1 0.33 ± 3% perf-profile.children.cycles-pp.rcu_all_qs 0.54 ± 11% +0.1 0.67 ± 2% perf-profile.children.cycles-pp.__cond_resched 0.33 ± 10% +0.1 0.48 perf-profile.children.cycles-pp.aa_file_perm 0.00 +0.1 0.15 ± 8% perf-profile.children.cycles-pp.try_charge_memcg 0.10 ± 16% +0.2 0.25 ± 17% perf-profile.children.cycles-pp.wake_up_q 0.26 ± 7% +0.2 0.42 perf-profile.children.cycles-pp.timestamp_truncate 0.23 ± 9% +0.2 0.40 ± 5% perf-profile.children.cycles-pp.make_kgid 0.24 ± 13% +0.2 0.42 ± 3% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 0.23 ± 9% +0.2 0.42 ± 2% perf-profile.children.cycles-pp.make_kuid 0.28 ± 8% +0.2 0.49 ± 6% perf-profile.children.cycles-pp.__x64_sys_read 0.30 ± 6% +0.3 0.55 ± 3% perf-profile.children.cycles-pp.__x64_sys_write 0.71 ± 10% +0.3 0.97 perf-profile.children.cycles-pp.__pthread_disable_asynccancel 0.00 +0.3 0.27 ± 9% perf-profile.children.cycles-pp.mutex_spin_on_owner 0.70 ± 9% +0.3 0.98 ± 2% perf-profile.children.cycles-pp.file_update_time 0.78 ± 10% +0.3 1.06 perf-profile.children.cycles-pp.__might_sleep 0.26 ± 20% +0.3 0.55 ± 6% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 0.40 ± 9% +0.3 0.73 ± 3% perf-profile.children.cycles-pp.map_id_range_down 0.50 ± 9% +0.4 0.89 ± 2% perf-profile.children.cycles-pp.__might_fault 0.68 ± 9% +0.4 1.09 ± 2% perf-profile.children.cycles-pp.current_time 0.16 ± 18% +0.5 0.65 ± 8% perf-profile.children.cycles-pp.__mutex_lock 0.55 ± 9% +0.5 1.05 ± 2% perf-profile.children.cycles-pp.__pthread_enable_asynccancel 0.45 ± 11% +0.6 1.00 ± 3% perf-profile.children.cycles-pp.apparmor_file_permission 0.56 ± 9% +0.6 1.14 ± 2% perf-profile.children.cycles-pp.fput_many 1.26 ± 8% +0.6 1.91 perf-profile.children.cycles-pp.__might_resched 1.13 ± 7% +0.7 1.84 perf-profile.children.cycles-pp.atime_needs_update 1.40 ± 6% +0.7 2.12 ± 2% perf-profile.children.cycles-pp.touch_atime 1.17 ± 9% +0.8 1.95 perf-profile.children.cycles-pp.copyout 1.08 ± 10% +0.8 1.85 perf-profile.children.cycles-pp.mutex_unlock 1.05 ± 9% +0.9 1.94 perf-profile.children.cycles-pp.copyin 2.12 ± 8% +1.4 3.49 ± 8% perf-profile.children.cycles-pp.__fget_light 2.20 ± 8% +1.4 3.64 ± 8% perf-profile.children.cycles-pp.__fdget_pos 2.69 ± 9% +1.5 4.18 ± 3% perf-profile.children.cycles-pp.mutex_lock 2.09 ± 9% +1.5 3.60 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 2.27 ± 9% +1.5 3.79 ± 2% perf-profile.children.cycles-pp.copy_page_from_iter 1.72 ± 6% +1.6 3.35 ± 7% perf-profile.children.cycles-pp.common_file_perm 2.30 ± 9% +1.7 4.03 perf-profile.children.cycles-pp.copy_page_to_iter 0.04 ± 71% +1.9 1.94 ± 4% perf-profile.children.cycles-pp.page_counter_charge 0.07 ± 18% +2.1 2.13 ± 4% perf-profile.children.cycles-pp.obj_cgroup_charge_pages 0.07 ± 15% +2.1 2.16 ± 4% perf-profile.children.cycles-pp.__memcg_kmem_charge_page 0.11 ± 16% +2.2 2.30 ± 4% perf-profile.children.cycles-pp.__alloc_pages 0.15 ± 18% +2.6 2.71 ± 2% perf-profile.children.cycles-pp.page_counter_cancel 2.48 ± 8% +2.7 5.21 ± 5% perf-profile.children.cycles-pp.security_file_permission 1.82 ± 11% +2.9 4.75 perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 17.36 ± 4% +3.8 21.12 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.08 ± 16% +3.8 3.88 ± 4% perf-profile.children.cycles-pp.propagate_protected_usage 45.67 +4.4 50.06 perf-profile.children.cycles-pp.__libc_read 6.37 ± 9% +4.7 11.04 perf-profile.children.cycles-pp.syscall_return_via_sysret 6.22 ± 9% +5.8 11.98 perf-profile.children.cycles-pp.__entry_text_start 0.22 ± 16% +5.8 6.05 ± 3% perf-profile.children.cycles-pp.page_counter_uncharge 0.23 ± 16% +6.0 6.20 ± 3% perf-profile.children.cycles-pp.uncharge_batch 0.24 ± 15% +6.0 6.25 ± 3% perf-profile.children.cycles-pp.__mem_cgroup_uncharge 0.24 ± 16% +6.0 6.26 ± 3% perf-profile.children.cycles-pp.__put_page 3.97 ± 50% -3.7 0.28 ±117% perf-profile.self.cycles-pp.unwind_next_frame 3.44 ± 20% -3.4 0.05 ±111% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 2.08 ± 51% -1.9 0.15 ±118% perf-profile.self.cycles-pp.__orc_find 1.97 ± 12% -1.8 0.16 ± 14% perf-profile.self.cycles-pp.update_cfs_group 1.51 ± 12% -1.4 0.09 ± 4% perf-profile.self.cycles-pp.__schedule 1.40 ± 13% -1.3 0.07 ± 5% perf-profile.self.cycles-pp.update_load_avg 1.19 ± 8% -1.1 0.06 ± 8% perf-profile.self.cycles-pp.update_curr 1.18 ± 51% -1.1 0.08 ±120% perf-profile.self.cycles-pp.orc_find 0.94 ± 11% -0.9 0.07 ± 8% perf-profile.self.cycles-pp.__switch_to_asm 1.01 ± 10% -0.8 0.16 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.86 ± 12% -0.8 0.09 ± 5% perf-profile.self.cycles-pp._raw_spin_lock 0.83 ± 9% -0.8 0.07 ± 7% perf-profile.self.cycles-pp.__update_load_avg_se 0.84 ± 12% -0.8 0.08 ± 4% perf-profile.self.cycles-pp.restore_fpregs_from_fpstate 0.80 ± 50% -0.8 0.06 ±119% perf-profile.self.cycles-pp.perf_callchain_kernel 0.80 ± 50% -0.7 0.06 ±119% perf-profile.self.cycles-pp.stack_access_ok 0.74 ± 10% -0.7 0.03 ± 70% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.76 ± 11% -0.7 0.06 perf-profile.self.cycles-pp.__switch_to 0.62 ± 9% -0.6 0.06 ± 6% perf-profile.self.cycles-pp.try_to_wake_up 0.42 ± 11% -0.3 0.08 ± 5% perf-profile.self.cycles-pp.__calc_delta 0.34 ± 10% -0.3 0.06 ± 7% perf-profile.self.cycles-pp.__list_del_entry_valid 0.27 ± 12% -0.2 0.09 ± 4% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.26 ± 12% -0.2 0.08 ± 5% perf-profile.self.cycles-pp.anon_pipe_buf_release 0.09 ± 4% -0.0 0.04 ± 45% perf-profile.self.cycles-pp.__mark_inode_dirty 0.18 ± 5% +0.1 0.24 ± 5% perf-profile.self.cycles-pp.touch_atime 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.wake_up_q 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.get_mem_cgroup_from_objcg 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.__count_memcg_events 0.05 ± 46% +0.1 0.11 ± 3% perf-profile.self.cycles-pp.wake_q_add 0.13 ± 14% +0.1 0.19 ± 7% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.13 ± 11% +0.1 0.19 ± 5% perf-profile.self.cycles-pp.__fdget_pos 0.06 ± 11% +0.1 0.12 ± 3% perf-profile.self.cycles-pp.copyout 0.06 ± 49% +0.1 0.13 ± 8% perf-profile.self.cycles-pp.__mutex_unlock_slowpath 0.16 ± 7% +0.1 0.23 perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.09 ± 9% +0.1 0.17 ± 2% perf-profile.self.cycles-pp.__might_fault 0.25 ± 7% +0.1 0.33 ± 2% perf-profile.self.cycles-pp.__cond_resched 0.09 ± 11% +0.1 0.17 ± 2% perf-profile.self.cycles-pp.iov_iter_init 0.00 +0.1 0.08 ± 4% perf-profile.self.cycles-pp.uncharge_batch 0.06 ± 9% +0.1 0.14 ± 5% perf-profile.self.cycles-pp.rw_verify_area 0.17 ± 11% +0.1 0.26 ± 2% perf-profile.self.cycles-pp.rcu_all_qs 0.06 ± 11% +0.1 0.16 ± 4% perf-profile.self.cycles-pp.copyin 0.28 ± 10% +0.1 0.39 ± 2% perf-profile.self.cycles-pp.file_update_time 0.41 ± 8% +0.1 0.52 ± 2% perf-profile.self.cycles-pp.do_syscall_64 0.50 ± 13% +0.1 0.62 ± 2% perf-profile.self.cycles-pp.new_sync_read 0.55 ± 9% +0.1 0.67 ± 3% perf-profile.self.cycles-pp.vfs_read 0.08 ± 23% +0.1 0.21 ± 8% perf-profile.self.cycles-pp.__mutex_lock 0.31 ± 10% +0.1 0.45 perf-profile.self.cycles-pp.aa_file_perm 0.18 ± 14% +0.1 0.32 ± 3% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 0.24 ± 7% +0.1 0.38 perf-profile.self.cycles-pp.timestamp_truncate 0.33 ± 8% +0.2 0.48 ± 3% perf-profile.self.cycles-pp.atime_needs_update 0.25 ± 7% +0.2 0.43 ± 7% perf-profile.self.cycles-pp.__x64_sys_read 1.29 ± 9% +0.2 1.48 perf-profile.self.cycles-pp.pipe_read 0.29 ± 9% +0.2 0.48 ± 3% perf-profile.self.cycles-pp.current_time 0.36 ± 11% +0.2 0.56 ± 8% perf-profile.self.cycles-pp.ksys_read 0.27 ± 5% +0.2 0.49 ± 4% perf-profile.self.cycles-pp.__x64_sys_write 0.65 ± 10% +0.2 0.89 perf-profile.self.cycles-pp.__pthread_disable_asynccancel 0.69 ± 9% +0.2 0.93 perf-profile.self.cycles-pp.__might_sleep 0.76 ± 10% +0.2 1.01 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.00 +0.3 0.27 ± 9% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.00 +0.3 0.29 ± 5% perf-profile.self.cycles-pp.page_counter_uncharge 0.53 ± 7% +0.3 0.84 ± 4% perf-profile.self.cycles-pp.new_sync_write 0.38 ± 8% +0.3 0.70 ± 3% perf-profile.self.cycles-pp.map_id_range_down 0.56 ± 6% +0.3 0.88 ± 2% perf-profile.self.cycles-pp.vfs_write 0.29 ± 10% +0.3 0.62 ± 5% perf-profile.self.cycles-pp.ksys_write 0.42 ± 8% +0.4 0.79 ± 2% perf-profile.self.cycles-pp.copy_page_from_iter 0.65 ± 10% +0.4 1.07 perf-profile.self.cycles-pp.__libc_read 0.54 ± 9% +0.4 0.98 perf-profile.self.cycles-pp.copy_page_to_iter 0.63 ± 10% +0.5 1.12 ± 2% perf-profile.self.cycles-pp.__libc_write 0.53 ± 9% +0.5 1.03 ± 2% perf-profile.self.cycles-pp.__pthread_enable_asynccancel 0.40 ± 11% +0.5 0.93 ± 3% perf-profile.self.cycles-pp.apparmor_file_permission 0.33 ± 11% +0.6 0.88 ± 13% perf-profile.self.cycles-pp.security_file_permission 0.54 ± 9% +0.6 1.11 ± 2% perf-profile.self.cycles-pp.fput_many 1.24 ± 8% +0.6 1.88 perf-profile.self.cycles-pp.__might_resched 1.46 ± 9% +0.7 2.20 ± 2% perf-profile.self.cycles-pp.pipe_write 1.05 ± 10% +0.8 1.82 perf-profile.self.cycles-pp.mutex_unlock 0.60 ± 11% +0.9 1.51 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 1.66 ± 10% +1.1 2.79 ± 4% perf-profile.self.cycles-pp.mutex_lock 0.03 ± 70% +1.1 1.17 ± 3% perf-profile.self.cycles-pp.page_counter_charge 2.04 ± 8% +1.4 3.41 ± 8% perf-profile.self.cycles-pp.__fget_light 1.37 ± 6% +1.5 2.83 ± 9% perf-profile.self.cycles-pp.common_file_perm 2.07 ± 9% +1.5 3.56 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.15 ± 18% +2.5 2.69 ± 2% perf-profile.self.cycles-pp.page_counter_cancel 0.08 ± 19% +3.8 3.85 ± 4% perf-profile.self.cycles-pp.propagate_protected_usage 6.36 ± 9% +4.7 11.02 perf-profile.self.cycles-pp.syscall_return_via_sysret 5.50 ± 9% +5.3 10.78 perf-profile.self.cycles-pp.__entry_text_start 11.40 ± 9% +8.9 20.30 perf-profile.self.cycles-pp.syscall_exit_to_user_mode *************************************************************************************************** lkp-csl-2sp8: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 128G memory ========================================================================================= compiler/cpufreq_governor/kconfig/option_a/option_b/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/x86_64-rhel-8.3/32/32/debian-x86_64-phoronix/lkp-csl-2sp8/schbench-1.0.0/phoronix-test-suite/0x5003006 commit: v5.16-rc6 b08e888d97 ("sched/fair: Force progress on min_vruntime") v5.16-rc6 b08e888d9715f01546da7a40ca3 ---------------- --------------------------- %stddev %change %stddev \ | \ 135133 -60.6% 53294 phoronix-test-suite.schbench.32.32.usec,_99.9th_latency_percentile 2038805 -15.1% 1730336 phoronix-test-suite.time.involuntary_context_switches 0.14 ± 2% -0.0 0.11 ± 4% mpstat.cpu.all.sys% 1096 ± 31% +45.4% 1594 ± 6% proc-vmstat.numa_huge_pte_updates 669991 ± 26% +38.5% 928203 ± 5% proc-vmstat.numa_pte_updates 11.39 ± 5% +1.7 13.12 ± 5% perf-stat.i.cache-miss-rate% 30800 -10.5% 27573 perf-stat.i.context-switches 3842 +22.1% 4690 perf-stat.i.cpu-migrations 609.10 -9.8% 549.40 vmstat.procs.r 29868 -9.4% 27049 vmstat.system.cs 201252 ± 9% +4.6% 210516 vmstat.system.in 0.84 ± 7% +0.2 1.07 ± 5% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 0.78 ± 7% +0.2 1.02 ± 5% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__softirqentry_text_start.irq_exit_rcu.sysvec_apic_timer_interrupt 0.72 ± 11% +0.3 1.03 ± 5% perf-profile.calltrace.cycles-pp.newidle_balance.pick_next_task_fair.__schedule.schedule.futex_wait_queue 0.69 ± 12% +0.3 1.00 ± 5% perf-profile.calltrace.cycles-pp.load_balance.newidle_balance.pick_next_task_fair.__schedule.schedule 0.90 ± 11% +0.3 1.22 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.futex_wait_queue.futex_wait 5.69 ± 3% +0.5 6.23 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.futex_wait_queue.futex_wait.do_futex 5.73 ± 3% +0.5 6.27 ± 4% perf-profile.calltrace.cycles-pp.schedule.futex_wait_queue.futex_wait.do_futex.__x64_sys_futex 5.82 ± 3% +0.6 6.38 ± 4% perf-profile.calltrace.cycles-pp.futex_wait_queue.futex_wait.do_futex.__x64_sys_futex.do_syscall_64 6.02 ± 3% +0.6 6.58 ± 3% perf-profile.calltrace.cycles-pp.futex_wait.do_futex.__x64_sys_futex.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.50 ± 4% -1.5 5.04 ± 5% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 6.19 ± 5% -1.3 4.89 ± 5% perf-profile.children.cycles-pp.irqentry_exit_to_user_mode 12.46 ± 2% -1.1 11.32 ± 3% perf-profile.children.cycles-pp.__schedule 12.52 ± 2% -1.1 11.39 ± 3% perf-profile.children.cycles-pp.schedule 1.00 ± 15% -0.8 0.20 ± 22% perf-profile.children.cycles-pp.asm_sysvec_reschedule_ipi 6.14 ± 6% -0.8 5.37 ± 4% perf-profile.children.cycles-pp.perf_trace_sched_switch 1.39 ± 10% -0.4 0.96 ± 10% perf-profile.children.cycles-pp._raw_spin_lock 0.89 ± 14% -0.4 0.49 ± 24% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.69 ± 14% -0.2 0.48 ± 20% perf-profile.children.cycles-pp.__x64_sys_nanosleep 0.68 ± 14% -0.2 0.47 ± 20% perf-profile.children.cycles-pp.hrtimer_nanosleep 0.68 ± 14% -0.2 0.47 ± 20% perf-profile.children.cycles-pp.do_nanosleep 0.38 ± 17% -0.2 0.22 ± 24% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.15 ± 19% -0.1 0.08 ± 48% perf-profile.children.cycles-pp.write 0.21 ± 19% +0.1 0.30 ± 12% perf-profile.children.cycles-pp.kvm_is_in_guest 0.84 ± 7% +0.2 1.07 ± 5% perf-profile.children.cycles-pp.rebalance_domains 0.45 ± 9% +0.3 0.70 ± 15% perf-profile.children.cycles-pp.__calc_delta 0.74 ± 11% +0.3 1.08 ± 6% perf-profile.children.cycles-pp.newidle_balance 1.49 ± 9% +0.6 2.07 ± 4% perf-profile.children.cycles-pp.load_balance 0.89 ± 15% -0.4 0.49 ± 23% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.38 ± 11% -0.1 0.30 ± 9% perf-profile.self.cycles-pp.__schedule 0.19 ± 19% +0.1 0.28 ± 12% perf-profile.self.cycles-pp.kvm_is_in_guest 0.44 ± 8% +0.2 0.69 ± 15% perf-profile.self.cycles-pp.__calc_delta *************************************************************************************************** lkp-icl-2sp2: 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz with 256G memory ========================================================================================= compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-9/performance/pipe/4/x86_64-rhel-8.3/threads/100%/debian-10.4-x86_64-20200603.cgz/lkp-icl-2sp2/hackbench/0xd000280 commit: v5.16-rc6 b08e888d97 ("sched/fair: Force progress on min_vruntime") v5.16-rc6 b08e888d9715f01546da7a40ca3 ---------------- --------------------------- %stddev %change %stddev \ | \ 653575 +46.6% 958089 hackbench.throughput 17908198 ± 5% -36.4% 11391663 ± 5% cpuidle..usage 155.26 -24.8% 116.73 ± 3% uptime.boot 0.03 ± 3% -0.0 0.03 ± 6% mpstat.cpu.all.soft% 9.86 +3.3 13.20 mpstat.cpu.all.usr% 2052 ± 20% -35.2% 1329 ± 23% numa-meminfo.node0.Active 2052 ± 20% -35.2% 1329 ± 23% numa-meminfo.node0.Active(anon) 3312 -16.9% 2753 ± 2% meminfo.Active 3312 -16.9% 2753 ± 2% meminfo.Active(anon) 128712 ± 2% -24.9% 96605 meminfo.AnonHugePages 2554 +1.2% 2584 turbostat.Bzy_MHz 17637175 ± 5% -35.5% 11382497 ± 5% turbostat.C1 1.321e+08 ± 4% -77.7% 29492706 ± 3% turbostat.IRQ 1213 +108.1% 2525 ± 3% vmstat.procs.r 23544038 -94.2% 1373085 ± 4% vmstat.system.cs 1267010 ± 3% -65.1% 441932 ± 2% vmstat.system.in 28404703 ± 7% +154.1% 72168987 ± 4% numa-numastat.node0.local_node 28445356 ± 7% +154.0% 72237824 ± 4% numa-numastat.node0.numa_hit 25297912 ± 6% +186.2% 72404938 ± 4% numa-numastat.node1.local_node 25371744 ± 6% +185.5% 72437776 ± 4% numa-numastat.node1.numa_hit 513.50 ± 21% -35.8% 329.67 ± 24% numa-vmstat.node0.nr_active_anon 513.50 ± 21% -35.8% 329.67 ± 24% numa-vmstat.node0.nr_zone_active_anon 16630487 ± 8% +129.4% 38157859 ± 4% numa-vmstat.node0.numa_hit 16587877 ± 8% +129.6% 38086060 ± 4% numa-vmstat.node0.numa_local 14598373 ± 8% +159.4% 37867111 ± 6% numa-vmstat.node1.numa_hit 14483676 ± 8% +160.9% 37790942 ± 6% numa-vmstat.node1.numa_local 6.63 ± 64% -6.0 0.62 ±223% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.63 ± 64% -6.0 0.62 ±223% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.do_sys_open.do_syscall_64 3.75 ± 72% -3.7 0.00 perf-profile.calltrace.cycles-pp.open64 5.44 ± 51% -2.9 2.52 ±163% perf-profile.calltrace.cycles-pp.copy_page_from_iter_atomic.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.new_sync_write 4.48 ± 57% -2.7 1.76 ±141% perf-profile.calltrace.cycles-pp.number.vsnprintf.seq_vprintf.seq_printf.show_interrupts 3.65 ± 77% -1.0 2.67 ±223% perf-profile.calltrace.cycles-pp.proc_pid_status.proc_single_show.seq_read_iter.seq_read.vfs_read 6.63 ± 64% -6.0 0.62 ±223% perf-profile.children.cycles-pp.do_sys_open 6.63 ± 64% -6.0 0.62 ±223% perf-profile.children.cycles-pp.do_sys_openat2 6.63 ± 64% -6.0 0.62 ±223% perf-profile.children.cycles-pp.do_filp_open 6.63 ± 64% -6.0 0.62 ±223% perf-profile.children.cycles-pp.path_openat 5.04 ± 64% -3.3 1.76 ±141% perf-profile.children.cycles-pp.number 5.44 ± 51% -2.9 2.52 ±163% perf-profile.children.cycles-pp.copy_page_from_iter_atomic 5.04 ± 64% -3.3 1.76 ±141% perf-profile.self.cycles-pp.number 826.83 -16.8% 688.33 ± 2% proc-vmstat.nr_active_anon 79216 -1.6% 77968 proc-vmstat.nr_anon_pages 84555 ± 2% -2.6% 82387 proc-vmstat.nr_inactive_anon 92536 -4.6% 88260 proc-vmstat.nr_kernel_stack 6216 ± 31% -16.0% 5220 proc-vmstat.nr_shmem 826.83 -16.8% 688.33 ± 2% proc-vmstat.nr_zone_active_anon 84555 ± 2% -2.6% 82387 proc-vmstat.nr_zone_inactive_anon 40595 ± 24% -85.3% 5957 ± 89% proc-vmstat.numa_hint_faults 20890 ± 41% -87.4% 2629 ±105% proc-vmstat.numa_hint_faults_local 53819891 +168.8% 1.447e+08 proc-vmstat.numa_hit 53705406 +169.2% 1.446e+08 proc-vmstat.numa_local 20887 ± 27% -84.1% 3328 ±100% proc-vmstat.numa_pages_migrated 72657 ± 30% -55.8% 32092 ± 61% proc-vmstat.numa_pte_updates 53822302 +168.7% 1.446e+08 proc-vmstat.pgalloc_normal 449333 ± 13% -33.5% 298589 ± 3% proc-vmstat.pgfault 53610460 +169.4% 1.444e+08 proc-vmstat.pgfree 20887 ± 27% -84.1% 3328 ±100% proc-vmstat.pgmigrate_success 20678 ± 2% -32.9% 13880 ± 4% proc-vmstat.pgreuse 7.20 ± 2% -56.8% 3.11 ± 2% perf-stat.i.MPKI 5.303e+10 -4.0% 5.092e+10 perf-stat.i.branch-instructions 0.63 -0.5 0.16 perf-stat.i.branch-miss-rate% 3.195e+08 -77.6% 71713145 perf-stat.i.branch-misses 7.04 ± 4% +18.3 25.32 ± 3% perf-stat.i.cache-miss-rate% 1.199e+08 ± 3% +41.4% 1.695e+08 ± 2% perf-stat.i.cache-misses 1.824e+09 -59.2% 7.436e+08 ± 2% perf-stat.i.cache-references 24276211 -94.4% 1358591 ± 4% perf-stat.i.context-switches 1521377 ± 3% -87.5% 190633 ± 7% perf-stat.i.cpu-migrations 3127 ± 5% -39.4% 1893 ± 2% perf-stat.i.cycles-between-cache-misses 0.00 ± 6% -0.0 0.00 ± 9% perf-stat.i.dTLB-load-miss-rate% 1804532 ± 5% -23.3% 1383626 ± 10% perf-stat.i.dTLB-load-misses 0.00 ± 8% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate% 343753 ± 3% -23.8% 261974 ± 4% perf-stat.i.dTLB-store-misses 4.648e+10 -3.9% 4.464e+10 perf-stat.i.dTLB-stores 2.617e+11 -3.4% 2.528e+11 perf-stat.i.instructions 0.82 -3.5% 0.79 perf-stat.i.ipc 517.04 ± 2% -9.8% 466.28 perf-stat.i.metric.K/sec 1407 -3.3% 1361 perf-stat.i.metric.M/sec 87.20 -17.6 69.59 perf-stat.i.node-load-miss-rate% 2795035 ± 6% +214.8% 8799183 ± 5% perf-stat.i.node-loads 59.48 +3.0 62.46 perf-stat.i.node-store-miss-rate% 11371827 ± 4% +62.7% 18504651 ± 2% perf-stat.i.node-store-misses 7495979 ± 3% +48.1% 11099696 perf-stat.i.node-stores 6.92 ± 2% -57.8% 2.92 ± 2% perf-stat.overall.MPKI 0.60 -0.5 0.14 perf-stat.overall.branch-miss-rate% 6.62 ± 4% +16.3 22.96 ± 4% perf-stat.overall.cache-miss-rate% 1.20 +4.7% 1.26 perf-stat.overall.cpi 2621 ± 3% -28.5% 1873 ± 2% perf-stat.overall.cycles-between-cache-misses 0.00 ± 5% -0.0 0.00 ± 11% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 2% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate% 0.83 -4.5% 0.80 perf-stat.overall.ipc 86.77 -17.9 68.83 perf-stat.overall.node-load-miss-rate% 60.35 +2.2 62.51 perf-stat.overall.node-store-miss-rate% 5.269e+10 -4.6% 5.025e+10 perf-stat.ps.branch-instructions 3.154e+08 -77.7% 70229688 perf-stat.ps.branch-misses 1.191e+08 ± 3% +40.4% 1.673e+08 ± 2% perf-stat.ps.cache-misses 1.8e+09 ± 2% -59.5% 7.291e+08 ± 2% perf-stat.ps.cache-references 24036022 -94.5% 1320301 ± 4% perf-stat.ps.context-switches 1470959 ± 3% -87.6% 182972 ± 7% perf-stat.ps.cpu-migrations 1759980 ± 5% -23.8% 1341985 ± 11% perf-stat.ps.dTLB-load-misses 335955 ± 2% -24.1% 255091 ± 4% perf-stat.ps.dTLB-store-misses 4.618e+10 -4.6% 4.406e+10 perf-stat.ps.dTLB-stores 2.6e+11 -4.1% 2.495e+11 perf-stat.ps.instructions 2784328 ± 5% +212.5% 8701815 ± 5% perf-stat.ps.node-loads 11244792 ± 4% +62.0% 18220133 ± 2% perf-stat.ps.node-store-misses 7386068 ± 3% +47.9% 10923448 perf-stat.ps.node-stores 2.568e+13 -37.0% 1.619e+13 perf-stat.total.instructions 18746 ± 12% -39.3% 11371 ± 9% softirqs.CPU0.RCU 19003 ± 13% -37.5% 11885 ± 16% softirqs.CPU1.RCU 19070 ± 12% -42.2% 11015 ± 9% softirqs.CPU10.RCU 19263 ± 6% -38.0% 11951 ± 6% softirqs.CPU100.RCU 19660 ± 7% -39.6% 11878 ± 5% softirqs.CPU101.RCU 19396 ± 6% -38.4% 11942 ± 5% softirqs.CPU102.RCU 19208 ± 6% -38.0% 11912 ± 5% softirqs.CPU103.RCU 19759 ± 7% -40.1% 11826 ± 5% softirqs.CPU104.RCU 19676 ± 6% -40.1% 11787 ± 5% softirqs.CPU105.RCU 19141 ± 6% -38.1% 11849 ± 6% softirqs.CPU106.RCU 19236 ± 6% -38.2% 11883 ± 6% softirqs.CPU107.RCU 19948 ± 5% -39.7% 12033 ± 5% softirqs.CPU108.RCU 19426 ± 6% -39.1% 11832 ± 6% softirqs.CPU109.RCU 18843 ± 11% -36.0% 12060 ± 21% softirqs.CPU11.RCU 19548 ± 6% -39.8% 11765 ± 6% softirqs.CPU110.RCU 19460 ± 6% -38.7% 11920 ± 5% softirqs.CPU111.RCU 17625 ± 6% -39.9% 10596 ± 4% softirqs.CPU112.RCU 17477 ± 6% -40.1% 10472 ± 5% softirqs.CPU113.RCU 17702 ± 8% -40.6% 10506 ± 5% softirqs.CPU114.RCU 17433 ± 7% -39.6% 10530 ± 4% softirqs.CPU115.RCU 17184 ± 9% -38.5% 10572 ± 5% softirqs.CPU116.RCU 17123 ± 6% -38.5% 10534 ± 8% softirqs.CPU117.RCU 17453 ± 6% -39.9% 10497 ± 5% softirqs.CPU118.RCU 17449 ± 7% -40.4% 10395 ± 5% softirqs.CPU119.RCU 18501 ± 12% -39.6% 11180 ± 10% softirqs.CPU12.RCU 17417 ± 6% -39.1% 10600 ± 5% softirqs.CPU120.RCU 17554 ± 6% -40.2% 10501 ± 6% softirqs.CPU121.RCU 18068 ± 12% -39.4% 10945 ± 7% softirqs.CPU122.RCU 17356 ± 5% -39.1% 10574 ± 5% softirqs.CPU123.RCU 17411 ± 6% -38.5% 10715 ± 5% softirqs.CPU124.RCU 17615 ± 6% -39.2% 10702 ± 6% softirqs.CPU125.RCU 17569 ± 6% -39.5% 10631 ± 5% softirqs.CPU126.RCU 17797 ± 6% -40.2% 10635 ± 5% softirqs.CPU127.RCU 19570 ± 13% -38.9% 11967 ± 21% softirqs.CPU13.RCU 18846 ± 11% -38.5% 11586 ± 13% softirqs.CPU14.RCU 18642 ± 13% -39.6% 11255 ± 10% softirqs.CPU15.RCU 18684 ± 12% -41.8% 10869 ± 10% softirqs.CPU16.RCU 18890 ± 14% -42.9% 10790 ± 11% softirqs.CPU17.RCU 18794 ± 13% -40.0% 11278 ± 6% softirqs.CPU18.RCU 18930 ± 13% -41.4% 11096 ± 11% softirqs.CPU19.RCU 18567 ± 13% -40.6% 11028 ± 9% softirqs.CPU2.RCU 18769 ± 12% -41.4% 11006 ± 9% softirqs.CPU20.RCU 18500 ± 12% -40.3% 11040 ± 12% softirqs.CPU21.RCU 18477 ± 14% -42.3% 10655 ± 10% softirqs.CPU22.RCU 18551 ± 12% -43.4% 10507 ± 18% softirqs.CPU23.RCU 18638 ± 12% -42.0% 10805 ± 21% softirqs.CPU24.RCU 18602 ± 12% -42.8% 10642 ± 8% softirqs.CPU25.RCU 18739 ± 12% -41.3% 11006 ± 9% softirqs.CPU26.RCU 18763 ± 13% -41.9% 10900 ± 10% softirqs.CPU27.RCU 18268 ± 19% -40.3% 10898 ± 12% softirqs.CPU28.RCU 18584 ± 12% -42.4% 10706 ± 9% softirqs.CPU29.RCU 19256 ± 12% -43.5% 10889 ± 11% softirqs.CPU3.RCU 18713 ± 12% -41.8% 10887 ± 8% softirqs.CPU30.RCU 18710 ± 13% -41.4% 10960 ± 9% softirqs.CPU31.RCU 18442 ± 5% -40.2% 11023 ± 4% softirqs.CPU32.RCU 17972 ± 7% -38.1% 11126 ± 5% softirqs.CPU33.RCU 18139 ± 6% -39.4% 10986 ± 4% softirqs.CPU34.RCU 17619 ± 6% -38.7% 10795 ± 5% softirqs.CPU35.RCU 17452 ± 6% -38.1% 10795 ± 3% softirqs.CPU36.RCU 17860 ± 7% -38.0% 11079 ± 4% softirqs.CPU37.RCU 17703 ± 6% -39.5% 10710 ± 10% softirqs.CPU38.RCU 17733 ± 4% -39.9% 10655 ± 5% softirqs.CPU39.RCU 19506 ± 15% -43.6% 11001 ± 8% softirqs.CPU4.RCU 18044 ± 6% -39.7% 10882 ± 5% softirqs.CPU40.RCU 18256 ± 6% -40.4% 10881 ± 5% softirqs.CPU41.RCU 17318 ± 7% -37.5% 10823 ± 5% softirqs.CPU42.RCU 17660 ± 7% -40.3% 10539 ± 5% softirqs.CPU43.RCU 18660 ± 5% -41.9% 10844 ± 4% softirqs.CPU44.RCU 17768 ± 6% -38.9% 10852 ± 6% softirqs.CPU45.RCU 17655 ± 7% -39.3% 10713 ± 5% softirqs.CPU46.RCU 17747 ± 6% -39.1% 10816 ± 5% softirqs.CPU47.RCU 17656 ± 6% -39.8% 10636 ± 5% softirqs.CPU48.RCU 17592 ± 6% -39.5% 10646 ± 4% softirqs.CPU49.RCU 19129 ± 13% -39.4% 11599 ± 19% softirqs.CPU5.RCU 17586 ± 6% -40.1% 10540 ± 4% softirqs.CPU50.RCU 17758 ± 5% -40.5% 10570 ± 4% softirqs.CPU51.RCU 17960 ± 7% -40.1% 10765 ± 4% softirqs.CPU52.RCU 16892 ± 8% -39.3% 10256 ± 7% softirqs.CPU53.RCU 17597 ± 5% -40.2% 10524 ± 4% softirqs.CPU54.RCU 17614 ± 6% -40.9% 10412 ± 5% softirqs.CPU55.RCU 17682 ± 7% -40.0% 10600 ± 4% softirqs.CPU56.RCU 18064 ± 7% -40.1% 10816 ± 5% softirqs.CPU57.RCU 17623 ± 6% -40.4% 10500 ± 4% softirqs.CPU58.RCU 17585 ± 6% -40.6% 10450 ± 4% softirqs.CPU59.RCU 18840 ± 12% -40.9% 11139 ± 8% softirqs.CPU6.RCU 17667 ± 6% -39.1% 10761 ± 7% softirqs.CPU60.RCU 17780 ± 6% -37.0% 11206 ± 5% softirqs.CPU61.RCU 17766 ± 6% -37.9% 11029 ± 9% softirqs.CPU62.RCU 17961 ± 7% -42.4% 10348 ± 8% softirqs.CPU63.RCU 18806 ± 15% -43.0% 10724 ± 8% softirqs.CPU64.RCU 18615 ± 13% -39.7% 11220 ± 10% softirqs.CPU65.RCU 18447 ± 13% -37.8% 11482 ± 16% softirqs.CPU66.RCU 18479 ± 13% -40.7% 10960 ± 11% softirqs.CPU67.RCU 18815 ± 13% -42.0% 10916 ± 10% softirqs.CPU68.RCU 18691 ± 13% -41.6% 10912 ± 9% softirqs.CPU69.RCU 19163 ± 13% -40.6% 11392 ± 13% softirqs.CPU7.RCU 18358 ± 13% -40.2% 10982 ± 9% softirqs.CPU70.RCU 18096 ± 19% -40.1% 10848 ± 9% softirqs.CPU71.RCU 18737 ± 13% -42.4% 10789 ± 8% softirqs.CPU72.RCU 19085 ± 11% -43.1% 10864 ± 7% softirqs.CPU73.RCU 18868 ± 13% -37.1% 11868 ± 9% softirqs.CPU74.RCU 18523 ± 13% -41.3% 10876 ± 11% softirqs.CPU75.RCU 18366 ± 13% -40.9% 10850 ± 10% softirqs.CPU76.RCU 19087 ± 12% -43.1% 10869 ± 9% softirqs.CPU77.RCU 18624 ± 11% -41.4% 10910 ± 9% softirqs.CPU78.RCU 18660 ± 12% -41.6% 10889 ± 10% softirqs.CPU79.RCU 18591 ± 12% -41.0% 10971 ± 9% softirqs.CPU8.RCU 18670 ± 13% -42.9% 10654 ± 8% softirqs.CPU80.RCU 19309 ± 16% -44.8% 10652 ± 11% softirqs.CPU81.RCU 18675 ± 12% -42.4% 10753 ± 9% softirqs.CPU82.RCU 18285 ± 13% -41.3% 10735 ± 10% softirqs.CPU83.RCU 18831 ± 13% -41.6% 11003 ± 8% softirqs.CPU84.RCU 18235 ± 13% -41.1% 10734 ± 11% softirqs.CPU85.RCU 18344 ± 13% -41.6% 10721 ± 7% softirqs.CPU86.RCU 18706 ± 13% -41.9% 10860 ± 14% softirqs.CPU87.RCU 18387 ± 13% -41.2% 10809 ± 13% softirqs.CPU88.RCU 18812 ± 12% -42.0% 10904 ± 8% softirqs.CPU89.RCU 18664 ± 11% -40.3% 11136 ± 10% softirqs.CPU9.RCU 18902 ± 13% -41.9% 10976 ± 10% softirqs.CPU90.RCU 18687 ± 12% -39.1% 11374 ± 14% softirqs.CPU91.RCU 18283 ± 14% -40.3% 10915 ± 10% softirqs.CPU92.RCU 18476 ± 12% -41.7% 10766 ± 10% softirqs.CPU93.RCU 18580 ± 13% -40.2% 11119 ± 7% softirqs.CPU94.RCU 18675 ± 13% -40.3% 11149 ± 10% softirqs.CPU95.RCU 19611 ± 6% -38.8% 12003 ± 5% softirqs.CPU96.RCU 19352 ± 6% -37.9% 12016 ± 5% softirqs.CPU97.RCU 20041 ± 5% -40.3% 11974 ± 4% softirqs.CPU98.RCU 19370 ± 6% -39.1% 11792 ± 6% softirqs.CPU99.RCU 2358890 ± 9% -40.3% 1409034 ± 6% softirqs.RCU 732676 ± 8% -36.8% 463351 ± 9% softirqs.SCHED 33955 ± 5% -19.4% 27373 ± 9% softirqs.TIMER *************************************************************************************************** lkp-csl-2sp9: 88 threads 2 sockets Intel(R) Xeon(R) Gold 6238M CPU @ 2.10GHz with 128G memory ========================================================================================= compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-9/performance/pipe/8/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp9/hackbench/0x5003006 commit: v5.16-rc6 b08e888d97 ("sched/fair: Force progress on min_vruntime") v5.16-rc6 b08e888d9715f01546da7a40ca3 ---------------- --------------------------- %stddev %change %stddev \ | \ 306153 ± 3% +82.2% 557858 hackbench.throughput 5.929e+08 ± 10% -21.0% 4.684e+08 ± 4% cpuidle..time 85185415 ± 2% -64.7% 30105807 ± 4% cpuidle..usage 331.48 ± 2% -43.3% 187.90 uptime.boot 3435 ± 5% -10.0% 3093 uptime.idle 2.46 ± 8% +0.8 3.31 ± 7% mpstat.cpu.all.idle% 0.07 ± 5% -0.0 0.06 ± 3% mpstat.cpu.all.soft% 6.52 +0.7 7.24 mpstat.cpu.all.usr% 43549184 +115.3% 93753986 numa-numastat.node0.local_node 43604110 +115.0% 93744217 numa-numastat.node0.numa_hit 45251285 +123.4% 1.011e+08 numa-numastat.node1.local_node 45266256 +123.3% 1.011e+08 numa-numastat.node1.numa_hit 2890160 +62.0% 4680855 ± 5% vmstat.memory.cache 663.80 +53.4% 1018 vmstat.procs.r 11780323 -87.9% 1428215 ± 2% vmstat.system.cs 1353520 -83.1% 229413 vmstat.system.in 31869811 ± 5% -35.4% 20598074 ± 4% turbostat.C1 0.55 ± 4% +0.7 1.21 ± 4% turbostat.C1% 481922 ± 30% +64.2% 791346 ± 12% turbostat.C1E 0.52 ± 60% +1.1 1.65 ± 26% turbostat.C1E% 1.70 ± 13% +85.9% 3.16 ± 5% turbostat.CPU%c1 4.019e+08 -91.1% 35603903 turbostat.IRQ 67.60 ± 2% +5.0% 71.00 turbostat.PkgTmp 54.89 +19.5% 65.57 turbostat.RAMWatt 174395 -13.7% 150468 meminfo.AnonHugePages 320426 +13.1% 362465 meminfo.AnonPages 2780524 +63.4% 4543649 ± 5% meminfo.Cached 1267846 ± 3% +141.6% 3063461 ± 9% meminfo.Committed_AS 10864230 ± 14% +34.7% 14630912 meminfo.DirectMap2M 522339 ± 5% +348.8% 2344046 ± 9% meminfo.Inactive 522188 ± 5% +348.9% 2343898 ± 9% meminfo.Inactive(anon) 144780 ± 11% +206.4% 443637 meminfo.Mapped 289684 ± 11% +608.4% 2052268 ± 13% meminfo.Shmem 251322 ± 6% +15.0% 289116 ± 4% numa-meminfo.node0.AnonPages 253166 ± 5% +17.7% 297948 ± 3% numa-meminfo.node0.Inactive 253014 ± 5% +17.7% 297800 ± 3% numa-meminfo.node0.Inactive(anon) 54497 ± 38% +48.3% 80817 numa-meminfo.node0.KReclaimable 37290 ± 31% +158.1% 96237 numa-meminfo.node0.Mapped 54497 ± 38% +48.3% 80817 numa-meminfo.node0.SReclaimable 218791 ± 13% +18.4% 259108 ± 2% numa-meminfo.node0.Slab 264384 ± 10% +677.0% 2054141 ± 12% numa-meminfo.node1.Inactive 264384 ± 10% +677.0% 2054141 ± 12% numa-meminfo.node1.Inactive(anon) 51486 ± 41% -47.0% 27310 ± 4% numa-meminfo.node1.KReclaimable 104995 ± 24% +232.5% 349071 ± 2% numa-meminfo.node1.Mapped 51486 ± 41% -47.0% 27310 ± 4% numa-meminfo.node1.SReclaimable 282998 ± 12% +624.5% 2050190 ± 13% numa-meminfo.node1.Shmem 62863 ± 6% +15.0% 72313 ± 4% numa-vmstat.node0.nr_anon_pages 63287 ± 5% +17.7% 74490 ± 4% numa-vmstat.node0.nr_inactive_anon 9356 ± 32% +155.7% 23923 ± 2% numa-vmstat.node0.nr_mapped 13624 ± 38% +48.3% 20203 numa-vmstat.node0.nr_slab_reclaimable 63287 ± 5% +17.7% 74490 ± 4% numa-vmstat.node0.nr_zone_inactive_anon 22307082 +117.3% 48473269 numa-vmstat.node0.numa_hit 22243588 +117.8% 48455183 numa-vmstat.node0.numa_local 65013 ± 37% -29.2% 46039 ± 51% numa-vmstat.node0.numa_other 65889 ± 10% +674.4% 510264 ± 12% numa-vmstat.node1.nr_inactive_anon 26129 ± 23% +231.9% 86733 numa-vmstat.node1.nr_mapped 70702 ± 12% +620.4% 509352 ± 14% numa-vmstat.node1.nr_shmem 12867 ± 41% -47.0% 6824 ± 4% numa-vmstat.node1.nr_slab_reclaimable 65888 ± 10% +674.4% 510262 ± 12% numa-vmstat.node1.nr_zone_inactive_anon 22961153 ± 2% +123.1% 51232967 numa-vmstat.node1.numa_hit 22925615 ± 2% +123.3% 51200328 numa-vmstat.node1.numa_local 80078 +13.3% 90710 proc-vmstat.nr_anon_pages 694929 +63.5% 1136182 ± 5% proc-vmstat.nr_file_pages 129914 ± 5% +351.4% 586398 ± 9% proc-vmstat.nr_inactive_anon 35791 ± 10% +210.4% 111105 proc-vmstat.nr_mapped 72218 ± 11% +610.8% 513336 ± 13% proc-vmstat.nr_shmem 26494 +2.1% 27043 proc-vmstat.nr_slab_reclaimable 129914 ± 5% +351.4% 586398 ± 9% proc-vmstat.nr_zone_inactive_anon 188590 ± 7% +54.5% 291286 proc-vmstat.numa_hint_faults 92457 ± 18% +106.8% 191238 ± 3% proc-vmstat.numa_hint_faults_local 88873875 +119.2% 1.948e+08 proc-vmstat.numa_hit 88803978 +119.4% 1.948e+08 proc-vmstat.numa_local 73586 ± 20% -86.0% 10275 ± 31% proc-vmstat.numa_pages_migrated 363990 ± 6% +25.4% 456525 ± 4% proc-vmstat.numa_pte_updates 575511 ± 11% -95.6% 25379 ± 11% proc-vmstat.pgactivate 88875945 +119.1% 1.948e+08 proc-vmstat.pgalloc_normal 1778434 -13.4% 1539544 proc-vmstat.pgfault 88878278 +117.2% 1.93e+08 proc-vmstat.pgfree 73586 ± 20% -86.0% 10275 ± 31% proc-vmstat.pgmigrate_success 79700 -26.3% 58766 proc-vmstat.pgreuse 15.10 -52.8% 7.12 perf-stat.i.MPKI 2.731e+10 +9.8% 2.998e+10 perf-stat.i.branch-instructions 1.05 -0.5 0.53 perf-stat.i.branch-miss-rate% 2.846e+08 -44.9% 1.568e+08 perf-stat.i.branch-misses 2.28 ± 3% +15.4 17.71 perf-stat.i.cache-miss-rate% 43697080 ± 3% +322.8% 1.847e+08 perf-stat.i.cache-misses 2.006e+09 -47.1% 1.062e+09 perf-stat.i.cache-references 11893072 -88.0% 1423249 ± 2% perf-stat.i.context-switches 1.80 -11.8% 1.59 perf-stat.i.cpi 2.413e+11 -1.2% 2.383e+11 perf-stat.i.cpu-cycles 1571433 -83.4% 260389 ± 3% perf-stat.i.cpu-migrations 5916 ± 3% -77.9% 1309 perf-stat.i.cycles-between-cache-misses 0.09 ± 4% -0.1 0.02 perf-stat.i.dTLB-load-miss-rate% 37010511 ± 3% -80.4% 7255935 perf-stat.i.dTLB-load-misses 3.979e+10 +15.4% 4.592e+10 perf-stat.i.dTLB-loads 0.01 ± 90% -0.0 0.00 perf-stat.i.dTLB-store-miss-rate% 1195730 ± 92% -75.8% 289743 perf-stat.i.dTLB-store-misses 2.294e+10 +10.4% 2.533e+10 perf-stat.i.dTLB-stores 63.68 +28.2 91.84 perf-stat.i.iTLB-load-miss-rate% 1.317e+08 -23.1% 1.013e+08 ± 3% perf-stat.i.iTLB-load-misses 74778299 -87.7% 9175043 perf-stat.i.iTLB-loads 1.351e+11 +11.0% 1.5e+11 perf-stat.i.instructions 1028 +45.9% 1501 ± 2% perf-stat.i.instructions-per-iTLB-miss 0.56 +12.3% 0.63 perf-stat.i.ipc 1.95 ± 8% +89.0% 3.68 ± 15% perf-stat.i.major-faults 2.74 -1.2% 2.71 perf-stat.i.metric.GHz 1243 -27.0% 907.17 perf-stat.i.metric.K/sec 1046 +11.1% 1162 perf-stat.i.metric.M/sec 5607 +64.2% 9210 perf-stat.i.minor-faults 84.88 -5.7 79.22 perf-stat.i.node-load-miss-rate% 9990810 ± 4% +153.9% 25363879 perf-stat.i.node-load-misses 1760915 +288.1% 6834461 perf-stat.i.node-loads 61.25 -6.8 54.48 perf-stat.i.node-store-miss-rate% 6249188 ± 4% +215.3% 19700759 perf-stat.i.node-store-misses 3834193 +337.7% 16783709 perf-stat.i.node-stores 5609 +64.2% 9213 perf-stat.i.page-faults 14.84 -52.3% 7.08 perf-stat.overall.MPKI 1.04 -0.5 0.52 perf-stat.overall.branch-miss-rate% 2.18 ± 3% +15.2 17.40 perf-stat.overall.cache-miss-rate% 1.79 -11.0% 1.59 perf-stat.overall.cpi 5525 ± 3% -76.7% 1289 perf-stat.overall.cycles-between-cache-misses 0.09 ± 4% -0.1 0.02 perf-stat.overall.dTLB-load-miss-rate% 0.01 ± 92% -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate% 63.78 +27.9 91.69 perf-stat.overall.iTLB-load-miss-rate% 1025 +44.5% 1482 ± 3% perf-stat.overall.instructions-per-iTLB-miss 0.56 +12.4% 0.63 perf-stat.overall.ipc 84.82 -6.1 78.68 perf-stat.overall.node-load-miss-rate% 61.93 -8.0 53.97 perf-stat.overall.node-store-miss-rate% 2.721e+10 +9.4% 2.978e+10 perf-stat.ps.branch-instructions 2.837e+08 -45.1% 1.558e+08 perf-stat.ps.branch-misses 43572385 ± 3% +321.3% 1.836e+08 perf-stat.ps.cache-misses 1.999e+09 -47.2% 1.055e+09 perf-stat.ps.cache-references 11855647 -88.1% 1414792 ± 2% perf-stat.ps.context-switches 2.405e+11 -1.5% 2.368e+11 perf-stat.ps.cpu-cycles 1566133 -83.5% 258729 ± 3% perf-stat.ps.cpu-migrations 36897722 ± 3% -80.5% 7210567 perf-stat.ps.dTLB-load-misses 3.965e+10 +15.1% 4.562e+10 perf-stat.ps.dTLB-loads 1196120 ± 93% -75.9% 288407 perf-stat.ps.dTLB-store-misses 2.286e+10 +10.1% 2.517e+10 perf-stat.ps.dTLB-stores 1.313e+08 -23.3% 1.007e+08 ± 3% perf-stat.ps.iTLB-load-misses 74541435 -87.8% 9121216 ± 2% perf-stat.ps.iTLB-loads 1.347e+11 +10.7% 1.491e+11 perf-stat.ps.instructions 1.92 ± 8% +89.4% 3.65 ± 17% perf-stat.ps.major-faults 5594 +63.0% 9121 perf-stat.ps.minor-faults 9953250 ± 4% +153.1% 25190078 perf-stat.ps.node-load-misses 1779285 +283.5% 6823202 perf-stat.ps.node-loads 6223057 ± 4% +214.4% 19564014 perf-stat.ps.node-store-misses 3822259 +336.5% 16685369 perf-stat.ps.node-stores 5596 +63.0% 9125 perf-stat.ps.page-faults 3.969e+13 -42.3% 2.289e+13 perf-stat.total.instructions 46802 ± 19% -50.8% 23023 ± 5% softirqs.CPU0.RCU 40693 ± 3% -59.8% 16375 ± 6% softirqs.CPU0.SCHED 45742 ± 19% -51.5% 22179 ± 7% softirqs.CPU1.RCU 37913 ± 3% -65.5% 13070 ± 4% softirqs.CPU1.SCHED 46439 ± 18% -54.7% 21056 ± 9% softirqs.CPU10.RCU 35358 ± 2% -65.7% 12116 softirqs.CPU10.SCHED 44730 ± 16% -51.8% 21549 ± 9% softirqs.CPU11.RCU 34166 ± 3% -64.8% 12037 ± 2% softirqs.CPU11.SCHED 45767 ± 20% -52.5% 21722 ± 7% softirqs.CPU12.RCU 34792 ± 3% -63.0% 12875 ± 4% softirqs.CPU12.SCHED 45780 ± 20% -51.2% 22324 ± 13% softirqs.CPU13.RCU 35293 ± 2% -66.6% 11776 softirqs.CPU13.SCHED 45758 ± 19% -54.0% 21062 ± 8% softirqs.CPU14.RCU 37324 -68.3% 11829 ± 3% softirqs.CPU14.SCHED 48752 ± 19% -50.3% 24248 ± 3% softirqs.CPU15.RCU 34954 ± 2% -64.0% 12585 ± 2% softirqs.CPU15.SCHED 48784 ± 19% -53.2% 22850 ± 7% softirqs.CPU16.RCU 35224 -66.3% 11857 ± 2% softirqs.CPU16.SCHED 48992 ± 21% -53.0% 23041 ± 9% softirqs.CPU17.RCU 35725 ± 2% -67.1% 11744 softirqs.CPU17.SCHED 48534 ± 19% -53.4% 22600 ± 7% softirqs.CPU18.RCU 36715 ± 3% -68.0% 11761 softirqs.CPU18.SCHED 48555 ± 19% -53.0% 22798 ± 8% softirqs.CPU19.RCU 36397 ± 3% -67.6% 11789 softirqs.CPU19.SCHED 46076 ± 19% -52.5% 21876 ± 5% softirqs.CPU2.RCU 37360 ± 2% -65.3% 12948 softirqs.CPU2.SCHED 49247 ± 19% -52.0% 23646 ± 10% softirqs.CPU20.RCU 35625 ± 4% -66.6% 11892 softirqs.CPU20.SCHED 48759 ± 20% -53.2% 22817 ± 6% softirqs.CPU21.RCU 34901 ± 2% -67.0% 11522 softirqs.CPU21.SCHED 47637 ± 19% -53.6% 22121 ± 4% softirqs.CPU22.RCU 35345 ± 3% -65.4% 12214 softirqs.CPU22.SCHED 47116 ± 20% -53.7% 21829 ± 4% softirqs.CPU23.RCU 38022 ± 3% -67.9% 12212 softirqs.CPU23.SCHED 46266 ± 19% -53.2% 21669 ± 4% softirqs.CPU24.RCU 39105 ± 4% -68.3% 12383 softirqs.CPU24.SCHED 47073 ± 19% -53.7% 21772 ± 6% softirqs.CPU25.RCU 38229 ± 5% -67.5% 12427 softirqs.CPU25.SCHED 47226 ± 20% -54.3% 21562 ± 6% softirqs.CPU26.RCU 38741 ± 4% -68.7% 12137 softirqs.CPU26.SCHED 47334 ± 19% -53.5% 22033 ± 5% softirqs.CPU27.RCU 37660 ± 2% -67.0% 12423 softirqs.CPU27.SCHED 47299 ± 20% -54.0% 21766 ± 5% softirqs.CPU28.RCU 37332 ± 3% -67.2% 12241 softirqs.CPU28.SCHED 47252 ± 19% -53.2% 22096 ± 3% softirqs.CPU29.RCU 36340 ± 3% -64.2% 13012 ± 7% softirqs.CPU29.SCHED 46079 ± 19% -52.2% 22040 ± 5% softirqs.CPU3.RCU 36081 ± 3% -66.8% 11981 ± 3% softirqs.CPU3.SCHED 46128 ± 19% -53.4% 21478 ± 5% softirqs.CPU30.RCU 37966 ± 3% -67.8% 12220 softirqs.CPU30.SCHED 46648 ± 20% -53.3% 21769 ± 5% softirqs.CPU31.RCU 39054 ± 3% -68.8% 12191 softirqs.CPU31.SCHED 46406 ± 19% -53.0% 21798 ± 5% softirqs.CPU32.RCU 35941 ± 5% -65.7% 12312 softirqs.CPU32.SCHED 46142 ± 19% -53.4% 21496 ± 6% softirqs.CPU33.RCU 37769 -68.4% 11945 softirqs.CPU33.SCHED 46688 ± 21% -53.8% 21549 ± 6% softirqs.CPU34.RCU 37085 -66.5% 12407 softirqs.CPU34.SCHED 45861 ± 19% -53.6% 21288 ± 7% softirqs.CPU35.RCU 39444 ± 3% -68.7% 12347 softirqs.CPU35.SCHED 46303 ± 19% -53.6% 21498 ± 7% softirqs.CPU36.RCU 38769 ± 6% -68.4% 12257 softirqs.CPU36.SCHED 46266 ± 19% -53.5% 21520 ± 7% softirqs.CPU37.RCU 36319 ± 2% -66.2% 12274 ± 2% softirqs.CPU37.SCHED 46429 ± 19% -53.2% 21708 ± 7% softirqs.CPU38.RCU 37051 ± 4% -66.8% 12298 ± 2% softirqs.CPU38.SCHED 46126 ± 19% -54.1% 21189 ± 7% softirqs.CPU39.RCU 36870 ± 2% -66.9% 12209 ± 2% softirqs.CPU39.SCHED 46058 ± 19% -53.1% 21624 ± 7% softirqs.CPU4.RCU 35416 ± 3% -65.5% 12227 ± 2% softirqs.CPU4.SCHED 45775 ± 19% -53.0% 21526 ± 7% softirqs.CPU40.RCU 37513 ± 3% -66.6% 12536 ± 3% softirqs.CPU40.SCHED 46193 ± 20% -52.6% 21887 ± 7% softirqs.CPU41.RCU 38403 ± 4% -66.1% 13037 ± 6% softirqs.CPU41.SCHED 46133 ± 19% -53.5% 21447 ± 7% softirqs.CPU42.RCU 36496 ± 3% -66.5% 12212 softirqs.CPU42.SCHED 47125 ± 19% -54.1% 21635 ± 6% softirqs.CPU43.RCU 36632 ± 4% -67.0% 12081 softirqs.CPU43.SCHED 41737 ± 18% -55.2% 18683 ± 6% softirqs.CPU44.RCU 34625 -66.4% 11627 ± 2% softirqs.CPU44.SCHED 45313 ± 20% -50.3% 22541 ± 12% softirqs.CPU45.RCU 36386 ± 2% -66.7% 12102 ± 2% softirqs.CPU45.SCHED 45903 ± 20% -54.1% 21087 ± 7% softirqs.CPU46.RCU 35799 ± 2% -67.7% 11560 softirqs.CPU46.SCHED 45951 ± 20% -54.5% 20907 ± 7% softirqs.CPU47.RCU 36536 ± 3% -68.3% 11594 softirqs.CPU47.SCHED 46089 ± 19% -53.5% 21410 ± 6% softirqs.CPU48.RCU 35653 ± 3% -67.4% 11635 softirqs.CPU48.SCHED 45741 ± 19% -53.2% 21396 ± 6% softirqs.CPU49.RCU 34829 ± 4% -66.2% 11762 softirqs.CPU49.SCHED 46086 ± 18% -54.3% 21064 ± 7% softirqs.CPU5.RCU 34449 ± 2% -65.6% 11842 ± 3% softirqs.CPU5.SCHED 45940 ± 19% -52.4% 21851 ± 8% softirqs.CPU50.RCU 35692 ± 3% -67.1% 11741 ± 2% softirqs.CPU50.SCHED 45487 ± 20% -51.9% 21868 ± 8% softirqs.CPU51.RCU 35300 ± 3% -66.8% 11723 ± 2% softirqs.CPU51.SCHED 45672 ± 20% -52.4% 21749 ± 6% softirqs.CPU52.RCU 35529 ± 3% -67.0% 11707 softirqs.CPU52.SCHED 3817 ±199% -100.0% 0.00 softirqs.CPU53.NET_RX 46083 ± 19% -51.9% 22185 ± 8% softirqs.CPU53.RCU 36167 ± 2% -67.1% 11907 softirqs.CPU53.SCHED 45970 ± 19% -53.1% 21542 ± 9% softirqs.CPU54.RCU 35497 ± 2% -65.6% 12203 ± 3% softirqs.CPU54.SCHED 45545 ± 19% -50.8% 22411 ± 8% softirqs.CPU55.RCU 35768 ± 3% -65.9% 12210 ± 6% softirqs.CPU55.SCHED 46198 ± 19% -52.5% 21932 ± 8% softirqs.CPU56.RCU 34683 ± 4% -64.9% 12165 ± 2% softirqs.CPU56.SCHED 46091 ± 19% -53.1% 21599 ± 7% softirqs.CPU57.RCU 35670 ± 4% -65.8% 12189 ± 2% softirqs.CPU57.SCHED 45592 ± 20% -51.8% 21956 ± 8% softirqs.CPU58.RCU 36998 ± 4% -68.1% 11784 ± 2% softirqs.CPU58.SCHED 46021 ± 21% -53.2% 21531 ± 4% softirqs.CPU59.RCU 35196 ± 4% -66.5% 11775 ± 2% softirqs.CPU59.SCHED 45068 ± 19% -53.3% 21042 ± 6% softirqs.CPU6.RCU 36122 ± 2% -66.9% 11939 softirqs.CPU6.SCHED 48915 ± 18% -53.5% 22763 ± 5% softirqs.CPU60.RCU 35236 -66.5% 11791 softirqs.CPU60.SCHED 48732 ± 21% -53.9% 22476 ± 5% softirqs.CPU61.RCU 35706 -67.0% 11766 softirqs.CPU61.SCHED 48713 ± 20% -53.3% 22757 ± 4% softirqs.CPU62.RCU 38731 ± 4% -69.8% 11703 softirqs.CPU62.SCHED 48929 ± 19% -54.8% 22101 ± 5% softirqs.CPU63.RCU 37530 -69.0% 11633 softirqs.CPU63.SCHED 48680 ± 19% -54.3% 22238 ± 5% softirqs.CPU64.RCU 36365 ± 3% -67.5% 11807 ± 2% softirqs.CPU64.SCHED 49248 ± 19% -53.8% 22745 ± 4% softirqs.CPU65.RCU 35362 ± 3% -66.7% 11768 softirqs.CPU65.SCHED 48262 ± 19% -53.1% 22655 ± 5% softirqs.CPU66.RCU 34413 ± 4% -65.2% 11968 ± 2% softirqs.CPU66.SCHED 48134 ± 19% -54.1% 22102 ± 5% softirqs.CPU67.RCU 37361 ± 3% -68.4% 11823 ± 2% softirqs.CPU67.SCHED 49278 ± 15% -55.2% 22054 ± 6% softirqs.CPU68.RCU 38547 ± 2% -68.0% 12351 ± 2% softirqs.CPU68.SCHED 47844 ± 19% -53.9% 22060 ± 5% softirqs.CPU69.RCU 37131 ± 4% -67.1% 12207 ± 2% softirqs.CPU69.SCHED 45743 ± 19% -53.0% 21499 ± 6% softirqs.CPU7.RCU 36268 ± 4% -66.0% 12338 ± 4% softirqs.CPU7.SCHED 47705 ± 19% -53.7% 22092 ± 5% softirqs.CPU70.RCU 38058 ± 2% -67.7% 12287 ± 2% softirqs.CPU70.SCHED 47966 ± 19% -54.0% 22087 ± 5% softirqs.CPU71.RCU 37137 ± 3% -66.9% 12302 softirqs.CPU71.SCHED 48056 ± 19% -53.3% 22434 ± 7% softirqs.CPU72.RCU 36812 ± 3% -67.3% 12039 ± 3% softirqs.CPU72.SCHED 48322 ± 19% -54.0% 22214 ± 6% softirqs.CPU73.RCU 36387 -65.0% 12729 ± 5% softirqs.CPU73.SCHED 48144 ± 19% -54.4% 21970 ± 5% softirqs.CPU74.RCU 37509 -67.5% 12208 softirqs.CPU74.SCHED 44617 ± 19% -53.9% 20575 ± 7% softirqs.CPU75.RCU 39672 ± 4% -69.4% 12159 softirqs.CPU75.SCHED 45105 ± 18% -54.5% 20511 ± 7% softirqs.CPU76.RCU 36036 ± 4% -65.9% 12285 softirqs.CPU76.SCHED 44035 ± 19% -53.8% 20342 ± 7% softirqs.CPU77.RCU 38013 ± 2% -68.6% 11941 softirqs.CPU77.SCHED 44575 ± 20% -53.0% 20941 ± 9% softirqs.CPU78.RCU 36763 ± 3% -64.3% 13137 ± 10% softirqs.CPU78.SCHED 44307 ± 19% -54.1% 20316 ± 7% softirqs.CPU79.RCU 39343 ± 3% -68.7% 12310 softirqs.CPU79.SCHED 45928 ± 19% -53.5% 21354 ± 6% softirqs.CPU8.RCU 35468 ± 2% -66.0% 12045 softirqs.CPU8.SCHED 44238 ± 19% -53.1% 20745 ± 5% softirqs.CPU80.RCU 38836 ± 5% -68.7% 12146 softirqs.CPU80.SCHED 44625 ± 19% -53.6% 20721 ± 8% softirqs.CPU81.RCU 37847 ± 3% -67.9% 12133 softirqs.CPU81.SCHED 45531 ± 16% -54.8% 20597 ± 7% softirqs.CPU82.RCU 37100 ± 3% -67.2% 12152 ± 2% softirqs.CPU82.SCHED 44042 ± 19% -54.2% 20181 ± 7% softirqs.CPU83.RCU 36876 ± 3% -67.1% 12122 softirqs.CPU83.SCHED 44368 ± 19% -53.9% 20467 ± 7% softirqs.CPU84.RCU 38721 ± 3% -68.2% 12323 softirqs.CPU84.SCHED 44148 ± 19% -53.5% 20547 ± 7% softirqs.CPU85.RCU 39152 ± 5% -68.7% 12241 ± 2% softirqs.CPU85.SCHED 44320 ± 19% -52.4% 21103 ± 4% softirqs.CPU86.RCU 37838 ± 2% -67.3% 12385 softirqs.CPU86.SCHED 44683 ± 19% -54.0% 20538 ± 7% softirqs.CPU87.RCU 37516 ± 3% -68.2% 11929 softirqs.CPU87.SCHED 45602 ± 19% -53.1% 21384 ± 8% softirqs.CPU9.RCU 35944 ± 4% -67.0% 11844 softirqs.CPU9.SCHED 34402 ± 85% -86.8% 4541 ± 15% softirqs.NET_RX 4089617 ± 19% -53.3% 1910471 ± 6% softirqs.RCU 3234656 -66.9% 1071147 softirqs.SCHED 47699 ± 13% -41.3% 27976 ± 2% softirqs.TIMER 41.30 ± 10% -36.5 4.76 ± 8% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event 41.30 ± 2% -35.1 6.23 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 39.40 ± 10% -34.9 4.52 ± 8% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow 39.24 ± 10% -34.7 4.50 ± 8% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow 40.54 ± 2% -34.4 6.13 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 40.27 ± 2% -34.2 6.08 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 39.97 ± 2% -34.0 6.02 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 29.87 ± 3% -24.8 5.10 perf-profile.calltrace.cycles-pp.schedule.pipe_read.new_sync_read.vfs_read.ksys_read 29.70 ± 3% -24.6 5.07 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.new_sync_read.vfs_read 25.71 ± 10% -22.8 2.93 ± 8% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward 18.68 ± 10% -16.0 2.72 ± 9% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr 18.60 ± 10% -15.9 2.70 ± 9% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime 15.40 ± 2% -12.9 2.47 ± 7% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 15.52 ± 2% -12.9 2.65 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 12.47 ± 10% -12.5 0.00 perf-profile.calltrace.cycles-pp.perf_callchain_user.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward 12.20 ± 11% -12.2 0.00 perf-profile.calltrace.cycles-pp.asm_exc_page_fault.__get_user_nocheck_8.perf_callchain_user.get_perf_callchain.perf_callchain 13.58 ± 3% -11.4 2.23 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.new_sync_read 11.20 ± 10% -11.2 0.00 perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample 12.43 ± 10% -11.2 1.26 ± 8% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule 12.41 ± 10% -11.1 1.26 ± 8% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch.__schedule 12.90 ± 6% -11.1 1.82 perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 12.33 ± 10% -11.1 1.25 ± 8% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_switch 9.80 ± 10% -9.8 0.00 perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain 11.02 ± 9% -9.6 1.45 ± 6% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up 11.25 ± 7% -9.3 1.91 ± 4% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read 8.98 ± 10% -9.0 0.00 perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample 10.18 ± 10% -8.9 1.29 ± 8% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair.ttwu_do_activate 56.59 -8.7 47.85 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 10.35 ± 9% -8.6 1.71 ± 6% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule 9.82 ± 10% -8.6 1.26 ± 9% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity.enqueue_task_fair 56.83 -8.4 48.45 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write 9.59 ± 10% -8.4 1.24 ± 9% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.enqueue_entity 9.75 ± 9% -8.2 1.59 ± 7% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 8.08 ± 4% -8.1 0.00 perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 9.62 ± 10% -8.1 1.55 ± 8% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule 9.43 ± 9% -7.9 1.53 ± 8% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common 9.39 ± 10% -7.9 1.53 ± 9% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair 7.72 ± 5% -7.7 0.00 perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe 9.14 ± 10% -7.6 1.49 ± 9% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity 7.63 ± 5% -7.6 0.00 perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64 8.96 ± 10% -7.5 1.45 ± 9% perf-profile.calltrace.cycles-pp.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function 8.95 ± 10% -7.5 1.45 ± 9% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up 8.92 ± 10% -7.5 1.44 ± 9% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_swevent_overflow.perf_tp_event.perf_trace_sched_wakeup_template 8.21 ± 4% -7.5 0.74 ± 3% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 8.64 ± 10% -7.3 1.35 ± 8% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.pipe_read.new_sync_read 8.30 ± 10% -7.0 1.30 ± 8% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.pipe_read 5.93 ± 11% -5.9 0.00 perf-profile.calltrace.cycles-pp.__get_user_nocheck_8.perf_callchain_user.get_perf_callchain.perf_callchain.perf_prepare_sample 46.41 -5.8 40.61 perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 6.56 ± 2% -5.7 0.83 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 6.23 ± 2% -5.4 0.78 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 58.19 -5.4 52.80 perf-profile.calltrace.cycles-pp.__libc_write 46.94 -4.7 42.23 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.53 ± 3% -2.9 0.62 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_read.new_sync_read 3.34 ± 3% -2.8 0.58 ± 4% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.pipe_read 47.65 -2.7 44.93 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 35.69 -2.3 33.35 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 48.12 -1.7 46.42 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 36.04 -1.6 34.43 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.20 ± 6% -0.4 0.78 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 0.77 ± 7% +0.4 1.19 perf-profile.calltrace.cycles-pp.schedule.pipe_write.new_sync_write.vfs_write.ksys_write 0.76 ± 7% +0.4 1.19 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_write.new_sync_write.vfs_write 0.00 +0.5 0.54 perf-profile.calltrace.cycles-pp.current_time.file_update_time.pipe_write.new_sync_write.vfs_write 0.00 +0.5 0.54 perf-profile.calltrace.cycles-pp.make_kgid.atime_needs_update.touch_atime.pipe_read.new_sync_read 0.00 +0.6 0.56 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_write.new_sync_write 0.00 +0.6 0.59 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_read 0.00 +0.6 0.61 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_write 0.84 ± 5% +0.6 1.48 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_read 0.85 ± 5% +0.6 1.49 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_read.new_sync_read 0.86 ± 5% +0.7 1.51 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_read.new_sync_read.vfs_read 0.00 +0.7 0.67 perf-profile.calltrace.cycles-pp.__might_fault.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 0.00 +0.7 0.70 ± 2% perf-profile.calltrace.cycles-pp.current_time.atime_needs_update.touch_atime.pipe_read.new_sync_read 0.90 ± 5% +0.7 1.63 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +0.8 0.75 ± 15% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 0.00 +0.8 0.83 ± 5% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +0.9 0.87 ± 5% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 0.00 +1.0 0.98 ± 17% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 0.00 +1.0 0.98 ± 17% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.00 +1.0 0.99 ± 5% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.0 1.05 ± 4% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write 0.00 +1.1 1.08 ± 5% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_charge.obj_cgroup_charge_pages.__memcg_kmem_charge_page.__alloc_pages 0.00 +1.1 1.15 perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.new_sync_write.vfs_write.ksys_write 0.00 +1.3 1.27 ± 2% perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.00 +1.3 1.29 perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +1.3 1.30 ± 3% perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64 37.34 +1.4 38.78 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 38.77 +1.5 40.25 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read 0.00 +1.5 1.48 ± 18% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.00 +1.5 1.49 ± 17% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.00 +1.5 1.49 ± 17% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 0.00 +1.5 1.50 ± 17% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 0.00 +1.6 1.65 ± 2% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.31 ± 81% +1.7 2.02 ± 3% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.71 ± 14% +1.8 2.48 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read 39.03 +1.9 40.89 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read 0.76 ± 14% +2.0 2.74 ± 2% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 0.00 +2.0 2.02 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write 0.24 ±122% +2.1 2.38 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.new_sync_read.vfs_read.ksys_read 0.33 ± 82% +2.2 2.50 perf-profile.calltrace.cycles-pp.atime_needs_update.touch_atime.pipe_read.new_sync_read.vfs_read 0.11 ±200% +2.2 2.31 perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 0.65 ± 11% +2.2 2.89 perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +2.3 2.33 ± 3% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.pipe_write.new_sync_write.vfs_write 0.00 +2.9 2.86 perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_read 0.00 +2.9 2.92 perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_write 1.14 ± 13% +3.1 4.27 perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 0.00 +3.9 3.93 ± 4% perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.pipe_write.new_sync_write.vfs_write 1.23 ± 14% +4.0 5.19 perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 0.00 +4.1 4.14 ± 5% perf-profile.calltrace.cycles-pp.page_counter_charge.obj_cgroup_charge_pages.__memcg_kmem_charge_page.__alloc_pages.pipe_write 40.97 +4.4 45.33 perf-profile.calltrace.cycles-pp.__libc_read 0.00 +4.5 4.45 ± 5% perf-profile.calltrace.cycles-pp.obj_cgroup_charge_pages.__memcg_kmem_charge_page.__alloc_pages.pipe_write.new_sync_write 0.00 +4.5 4.51 ± 5% perf-profile.calltrace.cycles-pp.__memcg_kmem_charge_page.__alloc_pages.pipe_write.new_sync_write.vfs_write 0.38 ± 83% +4.6 4.96 perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write 1.04 ± 15% +4.6 5.63 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 0.00 +4.7 4.67 ± 5% perf-profile.calltrace.cycles-pp.__alloc_pages.pipe_write.new_sync_write.vfs_write.ksys_write 0.00 +5.3 5.30 ± 4% perf-profile.calltrace.cycles-pp.propagate_protected_usage.page_counter_uncharge.uncharge_batch.__mem_cgroup_uncharge.__put_page 0.00 +6.2 6.24 ± 4% perf-profile.calltrace.cycles-pp.page_counter_cancel.page_counter_uncharge.uncharge_batch.__mem_cgroup_uncharge.__put_page 0.00 +7.4 7.43 ± 3% perf-profile.calltrace.cycles-pp.__mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 0.00 +11.7 11.66 ± 4% perf-profile.calltrace.cycles-pp.page_counter_uncharge.uncharge_batch.__mem_cgroup_uncharge.__put_page.pipe_read 0.00 +11.8 11.83 ± 4% perf-profile.calltrace.cycles-pp.uncharge_batch.__mem_cgroup_uncharge.__put_page.pipe_read.new_sync_read 0.00 +11.9 11.92 ± 4% perf-profile.calltrace.cycles-pp.__mem_cgroup_uncharge.__put_page.pipe_read.new_sync_read.vfs_read 0.00 +11.9 11.93 ± 4% perf-profile.calltrace.cycles-pp.__put_page.pipe_read.new_sync_read.vfs_read.ksys_read 45.68 ± 10% -37.7 8.00 ± 8% perf-profile.children.cycles-pp.perf_tp_event 44.29 ± 10% -36.5 7.77 ± 8% perf-profile.children.cycles-pp.perf_swevent_overflow 44.20 ± 10% -36.4 7.76 ± 8% perf-profile.children.cycles-pp.__perf_event_overflow 44.00 ± 10% -36.3 7.72 ± 8% perf-profile.children.cycles-pp.perf_event_output_forward 42.94 ± 10% -36.1 6.82 ± 8% perf-profile.children.cycles-pp.perf_prepare_sample 40.93 ± 10% -34.5 6.46 ± 8% perf-profile.children.cycles-pp.perf_callchain 42.22 ± 2% -34.4 7.86 perf-profile.children.cycles-pp.__wake_up_common_lock 40.75 ± 10% -34.3 6.43 ± 8% perf-profile.children.cycles-pp.get_perf_callchain 41.40 ± 2% -33.8 7.64 perf-profile.children.cycles-pp.__wake_up_common 41.14 ± 2% -33.6 7.56 perf-profile.children.cycles-pp.autoremove_wake_function 40.93 ± 2% -33.2 7.74 perf-profile.children.cycles-pp.try_to_wake_up 38.81 ± 3% -32.3 6.49 perf-profile.children.cycles-pp.schedule 38.78 ± 3% -31.9 6.88 perf-profile.children.cycles-pp.__schedule 26.99 ± 10% -22.7 4.27 ± 8% perf-profile.children.cycles-pp.perf_callchain_kernel 23.11 ± 8% -18.5 4.59 ± 6% perf-profile.children.cycles-pp.update_curr 20.31 ± 10% -17.1 3.21 ± 8% perf-profile.children.cycles-pp.unwind_next_frame 20.72 ± 10% -16.7 3.98 ± 8% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime 16.01 ± 2% -13.2 2.77 ± 2% perf-profile.children.cycles-pp.ttwu_do_activate 15.89 ± 2% -13.1 2.77 perf-profile.children.cycles-pp.enqueue_task_fair 13.84 ± 10% -11.9 1.91 ± 8% perf-profile.children.cycles-pp.perf_trace_sched_switch 14.00 ± 3% -11.1 2.86 perf-profile.children.cycles-pp.dequeue_task_fair 13.33 ± 6% -11.0 2.37 perf-profile.children.cycles-pp.enqueue_entity 12.97 ± 10% -10.9 2.03 ± 8% perf-profile.children.cycles-pp.perf_callchain_user 12.19 ± 10% -10.3 1.88 ± 8% perf-profile.children.cycles-pp.__get_user_nocheck_8 11.78 ± 10% -10.0 1.82 ± 8% perf-profile.children.cycles-pp.__unwind_start 11.36 ± 2% -9.3 2.03 ± 3% perf-profile.children.cycles-pp._raw_spin_lock 11.63 ± 7% -9.2 2.46 ± 3% perf-profile.children.cycles-pp.dequeue_entity 10.65 ± 2% -8.4 2.25 ± 3% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 9.19 ± 3% -8.2 0.98 ± 3% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 10.04 ± 9% -8.0 2.03 ± 7% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template 9.44 ± 3% -7.8 1.66 ± 2% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 95.44 -7.1 88.29 perf-profile.children.cycles-pp.do_syscall_64 8.21 ± 10% -7.0 1.21 ± 8% perf-profile.children.cycles-pp.asm_exc_page_fault 95.92 -6.4 89.48 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 46.45 -5.7 40.70 perf-profile.children.cycles-pp.pipe_write 58.45 -5.6 52.85 perf-profile.children.cycles-pp.__libc_write 46.95 -4.7 42.29 perf-profile.children.cycles-pp.new_sync_write 5.24 ± 10% -4.4 0.85 ± 7% perf-profile.children.cycles-pp.__orc_find 3.79 ± 9% -3.2 0.59 ± 8% perf-profile.children.cycles-pp.unwind_get_return_address 47.67 -2.7 44.99 perf-profile.children.cycles-pp.vfs_write 3.10 ± 7% -2.7 0.42 perf-profile.children.cycles-pp.set_task_cpu 3.08 ± 9% -2.6 0.47 ± 9% perf-profile.children.cycles-pp.__kernel_text_address 3.05 ± 9% -2.6 0.49 ± 7% perf-profile.children.cycles-pp.orc_find 2.64 ± 10% -2.3 0.34 ± 8% perf-profile.children.cycles-pp.perf_trace_sched_migrate_task 35.75 -2.3 33.47 perf-profile.children.cycles-pp.pipe_read 2.92 ± 10% -2.3 0.66 ± 19% perf-profile.children.cycles-pp.update_load_avg 2.82 ± 9% -2.2 0.58 ± 15% perf-profile.children.cycles-pp.select_task_rq_fair 2.56 ± 10% -2.2 0.36 ± 7% perf-profile.children.cycles-pp.exc_page_fault 2.58 ± 9% -2.2 0.40 ± 10% perf-profile.children.cycles-pp.kernel_text_address 2.28 ± 10% -1.9 0.34 ± 10% perf-profile.children.cycles-pp.stack_access_ok 2.09 ± 10% -1.8 0.26 ± 9% perf-profile.children.cycles-pp.kernelmode_fixup_or_oops 2.12 ± 9% -1.8 0.34 ± 6% perf-profile.children.cycles-pp.native_irq_return_iret 2.15 ± 7% -1.8 0.39 ± 12% perf-profile.children.cycles-pp.pick_next_task_fair 2.10 ± 33% -1.7 0.38 ± 29% perf-profile.children.cycles-pp.update_cfs_group 48.13 -1.7 46.45 perf-profile.children.cycles-pp.ksys_write 1.90 ± 10% -1.7 0.23 ± 10% perf-profile.children.cycles-pp.fixup_exception 2.01 ± 9% -1.7 0.36 ± 17% perf-profile.children.cycles-pp.select_idle_sibling 36.06 -1.6 34.51 perf-profile.children.cycles-pp.new_sync_read 1.74 ± 10% -1.5 0.21 ± 10% perf-profile.children.cycles-pp.search_exception_tables 1.70 ± 10% -1.5 0.21 ± 12% perf-profile.children.cycles-pp.search_extable 1.62 ± 10% -1.4 0.19 ± 11% perf-profile.children.cycles-pp.bsearch 1.48 ± 10% -1.2 0.27 ± 12% perf-profile.children.cycles-pp.switch_mm_irqs_off 1.22 ± 8% -1.0 0.23 ± 15% perf-profile.children.cycles-pp.select_idle_cpu 1.05 ± 10% -0.9 0.12 ± 7% perf-profile.children.cycles-pp.cmp_ex_search 1.07 ± 10% -0.9 0.18 ± 9% perf-profile.children.cycles-pp.__perf_event_header__init_id 0.96 ± 9% -0.8 0.13 ± 16% perf-profile.children.cycles-pp.load_new_mm_cr3 1.03 ± 8% -0.8 0.21 ± 14% perf-profile.children.cycles-pp.reweight_entity 0.85 ± 7% -0.7 0.14 ± 8% perf-profile.children.cycles-pp.switch_fpu_return 0.88 ± 7% -0.7 0.18 ± 18% perf-profile.children.cycles-pp.set_next_entity 0.82 ± 8% -0.7 0.12 ± 7% perf-profile.children.cycles-pp.core_kernel_text 0.76 ± 9% -0.6 0.16 ± 19% perf-profile.children.cycles-pp.available_idle_cpu 0.72 -0.6 0.12 ± 6% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.71 ± 7% -0.6 0.12 ± 14% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.68 ± 8% -0.6 0.11 ± 12% perf-profile.children.cycles-pp.check_preempt_curr 0.67 ± 10% -0.6 0.11 ± 8% perf-profile.children.cycles-pp.perf_output_begin_forward 0.68 ± 8% -0.6 0.13 ± 14% perf-profile.children.cycles-pp.__switch_to_asm 0.63 ± 10% -0.5 0.11 ± 7% perf-profile.children.cycles-pp.ftrace_graph_ret_addr 0.59 ± 9% -0.5 0.11 ± 7% perf-profile.children.cycles-pp.perf_event_pid_type 0.63 ± 7% -0.5 0.16 ± 11% perf-profile.children.cycles-pp.__update_load_avg_se 0.50 ± 7% -0.5 0.04 ± 71% perf-profile.children.cycles-pp.check_preempt_wakeup 0.55 ± 7% -0.5 0.09 ± 9% perf-profile.children.cycles-pp.restore_fpregs_from_fpstate 0.57 ± 7% -0.5 0.11 ± 16% perf-profile.children.cycles-pp.__switch_to 0.59 ± 9% -0.5 0.13 ± 16% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.60 ± 11% -0.4 0.17 ± 14% perf-profile.children.cycles-pp.update_rq_clock 0.54 ± 9% -0.4 0.10 ± 8% perf-profile.children.cycles-pp.__enqueue_entity 0.55 ± 3% -0.4 0.13 ± 7% perf-profile.children.cycles-pp.cpuacct_charge 0.48 ± 10% -0.4 0.06 ± 7% perf-profile.children.cycles-pp.update_min_vruntime 0.52 -0.4 0.12 ± 4% perf-profile.children.cycles-pp.sched_clock_cpu 0.59 ± 4% -0.4 0.23 ± 4% perf-profile.children.cycles-pp.finish_task_switch 0.43 ± 11% -0.4 0.07 ± 7% perf-profile.children.cycles-pp.bad_get_user 0.45 -0.4 0.10 ± 4% perf-profile.children.cycles-pp.native_sched_clock 0.43 ± 8% -0.3 0.09 ± 5% perf-profile.children.cycles-pp.__task_pid_nr_ns 0.37 ± 9% -0.3 0.04 ± 71% perf-profile.children.cycles-pp.save_fpregs_to_fpstate 0.35 ± 11% -0.3 0.03 ± 70% perf-profile.children.cycles-pp.perf_tp_event_match 0.37 ± 9% -0.3 0.06 ± 13% perf-profile.children.cycles-pp.error_entry 0.34 ± 5% -0.3 0.04 ± 70% perf-profile.children.cycles-pp.is_bpf_text_address 0.42 ± 9% -0.3 0.17 ± 5% perf-profile.children.cycles-pp.__list_del_entry_valid 0.30 ± 5% -0.2 0.06 ± 7% perf-profile.children.cycles-pp.perf_trace_buf_update 0.26 ± 11% -0.2 0.03 ± 70% perf-profile.children.cycles-pp.get_stack_info 0.28 ± 14% -0.2 0.07 ± 6% perf-profile.children.cycles-pp.update_cfs_rq_h_load 0.27 ± 10% -0.2 0.08 ± 11% perf-profile.children.cycles-pp.kvm_is_in_guest 0.88 ± 13% -0.2 0.71 ± 7% perf-profile.children.cycles-pp.prepare_to_wait_event 1.01 ± 12% -0.1 0.86 ± 7% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.26 ± 9% -0.1 0.15 ± 14% perf-profile.children.cycles-pp.__calc_delta 0.15 ± 16% -0.1 0.06 ± 8% perf-profile.children.cycles-pp.perf_misc_flags 0.10 -0.0 0.07 perf-profile.children.cycles-pp.generic_update_time 0.09 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.__mark_inode_dirty 0.05 ± 53% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.irq_exit_rcu 0.25 ± 13% +0.0 0.30 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.02 ±123% +0.1 0.08 ± 14% perf-profile.children.cycles-pp.wake_q_add 0.01 ±200% +0.1 0.07 ± 6% perf-profile.children.cycles-pp.__softirqentry_text_start 0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.newidle_balance 0.00 +0.1 0.07 ± 20% perf-profile.children.cycles-pp.shmem_add_to_page_cache 0.00 +0.1 0.07 ± 6% [email protected] 0.00 +0.1 0.08 ± 22% perf-profile.children.cycles-pp.perf_output_end 0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.uncharge_folio 0.00 +0.1 0.09 ± 9% perf-profile.children.cycles-pp.get_mem_cgroup_from_objcg 0.00 +0.1 0.09 perf-profile.children.cycles-pp.free_unref_page 0.00 +0.1 0.09 ± 5% perf-profile.children.cycles-pp.load_balance 0.02 ±122% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.schedule_preempt_disabled 0.06 ± 10% +0.1 0.16 ± 5% perf-profile.children.cycles-pp.__x64_sys_read 0.00 +0.1 0.10 ± 4% perf-profile.children.cycles-pp.rmqueue 0.08 ± 7% +0.1 0.19 ± 6% perf-profile.children.cycles-pp.__x64_sys_write 0.01 ±200% +0.1 0.12 ± 16% perf-profile.children.cycles-pp.copy_page_from_iter_atomic 0.67 ± 2% +0.1 0.79 ± 4% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.03 ±123% +0.1 0.14 ± 33% perf-profile.children.cycles-pp.ordered_events__queue 0.42 ± 6% +0.1 0.54 ± 5% perf-profile.children.cycles-pp.scheduler_tick 0.05 ± 90% +0.1 0.17 ± 29% perf-profile.children.cycles-pp.poll_idle 0.02 ±123% +0.1 0.14 ± 33% perf-profile.children.cycles-pp.queue_event 0.66 ± 2% +0.1 0.78 ± 4% perf-profile.children.cycles-pp.hrtimer_interrupt 0.03 ±123% +0.1 0.15 ± 31% perf-profile.children.cycles-pp.process_simple 0.00 +0.1 0.13 ± 3% perf-profile.children.cycles-pp.get_page_from_freelist 0.03 ±122% +0.1 0.16 ± 29% perf-profile.children.cycles-pp.record__finish_output 0.03 ±122% +0.1 0.16 ± 29% perf-profile.children.cycles-pp.perf_session__process_events 0.03 ±122% +0.1 0.17 ± 26% perf-profile.children.cycles-pp.cmd_record 0.37 ± 7% +0.1 0.50 ± 6% perf-profile.children.cycles-pp.task_tick_fair 0.00 +0.1 0.14 ± 23% perf-profile.children.cycles-pp.shmem_write_begin 0.00 +0.1 0.14 ± 23% perf-profile.children.cycles-pp.shmem_getpage_gfp 0.12 ± 32% +0.2 0.27 ± 12% perf-profile.children.cycles-pp.wake_up_q 0.49 ± 5% +0.2 0.64 ± 5% perf-profile.children.cycles-pp.tick_sched_timer 0.74 ± 3% +0.2 0.89 ± 4% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.53 ± 5% +0.2 0.69 ± 5% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.47 ± 5% +0.2 0.63 ± 5% perf-profile.children.cycles-pp.update_process_times 0.47 ± 5% +0.2 0.63 ± 6% perf-profile.children.cycles-pp.tick_sched_handle 0.08 ± 6% +0.2 0.25 ± 3% perf-profile.children.cycles-pp.kill_fasync 0.00 +0.2 0.18 ± 2% perf-profile.children.cycles-pp.osq_unlock 0.00 +0.2 0.18 perf-profile.children.cycles-pp.rw_verify_area 0.07 ± 18% +0.2 0.25 ± 3% perf-profile.children.cycles-pp.iov_iter_init 0.00 +0.2 0.21 ± 3% perf-profile.children.cycles-pp.page_counter_try_charge 0.07 ± 16% +0.2 0.28 ± 3% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64 0.79 ± 3% +0.2 1.00 ± 3% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.09 ± 5% +0.2 0.32 ± 2% perf-profile.children.cycles-pp.apparmor_file_permission 0.11 ± 10% +0.2 0.36 ± 18% perf-profile.children.cycles-pp.generic_file_write_iter 0.11 ± 10% +0.2 0.36 ± 18% perf-profile.children.cycles-pp.__generic_file_write_iter 0.11 ± 10% +0.2 0.36 ± 18% perf-profile.children.cycles-pp.generic_perform_write 0.14 ± 10% +0.3 0.39 perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.09 ± 15% +0.3 0.35 ± 17% perf-profile.children.cycles-pp.memcpy_erms 0.00 +0.3 0.26 ± 3% perf-profile.children.cycles-pp.try_charge_memcg 0.13 ± 6% +0.3 0.41 ± 3% perf-profile.children.cycles-pp.rcu_all_qs 0.10 ± 17% +0.3 0.40 ± 19% perf-profile.children.cycles-pp.perf_output_copy 0.23 ± 33% +0.3 0.53 ± 12% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 0.09 ± 93% +0.3 0.39 ± 16% perf-profile.children.cycles-pp.schedule_idle 0.09 ± 15% +0.3 0.41 ± 3% perf-profile.children.cycles-pp.syscall_enter_from_user_mode 0.09 ± 15% +0.4 0.47 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 0.08 ± 17% +0.4 0.50 ± 5% perf-profile.children.cycles-pp.make_kuid 0.12 ± 8% +0.4 0.54 ± 2% perf-profile.children.cycles-pp.timestamp_truncate 0.38 ± 10% +0.4 0.82 perf-profile.children.cycles-pp.__cond_resched 0.12 ± 19% +0.5 0.62 ± 3% perf-profile.children.cycles-pp.aa_file_perm 0.09 ± 15% +0.5 0.60 ± 2% perf-profile.children.cycles-pp.make_kgid 0.16 ± 15% +0.5 0.67 ± 18% perf-profile.children.cycles-pp.perf_output_sample 0.03 ±127% +0.8 0.80 ± 13% perf-profile.children.cycles-pp.intel_idle 0.37 ± 8% +0.8 1.17 perf-profile.children.cycles-pp.file_update_time 0.15 ± 14% +0.8 0.97 perf-profile.children.cycles-pp.map_id_range_down 0.09 ± 92% +0.9 0.99 ± 16% perf-profile.children.cycles-pp.cpuidle_enter 0.09 ± 92% +0.9 0.99 ± 16% perf-profile.children.cycles-pp.cpuidle_enter_state 0.48 ± 13% +0.9 1.41 ± 2% perf-profile.children.cycles-pp.__might_sleep 0.22 ± 14% +0.9 1.16 perf-profile.children.cycles-pp.__might_fault 0.29 ± 13% +1.1 1.36 perf-profile.children.cycles-pp.current_time 0.58 ± 13% +1.2 1.82 ± 5% perf-profile.children.cycles-pp.__fget_light 0.22 ± 67% +1.3 1.49 ± 17% perf-profile.children.cycles-pp.start_secondary 0.22 ± 67% +1.3 1.50 ± 17% perf-profile.children.cycles-pp.secondary_startup_64_no_verify 0.22 ± 67% +1.3 1.50 ± 17% perf-profile.children.cycles-pp.cpu_startup_entry 0.22 ± 67% +1.3 1.50 ± 17% perf-profile.children.cycles-pp.do_idle 0.63 ± 14% +1.4 2.07 ± 4% perf-profile.children.cycles-pp.__fdget_pos 37.35 +1.5 38.82 perf-profile.children.cycles-pp.ksys_read 0.66 ± 12% +1.8 2.42 perf-profile.children.cycles-pp.__might_resched 0.69 ± 9% +2.0 2.64 ± 2% perf-profile.children.cycles-pp.common_file_perm 0.49 ± 14% +2.0 2.45 perf-profile.children.cycles-pp.copyin 0.77 ± 14% +2.0 2.77 ± 2% perf-profile.children.cycles-pp.copyout 0.51 ± 12% +2.0 2.55 perf-profile.children.cycles-pp.atime_needs_update 0.66 ± 11% +2.3 2.91 perf-profile.children.cycles-pp.touch_atime 0.00 +2.4 2.44 ± 3% perf-profile.children.cycles-pp.osq_lock 0.68 ± 14% +2.6 3.33 perf-profile.children.cycles-pp.syscall_return_via_sysret 0.89 ± 9% +2.8 3.71 ± 2% perf-profile.children.cycles-pp.security_file_permission 0.68 ± 14% +3.0 3.64 perf-profile.children.cycles-pp.__entry_text_start 1.17 ± 13% +3.2 4.38 perf-profile.children.cycles-pp.copy_page_from_iter 1.20 ± 14% +3.8 4.96 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.00 +4.0 4.00 ± 4% perf-profile.children.cycles-pp.mutex_spin_on_owner 1.25 ± 13% +4.0 5.28 perf-profile.children.cycles-pp.copy_page_to_iter 0.00 +4.2 4.15 ± 5% perf-profile.children.cycles-pp.page_counter_charge 41.21 +4.2 45.37 perf-profile.children.cycles-pp.__libc_read 0.00 +4.5 4.46 ± 5% perf-profile.children.cycles-pp.obj_cgroup_charge_pages 0.00 +4.5 4.51 ± 5% perf-profile.children.cycles-pp.__memcg_kmem_charge_page 0.01 ±200% +4.7 4.70 ± 5% perf-profile.children.cycles-pp.__alloc_pages 0.79 ± 20% +5.5 6.31 perf-profile.children.cycles-pp.mutex_unlock 0.01 ±200% +6.2 6.25 ± 4% perf-profile.children.cycles-pp.page_counter_cancel 0.00 +6.5 6.52 ± 4% perf-profile.children.cycles-pp.propagate_protected_usage 1.59 ± 15% +6.6 8.15 perf-profile.children.cycles-pp.mutex_lock 0.16 ± 35% +7.7 7.83 ± 2% perf-profile.children.cycles-pp.__mutex_lock 0.02 ±125% +11.6 11.67 ± 4% perf-profile.children.cycles-pp.page_counter_uncharge 0.03 ±127% +11.8 11.83 ± 4% perf-profile.children.cycles-pp.uncharge_batch 0.03 ±127% +11.9 11.92 ± 4% perf-profile.children.cycles-pp.__mem_cgroup_uncharge 0.03 ±127% +11.9 11.93 ± 4% perf-profile.children.cycles-pp.__put_page 10.64 ± 2% -8.4 2.25 ± 3% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 9.56 ± 10% -8.1 1.48 ± 8% perf-profile.self.cycles-pp.unwind_next_frame 6.92 ± 10% -5.8 1.13 ± 8% perf-profile.self.cycles-pp.__get_user_nocheck_8 5.22 ± 10% -4.4 0.85 ± 7% perf-profile.self.cycles-pp.__orc_find 2.87 ± 9% -2.4 0.47 ± 7% perf-profile.self.cycles-pp.orc_find 2.11 ± 9% -1.8 0.34 ± 7% perf-profile.self.cycles-pp.native_irq_return_iret 2.09 ± 33% -1.7 0.38 ± 29% perf-profile.self.cycles-pp.update_cfs_group 1.93 ± 10% -1.6 0.29 ± 10% perf-profile.self.cycles-pp.stack_access_ok 1.95 ± 11% -1.6 0.32 ± 8% perf-profile.self.cycles-pp.perf_callchain_kernel 1.59 ± 12% -1.2 0.35 ± 23% perf-profile.self.cycles-pp.update_load_avg 1.49 ± 7% -1.2 0.32 ± 14% perf-profile.self.cycles-pp.__schedule 1.08 ± 10% -0.9 0.17 ± 11% perf-profile.self.cycles-pp.__unwind_start 1.02 ± 10% -0.9 0.12 ± 11% perf-profile.self.cycles-pp.cmp_ex_search 1.02 ± 11% -0.9 0.16 ± 10% perf-profile.self.cycles-pp.kernel_text_address 0.96 ± 9% -0.8 0.13 ± 16% perf-profile.self.cycles-pp.load_new_mm_cr3 1.07 ± 7% -0.8 0.26 ± 12% perf-profile.self.cycles-pp.update_curr 0.75 ± 9% -0.6 0.16 ± 19% perf-profile.self.cycles-pp.available_idle_cpu 0.71 ± 8% -0.6 0.12 ± 3% perf-profile.self.cycles-pp.perf_tp_event 0.69 ± 10% -0.6 0.13 ± 9% perf-profile.self.cycles-pp.perf_callchain_user 0.67 ± 11% -0.6 0.11 ± 4% perf-profile.self.cycles-pp.unwind_get_return_address 0.96 ± 11% -0.5 0.41 ± 7% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.68 ± 8% -0.5 0.13 ± 14% perf-profile.self.cycles-pp.__switch_to_asm 0.63 ± 10% -0.5 0.10 ± 8% perf-profile.self.cycles-pp.perf_output_begin_forward 0.75 ± 11% -0.5 0.23 ± 9% perf-profile.self.cycles-pp._raw_spin_lock 0.59 ± 11% -0.5 0.07 ± 12% perf-profile.self.cycles-pp.bsearch 0.56 ± 11% -0.5 0.09 ± 14% perf-profile.self.cycles-pp.get_perf_callchain 0.62 ± 7% -0.5 0.15 ± 12% perf-profile.self.cycles-pp.__update_load_avg_se 0.55 ± 7% -0.5 0.09 ± 9% perf-profile.self.cycles-pp.restore_fpregs_from_fpstate 0.54 ± 12% -0.5 0.08 ± 11% perf-profile.self.cycles-pp.perf_prepare_sample 0.57 ± 8% -0.4 0.13 ± 13% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.55 ± 7% -0.4 0.11 ± 16% perf-profile.self.cycles-pp.__switch_to 0.49 ± 6% -0.4 0.06 perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime 0.53 ± 10% -0.4 0.10 ± 8% perf-profile.self.cycles-pp.__enqueue_entity 0.51 ± 8% -0.4 0.08 ± 14% perf-profile.self.cycles-pp.select_idle_cpu 0.54 ± 3% -0.4 0.12 ± 3% perf-profile.self.cycles-pp.cpuacct_charge 0.48 ± 10% -0.4 0.07 ± 11% perf-profile.self.cycles-pp.perf_trace_sched_switch 0.50 ± 10% -0.4 0.09 ± 9% perf-profile.self.cycles-pp.ftrace_graph_ret_addr 0.46 ± 10% -0.4 0.06 ± 7% perf-profile.self.cycles-pp.update_min_vruntime 0.47 ± 8% -0.4 0.07 ± 6% perf-profile.self.cycles-pp.pick_next_task_fair 0.45 ± 10% -0.4 0.07 ± 11% perf-profile.self.cycles-pp.__kernel_text_address 0.45 ± 9% -0.4 0.07 ± 11% perf-profile.self.cycles-pp.core_kernel_text 0.87 ± 5% -0.4 0.50 perf-profile.self.cycles-pp.try_to_wake_up 0.41 ± 9% -0.4 0.04 ± 71% perf-profile.self.cycles-pp.reweight_entity 0.46 ± 8% -0.4 0.10 ± 17% perf-profile.self.cycles-pp.enqueue_entity 0.45 ± 9% -0.4 0.10 ± 16% perf-profile.self.cycles-pp.select_task_rq_fair 0.47 ± 13% -0.3 0.13 ± 10% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.44 -0.3 0.10 ± 4% perf-profile.self.cycles-pp.native_sched_clock 0.42 ± 9% -0.3 0.08 ± 12% perf-profile.self.cycles-pp.select_idle_sibling 0.42 ± 8% -0.3 0.09 ± 5% perf-profile.self.cycles-pp.__task_pid_nr_ns 0.37 ± 9% -0.3 0.04 ± 71% perf-profile.self.cycles-pp.save_fpregs_to_fpstate 0.41 ± 10% -0.3 0.09 ± 24% perf-profile.self.cycles-pp.enqueue_task_fair 0.43 ± 12% -0.3 0.12 ± 11% perf-profile.self.cycles-pp.update_rq_clock 0.34 ± 12% -0.3 0.04 ± 71% perf-profile.self.cycles-pp.asm_exc_page_fault 0.32 ± 9% -0.3 0.03 ± 70% perf-profile.self.cycles-pp.error_entry 0.29 ± 9% -0.3 0.04 ± 71% perf-profile.self.cycles-pp.switch_fpu_return 0.37 ± 12% -0.2 0.12 ± 11% perf-profile.self.cycles-pp.prepare_to_wait_event 0.38 ± 10% -0.2 0.16 ± 6% perf-profile.self.cycles-pp.__list_del_entry_valid 0.29 ± 9% -0.2 0.08 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair 0.28 ± 14% -0.2 0.07 ± 6% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.26 ± 12% -0.2 0.07 ± 14% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template 0.26 ± 8% -0.2 0.08 ± 12% perf-profile.self.cycles-pp.__wake_up_common 0.23 ± 9% -0.2 0.07 ± 7% perf-profile.self.cycles-pp.perf_event_output_forward 0.23 ± 10% -0.2 0.08 ± 6% perf-profile.self.cycles-pp.kvm_is_in_guest 0.26 ± 10% -0.1 0.15 ± 16% perf-profile.self.cycles-pp.__calc_delta 0.09 ± 5% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.__mark_inode_dirty 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.__memcg_kmem_charge_page 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.make_kuid 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.try_charge_memcg 0.02 ±123% +0.1 0.08 ± 10% perf-profile.self.cycles-pp.wake_q_add 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.make_kgid 0.00 +0.1 0.07 ± 6% [email protected] 0.04 ± 85% +0.1 0.12 ± 10% perf-profile.self.cycles-pp.__mutex_unlock_slowpath 0.06 ± 6% +0.1 0.14 ± 5% perf-profile.self.cycles-pp.__x64_sys_read 0.07 ± 6% +0.1 0.16 ± 5% perf-profile.self.cycles-pp.__x64_sys_write 0.00 +0.1 0.09 perf-profile.self.cycles-pp.page_counter_try_charge 0.00 +0.1 0.09 ± 9% perf-profile.self.cycles-pp.get_mem_cgroup_from_objcg 0.00 +0.1 0.10 ± 9% perf-profile.self.cycles-pp.page_counter_uncharge 0.05 ± 90% +0.1 0.16 ± 31% perf-profile.self.cycles-pp.poll_idle 0.02 ±123% +0.1 0.14 ± 32% perf-profile.self.cycles-pp.queue_event 0.00 +0.1 0.12 ± 10% perf-profile.self.cycles-pp.uncharge_batch 0.00 +0.1 0.12 ± 14% perf-profile.self.cycles-pp.perf_output_copy 0.00 +0.1 0.13 ± 3% perf-profile.self.cycles-pp.copyout 0.00 +0.1 0.13 ± 3% perf-profile.self.cycles-pp.copyin 0.00 +0.1 0.14 perf-profile.self.cycles-pp.rw_verify_area 0.06 ± 10% +0.1 0.20 ± 2% perf-profile.self.cycles-pp.kill_fasync 0.06 ± 15% +0.2 0.23 ± 2% perf-profile.self.cycles-pp.iov_iter_init 0.00 +0.2 0.17 ± 2% perf-profile.self.cycles-pp.osq_unlock 0.06 ± 14% +0.2 0.25 perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.01 ±200% +0.2 0.21 ± 18% perf-profile.self.cycles-pp.perf_output_sample 0.08 ± 5% +0.2 0.28 ± 3% perf-profile.self.cycles-pp.apparmor_file_permission 0.10 ± 5% +0.2 0.30 ± 2% perf-profile.self.cycles-pp.rcu_all_qs 0.15 ± 2% +0.2 0.36 perf-profile.self.cycles-pp.ksys_write 0.18 ± 7% +0.2 0.40 ± 3% perf-profile.self.cycles-pp.__cond_resched 0.01 ±200% +0.2 0.22 ± 2% perf-profile.self.cycles-pp.__might_fault 0.13 ± 10% +0.2 0.35 ± 2% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.08 ± 12% +0.2 0.31 ± 2% perf-profile.self.cycles-pp.touch_atime 0.16 ± 10% +0.2 0.39 ± 2% perf-profile.self.cycles-pp.ksys_read 0.12 ± 11% +0.2 0.36 perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.09 ± 18% +0.2 0.34 ± 18% perf-profile.self.cycles-pp.memcpy_erms 0.07 ± 11% +0.3 0.32 ± 2% perf-profile.self.cycles-pp.__fdget_pos 0.08 ± 17% +0.3 0.36 ± 3% perf-profile.self.cycles-pp.syscall_enter_from_user_mode 0.28 ± 9% +0.3 0.59 ± 3% perf-profile.self.cycles-pp.do_syscall_64 0.15 ± 8% +0.4 0.51 perf-profile.self.cycles-pp.file_update_time 0.09 ± 16% +0.4 0.46 perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.11 ± 9% +0.4 0.49 perf-profile.self.cycles-pp.timestamp_truncate 0.11 ± 18% +0.5 0.57 ± 3% perf-profile.self.cycles-pp.aa_file_perm 0.24 ± 6% +0.5 0.70 perf-profile.self.cycles-pp.exit_to_user_mode_prepare 0.11 ± 18% +0.5 0.58 ± 2% perf-profile.self.cycles-pp.current_time 0.19 ± 12% +0.5 0.72 ± 4% perf-profile.self.cycles-pp.atime_needs_update 0.32 ± 11% +0.6 0.87 ± 2% perf-profile.self.cycles-pp.vfs_read 0.30 ± 12% +0.6 0.91 perf-profile.self.cycles-pp.new_sync_read 0.16 ± 13% +0.6 0.81 perf-profile.self.cycles-pp.copy_page_from_iter 0.13 ± 14% +0.7 0.79 ± 5% perf-profile.self.cycles-pp.security_file_permission 0.30 ± 11% +0.7 0.98 perf-profile.self.cycles-pp.vfs_write 0.48 ± 10% +0.7 1.20 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.03 ±127% +0.8 0.79 ± 14% perf-profile.self.cycles-pp.intel_idle 0.35 ± 5% +0.8 1.13 ± 5% perf-profile.self.cycles-pp.new_sync_write 0.14 ± 15% +0.8 0.91 perf-profile.self.cycles-pp.map_id_range_down 0.43 ± 13% +0.8 1.21 ± 2% perf-profile.self.cycles-pp.__might_sleep 0.22 ± 10% +0.9 1.08 perf-profile.self.cycles-pp.copy_page_to_iter 0.72 ± 13% +0.9 1.60 ± 2% perf-profile.self.cycles-pp.pipe_read 0.05 ± 85% +0.9 0.97 ± 4% perf-profile.self.cycles-pp.__mutex_lock 0.54 ± 16% +1.0 1.58 perf-profile.self.cycles-pp.__libc_read 0.56 ± 14% +1.1 1.70 ± 4% perf-profile.self.cycles-pp.__fget_light 0.39 ± 12% +1.2 1.55 perf-profile.self.cycles-pp.__libc_write 0.28 ± 14% +1.2 1.50 ± 2% perf-profile.self.cycles-pp.__entry_text_start 0.56 ± 8% +1.5 2.03 ± 4% perf-profile.self.cycles-pp.common_file_perm 0.65 ± 11% +1.7 2.38 perf-profile.self.cycles-pp.__might_resched 0.00 +2.4 2.42 ± 3% perf-profile.self.cycles-pp.osq_lock 0.67 ± 13% +2.6 3.30 perf-profile.self.cycles-pp.syscall_return_via_sysret 0.78 ± 13% +3.0 3.76 perf-profile.self.cycles-pp.pipe_write 0.00 +3.0 3.04 ± 6% perf-profile.self.cycles-pp.page_counter_charge 1.18 ± 14% +3.7 4.87 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.00 +4.0 3.96 ± 4% perf-profile.self.cycles-pp.mutex_spin_on_owner 1.01 ± 17% +5.3 6.29 perf-profile.self.cycles-pp.mutex_lock 0.78 ± 21% +5.4 6.21 perf-profile.self.cycles-pp.mutex_unlock 0.01 ±200% +6.2 6.19 ± 4% perf-profile.self.cycles-pp.page_counter_cancel 0.00 +6.5 6.46 ± 4% perf-profile.self.cycles-pp.propagate_protected_usage Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang