November 2021 - LKP - Ml01.01.Org

[squash! clocksource] 0a526bb47c: WARNING:at_kernel/time/clocksource.c:#clocksource_watchdog

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 0a526bb47c44195b902ae91ff8c9dff7322324fb ("squash! clocksource: Forgive repeated long-latency watchdog clocksource reads") https://git.kernel.org/cgit/linux/kernel/git/paulmck/linux-rcu.git dev.2021.10.13a in testcase: stress-ng version: stress-ng-x86_64-0.11-06_20211029 with following parameters: nr_threads: 100% testtime: 60s class: cpu test: ioport cpufreq_governor: performance ucode: 0x5003006 on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 51.311345][ C93] WARNING: CPU: 93 PID: 5763 at kernel/time/clocksource.c:235 clocksource_watchdog (kernel/time/clocksource.c:235 kernel/time/clocksource.c:389) [ 51.326950][ C93] Modules linked in: binfmt_misc btrfs blake2b_generic xor zstd_compress raid6_pq libcrc32c sd_mod t10_pi sg intel_rapl_msr intel_rapl_common skx_edac nfit libnvdimm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ast drm_vram_helper kvm drm_ttm_helper ipmi_ssif ttm irqbypass crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel ghash_clmulni_intel rapl syscopyarea sysfillrect sysimgblt acpi_ipmi intel_cstate fb_sys_fops ahci ipmi_si mei_me libahci ipmi_devintf drm intel_uncore ioatdma libata mei joydev intel_pch_thermal dca wmi ipmi_msghandler acpi_pad acpi_power_meter ip_tables [ 51.331824][ T1303] /lkp/lkp/src/monitors/wrapper: 64: /lkp/lkp/src/monitors/wrapper: [[: not found [ 51.419746][ T1303] [ 51.419749][ C93] CPU: 93 PID: 5763 Comm: stress-ng Not tainted 5.15.0-rc1-00122-g0a526bb47c44 #1 [ 51.490062][ C93] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019 [ 51.516549][ T1303] /lkp/lkp/src/monitors/wrapper: 64: /lkp/lkp/src/monitors/wrapper: [[: not found [ 51.520735][ T1303] [ 51.522283][ C93] RIP: 0010:clocksource_watchdog (kernel/time/clocksource.c:235 kernel/time/clocksource.c:389) [ 51.575930][ C93] Code: 75 fe ff ff 48 8b 56 30 48 c7 c7 d0 93 55 82 4c 89 0c 24 65 44 8b 05 a0 43 e8 7e 44 89 c6 c6 05 e6 de c0 01 01 e8 87 b0 ad 00 <0f> 0b 4c 8b 0c 24 e9 fd fe ff ff e8 59 ec 00 00 e9 b4 fc ff ff 66 All code ======== 0: 75 fe jne 0x0 2: ff (bad) 3: ff 48 8b decl -0x75(%rax) 6: 56 push %rsi 7: 30 48 c7 xor %cl,-0x39(%rax) a: c7 (bad) b: d0 93 55 82 4c 89 rclb -0x76b37dab(%rbx) 11: 0c 24 or $0x24,%al 13: 65 44 8b 05 a0 43 e8 mov %gs:0x7ee843a0(%rip),%r8d # 0x7ee843bb 1a: 7e 1b: 44 89 c6 mov %r8d,%esi 1e: c6 05 e6 de c0 01 01 movb $0x1,0x1c0dee6(%rip) # 0x1c0df0b 25: e8 87 b0 ad 00 callq 0xadb0b1 2a:* 0f 0b ud2 <-- trapping instruction 2c: 4c 8b 0c 24 mov (%rsp),%r9 30: e9 fd fe ff ff jmpq 0xffffffffffffff32 35: e8 59 ec 00 00 callq 0xec93 3a: e9 b4 fc ff ff jmpq 0xfffffffffffffcf3 3f: 66 data16 Code starting with the faulting instruction =========================================== 0: 0f 0b ud2 2: 4c 8b 0c 24 mov (%rsp),%r9 6: e9 fd fe ff ff jmpq 0xffffffffffffff08 b: e8 59 ec 00 00 callq 0xec69 10: e9 b4 fc ff ff jmpq 0xfffffffffffffcc9 15: 66 data16 [ 51.610021][ C93] RSP: 0018:ffffc9000dc68e88 EFLAGS: 00010282 [ 51.618528][ T1303] /lkp/lkp/src/monitors/wrapper: 3: /lkp/lkp/src/lib/lkp_path.sh: [[: not found [ 51.619224][ T1303] [ 51.619226][ C93] RAX: 0000000000000000 RBX: ffffffff828232c0 RCX: 0000000000000000 [ 51.654375][ T1303] /lkp/lkp/src/monitors/wrapper: 5: /lkp/lkp/src/lib/lkp_path.sh: source: not found [ 51.661420][ T1303] [ 51.661449][ C93] RDX: ffff88ff48d63800 RSI: ffff88ff48d57b50 RDI: ffff88ff48d57b50 [ 51.687441][ C93] RBP: 0000000000000000 R08: ffff88ff48d57b50 R09: ffffc9000dc68ca8 [ 51.710127][ C93] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000049784638 [ 51.735896][ C93] R13: 00000dbb62d538fc R14: 0000000000000003 R15: 0000000000000003 [ 51.751265][ C93] FS: 00007fbde3ad7740(0000) GS:ffff88ff48d40000(0000) knlGS:0000000000000000 [ 51.779544][ C93] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 51.782253][ T1303] /lkp/lkp/src/monitors/wrapper: 3: /lkp/lkp/src/lib/lkp_path.sh: [[: not found [ 51.799053][ C93] CR2: 00007f52fb06ba20 CR3: 000000079349e004 CR4: 00000000007706e0 [ 51.799055][ C93] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 51.799214][ C93] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 51.799383][ C93] PKRU: 55555554 [ 51.799385][ C93] Call Trace: [ 51.799725][ C93] <IRQ> [ 51.832939][ T1303] [ 51.852125][ C93] ? trigger_load_balance (arch/x86/include/asm/atomic.h:29 include/linux/atomic/atomic-instrumented.h:28 kernel/sched/fair.c:10363 kernel/sched/fair.c:10919) [ 51.946975][ T1303] /lkp/lkp/src/monitors/wrapper: 5: /lkp/lkp/src/lib/lkp_path.sh: source: not found [ 51.948795][ T1303] [ 51.948794][ C93] ? __clocksource_register_scale (kernel/time/clocksource.c:366) [ 51.961958][ T1303] /lkp/lkp/src/monitors/wrapper: 64: /lkp/lkp/src/monitors/wrapper: [[: not found [ 51.967150][ C93] call_timer_fn (kernel/time/timer.c:1421) [ 51.967153][ T1303] [ 51.983240][ C93] run_timer_softirq (kernel/time/timer.c:1467 kernel/time/timer.c:1734 kernel/time/timer.c:1710 kernel/time/timer.c:1747) [ 51.988436][ C93] ? enqueue_hrtimer (kernel/time/hrtimer.c:1092) [ 51.990262][ T1303] /lkp/lkp/src/monitors/wrapper: 64: /lkp/lkp/src/monitors/wrapper: [[: not found [ 51.993387][ T1303] [ 51.993535][ C93] ? ktime_get (kernel/time/timekeeping.c:193 kernel/time/timekeeping.c:287 kernel/time/timekeeping.c:386 kernel/time/timekeeping.c:829 kernel/time/timekeeping.c:817) [ 52.009974][ C93] __do_softirq (kernel/softirq.c:558) [ 52.014815][ C93] irq_exit_rcu (kernel/softirq.c:432 kernel/softirq.c:637 kernel/softirq.c:649) [ 52.019617][ C93] sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1097 (discriminator 14)) [ 52.021485][ T1303] /lkp/lkp/src/monitors/wrapper: 3: /lkp/lkp/src/lib/lkp_path.sh: [[: not found [ 52.025661][ C93] </IRQ> [ 52.025899][ T1303] [ 52.034972][ C93] asm_sysvec_apic_timer_interrupt (arch/x86/include/asm/idtentry.h:638) [ 52.034980][ C93] RIP: 0010:osq_lock (kernel/locking/osq_lock.c:143) [ 52.034989][ C93] Code: 85 95 00 00 00 65 4c 8b 0c 25 00 6d 01 00 eb 21 48 8b 42 08 8b 78 14 83 ef 01 48 63 ff 48 31 c0 0f 1f 40 00 84 c0 75 10 f3 90 <8b> 42 10 85 c0 75 69 49 8b 01 a8 08 74 d8 31 ff eb 0d 8b 42 10 85 All code ======== 0: 85 95 00 00 00 65 test %edx,0x65000000(%rbp) 6: 4c 8b 0c 25 00 6d 01 mov 0x16d00,%r9 d: 00 e: eb 21 jmp 0x31 10: 48 8b 42 08 mov 0x8(%rdx),%rax 14: 8b 78 14 mov 0x14(%rax),%edi 17: 83 ef 01 sub $0x1,%edi 1a: 48 63 ff movslq %edi,%rdi 1d: 48 31 c0 xor %rax,%rax 20: 0f 1f 40 00 nopl 0x0(%rax) 24: 84 c0 test %al,%al 26: 75 10 jne 0x38 28: f3 90 pause 2a:* 8b 42 10 mov 0x10(%rdx),%eax <-- trapping instruction 2d: 85 c0 test %eax,%eax 2f: 75 69 jne 0x9a 31: 49 8b 01 mov (%r9),%rax 34: a8 08 test $0x8,%al 36: 74 d8 je 0x10 38: 31 ff xor %edi,%edi 3a: eb 0d jmp 0x49 3c: 8b 42 10 mov 0x10(%rdx),%eax 3f: 85 .byte 0x85 Code starting with the faulting instruction =========================================== 0: 8b 42 10 mov 0x10(%rdx),%eax 3: 85 c0 test %eax,%eax 5: 75 69 jne 0x70 7: 49 8b 01 mov (%r9),%rax a: a8 08 test $0x8,%al c: 74 d8 je 0xffffffffffffffe6 e: 31 ff xor %edi,%edi 10: eb 0d jmp 0x1f 12: 8b 42 10 mov 0x10(%rdx),%eax 15: 85 .byte 0x85 To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months

3
5
0 / 0

[sched/fair] 64228563c2: WARNING:at_kernel/kthread.c:#__kthread_bind_mask

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 64228563c20f024e40e4bdaa51eeec99002c489f ("[RFC PATCH 2/4] sched/fair: Introduce cfs_migration") url: https://github.com/0day-ci/linux/commits/Yafang-Shao/sched-Introduce-cfs_... base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git 8ea9183db4ad8afbcb7089a77c23eaf965b0cacd patch link: https://lore.kernel.org/lkml/[email protected] in testcase: leaking-addresses version: leaking-addresses-x86_64-cf2a85e-1_20211103 with following parameters: ucode: 0x28 on test machine: 8 threads 1 sockets Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz with 16G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +--------------------------------------------------+------------+------------+ | | 812ea7cfb1 | 64228563c2 | +--------------------------------------------------+------------+------------+ | boot_successes | 10 | 0 | | boot_failures | 0 | 11 | | WARNING:at_kernel/kthread.c:#__kthread_bind_mask | 0 | 11 | | RIP:__kthread_bind_mask | 0 | 11 | +--------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 3.072411][ T1] WARNING: CPU: 0 PID: 1 at kernel/kthread.c:465 __kthread_bind_mask (kernel/kthread.c:465 (discriminator 1)) [ 3.073411][ T1] Modules linked in: [ 3.074411][ T1] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.15.0-rc4-00071-g64228563c20f #1 [ 3.075411][ T1] Hardware name: Dell Inc. OptiPlex 9020/0DNKMN, BIOS A05 12/05/2013 [ 3.076411][ T1] RIP: 0010:__kthread_bind_mask (kernel/kthread.c:465 (discriminator 1)) [ 3.077411][ T1] Code: 89 e2 b7 00 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 41 55 41 54 55 48 89 f5 89 d6 53 48 89 fb e8 28 bb 01 00 48 85 c0 75 09 <0f> 0b 5b 5d 41 5c 41 5d c3 4c 8d ab a4 0c 00 00 4c 89 ef e8 4b ed All code ======== 0: 89 e2 mov %esp,%edx 2: b7 00 mov $0x0,%bh 4: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1) b: 00 00 d: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1) 12: 41 55 push %r13 14: 41 54 push %r12 16: 55 push %rbp 17: 48 89 f5 mov %rsi,%rbp 1a: 89 d6 mov %edx,%esi 1c: 53 push %rbx 1d: 48 89 fb mov %rdi,%rbx 20: e8 28 bb 01 00 callq 0x1bb4d 25: 48 85 c0 test %rax,%rax 28: 75 09 jne 0x33 2a:* 0f 0b ud2 <-- trapping instruction 2c: 5b pop %rbx 2d: 5d pop %rbp 2e: 41 5c pop %r12 30: 41 5d pop %r13 32: c3 retq 33: 4c 8d ab a4 0c 00 00 lea 0xca4(%rbx),%r13 3a: 4c 89 ef mov %r13,%rdi 3d: e8 .byte 0xe8 3e: 4b ed rex.WXB in (%dx),%eax Code starting with the faulting instruction =========================================== 0: 0f 0b ud2 2: 5b pop %rbx 3: 5d pop %rbp 4: 41 5c pop %r12 6: 41 5d pop %r13 8: c3 retq 9: 4c 8d ab a4 0c 00 00 lea 0xca4(%rbx),%r13 10: 4c 89 ef mov %r13,%rdi 13: e8 .byte 0xe8 14: 4b ed rex.WXB in (%dx),%eax [ 3.078411][ T1] RSP: 0000:ffffc9000002fdd8 EFLAGS: 00010246 [ 3.079411][ T1] RAX: 0000000000000000 RBX: ffff888100aed000 RCX: 0000000000000000 [ 3.080411][ T1] RDX: 0000000000000001 RSI: 0000000000000246 RDI: 00000000ffffffff [ 3.081411][ T1] RBP: ffffffff822101e0 R08: ffffffff8284cec8 R09: ffffffff8284cec8 [ 3.082411][ T1] R10: 0000000000000000 R11: ffff8883fda278f0 R12: 0000000000000008 [ 3.083411][ T1] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 [ 3.084411][ T1] FS: 0000000000000000(0000) GS:ffff8883fda00000(0000) knlGS:0000000000000000 [ 3.085411][ T1] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 3.086411][ T1] CR2: ffff88841ea01000 CR3: 000000041da10001 CR4: 00000000001706f0 [ 3.087411][ T1] Call Trace: [ 3.088413][ T1] kthread_unpark (kernel/kthread.c:478 kernel/kthread.c:570) [ 3.089411][ T1] cfs_migration_init (kernel/sched/fair.c:12029 (discriminator 3)) [ 3.090412][ T1] ? setup_sched_thermal_decay_shift (kernel/sched/fair.c:12014) [ 3.091411][ T1] do_one_initcall (init/main.c:1303) [ 3.092412][ T1] kernel_init_freeable (init/main.c:1419 init/main.c:1603) [ 3.093412][ T1] ? rest_init (init/main.c:1497) [ 3.094411][ T1] kernel_init (init/main.c:1507) [ 3.095411][ T1] ret_from_fork (arch/x86/entry/entry_64.S:301) [ 3.096413][ T1] ---[ end trace 221e592f8f64f075 ]--- [ 3.097411][ T1] rcu: Hierarchical SRCU implementation. [ 3.098989][ T5] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. [ 3.099473][ T1] smp: Bringing up secondary CPUs ... [ 3.100482][ T1] x86: Booting SMP configuration: [ 3.101412][ T1] .... node #0, CPUs: #1 [ 0.534696][ T0] masked ExtINT on CPU#1 [ 3.110490][ T1] #2 [ 0.534696][ T0] masked ExtINT on CPU#2 [ 3.117413][ T1] #3 [ 0.534696][ T0] masked ExtINT on CPU#3 [ 3.124184][ T1] #4 [ 0.534696][ T0] masked ExtINT on CPU#4 [ 3.130933][ T1] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details. [ 3.131497][ T1] #5 [ 0.534696][ T0] masked ExtINT on CPU#5 [ 3.138269][ T1] #6 [ 0.534696][ T0] masked ExtINT on CPU#6 [ 3.145166][ T1] #7 [ 0.534696][ T0] masked ExtINT on CPU#7 [ 3.152059][ T1] smp: Brought up 1 node, 8 CPUs [ 3.152412][ T1] smpboot: Max logical packages: 1 [ 3.153411][ T1] smpboot: Total of 8 processors activated (54273.93 BogoMIPS) [ 3.177958][ T60] node 0 deferred pages initialised in 22ms [ 3.184695][ T1] devtmpfs: initialized [ 3.185443][ T1] x86/mm: Memory block size: 128MB [ 3.187216][ T1] ACPI: PM: Registering ACPI NVS region [mem 0xd1695000-0xd169bfff] (28672 bytes) [ 3.187412][ T1] ACPI: PM: Registering ACPI NVS region [mem 0xda71c000-0xda7fffff] (933888 bytes) [ 3.188450][ T1] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 3.189413][ T1] futex hash table entries: 2048 (order: 5, 131072 bytes, linear) [ 3.190454][ T1] pinctrl core: initialized pinctrl subsystem [ 3.191527][ T1] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 3.192555][ T1] audit: initializing netlink subsys (disabled) [ 3.193423][ T71] audit: type=2000 audit(1636161192.177:1): state=initialized audit_enabled=0 res=1 [ 3.193471][ T1] thermal_sys: Registered thermal governor 'fair_share' [ 3.194412][ T1] thermal_sys: Registered thermal governor 'bang_bang' [ 3.195411][ T1] thermal_sys: Registered thermal governor 'step_wise' [ 3.196411][ T1] thermal_sys: Registered thermal governor 'user_space' [ 3.197420][ T1] cpuidle: using governor menu [ 3.199511][ T1] ACPI: bus type PCI registered [ 3.200412][ T1] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 3.201454][ T1] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000) [ 3.202412][ T1] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved in E820 [ 3.203416][ T1] pmd_set_huge: Cannot satisfy [mem 0xf8000000-0xf8200000] with a huge-page mapping due to MTRR override. [ 3.204444][ T1] PCI: Using configuration type 1 for base access [ 3.205522][ T1] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 3.207568][ T1] Kprobes globally optimized [ 3.208425][ T1] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 3.209412][ T1] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 3.210428][ T1] cryptd: max_cpu_qlen set to 1000 [ 3.211440][ T1] ACPI: Added _OSI(Module Device) [ 3.212415][ T1] ACPI: Added _OSI(Processor Device) [ 3.213411][ T1] ACPI: Added _OSI(3.0 _SCP Extensions) [ 3.214411][ T1] ACPI: Added _OSI(Processor Aggregator Device) [ 3.215411][ T1] ACPI: Added _OSI(Linux-Dell-Video) [ 3.216411][ T1] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) [ 3.217411][ T1] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) [ 3.224227][ T1] ACPI: 6 ACPI AML tables successfully acquired and loaded [ 3.225178][ T1] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored [ 3.225803][ T1] ACPI: Dynamic OEM Table Load: [ 3.226414][ T1] ACPI: SSDT 0xFFFF88841EAB5000 0003D3 (v01 PmRef Cpu0Cst 00003001 INTL 20120711) [ 3.227913][ T1] ACPI: Dynamic OEM Table Load: [ 3.228414][ T1] ACPI: SSDT 0xFFFF88841EAC4000 0005AA (v01 PmRef ApIst 00003000 INTL 20120711) [ 3.229916][ T1] ACPI: Dynamic OEM Table Load: [ 3.230413][ T1] ACPI: SSDT 0xFFFF888100EF8800 000119 (v01 PmRef ApCst 00003000 INTL 20120711) [ 3.232725][ T1] ACPI: Interpreter enabled [ 3.233432][ T1] ACPI: PM: (supports S0 S3 S4 S5) [ 3.234411][ T1] ACPI: Using IOAPIC for interrupt routing [ 3.235430][ T1] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 3.236560][ T1] ACPI: Enabled 9 GPEs in block 00 to 3F [ 3.243366][ T1] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-3e]) [ 3.243414][ T1] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] [ 3.244781][ T1] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug SHPCHotplug PME AER PCIeCapability LTR] [ 3.245679][ T1] PCI host bridge to bus 0000:00 [ 3.246412][ T1] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 3.247411][ T1] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 3.248411][ T1] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 3.249411][ T1] pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000d7fff window] [ 3.250411][ T1] pci_bus 0000:00: root bus resource [mem 0x000d8000-0x000dbfff window] [ 3.251411][ T1] pci_bus 0000:00: root bus resource [mem 0x000dc000-0x000dffff window] [ 3.252411][ T1] pci_bus 0000:00: root bus resource [mem 0x000e0000-0x000e3fff window] [ 3.253411][ T1] pci_bus 0000:00: root bus resource [mem 0x000e4000-0x000e7fff window] [ 3.254411][ T1] pci_bus 0000:00: root bus resource [mem 0xdf200000-0xfeafffff window] [ 3.255411][ T1] pci_bus 0000:00: root bus resource [bus 00-3e] [ 3.256436][ T1] pci 0000:00:00.0: [8086:0c00] type 00 class 0x060000 [ 3.257508][ T1] pci 0000:00:02.0: [8086:0412] type 00 class 0x030000 [ 3.258416][ T1] pci 0000:00:02.0: reg 0x10: [mem 0xf7800000-0xf7bfffff 64bit] [ 3.259414][ T1] pci 0000:00:02.0: reg 0x18: [mem 0xe0000000-0xefffffff 64bit pref] To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months

2
1
0 / 0

[xfs] 01e8f379a4: stress-ng.dir.ops_per_sec -23.7% regression

by kernel test robot

Greeting, FYI, we noticed a -23.7% regression of stress-ng.dir.ops_per_sec due to commit: commit: 01e8f379a4895a9a173391408db4fb49ec91e148 ("xfs: flush inode inactivation work when compiling usage statistics") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: stress-ng on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 512G memory with following parameters: nr_threads: 10% disk: 1HDD testtime: 60s fs: xfs class: filesystem test: dir cpufreq_governor: performance ucode: 0x5003006 If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. ========================================================================================= class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode: filesystem/gcc-9/performance/1HDD/xfs/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp7/dir/stress-ng/60s/0x5003006 commit: 2eb665027b ("xfs: inactivate inodes any time we try to free speculative preallocations") 01e8f379a4 ("xfs: flush inode inactivation work when compiling usage statistics") 2eb665027b6528c1 01e8f379a4895a9a173391408db ---------------- --------------------------- %stddev %change %stddev \ | \ 2285220 ± 5% -23.7% 1744748 ± 2% stress-ng.dir.ops 37771 ± 5% -23.7% 28804 stress-ng.dir.ops_per_sec 189150 ± 7% +1143.8% 2352581 stress-ng.time.involuntary_context_switches 275.25 +51.7% 417.50 stress-ng.time.percent_of_cpu_this_job_got 166.30 +54.3% 256.55 stress-ng.time.system_time 3113811 ± 4% +110.4% 6551734 ± 3% stress-ng.time.voluntary_context_switches 94.78 -1.0% 93.80 iostat.cpu.idle 4.99 +20.5% 6.01 iostat.cpu.system 5706 ± 3% +14.8% 6549 ± 3% meminfo.Active 5482 ± 3% +13.9% 6243 ± 3% meminfo.Active(anon) 0.00 ± 62% -0.0 0.00 ±145% mpstat.cpu.all.iowait% 4.05 +1.2 5.21 mpstat.cpu.all.sys% 0.22 ± 12% -0.0 0.17 ± 3% mpstat.cpu.all.usr% 11665 ± 5% -37.9% 7249 vmstat.io.bo 4.00 +37.5% 5.50 ± 9% vmstat.procs.r 156644 ± 5% +103.7% 319063 ± 2% vmstat.system.cs 158.00 +24.1% 196.00 ± 2% turbostat.Avg_MHz 6.27 ± 2% +1.1 7.35 turbostat.Busy% 2529 ± 3% +5.8% 2674 turbostat.Bzy_MHz 154.79 ± 3% +5.3% 162.98 turbostat.PkgWatt 8797 ± 5% +11.5% 9808 ± 4% slabinfo.kmalloc-128.active_objs 9424 ± 5% +12.8% 10628 ± 3% slabinfo.kmalloc-128.num_objs 5965 ± 5% -13.1% 5181 ± 4% slabinfo.kmalloc-2k.active_objs 6025 ± 5% -13.7% 5200 ± 4% slabinfo.kmalloc-2k.num_objs 3891 ± 4% -41.2% 2288 slabinfo.kmalloc-8k.active_objs 988.88 ± 3% -41.7% 576.33 slabinfo.kmalloc-8k.active_slabs 3957 ± 3% -41.7% 2307 slabinfo.kmalloc-8k.num_objs 988.88 ± 3% -41.7% 576.33 slabinfo.kmalloc-8k.num_slabs 1370 ± 3% +13.5% 1555 ± 3% proc-vmstat.nr_active_anon 18309 -3.7% 17635 proc-vmstat.nr_kernel_stack 4325 +4.2% 4509 proc-vmstat.nr_shmem 1370 ± 3% +13.5% 1555 ± 3% proc-vmstat.nr_zone_active_anon 790035 ± 2% -11.8% 696596 proc-vmstat.numa_hit 703246 ± 2% -13.3% 609791 proc-vmstat.numa_local 2587 ± 3% +22.0% 3156 ± 5% proc-vmstat.pgactivate 792943 ± 2% -11.8% 699547 proc-vmstat.pgalloc_normal 670410 ± 2% -14.0% 576782 proc-vmstat.pgfree 768391 ± 5% -38.0% 476034 ± 2% proc-vmstat.pgpgout 13643 ± 4% -33.3% 9100 softirqs.BLOCK 7786 ± 25% -30.7% 5393 ± 15% softirqs.CPU1.RCU 6841 ± 31% -37.3% 4291 ± 23% softirqs.CPU34.RCU 6283 ± 26% -35.9% 4025 ± 23% softirqs.CPU42.RCU 7196 ± 26% -42.2% 4162 ± 28% softirqs.CPU48.RCU 10076 ± 8% -13.6% 8707 ± 5% softirqs.CPU49.SCHED 6886 ± 30% -34.9% 4485 ± 17% softirqs.CPU6.RCU 6985 ± 26% -32.5% 4718 ± 16% softirqs.CPU61.RCU 6830 ± 27% -30.0% 4780 ± 20% softirqs.CPU9.RCU 604772 ± 16% -29.7% 424951 ± 20% softirqs.RCU 4.36 ± 3% +26.9% 5.53 ± 6% perf-stat.i.MPKI 0.66 ± 3% +0.0 0.69 ± 3% perf-stat.i.branch-miss-rate% 85063336 ± 6% +14.5% 97374871 ± 6% perf-stat.i.cache-references 161851 ± 5% +102.9% 328460 ± 3% perf-stat.i.context-switches 0.77 ± 5% +34.7% 1.03 ± 4% perf-stat.i.cpi 1.466e+10 +25.4% 1.838e+10 ± 3% perf-stat.i.cpu-cycles 2196 ± 10% +307.8% 8957 ± 5% perf-stat.i.cpu-migrations 0.02 ± 18% +0.0 0.03 ± 18% perf-stat.i.dTLB-load-miss-rate% 989191 ± 15% +38.7% 1371676 ± 18% perf-stat.i.dTLB-load-misses 5.112e+09 ± 4% -9.3% 4.635e+09 perf-stat.i.dTLB-loads 2.607e+09 ± 5% -8.8% 2.378e+09 perf-stat.i.dTLB-stores 36.52 +3.2 39.71 perf-stat.i.iTLB-load-miss-rate% 9729269 ± 4% -12.9% 8475091 ± 2% perf-stat.i.iTLB-loads 3695 ± 3% -8.5% 3381 ± 3% perf-stat.i.instructions-per-iTLB-miss 1.32 ± 6% -25.7% 0.98 ± 4% perf-stat.i.ipc 0.15 +25.4% 0.19 ± 3% perf-stat.i.metric.GHz 833.35 ± 13% -30.6% 578.12 ± 22% perf-stat.i.metric.K/sec 63.09 ± 32% +21.6 84.65 ± 2% perf-stat.i.node-load-miss-rate% 1534591 ± 16% -41.5% 897973 ± 10% perf-stat.i.node-loads 57.02 ± 36% +30.4 87.39 ± 3% perf-stat.i.node-store-miss-rate% 1665326 ± 44% +239.2% 5648703 ± 9% perf-stat.i.node-store-misses 850494 ± 28% -50.3% 422731 ± 10% perf-stat.i.node-stores 4.37 ± 3% +23.1% 5.38 ± 5% perf-stat.overall.MPKI 0.76 ± 6% +34.4% 1.02 ± 4% perf-stat.overall.cpi 0.02 ± 17% +0.0 0.03 ± 18% perf-stat.overall.dTLB-load-miss-rate% 36.21 +3.1 39.35 perf-stat.overall.iTLB-load-miss-rate% 1.33 ± 6% -25.8% 0.99 ± 4% perf-stat.overall.ipc 70.65 ± 23% +16.7 87.35 perf-stat.overall.node-load-miss-rate% 62.56 ± 30% +30.4 92.98 perf-stat.overall.node-store-miss-rate% 83708885 ± 6% +14.5% 95833320 ± 6% perf-stat.ps.cache-references 159241 ± 5% +102.9% 323144 ± 3% perf-stat.ps.context-switches 1.443e+10 +25.4% 1.809e+10 ± 3% perf-stat.ps.cpu-cycles 2160 ± 10% +307.9% 8811 ± 5% perf-stat.ps.cpu-migrations 973211 ± 15% +38.7% 1349799 ± 18% perf-stat.ps.dTLB-load-misses 5.031e+09 ± 4% -9.3% 4.563e+09 perf-stat.ps.dTLB-loads 2.566e+09 ± 5% -8.8% 2.341e+09 perf-stat.ps.dTLB-stores 9574218 ± 4% -12.9% 8340948 ± 2% perf-stat.ps.iTLB-loads 1509918 ± 16% -41.5% 883602 ± 10% perf-stat.ps.node-loads 1638853 ± 44% +239.1% 5557682 ± 9% perf-stat.ps.node-store-misses 836829 ± 28% -50.3% 416240 ± 10% perf-stat.ps.node-stores 130915 ± 3% +335.5% 570113 ± 6% interrupts.CAL:Function_call_interrupts 1595 ± 15% +287.4% 6181 ± 3% interrupts.CPU0.CAL:Function_call_interrupts 1424 ± 50% +272.5% 5306 ± 40% interrupts.CPU0.RES:Rescheduling_interrupts 1691 ± 31% +273.7% 6321 ± 5% interrupts.CPU1.CAL:Function_call_interrupts 1402 ± 50% +284.9% 5396 ± 42% interrupts.CPU1.RES:Rescheduling_interrupts 1349 ± 15% +377.6% 6445 ± 5% interrupts.CPU10.CAL:Function_call_interrupts 1400 ± 53% +345.2% 6236 ± 37% interrupts.CPU10.RES:Rescheduling_interrupts 1382 ± 14% +388.0% 6747 ± 6% interrupts.CPU11.CAL:Function_call_interrupts 1405 ± 52% +357.1% 6422 ± 35% interrupts.CPU11.RES:Rescheduling_interrupts 1426 ± 15% +375.1% 6776 ± 7% interrupts.CPU12.CAL:Function_call_interrupts 1411 ± 55% +348.8% 6333 ± 34% interrupts.CPU12.RES:Rescheduling_interrupts 1402 ± 6% +378.1% 6705 ± 6% interrupts.CPU13.CAL:Function_call_interrupts 1391 ± 54% +354.8% 6327 ± 34% interrupts.CPU13.RES:Rescheduling_interrupts 1407 ± 11% +395.0% 6967 ± 9% interrupts.CPU14.CAL:Function_call_interrupts 1423 ± 54% +349.3% 6396 ± 34% interrupts.CPU14.RES:Rescheduling_interrupts 1413 ± 9% +362.1% 6531 ± 4% interrupts.CPU15.CAL:Function_call_interrupts 1432 ± 54% +346.4% 6393 ± 34% interrupts.CPU15.RES:Rescheduling_interrupts 1372 ± 15% +393.8% 6776 ± 5% interrupts.CPU16.CAL:Function_call_interrupts 1419 ± 52% +351.5% 6409 ± 33% interrupts.CPU16.RES:Rescheduling_interrupts 1401 ± 9% +386.6% 6820 ± 6% interrupts.CPU17.CAL:Function_call_interrupts 1392 ± 53% +364.6% 6470 ± 32% interrupts.CPU17.RES:Rescheduling_interrupts 1362 ± 10% +395.0% 6745 ± 6% interrupts.CPU18.CAL:Function_call_interrupts 1361 ± 47% +374.2% 6455 ± 32% interrupts.CPU18.RES:Rescheduling_interrupts 1418 ± 22% +371.8% 6691 ± 5% interrupts.CPU19.CAL:Function_call_interrupts 1344 ± 48% +373.9% 6369 ± 33% interrupts.CPU19.RES:Rescheduling_interrupts 1594 ± 12% +304.2% 6445 ± 11% interrupts.CPU2.CAL:Function_call_interrupts 1416 ± 51% +281.8% 5408 ± 41% interrupts.CPU2.RES:Rescheduling_interrupts 1334 ± 8% +390.8% 6551 ± 6% interrupts.CPU20.CAL:Function_call_interrupts 1327 ± 50% +380.8% 6382 ± 32% interrupts.CPU20.RES:Rescheduling_interrupts 1366 ± 11% +386.5% 6648 ± 5% interrupts.CPU21.CAL:Function_call_interrupts 1382 ± 48% +359.1% 6346 ± 29% interrupts.CPU21.RES:Rescheduling_interrupts 1314 ± 4% +419.1% 6823 ± 5% interrupts.CPU22.CAL:Function_call_interrupts 1371 ± 53% +358.2% 6285 ± 28% interrupts.CPU22.RES:Rescheduling_interrupts 1359 ± 13% +412.5% 6965 ± 6% interrupts.CPU23.CAL:Function_call_interrupts 1356 ± 52% +370.9% 6389 ± 29% interrupts.CPU23.RES:Rescheduling_interrupts 2022 ± 50% +280.8% 7703 ± 20% interrupts.CPU24.CAL:Function_call_interrupts 691.75 ± 53% +168.4% 1856 ± 23% interrupts.CPU24.NMI:Non-maskable_interrupts 691.75 ± 53% +168.4% 1856 ± 23% interrupts.CPU24.PMI:Performance_monitoring_interrupts 1024 ± 40% +449.3% 5628 ± 39% interrupts.CPU24.RES:Rescheduling_interrupts 1424 ± 32% +364.6% 6617 ± 13% interrupts.CPU25.CAL:Function_call_interrupts 545.38 ± 60% +168.3% 1463 ± 25% interrupts.CPU25.NMI:Non-maskable_interrupts 545.38 ± 60% +168.3% 1463 ± 25% interrupts.CPU25.PMI:Performance_monitoring_interrupts 978.62 ± 42% +433.4% 5220 ± 36% interrupts.CPU25.RES:Rescheduling_interrupts 1639 ± 49% +283.2% 6282 ± 15% interrupts.CPU26.CAL:Function_call_interrupts 520.00 ± 62% +112.1% 1102 ± 34% interrupts.CPU26.NMI:Non-maskable_interrupts 520.00 ± 62% +112.1% 1102 ± 34% interrupts.CPU26.PMI:Performance_monitoring_interrupts 944.75 ± 41% +437.6% 5078 ± 37% interrupts.CPU26.RES:Rescheduling_interrupts 1174 ± 34% +426.8% 6187 ± 15% interrupts.CPU27.CAL:Function_call_interrupts 1001 ± 44% +423.8% 5245 ± 37% interrupts.CPU27.RES:Rescheduling_interrupts 490.62 ± 50% +97.0% 966.67 ± 29% interrupts.CPU28.NMI:Non-maskable_interrupts 490.62 ± 50% +97.0% 966.67 ± 29% interrupts.CPU28.PMI:Performance_monitoring_interrupts 955.38 ± 42% +456.4% 5315 ± 38% interrupts.CPU28.RES:Rescheduling_interrupts 1191 ± 28% +408.8% 6061 ± 14% interrupts.CPU29.CAL:Function_call_interrupts 943.25 ± 43% +443.4% 5125 ± 37% interrupts.CPU29.RES:Rescheduling_interrupts 1581 ± 15% +313.4% 6535 ± 7% interrupts.CPU3.CAL:Function_call_interrupts 1441 ± 48% +292.1% 5650 ± 40% interrupts.CPU3.RES:Rescheduling_interrupts 951.62 ± 45% +443.3% 5169 ± 35% interrupts.CPU30.RES:Rescheduling_interrupts 935.88 ± 42% +461.0% 5250 ± 36% interrupts.CPU31.RES:Rescheduling_interrupts 1132 ± 25% +427.0% 5968 ± 13% interrupts.CPU32.CAL:Function_call_interrupts 929.88 ± 40% +454.7% 5157 ± 35% interrupts.CPU32.RES:Rescheduling_interrupts 1201 ± 27% +401.0% 6021 ± 13% interrupts.CPU33.CAL:Function_call_interrupts 942.88 ± 42% +447.6% 5163 ± 35% interrupts.CPU33.RES:Rescheduling_interrupts 1196 ± 20% +393.5% 5905 ± 15% interrupts.CPU34.CAL:Function_call_interrupts 961.25 ± 42% +437.1% 5163 ± 33% interrupts.CPU34.RES:Rescheduling_interrupts 1169 ± 25% +410.1% 5965 ± 14% interrupts.CPU35.CAL:Function_call_interrupts 949.50 ± 41% +441.2% 5138 ± 34% interrupts.CPU35.RES:Rescheduling_interrupts 1158 ± 22% +411.2% 5922 ± 14% interrupts.CPU36.CAL:Function_call_interrupts 927.88 ± 41% +448.5% 5089 ± 33% interrupts.CPU36.RES:Rescheduling_interrupts 1150 ± 24% +412.6% 5897 ± 15% interrupts.CPU37.CAL:Function_call_interrupts 961.38 ± 42% +435.6% 5149 ± 35% interrupts.CPU37.RES:Rescheduling_interrupts 1219 ± 24% +385.7% 5921 ± 13% interrupts.CPU38.CAL:Function_call_interrupts 961.00 ± 40% +440.4% 5193 ± 33% interrupts.CPU38.RES:Rescheduling_interrupts 1542 ± 69% +279.6% 5853 ± 14% interrupts.CPU39.CAL:Function_call_interrupts 939.00 ± 42% +457.3% 5233 ± 34% interrupts.CPU39.RES:Rescheduling_interrupts 1617 ± 22% +301.8% 6497 ± 3% interrupts.CPU4.CAL:Function_call_interrupts 565.00 ± 36% +88.6% 1065 ± 23% interrupts.CPU4.NMI:Non-maskable_interrupts 565.00 ± 36% +88.6% 1065 ± 23% interrupts.CPU4.PMI:Performance_monitoring_interrupts 1353 ± 51% +323.2% 5727 ± 39% interrupts.CPU4.RES:Rescheduling_interrupts 1122 ± 28% +427.9% 5923 ± 11% interrupts.CPU40.CAL:Function_call_interrupts 897.38 ± 42% +475.1% 5160 ± 33% interrupts.CPU40.RES:Rescheduling_interrupts 1174 ± 24% +397.3% 5841 ± 14% interrupts.CPU41.CAL:Function_call_interrupts 944.62 ± 44% +429.8% 5004 ± 34% interrupts.CPU41.RES:Rescheduling_interrupts 1193 ± 29% +399.9% 5964 ± 13% interrupts.CPU42.CAL:Function_call_interrupts 961.50 ± 44% +435.3% 5147 ± 34% interrupts.CPU42.RES:Rescheduling_interrupts 1170 ± 29% +405.9% 5921 ± 14% interrupts.CPU43.CAL:Function_call_interrupts 954.62 ± 42% +448.9% 5240 ± 32% interrupts.CPU43.RES:Rescheduling_interrupts 1147 ± 26% +402.3% 5764 ± 13% interrupts.CPU44.CAL:Function_call_interrupts 923.38 ± 43% +451.8% 5095 ± 33% interrupts.CPU44.RES:Rescheduling_interrupts 1136 ± 22% +397.7% 5655 ± 11% interrupts.CPU45.CAL:Function_call_interrupts 926.12 ± 40% +453.2% 5123 ± 32% interrupts.CPU45.RES:Rescheduling_interrupts 1157 ± 26% +394.8% 5727 ± 12% interrupts.CPU46.CAL:Function_call_interrupts 935.50 ± 41% +457.0% 5211 ± 33% interrupts.CPU46.RES:Rescheduling_interrupts 1264 ± 24% +382.9% 6105 ± 13% interrupts.CPU47.CAL:Function_call_interrupts 925.88 ± 41% +479.1% 5361 ± 32% interrupts.CPU47.RES:Rescheduling_interrupts 1329 ± 9% +380.7% 6389 ± 2% interrupts.CPU48.CAL:Function_call_interrupts 1389 ± 54% +331.6% 5998 ± 34% interrupts.CPU48.RES:Rescheduling_interrupts 1388 ± 17% +340.7% 6119 ± 5% interrupts.CPU49.CAL:Function_call_interrupts 1391 ± 50% +331.4% 6002 ± 34% interrupts.CPU49.RES:Rescheduling_interrupts 1581 ± 39% +317.1% 6597 ± 6% interrupts.CPU5.CAL:Function_call_interrupts 1425 ± 53% +312.1% 5873 ± 39% interrupts.CPU5.RES:Rescheduling_interrupts 1339 ± 9% +365.2% 6230 ± 5% interrupts.CPU50.CAL:Function_call_interrupts 1379 ± 51% +343.0% 6109 ± 35% interrupts.CPU50.RES:Rescheduling_interrupts 1354 ± 11% +360.2% 6233 ± 2% interrupts.CPU51.CAL:Function_call_interrupts 1395 ± 53% +323.3% 5908 ± 34% interrupts.CPU51.RES:Rescheduling_interrupts 1309 ± 9% +366.2% 6105 ± 5% interrupts.CPU52.CAL:Function_call_interrupts 510.38 ± 49% +84.2% 940.17 ± 23% interrupts.CPU52.NMI:Non-maskable_interrupts 510.38 ± 49% +84.2% 940.17 ± 23% interrupts.CPU52.PMI:Performance_monitoring_interrupts 1392 ± 55% +320.0% 5848 ± 34% interrupts.CPU52.RES:Rescheduling_interrupts 1374 ± 14% +330.8% 5920 ± 5% interrupts.CPU53.CAL:Function_call_interrupts 1374 ± 52% +316.6% 5728 ± 34% interrupts.CPU53.RES:Rescheduling_interrupts 1320 ± 8% +343.2% 5851 interrupts.CPU54.CAL:Function_call_interrupts 1385 ± 55% +303.7% 5593 ± 33% interrupts.CPU54.RES:Rescheduling_interrupts 1389 ± 11% +314.2% 5756 ± 2% interrupts.CPU55.CAL:Function_call_interrupts 1381 ± 53% +300.2% 5528 ± 35% interrupts.CPU55.RES:Rescheduling_interrupts 1396 ± 6% +302.4% 5620 ± 4% interrupts.CPU56.CAL:Function_call_interrupts 1375 ± 53% +302.5% 5536 ± 36% interrupts.CPU56.RES:Rescheduling_interrupts 1394 ± 11% +303.5% 5625 ± 4% interrupts.CPU57.CAL:Function_call_interrupts 1406 ± 53% +290.8% 5497 ± 36% interrupts.CPU57.RES:Rescheduling_interrupts 1364 ± 13% +319.4% 5722 ± 3% interrupts.CPU58.CAL:Function_call_interrupts 1367 ± 54% +300.8% 5478 ± 34% interrupts.CPU58.RES:Rescheduling_interrupts 1354 ± 17% +315.6% 5628 ± 3% interrupts.CPU59.CAL:Function_call_interrupts 1375 ± 56% +292.8% 5403 ± 34% interrupts.CPU59.RES:Rescheduling_interrupts 1415 ± 16% +361.3% 6528 ± 5% interrupts.CPU6.CAL:Function_call_interrupts 1413 ± 53% +321.1% 5953 ± 36% interrupts.CPU6.RES:Rescheduling_interrupts 1324 ± 6% +322.5% 5597 ± 3% interrupts.CPU60.CAL:Function_call_interrupts 1436 ± 52% +272.6% 5351 ± 33% interrupts.CPU60.RES:Rescheduling_interrupts 1432 ± 15% +285.0% 5513 ± 4% interrupts.CPU61.CAL:Function_call_interrupts 1441 ± 57% +269.3% 5321 ± 34% interrupts.CPU61.RES:Rescheduling_interrupts 1369 ± 11% +300.8% 5488 ± 3% interrupts.CPU62.CAL:Function_call_interrupts 1358 ± 53% +287.6% 5266 ± 34% interrupts.CPU62.RES:Rescheduling_interrupts 1344 ± 11% +311.0% 5525 ± 5% interrupts.CPU63.CAL:Function_call_interrupts 1387 ± 54% +283.1% 5316 ± 33% interrupts.CPU63.RES:Rescheduling_interrupts 1348 ± 11% +307.0% 5486 interrupts.CPU64.CAL:Function_call_interrupts 1387 ± 51% +276.8% 5229 ± 35% interrupts.CPU64.RES:Rescheduling_interrupts 1317 ± 11% +308.9% 5387 ± 4% interrupts.CPU65.CAL:Function_call_interrupts 1386 ± 54% +276.9% 5227 ± 34% interrupts.CPU65.RES:Rescheduling_interrupts 1354 ± 11% +302.4% 5451 ± 3% interrupts.CPU66.CAL:Function_call_interrupts 1440 ± 51% +261.5% 5205 ± 35% interrupts.CPU66.RES:Rescheduling_interrupts 1298 ± 13% +305.5% 5265 ± 5% interrupts.CPU67.CAL:Function_call_interrupts 1408 ± 53% +266.9% 5166 ± 35% interrupts.CPU67.RES:Rescheduling_interrupts 1296 ± 9% +302.3% 5214 ± 3% interrupts.CPU68.CAL:Function_call_interrupts 1435 ± 55% +254.9% 5095 ± 34% interrupts.CPU68.RES:Rescheduling_interrupts 1380 ± 15% +289.9% 5382 ± 4% interrupts.CPU69.CAL:Function_call_interrupts 1407 ± 52% +259.6% 5060 ± 33% interrupts.CPU69.RES:Rescheduling_interrupts 1733 ± 32% +274.7% 6496 ± 7% interrupts.CPU7.CAL:Function_call_interrupts 1398 ± 49% +341.0% 6167 ± 38% interrupts.CPU7.RES:Rescheduling_interrupts 1297 ± 12% +306.5% 5274 ± 6% interrupts.CPU70.CAL:Function_call_interrupts 1360 ± 51% +259.5% 4891 ± 33% interrupts.CPU70.RES:Rescheduling_interrupts 1379 ± 18% +242.0% 4717 ± 5% interrupts.CPU71.CAL:Function_call_interrupts 1384 ± 55% +201.5% 4174 ± 34% interrupts.CPU71.RES:Rescheduling_interrupts 1190 ± 22% +385.1% 5772 ± 13% interrupts.CPU72.CAL:Function_call_interrupts 900.62 ± 42% +431.5% 4787 ± 34% interrupts.CPU72.RES:Rescheduling_interrupts 1355 ± 15% +319.8% 5688 ± 13% interrupts.CPU73.CAL:Function_call_interrupts 559.50 ± 49% +83.1% 1024 ± 26% interrupts.CPU73.NMI:Non-maskable_interrupts 559.50 ± 49% +83.1% 1024 ± 26% interrupts.CPU73.PMI:Performance_monitoring_interrupts 827.62 ± 42% +486.2% 4851 ± 35% interrupts.CPU73.RES:Rescheduling_interrupts 1185 ± 24% +378.5% 5671 ± 13% interrupts.CPU74.CAL:Function_call_interrupts 842.38 ± 45% +478.6% 4873 ± 34% interrupts.CPU74.RES:Rescheduling_interrupts 1149 ± 24% +396.6% 5705 ± 13% interrupts.CPU75.CAL:Function_call_interrupts 852.75 ± 43% +468.7% 4849 ± 35% interrupts.CPU75.RES:Rescheduling_interrupts 1189 ± 24% +362.3% 5499 ± 13% interrupts.CPU76.CAL:Function_call_interrupts 870.50 ± 43% +444.6% 4741 ± 34% interrupts.CPU76.RES:Rescheduling_interrupts 1142 ± 25% +385.2% 5542 ± 15% interrupts.CPU77.CAL:Function_call_interrupts 853.75 ± 45% +452.5% 4717 ± 34% interrupts.CPU77.RES:Rescheduling_interrupts 1135 ± 26% +387.8% 5540 ± 13% interrupts.CPU78.CAL:Function_call_interrupts 855.62 ± 43% +446.7% 4677 ± 35% interrupts.CPU78.RES:Rescheduling_interrupts 1188 ± 27% +363.6% 5508 ± 14% interrupts.CPU79.CAL:Function_call_interrupts 872.50 ± 44% +431.4% 4636 ± 35% interrupts.CPU79.RES:Rescheduling_interrupts 1436 ± 9% +352.5% 6499 ± 5% interrupts.CPU8.CAL:Function_call_interrupts 1389 ± 53% +351.8% 6277 ± 36% interrupts.CPU8.RES:Rescheduling_interrupts 1173 ± 25% +364.5% 5448 ± 14% interrupts.CPU80.CAL:Function_call_interrupts 480.12 ± 38% +67.6% 804.50 ± 21% interrupts.CPU80.NMI:Non-maskable_interrupts 480.12 ± 38% +67.6% 804.50 ± 21% interrupts.CPU80.PMI:Performance_monitoring_interrupts 893.00 ± 44% +420.4% 4647 ± 34% interrupts.CPU80.RES:Rescheduling_interrupts 1133 ± 22% +382.7% 5471 ± 15% interrupts.CPU81.CAL:Function_call_interrupts 849.62 ± 43% +456.0% 4724 ± 34% interrupts.CPU81.RES:Rescheduling_interrupts 1192 ± 17% +359.9% 5486 ± 14% interrupts.CPU82.CAL:Function_call_interrupts 900.75 ± 41% +421.5% 4697 ± 33% interrupts.CPU82.RES:Rescheduling_interrupts 1143 ± 20% +379.0% 5479 ± 12% interrupts.CPU83.CAL:Function_call_interrupts 896.00 ± 43% +422.8% 4684 ± 33% interrupts.CPU83.RES:Rescheduling_interrupts 1162 ± 22% +363.8% 5391 ± 13% interrupts.CPU84.CAL:Function_call_interrupts 885.88 ± 42% +421.6% 4621 ± 33% interrupts.CPU84.RES:Rescheduling_interrupts 1196 ± 17% +349.3% 5377 ± 12% interrupts.CPU85.CAL:Function_call_interrupts 871.50 ± 40% +430.4% 4622 ± 33% interrupts.CPU85.RES:Rescheduling_interrupts 1216 ± 21% +334.6% 5286 ± 11% interrupts.CPU86.CAL:Function_call_interrupts 876.00 ± 45% +417.9% 4536 ± 33% interrupts.CPU86.RES:Rescheduling_interrupts 1191 ± 26% +348.2% 5341 ± 13% interrupts.CPU87.CAL:Function_call_interrupts 870.75 ± 44% +424.4% 4566 ± 34% interrupts.CPU87.RES:Rescheduling_interrupts 1137 ± 23% +377.6% 5430 ± 12% interrupts.CPU88.CAL:Function_call_interrupts 861.00 ± 42% +445.9% 4700 ± 33% interrupts.CPU88.RES:Rescheduling_interrupts 1208 ± 21% +346.7% 5398 ± 14% interrupts.CPU89.CAL:Function_call_interrupts 876.12 ± 42% +420.0% 4556 ± 34% interrupts.CPU89.RES:Rescheduling_interrupts 1378 ± 11% +373.4% 6523 ± 5% interrupts.CPU9.CAL:Function_call_interrupts 1428 ± 54% +333.5% 6193 ± 37% interrupts.CPU9.RES:Rescheduling_interrupts 1196 ± 23% +346.9% 5348 ± 14% interrupts.CPU90.CAL:Function_call_interrupts 878.38 ± 43% +406.3% 4447 ± 34% interrupts.CPU90.RES:Rescheduling_interrupts 1182 ± 28% +342.0% 5227 ± 13% interrupts.CPU91.CAL:Function_call_interrupts 864.12 ± 45% +415.9% 4458 ± 34% interrupts.CPU91.RES:Rescheduling_interrupts 1154 ± 23% +356.2% 5267 ± 13% interrupts.CPU92.CAL:Function_call_interrupts 852.88 ± 43% +429.0% 4511 ± 33% interrupts.CPU92.RES:Rescheduling_interrupts 1176 ± 26% +346.0% 5247 ± 12% interrupts.CPU93.CAL:Function_call_interrupts 881.00 ± 42% +411.6% 4507 ± 33% interrupts.CPU93.RES:Rescheduling_interrupts 1155 ± 23% +343.0% 5118 ± 13% interrupts.CPU94.CAL:Function_call_interrupts 848.38 ± 42% +420.3% 4414 ± 34% interrupts.CPU94.RES:Rescheduling_interrupts 1181 ± 29% +312.7% 4875 ± 12% interrupts.CPU95.CAL:Function_call_interrupts 856.00 ± 44% +368.0% 4006 ± 34% interrupts.CPU95.RES:Rescheduling_interrupts 59034 ± 7% +24.3% 73355 ± 7% interrupts.NMI:Non-maskable_interrupts 59034 ± 7% +24.3% 73355 ± 7% interrupts.PMI:Performance_monitoring_interrupts 110578 ± 18% +363.6% 512616 ± 4% interrupts.RES:Rescheduling_interrupts 14.76 ± 3% -7.4 7.34 ± 6% perf-profile.calltrace.cycles-pp.xfs_inodegc_worker.process_one_work.worker_thread.kthread.ret_from_fork 14.62 ± 3% -7.4 7.26 ± 6% perf-profile.calltrace.cycles-pp.xfs_inactive.xfs_inodegc_worker.process_one_work.worker_thread.kthread 14.59 ± 3% -7.3 7.24 ± 6% perf-profile.calltrace.cycles-pp.xfs_inactive_ifree.xfs_inactive.xfs_inodegc_worker.process_one_work.worker_thread 15.54 ± 3% -6.7 8.88 ± 5% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 16.40 ± 6% -6.4 9.99 ± 7% perf-profile.calltrace.cycles-pp.xfs_generic_create.vfs_mkdir.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe 16.29 ± 6% -6.4 9.91 ± 7% perf-profile.calltrace.cycles-pp.xfs_create.xfs_generic_create.vfs_mkdir.do_mkdirat.do_syscall_64 15.64 ± 3% -6.3 9.37 ± 4% perf-profile.calltrace.cycles-pp.ret_from_fork 15.63 ± 3% -6.3 9.37 ± 4% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 15.56 ± 3% -6.2 9.33 ± 4% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 11.44 ± 5% -5.4 6.03 ± 6% perf-profile.calltrace.cycles-pp.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_inodegc_worker.process_one_work 10.77 ± 6% -4.7 6.11 ± 8% perf-profile.calltrace.cycles-pp.xfs_dialloc.xfs_create.xfs_generic_create.vfs_mkdir.do_mkdirat 9.80 ± 6% -4.6 5.22 ± 6% perf-profile.calltrace.cycles-pp.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_inodegc_worker 9.03 ± 7% -4.0 5.02 ± 8% perf-profile.calltrace.cycles-pp.xfs_check_agi_freecount.xfs_dialloc.xfs_create.xfs_generic_create.vfs_mkdir 50.94 -3.7 47.23 ± 2% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 8.25 ± 4% -3.7 4.54 ± 4% perf-profile.calltrace.cycles-pp.rmdir 50.42 -3.7 46.74 ± 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 50.40 -3.7 46.72 ± 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 50.42 -3.7 46.74 ± 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 8.16 ± 4% -3.7 4.50 ± 4% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.rmdir 8.15 ± 4% -3.7 4.50 ± 4% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.rmdir 8.05 ± 4% -3.6 4.44 ± 3% perf-profile.calltrace.cycles-pp.do_rmdir.do_syscall_64.entry_SYSCALL_64_after_hwframe.rmdir 47.43 ± 2% -3.6 43.85 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 7.68 ± 4% -3.5 4.19 ± 4% perf-profile.calltrace.cycles-pp.vfs_rmdir.do_rmdir.do_syscall_64.entry_SYSCALL_64_after_hwframe.rmdir 47.03 ± 2% -3.5 43.58 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 7.17 ± 4% -3.3 3.83 ± 4% perf-profile.calltrace.cycles-pp.xfs_vn_unlink.vfs_rmdir.do_rmdir.do_syscall_64.entry_SYSCALL_64_after_hwframe 7.16 ± 4% -3.3 3.82 ± 4% perf-profile.calltrace.cycles-pp.xfs_remove.xfs_vn_unlink.vfs_rmdir.do_rmdir.do_syscall_64 8.49 ± 6% -3.3 5.16 ± 7% perf-profile.calltrace.cycles-pp.mkdirat 8.45 ± 6% -3.3 5.14 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mkdirat 8.45 ± 6% -3.3 5.14 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdirat 8.44 ± 7% -3.3 5.13 ± 7% perf-profile.calltrace.cycles-pp.mkdir 8.41 ± 7% -3.3 5.12 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mkdir 8.42 ± 6% -3.3 5.13 ± 7% perf-profile.calltrace.cycles-pp.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdirat 8.41 ± 7% -3.3 5.12 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdir 8.38 ± 6% -3.3 5.11 ± 7% perf-profile.calltrace.cycles-pp.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdir 8.26 ± 6% -3.2 5.02 ± 7% perf-profile.calltrace.cycles-pp.vfs_mkdir.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdirat 8.18 ± 7% -3.2 4.99 ± 7% perf-profile.calltrace.cycles-pp.vfs_mkdir.do_mkdirat.do_syscall_64.entry_SYSCALL_64_after_hwframe.mkdir 7.51 ± 8% -3.0 4.46 ± 5% perf-profile.calltrace.cycles-pp.xfs_difree_inobt.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive 7.01 ± 8% -2.9 4.16 ± 5% perf-profile.calltrace.cycles-pp.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree.xfs_ifree.xfs_inactive_ifree 5.32 ± 7% -2.4 2.91 ± 8% perf-profile.calltrace.cycles-pp.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_dialloc.xfs_create.xfs_generic_create 42.52 ± 2% -2.3 40.24 ± 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 2.92 ± 4% -1.9 1.06 ± 6% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_inactive_ifree.xfs_inactive.xfs_inodegc_worker.process_one_work 2.82 ± 4% -1.8 1.00 ± 6% perf-profile.calltrace.cycles-pp.xlog_cil_commit.__xfs_trans_commit.xfs_inactive_ifree.xfs_inactive.xfs_inodegc_worker 4.31 ± 8% -1.8 2.56 ± 5% perf-profile.calltrace.cycles-pp.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree.xfs_ifree 2.23 ± 4% -1.5 0.72 ± 18% perf-profile.calltrace.cycles-pp.xfs_difree_finobt.xfs_difree.xfs_ifree.xfs_inactive_ifree.xfs_inactive 2.73 ± 5% -1.4 1.33 ± 5% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_remove.xfs_vn_unlink.vfs_rmdir.do_rmdir 2.63 ± 5% -1.4 1.25 ± 4% perf-profile.calltrace.cycles-pp.xlog_cil_commit.__xfs_trans_commit.xfs_remove.xfs_vn_unlink.vfs_rmdir 1.51 ± 6% -1.3 0.26 ±100% perf-profile.calltrace.cycles-pp.xlog_cil_insert_items.xlog_cil_commit.__xfs_trans_commit.xfs_inactive_ifree.xfs_inactive 2.75 ± 6% -1.2 1.51 ± 10% perf-profile.calltrace.cycles-pp.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_dialloc.xfs_create 2.73 ± 7% -1.2 1.50 ± 8% perf-profile.calltrace.cycles-pp.xfs_btree_increment.xfs_check_agi_freecount.xfs_dialloc.xfs_create.xfs_generic_create 4.33 ± 4% -1.2 3.10 ± 13% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 3.88 ± 5% -1.1 2.81 ± 13% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 1.49 ± 8% -1.1 0.42 ±108% perf-profile.calltrace.cycles-pp.xfs_dir_createname.xfs_create.xfs_generic_create.vfs_mkdir.do_mkdirat 1.59 ± 5% -1.1 0.54 ± 3% perf-profile.calltrace.cycles-pp.xfs_iunlink.xfs_remove.xfs_vn_unlink.vfs_rmdir.do_rmdir 2.20 ± 9% -0.9 1.25 ± 6% perf-profile.calltrace.cycles-pp.xfs_btree_increment.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree.xfs_ifree 2.21 ± 10% -0.9 1.28 ± 7% perf-profile.calltrace.cycles-pp.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree 1.38 ± 6% -0.8 0.59 ± 8% perf-profile.calltrace.cycles-pp.xfs_iunlink_remove.xfs_ifree.xfs_inactive_ifree.xfs_inactive.xfs_inodegc_worker 2.53 ± 7% -0.8 1.76 ± 8% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_create.xfs_generic_create.vfs_mkdir.do_mkdirat 2.48 ± 7% -0.8 1.73 ± 8% perf-profile.calltrace.cycles-pp.xlog_cil_commit.__xfs_trans_commit.xfs_create.xfs_generic_create.vfs_mkdir 2.12 ± 5% -0.7 1.42 ± 5% perf-profile.calltrace.cycles-pp.xfs_dir_removename.xfs_remove.xfs_vn_unlink.vfs_rmdir.do_rmdir 1.27 ± 7% -0.7 0.60 ± 3% perf-profile.calltrace.cycles-pp.xlog_cil_insert_items.xlog_cil_commit.__xfs_trans_commit.xfs_remove.xfs_vn_unlink 2.57 ± 7% -0.7 1.90 ± 14% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 2.53 ± 7% -0.7 1.88 ± 14% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state 1.94 ± 5% -0.6 1.32 ± 6% perf-profile.calltrace.cycles-pp.xfs_dir2_node_removename.xfs_dir_removename.xfs_remove.xfs_vn_unlink.vfs_rmdir 1.21 ± 9% -0.6 0.65 ± 6% perf-profile.calltrace.cycles-pp.xfs_btree_check_sblock.xfs_btree_increment.xfs_check_agi_freecount.xfs_difree_inobt.xfs_difree 1.22 ± 12% -0.5 0.68 ± 5% perf-profile.calltrace.cycles-pp.xfs_btree_check_sblock.xfs_btree_get_rec.xfs_inobt_get_rec.xfs_check_agi_freecount.xfs_difree_inobt 1.09 ± 6% -0.3 0.75 ± 7% perf-profile.calltrace.cycles-pp.xfs_da3_node_lookup_int.xfs_dir2_node_removename.xfs_dir_removename.xfs_remove.xfs_vn_unlink 0.00 +0.6 0.64 ± 4% perf-profile.calltrace.cycles-pp.xfs_inodegc_queue_all.xfs_inodegc_flush.xfs_fs_statfs.statfs_by_dentry.vfs_statfs 0.00 +0.8 0.75 ± 4% perf-profile.calltrace.cycles-pp.try_to_wake_up.swake_up_locked.complete.process_one_work.worker_thread 0.00 +0.8 0.78 ± 6% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.wait_for_completion.__flush_work 0.00 +0.8 0.79 ± 6% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_for_completion.__flush_work.xfs_inodegc_flush 0.00 +0.8 0.82 ± 3% perf-profile.calltrace.cycles-pp.swake_up_locked.complete.process_one_work.worker_thread.kthread 0.00 +0.8 0.84 ± 6% perf-profile.calltrace.cycles-pp.schedule_timeout.wait_for_completion.__flush_work.xfs_inodegc_flush.xfs_fs_statfs 0.00 +0.8 0.85 ± 3% perf-profile.calltrace.cycles-pp.complete.process_one_work.worker_thread.kthread.ret_from_fork 0.00 +1.0 0.96 ± 5% perf-profile.calltrace.cycles-pp.wait_for_completion.__flush_work.xfs_inodegc_flush.xfs_fs_statfs.statfs_by_dentry 0.00 +1.8 1.76 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__percpu_counter_sum.xfs_fs_statfs.statfs_by_dentry 0.07 ±264% +2.1 2.14 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__percpu_counter_sum.xfs_fs_statfs.statfs_by_dentry.vfs_statfs 2.95 ± 4% +9.6 12.58 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs.statvfs64 2.95 ± 4% +9.6 12.59 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__statfs.statvfs64 2.94 ± 4% +9.6 12.58 ± 2% perf-profile.calltrace.cycles-pp.__do_sys_statfs.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs.statvfs64 2.95 ± 4% +9.7 12.61 ± 2% perf-profile.calltrace.cycles-pp.__statfs.statvfs64 2.96 ± 4% +9.7 12.62 ± 2% perf-profile.calltrace.cycles-pp.statvfs64 3.64 ± 8% +10.9 14.57 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs 3.65 ± 8% +10.9 14.59 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__statfs 3.56 ± 4% +10.9 14.50 ± 2% perf-profile.calltrace.cycles-pp.__do_sys_statfs.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs 3.74 ± 8% +11.0 14.75 ± 2% perf-profile.calltrace.cycles-pp.__statfs 0.00 +14.5 14.46 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.__flush_work.xfs_inodegc_flush.xfs_fs_statfs.statfs_by_dentry 0.00 +18.7 18.73 ± 2% perf-profile.calltrace.cycles-pp.__flush_work.xfs_inodegc_flush.xfs_fs_statfs.statfs_by_dentry.vfs_statfs 0.00 +20.5 20.54 ± 2% perf-profile.calltrace.cycles-pp.xfs_inodegc_flush.xfs_fs_statfs.statfs_by_dentry.vfs_statfs.user_statfs 6.41 ± 4% +20.6 27.04 ± 2% perf-profile.calltrace.cycles-pp.user_statfs.__do_sys_statfs.do_syscall_64.entry_SYSCALL_64_after_hwframe.__statfs 5.58 ± 5% +20.8 26.36 ± 2% perf-profile.calltrace.cycles-pp.statfs_by_dentry.vfs_statfs.user_statfs.__do_sys_statfs.do_syscall_64 5.59 ± 5% +20.8 26.37 ± 2% perf-profile.calltrace.cycles-pp.vfs_statfs.user_statfs.__do_sys_statfs.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.54 ± 5% +20.8 26.34 ± 2% perf-profile.calltrace.cycles-pp.xfs_fs_statfs.statfs_by_dentry.vfs_statfs.user_statfs.__do_sys_statfs 17.82 ± 5% -8.2 9.61 ± 4% perf-profile.children.cycles-pp.xfs_check_agi_freecount 14.76 ± 3% -7.4 7.34 ± 6% perf-profile.children.cycles-pp.xfs_inodegc_worker 14.62 ± 3% -7.4 7.26 ± 6% perf-profile.children.cycles-pp.xfs_inactive 14.59 ± 3% -7.3 7.24 ± 6% perf-profile.children.cycles-pp.xfs_inactive_ifree 15.54 ± 3% -6.7 8.89 ± 5% perf-profile.children.cycles-pp.process_one_work 16.80 ± 6% -6.6 10.23 ± 7% perf-profile.children.cycles-pp.do_mkdirat 16.44 ± 6% -6.4 10.01 ± 7% perf-profile.children.cycles-pp.vfs_mkdir 16.41 ± 6% -6.4 9.99 ± 7% perf-profile.children.cycles-pp.xfs_generic_create 16.29 ± 6% -6.4 9.90 ± 7% perf-profile.children.cycles-pp.xfs_create 15.63 ± 3% -6.3 9.37 ± 4% perf-profile.children.cycles-pp.kthread 15.64 ± 3% -6.3 9.37 ± 4% perf-profile.children.cycles-pp.ret_from_fork 15.56 ± 3% -6.2 9.33 ± 4% perf-profile.children.cycles-pp.worker_thread 11.44 ± 5% -5.4 6.04 ± 6% perf-profile.children.cycles-pp.xfs_ifree 11.03 ± 5% -5.1 5.89 ± 4% perf-profile.children.cycles-pp.xfs_inobt_get_rec 10.77 ± 6% -4.7 6.11 ± 8% perf-profile.children.cycles-pp.xfs_dialloc 9.80 ± 6% -4.6 5.22 ± 6% perf-profile.children.cycles-pp.xfs_difree 8.32 ± 2% -4.1 4.22 ± 4% perf-profile.children.cycles-pp.__xfs_trans_commit 8.08 ± 2% -4.0 4.06 ± 4% perf-profile.children.cycles-pp.xlog_cil_commit 50.94 -3.7 47.22 ± 2% perf-profile.children.cycles-pp.do_idle 8.26 ± 4% -3.7 4.55 ± 4% perf-profile.children.cycles-pp.rmdir 50.94 -3.7 47.23 ± 2% perf-profile.children.cycles-pp.secondary_startup_64_no_verify 50.94 -3.7 47.23 ± 2% perf-profile.children.cycles-pp.cpu_startup_entry 50.42 -3.7 46.74 ± 3% perf-profile.children.cycles-pp.start_secondary 47.92 ± 2% -3.6 44.31 ± 2% perf-profile.children.cycles-pp.cpuidle_enter 47.91 ± 2% -3.6 44.30 ± 2% perf-profile.children.cycles-pp.cpuidle_enter_state 8.05 ± 4% -3.6 4.44 ± 3% perf-profile.children.cycles-pp.do_rmdir 7.68 ± 4% -3.5 4.19 ± 4% perf-profile.children.cycles-pp.vfs_rmdir 7.17 ± 4% -3.3 3.83 ± 4% perf-profile.children.cycles-pp.xfs_vn_unlink 7.16 ± 4% -3.3 3.82 ± 4% perf-profile.children.cycles-pp.xfs_remove 8.49 ± 6% -3.3 5.16 ± 7% perf-profile.children.cycles-pp.mkdirat 8.45 ± 7% -3.3 5.14 ± 7% perf-profile.children.cycles-pp.mkdir 6.66 ± 5% -3.3 3.38 ± 3% perf-profile.children.cycles-pp.xfs_btree_check_sblock 7.51 ± 8% -3.0 4.47 ± 5% perf-profile.children.cycles-pp.xfs_difree_inobt 5.72 ± 6% -2.7 3.02 ± 6% perf-profile.children.cycles-pp.xfs_btree_get_rec 5.56 ± 5% -2.6 2.91 ± 4% perf-profile.children.cycles-pp.xfs_btree_increment 42.75 ± 2% -2.2 40.56 ± 2% perf-profile.children.cycles-pp.intel_idle 4.13 ± 4% -2.1 2.00 ± 3% perf-profile.children.cycles-pp.__xfs_btree_check_sblock 4.10 ± 3% -2.1 2.04 ± 4% perf-profile.children.cycles-pp.xlog_cil_insert_items 5.01 ± 4% -2.0 2.97 ± 3% perf-profile.children.cycles-pp.xfs_trans_read_buf_map 3.72 ± 3% -1.6 2.08 ± 3% perf-profile.children.cycles-pp.xfs_buf_read_map 3.65 ± 3% -1.6 2.05 ± 3% perf-profile.children.cycles-pp.xfs_buf_get_map 2.23 ± 4% -1.5 0.72 ± 18% perf-profile.children.cycles-pp.xfs_difree_finobt 2.52 ± 3% -1.5 1.02 ± 6% perf-profile.children.cycles-pp.xfs_buf_item_format 3.39 ± 4% -1.5 1.91 ± 3% perf-profile.children.cycles-pp.xfs_buf_find 2.41 ± 3% -1.4 0.98 ± 7% perf-profile.children.cycles-pp.xfs_buf_item_format_segment 4.64 ± 4% -1.2 3.41 ± 13% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 2.24 ± 3% -1.1 1.14 ± 3% perf-profile.children.cycles-pp.xfs_read_agi 1.60 ± 5% -1.1 0.54 ± 3% perf-profile.children.cycles-pp.xfs_iunlink 4.14 ± 4% -1.1 3.08 ± 13% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 1.86 ± 4% -1.0 0.88 ± 6% perf-profile.children.cycles-pp.memcpy_erms 1.38 ± 6% -0.8 0.59 ± 8% perf-profile.children.cycles-pp.xfs_iunlink_remove 1.35 ± 8% -0.8 0.59 ± 7% perf-profile.children.cycles-pp.xfs_verify_agino 1.35 ± 5% -0.7 0.64 ± 7% perf-profile.children.cycles-pp.xfs_buf_item_release 2.12 ± 5% -0.7 1.42 ± 5% perf-profile.children.cycles-pp.xfs_dir_removename 1.18 ± 2% -0.7 0.51 ± 4% perf-profile.children.cycles-pp.xfs_buf_lock 1.54 ± 7% -0.7 0.88 ± 4% perf-profile.children.cycles-pp.xfs_errortag_test 1.15 ± 2% -0.7 0.50 ± 5% perf-profile.children.cycles-pp.down 1.23 ± 6% -0.7 0.58 ± 8% perf-profile.children.cycles-pp.xfs_buf_unlock 1.93 ± 5% -0.7 1.28 ± 3% perf-profile.children.cycles-pp.xfs_btree_lookup 2.76 ± 6% -0.6 2.11 ± 13% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 2.72 ± 6% -0.6 2.08 ± 13% perf-profile.children.cycles-pp.hrtimer_interrupt 1.13 ± 2% -0.6 0.49 ± 4% perf-profile.children.cycles-pp.__down 1.94 ± 5% -0.6 1.32 ± 6% perf-profile.children.cycles-pp.xfs_dir2_node_removename 1.17 ± 6% -0.6 0.55 ± 9% perf-profile.children.cycles-pp.up 1.15 ± 6% -0.6 0.58 ± 5% perf-profile.children.cycles-pp.xfs_inobt_btrec_to_irec 1.49 ± 8% -0.5 0.95 ± 5% perf-profile.children.cycles-pp.xfs_dir_createname 1.60 ± 3% -0.5 1.07 ± 6% perf-profile.children.cycles-pp.xfs_da3_node_lookup_int 1.59 ± 4% -0.5 1.11 ± 4% perf-profile.children.cycles-pp.xfs_da_read_buf 1.37 ± 6% -0.5 0.90 ± 5% perf-profile.children.cycles-pp.xfs_inobt_irec_to_allocmask 0.66 ± 4% -0.5 0.20 ± 10% perf-profile.children.cycles-pp.xfs_iunlink_update_inode 1.27 ± 8% -0.5 0.82 ± 5% perf-profile.children.cycles-pp.xfs_dir2_node_addname 1.04 ± 6% -0.4 0.60 ± 6% perf-profile.children.cycles-pp.xfs_next_bit 1.18 ± 12% -0.4 0.76 ± 35% perf-profile.children.cycles-pp.ktime_get 1.32 ± 6% -0.4 0.90 ± 3% perf-profile.children.cycles-pp.xfs_btree_lookup_get_block 0.70 ± 5% -0.4 0.31 ± 7% perf-profile.children.cycles-pp.xfs_imap_to_bp 0.50 ± 10% -0.4 0.15 ± 14% perf-profile.children.cycles-pp.xfs_buf_item_size 1.14 ± 6% -0.3 0.79 ± 3% perf-profile.children.cycles-pp.xfs_btree_read_buf_block 0.86 ± 6% -0.3 0.54 ± 7% perf-profile.children.cycles-pp.xfs_dir3_data_check 0.86 ± 7% -0.3 0.53 ± 7% perf-profile.children.cycles-pp.__xfs_dir3_data_check 0.63 ± 3% -0.3 0.31 ± 6% perf-profile.children.cycles-pp.xfs_inobt_get_maxrecs 0.66 ± 6% -0.3 0.35 ± 12% perf-profile.children.cycles-pp.xfs_btree_check_block 0.39 ± 12% -0.3 0.08 ± 20% perf-profile.children.cycles-pp.xfs_buf_item_size_segment 0.92 ± 9% -0.3 0.62 ± 5% perf-profile.children.cycles-pp.xfs_perag_get 1.74 ± 5% -0.3 1.45 ± 4% perf-profile.children.cycles-pp._raw_spin_lock 1.03 ± 6% -0.3 0.76 ± 5% perf-profile.children.cycles-pp._xfs_trans_bjoin 0.68 ± 6% -0.3 0.42 ± 7% perf-profile.children.cycles-pp.xfs_ialloc_read_agi 0.96 ± 10% -0.3 0.71 ± 10% perf-profile.children.cycles-pp.xfs_init_new_inode 0.56 ± 9% -0.2 0.31 ± 8% perf-profile.children.cycles-pp.xfs_btree_rec_offset 0.71 ± 6% -0.2 0.47 ± 6% perf-profile.children.cycles-pp.xfs_dir2_leafn_remove 0.78 ± 5% -0.2 0.54 ± 6% perf-profile.children.cycles-pp.xfs_da3_node_read 0.65 ± 6% -0.2 0.43 ± 7% perf-profile.children.cycles-pp.xfs_dir2_leafn_lookup_for_entry 0.43 ± 12% -0.2 0.21 ± 8% perf-profile.children.cycles-pp.xfs_btree_get_block 0.60 ± 7% -0.2 0.39 ± 6% perf-profile.children.cycles-pp.xfs_inode_item_format 0.59 ± 11% -0.2 0.39 ± 6% perf-profile.children.cycles-pp.xfs_dir2_node_addname_int 0.49 ± 10% -0.2 0.33 ± 11% perf-profile.children.cycles-pp.xfs_iget 0.25 ± 9% -0.2 0.09 ± 7% perf-profile.children.cycles-pp.kmem_alloc 0.61 ± 6% -0.2 0.45 ± 5% perf-profile.children.cycles-pp.filename_lookup 0.50 ± 7% -0.2 0.34 ± 6% perf-profile.children.cycles-pp.xfs_buf_rele 0.44 ± 7% -0.2 0.28 ± 11% perf-profile.children.cycles-pp.xfs_verify_dir_ino 0.41 ± 11% -0.2 0.26 ± 8% perf-profile.children.cycles-pp.xfs_dabuf_map 0.24 ± 8% -0.2 0.08 ± 8% perf-profile.children.cycles-pp.__kmalloc 0.67 ± 4% -0.1 0.53 ± 4% perf-profile.children.cycles-pp.irq_exit_rcu 0.44 ± 11% -0.1 0.29 ± 11% perf-profile.children.cycles-pp.__xstat64 0.54 ± 7% -0.1 0.39 ± 5% perf-profile.children.cycles-pp.path_lookupat 0.37 ± 18% -0.1 0.23 ± 16% perf-profile.children.cycles-pp.xlog_ioend_work 0.37 ± 18% -0.1 0.23 ± 17% perf-profile.children.cycles-pp.xlog_cil_process_committed 0.37 ± 18% -0.1 0.23 ± 17% perf-profile.children.cycles-pp.xlog_cil_committed 0.37 ± 18% -0.1 0.23 ± 16% perf-profile.children.cycles-pp.xlog_state_do_callback 0.32 ± 10% -0.1 0.18 ± 10% perf-profile.children.cycles-pp.xfs_btree_readahead 0.58 ± 4% -0.1 0.44 ± 4% perf-profile.children.cycles-pp.__softirqentry_text_start 0.39 ± 9% -0.1 0.26 ± 5% perf-profile.children.cycles-pp.__xfs_dir3_free_read 0.28 ± 29% -0.1 0.15 ± 23% perf-profile.children.cycles-pp.xlog_cil_push_work 0.39 ± 9% -0.1 0.26 perf-profile.children.cycles-pp.__list_del_entry_valid 0.32 ± 6% -0.1 0.19 ± 6% perf-profile.children.cycles-pp.xfs_perag_put 0.38 ± 6% -0.1 0.26 ± 9% perf-profile.children.cycles-pp.kmem_cache_alloc 0.37 ± 7% -0.1 0.24 ± 11% perf-profile.children.cycles-pp.getname_flags 0.40 ± 7% -0.1 0.27 ± 11% perf-profile.children.cycles-pp.__do_sys_newstat 0.25 ± 14% -0.1 0.13 ± 8% perf-profile.children.cycles-pp.d_lookup 0.25 ± 14% -0.1 0.13 ± 6% perf-profile.children.cycles-pp.__lookup_hash 0.25 ± 14% -0.1 0.13 ± 6% perf-profile.children.cycles-pp.lookup_dcache 0.25 ± 15% -0.1 0.13 ± 8% perf-profile.children.cycles-pp.__d_lookup 0.37 ± 7% -0.1 0.25 ± 13% perf-profile.children.cycles-pp.vfs_statx 0.46 ± 12% -0.1 0.34 ± 7% perf-profile.children.cycles-pp.xfs_buf_item_init 0.32 ± 8% -0.1 0.21 ± 12% perf-profile.children.cycles-pp.xfs_trans_log_buf 0.33 ± 11% -0.1 0.21 ± 9% perf-profile.children.cycles-pp.xfs_bmapi_read 0.14 ± 17% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__slab_free 0.38 ± 8% -0.1 0.26 ± 2% perf-profile.children.cycles-pp.vsnprintf 0.30 ± 9% -0.1 0.19 ± 9% perf-profile.children.cycles-pp.xfs_dialloc_ag_update_inobt 0.57 ± 7% -0.1 0.46 ± 10% perf-profile.children.cycles-pp.xfs_trans_log_inode 0.26 ± 8% -0.1 0.15 ± 14% perf-profile.children.cycles-pp.xfs_lookup_get_search_key 0.29 ± 17% -0.1 0.18 ± 17% perf-profile.children.cycles-pp.xfs_trans_committed_bulk 0.28 ± 10% -0.1 0.18 ± 10% perf-profile.children.cycles-pp.user_path_at_empty 0.27 ± 10% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.filename_create 0.22 ± 33% -0.1 0.11 ± 27% perf-profile.children.cycles-pp.xlog_write 0.28 ± 9% -0.1 0.18 ± 5% perf-profile.children.cycles-pp.xfs_inobt_update 0.21 ± 27% -0.1 0.11 ± 14% perf-profile.children.cycles-pp.irq_enter_rcu 0.38 ± 4% -0.1 0.28 ± 4% perf-profile.children.cycles-pp.xfs_dir3_data_read 0.27 ± 8% -0.1 0.18 ± 11% perf-profile.children.cycles-pp.xfs_trans_dirty_buf 0.25 ± 10% -0.1 0.15 ± 12% perf-profile.children.cycles-pp.strncpy_from_user 0.52 ± 5% -0.1 0.43 ± 5% perf-profile.children.cycles-pp.xfs_trans_alloc 0.20 ± 31% -0.1 0.10 ± 14% perf-profile.children.cycles-pp.tick_irq_enter 0.22 ± 7% -0.1 0.12 ± 11% perf-profile.children.cycles-pp.xfs_defer_finish_noroll 0.26 ± 7% -0.1 0.17 ± 4% perf-profile.children.cycles-pp.xfs_trans_del_item 0.29 ± 8% -0.1 0.20 ± 4% perf-profile.children.cycles-pp.vfprintf 0.26 ± 11% -0.1 0.17 ± 4% perf-profile.children.cycles-pp.xfs_btree_update 0.23 ± 9% -0.1 0.14 ± 5% perf-profile.children.cycles-pp.xfs_log_ticket_ungrant 0.19 ± 9% -0.1 0.10 ± 10% perf-profile.children.cycles-pp.xfs_buf_trylock 0.17 ± 20% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.ktime_get_update_offsets_now 0.17 ± 9% -0.1 0.10 ± 10% perf-profile.children.cycles-pp.down_trylock 0.17 ± 18% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.asm_sysvec_irq_work 0.17 ± 18% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.sysvec_irq_work 0.17 ± 18% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.__sysvec_irq_work 0.17 ± 18% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.irq_work_run 0.17 ± 18% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.irq_work_single 0.17 ± 18% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.printk 0.17 ± 18% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.vprintk_emit 0.17 ± 18% -0.1 0.10 ± 18% perf-profile.children.cycles-pp.console_unlock 0.13 ± 16% -0.1 0.05 ± 46% perf-profile.children.cycles-pp.xfs_trans_add_item 0.19 ± 10% -0.1 0.11 ± 12% perf-profile.children.cycles-pp.xfs_trans_ijoin 0.18 ± 17% -0.1 0.10 ± 17% perf-profile.children.cycles-pp.irq_work_run_list 0.16 ± 18% -0.1 0.09 ± 16% perf-profile.children.cycles-pp.wait_for_xmitr 0.16 ± 17% -0.1 0.09 ± 15% perf-profile.children.cycles-pp.serial8250_console_write 0.16 ± 18% -0.1 0.08 ± 20% perf-profile.children.cycles-pp.serial8250_console_putchar 0.17 ± 7% -0.1 0.10 ± 11% perf-profile.children.cycles-pp.xfs_inobt_init_cursor 0.10 ± 20% -0.1 0.03 ±101% perf-profile.children.cycles-pp.rcu_core 0.16 ± 19% -0.1 0.09 ± 16% perf-profile.children.cycles-pp.uart_console_write 0.13 ± 12% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.kfree 0.26 ± 8% -0.1 0.19 ± 9% perf-profile.children.cycles-pp.link_path_walk 0.14 ± 10% -0.1 0.08 ± 8% perf-profile.children.cycles-pp.kmem_cache_free 0.16 ± 7% -0.1 0.10 ± 13% perf-profile.children.cycles-pp.xfs_inobt_init_common 0.10 ± 6% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.xfs_btree_check_ptr 0.12 ± 9% -0.1 0.06 ± 46% perf-profile.children.cycles-pp.xfs_trans_buf_item_match 0.41 ± 5% -0.1 0.34 ± 8% perf-profile.children.cycles-pp.xfs_trans_reserve 0.14 ± 10% -0.1 0.07 ± 10% perf-profile.children.cycles-pp.xfs_btree_ptr_to_daddr 0.10 ± 14% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.xfs_defer_trans_roll 0.12 ± 13% -0.1 0.06 ± 8% perf-profile.children.cycles-pp.__up 0.12 ± 22% -0.1 0.06 ± 21% perf-profile.children.cycles-pp.io_serial_in 0.13 ± 18% -0.1 0.07 ± 16% perf-profile.children.cycles-pp.__raw_callee_save___native_queued_spin_unlock 0.20 ± 5% -0.1 0.14 ± 13% perf-profile.children.cycles-pp.down_write 0.14 ± 22% -0.1 0.09 ± 15% perf-profile.children.cycles-pp.xfs_iext_lookup_extent 0.23 ± 10% -0.1 0.17 ± 5% perf-profile.children.cycles-pp.complete_walk 0.14 ± 11% -0.1 0.08 ± 12% perf-profile.children.cycles-pp.xfs_trans_roll 0.16 ± 13% -0.1 0.11 ± 19% perf-profile.children.cycles-pp.xfs_inobt_init_key_from_rec 0.24 ± 6% -0.1 0.18 ± 21% perf-profile.children.cycles-pp.native_irq_return_iret 0.11 ± 13% -0.1 0.05 ± 48% perf-profile.children.cycles-pp._atomic_dec_and_lock 0.08 ± 10% -0.1 0.03 ±100% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.12 ± 10% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.memset_erms 0.09 ± 23% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.xfs_btree_del_cursor 0.08 ± 23% -0.1 0.02 ± 99% perf-profile.children.cycles-pp.do_statfs_native 0.09 ± 15% -0.1 0.04 ± 77% perf-profile.children.cycles-pp.path_init 0.09 ± 12% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.xfs_buf_item_put 0.09 ± 16% -0.0 0.04 ± 75% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.22 ± 12% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.try_to_unlazy 0.16 ± 6% -0.0 0.11 ± 13% perf-profile.children.cycles-pp.xfs_ialloc_ag_alloc 0.12 ± 15% -0.0 0.07 ± 12% perf-profile.children.cycles-pp.evict 0.12 ± 17% -0.0 0.07 ± 18% perf-profile.children.cycles-pp.xfs_inode_item_format_data_fork 0.15 ± 7% -0.0 0.10 ± 13% perf-profile.children.cycles-pp.irqtime_account_irq 0.20 ± 8% -0.0 0.16 ± 15% perf-profile.children.cycles-pp.read_tsc 0.12 ± 13% -0.0 0.08 ± 16% perf-profile.children.cycles-pp.__check_object_size 0.18 ± 10% -0.0 0.14 ± 7% perf-profile.children.cycles-pp.path_parentat 0.08 ± 24% -0.0 0.04 ± 45% perf-profile.children.cycles-pp._copy_to_user 0.20 ± 13% -0.0 0.16 ± 6% perf-profile.children.cycles-pp.__legitimize_path 0.10 ± 16% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.xfs_inodegc_set_reclaimable 0.19 ± 10% -0.0 0.15 ± 10% perf-profile.children.cycles-pp.filename_parentat 0.10 ± 22% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.xfs_buf_set_ref 0.10 ± 12% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.xfs_alloc_vextent 0.26 ± 5% -0.0 0.22 ± 7% perf-profile.children.cycles-pp.sched_clock_cpu 0.16 ± 10% -0.0 0.12 ± 18% perf-profile.children.cycles-pp.xfs_droplink 0.16 ± 7% -0.0 0.12 ± 12% perf-profile.children.cycles-pp.walk_component 0.10 ± 15% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.inode_permission 0.08 ± 12% -0.0 0.04 ± 44% perf-profile.children.cycles-pp.xfs_lock_two_inodes 0.13 ± 9% -0.0 0.10 ± 11% perf-profile.children.cycles-pp._IO_default_xsputn 0.08 ± 11% -0.0 0.05 ± 45% perf-profile.children.cycles-pp.__entry_text_start 0.07 ± 9% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.xfs_trans_get_buf_map 0.10 ± 17% -0.0 0.06 ± 17% perf-profile.children.cycles-pp.hrtimer_update_next_event 0.12 ± 10% -0.0 0.09 ± 14% perf-profile.children.cycles-pp.xfs_ilock 0.08 ± 13% -0.0 0.05 ± 46% perf-profile.children.cycles-pp.xfs_ialloc_inode_init 0.06 ± 17% +0.0 0.10 ± 14% perf-profile.children.cycles-pp.__update_load_avg_se 0.04 ± 78% +0.0 0.08 ± 14% perf-profile.children.cycles-pp.update_cfs_group 0.14 ± 12% +0.0 0.19 ± 5% perf-profile.children.cycles-pp.lockref_put_return 0.10 ± 16% +0.1 0.15 ± 16% perf-profile.children.cycles-pp.__switch_to_asm 0.03 ±102% +0.1 0.08 ± 11% perf-profile.children.cycles-pp.perf_tp_event 0.07 ± 19% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template 0.01 ±264% +0.1 0.07 ± 14% perf-profile.children.cycles-pp.available_idle_cpu 0.09 ± 13% +0.1 0.15 ± 15% perf-profile.children.cycles-pp.__switch_to 0.02 ±129% +0.1 0.08 ± 23% perf-profile.children.cycles-pp.__smp_call_single_queue 0.02 ±129% +0.1 0.09 ± 15% perf-profile.children.cycles-pp.llist_reverse_order 0.01 ±174% +0.1 0.08 ± 14% perf-profile.children.cycles-pp.cpumask_next_and 0.00 +0.1 0.07 ± 23% perf-profile.children.cycles-pp.__sysvec_call_function_single 0.10 ± 16% +0.1 0.17 ± 8% perf-profile.children.cycles-pp.select_task_rq_fair 0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.put_prev_entity 0.00 +0.1 0.08 ± 24% perf-profile.children.cycles-pp.cpuacct_charge 0.00 +0.1 0.08 ± 22% perf-profile.children.cycles-pp.sysvec_call_function_single 0.02 ±129% +0.1 0.10 ± 22% perf-profile.children.cycles-pp.llist_add_batch 0.03 ±100% +0.1 0.11 ± 11% perf-profile.children.cycles-pp.finish_task_switch 0.00 +0.1 0.09 ± 20% perf-profile.children.cycles-pp.rcu_read_unlock_strict 0.05 ± 85% +0.1 0.14 ± 16% perf-profile.children.cycles-pp.flush_smp_call_function_queue 0.00 +0.1 0.10 ± 17% perf-profile.children.cycles-pp.asm_sysvec_call_function_single 0.00 +0.1 0.11 ± 6% perf-profile.children.cycles-pp.insert_work 0.01 ±264% +0.1 0.12 ± 15% perf-profile.children.cycles-pp.rcu_all_qs 0.23 ± 13% +0.1 0.34 ± 7% perf-profile.children.cycles-pp.update_load_avg 0.25 ± 6% +0.1 0.37 ± 11% perf-profile.children.cycles-pp.dequeue_entity 0.13 ± 11% +0.1 0.25 ± 9% perf-profile.children.cycles-pp.update_curr 0.12 ± 35% +0.1 0.26 ± 11% perf-profile.children.cycles-pp.ttwu_queue_wakelist 0.30 ± 5% +0.1 0.43 ± 9% perf-profile.children.cycles-pp.dequeue_task_fair 0.00 +0.1 0.14 ± 34% perf-profile.children.cycles-pp.get_work_pool 0.24 ± 13% +0.1 0.39 ± 5% perf-profile.children.cycles-pp.enqueue_entity 0.09 ± 14% +0.2 0.26 ± 7% perf-profile.children.cycles-pp.__might_sleep 0.15 ± 9% +0.2 0.34 ± 8% perf-profile.children.cycles-pp.___might_sleep 0.14 ± 53% +0.2 0.33 ± 14% perf-profile.children.cycles-pp.sched_ttwu_pending 0.00 +0.2 0.19 ± 7% perf-profile.children.cycles-pp.preempt_schedule_common 0.31 ± 10% +0.2 0.51 ± 5% perf-profile.children.cycles-pp.enqueue_task_fair 0.32 ± 10% +0.2 0.53 ± 6% perf-profile.children.cycles-pp.ttwu_do_activate 0.00 +0.2 0.21 ± 3% perf-profile.children.cycles-pp.__queue_work 0.00 +0.2 0.24 ± 4% perf-profile.children.cycles-pp.queue_work_on 0.25 ± 45% +0.3 0.50 ± 12% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle 0.08 ± 14% +0.4 0.44 ± 6% perf-profile.children.cycles-pp.__cond_resched 0.34 ± 8% +0.4 0.73 ± 8% perf-profile.children.cycles-pp.update_sd_lb_stats 0.34 ± 8% +0.4 0.74 ± 8% perf-profile.children.cycles-pp.find_busiest_group 0.39 ± 8% +0.4 0.80 ± 7% perf-profile.children.cycles-pp.load_balance 0.28 ± 8% +0.5 0.76 ± 9% perf-profile.children.cycles-pp.newidle_balance 0.98 ± 3% +0.5 1.52 ± 3% perf-profile.children.cycles-pp.schedule_timeout 0.87 ± 7% +0.5 1.42 ± 4% perf-profile.children.cycles-pp.try_to_wake_up 0.47 ± 9% +0.6 1.10 ± 6% perf-profile.children.cycles-pp.pick_next_task_fair 0.95 ± 3% +0.8 1.72 ± 3% perf-profile.children.cycles-pp.schedule 0.20 ± 6% +0.8 0.98 ± 6% perf-profile.children.cycles-pp.__radix_tree_lookup 0.00 +0.8 0.82 ± 3% perf-profile.children.cycles-pp.swake_up_locked 0.00 +0.8 0.85 ± 3% perf-profile.children.cycles-pp.complete 1.36 ± 7% +1.0 2.39 ± 3% perf-profile.children.cycles-pp.__schedule 0.00 +1.0 1.05 ± 4% perf-profile.children.cycles-pp.xfs_inodegc_queue_all 0.00 +1.2 1.25 ± 5% perf-profile.children.cycles-pp.wait_for_completion 1.35 ± 8% +1.3 2.66 ± 4% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.51 ± 16% +1.9 2.40 ± 5% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 2.96 ± 4% +9.7 12.62 ± 2% perf-profile.children.cycles-pp.statvfs64 32.29 ± 3% +10.1 42.40 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 32.25 ± 3% +10.1 42.36 ± 3% perf-profile.children.cycles-pp.do_syscall_64 0.18 ± 10% +14.6 14.83 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.00 +18.9 18.90 ± 2% perf-profile.children.cycles-pp.__flush_work 6.50 ± 4% +20.6 27.08 ± 2% perf-profile.children.cycles-pp.__do_sys_statfs 0.00 +20.6 20.62 ± 2% perf-profile.children.cycles-pp.xfs_inodegc_flush 6.41 ± 4% +20.6 27.03 ± 2% perf-profile.children.cycles-pp.user_statfs 6.72 ± 6% +20.7 27.37 ± 2% perf-profile.children.cycles-pp.__statfs 5.58 ± 5% +20.8 26.36 ± 2% perf-profile.children.cycles-pp.statfs_by_dentry 5.59 ± 5% +20.8 26.38 ± 2% perf-profile.children.cycles-pp.vfs_statfs 5.55 ± 5% +20.8 26.34 ± 2% perf-profile.children.cycles-pp.xfs_fs_statfs 42.75 ± 2% -2.2 40.56 ± 2% perf-profile.self.cycles-pp.intel_idle 3.51 ± 4% -1.8 1.70 ± 4% perf-profile.self.cycles-pp.__xfs_btree_check_sblock 1.83 ± 4% -1.0 0.86 ± 6% perf-profile.self.cycles-pp.memcpy_erms 1.33 ± 8% -0.8 0.58 ± 8% perf-profile.self.cycles-pp.xfs_verify_agino 1.60 ± 7% -0.7 0.92 ± 4% perf-profile.self.cycles-pp.xfs_inobt_get_rec 1.46 ± 9% -0.7 0.81 ± 10% perf-profile.self.cycles-pp.xfs_btree_get_rec 1.47 ± 9% -0.6 0.82 ± 8% perf-profile.self.cycles-pp.xfs_btree_increment 1.50 ± 7% -0.6 0.87 ± 4% perf-profile.self.cycles-pp.xfs_errortag_test 1.08 ± 7% -0.6 0.53 ± 5% perf-profile.self.cycles-pp.xfs_inobt_btrec_to_irec 1.12 ± 9% -0.5 0.59 ± 7% perf-profile.self.cycles-pp.xfs_btree_check_sblock 2.86 ± 6% -0.5 2.32 ± 4% perf-profile.self.cycles-pp.__percpu_counter_sum 0.98 ± 6% -0.4 0.56 ± 6% perf-profile.self.cycles-pp.xfs_next_bit 1.02 ± 15% -0.4 0.63 ± 40% perf-profile.self.cycles-pp.ktime_get 0.62 ± 6% -0.3 0.32 ± 7% perf-profile.self.cycles-pp.xfs_btree_check_block 0.67 ± 7% -0.3 0.39 ± 7% perf-profile.self.cycles-pp.xfs_buf_find 0.53 ± 3% -0.3 0.25 ± 9% perf-profile.self.cycles-pp.xfs_inobt_get_maxrecs 0.40 ± 7% -0.3 0.13 ± 15% perf-profile.self.cycles-pp.xfs_buf_item_format_segment 1.63 ± 4% -0.3 1.37 ± 4% perf-profile.self.cycles-pp._raw_spin_lock 0.60 ± 10% -0.2 0.36 ± 6% perf-profile.self.cycles-pp.xlog_cil_commit 0.77 ± 10% -0.2 0.54 ± 5% perf-profile.self.cycles-pp.xfs_perag_get 0.47 ± 6% -0.2 0.27 ± 10% perf-profile.self.cycles-pp.xfs_btree_rec_offset 0.43 ± 10% -0.2 0.23 ± 9% perf-profile.self.cycles-pp.xfs_check_agi_freecount 0.38 ± 11% -0.2 0.19 ± 10% perf-profile.self.cycles-pp.xfs_btree_get_block 0.50 ± 8% -0.2 0.32 ± 7% perf-profile.self.cycles-pp.xfs_inobt_irec_to_allocmask 0.57 ± 2% -0.2 0.40 ± 15% perf-profile.self.cycles-pp.cpuidle_enter_state 0.40 ± 8% -0.2 0.24 ± 5% perf-profile.self.cycles-pp.__xfs_dir3_data_check 0.42 ± 4% -0.2 0.26 ± 9% perf-profile.self.cycles-pp.xfs_verify_dir_ino 0.44 ± 11% -0.1 0.30 ± 8% perf-profile.self.cycles-pp.xlog_cil_insert_items 0.30 ± 10% -0.1 0.17 ± 13% perf-profile.self.cycles-pp.xfs_btree_readahead 0.39 ± 10% -0.1 0.26 perf-profile.self.cycles-pp.__list_del_entry_valid 0.31 ± 5% -0.1 0.18 ± 6% perf-profile.self.cycles-pp.xfs_perag_put 0.45 ± 11% -0.1 0.33 ± 5% perf-profile.self.cycles-pp.xfs_buf_item_init 0.40 ± 10% -0.1 0.28 ± 5% perf-profile.self.cycles-pp.xfs_inode_item_format 0.49 ± 5% -0.1 0.38 ± 7% perf-profile.self.cycles-pp._xfs_trans_bjoin 0.29 ± 6% -0.1 0.18 ± 5% perf-profile.self.cycles-pp.xfs_btree_lookup 0.14 ± 16% -0.1 0.03 ±100% perf-profile.self.cycles-pp.__slab_free 0.19 ± 17% -0.1 0.09 ± 11% perf-profile.self.cycles-pp.__d_lookup 0.20 ± 12% -0.1 0.11 ± 14% perf-profile.self.cycles-pp.xfs_buf_get_map 0.27 ± 8% -0.1 0.17 ± 12% perf-profile.self.cycles-pp.xfs_trans_dirty_buf 0.28 ± 7% -0.1 0.20 ± 6% perf-profile.self.cycles-pp.vfprintf 0.25 ± 6% -0.1 0.16 ± 5% perf-profile.self.cycles-pp.xfs_trans_del_item 0.22 ± 10% -0.1 0.13 ± 7% perf-profile.self.cycles-pp.xfs_log_ticket_ungrant 0.15 ± 13% -0.1 0.07 ± 19% perf-profile.self.cycles-pp.xfs_trans_read_buf_map 0.14 ± 27% -0.1 0.06 ± 47% perf-profile.self.cycles-pp.ktime_get_update_offsets_now 0.11 ± 14% -0.1 0.03 ± 70% perf-profile.self.cycles-pp.xfs_buf_item_format 0.11 ± 21% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.xfs_trans_add_item 0.12 ± 12% -0.1 0.06 ± 6% perf-profile.self.cycles-pp.kfree 0.18 ± 8% -0.1 0.11 ± 20% perf-profile.self.cycles-pp.xfs_trans_log_inode 0.12 ± 11% -0.1 0.06 ± 46% perf-profile.self.cycles-pp.xfs_trans_buf_item_match 0.10 ± 14% -0.1 0.04 ± 72% perf-profile.self.cycles-pp._atomic_dec_and_lock 0.12 ± 22% -0.1 0.06 ± 21% perf-profile.self.cycles-pp.io_serial_in 0.08 ± 13% -0.1 0.02 ± 99% perf-profile.self.cycles-pp.xfs_dabuf_map 0.12 ± 9% -0.1 0.07 ± 11% perf-profile.self.cycles-pp.kmem_cache_free 0.16 ± 13% -0.1 0.10 ± 22% perf-profile.self.cycles-pp.xfs_inobt_init_key_from_rec 0.24 ± 6% -0.1 0.18 ± 21% perf-profile.self.cycles-pp.native_irq_return_iret 0.12 ± 12% -0.1 0.06 ± 16% perf-profile.self.cycles-pp.memset_erms 0.08 ± 10% -0.1 0.03 ±100% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.08 ± 15% -0.1 0.03 ±100% perf-profile.self.cycles-pp.__raw_callee_save___native_queued_spin_unlock 0.18 ± 7% -0.1 0.12 ± 12% perf-profile.self.cycles-pp.down_write 0.08 ± 14% -0.1 0.03 ±101% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.08 ± 10% -0.1 0.03 ±102% perf-profile.self.cycles-pp.xfs_buf_item_put 0.11 ± 7% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.xfs_buf_item_size 0.17 ± 9% -0.0 0.12 ± 14% perf-profile.self.cycles-pp.kmem_cache_alloc 0.09 ± 14% -0.0 0.05 ± 46% perf-profile.self.cycles-pp.xfs_buf_rele 0.13 ± 16% -0.0 0.08 ± 14% perf-profile.self.cycles-pp.xfs_iext_lookup_extent 0.09 ± 11% -0.0 0.05 ± 47% perf-profile.self.cycles-pp.strncpy_from_user 0.08 ± 17% -0.0 0.04 ± 73% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.20 ± 8% -0.0 0.16 ± 15% perf-profile.self.cycles-pp.read_tsc 0.10 ± 7% -0.0 0.07 ± 9% perf-profile.self.cycles-pp.xfs_bmapi_read 0.12 ± 10% -0.0 0.09 ± 9% perf-profile.self.cycles-pp._IO_default_xsputn 0.06 ± 17% +0.0 0.10 ± 15% perf-profile.self.cycles-pp.__update_load_avg_se 0.06 ± 15% +0.0 0.09 ± 14% perf-profile.self.cycles-pp.try_to_wake_up 0.14 ± 15% +0.0 0.18 ± 6% perf-profile.self.cycles-pp.update_rq_clock 0.03 ± 78% +0.0 0.08 ± 16% perf-profile.self.cycles-pp.update_cfs_group 0.09 ± 15% +0.0 0.14 ± 5% perf-profile.self.cycles-pp.update_load_avg 0.14 ± 12% +0.0 0.19 ± 6% perf-profile.self.cycles-pp.lockref_put_return 0.10 ± 16% +0.0 0.14 ± 15% perf-profile.self.cycles-pp.__switch_to_asm 0.06 ± 14% +0.0 0.11 ± 13% perf-profile.self.cycles-pp.schedule_timeout 0.07 ± 15% +0.1 0.12 ± 11% perf-profile.self.cycles-pp.enqueue_task_fair 0.04 ± 79% +0.1 0.09 ± 7% perf-profile.self.cycles-pp.flush_smp_call_function_from_idle 0.05 ± 62% +0.1 0.11 ± 6% perf-profile.self.cycles-pp.ttwu_queue_wakelist 0.01 ±264% +0.1 0.07 ± 14% perf-profile.self.cycles-pp.available_idle_cpu 0.09 ± 11% +0.1 0.15 ± 17% perf-profile.self.cycles-pp.__switch_to 0.01 ±264% +0.1 0.07 ± 8% perf-profile.self.cycles-pp.perf_tp_event 0.02 ±129% +0.1 0.09 ± 15% perf-profile.self.cycles-pp.llist_reverse_order 0.00 +0.1 0.08 ± 24% perf-profile.self.cycles-pp.cpuacct_charge 0.02 ±129% +0.1 0.10 ± 22% perf-profile.self.cycles-pp.llist_add_batch 0.00 +0.1 0.08 ± 14% perf-profile.self.cycles-pp.rcu_all_qs 0.00 +0.1 0.10 ± 9% perf-profile.self.cycles-pp.insert_work 0.00 +0.1 0.12 ± 8% perf-profile.self.cycles-pp.__cond_resched 0.24 ± 6% +0.1 0.37 ± 7% perf-profile.self.cycles-pp.__schedule 0.00 +0.1 0.14 ± 32% perf-profile.self.cycles-pp.get_work_pool 0.00 +0.2 0.16 ± 13% perf-profile.self.cycles-pp.process_one_work 0.08 ± 16% +0.2 0.24 ± 6% perf-profile.self.cycles-pp.__might_sleep 0.15 ± 10% +0.2 0.32 ± 8% perf-profile.self.cycles-pp.___might_sleep 0.24 ± 10% +0.3 0.56 ± 10% perf-profile.self.cycles-pp.update_sd_lb_stats 0.00 +0.3 0.32 ± 24% perf-profile.self.cycles-pp.xfs_inodegc_flush 0.00 +0.5 0.49 ± 5% perf-profile.self.cycles-pp.xfs_inodegc_queue_all 0.20 ± 6% +0.7 0.94 ± 6% perf-profile.self.cycles-pp.__radix_tree_lookup 0.00 +1.0 0.99 ± 4% perf-profile.self.cycles-pp.__flush_work 0.51 ± 17% +1.9 2.39 ± 4% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.17 ± 9% +14.0 14.22 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irq stress-ng.dir.ops_per_sec 50000 +-------------------------------------------------------------------+ | .+ | 45000 |.++.++.++ +. +.++.++.++.++.+ ++ + +.++.++.+ | | + : + :+ : : : | | + + :: : + + | 40000 |-+ + :+: ++ + .+ :| | O + :+ + + :.++ :| 35000 |-+O O O + ++ + | | O O O | 30000 |-O OO O O O O O O O O O O | | O O O OO O O O O O OO OO OO O O | | | 25000 |-+ O | | | 20000 +-------------------------------------------------------------------+ stress-ng.time.voluntary_context_switches 9e+06 +-------------------------------------------------------------------+ | O | 8e+06 |-+O O O O O | | O O O | 7e+06 |-O OO O O O O O | | O O O OO OO OO O O O O OO OO OO O O | 6e+06 |-+ | | O | 5e+06 |.++.++.++. +. +.++.++.++.++.+ .++.++. +.++.++.+ | | + + + + : | 4e+06 |-+ : | | :.+ +.| 3e+06 |-+ + +.++.++.++.++.+ | | | 2e+06 +-------------------------------------------------------------------+ stress-ng.time.involuntary_context_switches 3e+06 +-----------------------------------------------------------------+ | O O O O OO | 2.5e+06 |-+ O O O O O O O O | |O O O O O OO OO OO O O O O O OO OO OO OO | | | 2e+06 |-+ O | | | 1.5e+06 |-+ | | | 1e+06 |-+ | | | | | 500000 |-+ | | .++.++.++.++.++.++.+| 0 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months

1
0
0 / 0

[namei] ba73d98745: hackbench.throughput 4.3% improvement

by kernel test robot

Greeting, FYI, we noticed a 4.3% improvement of hackbench.throughput due to commit: commit: ba73d98745be1c10dc3cce68e8d7b95012d07d05 ("namei: handle idmapped mounts in may_*() helpers") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: hackbench on test machine: 96 threads 2 sockets Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory with following parameters: nr_threads: 50% iterations: 8 mode: process ipc: pipe cpufreq_governor: performance ucode: 0x4003006 test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler. test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sc... Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. ========================================================================================= compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-9/performance/pipe/8/x86_64-rhel-8.3/process/50%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp4/hackbench/0x4003006 commit: 0d56a4518d ("stat: handle idmapped mounts") ba73d98745 ("namei: handle idmapped mounts in may_*() helpers") 0d56a4518d5eaf59 ba73d98745be1c10dc3cce68e8d ---------------- --------------------------- %stddev %change %stddev \ | \ 351513 +4.3% 366744 hackbench.throughput 9.156e+08 ± 2% -14.7% 7.807e+08 ± 8% hackbench.time.involuntary_context_switches 1679 -5.5% 1586 ± 2% hackbench.time.user_time 2.496e+09 -7.4% 2.311e+09 ± 3% hackbench.time.voluntary_context_switches 3341 -2.0% 3275 boot-time.idle 88548243 +17.6% 1.042e+08 ± 6% cpuidle..usage 0.06 ± 2% +0.0 0.08 ± 5% mpstat.cpu.all.soft% 676.30 ± 9% +23.6% 836.03 ± 18% sched_debug.cpu.nr_uninterruptible.max 12398635 -5.2% 11754370 vmstat.system.cs 1342926 -5.8% 1264455 vmstat.system.in 43240116 ± 2% +22.4% 52937898 ± 8% turbostat.C1 0.77 ± 6% +0.2 0.98 ± 11% turbostat.C1% 3.796e+08 ± 2% -9.3% 3.443e+08 ± 4% turbostat.IRQ 0.00 +0.1 0.07 ± 9% perf-profile.children.cycles-pp.make_kuid 0.00 +0.1 0.09 ± 13% perf-profile.children.cycles-pp.make_kgid 0.00 +0.1 0.14 ± 9% perf-profile.children.cycles-pp.map_id_range_down 0.37 ± 9% +0.2 0.56 ± 9% perf-profile.children.cycles-pp.touch_atime 0.24 ± 13% +0.2 0.43 ± 11% perf-profile.children.cycles-pp.atime_needs_update 0.00 +0.1 0.13 ± 9% perf-profile.self.cycles-pp.map_id_range_down 0.34 ±114% -79.7% 0.07 ± 76% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables 0.01 ±124% +2430.1% 0.35 ±124% perf-sched.wait_time.avg.ms.preempt_schedule_common._cond_resched.dput.step_into.path_openat 0.53 ±115% -100.0% 0.00 perf-sched.wait_time.avg.ms.wait_for_partner.fifo_open.do_dentry_open.path_openat 0.02 ±130% +1574.5% 0.39 ±122% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.dput.step_into.path_openat 27.59 ± 55% +649.8% 206.87 ±147% perf-sched.wait_time.max.ms.preempt_schedule_common._cond_resched.pipe_write.new_sync_write.vfs_write 1.34 ±112% -100.0% 0.00 perf-sched.wait_time.max.ms.wait_for_partner.fifo_open.do_dentry_open.path_openat 14.79 -4.6% 14.11 perf-stat.i.MPKI 3.113e+10 +3.5% 3.222e+10 perf-stat.i.branch-instructions 1.20 -0.0 1.16 perf-stat.i.branch-miss-rate% 2.196e+09 -2.1% 2.151e+09 perf-stat.i.cache-references 12493419 -5.3% 11828083 perf-stat.i.context-switches 1.90 -2.5% 1.85 perf-stat.i.cpi 1567790 -7.1% 1456671 ± 3% perf-stat.i.cpu-migrations 4.432e+10 +2.3% 4.534e+10 perf-stat.i.dTLB-loads 2.594e+10 +1.5% 2.632e+10 perf-stat.i.dTLB-stores 69.41 +1.8 71.25 perf-stat.i.iTLB-load-miss-rate% 2.024e+08 +3.4% 2.092e+08 perf-stat.i.iTLB-load-misses 88106142 -5.4% 83326808 perf-stat.i.iTLB-loads 1.515e+11 +2.7% 1.556e+11 perf-stat.i.instructions 0.54 +2.7% 0.55 perf-stat.i.ipc 1079 +2.4% 1104 perf-stat.i.metric.M/sec 14.51 -4.7% 13.83 perf-stat.overall.MPKI 1.19 -0.0 1.15 perf-stat.overall.branch-miss-rate% 1.87 -2.6% 1.82 perf-stat.overall.cpi 69.64 +1.8 71.48 perf-stat.overall.iTLB-load-miss-rate% 0.53 +2.7% 0.55 perf-stat.overall.ipc 48.98 -1.3 47.65 perf-stat.overall.node-store-miss-rate% 3.101e+10 +3.5% 3.21e+10 perf-stat.ps.branch-instructions 2.189e+09 -2.1% 2.143e+09 perf-stat.ps.cache-references 12461918 -5.3% 11802321 perf-stat.ps.context-switches 1565708 -7.0% 1455380 ± 3% perf-stat.ps.cpu-migrations 4.415e+10 +2.3% 4.518e+10 perf-stat.ps.dTLB-loads 2.584e+10 +1.5% 2.622e+10 perf-stat.ps.dTLB-stores 2.016e+08 +3.4% 2.084e+08 perf-stat.ps.iTLB-load-misses 87887600 -5.4% 83148568 ± 2% perf-stat.ps.iTLB-loads 1.509e+11 +2.7% 1.55e+11 perf-stat.ps.instructions 31228 ± 4% +9.7% 34263 ± 2% softirqs.CPU1.SCHED 29873 ± 2% +10.2% 32919 ± 3% softirqs.CPU15.SCHED 29895 ± 2% +11.0% 33170 ± 2% softirqs.CPU16.SCHED 29884 ± 3% +12.1% 33513 ± 4% softirqs.CPU19.SCHED 29770 ± 2% +9.9% 32729 ± 3% softirqs.CPU20.SCHED 29584 ± 2% +11.0% 32839 ± 2% softirqs.CPU21.SCHED 30153 ± 2% +12.8% 34003 ± 6% softirqs.CPU24.SCHED 30757 ± 3% +10.9% 34123 ± 3% softirqs.CPU25.SCHED 30870 ± 2% +11.1% 34289 ± 3% softirqs.CPU26.SCHED 31234 ± 3% +10.1% 34391 ± 3% softirqs.CPU27.SCHED 30548 ± 2% +9.4% 33405 ± 3% softirqs.CPU30.SCHED 30163 ± 2% +10.7% 33383 ± 4% softirqs.CPU31.SCHED 30903 ± 3% +10.0% 33985 ± 3% softirqs.CPU32.SCHED 30884 ± 3% +11.1% 34297 ± 3% softirqs.CPU33.SCHED 29902 +11.1% 33217 ± 3% softirqs.CPU35.SCHED 30522 ± 3% +9.4% 33393 ± 3% softirqs.CPU36.SCHED 30582 ± 3% +10.3% 33727 ± 2% softirqs.CPU37.SCHED 30650 ± 2% +10.4% 33845 ± 3% softirqs.CPU38.SCHED 30637 ± 3% +11.0% 34002 ± 5% softirqs.CPU39.SCHED 30103 ± 2% +12.0% 33708 ± 2% softirqs.CPU4.SCHED 30160 ± 3% +12.3% 33868 ± 4% softirqs.CPU40.SCHED 30320 ± 3% +10.2% 33407 ± 4% softirqs.CPU41.SCHED 31093 ± 2% +8.1% 33609 ± 4% softirqs.CPU42.SCHED 30899 ± 2% +9.1% 33708 ± 4% softirqs.CPU44.SCHED 30808 ± 2% +10.5% 34051 ± 4% softirqs.CPU45.SCHED 30747 ± 2% +10.9% 34110 ± 3% softirqs.CPU46.SCHED 30485 ± 2% +11.1% 33859 ± 3% softirqs.CPU47.SCHED 30153 ± 3% +9.9% 33139 ± 2% softirqs.CPU48.SCHED 29785 ± 3% +10.8% 33015 ± 2% softirqs.CPU49.SCHED 29772 ± 2% +11.3% 33136 ± 3% softirqs.CPU52.SCHED 30346 ± 2% +11.9% 33958 ± 2% softirqs.CPU57.SCHED 29654 ± 2% +10.8% 32843 softirqs.CPU6.SCHED 30154 ± 2% +9.3% 32950 ± 4% softirqs.CPU63.SCHED 30397 ± 4% +9.2% 33202 ± 2% softirqs.CPU68.SCHED 29773 +11.8% 33283 ± 4% softirqs.CPU72.SCHED 30700 ± 2% +10.5% 33922 ± 4% softirqs.CPU73.SCHED 31490 ± 3% +9.8% 34568 ± 3% softirqs.CPU75.SCHED 30770 ± 2% +9.1% 33564 ± 4% softirqs.CPU77.SCHED 30476 ± 3% +9.9% 33504 ± 4% softirqs.CPU78.SCHED 31012 ± 3% +10.1% 34142 ± 3% softirqs.CPU80.SCHED 31266 ± 4% +10.2% 34464 ± 3% softirqs.CPU81.SCHED 30431 ± 3% +11.9% 34040 ± 3% softirqs.CPU85.SCHED 30765 ± 2% +10.7% 34052 ± 3% softirqs.CPU86.SCHED 30473 ± 3% +11.3% 33932 ± 3% softirqs.CPU88.SCHED 30285 ± 2% +11.7% 33838 ± 3% softirqs.CPU89.SCHED 30872 ± 3% +10.3% 34056 ± 3% softirqs.CPU90.SCHED 31275 ± 3% +9.9% 34361 ± 4% softirqs.CPU91.SCHED 30836 ± 2% +11.3% 34321 ± 4% softirqs.CPU94.SCHED 2929299 +9.9% 3217849 ± 2% softirqs.SCHED 3291660 ± 2% -10.2% 2955467 ± 4% interrupts.CPU0.RES:Rescheduling_interrupts 3209155 -9.9% 2890930 ± 4% interrupts.CPU1.RES:Rescheduling_interrupts 3167361 -10.1% 2848895 ± 5% interrupts.CPU12.RES:Rescheduling_interrupts 3205437 ± 2% -10.2% 2878328 ± 5% interrupts.CPU13.RES:Rescheduling_interrupts 3214503 -10.4% 2878875 ± 5% interrupts.CPU15.RES:Rescheduling_interrupts 3193257 -10.4% 2859582 ± 5% interrupts.CPU23.RES:Rescheduling_interrupts 3197798 ± 4% -10.8% 2851897 ± 4% interrupts.CPU24.RES:Rescheduling_interrupts 3324203 ± 3% -10.9% 2962555 ± 5% interrupts.CPU26.RES:Rescheduling_interrupts 3322402 ± 3% -11.0% 2955560 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts 3331711 ± 2% -13.1% 2893654 ± 5% interrupts.CPU28.RES:Rescheduling_interrupts 3348281 ± 3% -14.5% 2864319 ± 4% interrupts.CPU29.RES:Rescheduling_interrupts 3234270 ± 3% -11.7% 2855014 ± 4% interrupts.CPU30.RES:Rescheduling_interrupts 3277111 ± 2% -12.3% 2873186 ± 4% interrupts.CPU31.RES:Rescheduling_interrupts 3309840 ± 3% -12.5% 2896464 ± 2% interrupts.CPU32.RES:Rescheduling_interrupts 3303859 ± 3% -11.6% 2922260 ± 4% interrupts.CPU33.RES:Rescheduling_interrupts 3287003 ± 4% -12.8% 2866335 ± 4% interrupts.CPU34.RES:Rescheduling_interrupts 3268249 ± 2% -13.1% 2838933 ± 5% interrupts.CPU35.RES:Rescheduling_interrupts 3278415 ± 2% -12.5% 2867775 ± 4% interrupts.CPU36.RES:Rescheduling_interrupts 3252387 ± 3% -12.0% 2861895 ± 4% interrupts.CPU37.RES:Rescheduling_interrupts 3312763 ± 3% -12.3% 2905598 ± 3% interrupts.CPU38.RES:Rescheduling_interrupts 3299906 ± 3% -12.2% 2898481 ± 4% interrupts.CPU39.RES:Rescheduling_interrupts 3227478 ± 3% -10.0% 2904683 ± 3% interrupts.CPU40.RES:Rescheduling_interrupts 3233470 ± 4% -11.8% 2850359 ± 4% interrupts.CPU41.RES:Rescheduling_interrupts 3247904 ± 4% -11.7% 2869199 ± 2% interrupts.CPU42.RES:Rescheduling_interrupts 3313033 ± 4% -11.2% 2941349 ± 4% interrupts.CPU43.RES:Rescheduling_interrupts 3250621 ± 4% -10.9% 2896775 ± 3% interrupts.CPU44.RES:Rescheduling_interrupts 5474 ± 35% +51.3% 8281 interrupts.CPU45.NMI:Non-maskable_interrupts 5474 ± 35% +51.3% 8281 interrupts.CPU45.PMI:Performance_monitoring_interrupts 3278556 ± 3% -11.0% 2918741 ± 5% interrupts.CPU45.RES:Rescheduling_interrupts 3271070 ± 4% -10.3% 2933335 ± 5% interrupts.CPU46.RES:Rescheduling_interrupts 3253683 ± 4% -11.3% 2884737 ± 4% interrupts.CPU47.RES:Rescheduling_interrupts 3186697 ± 2% -9.3% 2891753 ± 5% interrupts.CPU48.RES:Rescheduling_interrupts 3198247 -9.6% 2891256 ± 5% interrupts.CPU49.RES:Rescheduling_interrupts 3252717 -10.1% 2923091 ± 4% interrupts.CPU56.RES:Rescheduling_interrupts 3241606 ± 2% -9.7% 2926074 ± 5% interrupts.CPU57.RES:Rescheduling_interrupts 3200963 ± 2% -9.8% 2886532 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts 3258239 ± 2% -10.3% 2921021 ± 6% interrupts.CPU67.RES:Rescheduling_interrupts 3205346 -9.5% 2901805 ± 4% interrupts.CPU68.RES:Rescheduling_interrupts 3202213 ± 2% -9.4% 2902413 ± 5% interrupts.CPU70.RES:Rescheduling_interrupts 3176968 ± 4% -11.4% 2813497 ± 5% interrupts.CPU72.RES:Rescheduling_interrupts 3259194 ± 4% -10.5% 2917966 ± 4% interrupts.CPU73.RES:Rescheduling_interrupts 3309819 ± 3% -11.3% 2937158 ± 4% interrupts.CPU74.RES:Rescheduling_interrupts 3368701 ± 4% -12.3% 2954496 ± 4% interrupts.CPU75.RES:Rescheduling_interrupts 3302093 ± 3% -12.9% 2874666 ± 3% interrupts.CPU76.RES:Rescheduling_interrupts 3322200 ± 4% -13.3% 2880899 ± 3% interrupts.CPU77.RES:Rescheduling_interrupts 3254923 ± 4% -12.3% 2854791 ± 3% interrupts.CPU78.RES:Rescheduling_interrupts 3298900 ± 2% -12.8% 2877407 ± 5% interrupts.CPU79.RES:Rescheduling_interrupts 3329217 ± 4% -12.0% 2928239 ± 3% interrupts.CPU80.RES:Rescheduling_interrupts 3350850 ± 3% -11.6% 2963550 ± 4% interrupts.CPU81.RES:Rescheduling_interrupts 3261881 ± 4% -11.6% 2883912 ± 3% interrupts.CPU82.RES:Rescheduling_interrupts 3262199 ± 4% -13.6% 2819681 ± 5% interrupts.CPU83.RES:Rescheduling_interrupts 3296607 ± 3% -12.4% 2887541 ± 4% interrupts.CPU84.RES:Rescheduling_interrupts 3247117 ± 4% -11.6% 2871867 ± 4% interrupts.CPU85.RES:Rescheduling_interrupts 3304686 ± 3% -11.4% 2927270 ± 4% interrupts.CPU86.RES:Rescheduling_interrupts 3313258 ± 3% -12.8% 2890225 ± 4% interrupts.CPU87.RES:Rescheduling_interrupts 3276588 ± 4% -11.1% 2911694 ± 4% interrupts.CPU88.RES:Rescheduling_interrupts 3245365 ± 4% -11.8% 2861921 ± 3% interrupts.CPU89.RES:Rescheduling_interrupts 3206511 -9.4% 2905199 ± 4% interrupts.CPU9.RES:Rescheduling_interrupts 3260447 ± 3% -11.2% 2894368 ± 3% interrupts.CPU90.RES:Rescheduling_interrupts 3318146 ± 4% -11.0% 2954676 ± 3% interrupts.CPU91.RES:Rescheduling_interrupts 3253573 ± 4% -10.1% 2924313 ± 4% interrupts.CPU92.RES:Rescheduling_interrupts 3278406 ± 5% -10.5% 2934051 ± 5% interrupts.CPU93.RES:Rescheduling_interrupts 3343345 ± 3% -11.4% 2962741 ± 4% interrupts.CPU94.RES:Rescheduling_interrupts 3292411 ± 4% -12.0% 2897405 ± 4% interrupts.CPU95.RES:Rescheduling_interrupts 3.111e+08 ± 2% -10.5% 2.783e+08 ± 4% interrupts.RES:Rescheduling_interrupts hackbench.throughput 385000 +------------------------------------------------------------------+ 380000 |O+ OO O | | O O O O O | 375000 |-+ O O O O O OO O O| 370000 |-+ O O O O O O OO O | 365000 |-+ O O OO O O | 360000 |-+O O O O + | | + O O + : | 355000 |-+ :: O O + : O O | 350000 |-+ :: + .+O +. + + :+ | 345000 |-++ : : +: +.+ +.++ +.+ + + +. : + | 340000 |-:: + ++ :.+ :.+ + + | |:: :+ + + | 335000 |++ + | 330000 +------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months

1
0
0 / 0

[sched/fair] c5b0a7eefc: aim7.jobs-per-min 9.6% improvement

by kernel test robot

Greeting, FYI, we noticed a 9.6% improvement of aim7.jobs-per-min due to commit: commit: c5b0a7eefc70150caf23e37bc9d639c68c87a097 ("sched/fair: Remove sysctl_sched_migration_cost condition") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: aim7 on test machine: 144 threads 4 sockets Intel(R) Xeon(R) Gold 5318H CPU @ 2.50GHz with 128G memory with following parameters: disk: 1BRD_48G fs: xfs test: sync_disk_rw load: 600 cpufreq_governor: performance ucode: 0x700001e test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system. test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase/ucode: gcc-9/performance/1BRD_48G/xfs/x86_64-rhel-8.3/600/debian-10.4-x86_64-20200603.cgz/lkp-cpl-4sp1/sync_disk_rw/aim7/0x700001e commit: e60b56e46b ("sched/fair: Wait before decaying max_newidle_lb_cost") c5b0a7eefc ("sched/fair: Remove sysctl_sched_migration_cost condition") e60b56e46b384cee c5b0a7eefc70150caf23e37bc9d ---------------- --------------------------- %stddev %change %stddev \ | \ 20720 +9.6% 22708 aim7.jobs-per-min 174.01 -8.7% 158.79 aim7.time.elapsed_time 174.01 -8.7% 158.79 aim7.time.elapsed_time.max 1954744 +28.8% 2517455 aim7.time.involuntary_context_switches 14820 -1.2% 14641 aim7.time.system_time 61688914 -9.6% 55763881 aim7.time.voluntary_context_switches 9.414e+09 -23.0% 7.253e+09 cpuidle..time 49644559 -18.0% 40697490 cpuidle..usage 36.35 -5.0 31.37 mpstat.cpu.all.idle% 1.13 -0.7 0.48 ± 5% mpstat.cpu.all.iowait% 37.04 -13.2% 32.13 iostat.cpu.idle 1.12 -57.4% 0.48 ± 5% iostat.cpu.iowait 61.57 +9.0% 67.09 iostat.cpu.system 789729 ± 2% -22.6% 611335 ± 5% meminfo.Active 789729 ± 2% -22.6% 611335 ± 5% meminfo.Active(anon) 50053 -17.1% 41506 meminfo.KernelStack 36.86 -14.0% 31.71 vmstat.cpu.id 845852 +7.4% 908831 vmstat.io.bo 213.86 ± 6% -23.6% 163.29 ± 4% vmstat.procs.r 589461 -1.6% 580038 vmstat.system.cs 20207 ± 17% -41.6% 11805 ± 40% numa-meminfo.node0.KernelStack 151950 ± 7% -14.8% 129403 ± 13% numa-meminfo.node0.SUnreclaim 188601 ± 56% -67.9% 60547 ± 19% numa-meminfo.node2.Active 188601 ± 56% -67.9% 60547 ± 19% numa-meminfo.node2.Active(anon) 9793 ± 7% -22.1% 7629 ± 9% numa-meminfo.node2.KernelStack 344052 ± 48% -58.6% 142287 ± 30% numa-meminfo.node2.Shmem 29323 ± 3% +12.6% 33027 ± 4% numa-meminfo.node3.Dirty 3710270 ± 6% -36.9% 2340623 turbostat.C1 1.08 ± 15% -0.5 0.62 ± 4% turbostat.C1% 44496648 ± 2% -16.6% 37121566 ± 2% turbostat.C1E 34.04 ± 7% -6.0 28.00 ± 9% turbostat.C1E% 36.56 ± 2% -16.0% 30.71 ± 2% turbostat.CPU%c1 446.22 +3.0% 459.49 turbostat.PkgWatt 39.04 +4.0% 40.61 turbostat.RAMWatt 20210 ± 17% -41.6% 11804 ± 40% numa-vmstat.node0.nr_kernel_stack 37988 ± 7% -14.8% 32351 ± 13% numa-vmstat.node0.nr_slab_unreclaimable 2700393 ± 11% -22.6% 2091340 ± 16% numa-vmstat.node0.numa_hit 2654128 ± 12% -22.8% 2049202 ± 16% numa-vmstat.node0.numa_local 47200 ± 56% -67.9% 15166 ± 19% numa-vmstat.node2.nr_active_anon 9793 ± 7% -22.1% 7629 ± 9% numa-vmstat.node2.nr_kernel_stack 86023 ± 48% -58.6% 35600 ± 30% numa-vmstat.node2.nr_shmem 47200 ± 56% -67.9% 15166 ± 19% numa-vmstat.node2.nr_zone_active_anon 7321 ± 4% +14.3% 8366 ± 5% numa-vmstat.node3.nr_dirty 8037 ± 3% +12.1% 9007 ± 4% numa-vmstat.node3.nr_zone_write_pending 1950 -41.4% 1143 ± 2% slabinfo.biovec-max.active_objs 1950 -41.4% 1143 ± 2% slabinfo.biovec-max.num_objs 6639 -10.0% 5973 slabinfo.ip6-frags.active_objs 6639 -10.0% 5973 slabinfo.ip6-frags.num_objs 4813 -9.7% 4347 slabinfo.sighand_cache.active_objs 4813 -9.6% 4350 slabinfo.sighand_cache.num_objs 3495 -15.8% 2942 slabinfo.task_struct.active_objs 3502 -15.7% 2952 slabinfo.task_struct.active_slabs 3502 -15.7% 2952 slabinfo.task_struct.num_objs 3502 -15.7% 2952 slabinfo.task_struct.num_slabs 197460 ± 2% -22.9% 152276 ± 5% proc-vmstat.nr_active_anon 1014492 -2.9% 984857 proc-vmstat.nr_file_pages 215625 ± 2% +7.8% 232516 ± 3% proc-vmstat.nr_inactive_anon 72096 -1.2% 71207 proc-vmstat.nr_inactive_file 50053 -17.1% 41510 proc-vmstat.nr_kernel_stack 125249 ± 3% +12.0% 140320 ± 3% proc-vmstat.nr_mapped 337780 -8.5% 309028 proc-vmstat.nr_shmem 104975 -2.6% 102291 proc-vmstat.nr_slab_unreclaimable 197460 ± 2% -22.9% 152276 ± 5% proc-vmstat.nr_zone_active_anon 215625 ± 2% +7.8% 232516 ± 3% proc-vmstat.nr_zone_inactive_anon 72096 -1.2% 71207 proc-vmstat.nr_zone_inactive_file 199209 -1.4% 196433 proc-vmstat.numa_other 602518 +3.6% 623992 proc-vmstat.pgactivate 1478725 -2.8% 1437852 ± 2% proc-vmstat.pgfault 1.495e+08 -1.8% 1.469e+08 proc-vmstat.pgpgout 58796 -6.4% 55008 proc-vmstat.pgreuse 0.01 ± 31% -73.0% 0.00 ± 35% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.13 -24.5% 0.09 ± 2% perf-sched.sch_delay.avg.ms.io_schedule.wait_on_page_bit.wait_on_page_writeback.__filemap_fdatawait_range 0.10 ± 3% -62.7% 0.04 ± 9% perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.__mutex_lock.isra.0.flush_workqueue 0.09 -11.6% 0.08 perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.flush_workqueue.xlog_cil_push_now 0.08 ± 19% -28.1% 0.06 ± 9% perf-sched.sch_delay.avg.ms.schedule_timeout.wait_for_completion.stop_two_cpus.migrate_swap 0.01 ± 6% -16.3% 0.01 ± 5% perf-sched.sch_delay.avg.ms.xlog_wait_on_iclog.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write 3.96 ± 32% -69.0% 1.23 ± 16% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single.[unknown] 2.17 ± 33% -57.1% 0.93 ± 32% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.dput.__fput.task_work_run 4.44 ± 22% -67.0% 1.47 ± 26% perf-sched.sch_delay.max.ms.preempt_schedule_common.__cond_resched.write_cache_pages.iomap_writepages.xfs_vm_writepages 2.22 ± 39% -59.9% 0.89 ± 37% perf-sched.sch_delay.max.ms.rwsem_down_read_slowpath.xlog_cil_commit.__xfs_trans_commit.xfs_free_eofblocks 0.08 ± 4% -16.9% 0.07 ± 7% perf-sched.total_sch_delay.average.ms 216.63 ± 3% -44.1% 121.06 ± 6% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.78 -25.6% 0.58 ± 2% perf-sched.wait_and_delay.avg.ms.schedule_preempt_disabled.__mutex_lock.isra.0.flush_workqueue 7.13 +13.5% 8.09 ± 3% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork 100.86 ± 10% -19.4% 81.29 ± 9% perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.generic_perform_write.__generic_file_write_iter.generic_file_write_iter 17962 ± 3% -16.0% 15079 ± 2% perf-sched.wait_and_delay.count.rwsem_down_read_slowpath.xlog_cil_commit.__xfs_trans_commit.xfs_iomap_write_unwritten 59797 ± 3% +10.3% 65945 ± 3% perf-sched.wait_and_delay.count.rwsem_down_read_slowpath.xlog_cil_commit.__xfs_trans_commit.xfs_vn_update_time 181332 ± 4% +12.6% 204242 perf-sched.wait_and_delay.count.xlog_wait_on_iclog.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write 73.57 ± 7% -14.3% 63.07 ± 5% perf-sched.wait_and_delay.max.ms.rwsem_down_read_slowpath.xlog_cil_commit.__xfs_trans_commit.xfs_vn_update_time 216.62 ± 3% -44.1% 121.06 ± 6% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.49 ± 9% +30.7% 0.64 ± 24% perf-sched.wait_time.avg.ms.preempt_schedule_common.__cond_resched.dput.__fput.task_work_run 0.68 -20.2% 0.54 ± 2% perf-sched.wait_time.avg.ms.schedule_preempt_disabled.__mutex_lock.isra.0.flush_workqueue 7.11 +13.5% 8.07 ± 3% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork 0.03 ± 10% -29.0% 0.02 ± 12% perf-sched.wait_time.avg.ms.xlog_wait_on_iclog.xfs_file_fsync.xfs_file_buffered_write.new_sync_write 75.83 ± 4% -9.0% 68.98 ± 2% perf-sched.wait_time.max.ms.io_schedule.wait_on_page_bit.wait_on_page_writeback.__filemap_fdatawait_range 71.76 ± 5% -12.1% 63.05 ± 5% perf-sched.wait_time.max.ms.rwsem_down_read_slowpath.xlog_cil_commit.__xfs_trans_commit.xfs_vn_update_time 78.96 ± 3% -9.4% 71.52 ± 2% perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.flush_workqueue.xlog_cil_push_now 75.38 ± 28% -25.2% 56.40 ± 6% perf-sched.wait_time.max.ms.schedule_timeout.wait_for_completion.stop_two_cpus.migrate_swap 6.459e+09 +14.9% 7.419e+09 perf-stat.i.branch-instructions 39222108 +57.1% 61599978 perf-stat.i.branch-misses 40.09 -9.4 30.71 ± 2% perf-stat.i.cache-miss-rate% 72330942 +29.8% 93921627 perf-stat.i.cache-misses 1.756e+08 +71.5% 3.012e+08 perf-stat.i.cache-references 596375 -1.5% 587688 perf-stat.i.context-switches 9.04 -6.7% 8.44 perf-stat.i.cpi 2.948e+11 +9.8% 3.237e+11 perf-stat.i.cpu-cycles 99840 +29.8% 129581 perf-stat.i.cpu-migrations 4207 ± 4% -15.2% 3569 ± 5% perf-stat.i.cycles-between-cache-misses 8.66e+09 +20.4% 1.043e+10 perf-stat.i.dTLB-loads 2.052e+09 +26.9% 2.603e+09 perf-stat.i.dTLB-stores 25.89 +1.7 27.61 perf-stat.i.iTLB-load-miss-rate% 5584929 +9.5% 6113345 perf-stat.i.iTLB-load-misses 3.179e+10 +17.1% 3.724e+10 perf-stat.i.instructions 5699 +6.5% 6069 perf-stat.i.instructions-per-iTLB-miss 0.13 ± 2% +6.8% 0.14 ± 2% perf-stat.i.ipc 2.05 +9.8% 2.25 perf-stat.i.metric.GHz 464.08 ± 3% +25.5% 582.27 ± 2% perf-stat.i.metric.K/sec 120.44 +19.6% 144.08 perf-stat.i.metric.M/sec 7887 +6.5% 8396 ± 2% perf-stat.i.minor-faults 28705491 +41.5% 40630186 perf-stat.i.node-load-misses 2519440 ± 4% +34.5% 3388635 ± 3% perf-stat.i.node-loads 13701813 +27.7% 17494602 perf-stat.i.node-store-misses 1443652 +8.0% 1559528 perf-stat.i.node-stores 7890 +6.4% 8399 ± 2% perf-stat.i.page-faults 5.52 +46.4% 8.09 perf-stat.overall.MPKI 0.61 +0.2 0.83 perf-stat.overall.branch-miss-rate% 41.19 -10.0 31.19 perf-stat.overall.cache-miss-rate% 9.27 -6.2% 8.69 perf-stat.overall.cpi 4076 -15.4% 3447 perf-stat.overall.cycles-between-cache-misses 0.04 ± 4% -0.0 0.03 ± 6% perf-stat.overall.dTLB-store-miss-rate% 25.44 ± 2% +1.7 27.15 perf-stat.overall.iTLB-load-miss-rate% 5692 +7.0% 6090 perf-stat.overall.instructions-per-iTLB-miss 0.11 +6.7% 0.12 perf-stat.overall.ipc 90.46 +1.3 91.81 perf-stat.overall.node-store-miss-rate% 6.424e+09 +14.8% 7.375e+09 perf-stat.ps.branch-instructions 38999514 +57.0% 61211279 perf-stat.ps.branch-misses 71925892 +29.8% 93340187 perf-stat.ps.cache-misses 1.746e+08 +71.4% 2.993e+08 perf-stat.ps.cache-references 592903 -1.5% 584027 perf-stat.ps.context-switches 2.931e+11 +9.7% 3.217e+11 perf-stat.ps.cpu-cycles 99246 +29.7% 128760 perf-stat.ps.cpu-migrations 8.614e+09 +20.3% 1.037e+10 perf-stat.ps.dTLB-loads 2.041e+09 +26.8% 2.588e+09 perf-stat.ps.dTLB-stores 5556625 +9.4% 6078609 perf-stat.ps.iTLB-load-misses 3.162e+10 +17.1% 3.702e+10 perf-stat.ps.instructions 7853 +6.3% 8352 ± 2% perf-stat.ps.minor-faults 28542787 +41.5% 40376546 perf-stat.ps.node-load-misses 2506460 ± 4% +34.4% 3367943 ± 3% perf-stat.ps.node-loads 13624704 +27.6% 17386445 perf-stat.ps.node-store-misses 1436181 +8.0% 1550585 perf-stat.ps.node-stores 7857 +6.3% 8354 ± 2% perf-stat.ps.page-faults 5.531e+12 +6.8% 5.908e+12 perf-stat.total.instructions 12.59 ± 2% -1.9 10.72 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 12.50 ± 2% -1.9 10.64 ± 2% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 12.50 ± 2% -1.8 10.65 ± 2% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 12.50 ± 2% -1.8 10.65 ± 2% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 12.05 ± 2% -1.8 10.20 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 12.04 ± 2% -1.8 10.19 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 11.66 ± 2% -1.8 9.90 ± 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 2.91 ± 19% -1.2 1.70 ± 19% perf-profile.calltrace.cycles-pp.xfs_iomap_write_unwritten.xfs_end_ioend.xfs_end_io.process_one_work.worker_thread 3.13 ± 18% -1.2 1.97 ± 16% perf-profile.calltrace.cycles-pp.xfs_end_io.process_one_work.worker_thread.kthread.ret_from_fork 3.13 ± 18% -1.2 1.97 ± 16% perf-profile.calltrace.cycles-pp.xfs_end_ioend.xfs_end_io.process_one_work.worker_thread.kthread 3.89 ± 14% -1.1 2.75 ± 11% perf-profile.calltrace.cycles-pp.ret_from_fork 3.89 ± 14% -1.1 2.75 ± 11% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 3.81 ± 14% -1.1 2.67 ± 12% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork 3.85 ± 14% -1.1 2.72 ± 12% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork 1.79 ± 16% -0.7 1.04 ± 14% perf-profile.calltrace.cycles-pp.xlog_cil_commit.__xfs_trans_commit.xfs_iomap_write_unwritten.xfs_end_ioend.xfs_end_io 1.80 ± 16% -0.7 1.05 ± 14% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_iomap_write_unwritten.xfs_end_ioend.xfs_end_io.process_one_work 3.52 ± 7% -0.4 3.15 ± 9% perf-profile.calltrace.cycles-pp.__xfs_trans_commit.xfs_vn_update_time.file_update_time.xfs_file_write_checks.xfs_file_buffered_write 3.50 ± 7% -0.4 3.13 ± 9% perf-profile.calltrace.cycles-pp.xlog_cil_commit.__xfs_trans_commit.xfs_vn_update_time.file_update_time.xfs_file_write_checks 1.92 ± 3% +0.1 2.03 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.remove_wait_queue.xlog_wait_on_iclog.xfs_log_force_seq.xfs_file_fsync 1.91 ± 3% +0.1 2.02 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.remove_wait_queue.xlog_wait_on_iclog.xfs_log_force_seq 1.93 ± 3% +0.1 2.04 ± 2% perf-profile.calltrace.cycles-pp.remove_wait_queue.xlog_wait_on_iclog.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write 0.89 ± 8% +0.2 1.06 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write 0.87 ± 8% +0.2 1.05 ± 10% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync 1.25 ± 5% +0.2 1.44 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq 1.29 ± 5% +0.2 1.48 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync 0.79 ± 3% +0.2 1.01 perf-profile.calltrace.cycles-pp.mutex_spin_on_owner.__mutex_lock.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq 2.02 ± 3% +0.2 2.24 ± 2% perf-profile.calltrace.cycles-pp.xlog_wait_on_iclog.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write.new_sync_write 0.07 ±244% +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.iomap_file_buffered_write.xfs_file_buffered_write.new_sync_write.vfs_write.ksys_write 0.00 +0.7 0.70 ± 10% perf-profile.calltrace.cycles-pp.newidle_balance.pick_next_task_fair.__schedule.schedule.schedule_timeout 0.00 +0.7 0.71 ± 10% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.schedule_timeout.wait_for_completion 0.00 +1.0 1.01 ± 8% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.wait_for_completion.flush_workqueue 0.00 +1.0 1.03 ± 8% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_for_completion.flush_workqueue.xlog_cil_push_now 0.00 +1.0 1.05 ± 7% perf-profile.calltrace.cycles-pp.schedule_timeout.wait_for_completion.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq 0.00 +1.1 1.11 ± 7% perf-profile.calltrace.cycles-pp.wait_for_completion.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq 67.03 +1.7 68.71 perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq 68.23 +2.2 70.47 perf-profile.calltrace.cycles-pp.__mutex_lock.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq 83.00 +3.0 85.98 perf-profile.calltrace.cycles-pp.xfs_file_buffered_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 83.02 +3.0 86.01 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 83.09 +3.0 86.08 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 83.08 +3.0 86.07 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 83.14 +3.0 86.13 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 83.13 +3.0 86.13 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 83.24 +3.0 86.23 perf-profile.calltrace.cycles-pp.write 69.57 +3.1 72.63 perf-profile.calltrace.cycles-pp.flush_workqueue.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync 70.93 +3.2 74.17 perf-profile.calltrace.cycles-pp.xlog_cil_push_now.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write 72.33 +3.4 75.74 perf-profile.calltrace.cycles-pp.xlog_cil_force_seq.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write.new_sync_write 74.63 +3.6 78.28 perf-profile.calltrace.cycles-pp.xfs_log_force_seq.xfs_file_fsync.xfs_file_buffered_write.new_sync_write.vfs_write 76.83 +4.0 80.84 perf-profile.calltrace.cycles-pp.xfs_file_fsync.xfs_file_buffered_write.new_sync_write.vfs_write.ksys_write 12.59 ± 2% -1.9 10.72 perf-profile.children.cycles-pp.do_idle 12.59 ± 2% -1.9 10.72 perf-profile.children.cycles-pp.secondary_startup_64_no_verify 12.59 ± 2% -1.9 10.72 perf-profile.children.cycles-pp.cpu_startup_entry 12.14 ± 2% -1.9 10.28 perf-profile.children.cycles-pp.cpuidle_enter 12.14 ± 2% -1.9 10.28 perf-profile.children.cycles-pp.cpuidle_enter_state 12.50 ± 2% -1.8 10.65 ± 2% perf-profile.children.cycles-pp.start_secondary 11.74 ± 2% -1.8 9.97 perf-profile.children.cycles-pp.intel_idle 9.95 ± 13% -1.7 8.27 ± 13% perf-profile.children.cycles-pp._raw_spin_lock 2.91 ± 19% -1.2 1.71 ± 19% perf-profile.children.cycles-pp.xfs_iomap_write_unwritten 2.21 ± 18% -1.2 1.02 ± 43% perf-profile.children.cycles-pp.xfs_log_space_wake 2.30 ± 17% -1.2 1.12 ± 39% perf-profile.children.cycles-pp.xfs_log_ticket_ungrant 3.13 ± 18% -1.2 1.97 ± 16% perf-profile.children.cycles-pp.xfs_end_io 3.13 ± 18% -1.2 1.97 ± 16% perf-profile.children.cycles-pp.xfs_end_ioend 3.89 ± 14% -1.1 2.75 ± 11% perf-profile.children.cycles-pp.ret_from_fork 3.89 ± 14% -1.1 2.75 ± 11% perf-profile.children.cycles-pp.kthread 3.81 ± 14% -1.1 2.67 ± 12% perf-profile.children.cycles-pp.process_one_work 3.85 ± 14% -1.1 2.72 ± 12% perf-profile.children.cycles-pp.worker_thread 5.36 ± 10% -1.1 4.24 ± 10% perf-profile.children.cycles-pp.__xfs_trans_commit 5.33 ± 10% -1.1 4.22 ± 10% perf-profile.children.cycles-pp.xlog_cil_commit 0.94 ± 30% -0.5 0.46 ± 40% perf-profile.children.cycles-pp.xfs_trans_alloc_inode 0.40 ± 6% -0.1 0.35 ± 7% perf-profile.children.cycles-pp._raw_spin_lock_irq 0.11 ± 4% -0.0 0.08 ± 4% perf-profile.children.cycles-pp.rwsem_wake 0.26 ± 3% -0.0 0.24 ± 2% perf-profile.children.cycles-pp.asm_sysvec_call_function_single 0.24 ± 3% -0.0 0.22 ± 2% perf-profile.children.cycles-pp.sysvec_call_function_single 0.14 ± 4% +0.0 0.16 ± 3% perf-profile.children.cycles-pp.xfs_bmapi_convert_unwritten 0.07 +0.0 0.08 ± 5% perf-profile.children.cycles-pp.update_rq_clock 0.16 ± 4% +0.0 0.18 perf-profile.children.cycles-pp.xfs_bmapi_write 0.07 ± 7% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.40 ± 3% +0.0 0.43 perf-profile.children.cycles-pp.brd_do_bvec 0.28 ± 2% +0.0 0.32 perf-profile.children.cycles-pp.update_load_avg 0.22 ± 3% +0.0 0.26 ± 2% perf-profile.children.cycles-pp.enqueue_entity 0.03 ± 86% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.set_task_cpu 0.21 ± 8% +0.0 0.26 ± 5% perf-profile.children.cycles-pp.iomap_finish_ioends 0.19 ± 2% +0.0 0.23 ± 3% perf-profile.children.cycles-pp.update_curr 0.00 +0.1 0.05 perf-profile.children.cycles-pp.migrate_task_rq_fair 0.00 +0.1 0.05 ± 6% perf-profile.children.cycles-pp.finish_task_switch 0.00 +0.1 0.05 ± 8% perf-profile.children.cycles-pp._find_next_bit 0.48 ± 4% +0.1 0.53 ± 2% perf-profile.children.cycles-pp.iomap_file_buffered_write 0.00 +0.1 0.05 ± 9% perf-profile.children.cycles-pp.cpumask_next_and 0.94 +0.1 1.00 perf-profile.children.cycles-pp.try_to_wake_up 0.32 ± 3% +0.1 0.38 perf-profile.children.cycles-pp.enqueue_task_fair 0.00 +0.1 0.06 perf-profile.children.cycles-pp.update_blocked_averages 0.16 ± 4% +0.1 0.24 ± 2% perf-profile.children.cycles-pp.dequeue_entity 0.22 ± 3% +0.1 0.30 perf-profile.children.cycles-pp.wake_up_q 0.23 ± 3% +0.1 0.32 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair 0.48 ± 6% +0.1 0.57 ± 5% perf-profile.children.cycles-pp.rwsem_down_read_slowpath 0.00 +0.1 0.11 ± 13% perf-profile.children.cycles-pp.idle_cpu 0.00 +0.1 0.13 ± 8% perf-profile.children.cycles-pp.schedule_preempt_disabled 0.14 ± 5% +0.1 0.28 ± 9% perf-profile.children.cycles-pp.raw_spin_rq_lock_nested 0.20 ± 7% +0.1 0.34 ± 3% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 0.79 ± 3% +0.2 1.01 ± 2% perf-profile.children.cycles-pp.mutex_spin_on_owner 2.04 ± 3% +0.2 2.28 ± 2% perf-profile.children.cycles-pp.xlog_wait_on_iclog 0.10 ± 5% +0.6 0.70 ± 12% perf-profile.children.cycles-pp.update_sd_lb_stats 0.10 ± 4% +0.6 0.71 ± 12% perf-profile.children.cycles-pp.find_busiest_group 0.45 ± 3% +0.7 1.11 ± 7% perf-profile.children.cycles-pp.wait_for_completion 0.39 ± 3% +0.7 1.05 ± 8% perf-profile.children.cycles-pp.schedule_timeout 0.18 ± 4% +0.9 1.10 ± 10% perf-profile.children.cycles-pp.load_balance 0.23 ± 4% +1.0 1.25 ± 9% perf-profile.children.cycles-pp.pick_next_task_fair 0.16 ± 6% +1.0 1.18 ± 9% perf-profile.children.cycles-pp.newidle_balance 0.78 ± 2% +1.1 1.88 ± 6% perf-profile.children.cycles-pp.__schedule 0.70 ± 2% +1.1 1.79 ± 6% perf-profile.children.cycles-pp.schedule 67.06 +1.7 68.75 perf-profile.children.cycles-pp.osq_lock 68.23 +2.2 70.47 perf-profile.children.cycles-pp.__mutex_lock 83.04 +3.0 86.02 perf-profile.children.cycles-pp.new_sync_write 83.00 +3.0 85.99 perf-profile.children.cycles-pp.xfs_file_buffered_write 83.10 +3.0 86.09 perf-profile.children.cycles-pp.ksys_write 83.10 +3.0 86.08 perf-profile.children.cycles-pp.vfs_write 83.31 +3.0 86.30 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 83.30 +3.0 86.30 perf-profile.children.cycles-pp.do_syscall_64 83.25 +3.0 86.26 perf-profile.children.cycles-pp.write 69.57 +3.1 72.63 perf-profile.children.cycles-pp.flush_workqueue 70.93 +3.2 74.17 perf-profile.children.cycles-pp.xlog_cil_push_now 72.33 +3.4 75.74 perf-profile.children.cycles-pp.xlog_cil_force_seq 74.63 +3.6 78.28 perf-profile.children.cycles-pp.xfs_log_force_seq 76.83 +4.0 80.84 perf-profile.children.cycles-pp.xfs_file_fsync 11.74 ± 2% -1.8 9.97 perf-profile.self.cycles-pp.intel_idle 0.06 ± 7% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.16 ± 3% +0.0 0.17 ± 2% perf-profile.self.cycles-pp.update_load_avg 0.04 ± 40% +0.0 0.06 perf-profile.self.cycles-pp.update_rq_clock 0.06 ± 7% +0.0 0.08 ± 6% perf-profile.self.cycles-pp.update_curr 0.28 ± 3% +0.0 0.31 ± 2% perf-profile.self.cycles-pp.brd_do_bvec 0.10 ± 6% +0.0 0.13 ± 4% perf-profile.self.cycles-pp.__mutex_lock 0.02 ±115% +0.0 0.07 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair 0.37 ± 2% +0.0 0.42 perf-profile.self.cycles-pp._raw_spin_lock 0.00 +0.1 0.07 ± 12% perf-profile.self.cycles-pp.load_balance 0.00 +0.1 0.11 ± 10% perf-profile.self.cycles-pp.idle_cpu 0.78 ± 3% +0.2 1.00 perf-profile.self.cycles-pp.mutex_spin_on_owner 0.08 ± 6% +0.5 0.54 ± 12% perf-profile.self.cycles-pp.update_sd_lb_stats 66.30 +1.7 67.99 perf-profile.self.cycles-pp.osq_lock 40636 ± 2% +13.7% 46188 ± 4% softirqs.CPU1.SCHED 40402 ± 2% +10.0% 44439 softirqs.CPU10.SCHED 39548 +12.2% 44389 softirqs.CPU100.SCHED 39617 +11.4% 44124 softirqs.CPU101.SCHED 39732 +11.0% 44117 ± 2% softirqs.CPU102.SCHED 39719 +10.7% 43971 ± 2% softirqs.CPU103.SCHED 39569 +12.3% 44429 softirqs.CPU104.SCHED 39695 +11.8% 44369 softirqs.CPU105.SCHED 39780 +11.8% 44474 softirqs.CPU106.SCHED 39704 +11.6% 44323 softirqs.CPU107.SCHED 39774 +11.4% 44293 softirqs.CPU108.SCHED 39417 +12.7% 44405 softirqs.CPU109.SCHED 40438 ± 3% +11.2% 44971 ± 2% softirqs.CPU11.SCHED 39889 +11.4% 44417 softirqs.CPU110.SCHED 39655 +11.9% 44368 softirqs.CPU111.SCHED 39585 +11.6% 44177 softirqs.CPU112.SCHED 39577 +12.3% 44447 softirqs.CPU113.SCHED 39611 +11.5% 44181 softirqs.CPU114.SCHED 39986 ± 2% +10.8% 44286 softirqs.CPU115.SCHED 39582 +11.5% 44144 softirqs.CPU116.SCHED 39611 +11.8% 44304 softirqs.CPU117.SCHED 39809 +11.4% 44336 softirqs.CPU118.SCHED 39814 +12.1% 44612 softirqs.CPU119.SCHED 39940 +12.1% 44760 softirqs.CPU12.SCHED 39749 +11.7% 44401 softirqs.CPU120.SCHED 39783 +11.0% 44178 softirqs.CPU121.SCHED 39605 +11.7% 44229 softirqs.CPU122.SCHED 39618 +11.5% 44168 softirqs.CPU123.SCHED 39619 +11.4% 44131 softirqs.CPU124.SCHED 39807 +11.5% 44393 softirqs.CPU125.SCHED 39451 +12.1% 44238 softirqs.CPU126.SCHED 39502 +12.5% 44436 softirqs.CPU127.SCHED 39511 +12.6% 44482 softirqs.CPU128.SCHED 39667 +12.6% 44650 softirqs.CPU129.SCHED 40439 ± 2% -9.6% 36553 softirqs.CPU13.RCU 39894 +11.9% 44647 softirqs.CPU13.SCHED 39549 +12.1% 44329 softirqs.CPU130.SCHED 39501 +12.0% 44231 softirqs.CPU131.SCHED 39530 +12.5% 44454 softirqs.CPU132.SCHED 39476 +12.8% 44530 softirqs.CPU133.SCHED 39479 +12.3% 44316 softirqs.CPU134.SCHED 39573 +12.6% 44549 softirqs.CPU135.SCHED 39588 +12.5% 44555 softirqs.CPU136.SCHED 39608 +13.5% 44967 softirqs.CPU137.SCHED 39541 +12.8% 44616 softirqs.CPU138.SCHED 39610 +12.4% 44540 softirqs.CPU139.SCHED 39834 +11.3% 44317 softirqs.CPU14.SCHED 39624 +11.8% 44296 softirqs.CPU140.SCHED 39527 +12.3% 44372 softirqs.CPU141.SCHED 39620 +12.0% 44370 softirqs.CPU142.SCHED 38415 +12.9% 43354 softirqs.CPU143.SCHED 39602 +12.6% 44590 softirqs.CPU15.SCHED 39793 +11.9% 44530 softirqs.CPU16.SCHED 39640 +12.4% 44540 softirqs.CPU17.SCHED 39956 +10.7% 44217 softirqs.CPU18.SCHED 39621 +11.6% 44204 softirqs.CPU20.SCHED 39654 +11.9% 44384 softirqs.CPU21.SCHED 39758 +11.8% 44468 softirqs.CPU22.SCHED 39464 +12.1% 44230 softirqs.CPU23.SCHED 39527 +12.5% 44462 softirqs.CPU24.SCHED 39573 +12.9% 44696 softirqs.CPU25.SCHED 39511 +13.2% 44733 softirqs.CPU26.SCHED 39301 +12.8% 44327 softirqs.CPU27.SCHED 39594 +12.2% 44440 softirqs.CPU28.SCHED 39530 +12.1% 44331 softirqs.CPU29.SCHED 40745 ± 2% +10.5% 45016 softirqs.CPU3.SCHED 39593 +11.2% 44014 ± 2% softirqs.CPU30.SCHED 39404 +11.9% 44111 softirqs.CPU31.SCHED 39750 +12.3% 44644 softirqs.CPU32.SCHED 39527 +12.3% 44373 softirqs.CPU33.SCHED 39569 +12.6% 44562 softirqs.CPU34.SCHED 39614 +11.8% 44274 softirqs.CPU35.SCHED 39718 +12.8% 44784 ± 2% softirqs.CPU36.SCHED 39774 +11.7% 44447 softirqs.CPU37.SCHED 39537 +12.4% 44453 softirqs.CPU38.SCHED 39810 +11.2% 44249 softirqs.CPU39.SCHED 40382 +9.3% 44158 softirqs.CPU4.SCHED 39613 +12.0% 44357 softirqs.CPU40.SCHED 39493 +11.9% 44204 softirqs.CPU41.SCHED 39795 +11.3% 44287 softirqs.CPU42.SCHED 39513 +12.3% 44366 softirqs.CPU43.SCHED 39584 +12.8% 44666 softirqs.CPU44.SCHED 39826 +11.7% 44495 softirqs.CPU45.SCHED 39591 +12.0% 44349 softirqs.CPU46.SCHED 39825 +11.4% 44360 softirqs.CPU47.SCHED 39847 +11.1% 44287 softirqs.CPU48.SCHED 39624 +12.5% 44565 softirqs.CPU49.SCHED 40050 ± 2% +11.0% 44437 softirqs.CPU5.SCHED 39494 +12.8% 44543 softirqs.CPU50.SCHED 39528 +12.4% 44430 softirqs.CPU51.SCHED 39751 +11.5% 44340 softirqs.CPU52.SCHED 39569 +12.2% 44416 softirqs.CPU53.SCHED 39671 +12.5% 44619 softirqs.CPU54.SCHED 39458 +13.0% 44570 softirqs.CPU55.SCHED 39296 +12.8% 44309 softirqs.CPU56.SCHED 39779 ± 2% +11.8% 44471 softirqs.CPU57.SCHED 39415 +12.9% 44485 softirqs.CPU58.SCHED 39429 +13.3% 44686 softirqs.CPU59.SCHED 39755 +11.5% 44323 softirqs.CPU6.SCHED 39560 +12.1% 44359 softirqs.CPU60.SCHED 39407 +12.9% 44503 softirqs.CPU61.SCHED 39411 +12.6% 44363 softirqs.CPU62.SCHED 39374 +12.7% 44381 softirqs.CPU63.SCHED 39410 +12.6% 44369 softirqs.CPU64.SCHED 39474 +12.6% 44429 softirqs.CPU65.SCHED 39445 +13.0% 44572 softirqs.CPU66.SCHED 39465 +12.5% 44414 softirqs.CPU67.SCHED 39264 +13.8% 44688 softirqs.CPU68.SCHED 39491 +12.7% 44490 softirqs.CPU69.SCHED 40205 +10.5% 44436 softirqs.CPU7.SCHED 39377 +13.1% 44544 softirqs.CPU70.SCHED 38893 +12.8% 43859 softirqs.CPU71.SCHED 38570 ± 2% +12.8% 43503 ± 3% softirqs.CPU72.SCHED 39217 +15.5% 45292 ± 3% softirqs.CPU73.SCHED 39924 +11.2% 44388 softirqs.CPU74.SCHED 38940 ± 2% +13.3% 44123 ± 2% softirqs.CPU75.SCHED 39777 ± 2% +11.4% 44307 softirqs.CPU76.SCHED 39677 +12.0% 44440 softirqs.CPU77.SCHED 39672 +12.2% 44516 softirqs.CPU78.SCHED 39834 +11.9% 44563 softirqs.CPU79.SCHED 39728 +13.1% 44944 softirqs.CPU8.SCHED 39883 +11.1% 44325 softirqs.CPU80.SCHED 39893 +10.5% 44067 softirqs.CPU81.SCHED 39846 +12.0% 44618 softirqs.CPU82.SCHED 39675 +11.8% 44339 softirqs.CPU83.SCHED 39910 +11.3% 44403 softirqs.CPU84.SCHED 39656 +11.7% 44310 softirqs.CPU85.SCHED 40229 ± 2% +9.9% 44199 softirqs.CPU86.SCHED 39820 +11.6% 44429 softirqs.CPU87.SCHED 39768 +12.0% 44535 softirqs.CPU88.SCHED 39855 +11.4% 44408 softirqs.CPU89.SCHED 40123 +10.9% 44489 softirqs.CPU9.SCHED 39750 +10.6% 43956 ± 2% softirqs.CPU90.SCHED 39661 +10.9% 43983 softirqs.CPU91.SCHED 39733 +12.1% 44522 softirqs.CPU92.SCHED 39721 +11.9% 44447 softirqs.CPU93.SCHED 39530 +12.9% 44649 softirqs.CPU94.SCHED 39469 +11.9% 44172 softirqs.CPU95.SCHED 39562 +12.2% 44399 softirqs.CPU96.SCHED 39536 +11.6% 44122 softirqs.CPU97.SCHED 39631 +12.4% 44545 softirqs.CPU98.SCHED 39610 +12.1% 44410 softirqs.CPU99.SCHED 5716307 +11.9% 6399285 softirqs.SCHED 16571119 -12.3% 14527643 interrupts.CAL:Function_call_interrupts 117562 ± 2% -14.6% 100444 ± 2% interrupts.CPU0.CAL:Function_call_interrupts 4567 ± 2% +28.6% 5871 ± 4% interrupts.CPU0.RES:Rescheduling_interrupts 118438 ± 3% -13.5% 102456 ± 2% interrupts.CPU1.CAL:Function_call_interrupts 4650 ± 3% +29.1% 6003 ± 3% interrupts.CPU1.RES:Rescheduling_interrupts 120051 ± 3% -15.8% 101126 ± 4% interrupts.CPU10.CAL:Function_call_interrupts 4636 ± 3% +29.9% 6024 ± 3% interrupts.CPU10.RES:Rescheduling_interrupts 113920 ± 2% -12.6% 99611 interrupts.CPU100.CAL:Function_call_interrupts 4408 ± 2% +34.0% 5908 ± 2% interrupts.CPU100.RES:Rescheduling_interrupts 113352 -12.1% 99611 ± 2% interrupts.CPU101.CAL:Function_call_interrupts 4397 ± 2% +32.5% 5824 ± 2% interrupts.CPU101.RES:Rescheduling_interrupts 113263 -13.8% 97676 ± 3% interrupts.CPU102.CAL:Function_call_interrupts 4355 +30.2% 5670 ± 2% interrupts.CPU102.RES:Rescheduling_interrupts 113423 ± 2% -12.8% 98939 interrupts.CPU103.CAL:Function_call_interrupts 4416 ± 2% +29.9% 5738 ± 3% interrupts.CPU103.RES:Rescheduling_interrupts 112381 -11.6% 99361 ± 2% interrupts.CPU104.CAL:Function_call_interrupts 4454 ± 2% +30.3% 5803 ± 2% interrupts.CPU104.RES:Rescheduling_interrupts 113390 -11.9% 99946 ± 2% interrupts.CPU105.CAL:Function_call_interrupts 4430 ± 2% +32.9% 5887 interrupts.CPU105.RES:Rescheduling_interrupts 113283 -12.3% 99332 ± 3% interrupts.CPU106.CAL:Function_call_interrupts 4340 ± 2% +33.5% 5795 ± 2% interrupts.CPU106.RES:Rescheduling_interrupts 113098 ± 2% -12.6% 98829 ± 2% interrupts.CPU107.CAL:Function_call_interrupts 4476 +28.4% 5746 ± 2% interrupts.CPU107.RES:Rescheduling_interrupts 113419 ± 3% -12.6% 99149 interrupts.CPU108.CAL:Function_call_interrupts 4504 ± 3% +29.8% 5846 interrupts.CPU108.RES:Rescheduling_interrupts 113307 ± 4% -11.9% 99841 ± 2% interrupts.CPU109.CAL:Function_call_interrupts 4548 ± 4% +29.1% 5871 interrupts.CPU109.RES:Rescheduling_interrupts 119217 ± 2% -14.7% 101648 ± 2% interrupts.CPU11.CAL:Function_call_interrupts 4683 ± 3% +26.1% 5903 interrupts.CPU11.RES:Rescheduling_interrupts 114026 ± 3% -13.7% 98387 ± 2% interrupts.CPU110.CAL:Function_call_interrupts 4574 ± 6% +26.3% 5778 interrupts.CPU110.RES:Rescheduling_interrupts 113317 ± 3% -12.6% 99076 interrupts.CPU111.CAL:Function_call_interrupts 4566 ± 5% +27.9% 5839 interrupts.CPU111.RES:Rescheduling_interrupts 113456 ± 3% -13.5% 98087 interrupts.CPU112.CAL:Function_call_interrupts 4505 ± 5% +27.4% 5742 interrupts.CPU112.RES:Rescheduling_interrupts 114088 ± 3% -12.9% 99376 interrupts.CPU113.CAL:Function_call_interrupts 4554 ± 4% +27.4% 5804 interrupts.CPU113.RES:Rescheduling_interrupts 113096 ± 4% -12.7% 98700 interrupts.CPU114.CAL:Function_call_interrupts 4521 ± 4% +29.2% 5840 interrupts.CPU114.RES:Rescheduling_interrupts 114568 ± 3% -13.0% 99623 interrupts.CPU115.CAL:Function_call_interrupts 4570 ± 4% +25.4% 5729 interrupts.CPU115.RES:Rescheduling_interrupts 113198 ± 3% -12.5% 99096 interrupts.CPU116.CAL:Function_call_interrupts 4521 ± 4% +26.2% 5704 interrupts.CPU116.RES:Rescheduling_interrupts 114060 ± 4% -11.8% 100611 interrupts.CPU117.CAL:Function_call_interrupts 4531 ± 3% +29.3% 5858 interrupts.CPU117.RES:Rescheduling_interrupts 114659 ± 4% -13.0% 99794 interrupts.CPU118.CAL:Function_call_interrupts 4581 ± 4% +27.5% 5842 ± 2% interrupts.CPU118.RES:Rescheduling_interrupts 113876 ± 4% -12.3% 99878 interrupts.CPU119.CAL:Function_call_interrupts 4642 ± 5% +24.5% 5779 interrupts.CPU119.RES:Rescheduling_interrupts 119743 ± 3% -15.5% 101160 ± 3% interrupts.CPU12.CAL:Function_call_interrupts 4663 ± 3% +24.8% 5818 ± 3% interrupts.CPU12.RES:Rescheduling_interrupts 113227 ± 3% -12.3% 99343 ± 2% interrupts.CPU120.CAL:Function_call_interrupts 4518 ± 4% +26.8% 5730 interrupts.CPU120.RES:Rescheduling_interrupts 113852 ± 3% -13.5% 98469 interrupts.CPU121.CAL:Function_call_interrupts 4587 ± 4% +22.3% 5612 ± 2% interrupts.CPU121.RES:Rescheduling_interrupts 113775 ± 4% -12.2% 99917 interrupts.CPU122.CAL:Function_call_interrupts 4552 ± 4% +30.5% 5940 ± 2% interrupts.CPU122.RES:Rescheduling_interrupts 114003 ± 3% -12.6% 99683 interrupts.CPU123.CAL:Function_call_interrupts 4581 ± 4% +29.1% 5914 interrupts.CPU123.RES:Rescheduling_interrupts 113665 ± 3% -12.5% 99437 interrupts.CPU124.CAL:Function_call_interrupts 4562 ± 3% +25.9% 5745 interrupts.CPU124.RES:Rescheduling_interrupts 114325 ± 3% -12.8% 99659 interrupts.CPU125.CAL:Function_call_interrupts 4556 ± 3% +25.2% 5703 interrupts.CPU125.RES:Rescheduling_interrupts 4409 +40.8% 6209 ± 4% interrupts.CPU126.RES:Rescheduling_interrupts 4502 +37.7% 6197 ± 2% interrupts.CPU127.RES:Rescheduling_interrupts 4449 +38.5% 6161 ± 4% interrupts.CPU128.RES:Rescheduling_interrupts 4531 +35.1% 6120 ± 5% interrupts.CPU129.RES:Rescheduling_interrupts 119801 ± 3% -15.5% 101174 ± 2% interrupts.CPU13.CAL:Function_call_interrupts 4638 ± 3% +25.5% 5820 ± 3% interrupts.CPU13.RES:Rescheduling_interrupts 4470 +34.9% 6029 ± 3% interrupts.CPU130.RES:Rescheduling_interrupts 4450 +36.8% 6089 ± 5% interrupts.CPU131.RES:Rescheduling_interrupts 4479 ± 2% +39.0% 6224 ± 4% interrupts.CPU132.RES:Rescheduling_interrupts 4441 +39.3% 6188 ± 4% interrupts.CPU133.RES:Rescheduling_interrupts 4452 +35.7% 6042 ± 4% interrupts.CPU134.RES:Rescheduling_interrupts 4409 ± 2% +41.0% 6215 ± 4% interrupts.CPU135.RES:Rescheduling_interrupts 4465 +38.4% 6178 ± 4% interrupts.CPU136.RES:Rescheduling_interrupts 4432 +38.4% 6135 ± 4% interrupts.CPU137.RES:Rescheduling_interrupts 4460 +35.7% 6052 ± 5% interrupts.CPU138.RES:Rescheduling_interrupts 4421 ± 2% +35.6% 5995 ± 5% interrupts.CPU139.RES:Rescheduling_interrupts 121313 ± 2% -15.9% 102004 ± 2% interrupts.CPU14.CAL:Function_call_interrupts 4695 ± 3% +27.6% 5990 ± 2% interrupts.CPU14.RES:Rescheduling_interrupts 4436 +40.4% 6229 ± 4% interrupts.CPU140.RES:Rescheduling_interrupts 4457 +37.9% 6148 ± 6% interrupts.CPU141.RES:Rescheduling_interrupts 4475 ± 2% +36.1% 6089 ± 5% interrupts.CPU142.RES:Rescheduling_interrupts 4404 +34.6% 5931 ± 6% interrupts.CPU143.RES:Rescheduling_interrupts 120281 ± 3% -14.8% 102424 ± 3% interrupts.CPU15.CAL:Function_call_interrupts 4692 ± 3% +26.9% 5957 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts 120055 ± 3% -15.8% 101111 ± 4% interrupts.CPU16.CAL:Function_call_interrupts 4659 ± 4% +27.4% 5935 ± 3% interrupts.CPU16.RES:Rescheduling_interrupts 118571 ± 3% -14.5% 101336 ± 2% interrupts.CPU17.CAL:Function_call_interrupts 4604 ± 3% +26.7% 5833 ± 3% interrupts.CPU17.RES:Rescheduling_interrupts 113394 -13.5% 98125 ± 3% interrupts.CPU18.CAL:Function_call_interrupts 4438 ± 2% +33.7% 5932 ± 3% interrupts.CPU18.RES:Rescheduling_interrupts 113233 -13.5% 97937 ± 2% interrupts.CPU19.CAL:Function_call_interrupts 4495 +30.6% 5870 ± 3% interrupts.CPU19.RES:Rescheduling_interrupts 120349 ± 3% -15.2% 102075 ± 2% interrupts.CPU2.CAL:Function_call_interrupts 4765 ± 4% +25.5% 5980 interrupts.CPU2.RES:Rescheduling_interrupts 112237 -11.9% 98873 interrupts.CPU20.CAL:Function_call_interrupts 4500 ± 2% +31.1% 5899 ± 2% interrupts.CPU20.RES:Rescheduling_interrupts 113072 -12.3% 99167 ± 2% interrupts.CPU21.CAL:Function_call_interrupts 4429 ± 3% +31.7% 5833 ± 2% interrupts.CPU21.RES:Rescheduling_interrupts 113179 ± 2% -12.4% 99128 ± 2% interrupts.CPU22.CAL:Function_call_interrupts 4442 +30.9% 5813 ± 2% interrupts.CPU22.RES:Rescheduling_interrupts 113630 ± 2% -13.1% 98735 ± 2% interrupts.CPU23.CAL:Function_call_interrupts 4509 ± 3% +30.7% 5894 ± 2% interrupts.CPU23.RES:Rescheduling_interrupts 113299 -13.0% 98576 ± 2% interrupts.CPU24.CAL:Function_call_interrupts 4507 ± 3% +30.8% 5896 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts 113167 -11.8% 99784 ± 2% interrupts.CPU25.CAL:Function_call_interrupts 4470 +31.9% 5897 ± 2% interrupts.CPU25.RES:Rescheduling_interrupts 113540 ± 2% -12.5% 99356 ± 2% interrupts.CPU26.CAL:Function_call_interrupts 4476 ± 2% +28.3% 5740 ± 2% interrupts.CPU26.RES:Rescheduling_interrupts 112997 -11.0% 100561 interrupts.CPU27.CAL:Function_call_interrupts 4469 +34.0% 5991 ± 2% interrupts.CPU27.RES:Rescheduling_interrupts 113418 -12.1% 99670 interrupts.CPU28.CAL:Function_call_interrupts 4490 +32.0% 5926 interrupts.CPU28.RES:Rescheduling_interrupts 113460 -12.3% 99507 interrupts.CPU29.CAL:Function_call_interrupts 4495 ± 2% +31.1% 5894 interrupts.CPU29.RES:Rescheduling_interrupts 119931 ± 3% -15.2% 101684 ± 2% interrupts.CPU3.CAL:Function_call_interrupts 4670 ± 4% +27.2% 5941 ± 3% interrupts.CPU3.RES:Rescheduling_interrupts 113980 -13.9% 98187 ± 3% interrupts.CPU30.CAL:Function_call_interrupts 4505 +27.9% 5763 ± 2% interrupts.CPU30.RES:Rescheduling_interrupts 113880 ± 2% -13.1% 98939 interrupts.CPU31.CAL:Function_call_interrupts 4474 ± 4% +30.3% 5828 ± 2% interrupts.CPU31.RES:Rescheduling_interrupts 113746 -12.7% 99354 ± 2% interrupts.CPU32.CAL:Function_call_interrupts 4496 ± 2% +32.9% 5973 interrupts.CPU32.RES:Rescheduling_interrupts 113612 -12.3% 99599 interrupts.CPU33.CAL:Function_call_interrupts 4415 +34.6% 5941 interrupts.CPU33.RES:Rescheduling_interrupts 113571 -11.9% 100037 interrupts.CPU34.CAL:Function_call_interrupts 4418 +34.1% 5925 interrupts.CPU34.RES:Rescheduling_interrupts 113216 -12.9% 98557 interrupts.CPU35.CAL:Function_call_interrupts 4455 +29.8% 5783 interrupts.CPU35.RES:Rescheduling_interrupts 113931 ± 3% -12.2% 100068 interrupts.CPU36.CAL:Function_call_interrupts 4558 ± 3% +28.6% 5863 interrupts.CPU36.RES:Rescheduling_interrupts 113357 ± 3% -11.9% 99846 interrupts.CPU37.CAL:Function_call_interrupts 4578 ± 3% +30.9% 5991 interrupts.CPU37.RES:Rescheduling_interrupts 114061 ± 3% -12.8% 99425 ± 2% interrupts.CPU38.CAL:Function_call_interrupts 4579 ± 4% +29.0% 5908 interrupts.CPU38.RES:Rescheduling_interrupts 114093 ± 3% -13.4% 98844 interrupts.CPU39.CAL:Function_call_interrupts 4545 ± 4% +28.5% 5839 ± 2% interrupts.CPU39.RES:Rescheduling_interrupts 119669 ± 3% -15.4% 101264 ± 4% interrupts.CPU4.CAL:Function_call_interrupts 4680 ± 5% +23.8% 5795 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts 114128 ± 3% -13.4% 98821 interrupts.CPU40.CAL:Function_call_interrupts 4560 ± 4% +27.8% 5830 interrupts.CPU40.RES:Rescheduling_interrupts 113453 ± 3% -13.5% 98157 interrupts.CPU41.CAL:Function_call_interrupts 4527 ± 4% +29.2% 5848 interrupts.CPU41.RES:Rescheduling_interrupts 114132 ± 3% -12.7% 99688 interrupts.CPU42.CAL:Function_call_interrupts 4546 ± 4% +29.9% 5908 interrupts.CPU42.RES:Rescheduling_interrupts 114812 ± 3% -13.3% 99518 interrupts.CPU43.CAL:Function_call_interrupts 4630 ± 3% +25.8% 5827 interrupts.CPU43.RES:Rescheduling_interrupts 113893 ± 3% -12.7% 99442 interrupts.CPU44.CAL:Function_call_interrupts 4569 ± 3% +26.1% 5760 interrupts.CPU44.RES:Rescheduling_interrupts 113936 ± 3% -12.4% 99787 interrupts.CPU45.CAL:Function_call_interrupts 4602 ± 3% +27.6% 5871 interrupts.CPU45.RES:Rescheduling_interrupts 114508 ± 3% -13.4% 99177 ± 2% interrupts.CPU46.CAL:Function_call_interrupts 4604 ± 4% +29.7% 5970 ± 2% interrupts.CPU46.RES:Rescheduling_interrupts 114139 ± 3% -13.2% 99111 interrupts.CPU47.CAL:Function_call_interrupts 4599 ± 4% +28.2% 5894 ± 2% interrupts.CPU47.RES:Rescheduling_interrupts 114560 ± 3% -13.2% 99482 ± 2% interrupts.CPU48.CAL:Function_call_interrupts 4538 ± 5% +28.6% 5837 ± 2% interrupts.CPU48.RES:Rescheduling_interrupts 114404 ± 3% -13.6% 98807 interrupts.CPU49.CAL:Function_call_interrupts 4576 ± 3% +26.1% 5771 interrupts.CPU49.RES:Rescheduling_interrupts 120437 ± 3% -16.7% 100268 ± 3% interrupts.CPU5.CAL:Function_call_interrupts 4754 ± 3% +25.3% 5956 ± 2% interrupts.CPU5.RES:Rescheduling_interrupts 114130 ± 4% -12.6% 99805 interrupts.CPU50.CAL:Function_call_interrupts 4586 ± 4% +27.6% 5852 interrupts.CPU50.RES:Rescheduling_interrupts 114785 ± 3% -12.8% 100057 interrupts.CPU51.CAL:Function_call_interrupts 4560 ± 3% +28.0% 5837 interrupts.CPU51.RES:Rescheduling_interrupts 114501 ± 3% -13.4% 99161 interrupts.CPU52.CAL:Function_call_interrupts 4526 ± 3% +28.2% 5805 interrupts.CPU52.RES:Rescheduling_interrupts 114922 ± 3% -13.0% 100003 interrupts.CPU53.CAL:Function_call_interrupts 4517 ± 4% +28.2% 5790 interrupts.CPU53.RES:Rescheduling_interrupts 4469 ± 2% +39.8% 6249 ± 4% interrupts.CPU54.RES:Rescheduling_interrupts 4529 +37.6% 6231 ± 5% interrupts.CPU55.RES:Rescheduling_interrupts 4504 +38.1% 6218 ± 4% interrupts.CPU56.RES:Rescheduling_interrupts 4523 +37.3% 6209 ± 5% interrupts.CPU57.RES:Rescheduling_interrupts 4461 +37.1% 6117 ± 4% interrupts.CPU58.RES:Rescheduling_interrupts 4471 +40.2% 6267 ± 5% interrupts.CPU59.RES:Rescheduling_interrupts 120155 ± 3% -15.2% 101834 ± 2% interrupts.CPU6.CAL:Function_call_interrupts 4692 ± 4% +27.5% 5984 ± 2% interrupts.CPU6.RES:Rescheduling_interrupts 4495 +38.6% 6232 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts 4543 +36.3% 6192 ± 5% interrupts.CPU61.RES:Rescheduling_interrupts 4523 +34.6% 6086 ± 5% interrupts.CPU62.RES:Rescheduling_interrupts 4531 ± 2% +38.1% 6256 ± 4% interrupts.CPU63.RES:Rescheduling_interrupts 4510 ± 2% +39.3% 6283 ± 4% interrupts.CPU64.RES:Rescheduling_interrupts 4531 +37.4% 6227 ± 5% interrupts.CPU65.RES:Rescheduling_interrupts 4486 ± 2% +35.2% 6065 ± 4% interrupts.CPU66.RES:Rescheduling_interrupts 4459 ± 2% +37.8% 6145 ± 4% interrupts.CPU67.RES:Rescheduling_interrupts 4494 +38.8% 6239 ± 4% interrupts.CPU68.RES:Rescheduling_interrupts 4495 +35.8% 6105 ± 4% interrupts.CPU69.RES:Rescheduling_interrupts 120123 ± 3% -15.7% 101289 ± 2% interrupts.CPU7.CAL:Function_call_interrupts 4712 ± 2% +24.9% 5887 ± 2% interrupts.CPU7.RES:Rescheduling_interrupts 4488 ± 2% +36.1% 6108 ± 4% interrupts.CPU70.RES:Rescheduling_interrupts 4520 ± 2% +32.9% 6006 ± 5% interrupts.CPU71.RES:Rescheduling_interrupts 119021 ± 2% -17.1% 98617 ± 3% interrupts.CPU72.CAL:Function_call_interrupts 4529 ± 3% +27.5% 5776 ± 4% interrupts.CPU72.RES:Rescheduling_interrupts 118495 ± 4% -14.7% 101128 ± 2% interrupts.CPU73.CAL:Function_call_interrupts 4690 ± 4% +27.1% 5962 interrupts.CPU73.RES:Rescheduling_interrupts 119467 ± 3% -14.8% 101845 ± 3% interrupts.CPU74.CAL:Function_call_interrupts 4653 ± 4% +26.7% 5894 ± 2% interrupts.CPU74.RES:Rescheduling_interrupts 119849 ± 3% -15.6% 101207 ± 3% interrupts.CPU75.CAL:Function_call_interrupts 4600 ± 2% +27.2% 5850 ± 2% interrupts.CPU75.RES:Rescheduling_interrupts 119432 ± 2% -15.3% 101151 ± 2% interrupts.CPU76.CAL:Function_call_interrupts 4643 ± 3% +24.0% 5757 ± 2% interrupts.CPU76.RES:Rescheduling_interrupts 119531 ± 3% -14.9% 101762 ± 2% interrupts.CPU77.CAL:Function_call_interrupts 4647 ± 3% +26.8% 5895 ± 2% interrupts.CPU77.RES:Rescheduling_interrupts 119283 ± 3% -15.1% 101219 ± 2% interrupts.CPU78.CAL:Function_call_interrupts 4660 ± 4% +27.1% 5922 ± 3% interrupts.CPU78.RES:Rescheduling_interrupts 119825 ± 2% -15.8% 100865 ± 2% interrupts.CPU79.CAL:Function_call_interrupts 4652 ± 3% +25.2% 5826 ± 2% interrupts.CPU79.RES:Rescheduling_interrupts 120936 ± 3% -15.8% 101816 ± 3% interrupts.CPU8.CAL:Function_call_interrupts 4686 ± 4% +25.4% 5877 ± 3% interrupts.CPU8.RES:Rescheduling_interrupts 120140 ± 3% -15.7% 101313 ± 3% interrupts.CPU80.CAL:Function_call_interrupts 4603 ± 3% +26.0% 5800 ± 3% interrupts.CPU80.RES:Rescheduling_interrupts 119906 ± 3% -15.6% 101215 ± 2% interrupts.CPU81.CAL:Function_call_interrupts 4622 ± 3% +27.3% 5882 ± 2% interrupts.CPU81.RES:Rescheduling_interrupts 120619 ± 3% -16.1% 101195 ± 2% interrupts.CPU82.CAL:Function_call_interrupts 4705 ± 4% +25.8% 5919 ± 2% interrupts.CPU82.RES:Rescheduling_interrupts 119235 ± 2% -15.2% 101105 ± 3% interrupts.CPU83.CAL:Function_call_interrupts 4593 ± 4% +26.9% 5830 ± 3% interrupts.CPU83.RES:Rescheduling_interrupts 120080 ± 3% -15.7% 101259 ± 2% interrupts.CPU84.CAL:Function_call_interrupts 4627 ± 3% +24.8% 5772 ± 3% interrupts.CPU84.RES:Rescheduling_interrupts 119739 ± 3% -15.5% 101237 ± 2% interrupts.CPU85.CAL:Function_call_interrupts 4609 ± 3% +25.6% 5788 ± 3% interrupts.CPU85.RES:Rescheduling_interrupts 120114 ± 2% -15.2% 101826 ± 2% interrupts.CPU86.CAL:Function_call_interrupts 4653 ± 4% +26.6% 5893 ± 2% interrupts.CPU86.RES:Rescheduling_interrupts 119989 ± 2% -14.8% 102229 ± 2% interrupts.CPU87.CAL:Function_call_interrupts 4692 ± 3% +27.2% 5967 ± 2% interrupts.CPU87.RES:Rescheduling_interrupts 119033 ± 3% -14.1% 102257 ± 3% interrupts.CPU88.CAL:Function_call_interrupts 4619 ± 2% +28.5% 5937 ± 3% interrupts.CPU88.RES:Rescheduling_interrupts 119930 ± 4% -15.7% 101158 interrupts.CPU89.CAL:Function_call_interrupts 4619 ± 4% +24.2% 5739 ± 2% interrupts.CPU89.RES:Rescheduling_interrupts 120300 ± 3% -15.5% 101627 ± 2% interrupts.CPU9.CAL:Function_call_interrupts 4738 ± 3% +26.4% 5989 ± 2% interrupts.CPU9.RES:Rescheduling_interrupts 113206 -13.9% 97437 ± 4% interrupts.CPU90.CAL:Function_call_interrupts 4364 ± 2% +32.1% 5767 ± 3% interrupts.CPU90.RES:Rescheduling_interrupts 113083 ± 2% -13.3% 98039 interrupts.CPU91.CAL:Function_call_interrupts 4526 ± 2% +29.2% 5846 ± 2% interrupts.CPU91.RES:Rescheduling_interrupts 114034 -13.1% 99044 interrupts.CPU92.CAL:Function_call_interrupts 4470 +30.8% 5848 interrupts.CPU92.RES:Rescheduling_interrupts 113900 ± 2% -12.5% 99697 ± 2% interrupts.CPU93.CAL:Function_call_interrupts 4493 +28.4% 5770 interrupts.CPU93.RES:Rescheduling_interrupts 113290 -12.6% 99000 ± 2% interrupts.CPU94.CAL:Function_call_interrupts 4455 ± 2% +30.1% 5795 interrupts.CPU94.RES:Rescheduling_interrupts 112981 -13.2% 98041 ± 2% interrupts.CPU95.CAL:Function_call_interrupts 4404 +31.2% 5779 ± 2% interrupts.CPU95.RES:Rescheduling_interrupts 114402 ± 2% -12.9% 99619 ± 2% interrupts.CPU96.CAL:Function_call_interrupts 4428 ± 2% +35.7% 6009 ± 2% interrupts.CPU96.RES:Rescheduling_interrupts 113804 -12.8% 99229 ± 2% interrupts.CPU97.CAL:Function_call_interrupts 4446 ± 2% +30.8% 5814 ± 2% interrupts.CPU97.RES:Rescheduling_interrupts 113096 -12.2% 99332 ± 2% interrupts.CPU98.CAL:Function_call_interrupts 4421 ± 2% +31.1% 5795 interrupts.CPU98.RES:Rescheduling_interrupts 113021 ± 2% -12.1% 99328 ± 2% interrupts.CPU99.CAL:Function_call_interrupts 4367 +33.3% 5821 ± 2% interrupts.CPU99.RES:Rescheduling_interrupts 653136 +30.7% 853583 interrupts.RES:Rescheduling_interrupts aim7.jobs-per-min 23000 +-------------------------------------------------------------------+ |O OO O OO OO OO O OOO O O O O O OO OO O O | 22500 |-+O O O O O O O O O O O O O O | | O O O O O | | | 22000 |-+ | | | 21500 |-+ | | | 21000 |-+ .++ | |+. + .+ + +. + .+++. + +.+ + .+ .+ ++ +.+ | |: + ++ ++.+ ++.++ ++ + ++ :.++ +.++ + + ++ + +++.+| 20500 |-+ + + | | | 20000 +-------------------------------------------------------------------+ 0.72 +--------------------------------------------------------------------+ 0.7 |-+ + + | | .+ .++ +.+ + :: + .+ :+ | 0.68 |+.+ +.+ + + :+ + + .+ +. .+ +.+ : : +: +.++ + + +| 0.66 |-+ + ++ + : + + + ++++.+++ +++ ++ :: + | 0.64 |-+ ++ + | 0.62 |-+ | | | 0.6 |-+ | 0.58 |-+ | 0.56 |O+ O O O OO | 0.54 |-+ O O O O O OO OO O O O O O O O | | O O O O O O O O O O OO OO O O | 0.52 |-+ O O O O OO | 0.5 +--------------------------------------------------------------------+ 0.13 +--------------------------------------------------------------------+ 0.12 |+. + +. +.++++.+ .+ +. | | ++ + +++. + ++ ++ + ++++.+++ | 0.11 |-+ + + : + | 0.1 |-+ : + .+ +.+ +.+ +. + + ++.+| | + + ++ :: + ++ + | 0.09 |-+ + | 0.08 |-+ | 0.07 |-+ | | | 0.06 |-+ | 0.05 |-+ | |O O O O O | 0.04 |-+O O OOOO OOO OOO O OO OO OOO OOO O O OOO OOO OOO O | 0.03 +--------------------------------------------------------------------+ 0.85 +--------------------------------------------------------------------+ | .+ +. | 0.8 |+. .+ + +.+ + ++ ++ .+ +. + + .+ +. | | +++ ++ + + + + + ++++.+++.+ +.+ + :+.+ +.++++ + +| 0.75 |-+ + +++ + + | | | 0.7 |-+ | | | 0.65 |-+ | | | 0.6 |O+ O O OOO O OO O | | OOO O O O O O OO OOO O O O O OO OO O O O | 0.55 |-+ OO O OO O O O OO | | | 0.5 +--------------------------------------------------------------------+ 240 +---------------------------------------------------------------------+ | .+ +. +. + .+ + + + + | 220 |+.+++ : .+++.+ + ++ +++. :+ :+.+ +.+ +. ::.+++.+ ++.++ ++ + + | | ++ : : + + + + :: :+ + +.+| 200 |-+ : + + | | + | 180 |-+ | | | 160 |-+ | | O O | 140 |O+O O O O O O OO O O O | | O O O O O O O O O OO O O O OO | 120 |-+ O O O O O O OO OO | | O O O O | 100 +---------------------------------------------------------------------+ 240 +---------------------------------------------------------------------+ | .+ +. +. + .+ + + + + | 220 |+.+++ : .+++.+ + ++ +++. :+ :+.+ +.+ +. ::.+++.+ ++.++ ++ + + | | ++ : : + + + + :: :+ + +.+| 200 |-+ : + + | | + | 180 |-+ | | | 160 |-+ | | O O | 140 |O+O O O O O O OO O O O | | O O O O O O O O O OO O O O OO | 120 |-+ O O O O O O OO OO | | O O O O | 100 +---------------------------------------------------------------------+ aim7.time.elapsed_time 178 +---------------------------------------------------------------------+ 176 |-+ +. +. | | .+ +.+ +.+ +++ +.+ : ++ +++. + ++.+| 174 |++ + + + + +.+++.+++.++ + +++.+ +.++++.++ +.+ | 172 |-+ + +.++ | 170 |-+ | 168 |-+ | | | 166 |-+ | 164 |-+ | 162 |-+ O | 160 |-+ O O O O | |O O O OO OO O O OO OO OOO O O OO OO O O O OOO OO | 158 |-+ O O O O O O O O O | 156 +---------------------------------------------------------------------+ aim7.time.elapsed_time.max 178 +---------------------------------------------------------------------+ 176 |-+ +. +. | | .+ +.+ +.+ +++ +.+ : ++ +++. + ++.+| 174 |++ + + + + +.+++.+++.++ + +++.+ +.++++.++ +.+ | 172 |-+ + +.++ | 170 |-+ | 168 |-+ | | | 166 |-+ | 164 |-+ | 162 |-+ O | 160 |-+ O O O O | |O O O OO OO O O OO OO OOO O O OO OO O O O OOO OO | 158 |-+ O O O O O O O O O | 156 +---------------------------------------------------------------------+ aim7.time.involuntary_context_switches 2.6e+06 +-----------------------------------------------------------------+ | O OO OOOO OO O O O O OOO OO O O O OOO O | 2.5e+06 |OO O OO OO OOO O OO O O O O O O | | | 2.4e+06 |-+ | 2.3e+06 |-+ | | | 2.2e+06 |-+ | | | 2.1e+06 |-+ | 2e+06 |-+ | | +. ++ .++++.+ +.++++.++ +.++++.++| 1.9e+06 |++.+++ .++ +.++++.++++.++ +.++ : + + ++ + | | + + + + | 1.8e+06 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months

1
0
0 / 0

[crypto] 1ecd8ea291: INFO:rcu_preempt_detected_stalls_on_CPUs/tasks

by kernel test robot

Greeting, FYI, we noticed the following commit (built with clang-14): commit: 1ecd8ea291a89955841d6beede2679677433eef9 ("Revert "crypto: api - Fix built-in testing dependency failures"") https://github.com/krzk/linux wip/dt-bindings-max77802 in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +--------------------------------------------------------------+------------+------------+ | | 875fd02ecd | 1ecd8ea291 | +--------------------------------------------------------------+------------+------------+ | BUG:workqueue_lockup-pool | 179 | 169 | | INFO:rcu_preempt_detected_stalls_on_CPUs/tasks | 0 | 41 | +--------------------------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 123.368914][ C0] BUG: workqueue lockup - pool cpus=0 flags=0x4 nice=0 stuck for 59s! [ 123.370957][ C0] Showing busy workqueues and worker pools: [ 123.372271][ C0] workqueue events: flags=0x0 [ 123.373291][ C0] pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 [ 123.374927][ C0] pending: stop_one_cpu_nowait_workfn [ 123.376231][ C0] workqueue events_unbound: flags=0x2 [ 123.377440][ C0] pwq 2: cpus=0 flags=0x4 nice=0 active=1/512 refcnt=3 [ 123.379065][ C0] in-flight: 7:async_run_entry_fn [ 123.380256][ C0] workqueue events_power_efficient: flags=0x82 [ 123.381496][ C0] pwq 2: cpus=0 flags=0x4 nice=0 active=3/256 refcnt=5 [ 123.382913][ C0] pending: do_cache_clean, neigh_periodic_work, check_lifetime [ 123.384712][ C0] pool 2: cpus=0 flags=0x4 nice=0 hung=59s workers=3 idle: 82 76 [ 144.562054][ T1] test_free_pages: Testing with GFP_KERNEL | __GFP_COMP [ 165.811939][ C0] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 165.811939][ C0] (detected by 0, t=31502 jiffies, g=21557, q=6) [ 165.811939][ C0] rcu: All QSes seen, last rcu_preempt kthread activity 31502 (4294926781-4294895279), jiffies_till_next_fqs=2, root ->qsmask 0x0 [ 165.811939][ C0] rcu: rcu_preempt kthread starved for 31502 jiffies! g21557 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 [ 165.811939][ C0] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 165.811939][ C0] rcu: RCU grace-period kthread stack dump: [ 165.811939][ C0] task:rcu_preempt state:R running task stack:14928 pid: 14 ppid: 2 flags:0x00004000 [ 165.811939][ C0] Call Trace: [ 165.811939][ C0] <TASK> [ 165.811939][ C0] __schedule (kernel/sched/core.c:4943 kernel/sched/core.c:6221) [ 165.811939][ C0] schedule (kernel/sched/core.c:6301) [ 165.811939][ C0] schedule_timeout (kernel/time/timer.c:1882) [ 165.811939][ C0] ? update_process_times (kernel/time/timer.c:1805) [ 165.811939][ C0] rcu_gp_fqs_loop (kernel/rcu/tree.c:1959) [ 165.811939][ C0] ? rcu_gp_init (kernel/rcu/tree.c:1849) [ 165.811939][ C0] ? preempt_count_sub (kernel/sched/core.c:5432) [ 165.811939][ C0] rcu_gp_kthread (kernel/rcu/tree.c:2135) [ 165.811939][ C0] kthread (kernel/kthread.c:328) [ 165.811939][ C0] ? rcu_report_qs_rsp (kernel/rcu/tree.c:2107) [ 165.811939][ C0] ? kthread_unuse_mm (kernel/kthread.c:272) [ 165.811939][ C0] ret_from_fork (??:?) [ 165.811939][ C0] </TASK> [ 165.811939][ C0] rcu: Stack dump where RCU GP kthread last ran: [ 165.811939][ C0] NMI backtrace for cpu 0 [ 165.811939][ C0] CPU: 0 PID: 1 Comm: swapper Not tainted 5.15.0-rc3-03890-g1ecd8ea291a8 #1 [ 165.811939][ C0] Call Trace: [ 165.811939][ C0] <IRQ> [ 165.811939][ C0] dump_stack_lvl (lib/dump_stack.c:108) [ 165.811939][ C0] dump_stack (lib/dump_stack.c:114) [ 165.811939][ C0] nmi_trigger_cpumask_backtrace (include/linux/instrumented.h:87 include/asm-generic/bitops/instrumented-atomic.h:41 include/linux/cpumask.h:327 lib/nmi_backtrace.c:108 lib/nmi_backtrace.c:62) [ 165.811939][ C0] arch_trigger_cpumask_backtrace (arch/x86/kernel/apic/hw_nmi.c:41) [ 165.811939][ C0] rcu_check_gp_kthread_starvation (kernel/rcu/tree_stall.h:485) [ 165.811939][ C0] rcu_sched_clock_irq (kernel/rcu/tree_stall.h:114 kernel/rcu/tree_stall.h:588 kernel/rcu/tree_stall.h:729 kernel/rcu/tree.c:3882 kernel/rcu/tree.c:2601) [ 165.811939][ C0] ? _raw_spin_unlock_irqrestore (arch/x86/include/asm/preempt.h:103 include/linux/spinlock_api_smp.h:161 kernel/locking/spinlock.c:194) [ 165.811939][ C0] update_process_times (arch/x86/include/asm/preempt.h:27 kernel/time/timer.c:1787) [ 165.811939][ C0] tick_nohz_handler (kernel/time/tick-sched.c:227 kernel/time/tick-sched.c:1320) [ 165.811939][ C0] __sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1104) [ 165.811939][ C0] sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1097) [ 165.811939][ C0] </IRQ> [ 165.811939][ C0] <TASK> [ 165.811939][ C0] asm_sysvec_apic_timer_interrupt (??:?) [ 165.811939][ C0] RIP: 0010:kcsan_setup_watchpoint (kernel/kcsan/core.c:338 kernel/kcsan/core.c:602) [ 165.811939][ C0] Code: 00 48 81 c5 00 08 00 00 80 3d 36 ab 53 03 00 75 15 48 8b 44 24 20 25 00 02 00 00 48 85 c0 74 06 e8 10 87 fb ff fb 48 8b 43 30 <48> 89 45 30 48 8b 43 28 48 89 45 28 48 8b 43 20 48 89 45 20 48 8b All code ======== 0: 00 48 81 add %cl,-0x7f(%rax) 3: c5 00 08 (bad) 6: 00 00 add %al,(%rax) 8: 80 3d 36 ab 53 03 00 cmpb $0x0,0x353ab36(%rip) # 0x353ab45 f: 75 15 jne 0x26 11: 48 8b 44 24 20 mov 0x20(%rsp),%rax 16: 25 00 02 00 00 and $0x200,%eax 1b: 48 85 c0 test %rax,%rax 1e: 74 06 je 0x26 20: e8 10 87 fb ff callq 0xfffffffffffb8735 25: fb sti 26: 48 8b 43 30 mov 0x30(%rbx),%rax 2a:* 48 89 45 30 mov %rax,0x30(%rbp) <-- trapping instruction 2e: 48 8b 43 28 mov 0x28(%rbx),%rax 32: 48 89 45 28 mov %rax,0x28(%rbp) 36: 48 8b 43 20 mov 0x20(%rbx),%rax 3a: 48 89 45 20 mov %rax,0x20(%rbp) 3e: 48 rex.W 3f: 8b .byte 0x8b Code starting with the faulting instruction =========================================== 0: 48 89 45 30 mov %rax,0x30(%rbp) 4: 48 8b 43 28 mov 0x28(%rbx),%rax 8: 48 89 45 28 mov %rax,0x28(%rbp) c: 48 8b 43 20 mov 0x20(%rbx),%rax 10: 48 89 45 20 mov %rax,0x20(%rbp) 14: 48 rex.W 15: 8b .byte 0x8b [ 165.811939][ C0] RSP: 0000:ffff8881001a7870 EFLAGS: 00000282 [ 165.811939][ C0] RAX: 00c4addf00c4add4 RBX: ffff888100262178 RCX: 0000000000000000 [ 165.811939][ C0] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffffffff835cffd8 [ 165.811939][ C0] RBP: ffff888100260800 R08: 00018881001a79bb R09: 0000000000000000 [ 165.811939][ C0] R10: 0000000000000010 R11: 0000000000000010 R12: ffff8881001a79b8 [ 165.811939][ C0] R13: aaaaaaaaaaaaaaaa R14: ffff888100262150 R15: 0000000000000000 [ 165.811939][ C0] ? stack_trace_consume_entry (kernel/stacktrace.c:93) [ 165.811939][ C0] stack_trace_consume_entry (kernel/stacktrace.c:93) [ 165.811939][ C0] ? kernel_init_freeable (init/main.c:1618) [ 165.811939][ C0] ? stack_trace_save (kernel/stacktrace.c:82) [ 165.811939][ C0] arch_stack_walk (arch/x86/kernel/stacktrace.c:27) [ 165.811939][ C0] ? kernel_init_freeable (init/main.c:1618) [ 165.811939][ C0] stack_trace_save (kernel/stacktrace.c:122) [ 165.811939][ C0] save_stack (mm/page_owner.c:120) [ 165.811939][ C0] ? free_pcp_prepare (include/linux/page_owner.h:? mm/page_alloc.c:1338 mm/page_alloc.c:1411) [ 165.811939][ C0] ? free_unref_page (mm/page_alloc.c:3315 mm/page_alloc.c:3394) [ 165.811939][ C0] ? __put_page (mm/swap.c:131) [ 165.811939][ C0] ? test_free_pages (include/linux/mm.h:? include/linux/mm.h:1287 lib/test_free_pages.c:25) [ 165.811939][ C0] ? m_in (lib/test_free_pages.c:35) [ 165.811939][ C0] ? do_one_initcall (init/main.c:1303) [ 165.811939][ C0] ? do_initcall_level (init/main.c:1375) [ 165.811939][ C0] ? do_initcalls (init/main.c:1389) [ 165.811939][ C0] ? do_basic_setup (init/main.c:1412) [ 165.811939][ C0] ? kernel_init_freeable (init/main.c:1618) [ 165.811939][ C0] __reset_page_owner (mm/page_owner.c:140) [ 165.811939][ C0] free_pcp_prepare (include/linux/page_owner.h:? mm/page_alloc.c:1338 mm/page_alloc.c:1411) [ 165.811939][ C0] free_unref_page (mm/page_alloc.c:3315 mm/page_alloc.c:3394) [ 165.811939][ C0] __put_page (mm/swap.c:131) [ 165.811939][ C0] test_free_pages (include/linux/mm.h:? include/linux/mm.h:1287 lib/test_free_pages.c:25) [ 165.811939][ C0] ? __leaf_switch_none (lib/test_free_pages.c:31) [ 165.811939][ C0] m_in (lib/test_free_pages.c:35) [ 165.811939][ C0] do_one_initcall (init/main.c:1303) [ 165.811939][ C0] ? rcu_read_lock_sched_held (kernel/rcu/update.c:125) [ 165.811939][ C0] do_initcall_level (init/main.c:1375) [ 165.811939][ C0] do_initcalls (init/main.c:1389) [ 165.811939][ C0] do_basic_setup (init/main.c:1412) [ 165.811939][ C0] kernel_init_freeable (init/main.c:1618) [ 165.811939][ C0] ? rest_init (init/main.c:1497) [ 165.811939][ C0] kernel_init (init/main.c:1507) [ 165.811939][ C0] ? rest_init (init/main.c:1497) [ 165.811939][ C0] ret_from_fork (??:?) [ 165.811939][ C0] </TASK> [ 175.608197][ T1] test_free_pages: Test completed [ 175.609667][ T1] glob: 64 self-tests passed, 0 failed [ 175.612731][ T1] gpio_winbond: chip ID at 2e is ffff [ 175.613875][ T1] gpio_winbond: not an our chip [ 175.614961][ T1] gpio_winbond: chip ID at 4e is ffff [ 175.616198][ T1] gpio_winbond: not an our chip [ 175.619125][ T1] IPMI message handler: version 39.2 [ 175.620532][ T1] ipmi device interface [ 175.621713][ T1] ipmi_si: IPMI System Interface driver [ 175.623807][ T1] ipmi_si: Unable to find any System Interface(s) [ 175.625144][ T1] ipmi_ssif: IPMI SSIF Interface driver [ 175.627821][ T1] ERST DBG: ERST support is disabled. [ 175.632004][ T101] clocksource_wdtest: --- Verify jiffies-like uncertainty margin. [ 175.633769][ T101] clocksource: wdtest-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 6370867519511994 ns [ 175.807028][ T1] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 175.809360][ T1] serial 00:05: GPIO lookup for consumer rs485-term [ 175.810908][ T1] serial 00:05: using ACPI for GPIO lookup [ 175.812252][ T1] acpi PNP0501:00: GPIO: looking up rs485-term-gpios [ 175.813692][ T1] acpi PNP0501:00: GPIO: looking up rs485-term-gpio [ 175.815081][ T1] serial 00:05: using lookup tables for GPIO lookup [ 175.816663][ T1] serial 00:05: No GPIO consumer rs485-term found [ 175.855621][ T1] 00:05: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 175.859039][ T1] serial 00:06: GPIO lookup for consumer rs485-term [ 175.860433][ T1] serial 00:06: using ACPI for GPIO lookup [ 175.861647][ T1] acpi PNP0501:01: GPIO: looking up rs485-term-gpios [ 175.863108][ T1] acpi PNP0501:01: GPIO: looking up rs485-term-gpio [ 175.864457][ T1] serial 00:06: using lookup tables for GPIO lookup [ 175.865837][ T1] serial 00:06: No GPIO consumer rs485-term found [ 175.900036][ T1] 00:06: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 175.907614][ T1] Non-volatile memory driver v1.3 [ 175.912094][ T1] SPI driver at25 has no spi_device_id for cypress,fm25 [ 175.918473][ T1] vcan: Virtual CAN interface driver [ 175.919701][ T1] CAN device driver interface [ 175.920953][ T1] e1000: Intel(R) PRO/1000 Network Driver [ 175.922231][ T1] e1000: Copyright (c) 1999-2006 Intel Corporation. To reproduce: # build kernel cd linux cp config-5.15.0-rc3-03890-g1ecd8ea291a8 .config make HOSTCC=clang-14 CC=clang-14 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months

1
0
0 / 0

54b675d9b2: WARNING:at_kernel/fork.c:#__mmdrop

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 54b675d9b28d9a56289d06a813250472bc621f40 ("[HACK] demonstrate lazy tlb issues") https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git sched/bad_lazymm in testcase: xfstests version: xfstests-x86_64-99bc497-1_20211101 with following parameters: disk: 2pmem fs: ext4 test: ext4-dax ucode: 0x7000019 test-description: xfstests is a regression test suite for xfs and other files ystems. test-url: git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git on test machine: 16 threads 1 sockets Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz with 48G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +------------------------------------+------------+------------+ | | 0304e8588d | 54b675d9b2 | +------------------------------------+------------+------------+ | boot_successes | 11 | 0 | | boot_failures | 0 | 13 | | WARNING:at_kernel/fork.c:#__mmdrop | 0 | 13 | | RIP:__mmdrop | 0 | 13 | +------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 11.139437][ T230] WARNING: CPU: 10 PID: 230 at kernel/fork.c:749 __mmdrop (kernel/fork.c:749 (discriminator 1)) [ 11.147364][ T230] Modules linked in: [ 11.151125][ T230] CPU: 10 PID: 230 Comm: modprobe Not tainted 5.15.0-00004-g54b675d9b28d #1 [ 11.159670][ T230] Hardware name: Supermicro SYS-5018D-FN4T/X10SDV-8C-TLN4F, BIOS 1.1 03/02/2016 [ 11.168529][ T230] RIP: 0010:__mmdrop (kernel/fork.c:749 (discriminator 1)) [ 11.172685][ T209] usb 3-4.1: new low-speed USB device number 4 using xhci_hcd [ 11.173251][ T230] Code: 48 89 ee 5d 41 5c e9 ad 1c 28 00 be 03 00 00 00 4c 89 c7 e8 60 62 50 00 eb de e8 d9 af 0f 00 eb d7 0f 0b 0f 0b e9 3d ff ff ff <0f> 0b e9 4c ff ff ff 48 89 ef e8 7f 41 27 00 e9 61 ff ff ff 66 66 All code ======== 0: 48 89 ee mov %rbp,%rsi 3: 5d pop %rbp 4: 41 5c pop %r12 6: e9 ad 1c 28 00 jmpq 0x281cb8 b: be 03 00 00 00 mov $0x3,%esi 10: 4c 89 c7 mov %r8,%rdi 13: e8 60 62 50 00 callq 0x506278 18: eb de jmp 0xfffffffffffffff8 1a: e8 d9 af 0f 00 callq 0xfaff8 1f: eb d7 jmp 0xfffffffffffffff8 21: 0f 0b ud2 23: 0f 0b ud2 25: e9 3d ff ff ff jmpq 0xffffffffffffff67 2a:* 0f 0b ud2 <-- trapping instruction 2c: e9 4c ff ff ff jmpq 0xffffffffffffff7d 31: 48 89 ef mov %rbp,%rdi 34: e8 7f 41 27 00 callq 0x2741b8 39: e9 61 ff ff ff jmpq 0xffffffffffffff9f 3e: 66 data16 3f: 66 data16 Code starting with the faulting instruction =========================================== 0: 0f 0b ud2 2: e9 4c ff ff ff jmpq 0xffffffffffffff53 7: 48 89 ef mov %rbp,%rdi a: e8 7f 41 27 00 callq 0x27418e f: e9 61 ff ff ff jmpq 0xffffffffffffff75 14: 66 data16 15: 66 data16 [ 11.199988][ T230] RSP: 0018:ffffc900005bbe88 EFLAGS: 00010246 [ 11.205915][ T230] RAX: ffff8881011b4f80 RBX: ffff8881011b4f80 RCX: ffff8881584c5300 [ 11.213733][ T230] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff888103960cc0 [ 11.221550][ T230] RBP: ffff888103960cc0 R08: ffff8881584c4d00 R09: ffffffff81300300 [ 11.229384][ T230] R10: ffff888c7cf09b58 R11: 0000000000000001 R12: ffff8881011b4f80 [ 11.237220][ T230] R13: 0000000000000001 R14: 0000000000000000 R15: ffff8881011b5bf0 [ 11.245055][ T230] FS: 0000000000000000(0000) GS:ffff888c3d080000(0000) knlGS:0000000000000000 [ 11.253842][ T230] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 11.260271][ T230] CR2: 00007f4c016bf114 CR3: 0000000c7ec10001 CR4: 00000000003706e0 [ 11.268115][ T230] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 11.275965][ T230] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 11.283802][ T230] Call Trace: [ 11.283994][ T209] usb 3-4.1: New USB device found, idVendor=0557, idProduct=2419, bcdDevice= 1.00 [ 11.286944][ T230] do_exit (arch/x86/include/asm/bitops.h:207 include/asm-generic/bitops/instrumented-non-atomic.h:135 include/linux/thread_info.h:118 kernel/exit.c:502 kernel/exit.c:812) [ 11.295979][ T209] usb 3-4.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 11.299897][ T230] do_group_exit (include/linux/list.h:282 include/linux/sched/signal.h:686 kernel/exit.c:907) [ 11.312160][ T230] __x64_sys_exit_group (kernel/exit.c:933) [ 11.313067][ T209] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-4/3-4.1/3-4.1:1.0/0003:0557:2419.0002/input/input4 [ 11.317038][ T230] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) [ 11.317044][ T230] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:113) [ 11.339130][ T230] RIP: 0033:0x7f4c012c19d6 [ 11.343393][ T230] Code: Unable to access opcode bytes at RIP 0x7f4c012c19ac. Code starting with the faulting instruction =========================================== [ 11.350621][ T230] RSP: 002b:00007ffe398efea8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7 [ 11.358889][ T230] RAX: ffffffffffffffda RBX: 00007f4c013b2760 RCX: 00007f4c012c19d6 [ 11.366706][ T230] RDX: 0000000000000001 RSI: 000000000000003c RDI: 0000000000000001 [ 11.374524][ T230] RBP: 0000000000000001 R08: 00000000000000e7 R09: ffffffffffffff80 [ 11.380812][ T209] hid-generic 0003:0557:2419.0002: input,hidraw1: USB HID v1.00 Keyboard [HID 0557:2419] on usb-0000:00:14.0-4.1/input0 [ 11.382358][ T230] R10: 00007ffe398efd5c R11: 0000000000000246 R12: 00007f4c013b2760 [ 11.382360][ T230] R13: 0000000000000001 R14: 00007f4c013bb428 R15: 0000000000000000 [ 11.395475][ T209] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb3/3-4/3-4.1/3-4.1:1.1/0003:0557:2419.0003/input/input5 [ 11.402518][ T230] ---[ end trace 8a0cdd37e7ac904b ]--- [ 11.404617][ T1] Loaded X.509 cert 'Build time autogenerated kernel key: 825f8f632f2835177197bb4b0ca2da2b106df827' [ 11.410430][ T209] hid-generic 0003:0557:2419.0003: input,hidraw2: USB HID v1.00 Mouse [HID 0557:2419] on usb-0000:00:14.0-4.1/input1 [ 11.422482][ T1] zswap: loaded using pool lzo/zbud [ 11.455839][ T1] Key type ._fscrypt registered [ 11.460543][ T1] Key type .fscrypt registered [ 11.465177][ T1] Key type fscrypt-provisioning registered [ 11.470889][ T1] pstore: Using crash dump compression: deflate [ 11.479754][ T1] Key type encrypted registered [ 11.658636][ T1] pps pps0: new PPS source ptp2 [ 11.663412][ T1] ixgbe 0000:03:00.0: registered PHC device on eth2 [ 11.876201][ T1] pps pps1: new PPS source ptp3 [ 11.880977][ T1] ixgbe 0000:03:00.1: registered PHC device on eth3 [ 12.845928][ C4] random: fast init done [ 15.035123][ T195] igb 0000:05:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX [ 15.045789][ T195] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [ 15.057691][ T1] Sending DHCP requests ...., OK [ 26.900567][ T1] IP-Config: Got DHCP answer from 192.168.3.2, my address is 192.168.3.82 [ 26.908908][ T1] IP-Config: Complete: [ 26.912824][ T1] device=eth0, hwaddr=0c:c4:7a:c4:ab:7a, ipaddr=192.168.3.82, mask=255.255.255.0, gw=192.168.3.200 [ 26.923770][ T1] host=lkp-bdw-de1, domain=lkp.intel.com, nis-domain=(none) [ 26.931351][ T1] bootserver=192.168.3.200, rootserver=192.168.3.200, rootpath= [ 26.931353][ T1] nameserver0=192.168.3.200 [ 26.989125][ T1] ixgbe 0000:03:00.0: removed PHC on eth2 [ 28.116657][ T1] ixgbe 0000:03:00.1: removed PHC on eth3 [ 29.605079][ T1] Freeing unused kernel image (initmem) memory: 2548K [ 29.613687][ T1] Write protecting the kernel read-only data: 24576k [ 29.620764][ T1] Freeing unused kernel image (text/rodata gap) memory: 2036K [ 29.628382][ T1] Freeing unused kernel image (rodata/data gap) memory: 1340K [ 29.644737][ T1] Run /init as init process [ 29.649096][ T1] with arguments: [ 29.652777][ T1] /init [ 29.655741][ T1] nokaslr [ 29.658877][ T1] with environment: [ 29.662709][ T1] HOME=/ [ 29.665758][ T1] TERM=linux [ 29.669155][ T1] user=lkp [ 29.672381][ T1] job=/lkp/jobs/scheduled/lkp-bdw-de1/xfstests-2pmem-ext4-ext4-dax-ucode=0x7000019-debian-10.4-x86_64-20200603.cgz-54b675d9b28d9a56289d06a813250472bc621f40-20211107-33854-12dhxvn-6.yaml [ 29.690798][ T1] ARCH=x86_64 [ 29.694280][ T1] kconfig=x86_64-rhel-8.3-func [ 29.699238][ T1] branch=luto/sched/bad_lazymm [ 29.704198][ T1] commit=54b675d9b28d9a56289d06a813250472bc621f40 [ 29.710809][ T1] BOOT_IMAGE=/pkg/linux/x86_64-rhel-8.3-func/gcc-9/54b675d9b28d9a56289d06a813250472bc621f40/vmlinuz-5.15.0-00004-g54b675d9b28d [ 29.724102][ T1] max_uptime=2100 [ 29.727950][ T1] RESULT_ROOT=/result/xfstests/2pmem-ext4-ext4-dax-ucode=0x7000019/lkp-bdw-de1/debian-10.4-x86_64-20200603.cgz/x86_64-rhel-8.3-func/gcc-9/54b675d9b28d9a56289d06a813250472bc621f40/6 [ 29.745925][ T1] LKP_SERVER=internal-lkp-server [ 29.751073][ T1] selinux=0 [ 29.754400][ T1] softlockup_panic=1 [ 29.758510][ T1] prompt_ramdisk=0 [ 29.767904][ T1] systemd[1]: RTC configured in localtime, applying delta of 0 minutes to system time. [ 29.815353][ T252] random: lvmconfig: uninitialized urandom read (4 bytes read) [ 29.870206][ T280] random: systemd-random-: uninitialized urandom read (512 bytes read) [ 29.969670][ T321] IPMI message handler: version 39.2 [ 29.976632][ T321] ipmi device interface [ 29.980769][ T329] dca service started, version 1.12.1 [ 29.987470][ T321] ipmi_si: IPMI System Interface driver [ 29.993083][ T321] ipmi_si dmi-ipmi-si.0: ipmi_platform: probing via SMBIOS [ 30.000160][ T321] ipmi_platform: ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0 [ 30.011663][ T321] ipmi_si: Adding SMBIOS-specified kcs state machine [ 30.018552][ T321] ipmi_si IPI0001:00: ipmi_platform: probing via ACPI [ 30.025355][ T321] ipmi_si IPI0001:00: ipmi_platform: [io 0x0ca2] regsize 1 spacing 1 irq 0 [ 30.042600][ T327] gpio_ich gpio_ich.1.auto: GPIO from 948 to 1023 [ 30.049182][ T326] ioatdma: Intel(R) QuickData Technology Driver 5.00 [ 30.056063][ T321] ipmi_si dmi-ipmi-si.0: Removing SMBIOS-specified kcs state machine in favor of ACPI [ 30.065467][ T321] ipmi_si: Adding ACPI-specified kcs state machine [ 30.065565][ T321] ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0 [ 30.077176][ T326] IOAPIC[9]: Set IRTE entry (P:1 FPD:0 Dst_Mode:0 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000000 SID:002C SQ:0 SVT:1) [ 30.096532][ T326] IOAPIC[1]: Preconfigured routing entry (9-13 -> IRQ 88 Level:1 ActiveLow:1) [ 30.105769][ T358] libata version 3.00 loaded. [ 30.106791][ T326] IOAPIC[9]: Set IRTE entry (P:1 FPD:0 Dst_Mode:0 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000000 SID:002C SQ:0 SVT:1) [ 30.121899][ T435] random: ln: uninitialized urandom read (6 bytes read) [ 30.124481][ T326] IOAPIC[1]: Preconfigured routing entry (9-14 -> IRQ 90 Level:1 ActiveLow:1) LKP: HOSTNAME lkp-bdw-de1, MAC 0c:c4:7a:c4:ab:7a, kernel 5.15.0-00004-g54b675d9b28d 1, serial console /dev/ttyS0 [ 30.161780][ T358] ahci 0000:00:1f.2: version 3.0 [ 30.166594][ T321] ipmi_si IPI0001:00: The BMC does not support clearing the recv irq bit, compensating, but the BMC needs to be fixed. [ 30.167637][ T358] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode [ 30.167649][ T358] ahci 0000:00:1f.2: flags: 64bit ncq pm led clo pio slum part ems apst [ 30.170722][ T94] mei_me 0000:00:16.0: Device doesn't have valid ME Interface [ 30.184055][ T358] scsi host0: ahci [ 30.191544][ T360] nd_pmem namespace0.0: unable to guarantee persistence of writes To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months

1
0
0 / 0

[perf tools] eb0ce82731: perf-sanity-tests.Parsing_of_PMU_event_table_metrics.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: eb0ce827314d6f0e0d07a04f000ade0ababd960d ("perf tools: Move out parse_events_add_pmu fallback from parser code") https://git.kernel.org/cgit/linux/kernel/git/jolsa/perf.git libperf/parse_6 in testcase: perf-sanity-tests version: perf-x86_64-cc0356d6a02e-1_20211102 with following parameters: perf_compiler: gcc ucode: 0xde on test machine: 8 threads 1 sockets Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz with 32G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> 2021-11-04 12:20:38 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-func-eb0ce827314d6f0e0d07a04f000ade0ababd960d/tools/perf/perf test 10 10: PMU events : 10.1: PMU event table sanity : Ok 10.2: PMU event map aliases : Ok 10.3: Parsing of PMU event table metrics : FAILED! 10.4: Parsing of PMU event table metrics with fake PMUs : Ok To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months, 1 week

1
0
0 / 0

[drm] a31246115b: BUG:kernel_NULL_pointer_dereference,address

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: a31246115b33b3c3ab456e3f689174a076f09bbf ("[RFC PATCH 5/8] drm: start using drm_gem_trace_gpu_mem_instance") url: https://github.com/0day-ci/linux/commits/Gurchetan-Singh/GPU-memory-trace... base: git://anongit.freedesktop.org/drm-intel for-linux-next patch link: https://lore.kernel.org/dri-devel/[email protected] in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -cpu Icelake-Server -smp 4 -m 16G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): +---------------------------------------------+------------+------------+ | | 4f27e9667d | a31246115b | +---------------------------------------------+------------+------------+ | boot_successes | 16 | 0 | | boot_failures | 0 | 16 | | BUG:kernel_NULL_pointer_dereference,address | 0 | 16 | | Oops:#[##] | 0 | 16 | | RIP:drm_gem_trace_gpu_mem_instance | 0 | 16 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 16 | +---------------------------------------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> [ 23.584758][ T1] BUG: kernel NULL pointer dereference, address: 0000000000000020 [ 23.586495][ T1] #PF: supervisor read access in kernel mode [ 23.587820][ T1] #PF: error_code(0x0000) - not-present page [ 23.589173][ T1] PGD 0 P4D 0 [ 23.589661][ T1] Oops: 0000 [#1] SMP [ 23.589661][ T1] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.15.0-rc2-01062-ga31246115b33 #1 [ 23.589661][ T1] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 23.589661][ T1] RIP: 0010:drm_gem_trace_gpu_mem_instance (drivers/gpu/drm/drm_gem.c:184) [ 23.589661][ T1] Code: 48 83 05 2c 56 e0 05 01 e8 c7 c1 0b ff 48 8b 83 d0 01 00 00 4c 8b ab 78 04 00 00 4c 8b a3 70 04 00 00 48 83 05 7a 54 e0 05 01 <48> 8b 40 20 48 8b 58 40 48 8b 85 80 00 00 00 8b 28 66 90 e8 92 c1 All code ======== 0: 48 83 05 2c 56 e0 05 addq $0x1,0x5e0562c(%rip) # 0x5e05634 7: 01 8: e8 c7 c1 0b ff callq 0xffffffffff0bc1d4 d: 48 8b 83 d0 01 00 00 mov 0x1d0(%rbx),%rax 14: 4c 8b ab 78 04 00 00 mov 0x478(%rbx),%r13 1b: 4c 8b a3 70 04 00 00 mov 0x470(%rbx),%r12 22: 48 83 05 7a 54 e0 05 addq $0x1,0x5e0547a(%rip) # 0x5e054a4 29: 01 2a:* 48 8b 40 20 mov 0x20(%rax),%rax <-- trapping instruction 2e: 48 8b 58 40 mov 0x40(%rax),%rbx 32: 48 8b 85 80 00 00 00 mov 0x80(%rbp),%rax 39: 8b 28 mov (%rax),%ebp 3b: 66 90 xchg %ax,%ax 3d: e8 .byte 0xe8 3e: 92 xchg %eax,%edx 3f: c1 .byte 0xc1 Code starting with the faulting instruction =========================================== 0: 48 8b 40 20 mov 0x20(%rax),%rax 4: 48 8b 58 40 mov 0x40(%rax),%rbx 8: 48 8b 85 80 00 00 00 mov 0x80(%rbp),%rax f: 8b 28 mov (%rax),%ebp 11: 66 90 xchg %ax,%ax 13: e8 .byte 0xe8 14: 92 xchg %eax,%edx 15: c1 .byte 0xc1 [ 23.589661][ T1] RSP: 0000:ffffc900000138f8 EFLAGS: 00010202 [ 23.589661][ T1] RAX: 0000000000000000 RBX: ffff888114cd4800 RCX: 0000000000000000 [ 23.589661][ T1] RDX: ffff8881002d8000 RSI: ffffffff8221ce49 RDI: ffff88810f9c6000 [ 23.589661][ T1] RBP: ffff88810f9c6000 R08: 0000000000000000 R09: 0000000000000001 [ 23.589661][ T1] R10: 00000000e4a45f4b R11: 000000000000007f R12: 0000000000300000 [ 23.589661][ T1] R13: 0000000000000000 R14: ffff888114cd48e0 R15: ffff88810f985418 [ 23.589661][ T1] FS: 0000000000000000(0000) GS:ffff88842fa00000(0000) knlGS:0000000000000000 [ 23.589661][ T1] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 23.589661][ T1] CR2: 0000000000000020 CR3: 0000000004e6a000 CR4: 0000000000000ea0 [ 23.589661][ T1] Call Trace: [ 23.589661][ T1] drm_gem_handle_create_tail (drivers/gpu/drm/drm_gem.c:452) [ 23.589661][ T1] drm_gem_handle_create (drivers/gpu/drm/drm_gem.c:486) [ 23.589661][ T1] drm_gem_vram_fill_create_dumb (drivers/gpu/drm/drm_gem_vram_helper.c:527) [ 23.589661][ T1] drm_gem_vram_driver_dumb_create (drivers/gpu/drm/drm_gem_vram_helper.c:624) [ 23.589661][ T1] drm_mode_create_dumb (drivers/gpu/drm/drm_dumb_buffers.c:96) [ 23.589661][ T1] drm_client_framebuffer_create (drivers/gpu/drm/drm_client.c:268 drivers/gpu/drm/drm_client.c:418) [ 23.589661][ T1] drm_fb_helper_generic_probe (drivers/gpu/drm/drm_fb_helper.c:2321 (discriminator 4)) [ 23.589661][ T1] drm_fb_helper_single_fb_probe (drivers/gpu/drm/drm_fb_helper.c:1668) [ 23.589661][ T1] __drm_fb_helper_initial_config_and_unlock (drivers/gpu/drm/drm_fb_helper.c:1827) [ 23.589661][ T1] drm_fb_helper_initial_config (drivers/gpu/drm/drm_fb_helper.c:1921) [ 23.589661][ T1] drm_fbdev_client_hotplug (drivers/gpu/drm/drm_fb_helper.c:2423) [ 23.589661][ T1] drm_fbdev_generic_setup (drivers/gpu/drm/drm_fb_helper.c:2510) [ 23.589661][ T1] bochs_pci_probe (drivers/gpu/drm/tiny/bochs.c:667) [ 23.589661][ T1] local_pci_probe (drivers/pci/pci-driver.c:323) [ 23.589661][ T1] pci_device_probe (drivers/pci/pci-driver.c:380 drivers/pci/pci-driver.c:405 drivers/pci/pci-driver.c:448) [ 23.589661][ T1] ? pci_device_remove (drivers/pci/pci-driver.c:433) [ 23.589661][ T1] really_probe (drivers/base/dd.c:515 drivers/base/dd.c:596) [ 23.589661][ T1] __driver_probe_device (drivers/base/dd.c:751) [ 23.589661][ T1] driver_probe_device (drivers/base/dd.c:781) [ 23.589661][ T1] __driver_attach (drivers/base/dd.c:1141) [ 23.589661][ T1] ? driver_allows_async_probing (drivers/base/dd.c:1093) [ 23.589661][ T1] bus_for_each_dev (drivers/base/bus.c:301) [ 23.589661][ T1] driver_attach (drivers/base/dd.c:1157) [ 23.589661][ T1] bus_add_driver (drivers/base/bus.c:618) [ 23.589661][ T1] driver_register (drivers/base/driver.c:171) [ 23.589661][ T1] __pci_register_driver (drivers/pci/pci-driver.c:1407) [ 23.589661][ T1] ? ch7006_init (drivers/gpu/drm/tiny/bochs.c:721) [ 23.589661][ T1] bochs_init (drivers/gpu/drm/tiny/bochs.c:728) [ 23.589661][ T1] do_one_initcall (init/main.c:1303) [ 23.589661][ T1] ? rcu_read_lock_sched_held (include/linux/lockdep.h:283 kernel/rcu/update.c:125) [ 23.589661][ T1] do_initcalls (init/main.c:1376 init/main.c:1392) [ 23.589661][ T1] kernel_init_freeable (init/main.c:1411 init/main.c:1614) [ 23.589661][ T1] ? rest_init (init/main.c:1497) [ 23.589661][ T1] kernel_init (init/main.c:1505) [ 23.589661][ T1] ret_from_fork (arch/x86/entry/entry_64.S:301) [ 23.589661][ T1] Modules linked in: [ 23.589661][ T1] CR2: 0000000000000020 [ 23.589661][ T1] ---[ end trace 2603038b65df9faf ]--- [ 23.589661][ T1] RIP: 0010:drm_gem_trace_gpu_mem_instance (drivers/gpu/drm/drm_gem.c:184) [ 23.589661][ T1] Code: 48 83 05 2c 56 e0 05 01 e8 c7 c1 0b ff 48 8b 83 d0 01 00 00 4c 8b ab 78 04 00 00 4c 8b a3 70 04 00 00 48 83 05 7a 54 e0 05 01 <48> 8b 40 20 48 8b 58 40 48 8b 85 80 00 00 00 8b 28 66 90 e8 92 c1 All code ======== 0: 48 83 05 2c 56 e0 05 addq $0x1,0x5e0562c(%rip) # 0x5e05634 7: 01 8: e8 c7 c1 0b ff callq 0xffffffffff0bc1d4 d: 48 8b 83 d0 01 00 00 mov 0x1d0(%rbx),%rax 14: 4c 8b ab 78 04 00 00 mov 0x478(%rbx),%r13 1b: 4c 8b a3 70 04 00 00 mov 0x470(%rbx),%r12 22: 48 83 05 7a 54 e0 05 addq $0x1,0x5e0547a(%rip) # 0x5e054a4 29: 01 2a:* 48 8b 40 20 mov 0x20(%rax),%rax <-- trapping instruction 2e: 48 8b 58 40 mov 0x40(%rax),%rbx 32: 48 8b 85 80 00 00 00 mov 0x80(%rbp),%rax 39: 8b 28 mov (%rax),%ebp 3b: 66 90 xchg %ax,%ax 3d: e8 .byte 0xe8 3e: 92 xchg %eax,%edx 3f: c1 .byte 0xc1 Code starting with the faulting instruction =========================================== 0: 48 8b 40 20 mov 0x20(%rax),%rax 4: 48 8b 58 40 mov 0x40(%rax),%rbx 8: 48 8b 85 80 00 00 00 mov 0x80(%rbp),%rax f: 8b 28 mov (%rax),%ebp 11: 66 90 xchg %ax,%ax 13: e8 .byte 0xe8 14: 92 xchg %eax,%edx 15: c1 .byte 0xc1 To reproduce: # build kernel cd linux cp config-5.15.0-rc2-01062-ga31246115b33 .config make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months, 1 week

1
0
0 / 0

[perf metrics] 5ecd5a0c7d: perf-sanity-tests.Parse_and_process_metrics.fail

by kernel test robot

Greeting, FYI, we noticed the following commit (built with gcc-9): commit: 5ecd5a0c7d1cca79f1431093d12e4cd9893b0331 ("perf metrics: Modify setup and deduplication") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: perf-sanity-tests version: perf-x86_64-cc0356d6a02e-1_20211102 with following parameters: perf_compiler: gcc ucode: 0xde on test machine: 8 threads 1 sockets Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz with 32G memory caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> 2021-11-04 13:27:18 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-func-5ecd5a0c7d1cca79f1431093d12e4cd9893b0331/tools/perf/perf test 68 68: Parse and process metrics : FAILED! >>> another failed test but can pass on parent 2021-11-04 13:27:18 sudo /usr/src/perf_selftests-x86_64-rhel-8.3-func-5ecd5a0c7d1cca79f1431093d12e4cd9893b0331/tools/perf/perf test 70 70: Event expansion for cgroups : FAILED! To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests sudo bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run sudo bin/lkp run generated-yaml-file # if come across any failure that blocks the test, # please remove ~/.lkp and /lkp dir to run from a clean state. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/[email protected] Intel Corporation Thanks, Oliver Sang

9 months, 1 week

2
1
0 / 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

LKP November 2021