[sched/hotplug] c59ee15948: kernel_BUG_at_kernel/sched/core.c
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: c59ee15948b8de2828d9175a4b58e2b8ce1c9131 ("sched/hotplug: Consolidate task migration on CPU unplug")
https://git.kernel.org/cgit/linux/kernel/git/tglx/devel.git highmem
in testcase: locktorture
version:
with following parameters:
runtime: 300s
test: cpuhotplug
test-description: This torture test consists of creating a number of kernel threads which acquire the lock and hold it for specific amount of time, thus simulating different critical region behaviors.
test-url: https://www.kernel.org/doc/Documentation/locking/locktorture.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+------------------------------------------+------------+------------+
| | 98f6f9834e | c59ee15948 |
+------------------------------------------+------------+------------+
| boot_successes | 6 | 2 |
| boot_failures | 0 | 5 |
| kernel_BUG_at_kernel/sched/core.c | 0 | 5 |
| invalid_opcode:#[##] | 0 | 5 |
| RIP:sched_cpu_dying | 0 | 5 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 5 |
+------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 310.168118] kernel BUG at kernel/sched/core.c:7081!
[ 310.168945] invalid opcode: 0000 [#1] SMP PTI
[ 310.169679] CPU: 0 PID: 11 Comm: migration/0 Not tainted 5.9.0-rc6-00005-gc59ee15948b8d #1
[ 310.170965] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 310.172371] Stopper: multi_cpu_stop+0x0/0x120 <- 0x0
[ 310.173247] RIP: 0010:sched_cpu_dying+0x153/0x160
[ 310.173920] Code: 5d 5d c3 80 3d 16 8f 75 01 00 0f 85 0c ff ff ff 48 c7 c7 b3 07 35 82 c6 05 02 8f 75 01 01 e8 56 20 fc ff 0f 0b e9 f2 fe ff ff <0f> 0b 0f 0b 0f 0b eb 86 0f 1f 44 00 00 66 66 66 66 90 55 48 89 e5
[ 310.189289] RSP: 0018:ffffc90000063db0 EFLAGS: 00010002
[ 310.190174] RAX: ffff88822a504860 RBX: ffff88823fc2c100 RCX: 0000000000000000
[ 310.191351] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88823fc2c180
[ 310.192513] RBP: ffffc90000063dc8 R08: 0000000000000000 R09: ffff88822a504bb0
[ 310.193731] R10: 0000000000000006 R11: 0000000000000000 R12: 0000000000000000
[ 310.194919] R13: 0000000000000086 R14: ffffffff810db6e0 R15: 0000000000000003
[ 310.196109] FS: 0000000000000000(0000) GS:ffff88823fc00000(0000) knlGS:0000000000000000
[ 310.197477] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 310.198422] CR2: 00000000080a2190 CR3: 000000000260a000 CR4: 00000000000006f0
[ 310.199607] Call Trace:
[ 310.200089] cpuhp_invoke_callback+0x8d/0x5c0
[ 310.212974] ? native_apic_msr_write+0x27/0x40
[ 310.213855] take_cpu_down+0x60/0xa0
[ 310.214483] multi_cpu_stop+0x6d/0x120
[ 310.215089] ? stop_machine_yield+0x20/0x20
[ 310.215819] cpu_stopper_thread+0x59/0x140
[ 310.216558] ? smpboot_thread_fn+0x26/0x1e0
[ 310.217312] smpboot_thread_fn+0x10b/0x1e0
[ 310.218028] ? sort_range+0x20/0x20
[ 310.218629] kthread+0x116/0x160
[ 310.219202] ? kthread_park+0xa0/0xa0
[ 310.219843] ret_from_fork+0x22/0x30
[ 310.220473] Modules linked in: locktorture(-) torture sr_mod cdrom ata_generic pata_acpi ppdev parport_pc parport floppy ata_piix libata i2c_piix4
[ 310.222706] ---[ end trace ae2e7e9fc379c741 ]---
To reproduce:
# build kernel
cd linux
cp config-5.9.0-rc6-00005-gc59ee15948b8d .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage modules
make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 INSTALL_MOD_PATH=<mod-install-dir> modules_install
cd <mod-install-dir>
find lib/ | cpio -o -H newc --quiet | gzip > modules.cgz
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> -m modules.cgz job-script # job-script is attached in this email
Thanks,
lkp
1 year, 10 months
[mm/lru] 28e332d5c5: kernel_BUG_at_mm/swap.c
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 28e332d5c5943b86b08af15abd745708b2af44c9 ("mm/lru: replace pgdat lru_lock with lruvec lock")
https://github.com/alexshi/linux.git lruv19.6
in testcase: boot
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+------------------------------------------+------------+------------+
| | ec6f8eee56 | 28e332d5c5 |
+------------------------------------------+------------+------------+
| boot_successes | 8 | 0 |
| boot_failures | 0 | 9 |
| kernel_BUG_at_mm/swap.c | 0 | 9 |
| invalid_opcode:#[##] | 0 | 9 |
| RIP:__page_cache_release | 0 | 9 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 9 |
+------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 4.101691] kernel BUG at mm/swap.c:87!
[ 4.102693] invalid opcode: 0000 [#1] SMP PTI
[ 4.103509] CPU: 1 PID: 442 Comm: khugepaged Not tainted 5.9.0-rc6-00018-g28e332d5c5943 #1
[ 4.104825] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 4.106188] RIP: 0010:__page_cache_release+0x6f/0x155
[ 4.107055] Code: e6 48 89 ef e8 2a 3a 06 00 48 89 ef 49 89 c5 e8 f5 f4 ff ff 48 8b 00 a8 10 75 11 48 c7 c6 9b b7 2b 82 48 89 ef e8 fe ee 01 00 <0f> 0b 48 89 ef e8 d5 f4 ff ff 48 89 ef 48 8b 00 0f ba e0 14 73 12
[ 4.116822] RSP: 0018:ffffc90000bd7c40 EFLAGS: 00010082
[ 4.117876] RAX: 0000000000000000 RBX: ffff8882235c4020 RCX: 0000000000000000
[ 4.118989] RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffffffff82c8370c
[ 4.120139] RBP: ffffea0008625180 R08: 0000000000000000 R09: ffffffff82c8574f
[ 4.121244] R10: 0000000000000000 R11: ffffc90000bd79ad R12: 0000000000800000
[ 4.122316] R13: ffff88822a585000 R14: ffffea0008645b28 R15: ffff88821916c060
[ 4.123488] FS: 0000000000000000(0000) GS:ffff88823fd00000(0000) knlGS:0000000000000000
[ 4.124867] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4.125788] CR2: 0000564112a7e8b8 CR3: 000000021915a000 CR4: 00000000000406e0
[ 4.126883] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 4.128004] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 4.129305] Call Trace:
[ 4.129859] __put_page+0x6f/0x82
[ 4.130493] collapse_huge_page+0x4a8/0x6ea
[ 4.131203] khugepaged_scan_pmd+0x332/0x3d8
[ 4.131979] khugepaged_scan_mm_slot+0x4dc/0x592
[ 4.132760] khugepaged+0x18e/0x394
[ 4.133396] ? prepare_to_wait_exclusive+0x89/0x89
[ 4.134224] ? khugepaged_scan_mm_slot+0x592/0x592
[ 4.135032] kthread+0x107/0x10f
[ 4.135631] ? kthread_flush_worker+0x75/0x75
[ 4.136364] ret_from_fork+0x22/0x30
[ 4.137047] Modules linked in:
[ 4.137621] ---[ end trace 5757c96c397e7d37 ]---
To reproduce:
# build kernel
cd linux
cp config-5.9.0-rc6-00018-g28e332d5c5943 .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
1 year, 10 months
[sched] 25d95d0623: BUG:assuming_atomic_context_at_include/linux/filter.h
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 25d95d0623751d029c3177527d44563040ae828b ("sched: Make migrate_disable/enable() independent of RT")
https://git.kernel.org/cgit/linux/kernel/git/tglx/devel.git highmem
in testcase: boot
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-------------------------------------------------------+------------+------------+
| | c321556668 | 25d95d0623 |
+-------------------------------------------------------+------------+------------+
| boot_successes | 4 | 0 |
| boot_failures | 0 | 4 |
| BUG:assuming_atomic_context_at_include/linux/filter.h | 0 | 4 |
+-------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 9.819373] BUG: assuming atomic context at include/linux/filter.h:597
[ 9.820843] in_atomic(): 0, irqs_disabled(): 0, pid: 156, name: (journald)
[ 9.822369] CPU: 1 PID: 156 Comm: (journald) Not tainted 5.9.0-rc6-00010-g25d95d0623751 #1
[ 9.824286] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 9.826275] Call Trace:
[ 9.826998] dump_stack+0x57/0x80
[ 9.827909] __cant_sleep.cold+0x68/0x75
[ 9.828932] __seccomp_filter+0x7e/0x680
[ 9.829876] syscall_trace_enter+0x83/0x1e0
[ 9.830892] do_syscall_64+0xf/0x40
[ 9.831719] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 9.832868] RIP: 0033:0x7fc1402c6f59
[ 9.833730] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 07 6f 0c 00 f7 d8 64 89 01 48
[ 9.837478] RSP: 002b:00007ffd5ecb9488 EFLAGS: 00000246 ORIG_RAX: 000000000000013d
[ 9.839240] RAX: ffffffffffffffda RBX: 0000561bf42ee750 RCX: 00007fc1402c6f59
[ 9.840827] RDX: 0000561bf42ee750 RSI: 0000000000000000 RDI: 0000000000000001
[ 9.842247] RBP: 0000561bf434d700 R08: 0000561bf42f5970 R09: 00000000c000003e
[ 9.843799] R10: 0000561bf42f59d0 R11: 0000000000000246 R12: 00007ffd5ecb94e0
[ 9.845242] R13: 00007ffd5ecb94d8 R14: 00007fc1400f7818 R15: 000000000000002c
[ OK ] Started Load Kernel Modules.
Mounting Kernel Configuration File System...
Starting Apply Kernel Variables...
[ OK ] Started Apply Kernel Variables.
[ OK ] Mounted Kernel Configuration File System.
[ OK ] Started Remount Root and Kernel File Systems.
Starting Create System Users...
Starting Load/Save Random Seed...
[ OK ] Started Create System Users.
Starting Create Static Device Nodes in /dev...
[ OK ] Started Load/Save Random Seed.
[ OK ] Started Create Static Device Nodes in /dev.
Starting udev Kernel Device Manager...
[ OK ] Reached target Local File Systems (Pre).
[ OK ] Reached target Local File Systems.
Starting Preprocess NFS configuration...
[ OK ] Started udev Kernel Device Manager.
[ OK ] Started Preprocess NFS configuration.
[ OK ] Reached target NFS client services.
[ OK ] Started Journal Service.
Starting Flush Journal to Persistent Storage...
[ OK ] Started udev Coldplug all Devices.
Starting Helper to synchronize boot up for ifupdown...
[ OK ] Started Flush Journal to Persistent Storage.
[ OK ] Started Helper to synchronize boot up for ifupdown.
Starting Raise network interfaces...
Starting Create Volatile Files and Directories...
[ OK ] Started Raise network interfaces.
[ OK ] Reached target Network.
[ OK ] Started Create Volatile Files and Directories.
Starting Update UTMP about System Boot/Shutdown...
Starting Network Time Synchronization...
Starting RPC bind portmap service...
[ OK ] Started RPC bind portmap service.
[ OK ] Started Update UTMP about System Boot/Shutdown.
[ OK ] Reached target Remote File Systems (Pre).
[ OK ] Reached target Remote File Systems.
[ OK ] Reached target RPC Port Mapper.
[ 10.337714] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
[ 10.450838] libata version 3.00 loaded.
[ OK ] Started Network Time Synchronization.
[ OK ] Reached target System Initialization.
[ OK ] Listening on D-Bus System Message Bus Socket.
[ OK ] Reached target Sockets.
[ OK ] Reached target Basic System.
Starting LKP bootstrap...
Starting Permit User Sessions...
[ OK ] Started D-Bus System Message Bus.
Starting System Logging Service...
Starting Login Service...
[ 10.531472] parport_pc 00:04: reported by Plug and Play ACPI
Starting /etc/rc.local Compatibility...
[ 10.537909] parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
Starting OpenBSD Secure Shell server...
[ OK ] Started Regular background program processing daemon.
Starting LSB: OpenIPMI Driver init script...
[ OK ] Started Daily Cleanup of Temporary Directories.
[ OK ] Reached target System Time Synchronized.
[ OK ] Started Daily rotation of log files.
[ OK ] Started Daily apt download activities.
[ OK ] Started Daily apt upgrade and clean activities.
[ OK ] Reached target Timers.
[ OK ] Started System Logging Service.
[ OK ] Started LKP bootstrap.
[ OK ] Started Permit User Sessions.
[ 10.757740] ata_piix 0000:00:01.1: version 2.13
[ 10.820897] BUG: assuming atomic context at include/linux/filter.h:597
[ 10.824958] in_atomic(): 0, irqs_disabled(): 0, pid: 189, name: systemd-udevd
[ 10.826545] CPU: 0 PID: 189 Comm: systemd-udevd Tainted: G W 5.9.0-rc6-00010-g25d95d0623751 #1
[ 10.828636] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 10.830631] Call Trace:
[ 10.831348] dump_stack+0x57/0x80
[ 10.832253] __cant_sleep.cold+0x68/0x75
[ 10.833228] __seccomp_filter+0x7e/0x680
[ 10.834143] ? __switch_to_asm+0x42/0x70
[ 10.835154] syscall_trace_enter+0x83/0x1e0
[ 10.836505] do_syscall_64+0xf/0x40
[ 10.837349] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 10.838368] RIP: 0033:0x7f31b4f36fd7
[ 10.839137] Code: 48 83 ec 80 5b 5d 41 5c c3 66 2e 0f 1f 84 00 00 00 00 00 b8 ff ff ff ff eb d6 e8 e4 07 02 00 0f 1f 40 00 b8 5f 00 00 00 0f 05 <c3> 0f 1f 84 00 00 00 00 00 b8 5a 00 00 00 0f 05 48 3d 01 f0 ff ff
[ 10.842410] RSP: 002b:00007fff9f8539f8 EFLAGS: 00000206 ORIG_RAX: 000000000000005f
[ 10.843757] RAX: ffffffffffffffda RBX: 0000559f0874b960 RCX: 00007f31b4f36fd7
[ 10.845012] RDX: 00000000000800c2 RSI: 0000559f0874b960 RDI: 0000000000000012
[ 10.846208] RBP: 00007fff9f853a80 R08: 0000000000000016 R09: 0000559f0874b97c
[ 10.847426] R10: 0000000000000180 R11: 0000000000000206 R12: 00007fff9f853a78
[ 10.848608] R13: 00007fff9f853a4f R14: 0000000000000012 R15: 0000000000000006
Starting LSB: Load kernel image with kexec...
[ OK ] Started LSB: Load kernel image with kexec.
[ 10.893602] scsi host0: ata_piix
[ 10.896952] IPMI message handler: version 39.2
[ 10.905583] ipmi device interface
[ OK ] Started Login Service.
LKP: HOSTNAME vm-snb-11, MAC 52:54:00:12:34:56, kernel 5.9.0-rc6-00010-g25d95d0623751 1, serial console /dev/ttyS0
[ 10.920625] scsi host1: ata_piix
[ 10.921689] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc040 irq 14
[ 10.923123] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc048 irq 15
[ 10.985404] ipmi_si: IPMI System Interface driver
[ 10.986829] ipmi_si: Unable to find any System Interface(s)
[FAILED] Failed to start LSB: OpenIPMI Driver init script.
See 'systemctl status openipmi.service' for details.
[ 11.090976] ata2.01: NODEV after polling detection
[ 11.096434] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
[ 11.099079] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5
[ 11.120492] ppdev: user-space parallel port driver
[ 11.193498] scsi 1:0:0:0: Attached scsi generic sg0 type 5
[ 11.261908] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
[ 11.263319] cdrom: Uniform CD-ROM driver Revision: 3.20
[ 11.273346] sr 1:0:0:0: Attached scsi CD-ROM sr0
[ 11.291046] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console
[ 11.302594] Console: switching to colour dummy device 80x25
[ 11.304468] [drm] Found bochs VGA, ID 0xb0c0.
[ 11.305151] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebf0000.
[ 11.309645] [TTM] Zone kernel: Available graphics memory: 4074508 KiB
[ 11.310875] [TTM] Zone dma32: Available graphics memory: 2097152 KiB
[ 11.311998] [TTM] Initializing pool allocator
[ 11.313245] [TTM] Initializing DMA pool allocator
[ 11.314735] [drm] Initialized bochs-drm 1.0.0 20130925 for 0000:00:02.0 on minor 0
[ 11.319970] fbcon: bochs-drmdrmfb (fb0) is primary device
[ 11.336008] Console: switching to colour frame buffer device 128x48
[ 11.345721] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device
[ 12.108197] BUG: assuming atomic context at include/linux/filter.h:597
[ 12.112588] in_atomic(): 0, irqs_disabled(): 0, pid: 223, name: systemd-logind
[ 12.113809] CPU: 0 PID: 223 Comm: systemd-logind Tainted: G W 5.9.0-rc6-00010-g25d95d0623751 #1
[ 12.113810] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 12.113811] Call Trace:
[ 12.113822] dump_stack+0x57/0x80
[ 12.117718] __seccomp_filter+0x7e/0x680
[ 12.118367] ? ep_poll+0x1e9/0x4e0
[ 12.118958] syscall_trace_enter+0x83/0x1e0
[ 12.119783] do_syscall_64+0xf/0x40
[ 12.120371] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 12.121214] RIP: 0033:0x7fc0b76567b4
[ 12.121807] Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b1 0f 1f 80 00 00 00 00 48 8d 05 49 5f 0c 00 8b 00 85 c0 75 13 b8 2f 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 41 89 d4 55 48 89 f5 53
[ 12.126243] RSP: 002b:00007ffe31fc6ca8 EFLAGS: 00000246 ORIG_RAX: 000000000000002f
[ 12.127517] RAX: ffffffffffffffda RBX: 000055a169cb2e30 RCX: 00007fc0b76567b4
[ 12.128510] RDX: 0000000040000040 RSI: 00007ffe31fc6ce0 RDI: 000000000000000e
[ 12.129450] RBP: 00007ffe31fc6cd0 R08: 0000000000b69f94 R09: 000000000000000b
[ 12.130617] R10: 00007ffe31fec000 R11: 0000000000000246 R12: 00007ffe31fc6cc8
[ 12.131809] R13: 000055a169cb2e30 R14: 0000000000000000 R15: 0000000000000000
9mPreprocess NFS configuration...
[ OK ] Reached target Host and Network Name Lookups.
[ OK ] Started Preprocess NFS configuration.
Starting NFS status monitor for NFSv2/3 locking....
Starting Notify NFS peers of a restart...
[ OK ] Started Notify NFS peers of a restart.
[ OK ] Started NFS status monitor for NFSv2/3 locking..
[ 12.284556] random: crng init done
[ 12.286119] random: 7 urandom warning(s) missed due to ratelimiting
[ OK ] Started OpenBSD Secure Shell server.
[ 15.538606] BUG: assuming atomic context at include/linux/filter.h:597
[ 15.541153] in_atomic(): 0, irqs_disabled(): 0, pid: 165, name: systemd-udevd
[ 15.543729] CPU: 0 PID: 165 Comm: systemd-udevd Tainted: G W 5.9.0-rc6-00010-g25d95d0623751 #1
[ 15.546685] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 15.549225] Call Trace:
[ 15.550836] dump_stack+0x57/0x80
[ 15.552645] __cant_sleep.cold+0x68/0x75
[ 15.554421] __seccomp_filter+0x7e/0x680
[ 15.556343] ? ep_poll+0x1e9/0x4e0
[ 15.558116] syscall_trace_enter+0x83/0x1e0
[ 15.560069] do_syscall_64+0xf/0x40
[ 15.561873] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 15.568005] RIP: 0033:0x7f31b4f37461
[ 15.569862] Code: fe ff ff 50 48 8d 3d fe d0 09 00 e8 e9 03 02 00 66 0f 1f 84 00 00 00 00 00 48 8d 05 99 62 0d 00 8b 00 85 c0 75 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 41 54 49 89 d4 55 48
[ 15.575428] RSP: 002b:00007fff9f853e68 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[ 15.577996] RAX: ffffffffffffffda RBX: 0000559f08744db8 RCX: 00007f31b4f37461
[ 15.580609] RDX: 0000000000000008 RSI: 00007fff9f853e70 RDI: 000000000000000d
[ 15.582964] RBP: 00007fff9f854000 R08: 0000000000eaf7a2 R09: 000000000000000f
[ 15.585537] R10: 00007fff9f9a1000 R11: 0000000000000246 R12: 00007fff9f853e90
[ 15.587966] R13: 0000559f08744da0 R14: 0000000000000001 R15: 00007fff9f853e9c
[ 20.538604] BUG: assuming atomic context at include/linux/filter.h:597
[ 20.541188] in_atomic(): 0, irqs_disabled(): 0, pid: 209, name: systemd-timesyn
[ 20.543554] CPU: 0 PID: 209 Comm: systemd-timesyn Tainted: G W 5.9.0-rc6-00010-g25d95d0623751 #1
[ 20.546364] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 20.548822] Call Trace:
[ 20.550416] dump_stack+0x57/0x80
[ 20.552250] __cant_sleep.cold+0x68/0x75
[ 20.554054] __seccomp_filter+0x7e/0x680
[ 20.555920] ? ep_poll+0x1e9/0x4e0
[ 20.557705] syscall_trace_enter+0x83/0x1e0
[ 20.559641] do_syscall_64+0xf/0x40
[ 20.561222] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 20.563248] RIP: 0033:0x7fe472a8f494
[ 20.573230] Code: 66 0f 1f 44 00 00 41 54 49 89 d4 55 48 89 f5 53 89 fb 48 83 ec 10 e8 cb c2 01 00 4c 89 e2 48 89 ee 89 df 41 89 c0 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 38 44 89 c7 48 89 44 24 08 e8 07 c3 01 00 48
[ 20.578771] RSP: 002b:00007fff46f52e10 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[ 20.581396] RAX: ffffffffffffffda RBX: 0000000000000010 RCX: 00007fe472a8f494
[ 20.583703] RDX: 0000000000000008 RSI: 00007fff46f52e40 RDI: 0000000000000010
[ 20.585996] RBP: 00007fff46f52e40 R08: 0000000000000000 R09: 0000000000000014
[ 20.588598] R10: 00007fff46f6a000 R11: 0000000000000246 R12: 0000000000000008
[ 20.590957] R13: 0000000000000001 R14: 00007fff46f52e6c R15: 00007fff46f52e60
[ 23.728449] sysrq: Emergency Sync
[ 23.731860] BUG: assuming atomic context at include/linux/filter.h:597
[ 23.732416] Emergency Sync complete
[ 23.734323] in_atomic(): 0, irqs_disabled(): 0, pid: 156, name: systemd-journal
[ 23.736313] sysrq: Resetting
[
Kboot worker: lkp-worker47
Elapsed time: 60
kvm=(
qemu-system-x86_64
-enable-kvm
-cpu SandyBridge
-kernel $kernel
-initrd initrd-vm-snb-11.cgz
-m 8192
-smp 2
-device e1000,netdev=net0
-netdev user,id=net0,hostfwd=tcp::32032-:22
-boot order=nc
-no-reboot
-watchdog i6300esb
-watchdog-action debug
-rtc base=localtime
-serial stdio
-display none
-monitor null
)
append=(
ip=::::vm-snb-11::dhcp
root=/dev/ram0
user=lkp
job=/job-script
ARCH=x86_64
kconfig=x86_64-rhel-8.3
branch=tglx-devel/highmem
commit=25d95d0623751d029c3177527d44563040ae828b
BOOT_IMAGE=/pkg/linux/x86_64-rhel-8.3/gcc-9/25d95d0623751d029c3177527d44563040ae828b/vmlinuz-5.9.0-rc6-00010-g25d95d0623751
max_uptime=600
RESULT_ROOT=/result/boot/1/vm-snb/debian-10.4-x86_64-20200603.cgz/x86_64-rhel-8.3/gcc-9/25d95d0623751d029c3177527d44563040ae828b/3
result_service=tmpfs
selinux=0
debug
apic=debug
sysrq_always_enabled
rcupdate.rcu_cpu_stall_timeout=100
net.ifnames=0
printk.devkmsg=on
panic=-1
softlockup_panic=1
nmi_watchdog=panic
oops=panic
load_ramdisk=2
prompt_ramdisk=0
drbd.minor_count=8
systemd.log_level=err
ignore_loglevel
console=tty0
earlyprintk=ttyS0,115200
console=ttyS0,115200
vga=normal
rw
rcuperf.shutdown=0
watchdog_thresh=60
)
"${kvm[@]}" -append "${append[*]}"
To reproduce:
# build kernel
cd linux
cp config-5.9.0-rc6-00010-g25d95d0623751 .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
1 year, 10 months
[x86/msi] 3b9c1d377d: BUG:kernel_NULL_pointer_dereference,address
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 3b9c1d377d67072d1d8a2373b4969103cca00dab ("x86/msi: Consolidate MSI allocation")
https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git x86/irq
in testcase: boot
on test machine: 8 threads Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz with 16G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 6.377874] BUG: kernel NULL pointer dereference, address: 0000000000000018
[ 6.378864] #PF: supervisor read access in kernel mode
[ 6.378864] #PF: error_code(0x0000) - not-present page
[ 6.378864] PGD 0 P4D 0
[ 6.378864] Oops: 0000 [#1] SMP PTI
[ 6.378864] CPU: 3 PID: 1 Comm: swapper/0 Tainted: G I 5.9.0-rc5-00018-g3b9c1d377d670 #1
[ 6.378864] Hardware name: Dell Inc. OptiPlex 7040/0Y7WYT, BIOS 1.2.8 01/26/2016
[ 6.378864] RIP: 0010:msi_desc_to_pci_dev+0x5/0x20
[ 6.378864] Code: cc cc cc cc cc 0f 1f 44 00 00 8b 05 35 cc 12 01 c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 <48> 8b 47 18 48 2d b0 00 00 00 c3 66 66 2e 0f 1f 84 00 00 00 00 00
[ 6.378864] RSP: 0000:ffffc9000002bad8 EFLAGS: 00010297
[ 6.378864] RAX: 0000000000000000 RBX: ffff88845956a000 RCX: 0000000000000000
[ 6.378864] RDX: ffffc9000002bae8 RSI: 0000000000000000 RDI: 0000000000000000
[ 6.378864] RBP: ffff888459a98c00 R08: ffff88845956a2e0 R09: 0000000000000000
[ 6.378864] R10: 0000000000000008 R11: 0000000000000000 R12: 0000000000000001
[ 6.378864] R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000000
[ 6.378864] FS: 0000000000000000(0000) GS:ffff888459cc0000(0000) knlGS:0000000000000000
[ 6.378864] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6.378864] CR2: 0000000000000018 CR3: 000000045860a001 CR4: 00000000003706e0
[ 6.378864] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6.378864] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 6.378864] Call Trace:
[ 6.378864] intel_get_irq_domain+0x24/0xc0
[ 6.378864] native_setup_msi_irqs+0x3b/0xa0
[ 6.378864] __pci_enable_msi_range+0x187/0x2c0
[ 6.378864] pci_alloc_irq_vectors_affinity+0xbf/0x100
[ 6.378864] pcie_port_device_register+0x138/0x560
[ 6.378864] ? _cond_resched+0x19/0x40
[ 6.378864] pcie_portdrv_probe+0x2d/0xc0
[ 6.378864] local_pci_probe+0x42/0x80
[ 6.378864] ? _cond_resched+0x19/0x40
[ 6.378864] pci_device_probe+0x107/0x1c0
[ 6.378864] really_probe+0x1ca/0x420
[ 6.378864] driver_probe_device+0xe1/0x160
[ 6.378864] device_driver_attach+0x53/0x60
[ 6.378864] __driver_attach+0x8a/0x160
[ 6.378864] ? device_driver_attach+0x60/0x60
[ 6.378864] ? device_driver_attach+0x60/0x60
[ 6.378864] bus_for_each_dev+0x78/0xc0
[ 6.378864] bus_add_driver+0x14d/0x200
[ 6.378864] driver_register+0x6c/0xc0
[ 6.378864] ? dmi_pcie_pme_disable_msi+0x1f/0x1f
[ 6.378864] do_one_initcall+0x46/0x204
[ 6.378864] kernel_init_freeable+0x1da/0x23c
[ 6.378864] ? rest_init+0xc6/0xc6
[ 6.378864] kernel_init+0xa/0x11a
[ 6.378864] ret_from_fork+0x22/0x30
[ 6.378864] Modules linked in:
[ 6.378864] CR2: 0000000000000018
[ 6.378864] ---[ end trace 33aa934cfab777f0 ]---
To reproduce:
# build kernel
cd linux
cp config-5.9.0-rc5-00018-g3b9c1d377d670 .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
1 year, 10 months
[mm/lru] 44c86dfabf: fio.read_iops -11.8% regression
by kernel test robot
Greeting,
FYI, we noticed a -11.8% regression of fio.read_iops due to commit:
commit: 44c86dfabfbbb310b5135c0e8b66b193b9d7e896 ("mm/lru: replace pgdat lru_lock with lruvec lock")
https://github.com/alexshi/linux.git lruv19.5
in testcase: fio-basic
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
with following parameters:
disk: 2pmem
fs: btrfs
runtime: 200s
nr_task: 50%
time_based: tb
rw: read
bs: 2M
ioengine: sync
test_size: 100G
cpufreq_governor: performance
ucode: 0x5002f01
test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio
In addition to that, the commit also has significant impact on the following tests:
+------------------+----------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_process_ops -11.1% regression |
| test machine | 104 threads Skylake with 192G memory |
| test parameters | cpufreq_governor=performance |
| | mode=process |
| | nr_task=100% |
| | test=page_fault2 |
| | ucode=0x2006906 |
+------------------+----------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
2M/gcc-9/performance/2pmem/btrfs/sync/x86_64-rhel-8.3/50%/debian-10.4-x86_64-20200603.cgz/200s/read/lkp-csl-2sp6/100G/fio-basic/tb/0x5002f01
commit:
0c5e8d9feb ("mm/mlock: remove __munlock_isolate_lru_page")
44c86dfabf ("mm/lru: replace pgdat lru_lock with lruvec lock")
0c5e8d9febd9f026 44c86dfabfbbb310b5135c0e8b6
---------------- ---------------------------
%stddev %change %stddev
\ | \
10.99 ± 3% -1.0 10.00 fio.latency_1000us%
0.94 ± 3% +0.3 1.26 ± 10% fio.latency_100ms%
7.42 -2.9 4.51 ± 2% fio.latency_10ms%
10.09 +0.9 11.01 fio.latency_20ms%
24.65 -2.0 22.69 fio.latency_2ms%
4.20 ± 8% -0.9 3.29 ± 3% fio.latency_4ms%
17.35 ± 3% +5.6 22.96 fio.latency_500us%
6.55 +1.6 8.12 fio.latency_50ms%
17.80 -1.7 16.14 fio.latency_750us%
15477 -11.8% 13658 fio.read_bw_MBps
15597568 +22.3% 19070976 fio.read_clat_90%_us
33161216 ± 2% +7.1% 35520512 fio.read_clat_95%_us
49545216 +4.8% 51904512 fio.read_clat_99%_us
5548318 +9.8% 6093570 fio.read_clat_mean_us
10774701 +7.7% 11604571 fio.read_clat_stddev
7738 -11.8% 6829 fio.read_iops
6.35e+09 -11.7% 5.604e+09 fio.time.file_system_inputs
264480 -12.5% 231508 fio.time.involuntary_context_switches
18.04 -10.7% 16.11 fio.time.user_time
1548992 -11.7% 1367144 fio.workload
7.31 ± 10% +13.9% 8.32 ± 6% sched_debug.cpu.nr_uninterruptible.stddev
37.03 +6.0% 39.25 iostat.cpu.idle
62.85 -3.5% 60.64 iostat.cpu.system
1.09 -0.1 0.96 mpstat.cpu.all.irq%
0.11 ± 2% -0.0 0.08 mpstat.cpu.all.soft%
2121 ± 10% +14.9% 2436 ± 6% slabinfo.kmalloc-rcl-96.active_objs
2121 ± 10% +14.9% 2436 ± 6% slabinfo.kmalloc-rcl-96.num_objs
3.314e+08 ± 3% -11.4% 2.937e+08 ± 2% numa-numastat.node0.local_node
3.314e+08 ± 3% -11.4% 2.938e+08 ± 2% numa-numastat.node0.numa_hit
3.425e+08 ± 4% -9.5% 3.1e+08 numa-numastat.node1.local_node
3.425e+08 ± 4% -9.5% 3.1e+08 numa-numastat.node1.numa_hit
36.25 +6.9% 38.75 vmstat.cpu.id
62.25 -3.6% 60.00 vmstat.cpu.sy
15563182 -11.8% 13719448 vmstat.io.bi
32059 ± 6% -16.0% 26929 vmstat.system.cs
168.00 ± 4% -36.3% 107.00 ± 11% numa-vmstat.node0.nr_isolated_file
1.781e+08 ± 6% -14.2% 1.528e+08 ± 8% numa-vmstat.node0.numa_hit
1.78e+08 ± 6% -14.2% 1.527e+08 ± 8% numa-vmstat.node0.numa_local
20557 ± 3% -15.4% 17389 ± 6% numa-vmstat.node0.workingset_nodes
155.00 ± 6% -30.3% 108.00 ± 8% numa-vmstat.node1.nr_isolated_file
8686 ± 11% -11.4% 7691 softirqs.CPU23.RCU
12327 ± 17% +29.9% 16011 ± 8% softirqs.CPU23.SCHED
8707 ± 4% +77.0% 15413 ± 44% softirqs.CPU26.RCU
15704 ± 5% -24.2% 11902 ± 8% softirqs.CPU34.SCHED
15612 ± 5% -19.8% 12523 ± 11% softirqs.CPU46.SCHED
19364 ± 7% -11.7% 17107 ± 6% softirqs.CPU74.SCHED
18491 ± 8% -10.2% 16597 ± 8% softirqs.CPU80.SCHED
9169 ± 24% -25.5% 6833 ± 4% softirqs.CPU82.RCU
16991 ± 7% -19.4% 13690 ± 11% softirqs.CPU9.SCHED
100144 ± 2% -15.2% 84953 proc-vmstat.allocstall_movable
5335 ± 19% -46.5% 2855 ± 19% proc-vmstat.allocstall_normal
1127 ± 5% +18.3% 1333 ± 10% proc-vmstat.compact_daemon_wake
663260 ± 29% +60.5% 1064753 ± 6% proc-vmstat.compact_isolated
2941 ± 4% +11.4% 3276 proc-vmstat.kswapd_low_wmark_hit_quickly
324.75 -33.3% 216.50 ± 2% proc-vmstat.nr_isolated_file
1.201e+08 ± 3% -18.6% 97790056 ± 5% proc-vmstat.numa_foreign
6.726e+08 -10.4% 6.025e+08 proc-vmstat.numa_hit
6.726e+08 -10.4% 6.025e+08 proc-vmstat.numa_local
1.201e+08 ± 3% -18.6% 97790056 ± 5% proc-vmstat.numa_miss
1.201e+08 ± 3% -18.6% 97821690 ± 5% proc-vmstat.numa_other
3009 ± 4% +15.6% 3478 proc-vmstat.pageoutrun
28720822 -12.6% 25115422 ± 2% proc-vmstat.pgalloc_dma32
7.667e+08 -11.7% 6.772e+08 proc-vmstat.pgalloc_normal
7.856e+08 -11.8% 6.929e+08 proc-vmstat.pgfree
341910 ± 28% +60.0% 546934 ± 7% proc-vmstat.pgmigrate_success
3.173e+09 -11.7% 2.8e+09 proc-vmstat.pgpgin
2.665e+08 -16.8% 2.217e+08 proc-vmstat.pgscan_direct
3.527e+08 -15.5% 2.98e+08 proc-vmstat.pgscan_file
86172597 -11.4% 76352858 ± 2% proc-vmstat.pgscan_kswapd
2.665e+08 -16.8% 2.216e+08 proc-vmstat.pgsteal_direct
3.527e+08 -15.5% 2.98e+08 proc-vmstat.pgsteal_file
86162757 -11.4% 76344354 ± 2% proc-vmstat.pgsteal_kswapd
32455 -9.2% 29454 proc-vmstat.workingset_nodes
29.22 -3.8% 28.12 perf-stat.i.MPKI
9.106e+09 -7.8% 8.4e+09 perf-stat.i.branch-instructions
0.31 -0.0 0.28 perf-stat.i.branch-miss-rate%
27498123 -17.1% 22805880 perf-stat.i.branch-misses
1.149e+09 -12.2% 1.009e+09 perf-stat.i.cache-misses
1.437e+09 -12.3% 1.26e+09 perf-stat.i.cache-references
32277 ± 6% -15.8% 27164 perf-stat.i.context-switches
3.47 +6.1% 3.68 perf-stat.i.cpi
1.694e+11 -3.3% 1.638e+11 perf-stat.i.cpu-cycles
199.09 ± 2% -10.8% 177.56 perf-stat.i.cpu-migrations
155.84 +9.9% 171.22 perf-stat.i.cycles-between-cache-misses
1.316e+10 -9.2% 1.195e+10 perf-stat.i.dTLB-loads
6.432e+09 -12.3% 5.64e+09 perf-stat.i.dTLB-stores
2945413 -6.9% 2742636 ± 2% perf-stat.i.iTLB-load-misses
2348427 ± 2% -4.9% 2233557 ± 3% perf-stat.i.iTLB-loads
4.901e+10 -8.9% 4.465e+10 perf-stat.i.instructions
0.29 -5.8% 0.27 perf-stat.i.ipc
1.76 -3.3% 1.71 perf-stat.i.metric.GHz
0.40 -10.8% 0.35 perf-stat.i.metric.K/sec
317.63 -9.6% 287.13 perf-stat.i.metric.M/sec
25.87 ± 2% -1.3 24.57 perf-stat.i.node-load-miss-rate%
54426063 ± 2% -17.3% 45032091 perf-stat.i.node-load-misses
1.598e+08 -11.1% 1.421e+08 perf-stat.i.node-loads
33.50 ± 2% -2.0 31.51 ± 3% perf-stat.i.node-store-miss-rate%
47107474 ± 3% -17.3% 38976821 ± 4% perf-stat.i.node-store-misses
94320159 -9.4% 85453107 perf-stat.i.node-stores
29.33 -3.7% 28.23 perf-stat.overall.MPKI
0.30 -0.0 0.27 perf-stat.overall.branch-miss-rate%
3.46 +6.1% 3.67 perf-stat.overall.cpi
147.39 +10.1% 162.33 perf-stat.overall.cycles-between-cache-misses
0.29 -5.8% 0.27 perf-stat.overall.ipc
25.38 ± 2% -1.3 24.03 perf-stat.overall.node-load-miss-rate%
33.29 ± 2% -2.0 31.27 ± 3% perf-stat.overall.node-store-miss-rate%
6360319 +3.3% 6568417 perf-stat.overall.path-length
9.052e+09 -7.7% 8.354e+09 perf-stat.ps.branch-instructions
27321158 -17.0% 22669616 perf-stat.ps.branch-misses
1.143e+09 -12.2% 1.004e+09 perf-stat.ps.cache-misses
1.429e+09 -12.3% 1.254e+09 perf-stat.ps.cache-references
32159 ± 6% -16.0% 27007 perf-stat.ps.context-switches
1.685e+11 -3.3% 1.63e+11 perf-stat.ps.cpu-cycles
198.36 ± 2% -10.7% 177.07 perf-stat.ps.cpu-migrations
1.308e+10 -9.2% 1.188e+10 perf-stat.ps.dTLB-loads
6.397e+09 -12.3% 5.612e+09 perf-stat.ps.dTLB-stores
2928738 -6.9% 2727615 ± 2% perf-stat.ps.iTLB-load-misses
2334022 ± 2% -4.9% 2220752 ± 3% perf-stat.ps.iTLB-loads
4.873e+10 -8.9% 4.441e+10 perf-stat.ps.instructions
54095968 ± 2% -17.3% 44762722 perf-stat.ps.node-load-misses
1.591e+08 -11.0% 1.416e+08 perf-stat.ps.node-loads
46879405 ± 3% -17.3% 38763251 ± 4% perf-stat.ps.node-store-misses
93903849 -9.3% 85155066 perf-stat.ps.node-stores
9.852e+12 -8.9% 8.98e+12 perf-stat.total.instructions
317.00 ±150% -99.4% 1.75 ±142% interrupts.56:PCI-MSI.31981589-edge.i40e-eth0-TxRx-20
187865 ± 20% -25.6% 139835 ± 3% interrupts.CAL:Function_call_interrupts
1073 ± 7% -21.1% 847.00 ± 7% interrupts.CPU0.RES:Rescheduling_interrupts
2159 ± 51% -43.3% 1225 ± 4% interrupts.CPU1.CAL:Function_call_interrupts
853.25 ± 23% -30.9% 589.50 ± 8% interrupts.CPU1.RES:Rescheduling_interrupts
734.75 ± 9% -16.8% 611.00 ± 13% interrupts.CPU10.RES:Rescheduling_interrupts
1568 ± 17% -19.6% 1260 ± 6% interrupts.CPU11.CAL:Function_call_interrupts
728.50 ± 10% -17.6% 600.25 ± 11% interrupts.CPU11.RES:Rescheduling_interrupts
699.50 ± 12% -17.1% 579.75 ± 7% interrupts.CPU12.RES:Rescheduling_interrupts
1610 ± 8% -23.3% 1234 ± 9% interrupts.CPU13.CAL:Function_call_interrupts
727.00 ± 11% -21.9% 568.00 ± 11% interrupts.CPU13.RES:Rescheduling_interrupts
764.00 ± 10% -15.2% 647.75 ± 10% interrupts.CPU14.RES:Rescheduling_interrupts
2019 ± 42% -37.2% 1269 ± 15% interrupts.CPU15.CAL:Function_call_interrupts
755.00 ± 22% -18.7% 613.75 ± 6% interrupts.CPU16.RES:Rescheduling_interrupts
736.25 ± 10% -18.6% 599.00 ± 7% interrupts.CPU17.RES:Rescheduling_interrupts
1596 ± 19% -22.3% 1240 ± 9% interrupts.CPU18.CAL:Function_call_interrupts
706.00 ± 11% -15.1% 599.25 ± 12% interrupts.CPU18.RES:Rescheduling_interrupts
702.25 ± 8% -17.1% 582.50 ± 7% interrupts.CPU19.RES:Rescheduling_interrupts
2081 ± 40% -39.3% 1262 ± 10% interrupts.CPU2.CAL:Function_call_interrupts
316.75 ±150% -99.6% 1.25 ±173% interrupts.CPU20.56:PCI-MSI.31981589-edge.i40e-eth0-TxRx-20
1744 ± 43% -33.4% 1161 ± 5% interrupts.CPU20.CAL:Function_call_interrupts
758.25 ± 23% -20.6% 602.25 ± 7% interrupts.CPU20.RES:Rescheduling_interrupts
1786 ± 32% -33.7% 1184 ± 7% interrupts.CPU22.CAL:Function_call_interrupts
764.50 ± 15% -25.4% 570.50 ± 8% interrupts.CPU22.RES:Rescheduling_interrupts
7068 ± 7% -28.6% 5048 ± 33% interrupts.CPU23.NMI:Non-maskable_interrupts
7068 ± 7% -28.6% 5048 ± 33% interrupts.CPU23.PMI:Performance_monitoring_interrupts
692.25 ± 10% -15.3% 586.00 ± 6% interrupts.CPU23.RES:Rescheduling_interrupts
1579 ± 9% -14.4% 1352 ± 4% interrupts.CPU24.CAL:Function_call_interrupts
653.00 ± 6% -11.2% 579.75 ± 5% interrupts.CPU24.RES:Rescheduling_interrupts
661.25 ± 6% -14.1% 567.75 ± 8% interrupts.CPU25.RES:Rescheduling_interrupts
707.50 ± 6% -15.6% 597.25 ± 14% interrupts.CPU27.RES:Rescheduling_interrupts
1891 ± 12% -26.5% 1390 ± 7% interrupts.CPU29.CAL:Function_call_interrupts
688.25 ± 5% -18.3% 562.50 ± 6% interrupts.CPU29.RES:Rescheduling_interrupts
1756 ± 9% -20.7% 1391 ± 5% interrupts.CPU32.CAL:Function_call_interrupts
688.25 ± 6% -11.5% 609.25 ± 3% interrupts.CPU32.RES:Rescheduling_interrupts
1791 ± 17% -27.8% 1292 ± 15% interrupts.CPU33.CAL:Function_call_interrupts
1776 ± 6% -26.1% 1312 ± 4% interrupts.CPU34.CAL:Function_call_interrupts
669.25 ± 6% -11.5% 592.50 ± 2% interrupts.CPU34.RES:Rescheduling_interrupts
6762 ± 8% -46.8% 3599 ± 13% interrupts.CPU35.NMI:Non-maskable_interrupts
6762 ± 8% -46.8% 3599 ± 13% interrupts.CPU35.PMI:Performance_monitoring_interrupts
1900 ± 11% -19.9% 1521 ± 19% interrupts.CPU37.CAL:Function_call_interrupts
708.75 ± 3% -12.5% 620.25 ± 14% interrupts.CPU37.RES:Rescheduling_interrupts
6945 ± 6% -32.5% 4685 ± 35% interrupts.CPU39.NMI:Non-maskable_interrupts
6945 ± 6% -32.5% 4685 ± 35% interrupts.CPU39.PMI:Performance_monitoring_interrupts
1750 ± 31% -26.9% 1279 ± 7% interrupts.CPU4.CAL:Function_call_interrupts
722.00 ± 9% -16.8% 600.50 ± 11% interrupts.CPU4.RES:Rescheduling_interrupts
1699 ± 7% -17.5% 1401 ± 13% interrupts.CPU41.CAL:Function_call_interrupts
651.75 ± 7% -13.8% 561.75 ± 8% interrupts.CPU41.RES:Rescheduling_interrupts
1672 ± 4% -16.0% 1405 ± 16% interrupts.CPU42.CAL:Function_call_interrupts
1563 ± 6% -12.6% 1366 ± 2% interrupts.CPU44.CAL:Function_call_interrupts
667.25 ± 5% -17.4% 551.00 ± 8% interrupts.CPU45.RES:Rescheduling_interrupts
903.00 ± 29% -32.6% 608.50 ± 12% interrupts.CPU48.RES:Rescheduling_interrupts
2112 ± 18% -35.6% 1359 ± 6% interrupts.CPU49.CAL:Function_call_interrupts
797.25 ± 11% -23.9% 607.00 ± 11% interrupts.CPU49.RES:Rescheduling_interrupts
1880 ± 38% -35.5% 1213 ± 9% interrupts.CPU5.CAL:Function_call_interrupts
734.50 ± 16% -22.2% 571.75 ± 8% interrupts.CPU5.RES:Rescheduling_interrupts
2418 ± 46% -42.3% 1395 ± 8% interrupts.CPU50.CAL:Function_call_interrupts
830.50 ± 17% -30.2% 579.75 ± 12% interrupts.CPU50.RES:Rescheduling_interrupts
2390 ± 48% -42.2% 1382 interrupts.CPU51.CAL:Function_call_interrupts
2274 ± 35% -36.0% 1455 ± 6% interrupts.CPU52.CAL:Function_call_interrupts
827.75 ± 29% -28.5% 592.25 ± 5% interrupts.CPU53.RES:Rescheduling_interrupts
2180 ± 34% -31.8% 1486 ± 8% interrupts.CPU54.CAL:Function_call_interrupts
774.00 ± 18% -22.7% 598.00 ± 8% interrupts.CPU54.RES:Rescheduling_interrupts
2461 ± 46% -39.9% 1480 interrupts.CPU55.CAL:Function_call_interrupts
818.75 ± 17% -25.3% 611.75 ± 11% interrupts.CPU55.RES:Rescheduling_interrupts
1969 ± 8% -25.4% 1469 ± 11% interrupts.CPU56.CAL:Function_call_interrupts
783.50 ± 9% -22.0% 610.75 ± 9% interrupts.CPU56.RES:Rescheduling_interrupts
1811 ± 8% -17.7% 1491 ± 9% interrupts.CPU59.CAL:Function_call_interrupts
709.50 ± 5% -18.7% 577.00 ± 10% interrupts.CPU59.RES:Rescheduling_interrupts
2466 ± 35% -43.3% 1397 ± 6% interrupts.CPU60.CAL:Function_call_interrupts
4211 ± 27% +57.4% 6629 ± 14% interrupts.CPU60.NMI:Non-maskable_interrupts
4211 ± 27% +57.4% 6629 ± 14% interrupts.CPU60.PMI:Performance_monitoring_interrupts
796.75 ± 16% -27.4% 578.50 ± 10% interrupts.CPU60.RES:Rescheduling_interrupts
2703 ± 59% -45.8% 1464 ± 11% interrupts.CPU61.CAL:Function_call_interrupts
823.75 ± 20% -25.1% 617.25 ± 9% interrupts.CPU61.RES:Rescheduling_interrupts
2957 ± 67% -50.5% 1464 ± 5% interrupts.CPU62.CAL:Function_call_interrupts
933.00 ± 30% -33.9% 616.50 ± 4% interrupts.CPU62.RES:Rescheduling_interrupts
2366 ± 40% -37.6% 1477 ± 8% interrupts.CPU63.CAL:Function_call_interrupts
752.50 ± 5% -18.7% 612.00 ± 8% interrupts.CPU64.RES:Rescheduling_interrupts
1822 ± 15% -22.3% 1415 ± 6% interrupts.CPU65.CAL:Function_call_interrupts
703.00 ± 13% -19.7% 564.75 ± 4% interrupts.CPU65.RES:Rescheduling_interrupts
2142 ± 31% -33.2% 1431 ± 4% interrupts.CPU66.CAL:Function_call_interrupts
706.50 ± 15% -20.8% 559.25 ± 6% interrupts.CPU66.RES:Rescheduling_interrupts
2465 ± 45% -40.4% 1469 ± 6% interrupts.CPU67.CAL:Function_call_interrupts
1864 ± 7% -22.9% 1438 ± 2% interrupts.CPU68.CAL:Function_call_interrupts
786.50 ± 7% -25.3% 587.25 ± 4% interrupts.CPU68.RES:Rescheduling_interrupts
2686 ± 43% -42.6% 1541 ± 5% interrupts.CPU69.CAL:Function_call_interrupts
847.00 ± 26% -28.4% 606.50 ± 5% interrupts.CPU69.RES:Rescheduling_interrupts
2006 ± 36% -39.2% 1220 ± 4% interrupts.CPU7.CAL:Function_call_interrupts
2426 ± 41% -36.4% 1544 ± 7% interrupts.CPU70.CAL:Function_call_interrupts
755.75 ± 22% -23.2% 580.50 ± 7% interrupts.CPU70.RES:Rescheduling_interrupts
3307 ± 63% -58.3% 1378 ± 8% interrupts.CPU71.CAL:Function_call_interrupts
864.50 ± 28% -34.0% 570.50 ± 11% interrupts.CPU71.RES:Rescheduling_interrupts
1897 ± 7% -14.6% 1619 ± 8% interrupts.CPU77.CAL:Function_call_interrupts
2209 ± 21% -30.4% 1537 ± 5% interrupts.CPU78.CAL:Function_call_interrupts
667.75 ± 5% -16.8% 555.75 ± 11% interrupts.CPU78.RES:Rescheduling_interrupts
1906 ± 3% -16.2% 1598 ± 12% interrupts.CPU79.CAL:Function_call_interrupts
622.25 ± 8% -13.5% 538.00 ± 9% interrupts.CPU81.RES:Rescheduling_interrupts
1917 -14.8% 1632 ± 10% interrupts.CPU83.CAL:Function_call_interrupts
1985 ± 8% -19.0% 1608 ± 8% interrupts.CPU84.CAL:Function_call_interrupts
2316 ± 5% -27.5% 1679 ± 8% interrupts.CPU85.CAL:Function_call_interrupts
679.75 ± 9% -17.6% 560.00 ± 3% interrupts.CPU85.RES:Rescheduling_interrupts
677.75 ± 3% -15.4% 573.25 ± 12% interrupts.CPU86.RES:Rescheduling_interrupts
2145 ± 14% -30.8% 1484 ± 8% interrupts.CPU87.CAL:Function_call_interrupts
641.25 ± 5% -13.9% 552.25 ± 3% interrupts.CPU87.RES:Rescheduling_interrupts
1923 ± 6% -13.3% 1666 ± 4% interrupts.CPU88.CAL:Function_call_interrupts
2201 ± 7% -18.2% 1799 ± 14% interrupts.CPU89.CAL:Function_call_interrupts
2369 ± 60% -48.7% 1215 ± 14% interrupts.CPU9.CAL:Function_call_interrupts
791.00 ± 26% -23.7% 603.25 ± 13% interrupts.CPU9.RES:Rescheduling_interrupts
1818 ± 7% -10.1% 1634 ± 7% interrupts.CPU90.CAL:Function_call_interrupts
1897 ± 6% -16.2% 1590 ± 10% interrupts.CPU92.CAL:Function_call_interrupts
2067 ± 7% -24.2% 1567 ± 9% interrupts.CPU93.CAL:Function_call_interrupts
2059 ± 10% -21.8% 1611 ± 10% interrupts.CPU95.CAL:Function_call_interrupts
68476 ± 6% -17.0% 56847 interrupts.RES:Rescheduling_interrupts
5.80 ± 5% -5.8 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__pagevec_lru_add.lru_cache_add.add_to_page_cache_lru.page_cache_readahead_unbounded
5.75 ± 5% -5.7 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__pagevec_lru_add.lru_cache_add.add_to_page_cache_lru
22.83 -5.3 17.49 ± 4% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.page_cache_readahead_unbounded.generic_file_buffered_read.new_sync_read.vfs_read
12.61 ± 4% -5.1 7.54 ± 7% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
7.78 ± 8% -3.3 4.47 ± 9% perf-profile.calltrace.cycles-pp.btrfs_releasepage.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
7.67 ± 8% -3.3 4.40 ± 9% perf-profile.calltrace.cycles-pp.try_release_extent_mapping.btrfs_releasepage.shrink_page_list.shrink_inactive_list.shrink_lruvec
18.40 ± 2% -2.7 15.65 ± 5% perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.page_cache_readahead_unbounded.generic_file_buffered_read.new_sync_read
4.31 ± 7% -2.6 1.72 ± 7% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.page_cache_readahead_unbounded.generic_file_buffered_read.new_sync_read
17.87 ± 4% -2.6 15.31 ± 5% perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask
17.75 ± 4% -2.5 15.23 ± 5% perf-profile.calltrace.cycles-pp.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath
17.74 ± 4% -2.5 15.22 ± 5% perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages
3.98 ± 7% -2.5 1.50 ± 9% perf-profile.calltrace.cycles-pp.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.page_cache_readahead_unbounded.generic_file_buffered_read
3.74 ± 8% -2.5 1.29 ± 10% perf-profile.calltrace.cycles-pp.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.page_cache_readahead_unbounded
3.08 ± 8% -2.4 0.73 ± 13% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist
3.08 ± 9% -2.4 0.73 ± 13% perf-profile.calltrace.cycles-pp._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask
17.59 ± 3% -2.3 15.31 ± 5% perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.page_cache_readahead_unbounded.generic_file_buffered_read
17.58 ± 3% -2.3 15.31 ± 5% perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.page_cache_readahead_unbounded
4.49 ± 4% -2.0 2.44 ± 6% perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
3.32 -1.6 1.69 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
3.15 -1.6 1.56 ± 8% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list
3.65 ± 7% -1.4 2.23 ± 10% perf-profile.calltrace.cycles-pp.__clear_extent_bit.try_release_extent_mapping.btrfs_releasepage.shrink_page_list.shrink_inactive_list
3.26 ± 7% -1.3 1.94 ± 9% perf-profile.calltrace.cycles-pp.test_range_bit.try_release_extent_mapping.btrfs_releasepage.shrink_page_list.shrink_inactive_list
3.15 ± 7% -1.3 1.87 ± 11% perf-profile.calltrace.cycles-pp._raw_spin_lock.__clear_extent_bit.try_release_extent_mapping.btrfs_releasepage.shrink_page_list
3.03 ± 7% -1.2 1.79 ± 11% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__clear_extent_bit.try_release_extent_mapping.btrfs_releasepage
3.02 ± 7% -1.2 1.78 ± 10% perf-profile.calltrace.cycles-pp._raw_spin_lock.test_range_bit.try_release_extent_mapping.btrfs_releasepage.shrink_page_list
2.93 ± 8% -1.2 1.73 ± 10% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.test_range_bit.try_release_extent_mapping.btrfs_releasepage
1.82 ± 7% -0.7 1.12 ± 6% perf-profile.calltrace.cycles-pp.free_unref_page_list.release_pages.__pagevec_release.invalidate_mapping_pages.generic_fadvise
1.72 ± 7% -0.7 1.03 ± 6% perf-profile.calltrace.cycles-pp.free_pcppages_bulk.free_unref_page_list.release_pages.__pagevec_release.invalidate_mapping_pages
2.14 ± 3% -0.5 1.61 ± 4% perf-profile.calltrace.cycles-pp.__add_to_page_cache_locked.add_to_page_cache_lru.page_cache_readahead_unbounded.generic_file_buffered_read.new_sync_read
4.13 ± 6% -0.4 3.69 ± 2% perf-profile.calltrace.cycles-pp.ret_from_fork
4.13 ± 6% -0.4 3.69 ± 2% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
1.36 ± 9% -0.4 0.96 ± 4% perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node.balance_pgdat
0.58 ± 6% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.__list_del_entry_valid.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask
2.12 ± 7% -0.3 1.81 perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
2.02 ± 7% -0.3 1.73 perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
2.57 -0.3 2.32 ± 7% perf-profile.calltrace.cycles-pp.lru_note_cost.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
1.59 ± 5% -0.2 1.38 ± 2% perf-profile.calltrace.cycles-pp.pagecache_get_page.generic_file_buffered_read.new_sync_read.vfs_read.ksys_read
1.56 ± 5% -0.2 1.36 ± 2% perf-profile.calltrace.cycles-pp.find_get_entry.pagecache_get_page.generic_file_buffered_read.new_sync_read.vfs_read
0.83 ± 8% -0.2 0.64 ± 2% perf-profile.calltrace.cycles-pp.mem_cgroup_charge.__add_to_page_cache_locked.add_to_page_cache_lru.page_cache_readahead_unbounded.generic_file_buffered_read
1.12 ± 5% -0.1 0.99 perf-profile.calltrace.cycles-pp.btrfs_submit_bio_hook.submit_one_bio.extent_readahead.read_pages.page_cache_readahead_unbounded
1.10 ± 5% -0.1 0.97 perf-profile.calltrace.cycles-pp.btrfs_map_bio.btrfs_submit_bio_hook.submit_one_bio.extent_readahead.read_pages
1.10 ± 5% -0.1 0.97 perf-profile.calltrace.cycles-pp.submit_bio.btrfs_map_bio.btrfs_submit_bio_hook.submit_one_bio.extent_readahead
1.12 ± 5% -0.1 0.99 perf-profile.calltrace.cycles-pp.submit_one_bio.extent_readahead.read_pages.page_cache_readahead_unbounded.generic_file_buffered_read
0.80 ± 6% -0.1 0.68 ± 3% perf-profile.calltrace.cycles-pp.crc_42
6.53 ± 8% +1.9 8.44 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_fadvise64.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.53 ± 8% +1.9 8.44 ± 5% perf-profile.calltrace.cycles-pp.ksys_fadvise64_64.__x64_sys_fadvise64.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.53 ± 8% +1.9 8.44 ± 5% perf-profile.calltrace.cycles-pp.generic_fadvise.ksys_fadvise64_64.__x64_sys_fadvise64.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.52 ± 8% +1.9 8.43 ± 5% perf-profile.calltrace.cycles-pp.invalidate_mapping_pages.generic_fadvise.ksys_fadvise64_64.__x64_sys_fadvise64.do_syscall_64
4.73 ± 8% +2.1 6.82 ± 6% perf-profile.calltrace.cycles-pp.__pagevec_release.invalidate_mapping_pages.generic_fadvise.ksys_fadvise64_64.__x64_sys_fadvise64
4.73 ± 8% +2.1 6.82 ± 6% perf-profile.calltrace.cycles-pp.release_pages.__pagevec_release.invalidate_mapping_pages.generic_fadvise.ksys_fadvise64_64
47.16 ± 3% +2.3 49.51 perf-profile.calltrace.cycles-pp.page_cache_readahead_unbounded.generic_file_buffered_read.new_sync_read.vfs_read.ksys_read
1.88 +3.0 4.91 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
63.88 ± 4% +3.2 67.03 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
63.88 ± 4% +3.2 67.03 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
1.86 +3.2 5.03 ± 11% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node
0.00 +5.0 4.96 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.lock_page_lruvec_irqsave.release_pages.__pagevec_release
0.00 +5.0 5.00 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.lock_page_lruvec_irqsave.release_pages.__pagevec_release.invalidate_mapping_pages
0.00 +5.0 5.00 ± 7% perf-profile.calltrace.cycles-pp.lock_page_lruvec_irqsave.release_pages.__pagevec_release.invalidate_mapping_pages.generic_fadvise
8.93 ± 4% +9.4 18.30 perf-profile.calltrace.cycles-pp.add_to_page_cache_lru.page_cache_readahead_unbounded.generic_file_buffered_read.new_sync_read.vfs_read
6.76 ± 5% +9.9 16.65 ± 2% perf-profile.calltrace.cycles-pp.lru_cache_add.add_to_page_cache_lru.page_cache_readahead_unbounded.generic_file_buffered_read.new_sync_read
6.68 ± 5% +9.9 16.59 ± 2% perf-profile.calltrace.cycles-pp.__pagevec_lru_add.lru_cache_add.add_to_page_cache_lru.page_cache_readahead_unbounded.generic_file_buffered_read
0.00 +15.3 15.27 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.lock_page_lruvec_irqsave.__pagevec_lru_add.lru_cache_add
0.00 +15.3 15.33 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.lock_page_lruvec_irqsave.__pagevec_lru_add.lru_cache_add.add_to_page_cache_lru
0.00 +15.3 15.33 ± 2% perf-profile.calltrace.cycles-pp.lock_page_lruvec_irqsave.__pagevec_lru_add.lru_cache_add.add_to_page_cache_lru.page_cache_readahead_unbounded
12.23 ± 2% -6.5 5.76 ± 8% perf-profile.children.cycles-pp._raw_spin_lock
14.49 ± 4% -5.8 8.70 ± 6% perf-profile.children.cycles-pp.shrink_page_list
23.67 -5.7 17.93 ± 4% perf-profile.children.cycles-pp.__alloc_pages_nodemask
8.61 ± 7% -3.3 5.31 ± 8% perf-profile.children.cycles-pp.btrfs_releasepage
8.42 ± 7% -3.3 5.16 ± 8% perf-profile.children.cycles-pp.try_release_extent_mapping
19.13 ± 3% -3.1 16.05 ± 5% perf-profile.children.cycles-pp.__alloc_pages_slowpath
5.15 ± 6% -3.1 2.08 ± 6% perf-profile.children.cycles-pp.get_page_from_freelist
4.78 ± 6% -3.0 1.83 ± 7% perf-profile.children.cycles-pp.rmqueue
4.40 ± 7% -2.9 1.55 ± 8% perf-profile.children.cycles-pp.rmqueue_bulk
20.22 ± 3% -2.7 17.50 ± 5% perf-profile.children.cycles-pp.shrink_node
20.09 ± 3% -2.7 17.42 ± 5% perf-profile.children.cycles-pp.shrink_lruvec
20.08 ± 3% -2.7 17.41 ± 5% perf-profile.children.cycles-pp.shrink_inactive_list
18.30 ± 4% -2.6 15.70 ± 5% perf-profile.children.cycles-pp.try_to_free_pages
18.29 ± 4% -2.6 15.69 ± 5% perf-profile.children.cycles-pp.do_try_to_free_pages
5.63 -2.2 3.40 ± 5% perf-profile.children.cycles-pp.__remove_mapping
4.42 ± 7% -1.6 2.77 ± 8% perf-profile.children.cycles-pp.__clear_extent_bit
3.96 ± 7% -1.6 2.40 ± 8% perf-profile.children.cycles-pp.test_range_bit
2.45 ± 4% -1.0 1.48 ± 5% perf-profile.children.cycles-pp.free_pcppages_bulk
2.54 ± 4% -1.0 1.58 ± 3% perf-profile.children.cycles-pp.free_unref_page_list
2.15 ± 3% -0.5 1.62 ± 4% perf-profile.children.cycles-pp.__add_to_page_cache_locked
4.13 ± 6% -0.4 3.69 ± 2% perf-profile.children.cycles-pp.kthread
4.14 ± 6% -0.4 3.70 ± 3% perf-profile.children.cycles-pp.ret_from_fork
0.99 ± 20% -0.4 0.59 ± 19% perf-profile.children.cycles-pp.kmem_cache_alloc
0.79 ± 26% -0.4 0.41 ± 27% perf-profile.children.cycles-pp.allocate_slab
0.80 ± 26% -0.4 0.42 ± 27% perf-profile.children.cycles-pp.___slab_alloc
0.80 ± 26% -0.4 0.42 ± 27% perf-profile.children.cycles-pp.__slab_alloc
2.12 ± 7% -0.3 1.81 perf-profile.children.cycles-pp.worker_thread
2.91 -0.3 2.60 ± 7% perf-profile.children.cycles-pp.lru_note_cost
2.02 ± 7% -0.3 1.73 perf-profile.children.cycles-pp.process_one_work
1.66 ± 3% -0.3 1.37 perf-profile.children.cycles-pp.__list_del_entry_valid
0.66 ± 22% -0.2 0.42 ± 19% perf-profile.children.cycles-pp.alloc_extent_state
0.69 ± 19% -0.2 0.46 ± 17% perf-profile.children.cycles-pp.__set_extent_bit
0.62 ± 22% -0.2 0.40 ± 20% perf-profile.children.cycles-pp.lock_extent_bits
0.64 ± 21% -0.2 0.42 ± 19% perf-profile.children.cycles-pp.btrfs_lock_and_flush_ordered_range
1.24 ± 3% -0.2 1.03 ± 3% perf-profile.children.cycles-pp.__delete_from_page_cache
1.59 ± 5% -0.2 1.38 ± 2% perf-profile.children.cycles-pp.pagecache_get_page
1.57 ± 5% -0.2 1.36 ± 2% perf-profile.children.cycles-pp.find_get_entry
0.84 ± 7% -0.2 0.65 ± 2% perf-profile.children.cycles-pp.mem_cgroup_charge
0.51 ± 4% -0.1 0.38 ± 5% perf-profile.children.cycles-pp.__etree_search
0.75 ± 4% -0.1 0.62 ± 2% perf-profile.children.cycles-pp.xas_store
0.23 ± 30% -0.1 0.10 ± 32% perf-profile.children.cycles-pp.xas_nomem
0.39 ± 3% -0.1 0.28 ± 4% perf-profile.children.cycles-pp.xas_create
0.55 ± 5% -0.1 0.46 perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.51 ± 8% -0.1 0.42 ± 3% perf-profile.children.cycles-pp.__mod_memcg_state
0.47 ± 6% -0.1 0.38 ± 2% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.47 ± 5% -0.1 0.39 ± 2% perf-profile.children.cycles-pp.asm_call_on_stack
0.32 ± 3% -0.1 0.24 ± 4% perf-profile.children.cycles-pp.workingset_eviction
0.30 ± 3% -0.1 0.22 ± 3% perf-profile.children.cycles-pp.unlock_page
0.51 ± 5% -0.1 0.43 ± 7% perf-profile.children.cycles-pp.unaccount_page_cache_page
0.39 ± 4% -0.1 0.32 ± 2% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.40 ± 4% -0.1 0.33 ± 4% perf-profile.children.cycles-pp.clear_state_bit
0.44 ± 6% -0.1 0.37 ± 6% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.38 ± 5% -0.1 0.31 ± 2% perf-profile.children.cycles-pp.hrtimer_interrupt
0.18 ± 6% -0.1 0.12 ± 3% perf-profile.children.cycles-pp._raw_write_lock
0.36 ± 6% -0.1 0.30 ± 5% perf-profile.children.cycles-pp.uncharge_batch
0.23 ± 9% -0.1 0.18 ± 2% perf-profile.children.cycles-pp.__count_memcg_events
0.17 ± 11% -0.1 0.12 ± 7% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm
0.31 ± 5% -0.1 0.26 perf-profile.children.cycles-pp.try_charge
0.12 ± 7% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.xas_alloc
0.30 ± 7% -0.0 0.26 ± 5% perf-profile.children.cycles-pp.page_counter_uncharge
0.21 ± 3% -0.0 0.16 ± 5% perf-profile.children.cycles-pp.__mod_node_page_state
0.16 ± 2% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.split_state
0.24 ± 2% -0.0 0.20 perf-profile.children.cycles-pp.xas_init_marks
0.24 ± 5% -0.0 0.20 ± 4% perf-profile.children.cycles-pp.__mod_lruvec_state
0.18 ± 10% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
0.30 ± 6% -0.0 0.26 ± 5% perf-profile.children.cycles-pp.page_counter_cancel
0.26 ± 7% -0.0 0.22 ± 3% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.13 ± 8% -0.0 0.09 ± 7% perf-profile.children.cycles-pp.__softirqentry_text_start
0.19 ± 7% -0.0 0.15 ± 5% perf-profile.children.cycles-pp.kmem_cache_free
0.20 ± 8% -0.0 0.17 ± 6% perf-profile.children.cycles-pp.tick_sched_timer
0.22 ± 3% -0.0 0.18 ± 2% perf-profile.children.cycles-pp.xa_load
0.18 ± 8% -0.0 0.15 ± 6% perf-profile.children.cycles-pp.update_process_times
0.11 ± 9% -0.0 0.08 ± 8% perf-profile.children.cycles-pp.rcu_core
0.18 ± 8% -0.0 0.15 ± 5% perf-profile.children.cycles-pp.tick_sched_handle
0.11 ± 10% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.rcu_do_batch
0.08 ± 10% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.run_ksoftirqd
0.12 ± 8% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.scheduler_tick
0.12 ± 5% -0.0 0.10 ± 5% perf-profile.children.cycles-pp.__mod_zone_page_state
0.16 ± 6% -0.0 0.14 ± 5% perf-profile.children.cycles-pp.page_counter_try_charge
0.08 ± 10% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.smpboot_thread_fn
0.12 ± 5% -0.0 0.10 ± 8% perf-profile.children.cycles-pp.workingset_age_nonresident
0.15 -0.0 0.13 perf-profile.children.cycles-pp.xas_clear_mark
0.12 ± 5% -0.0 0.10 perf-profile.children.cycles-pp.__schedule
0.10 ± 4% -0.0 0.08 perf-profile.children.cycles-pp.schedule
0.08 ± 8% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.task_tick_fair
0.11 ± 4% -0.0 0.09 perf-profile.children.cycles-pp.xas_start
0.08 ± 6% -0.0 0.06 ± 11% perf-profile.children.cycles-pp._cond_resched
0.10 ± 4% -0.0 0.09 perf-profile.children.cycles-pp.free_extent_state
0.06 ± 6% -0.0 0.05 perf-profile.children.cycles-pp.pick_next_task_fair
0.15 ± 3% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.__lookup_extent_mapping
0.06 -0.0 0.05 perf-profile.children.cycles-pp.workingset_update_node
0.14 ± 6% +0.0 0.15 ± 3% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.09 ± 7% +0.0 0.13 ± 6% perf-profile.children.cycles-pp.__list_add_valid
0.50 ± 3% +0.0 0.55 ± 4% perf-profile.children.cycles-pp.isolate_lru_pages
6.53 ± 8% +1.9 8.44 ± 5% perf-profile.children.cycles-pp.__x64_sys_fadvise64
6.53 ± 8% +1.9 8.44 ± 5% perf-profile.children.cycles-pp.ksys_fadvise64_64
6.53 ± 8% +1.9 8.44 ± 5% perf-profile.children.cycles-pp.generic_fadvise
6.52 ± 8% +1.9 8.44 ± 5% perf-profile.children.cycles-pp.invalidate_mapping_pages
4.83 ± 8% +2.1 6.90 ± 6% perf-profile.children.cycles-pp.release_pages
4.73 ± 8% +2.1 6.82 ± 6% perf-profile.children.cycles-pp.__pagevec_release
47.17 ± 3% +2.3 49.51 perf-profile.children.cycles-pp.page_cache_readahead_unbounded
5.29 +3.0 8.26 ± 7% perf-profile.children.cycles-pp._raw_spin_lock_irq
63.99 ± 4% +3.1 67.13 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
63.98 ± 4% +3.1 67.12 perf-profile.children.cycles-pp.do_syscall_64
28.11 ± 2% +7.2 35.36 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
8.94 ± 4% +9.4 18.30 perf-profile.children.cycles-pp.add_to_page_cache_lru
6.76 ± 5% +9.9 16.67 ± 2% perf-profile.children.cycles-pp.lru_cache_add
6.70 ± 5% +9.9 16.64 ± 2% perf-profile.children.cycles-pp.__pagevec_lru_add
11.74 ± 4% +10.6 22.30 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +20.4 20.40 ± 3% perf-profile.children.cycles-pp.lock_page_lruvec_irqsave
1.66 ± 3% -0.3 1.37 perf-profile.self.cycles-pp.__list_del_entry_valid
1.22 ± 8% -0.2 1.03 ± 2% perf-profile.self.cycles-pp.end_bio_extent_readpage
1.32 ± 5% -0.2 1.14 ± 2% perf-profile.self.cycles-pp.find_get_entry
0.49 ± 5% -0.1 0.36 ± 5% perf-profile.self.cycles-pp.__etree_search
0.63 ± 4% -0.1 0.52 perf-profile.self.cycles-pp._raw_spin_lock
0.32 ± 5% -0.1 0.21 ± 2% perf-profile.self.cycles-pp.get_page_from_freelist
0.73 ± 5% -0.1 0.63 perf-profile.self.cycles-pp.free_pcppages_bulk
0.50 ± 9% -0.1 0.42 ± 3% perf-profile.self.cycles-pp.__mod_memcg_state
0.29 ± 2% -0.1 0.22 perf-profile.self.cycles-pp.unlock_page
0.26 ± 3% -0.1 0.19 ± 3% perf-profile.self.cycles-pp.__remove_mapping
0.37 ± 3% -0.1 0.30 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.18 ± 6% -0.1 0.12 ± 3% perf-profile.self.cycles-pp._raw_write_lock
0.20 ± 6% -0.1 0.14 ± 3% perf-profile.self.cycles-pp.workingset_eviction
0.27 -0.1 0.21 ± 3% perf-profile.self.cycles-pp.xas_create
0.23 ± 9% -0.1 0.18 ± 2% perf-profile.self.cycles-pp.__count_memcg_events
0.17 ± 11% -0.1 0.12 ± 7% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm
0.30 ± 6% -0.0 0.25 ± 7% perf-profile.self.cycles-pp.__add_to_page_cache_locked
0.16 ± 4% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.try_release_extent_mapping
0.37 ± 3% -0.0 0.33 ± 2% perf-profile.self.cycles-pp.__do_readpage
0.20 ± 5% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.__mod_node_page_state
0.19 ± 6% -0.0 0.15 ± 2% perf-profile.self.cycles-pp.btrfs_releasepage
0.16 ± 8% -0.0 0.12 ± 7% perf-profile.self.cycles-pp.mem_cgroup_charge
0.12 ± 5% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.split_state
0.26 ± 7% -0.0 0.22 ± 6% perf-profile.self.cycles-pp.page_counter_cancel
0.19 ± 4% -0.0 0.16 ± 10% perf-profile.self.cycles-pp.unaccount_page_cache_page
0.15 ± 2% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.kmem_cache_alloc
0.15 ± 5% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.page_counter_try_charge
0.12 ± 3% -0.0 0.09 ± 7% perf-profile.self.cycles-pp.__mod_zone_page_state
0.12 ± 10% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.crc_36
0.15 ± 3% -0.0 0.12 perf-profile.self.cycles-pp.xas_clear_mark
0.11 ± 6% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.rmqueue_bulk
0.10 ± 4% -0.0 0.07 ± 5% perf-profile.self.cycles-pp.xas_init_marks
0.12 ± 5% -0.0 0.10 ± 8% perf-profile.self.cycles-pp.workingset_age_nonresident
0.10 ± 8% -0.0 0.08 ± 6% perf-profile.self.cycles-pp.__delete_from_page_cache
0.08 -0.0 0.06 ± 6% perf-profile.self.cycles-pp.clear_state_bit
0.15 ± 3% -0.0 0.13 ± 5% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.09 ± 4% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.test_range_bit
0.07 ± 5% -0.0 0.06 perf-profile.self.cycles-pp.pmem_submit_bio
0.15 ± 3% -0.0 0.14 ± 3% perf-profile.self.cycles-pp.xas_store
0.08 ± 8% +0.0 0.12 ± 6% perf-profile.self.cycles-pp.__list_add_valid
0.60 ± 5% +0.1 0.68 ± 2% perf-profile.self.cycles-pp.extent_readahead
0.11 ± 3% +0.1 0.25 ± 2% perf-profile.self.cycles-pp.isolate_lru_pages
0.29 ± 3% +0.4 0.66 perf-profile.self.cycles-pp.__pagevec_lru_add
28.07 ± 2% +7.3 35.34 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
fio.read_bw_MBps
16000 +-------------------------------------------------------------------+
| .+. : +.. + +. |
15500 |.+ +..+. : +.+..+.+.. .. + .. +.. .+.+..+..+. |
| +..+..+ + + +. +..+ |
| |
15000 |-+ |
| |
14500 |-+ |
| |
14000 |-+ O |
| O O O |
| O O O O O O O O O O |
13500 |-+O O O O O O O |
| O O O O |
13000 +-------------------------------------------------------------------+
fio.read_iops
8000 +--------------------------------------------------------------------+
| +. |
7800 |-.+.+..+.. .. +..+.. .+..+.. .+.. .+..+.. .+..+.. .+.. |
|. +.+..+ + + + + + +..+ |
7600 |-+ |
| |
7400 |-+ |
| |
7200 |-+ |
| |
7000 |-+ O |
| O O O O O |
6800 |-+ O O O O O O O O O O |
| O O O O O |
6600 +--------------------------------------------------------------------+
fio.read_clat_mean_us
6.4e+06 +-----------------------------------------------------------------+
6.3e+06 |-+ O |
| O O |
6.2e+06 |-+O O O O O O O O O O |
6.1e+06 |-+ O O O O O O O O O |
6e+06 |-+ O O O |
5.9e+06 |-+ |
| |
5.8e+06 |-+ |
5.7e+06 |-+ |
5.6e+06 |-+ +..+ |
5.5e+06 |.. .+.+.. .+.. .+.. .+.. .+..+.+..+.+.. + |
| +.+..+.+. .+..+ +.+. + + + |
5.4e+06 |-+ + |
5.3e+06 +-----------------------------------------------------------------+
fio.read_clat_90__us
2.05e+07 +----------------------------------------------------------------+
2e+07 |-+ O |
| O O O O O O O O O |
1.95e+07 |-+ O O O O O O |
1.9e+07 |-+ O O O O O O O O |
1.85e+07 |-+ O |
1.8e+07 |-+ |
| |
1.75e+07 |-+ |
1.7e+07 |-+ |
1.65e+07 |-+ |
1.6e+07 |.. .+.+..+.+.. .+.+.. .+. .+. .+.+.. |
| +.+. .+. +.+. +. +. +.+..+. .+.+ |
1.55e+07 |-+ + +. |
1.5e+07 +----------------------------------------------------------------+
fio.latency_500us_
26 +----------------------------------------------------------------------+
25 |-+ O |
| O O O O O O O O O |
24 |-+ O O O O O O O O |
23 |-+ O O O O O O O |
22 |-+ |
21 |-+ |
| |
20 |-+ |
19 |-+ |
18 |-+ .+ .+ |
17 |..+. .+.. .+. + .+.. +..+.+.. .+. |
| +. +..+.+. + .+..+..+..+.+..+..+ .. +. |
16 |-+ + + |
15 +----------------------------------------------------------------------+
fio.latency_10ms_
8 +---------------------------------------------------------------------+
| .+ |
7.5 |-+ +..+..+.+..+. |
7 |..+. .+.. .+..+. .+.+.. .+. .+..+ + |
| +. +.+..+. +..+. +. +. + + |
6.5 |-+ + |
6 |-+ |
| |
5.5 |-+ |
5 |-+ |
| O O O O O |
4.5 |-+ O O O O O O O O O O O O O |
4 |-+ O O O O O |
| O O |
3.5 +---------------------------------------------------------------------+
fio.latency_50ms_
9 +---------------------------------------------------------------------+
| O |
8.5 |-+ O O O |
| O O O O O O O O |
| O O O O O O O O O |
8 |-+ O O O O |
| |
7.5 |-+ |
| |
7 |-+ .+. .+.. .+. |
|..+.+..+. +..+..+..+ +.. .+.. .+. .+. +.. |
| + +. +. +..+..+. .+..+ |
6.5 |-+ +. |
| |
6 +---------------------------------------------------------------------+
fio.workload
1.6e+06 +----------------------------------------------------------------+
| .+. + +.. +.. +.. |
1.55e+06 |.+ +..+.+.. + +.+..+.+.. + + +. .+.+..+.+.. |
| +.+ + + +. +.+ |
| |
1.5e+06 |-+ |
| |
1.45e+06 |-+ |
| |
1.4e+06 |-+ O |
| O O O O |
| O O O O O O O O O |
1.35e+06 |-+O O O O O O O |
| O O O O |
1.3e+06 +----------------------------------------------------------------+
fio.time.file_system_inputs
6.6e+09 +-----------------------------------------------------------------+
| +. |
6.4e+09 |-.+.+..+. .. +.. .+.+.. .+. .+.+.. .+. |
|. +..+.+ +.+. +. +. +.+..+.+. +..+ |
| |
6.2e+09 |-+ |
| |
6e+09 |-+ |
| |
5.8e+09 |-+ |
| O |
| O O O O O O O |
5.6e+09 |-+ O O O O O O O O O |
| O O O O O O O |
5.4e+09 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-skl-fpga01: 104 threads Skylake with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/process/100%/debian-10.4-x86_64-20200603.cgz/lkp-skl-fpga01/page_fault2/will-it-scale/0x2006906
commit:
0c5e8d9feb ("mm/mlock: remove __munlock_isolate_lru_page")
44c86dfabf ("mm/lru: replace pgdat lru_lock with lruvec lock")
0c5e8d9febd9f026 44c86dfabfbbb310b5135c0e8b6
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:2 64% 2:4 perf-profile.calltrace.cycles-pp.sync_regs.error_entry.testcase
2:2 130% 5:4 perf-profile.calltrace.cycles-pp.error_entry.testcase
3:2 136% 5:4 perf-profile.children.cycles-pp.error_entry
1:2 68% 2:4 perf-profile.self.cycles-pp.error_entry
%stddev %change %stddev
\ | \
74710 -11.1% 66448 will-it-scale.per_process_ops
7769969 -11.1% 6910637 will-it-scale.workload
3.64 ± 3% -0.5 3.11 mpstat.cpu.all.usr%
1390 -2.0% 1362 vmstat.system.cs
1.151e+09 -11.0% 1.025e+09 ± 2% numa-numastat.node0.local_node
1.151e+09 -11.0% 1.025e+09 ± 2% numa-numastat.node0.numa_hit
1.198e+09 -11.1% 1.066e+09 numa-numastat.node1.local_node
1.198e+09 -11.1% 1.066e+09 numa-numastat.node1.numa_hit
1670 ± 14% +25.3% 2091 ± 4% slabinfo.dmaengine-unmap-16.active_objs
1670 ± 14% +25.3% 2091 ± 4% slabinfo.dmaengine-unmap-16.num_objs
918.00 +20.4% 1105 ± 6% slabinfo.mnt_cache.active_objs
918.00 +20.4% 1105 ± 6% slabinfo.mnt_cache.num_objs
0.50 ±100% +12550.0% 63.25 ±114% numa-vmstat.node1.nr_active_file
5990 ± 4% -7.2% 5560 ± 2% numa-vmstat.node1.nr_mapped
0.50 ±100% +12550.0% 63.25 ±114% numa-vmstat.node1.nr_zone_active_file
6.007e+08 -11.2% 5.332e+08 numa-vmstat.node1.numa_hit
6.005e+08 -11.2% 5.331e+08 numa-vmstat.node1.numa_local
2.347e+09 -11.1% 2.088e+09 proc-vmstat.numa_hit
2.347e+09 -11.1% 2.088e+09 proc-vmstat.numa_local
2.35e+09 -11.1% 2.09e+09 proc-vmstat.pgalloc_normal
2.34e+09 -11.1% 2.081e+09 proc-vmstat.pgfault
2.349e+09 -11.1% 2.087e+09 proc-vmstat.pgfree
67117 -3.0% 65096 proc-vmstat.pgreuse
165746 ± 16% -44.0% 92831 ± 45% sched_debug.cfs_rq:/.MIN_vruntime.avg
12794315 ± 5% -50.2% 6372286 ± 28% sched_debug.cfs_rq:/.MIN_vruntime.max
1408188 ± 9% -48.8% 720458 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.stddev
80320 ± 4% -31.3% 55186 ± 42% sched_debug.cfs_rq:/.load.stddev
165746 ± 16% -44.0% 92831 ± 45% sched_debug.cfs_rq:/.max_vruntime.avg
12794315 ± 5% -50.2% 6372286 ± 28% sched_debug.cfs_rq:/.max_vruntime.max
1408188 ± 9% -48.8% 720458 ± 33% sched_debug.cfs_rq:/.max_vruntime.stddev
59.32 ± 3% -8.7% 54.14 ± 6% sched_debug.cfs_rq:/.nr_spread_over.stddev
125.03 +13.9% 142.40 ± 4% sched_debug.cfs_rq:/.runnable_avg.stddev
248861 ± 91% +124.5% 558786 ± 3% sched_debug.cpu.avg_idle.min
155314 ± 4% -16.1% 130352 ± 7% sched_debug.cpu.avg_idle.stddev
14.05 ± 23% -19.0% 11.38 ± 25% sched_debug.cpu.clock.stddev
732.03 ± 4% -11.3% 648.96 ± 4% sched_debug.cpu.clock_task.stddev
605543 ± 7% -10.5% 541811 ± 4% sched_debug.cpu.max_idle_balance_cost.max
12349 ± 22% -48.9% 6314 ± 62% sched_debug.cpu.max_idle_balance_cost.stddev
13987 ± 4% +18.9% 16636 ± 6% sched_debug.cpu.nr_switches.max
1596 ± 6% -9.4% 1446 ± 7% sched_debug.cpu.sched_count.stddev
730.51 ± 2% -12.7% 637.48 ± 7% sched_debug.cpu.ttwu_count.stddev
60057 -2.6% 58474 interrupts.CAL:Function_call_interrupts
562.50 ± 8% -32.2% 381.50 ± 8% interrupts.CPU2.RES:Rescheduling_interrupts
318.50 ± 30% -32.3% 215.75 ± 2% interrupts.CPU21.RES:Rescheduling_interrupts
212.50 +49.1% 316.75 ± 47% interrupts.CPU25.RES:Rescheduling_interrupts
1156 ± 43% -45.6% 629.00 ± 25% interrupts.CPU29.CAL:Function_call_interrupts
317.50 ± 18% -19.9% 254.25 ± 9% interrupts.CPU31.RES:Rescheduling_interrupts
258.00 ± 5% -8.7% 235.50 ± 2% interrupts.CPU32.RES:Rescheduling_interrupts
525.00 -9.8% 473.75 interrupts.CPU37.CAL:Function_call_interrupts
435.50 ± 40% -57.9% 183.25 ± 5% interrupts.CPU37.RES:Rescheduling_interrupts
141.00 ± 4% -19.1% 114.00 ± 15% interrupts.CPU45.RES:Rescheduling_interrupts
124.50 ± 17% -31.5% 85.25 ± 11% interrupts.CPU48.RES:Rescheduling_interrupts
826.50 ± 2% -12.7% 721.75 ± 3% interrupts.CPU53.CAL:Function_call_interrupts
859.00 ± 29% -29.5% 606.00 ± 2% interrupts.CPU58.CAL:Function_call_interrupts
763.50 ± 28% -19.1% 617.75 ± 33% interrupts.CPU6.CAL:Function_call_interrupts
791.00 ± 11% -27.9% 570.25 ± 6% interrupts.CPU60.CAL:Function_call_interrupts
77.50 ± 14% +441.9% 420.00 ± 81% interrupts.CPU62.RES:Rescheduling_interrupts
3911 +101.9% 7896 interrupts.CPU63.NMI:Non-maskable_interrupts
3911 +101.9% 7896 interrupts.CPU63.PMI:Performance_monitoring_interrupts
36.50 ± 17% +90.4% 69.50 ± 35% interrupts.CPU65.RES:Rescheduling_interrupts
26.00 ± 11% +749.0% 220.75 ±106% interrupts.CPU68.RES:Rescheduling_interrupts
7859 -37.3% 4927 ± 34% interrupts.CPU70.NMI:Non-maskable_interrupts
7859 -37.3% 4927 ± 34% interrupts.CPU70.PMI:Performance_monitoring_interrupts
7839 -37.2% 4926 ± 34% interrupts.CPU71.NMI:Non-maskable_interrupts
7839 -37.2% 4926 ± 34% interrupts.CPU71.PMI:Performance_monitoring_interrupts
503.50 ± 3% -7.2% 467.25 interrupts.CPU72.CAL:Function_call_interrupts
506.00 ± 3% -6.8% 471.75 interrupts.CPU74.CAL:Function_call_interrupts
799.00 ± 8% -9.3% 724.50 ± 6% interrupts.CPU78.CAL:Function_call_interrupts
1163 ± 10% -34.9% 757.75 ± 8% interrupts.CPU80.CAL:Function_call_interrupts
941.50 ± 14% -25.3% 703.50 ± 10% interrupts.CPU81.CAL:Function_call_interrupts
1109 ± 35% -67.5% 360.50 ± 7% interrupts.CPU81.RES:Rescheduling_interrupts
891.00 ± 18% -28.8% 634.50 ± 4% interrupts.CPU82.CAL:Function_call_interrupts
988.50 -59.2% 403.75 ± 32% interrupts.CPU82.RES:Rescheduling_interrupts
116.50 ± 9% -28.1% 83.75 ± 15% interrupts.CPU88.RES:Rescheduling_interrupts
574.00 ± 5% -14.6% 490.25 ± 3% interrupts.CPU9.CAL:Function_call_interrupts
489.00 ± 33% -50.3% 243.00 ± 8% interrupts.CPU9.RES:Rescheduling_interrupts
473.50 +24.6% 589.75 ± 21% interrupts.CPU91.CAL:Function_call_interrupts
3904 +102.7% 7914 interrupts.CPU95.NMI:Non-maskable_interrupts
3904 +102.7% 7914 interrupts.CPU95.PMI:Performance_monitoring_interrupts
22.50 ± 6% +522.2% 140.00 ±113% interrupts.CPU98.RES:Rescheduling_interrupts
258.00 ± 75% -83.6% 42.25 ± 53% interrupts.CPU99.RES:Rescheduling_interrupts
7.708e+09 -7.7% 7.113e+09 perf-stat.i.branch-instructions
30906896 -8.1% 28391684 perf-stat.i.branch-misses
74.70 -2.4 72.34 perf-stat.i.cache-miss-rate%
6.82e+08 -11.1% 6.063e+08 perf-stat.i.cache-misses
9.092e+08 -8.2% 8.343e+08 perf-stat.i.cache-references
1361 -1.9% 1336 perf-stat.i.context-switches
7.27 +9.4% 7.95 perf-stat.i.cpi
429.82 +12.9% 485.13 perf-stat.i.cycles-between-cache-misses
17892232 ± 2% -13.7% 15448523 ± 3% perf-stat.i.dTLB-load-misses
1.058e+10 -8.9% 9.64e+09 perf-stat.i.dTLB-loads
1.24 -0.0 1.22 perf-stat.i.dTLB-store-miss-rate%
65867958 -12.5% 57633448 perf-stat.i.dTLB-store-misses
5.205e+09 -11.0% 4.633e+09 perf-stat.i.dTLB-stores
15554157 ± 3% -8.4% 14253928 ± 2% perf-stat.i.iTLB-load-misses
3.954e+10 -8.5% 3.618e+10 perf-stat.i.instructions
0.14 -8.1% 0.13 perf-stat.i.ipc
0.74 +8.4% 0.80 perf-stat.i.metric.K/sec
236.67 -9.1% 215.25 perf-stat.i.metric.M/sec
7714008 -10.9% 6876972 perf-stat.i.minor-faults
2920678 -11.5% 2583907 perf-stat.i.node-load-misses
1.514e+08 -10.8% 1.35e+08 perf-stat.i.node-loads
3356543 -11.1% 2983135 perf-stat.i.node-store-misses
44021143 -11.7% 38892081 ± 2% perf-stat.i.node-stores
7714009 -10.9% 6876973 perf-stat.i.page-faults
0.40 -0.0 0.39 perf-stat.overall.branch-miss-rate%
75.04 -2.3 72.71 perf-stat.overall.cache-miss-rate%
7.29 +9.5% 7.99 perf-stat.overall.cpi
422.48 +12.7% 475.92 perf-stat.overall.cycles-between-cache-misses
1.25 -0.0 1.23 perf-stat.overall.dTLB-store-miss-rate%
0.14 -8.7% 0.13 perf-stat.overall.ipc
1534009 +2.6% 1573577 perf-stat.overall.path-length
7.659e+09 -7.9% 7.057e+09 perf-stat.ps.branch-instructions
30443566 -8.9% 27747536 perf-stat.ps.branch-misses
6.784e+08 -11.2% 6.026e+08 perf-stat.ps.cache-misses
9.041e+08 -8.3% 8.289e+08 perf-stat.ps.cache-references
1346 -2.1% 1318 perf-stat.ps.context-switches
104.41 -1.4% 102.98 perf-stat.ps.cpu-migrations
17825946 ± 2% -13.5% 15414423 ± 3% perf-stat.ps.dTLB-load-misses
1.052e+10 -9.0% 9.567e+09 perf-stat.ps.dTLB-loads
65493569 -12.6% 57235079 perf-stat.ps.dTLB-store-misses
5.173e+09 -11.1% 4.599e+09 perf-stat.ps.dTLB-stores
15462920 ± 3% -8.4% 14157175 ± 2% perf-stat.ps.iTLB-load-misses
3.929e+10 -8.6% 3.589e+10 perf-stat.ps.instructions
0.81 ± 2% -5.8% 0.76 ± 5% perf-stat.ps.major-faults
7671341 -10.9% 6832656 perf-stat.ps.minor-faults
2898356 -11.6% 2563559 perf-stat.ps.node-load-misses
1.508e+08 -10.8% 1.345e+08 perf-stat.ps.node-loads
3337117 -11.2% 2963007 perf-stat.ps.node-store-misses
43687678 -11.7% 38554571 ± 2% perf-stat.ps.node-stores
7671342 -10.9% 6832657 perf-stat.ps.page-faults
1.192e+13 -8.8% 1.087e+13 perf-stat.total.instructions
45.97 -46.0 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__pagevec_lru_add.lru_cache_add.alloc_set_pte.finish_fault
45.92 -45.9 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__pagevec_lru_add.lru_cache_add.alloc_set_pte
20.05 -20.1 0.00 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.zap_pte_range.unmap_page_range
20.04 -20.0 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.release_pages.tlb_flush_mmu.zap_pte_range
3.42 -0.8 2.62 perf-profile.calltrace.cycles-pp.alloc_pages_vma.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
3.16 -0.8 2.36 perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.do_fault.__handle_mm_fault.handle_mm_fault
2.93 -0.8 2.15 perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_fault.__handle_mm_fault
2.76 -0.8 2.00 perf-profile.calltrace.cycles-pp.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.do_fault
2.42 -0.7 1.68 ± 2% perf-profile.calltrace.cycles-pp.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma
1.56 -0.7 0.89 ± 4% perf-profile.calltrace.cycles-pp._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask
1.55 -0.7 0.88 ± 4% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist
2.39 -0.7 1.73 ± 3% perf-profile.calltrace.cycles-pp.free_unref_page_list.release_pages.tlb_flush_mmu.zap_pte_range.unmap_page_range
2.19 -0.6 1.56 ± 3% perf-profile.calltrace.cycles-pp.free_pcppages_bulk.free_unref_page_list.release_pages.tlb_flush_mmu.zap_pte_range
6.82 -0.6 6.25 perf-profile.calltrace.cycles-pp.copy_page.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
1.31 -0.5 0.79 ± 6% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages
1.31 -0.5 0.80 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages.tlb_flush_mmu
1.62 -0.3 1.37 perf-profile.calltrace.cycles-pp.mem_cgroup_charge.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
1.29 ± 2% -0.1 1.19 perf-profile.calltrace.cycles-pp._raw_spin_lock.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault
1.50 -0.1 1.43 perf-profile.calltrace.cycles-pp.irqentry_exit_to_user_mode.asm_exc_page_fault.testcase
0.67 ± 2% -0.1 0.60 ± 5% perf-profile.calltrace.cycles-pp.page_add_new_anon_rmap.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault
0.73 -0.0 0.68 perf-profile.calltrace.cycles-pp.__list_del_entry_valid.rmqueue_bulk.rmqueue.get_page_from_freelist.__alloc_pages_nodemask
0.77 -0.0 0.73 perf-profile.calltrace.cycles-pp.__irqentry_text_end.testcase
2.97 +0.1 3.02 perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap
2.97 +0.1 3.02 perf-profile.calltrace.cycles-pp.tlb_finish_mmu.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
2.95 +0.1 3.00 perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region.__do_munmap
1.19 +0.1 1.33 perf-profile.calltrace.cycles-pp.__do_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.99 +0.1 1.13 perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_fault.__do_fault.do_fault.__handle_mm_fault
1.13 +0.1 1.27 perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.do_fault.__handle_mm_fault.handle_mm_fault
0.89 +0.2 1.04 perf-profile.calltrace.cycles-pp.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault.do_fault
0.69 +0.2 0.84 perf-profile.calltrace.cycles-pp.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault
64.82 +0.2 65.00 perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault.testcase
64.71 +0.2 64.90 perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.testcase
64.06 +0.2 64.28 perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.testcase
23.55 +0.3 23.84 perf-profile.calltrace.cycles-pp.tlb_flush_mmu.zap_pte_range.unmap_page_range.unmap_vmas.unmap_region
63.25 +0.3 63.57 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
62.92 +0.3 63.24 perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
23.37 +0.3 23.69 perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.zap_pte_range.unmap_page_range.unmap_vmas
49.48 +1.9 51.34 perf-profile.calltrace.cycles-pp.finish_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
49.45 +1.9 51.31 perf-profile.calltrace.cycles-pp.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault.handle_mm_fault
47.22 +2.0 49.24 perf-profile.calltrace.cycles-pp.__pagevec_lru_add.lru_cache_add.alloc_set_pte.finish_fault.do_fault
47.31 +2.0 49.34 perf-profile.calltrace.cycles-pp.lru_cache_add.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault
0.00 +2.7 2.67 perf-profile.calltrace.cycles-pp.lock_page_lruvec_irqsave.release_pages.tlb_flush_mmu.tlb_finish_mmu.unmap_region
0.00 +2.7 2.67 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.lock_page_lruvec_irqsave.release_pages.tlb_flush_mmu.tlb_finish_mmu
0.00 +21.1 21.15 perf-profile.calltrace.cycles-pp.lock_page_lruvec_irqsave.release_pages.tlb_flush_mmu.zap_pte_range.unmap_page_range
0.00 +21.1 21.15 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.lock_page_lruvec_irqsave.release_pages.tlb_flush_mmu.zap_pte_range
0.00 +23.8 23.81 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.lock_page_lruvec_irqsave.release_pages.tlb_flush_mmu
0.00 +47.8 47.75 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.lock_page_lruvec_irqsave.__pagevec_lru_add.lru_cache_add
0.00 +47.8 47.79 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.lock_page_lruvec_irqsave.__pagevec_lru_add.lru_cache_add.alloc_set_pte
0.00 +47.8 47.80 perf-profile.calltrace.cycles-pp.lock_page_lruvec_irqsave.__pagevec_lru_add.lru_cache_add.alloc_set_pte.finish_fault
4.37 -1.4 3.00 ± 3% perf-profile.children.cycles-pp._raw_spin_lock
3.45 -0.8 2.63 perf-profile.children.cycles-pp.alloc_pages_vma
3.21 -0.8 2.40 perf-profile.children.cycles-pp.__alloc_pages_nodemask
2.96 -0.8 2.19 perf-profile.children.cycles-pp.get_page_from_freelist
2.79 -0.8 2.02 perf-profile.children.cycles-pp.rmqueue
2.74 -0.8 1.97 ± 2% perf-profile.children.cycles-pp.free_unref_page_list
2.50 -0.7 1.77 ± 3% perf-profile.children.cycles-pp.free_pcppages_bulk
2.42 -0.7 1.70 ± 2% perf-profile.children.cycles-pp.rmqueue_bulk
6.84 -0.6 6.26 perf-profile.children.cycles-pp.copy_page
1.62 -0.2 1.38 perf-profile.children.cycles-pp.mem_cgroup_charge
1.14 ± 4% -0.1 1.01 ± 4% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
71.75 -0.1 71.63 perf-profile.children.cycles-pp.testcase
0.82 ± 3% -0.1 0.70 perf-profile.children.cycles-pp.__mod_memcg_state
0.57 -0.1 0.46 ± 4% perf-profile.children.cycles-pp.page_remove_rmap
0.56 -0.1 0.44 ± 4% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm
0.64 ± 3% -0.1 0.53 ± 3% perf-profile.children.cycles-pp.__count_memcg_events
1.21 -0.1 1.12 perf-profile.children.cycles-pp.__list_del_entry_valid
1.52 -0.1 1.44 perf-profile.children.cycles-pp.irqentry_exit_to_user_mode
0.67 ± 2% -0.1 0.60 ± 6% perf-profile.children.cycles-pp.page_add_new_anon_rmap
0.28 ± 3% -0.0 0.24 perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
0.13 -0.0 0.09 ± 7% perf-profile.children.cycles-pp.lock_page_memcg
0.77 -0.0 0.73 perf-profile.children.cycles-pp.__irqentry_text_end
0.29 -0.0 0.26 perf-profile.children.cycles-pp.__mod_lruvec_state
0.24 -0.0 0.20 ± 2% perf-profile.children.cycles-pp.__mod_node_page_state
0.96 -0.0 0.92 perf-profile.children.cycles-pp.native_irq_return_iret
0.72 -0.0 0.69 perf-profile.children.cycles-pp.sync_regs
0.40 -0.0 0.37 ± 2% perf-profile.children.cycles-pp.__perf_sw_event
0.29 -0.0 0.27 ± 3% perf-profile.children.cycles-pp.___perf_sw_event
0.18 -0.0 0.15 ± 3% perf-profile.children.cycles-pp.free_pages_and_swap_cache
0.15 ± 3% -0.0 0.14 ± 7% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.27 -0.0 0.25 ± 2% perf-profile.children.cycles-pp.xas_load
0.12 ± 4% -0.0 0.11 perf-profile.children.cycles-pp.__mod_zone_page_state
0.07 -0.0 0.06 perf-profile.children.cycles-pp.find_vma
2.99 +0.1 3.04 perf-profile.children.cycles-pp.tlb_finish_mmu
0.11 ± 9% +0.1 0.20 ± 6% perf-profile.children.cycles-pp.__list_add_valid
1.19 +0.1 1.33 perf-profile.children.cycles-pp.__do_fault
1.13 +0.1 1.27 perf-profile.children.cycles-pp.shmem_fault
0.90 +0.2 1.05 perf-profile.children.cycles-pp.find_lock_entry
1.00 +0.2 1.16 perf-profile.children.cycles-pp.shmem_getpage_gfp
0.69 +0.2 0.85 perf-profile.children.cycles-pp.find_get_entry
64.84 +0.2 65.02 perf-profile.children.cycles-pp.exc_page_fault
64.73 +0.2 64.92 perf-profile.children.cycles-pp.do_user_addr_fault
64.08 +0.2 64.30 perf-profile.children.cycles-pp.handle_mm_fault
63.27 +0.3 63.58 perf-profile.children.cycles-pp.__handle_mm_fault
62.94 +0.3 63.26 perf-profile.children.cycles-pp.do_fault
26.53 +0.4 26.88 perf-profile.children.cycles-pp.tlb_flush_mmu
26.45 +0.4 26.82 perf-profile.children.cycles-pp.release_pages
49.50 +1.8 51.34 perf-profile.children.cycles-pp.finish_fault
49.47 +1.9 51.32 perf-profile.children.cycles-pp.alloc_set_pte
71.62 +1.9 73.48 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
47.34 +2.0 49.36 perf-profile.children.cycles-pp.lru_cache_add
47.28 +2.0 49.32 perf-profile.children.cycles-pp.__pagevec_lru_add
68.60 +3.1 71.70 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +71.7 71.70 perf-profile.children.cycles-pp.lock_page_lruvec_irqsave
6.81 -0.6 6.25 perf-profile.self.cycles-pp.copy_page
0.94 -0.1 0.79 ± 3% perf-profile.self.cycles-pp.zap_pte_range
0.81 ± 4% -0.1 0.70 perf-profile.self.cycles-pp.__mod_memcg_state
0.56 -0.1 0.44 ± 3% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm
0.78 -0.1 0.67 perf-profile.self.cycles-pp.free_pcppages_bulk
0.63 ± 3% -0.1 0.53 ± 3% perf-profile.self.cycles-pp.__count_memcg_events
0.40 ± 2% -0.1 0.30 ± 2% perf-profile.self.cycles-pp.mem_cgroup_charge
1.30 ± 2% -0.1 1.21 perf-profile.self.cycles-pp._raw_spin_lock
1.21 -0.1 1.12 perf-profile.self.cycles-pp.__list_del_entry_valid
2.11 -0.1 2.03 perf-profile.self.cycles-pp.testcase
1.46 -0.1 1.40 perf-profile.self.cycles-pp.irqentry_exit_to_user_mode
0.96 -0.0 0.92 perf-profile.self.cycles-pp.native_irq_return_iret
0.77 -0.0 0.73 perf-profile.self.cycles-pp.__irqentry_text_end
0.23 -0.0 0.20 ± 2% perf-profile.self.cycles-pp.__mod_node_page_state
0.12 -0.0 0.09 ± 4% perf-profile.self.cycles-pp.lock_page_memcg
0.33 -0.0 0.29 perf-profile.self.cycles-pp.release_pages
0.20 ± 2% -0.0 0.17 ± 6% perf-profile.self.cycles-pp.page_remove_rmap
0.71 -0.0 0.69 perf-profile.self.cycles-pp.sync_regs
0.22 -0.0 0.19 ± 2% perf-profile.self.cycles-pp.rmqueue
0.17 ± 2% -0.0 0.15 ± 4% perf-profile.self.cycles-pp.free_pages_and_swap_cache
0.12 ± 4% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.__mod_zone_page_state
0.12 ± 8% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.page_add_new_anon_rmap
0.24 -0.0 0.22 perf-profile.self.cycles-pp.___perf_sw_event
0.21 -0.0 0.19 ± 4% perf-profile.self.cycles-pp.do_fault
0.19 -0.0 0.17 ± 2% perf-profile.self.cycles-pp.try_charge
0.33 -0.0 0.31 perf-profile.self.cycles-pp.__handle_mm_fault
0.10 ± 10% +0.1 0.19 ± 6% perf-profile.self.cycles-pp.__list_add_valid
0.43 +0.1 0.57 ± 6% perf-profile.self.cycles-pp.__pagevec_lru_add
0.42 +0.2 0.59 perf-profile.self.cycles-pp.find_get_entry
71.62 +1.9 73.48 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 10 months
[core/entry] b6ec413461: will-it-scale.per_process_ops 5.0% improvement
by kernel test robot
Greeting,
FYI, we noticed a 5.0% improvement of will-it-scale.per_process_ops due to commit:
commit: b6ec413461034d49f9e586845825adb35ba308f6 ("core/entry: Report syscall correctly for trace and audit")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: will-it-scale
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:
nr_task: 16
mode: process
test: lseek2
cpufreq_governor: performance
ucode: 0x5002f01
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
In addition to that, the commit also has significant impact on the following tests:
+------------------+---------------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_process_ops 3.2% improvement |
| test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters | cpufreq_governor=performance |
| | mode=process |
| | nr_task=16 |
| | test=eventfd1 |
| | ucode=0x5002f01 |
+------------------+---------------------------------------------------------------------------+
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/process/16/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap2/lseek2/will-it-scale/0x5002f01
commit:
v5.9-rc5
b6ec413461 ("core/entry: Report syscall correctly for trace and audit")
v5.9-rc5 b6ec413461034d49f9e58684582
---------------- ---------------------------
%stddev %change %stddev
\ | \
9885233 +5.0% 10378311 will-it-scale.per_process_ops
1.582e+08 +5.0% 1.661e+08 will-it-scale.workload
24202 ± 4% +8.4% 26235 ± 4% cpuidle.C1.usage
14616 ± 20% +18.7% 17353 ± 8% sched_debug.cfs_rq:/.load.avg
235.73 ± 31% -28.8% 167.73 ± 8% sched_debug.cfs_rq:/.load_avg.max
17260 ± 31% +44.7% 24968 ± 18% softirqs.CPU10.RCU
14774 ± 48% +62.3% 23974 ± 22% softirqs.NET_RX
17491 ± 6% +16.9% 20449 ± 6% meminfo.Active
17487 ± 6% +16.9% 20445 ± 6% meminfo.Active(anon)
37041 ± 3% +8.2% 40071 ± 3% meminfo.Shmem
148143 ± 74% -93.9% 9079 ± 6% numa-numastat.node0.local_node
160532 ± 62% -75.1% 40045 numa-numastat.node0.numa_hit
12401 ±122% +149.7% 30966 numa-numastat.node0.other_node
2079 ± 61% -94.0% 125.00 ± 52% numa-meminfo.node0.Active
2078 ± 61% -94.0% 125.00 ± 52% numa-meminfo.node0.Active(anon)
79197 ± 18% -28.5% 56601 ± 15% numa-meminfo.node1.SUnreclaim
13420 ± 7% +25.0% 16775 ± 16% numa-meminfo.node3.Active
13420 ± 7% +25.0% 16774 ± 16% numa-meminfo.node3.Active(anon)
519.40 ± 61% -94.1% 30.50 ± 54% numa-vmstat.node0.nr_active_anon
519.40 ± 61% -94.1% 30.50 ± 54% numa-vmstat.node0.nr_zone_active_anon
29263 ±127% +188.4% 84386 ± 35% numa-vmstat.node0.numa_other
19799 ± 18% -28.5% 14150 ± 15% numa-vmstat.node1.nr_slab_unreclaimable
3349 ± 7% +25.0% 4186 ± 16% numa-vmstat.node3.nr_active_anon
3349 ± 7% +25.0% 4186 ± 16% numa-vmstat.node3.nr_zone_active_anon
4372 ± 6% +16.9% 5109 ± 6% proc-vmstat.nr_active_anon
9269 ± 3% +8.2% 10026 ± 3% proc-vmstat.nr_shmem
31421 -2.4% 30681 proc-vmstat.nr_slab_reclaimable
72340 -3.8% 69626 proc-vmstat.nr_slab_unreclaimable
4372 ± 6% +16.9% 5109 ± 6% proc-vmstat.nr_zone_active_anon
4838 ± 8% +24.9% 6045 ± 9% proc-vmstat.pgactivate
8152 ± 6% -16.4% 6812 ± 6% slabinfo.Acpi-State.active_objs
8152 ± 6% -16.4% 6812 ± 6% slabinfo.Acpi-State.num_objs
6952 ± 3% -18.4% 5674 ± 7% slabinfo.files_cache.active_objs
6952 ± 3% -18.4% 5674 ± 7% slabinfo.files_cache.num_objs
5555 ± 4% -12.6% 4854 ± 6% slabinfo.mm_struct.active_objs
5555 ± 4% -12.6% 4854 ± 6% slabinfo.mm_struct.num_objs
3792 ± 3% -11.5% 3357 ± 2% slabinfo.sighand_cache.active_objs
3865 ± 3% -10.6% 3454 slabinfo.sighand_cache.num_objs
6037 ± 2% -11.9% 5318 ± 4% slabinfo.signal_cache.num_objs
9626 ± 3% -14.6% 8218 ± 4% slabinfo.task_delay_info.active_objs
9626 ± 3% -14.6% 8218 ± 4% slabinfo.task_delay_info.num_objs
817.40 +80.0% 1471 ± 64% interrupts.CPU1.CAL:Function_call_interrupts
2575 ± 22% +94.1% 4999 ± 24% interrupts.CPU1.NMI:Non-maskable_interrupts
2575 ± 22% +94.1% 4999 ± 24% interrupts.CPU1.PMI:Performance_monitoring_interrupts
3166 ± 19% +95.3% 6184 ± 42% interrupts.CPU10.NMI:Non-maskable_interrupts
3166 ± 19% +95.3% 6184 ± 42% interrupts.CPU10.PMI:Performance_monitoring_interrupts
203.00 ± 97% -83.1% 34.25 ± 61% interrupts.CPU112.RES:Rescheduling_interrupts
244.60 ±193% -99.0% 2.50 ± 87% interrupts.CPU147.RES:Rescheduling_interrupts
49.80 ± 86% -95.0% 2.50 ±173% interrupts.CPU20.RES:Rescheduling_interrupts
824.20 ± 4% +32.6% 1092 ± 32% interrupts.CPU49.CAL:Function_call_interrupts
191937 ± 9% +22.9% 235809 ± 7% interrupts.NMI:Non-maskable_interrupts
191937 ± 9% +22.9% 235809 ± 7% interrupts.PMI:Performance_monitoring_interrupts
6526 ± 36% -38.0% 4047 ± 22% interrupts.RES:Rescheduling_interrupts
1.025e+10 +4.9% 1.076e+10 perf-stat.i.branch-instructions
1.13 -3.9% 1.09 perf-stat.i.cpi
1.54e+10 +4.5% 1.609e+10 perf-stat.i.dTLB-loads
1.016e+10 +4.9% 1.066e+10 perf-stat.i.dTLB-stores
1.616e+08 +1.7% 1.644e+08 perf-stat.i.iTLB-load-misses
5.044e+10 +4.9% 5.292e+10 perf-stat.i.instructions
313.59 +3.1% 323.35 perf-stat.i.instructions-per-iTLB-miss
0.88 +4.1% 0.92 perf-stat.i.ipc
186.80 +4.7% 195.64 perf-stat.i.metric.M/sec
1.13 -3.9% 1.09 perf-stat.overall.cpi
312.10 +3.2% 321.96 perf-stat.overall.instructions-per-iTLB-miss
0.88 +4.1% 0.92 perf-stat.overall.ipc
1.022e+10 +4.9% 1.072e+10 perf-stat.ps.branch-instructions
1.535e+10 +4.5% 1.604e+10 perf-stat.ps.dTLB-loads
1.012e+10 +4.9% 1.062e+10 perf-stat.ps.dTLB-stores
1.611e+08 +1.7% 1.638e+08 perf-stat.ps.iTLB-load-misses
5.027e+10 +4.9% 5.275e+10 perf-stat.ps.instructions
1.517e+13 +5.0% 1.592e+13 perf-stat.total.instructions
33.08 ± 6% -4.7 28.41 ± 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.llseek
24.83 ± 7% -4.4 20.40 ± 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.llseek
4.24 ± 7% -1.2 3.08 ± 5% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
30.98 ± 14% +6.4 37.39 ± 12% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
29.97 ± 15% +6.8 36.78 ± 12% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
27.84 ± 16% +8.0 35.88 ± 15% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
33.38 ± 6% -4.7 28.66 ± 9% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
24.90 ± 6% -4.2 20.66 ± 9% perf-profile.children.cycles-pp.do_syscall_64
2.49 ± 6% -2.1 0.40 ± 11% perf-profile.children.cycles-pp.__x64_sys_lseek
3.56 ± 7% -0.8 2.77 ± 10% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.21 ± 7% -0.0 0.18 ± 10% perf-profile.children.cycles-pp.rebalance_domains
0.31 ± 6% -0.0 0.28 ± 3% [email protected]
0.67 ± 7% +0.1 0.82 ± 9% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
32.34 ± 14% +7.0 39.30 ± 14% perf-profile.children.cycles-pp.cpuidle_enter_state
32.35 ± 14% +7.0 39.31 ± 14% perf-profile.children.cycles-pp.cpuidle_enter
27.83 ± 16% +8.0 35.88 ± 15% perf-profile.children.cycles-pp.intel_idle
2.26 ± 6% -2.1 0.21 ± 13% perf-profile.self.cycles-pp.__x64_sys_lseek
3.02 ± 6% -0.9 2.09 ± 9% perf-profile.self.cycles-pp.ksys_lseek
1.69 ± 6% -0.7 1.02 ± 9% perf-profile.self.cycles-pp.do_syscall_64
0.11 ± 9% -0.0 0.09 ± 7% perf-profile.self.cycles-pp.read_tsc
27.83 ± 16% +8.0 35.88 ± 15% perf-profile.self.cycles-pp.intel_idle
will-it-scale.per_process_ops
1.04e+07 +----------------------------------------------------------------+
| O O O O O O O O O O |
1.03e+07 |-+ |
| |
| |
1.02e+07 |-+ |
| |
1.01e+07 |-+ |
| |
1e+07 |-+ |
| |
| +. +. |
9.9e+06 |..+.+..+.. .. +.. .. +..+.. .+.+..+..+.+..+..+. .+..+..+.+..|
| + + +. +. |
9.8e+06 +----------------------------------------------------------------+
will-it-scale.workload
1.67e+08 +----------------------------------------------------------------+
1.66e+08 |-+O O O O O O O O O O O O O |
| O |
1.65e+08 |-+ |
1.64e+08 |-+ |
| |
1.63e+08 |-+ |
1.62e+08 |-+ |
1.61e+08 |-+ |
| |
1.6e+08 |-+ |
1.59e+08 |-+ |
|..+.+..+.. .+.+.. .+. .+.. .+. .+..+.+..+..+. .+..+..+.+..|
1.58e+08 |-+ +. +. +. +. +. +. |
1.57e+08 +----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-csl-2ap2: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/process/16/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap2/eventfd1/will-it-scale/0x5002f01
commit:
v5.9-rc5
b6ec413461 ("core/entry: Report syscall correctly for trace and audit")
v5.9-rc5 b6ec413461034d49f9e58684582
---------------- ---------------------------
%stddev %change %stddev
\ | \
2798280 +3.2% 2887533 will-it-scale.per_process_ops
44772485 +3.2% 46200540 will-it-scale.workload
87.60 ± 24% +82.1% 159.50 ± 28% numa-vmstat.node3.nr_page_table_pages
20657 ± 22% +28.0% 26436 ± 15% sched_debug.cfs_rq:/.min_vruntime.min
2161 +2.3% 2210 vmstat.system.cs
191992 ±105% +165.2% 509166 ± 44% numa-numastat.node1.local_node
216868 ± 88% +142.0% 524722 ± 40% numa-numastat.node1.numa_hit
7105 ± 11% +17.0% 8311 ± 13% slabinfo.eventpoll_pwq.active_objs
7105 ± 11% +17.0% 8311 ± 13% slabinfo.eventpoll_pwq.num_objs
1.43 ± 9% +0.3 1.72 ± 8% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.82 ± 12% -0.7 0.12 ± 12% perf-profile.children.cycles-pp.__x64_sys_read
0.77 ± 10% -0.7 0.12 ± 7% perf-profile.children.cycles-pp.__x64_sys_write
0.52 ± 10% -0.1 0.41 ± 8% perf-profile.children.cycles-pp.apparmor_file_permission
0.47 ± 10% +0.1 0.61 ± 6% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.77 ± 11% -0.7 0.07 ± 21% perf-profile.self.cycles-pp.__x64_sys_read
0.71 ± 10% -0.7 0.06 ± 14% perf-profile.self.cycles-pp.__x64_sys_write
0.76 ± 9% -0.3 0.48 ± 6% perf-profile.self.cycles-pp.do_syscall_64
0.84 ± 10% -0.2 0.61 ± 11% perf-profile.self.cycles-pp.ksys_write
0.84 ± 11% -0.2 0.64 ± 9% perf-profile.self.cycles-pp.ksys_read
0.41 ± 11% -0.1 0.30 ± 8% perf-profile.self.cycles-pp.apparmor_file_permission
0.58 ± 89% -48.3% 0.30 ± 3% perf-stat.i.MPKI
1.506e+10 +3.1% 1.553e+10 perf-stat.i.branch-instructions
2117 +2.3% 2165 perf-stat.i.context-switches
0.74 -3.8% 0.71 perf-stat.i.cpi
0.00 ±185% -0.0 0.00 ± 11% perf-stat.i.dTLB-load-miss-rate%
699002 ±185% -94.3% 40038 ± 12% perf-stat.i.dTLB-load-misses
2.279e+10 +2.9% 2.346e+10 perf-stat.i.dTLB-loads
1.478e+10 +3.2% 1.525e+10 perf-stat.i.dTLB-stores
1.767e+08 ± 2% -2.6% 1.722e+08 perf-stat.i.iTLB-load-misses
3908623 ± 14% +8.4% 4235861 perf-stat.i.iTLB-loads
7.678e+10 +3.1% 7.919e+10 perf-stat.i.instructions
436.22 ± 2% +5.7% 461.03 perf-stat.i.instructions-per-iTLB-miss
1.35 +3.9% 1.40 perf-stat.i.ipc
274.35 +3.0% 282.62 perf-stat.i.metric.M/sec
0.58 ± 90% -48.6% 0.30 ± 3% perf-stat.overall.MPKI
0.74 -3.8% 0.71 perf-stat.overall.cpi
0.00 ±185% -0.0 0.00 ± 12% perf-stat.overall.dTLB-load-miss-rate%
434.74 ± 2% +5.8% 459.94 perf-stat.overall.instructions-per-iTLB-miss
1.35 +3.9% 1.40 perf-stat.overall.ipc
1.501e+10 +3.1% 1.548e+10 perf-stat.ps.branch-instructions
2110 +2.3% 2158 perf-stat.ps.context-switches
696387 ±185% -94.3% 39917 ± 12% perf-stat.ps.dTLB-load-misses
2.271e+10 +2.9% 2.338e+10 perf-stat.ps.dTLB-loads
1.473e+10 +3.2% 1.519e+10 perf-stat.ps.dTLB-stores
1.761e+08 ± 2% -2.6% 1.716e+08 perf-stat.ps.iTLB-load-misses
3895475 ± 14% +8.4% 4221532 perf-stat.ps.iTLB-loads
7.652e+10 +3.1% 7.892e+10 perf-stat.ps.instructions
2.31e+13 +3.1% 2.382e+13 perf-stat.total.instructions
12126 ± 28% +39.2% 16874 ± 34% softirqs.CPU100.RCU
11840 ± 31% +55.4% 18401 ± 39% softirqs.CPU102.RCU
10572 ± 11% +55.6% 16448 ± 35% softirqs.CPU104.RCU
11684 ± 29% +40.2% 16381 ± 34% softirqs.CPU106.RCU
11734 ± 31% +45.6% 17086 ± 36% softirqs.CPU108.RCU
10703 ± 7% +51.0% 16164 ± 30% softirqs.CPU11.RCU
13037 ± 30% +25.5% 16363 ± 35% softirqs.CPU110.RCU
9291 ± 6% +9.0% 10126 ± 6% softirqs.CPU115.RCU
9701 ± 9% +10.9% 10755 softirqs.CPU121.RCU
9730 ± 7% +9.9% 10689 ± 3% softirqs.CPU122.RCU
11788 ± 28% +42.5% 16796 ± 32% softirqs.CPU13.RCU
10971 ± 10% +18.5% 12996 ± 11% softirqs.CPU156.RCU
10974 ± 9% +10.8% 12158 ± 3% softirqs.CPU160.RCU
10794 ± 13% +14.6% 12365 ± 3% softirqs.CPU162.RCU
11211 ± 9% +10.2% 12359 ± 3% softirqs.CPU163.RCU
11176 ± 7% +9.5% 12240 ± 3% softirqs.CPU165.RCU
11092 ± 8% +9.7% 12168 ± 3% softirqs.CPU167.RCU
10702 ± 8% +10.7% 11842 ± 4% softirqs.CPU168.RCU
10387 ± 10% +13.2% 11756 ± 3% softirqs.CPU169.RCU
10337 ± 9% +12.9% 11672 ± 3% softirqs.CPU170.RCU
10579 ± 9% +12.6% 11911 ± 5% softirqs.CPU171.RCU
10539 ± 9% +13.9% 12000 ± 7% softirqs.CPU172.RCU
10516 ± 9% +10.2% 11592 ± 4% softirqs.CPU173.RCU
10491 ± 10% +11.3% 11682 ± 5% softirqs.CPU174.RCU
10410 ± 10% +11.7% 11628 ± 4% softirqs.CPU175.RCU
10714 ± 9% +11.8% 11973 ± 3% softirqs.CPU181.RCU
10813 ± 10% +13.2% 12240 ± 7% softirqs.CPU189.RCU
10340 ± 6% +14.0% 11788 ± 3% softirqs.CPU22.RCU
10412 ± 6% +16.3% 12106 ± 3% softirqs.CPU23.RCU
10211 ± 8% +11.2% 11360 ± 4% softirqs.CPU25.RCU
10031 ± 8% +10.7% 11107 ± 2% softirqs.CPU26.RCU
11991 ± 9% +7.6% 12904 ± 3% softirqs.CPU34.RCU
11613 ± 9% +9.5% 12712 ± 3% softirqs.CPU38.RCU
11600 ±117% +91.6% 22232 ± 76% softirqs.CPU4.SCHED
11729 ± 8% +10.0% 12899 ± 3% softirqs.CPU40.RCU
11808 ± 9% +11.9% 13211 ± 4% softirqs.CPU45.RCU
14100 ± 33% +18.9% 16772 ± 30% softirqs.CPU5.RCU
11311 ± 9% +11.4% 12604 ± 2% softirqs.CPU64.RCU
11492 ± 9% +11.8% 12848 ± 5% softirqs.CPU65.RCU
11340 ± 9% +12.3% 12734 ± 4% softirqs.CPU71.RCU
10991 ± 10% +12.5% 12364 ± 5% softirqs.CPU73.RCU
10941 ± 8% +11.5% 12195 ± 4% softirqs.CPU74.RCU
11032 ± 9% +11.3% 12274 ± 5% softirqs.CPU75.RCU
11103 ± 9% +8.4% 12033 ± 4% softirqs.CPU76.RCU
10933 ± 9% +9.8% 12000 ± 4% softirqs.CPU77.RCU
10915 ± 9% +11.5% 12173 ± 5% softirqs.CPU78.RCU
10888 ± 8% +9.8% 11957 ± 4% softirqs.CPU79.RCU
17681 ± 3% +15.6% 20433 ± 8% softirqs.CPU96.RCU
17146 ± 22% +27.1% 21799 ± 10% softirqs.CPU97.RCU
10.60 ± 95% +725.5% 87.50 ±138% interrupts.CPU120.RES:Rescheduling_interrupts
1135 ± 36% -29.0% 806.25 interrupts.CPU121.CAL:Function_call_interrupts
810.20 +52.3% 1234 ± 44% interrupts.CPU127.CAL:Function_call_interrupts
4.40 ± 54% +621.6% 31.75 ±133% interrupts.CPU146.TLB:TLB_shootdowns
139.40 ± 15% -31.1% 96.00 ± 23% interrupts.CPU147.NMI:Non-maskable_interrupts
139.40 ± 15% -31.1% 96.00 ± 23% interrupts.CPU147.PMI:Performance_monitoring_interrupts
4.00 ± 54% +806.2% 36.25 ±142% interrupts.CPU149.TLB:TLB_shootdowns
133.60 ± 8% -38.2% 82.50 ± 32% interrupts.CPU165.NMI:Non-maskable_interrupts
133.60 ± 8% -38.2% 82.50 ± 32% interrupts.CPU165.PMI:Performance_monitoring_interrupts
133.20 ± 9% -38.1% 82.50 ± 33% interrupts.CPU166.NMI:Non-maskable_interrupts
133.20 ± 9% -38.1% 82.50 ± 33% interrupts.CPU166.PMI:Performance_monitoring_interrupts
155.40 ± 33% -46.6% 83.00 ± 33% interrupts.CPU167.NMI:Non-maskable_interrupts
155.40 ± 33% -46.6% 83.00 ± 33% interrupts.CPU167.PMI:Performance_monitoring_interrupts
134.20 ± 11% -36.1% 85.75 ± 34% interrupts.CPU168.NMI:Non-maskable_interrupts
134.20 ± 11% -36.1% 85.75 ± 34% interrupts.CPU168.PMI:Performance_monitoring_interrupts
160.40 ± 37% -46.7% 85.50 ± 31% interrupts.CPU169.NMI:Non-maskable_interrupts
160.40 ± 37% -46.7% 85.50 ± 31% interrupts.CPU169.PMI:Performance_monitoring_interrupts
133.00 ± 11% -36.7% 84.25 ± 34% interrupts.CPU170.NMI:Non-maskable_interrupts
133.00 ± 11% -36.7% 84.25 ± 34% interrupts.CPU170.PMI:Performance_monitoring_interrupts
138.60 ± 20% -48.8% 71.00 ± 39% interrupts.CPU171.NMI:Non-maskable_interrupts
138.60 ± 20% -48.8% 71.00 ± 39% interrupts.CPU171.PMI:Performance_monitoring_interrupts
132.20 ± 11% -35.9% 84.75 ± 35% interrupts.CPU172.NMI:Non-maskable_interrupts
132.20 ± 11% -35.9% 84.75 ± 35% interrupts.CPU172.PMI:Performance_monitoring_interrupts
20.60 ±145% +637.9% 152.00 ±124% interrupts.CPU18.RES:Rescheduling_interrupts
144.00 ± 9% -34.5% 94.25 ± 31% interrupts.CPU191.NMI:Non-maskable_interrupts
144.00 ± 9% -34.5% 94.25 ± 31% interrupts.CPU191.PMI:Performance_monitoring_interrupts
83.80 ±167% -99.1% 0.75 ±173% interrupts.CPU23.RES:Rescheduling_interrupts
806.40 +8.8% 877.50 ± 6% interrupts.CPU28.CAL:Function_call_interrupts
805.60 +205.7% 2463 ± 97% interrupts.CPU31.CAL:Function_call_interrupts
1.60 ±108% +4181.2% 68.50 ±123% interrupts.CPU31.RES:Rescheduling_interrupts
816.00 +29.5% 1056 ± 21% interrupts.CPU39.CAL:Function_call_interrupts
804.00 +16.6% 937.25 ± 15% interrupts.CPU42.CAL:Function_call_interrupts
803.00 +16.3% 934.25 ± 12% interrupts.CPU43.CAL:Function_call_interrupts
803.00 +231.5% 2662 ± 99% interrupts.CPU44.CAL:Function_call_interrupts
803.60 +25.1% 1005 ± 19% interrupts.CPU45.CAL:Function_call_interrupts
133.40 ± 11% -37.2% 83.75 ± 35% interrupts.CPU48.NMI:Non-maskable_interrupts
133.40 ± 11% -37.2% 83.75 ± 35% interrupts.CPU48.PMI:Performance_monitoring_interrupts
130.60 ± 10% -37.4% 81.75 ± 35% interrupts.CPU50.NMI:Non-maskable_interrupts
130.60 ± 10% -37.4% 81.75 ± 35% interrupts.CPU50.PMI:Performance_monitoring_interrupts
131.80 ± 10% -27.7% 95.25 ± 24% interrupts.CPU51.NMI:Non-maskable_interrupts
131.80 ± 10% -27.7% 95.25 ± 24% interrupts.CPU51.PMI:Performance_monitoring_interrupts
726.40 ± 20% +33.8% 971.75 ± 16% interrupts.CPU73.CAL:Function_call_interrupts
129.00 ± 11% -35.1% 83.75 ± 35% interrupts.CPU89.NMI:Non-maskable_interrupts
129.00 ± 11% -35.1% 83.75 ± 35% interrupts.CPU89.PMI:Performance_monitoring_interrupts
847.20 ± 10% +76.8% 1498 ± 46% interrupts.CPU9.CAL:Function_call_interrupts
129.80 ± 11% -25.3% 97.00 ± 23% interrupts.CPU90.NMI:Non-maskable_interrupts
129.80 ± 11% -25.3% 97.00 ± 23% interrupts.CPU90.PMI:Performance_monitoring_interrupts
128.40 ± 11% -35.4% 83.00 ± 35% interrupts.CPU91.NMI:Non-maskable_interrupts
128.40 ± 11% -35.4% 83.00 ± 35% interrupts.CPU91.PMI:Performance_monitoring_interrupts
128.80 ± 12% -25.3% 96.25 ± 23% interrupts.CPU92.NMI:Non-maskable_interrupts
128.80 ± 12% -25.3% 96.25 ± 23% interrupts.CPU92.PMI:Performance_monitoring_interrupts
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 10 months
[seqlock] 1909760f5f: WARNING:inconsistent_lock_state
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 1909760f5fc3f123e47b4e24e0ccdc0fc8f3f106 ("seqlock: PREEMPT_RT: Do not starve seqlock_t writers")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: kernel-selftests
version: kernel-selftests-x86_64-e8e8f16e-1_20200807
with following parameters:
group: kselftests-net
ucode: 0x21
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 8G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+---------------------------------------+------------+------------+
| | 8117ab508f | 1909760f5f |
+---------------------------------------+------------+------------+
| boot_successes | 0 | 0 |
| boot_failures | 0 | 27 |
| dmesg.WARNING:inconsistent_lock_state | 0 | 27 |
+---------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
kern :warn : [ 309.808218] WARNING: inconsistent lock state
kern :warn : [ 309.813923] 5.9.0-rc2-00048-g1909760f5fc3f #1 Not tainted
kern :warn : [ 309.820815] --------------------------------
kern :warn : [ 309.826576] inconsistent {IN-SOFTIRQ-R} -> {SOFTIRQ-ON-W} usage.
kern :warn : [ 309.834089] ip/16990 [HC0[0]:SC0[0]:HE1:SE1] takes:
kern :warn : [ 309.840492] ffff88803171b290 (&s->seqcount#10){+.+-}-{0:0}, at: xfrm_user_rcv_msg+0x135/0x1e0
kern :warn : [ 309.850616] {IN-SOFTIRQ-R} state was registered at:
kern :warn : [ 309.857021] lock_acquire+0xaf/0x380
kern :warn : [ 309.862214] xfrm_policy_lookup_inexact_addr+0x101/0x120
kern :warn : [ 309.869157] xfrm_policy_find_inexact_candidates+0x50/0xc0
kern :warn : [ 309.876895] xfrm_policy_lookup_bytype+0x48a/0x760
kern :warn : [ 309.883326] xfrm_lookup_with_ifid+0x212/0xc20
kern :warn : [ 309.889390] mld_sendpack+0x187/0x3a0
kern :warn : [ 309.894657] mld_ifc_timer_expire+0x1bb/0x360
kern :warn : [ 309.900603] call_timer_fn+0xa6/0x2e0
kern :warn : [ 309.905848] run_timer_softirq+0x40c/0x5c0
kern :warn : [ 309.911556] __do_softirq+0xea/0x4eb
kern :warn : [ 309.916720] asm_call_on_stack+0x12/0x20
kern :warn : [ 309.922222] do_softirq_own_stack+0x78/0xa0
kern :warn : [ 309.927976] irq_exit_rcu+0xcf/0x120
kern :warn : [ 309.933110] sysvec_apic_timer_interrupt+0x58/0xe0
kern :warn : [ 309.939505] asm_sysvec_apic_timer_interrupt+0x12/0x20
kern :warn : [ 309.946271] cpuidle_enter_state+0x110/0x4e0
kern :warn : [ 309.952154] cpuidle_enter+0x29/0x40
kern :warn : [ 309.957344] do_idle+0x201/0x280
kern :warn : [ 309.962181] cpu_startup_entry+0x19/0x20
kern :warn : [ 309.967725] start_secondary+0x120/0x160
kern :warn : [ 309.973257] secondary_startup_64+0xb6/0xc0
kern :warn : [ 309.979048] irq event stamp: 5701
kern :warn : [ 309.983904] hardirqs last enabled at (5701): [<ffffffff81d96b8a>] syscall_enter_from_user_mode+0x2a/0x2a0
kern :warn : [ 309.995231] hardirqs last disabled at (5700): [<ffffffff81d916cf>] do_syscall_64+0xf/0x40
kern :warn : [ 310.005065] softirqs last enabled at (5688): [<ffffffff81b8b030>] netlink_insert+0x50/0x120
kern :warn : [ 310.015146] softirqs last disabled at (5686): [<ffffffff81ae0cd9>] release_sock+0x19/0xa0
kern :warn : [ 310.024945]
other info that might help us debug this:
kern :warn : [ 310.034639] Possible unsafe locking scenario:
kern :warn : [ 310.043674] CPU0
kern :warn : [ 310.047673] ----
kern :warn : [ 310.051631] lock(&s->seqcount#10);
kern :warn : [ 310.056738] <Interrupt>
kern :warn : [ 310.060875] lock(&s->seqcount#10);
kern :warn : [ 310.066130]
*** DEADLOCK ***
kern :warn : [ 310.076439] 2 locks held by ip/16990:
kern :warn : [ 310.081591] #0: ffff88803171b628 (&net->xfrm.xfrm_cfg_mutex){+.+.}-{3:3}, at: xfrm_netlink_rcv+0x24/0x40
kern :warn : [ 310.092781] #1: ffff88803171b2d8 (&(&net->xfrm.policy_hthresh.lock)->lock){+.+.}-{2:2}, at: xfrm_set_spdinfo+0xab/0x160
kern :warn : [ 310.105327]
stack backtrace:
kern :warn : [ 310.112731] CPU: 2 PID: 16990 Comm: ip Not tainted 5.9.0-rc2-00048-g1909760f5fc3f #1
kern :warn : [ 310.122097] Hardware name: Hewlett-Packard HP Pro 3340 MT/17A1, BIOS 8.07 01/24/2013
kern :warn : [ 310.131472] Call Trace:
kern :warn : [ 310.135475] dump_stack+0x8d/0xc0
kern :warn : [ 310.140377] mark_lock_irq+0x4f2/0x500
kern :warn : [ 310.145730] ? stack_trace_save+0x4b/0x80
kern :warn : [ 310.151298] ? save_trace+0x42/0x360
kern :warn : [ 310.156420] mark_lock+0x1c1/0x500
kern :warn : [ 310.161338] __lock_acquire+0x393/0xb80
kern :warn : [ 310.166684] lock_acquire+0xaf/0x380
kern :warn : [ 310.171732] ? xfrm_user_rcv_msg+0x135/0x1e0
kern :warn : [ 310.177509] xfrm_set_spdinfo+0xd4/0x160
kern :warn : [ 310.182918] ? xfrm_user_rcv_msg+0x135/0x1e0
kern :warn : [ 310.188671] xfrm_user_rcv_msg+0x135/0x1e0
kern :warn : [ 310.194248] ? __mutex_lock+0x21d/0x9e0
kern :warn : [ 310.199553] ? preempt_count_sub+0xa1/0xe0
kern :warn : [ 310.205144] ? __mutex_lock+0x21d/0x9e0
kern :warn : [ 310.210457] ? lock_acquire+0xaf/0x380
kern :warn : [ 310.215655] ? netlink_deliver_tap+0x93/0x3e0
kern :warn : [ 310.221468] ? xfrm_netlink_rcv+0x24/0x40
kern :warn : [ 310.226901] ? xfrm_user_state_lookup+0xc0/0xc0
kern :warn : [ 310.233909] netlink_rcv_skb+0x49/0x120
kern :warn : [ 310.239169] xfrm_netlink_rcv+0x33/0x40
kern :warn : [ 310.244425] netlink_unicast+0x191/0x240
kern :warn : [ 310.249721] netlink_sendmsg+0x24f/0x480
kern :warn : [ 310.255048] sock_sendmsg+0x5e/0x60
kern :warn : [ 310.259910] ____sys_sendmsg+0x1f6/0x260
kern :warn : [ 310.265207] ? copy_msghdr_from_user+0x5c/0xa0
kern :warn : [ 310.271035] ___sys_sendmsg+0x81/0xc0
kern :warn : [ 310.276081] ? lock_acquire+0xaf/0x380
kern :warn : [ 310.281224] ? find_held_lock+0x2b/0x80
kern :warn : [ 310.286438] ? __might_fault+0x36/0x80
kern :warn : [ 310.291566] ? _copy_to_user+0x69/0x80
kern :warn : [ 310.296684] ? move_addr_to_user+0xc3/0x100
kern :warn : [ 310.302249] __sys_sendmsg+0x59/0xa0
kern :warn : [ 310.307188] ? lockdep_hardirqs_on+0x7d/0x100
kern :warn : [ 310.312911] do_syscall_64+0x33/0x40
kern :warn : [ 310.317857] entry_SYSCALL_64_after_hwframe+0x44/0xa9
kern :warn : [ 310.324292] RIP: 0033:0x7fc920c9e914
kern :warn : [ 310.329272] Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b5 0f 1f 80 00 00 00 00 48 8d 05 e9 5d 0c 00 8b 00 85 c0 75 13 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 41 89 d4 55 48 89 f5 53
kern :warn : [ 310.351209] RSP: 002b:00007fff0c088ff8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
kern :warn : [ 310.360381] RAX: ffffffffffffffda RBX: 000000005f6a1c6c RCX: 00007fc920c9e914
kern :warn : [ 310.369104] RDX: 0000000000000000 RSI: 00007fff0c089060 RDI: 0000000000000006
kern :warn : [ 310.377830] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000006
kern :warn : [ 310.386538] R10: fffffffffffff1f9 R11: 0000000000000246 R12: 0000000000000001
kern :warn : [ 310.395277] R13: 00007fff0c089140 R14: 00007fff0c08b436 R15: 00007fff0c08b445
user :notice: [ 332.480919] # PASS: ping to .254 bypassed ipsec tunnel (exceptions and block policies after hresh changes)
user :notice: [ 332.549293] # PASS: direct policy matches (exceptions and block policies after hresh changes)
user :notice: [ 332.617915] # PASS: policy matches (exceptions and block policies after hresh changes)
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Thanks,
lkp
1 year, 10 months
Re: [mm/debug_vm_pgtable/locks] e2aad6f1d2: BUG:unable_to_handle_page_fault_for_address
by Anshuman Khandual
On 09/22/2020 02:50 PM, Aneesh Kumar K.V wrote:
> On 9/22/20 2:22 PM, Anshuman Khandual wrote:
>>
>>
>> On 09/22/2020 09:33 AM, Aneesh Kumar K.V wrote:
>>> On 9/21/20 2:51 PM, kernel test robot wrote:
>>>> Greeting,
>>>>
>>>> FYI, we noticed the following commit (built with gcc-9):
>>>>
>>>> commit: e2aad6f1d232b457ea6a3194992dd4c0a83534a5 ("mm/debug_vm_pgtable/locks: take correct page table lock")
>>>> https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
>>>>
>>>>
>>>> in testcase: trinity
>>>> version: trinity-i386
>>>> with following parameters:
>>>>
>>>> ����runtime: 300s
>>>>
>>>> test-description: Trinity is a linux system call fuzz tester.
>>>> test-url: http://codemonkey.org.uk/projects/trinity/
>>>>
>>>>
>>>> on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
>>>>
>>>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
>>>>
>>>>
>>>> +----------------------------------------------------------------------+------------+------------+
>>>> |��������������������������������������������������������������������� | c50eb1ed65 | e2aad6f1d2 |
>>>> +----------------------------------------------------------------------+------------+------------+
>>>> | boot_successes������������������������������������������������������ | 0��������� | 0��������� |
>>>> | boot_failures������������������������������������������������������� | 61�������� | 17�������� |
>>>> | BUG:workqueue_lockup-pool������������������������������������������� | 1��������� |����������� |
>>>> | BUG:sleeping_function_called_from_invalid_context_at_mm/page_alloc.c | 60�������� | 17�������� |
>>>> | BUG:unable_to_handle_page_fault_for_address������������������������� | 0��������� | 17�������� |
>>>> | Oops:#[##]���������������������������������������������������������� | 0��������� | 17�������� |
>>>> | EIP:ptep_get�������������������������������������������������������� | 0��������� | 17�������� |
>>>> | Kernel_panic-not_syncing:Fatal_exception���������������������������� | 0��������� | 17�������� |
>>>> +----------------------------------------------------------------------+------------+------------+
>>>>
>>>>
>>>> If you fix the issue, kindly add following tag
>>>> Reported-by: kernel test robot <rong.a.chen(a)intel.com>
>>>>
>>>>
>>>> [�� 28.726464] BUG: sleeping function called from invalid context at mm/page_alloc.c:4822
>>>> [�� 28.727835] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1, name: swapper
>>>> [�� 28.729221] no locks held by swapper/1.
>>>> [�� 28.729954] CPU: 0 PID: 1 Comm: swapper Not tainted 5.9.0-rc3-00324-ge2aad6f1d232b4 #1
>>>> [�� 28.731484] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
>>>> [�� 28.732891] Call Trace:
>>>> [�� 28.733295]� ? show_stack+0x48/0x50
>>>> [�� 28.733943]� dump_stack+0x1b/0x1d
>>>> [�� 28.734569]� ___might_sleep+0x205/0x219
>>>> [�� 28.735292]� __might_sleep+0x106/0x10f
>>>> [�� 28.736022]� __alloc_pages_nodemask+0xe0/0x2c8
>>>> [�� 28.736845]� swap_migration_tests+0x62/0x295
>>>> [�� 28.737639]� debug_vm_pgtable+0x587/0x9b5
>>>> [�� 28.738374]� ? pte_advanced_tests+0x267/0x267
>>>> [�� 28.739318]� do_one_initcall+0x129/0x31c
>>>> [�� 28.740023]� ? rcu_read_lock_sched_held+0x46/0x74
>>>> [�� 28.740944]� kernel_init_freeable+0x201/0x250
>>>> [�� 28.741763]� ? rest_init+0xf8/0xf8
>>>> [�� 28.742401]� kernel_init+0xe/0x15d
>>>> [�� 28.743040]� ? rest_init+0xf8/0xf8
>>>> [�� 28.743694]� ret_from_fork+0x1c/0x30
>>>
>>>
>>> This should be fixed by
>>> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commi...
>>>
>>>> [�� 28.744364] BUG: unable to handle page fault for address: fffbbea4
>>>> [�� 28.745465] #PF: supervisor read access in kernel mode
>>>> [�� 28.746373] #PF: error_code(0x0000) - not-present page
>>>> [�� 28.747275] *pde = 0492b067 *pte = 00000000
>>>> [�� 28.748054] Oops: 0000 [#1]
>>>> [�� 28.748548] CPU: 0 PID: 1 Comm: swapper Tainted: G������� W�������� 5.9.0-rc3-00324-ge2aad6f1d232b4 #1
>>>> [�� 28.750188] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
>>>> [�� 28.751641] EIP: ptep_get+0x0/0x3
>>>> [�� 28.752226] Code: 5d fc c9 c3 55 c1 e8 1a 89 e5 53 31 db 83 f8 1f 6a 00 0f 94 c3 b8 80 67 02 c4 31 c9 89 da e8 16 5c f1 ff 89 d8 8b 5d fc c9 c3 <8b> 00 c3 55 31 c9 89 e5 57 56 53 8b 70 04 89 c3 b8 10 68 02 c4 6a
>>>> [�� 28.755465] EAX: fffbbea4 EBX: fffbbea4 ECX: 000047bd EDX: fffbbea4
>>>> [�� 28.756418] ESI: 000047bd EDI: 00000025 EBP: f406bed8 ESP: f406bebc
>>>> [�� 28.757522] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00010286
>>>> [�� 28.758739] CR0: 80050033 CR2: fffbbea4 CR3: 04928000 CR4: 000406d0
>>>> [�� 28.759828] Call Trace:
>>>> [�� 28.760235]� ? hugetlb_advanced_tests+0x2a/0x27f
>>>> [�� 28.761099]� ? do_raw_spin_unlock+0xd7/0x112
>>>> [�� 28.761872]� debug_vm_pgtable+0x927/0x9b5
>>>> [�� 28.762578]� ? pte_advanced_tests+0x267/0x267
>>>> [�� 28.763462]� do_one_initcall+0x129/0x31c
>>>> [�� 28.764134]� ? rcu_read_lock_sched_held+0x46/0x74
>>>> [�� 28.764948]� kernel_init_freeable+0x201/0x250
>>>> [�� 28.765654]� ? rest_init+0xf8/0xf8
>>>> [�� 28.766277]� kernel_init+0xe/0x15d
>>>> [�� 28.766878]� ? rest_init+0xf8/0xf8
>>>> [�� 28.767488]� ret_from_fork+0x1c/0x30
>>>> [�� 28.768052] Modules linked in:
>>>> [�� 28.768532] CR2: 00000000fffbbea4
>>>> [�� 28.769065] ---[ end trace 9c4395cf49c7b3e7 ]---
>>>>
>>>
>>> IIUC, Anshuman is reworking the test to follow the page table update rules.
>>
>> The failure here has been attributed to this particular patch, and
>> IIRC we did not see this particular failure before this series.
>>
>
> IIUC that crash is because the usage of pte_t pointer. With highmem that address is unmapped earlier with the series as expected. But hugetlb_advanced_test is wrongly using that pointer instead of using a pte_t * that should be allocated by huge_pte_alloc().
Hence, now it does not really make sense to keep hugetlb_advanced_test()
test and create this regression on i386. Will probably send a patch to
drop this test altogether for now.
1 year, 10 months
[x86/copy_mc] a0ac629ebe: fio.read_iops -43.3% regression
by kernel test robot
Greeting,
FYI, we noticed a -43.3% regression of fio.read_iops due to commit:
commit: a0ac629ebe7b3d248cb93807782a00d9142fdb98 ("x86/copy_mc: Introduce copy_mc_generic()")
url: https://github.com/0day-ci/linux/commits/Dan-Williams/Renovate-memcpy_mcs...
in testcase: fio-basic
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
with following parameters:
disk: 2pmem
fs: xfs
mount_option: dax
runtime: 200s
nr_task: 50%
time_based: tb
rw: read
bs: 2M
ioengine: libaio
test_size: 200G
cpufreq_governor: performance
ucode: 0x5002f01
test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio
In addition to that, the commit also has significant impact on the following tests:
+------------------+----------------------------------------------------------------------+
| testcase: change | fio-basic: fio.read_iops -55.6% regression |
| test machine | 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory |
| test parameters | bs=2M |
| | cpufreq_governor=performance |
| | disk=2pmem |
| | fs=xfs |
| | ioengine=sync |
| | mount_option=dax |
| | nr_task=50% |
| | runtime=200s |
| | rw=read |
| | test_size=200G |
| | time_based=tb |
| | ucode=0x5002f01 |
+------------------+----------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/mount_option/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
2M/gcc-9/performance/2pmem/xfs/libaio/x86_64-rhel-8.3/dax/50%/debian-10.4-x86_64-20200603.cgz/200s/read/lkp-csl-2sp6/200G/fio-basic/tb/0x5002f01
commit:
7476b91d4d ("x86, powerpc: Rename memcpy_mcsafe() to copy_mc_to_{user, kernel}()")
a0ac629ebe ("x86/copy_mc: Introduce copy_mc_generic()")
7476b91d4db369d8 a0ac629ebe7b3d248cb93807782
---------------- ---------------------------
%stddev %change %stddev
\ | \
97.22 -96.0 1.19 ± 21% fio.latency_100ms%
0.14 -0.1 0.05 fio.latency_10ms%
0.27 ± 13% -0.1 0.14 fio.latency_20ms%
0.04 ± 6% -0.0 0.03 ± 12% fio.latency_20us%
1.00 ± 28% +96.6 97.57 fio.latency_250ms%
0.05 -0.0 0.05 fio.latency_4ms%
0.02 ± 48% +0.3 0.31 ± 15% fio.latency_500ms%
1.25 ± 47% -0.6 0.63 ± 11% fio.latency_50ms%
0.01 ± 9% +0.0 0.02 ± 24% fio.latency_50us%
44292 -43.3% 25124 fio.read_bw_MBps
67895296 +76.8% 1.201e+08 fio.read_clat_90%_us
68681728 +76.7% 1.214e+08 fio.read_clat_95%_us
98304000 ± 19% +80.3% 1.772e+08 ± 4% fio.read_clat_99%_us
66674508 +76.2% 1.175e+08 fio.read_clat_mean_us
9950116 ± 12% +80.3% 17935634 fio.read_clat_stddev
22146 -43.3% 12562 fio.read_iops
2152824 +76.8% 3805428 fio.read_slat_mean_us
291719 ± 14% +86.6% 544324 fio.read_slat_stddev
12923 -2.5% 12594 fio.time.involuntary_context_switches
77.65 ± 3% -39.1% 47.29 fio.time.user_time
4429275 -43.3% 2512537 fio.workload
0.14 ± 3% +0.0 0.16 ± 4% mpstat.cpu.all.soft%
0.47 ± 3% -0.2 0.31 mpstat.cpu.all.usr%
53185 ± 91% +121.2% 117642 ± 40% numa-vmstat.node0.numa_other
122640 ± 39% -52.6% 58092 ± 81% numa-vmstat.node1.numa_other
60096 +1.5% 61021 proc-vmstat.nr_slab_unreclaimable
20103 ± 5% -17.9% 16495 ± 12% proc-vmstat.pgactivate
49.00 -2.0% 48.00 vmstat.cpu.id
1612 -1.6% 1587 vmstat.system.cs
2713 ± 4% +8.0% 2931 ± 4% slabinfo.PING.active_objs
2713 ± 4% +8.0% 2931 ± 4% slabinfo.PING.num_objs
1164 ± 9% +16.8% 1360 ± 6% slabinfo.task_group.active_objs
1164 ± 9% +16.8% 1360 ± 6% slabinfo.task_group.num_objs
379.25 ± 85% +279.7% 1439 ± 75% sched_debug.cfs_rq:/.exec_clock.min
29948 ± 5% -15.5% 25309 ± 5% sched_debug.cfs_rq:/.exec_clock.stddev
21606 ± 7% +25.1% 27034 ± 7% sched_debug.cfs_rq:/.min_vruntime.min
33321 ± 6% -16.5% 27820 ± 6% sched_debug.cfs_rq:/.min_vruntime.stddev
13783 ±109% +184.1% 39158 ± 20% sched_debug.cfs_rq:/.spread0.avg
-38497 -76.6% -9012 sched_debug.cfs_rq:/.spread0.min
33321 ± 6% -16.5% 27820 ± 6% sched_debug.cfs_rq:/.spread0.stddev
12.22 ± 10% +27.9% 15.62 ± 3% sched_debug.cpu.clock.stddev
3716 ±173% -100.0% 1.50 ± 57% softirqs.CPU10.NET_RX
17411 ± 36% -41.8% 10126 ± 19% softirqs.CPU24.SCHED
9179 ± 67% +87.1% 17173 ± 23% softirqs.CPU35.SCHED
9611 ± 34% -58.9% 3951 ± 10% softirqs.CPU48.SCHED
17177 ± 30% -42.6% 9864 ± 37% softirqs.CPU69.SCHED
86644 ± 29% -22.3% 67339 ± 5% softirqs.CPU76.TIMER
6339 ± 66% +115.9% 13686 ± 31% softirqs.CPU78.SCHED
10156 ± 64% +91.8% 19477 ± 25% softirqs.CPU81.SCHED
1239 ±172% -100.0% 0.00 interrupts.62:PCI-MSI.31981595-edge.i40e-eth0-TxRx-26
47482 +5.4% 50055 ± 4% interrupts.CAL:Function_call_interrupts
209.00 ± 23% -50.4% 103.75 ± 8% interrupts.CPU0.RES:Rescheduling_interrupts
146.25 ± 16% -27.4% 106.25 ± 16% interrupts.CPU15.RES:Rescheduling_interrupts
168.75 ± 81% -64.6% 59.75 ± 33% interrupts.CPU15.TLB:TLB_shootdowns
7321 ± 5% -52.7% 3461 ± 39% interrupts.CPU20.NMI:Non-maskable_interrupts
7321 ± 5% -52.7% 3461 ± 39% interrupts.CPU20.PMI:Performance_monitoring_interrupts
6665 ± 14% -61.2% 2586 ± 26% interrupts.CPU21.NMI:Non-maskable_interrupts
6665 ± 14% -61.2% 2586 ± 26% interrupts.CPU21.PMI:Performance_monitoring_interrupts
64.50 ± 23% +41.9% 91.50 ± 22% interrupts.CPU21.TLB:TLB_shootdowns
100.00 ± 41% +66.0% 166.00 ± 9% interrupts.CPU24.RES:Rescheduling_interrupts
1238 ±173% -100.0% 0.00 interrupts.CPU26.62:PCI-MSI.31981595-edge.i40e-eth0-TxRx-26
438.25 ± 4% +16.1% 509.00 ± 18% interrupts.CPU28.CAL:Function_call_interrupts
145.50 ± 20% -34.4% 95.50 ± 25% interrupts.CPU35.RES:Rescheduling_interrupts
7134 ± 11% -28.3% 5118 ± 19% interrupts.CPU41.NMI:Non-maskable_interrupts
7134 ± 11% -28.3% 5118 ± 19% interrupts.CPU41.PMI:Performance_monitoring_interrupts
107.75 ± 34% -47.3% 56.75 ± 40% interrupts.CPU93.RES:Rescheduling_interrupts
63.18 ± 12% -26.1 37.12 ± 15% perf-profile.calltrace.cycles-pp.copy_mc_fragile.copy_mc_to_user.copyout_mc._copy_mc_to_iter.dax_iomap_actor
0.00 +3.7 3.72 ± 52% perf-profile.calltrace.cycles-pp.copy_mc_generic.copy_mc_to_user.copyout_mc._copy_mc_to_iter.dax_iomap_actor
0.00 +37.8 37.83 ± 12% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.copy_mc_generic.copy_mc_to_user.copyout_mc._copy_mc_to_iter
63.34 ± 12% -26.2 37.14 ± 15% perf-profile.children.cycles-pp.copy_mc_fragile
2.41 ±112% -2.2 0.25 ±108% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
2.26 ±109% -2.0 0.29 ± 89% perf-profile.children.cycles-pp.asm_call_on_stack
2.15 ±112% -1.9 0.23 ±110% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
2.12 ±113% -1.9 0.23 ±110% perf-profile.children.cycles-pp.hrtimer_interrupt
1.68 ±114% -1.5 0.17 ±119% perf-profile.children.cycles-pp.__hrtimer_run_queues
1.48 ±123% -1.3 0.15 ±121% perf-profile.children.cycles-pp.tick_sched_timer
1.34 ±120% -1.2 0.14 ±122% perf-profile.children.cycles-pp.tick_sched_handle
1.28 ±119% -1.1 0.14 ±122% perf-profile.children.cycles-pp.update_process_times
0.70 ±107% -0.6 0.10 ±120% perf-profile.children.cycles-pp.scheduler_tick
2.65 ±106% +16.5 19.13 ± 12% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.00 +22.6 22.58 ± 7% perf-profile.children.cycles-pp.copy_mc_generic
62.52 ± 12% -25.5 37.00 ± 15% perf-profile.self.cycles-pp.copy_mc_fragile
0.00 +22.4 22.41 ± 6% perf-profile.self.cycles-pp.copy_mc_generic
42.43 +68.7% 71.58 perf-stat.i.MPKI
5.949e+09 -42.2% 3.44e+09 perf-stat.i.branch-instructions
0.07 +0.0 0.10 ± 5% perf-stat.i.branch-miss-rate%
3554006 ± 2% -7.5% 3286479 ± 3% perf-stat.i.branch-misses
95.02 -2.4 92.63 perf-stat.i.cache-miss-rate%
1.444e+09 -5.2% 1.369e+09 perf-stat.i.cache-misses
1.513e+09 -2.8% 1.471e+09 perf-stat.i.cache-references
3.81 +72.5% 6.58 perf-stat.i.cpi
102.49 +4.5% 107.13 perf-stat.i.cycles-between-cache-misses
0.00 ± 4% +0.0 0.00 ± 41% perf-stat.i.dTLB-load-miss-rate%
6.03e+09 -42.0% 3.495e+09 perf-stat.i.dTLB-loads
0.00 ± 5% +0.0 0.00 ± 7% perf-stat.i.dTLB-store-miss-rate%
5.909e+09 -42.5% 3.4e+09 perf-stat.i.dTLB-stores
47.00 +1.4 48.45 perf-stat.i.iTLB-load-miss-rate%
2270674 -11.0% 2021114 perf-stat.i.iTLB-load-misses
2563127 -16.0% 2151931 perf-stat.i.iTLB-loads
3.548e+10 -42.4% 2.044e+10 perf-stat.i.instructions
15634 -35.2% 10127 perf-stat.i.instructions-per-iTLB-miss
0.26 -41.6% 0.15 perf-stat.i.ipc
207.77 -37.5% 129.85 perf-stat.i.metric.M/sec
78061415 ± 13% +98.0% 1.546e+08 ± 20% perf-stat.i.node-load-misses
85582855 ± 11% +58.1% 1.353e+08 ± 20% perf-stat.i.node-loads
3.817e+08 -2.8% 3.709e+08 perf-stat.i.node-stores
42.66 +68.7% 71.96 perf-stat.overall.MPKI
0.06 +0.0 0.09 ± 3% perf-stat.overall.branch-miss-rate%
95.45 -2.4 93.07 perf-stat.overall.cache-miss-rate%
3.81 +73.0% 6.59 perf-stat.overall.cpi
93.55 +5.2% 98.41 perf-stat.overall.cycles-between-cache-misses
0.00 ± 5% +0.0 0.00 ± 13% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 5% +0.0 0.00 ± 2% perf-stat.overall.dTLB-store-miss-rate%
46.98 +1.5 48.43 perf-stat.overall.iTLB-load-miss-rate%
15639 -35.2% 10127 perf-stat.overall.instructions-per-iTLB-miss
0.26 -42.2% 0.15 perf-stat.overall.ipc
1605743 +1.5% 1630326 perf-stat.overall.path-length
5.919e+09 -42.2% 3.422e+09 perf-stat.ps.branch-instructions
3519866 ± 2% -7.8% 3245208 ± 3% perf-stat.ps.branch-misses
1.437e+09 -5.2% 1.362e+09 perf-stat.ps.cache-misses
1.506e+09 -2.8% 1.463e+09 perf-stat.ps.cache-references
1552 -1.4% 1530 perf-stat.ps.context-switches
6e+09 -42.1% 3.477e+09 perf-stat.ps.dTLB-loads
5.88e+09 -42.5% 3.382e+09 perf-stat.ps.dTLB-stores
2257568 -11.0% 2008542 perf-stat.ps.iTLB-load-misses
2547705 -16.1% 2138603 perf-stat.ps.iTLB-loads
3.53e+10 -42.4% 2.034e+10 perf-stat.ps.instructions
77685715 ± 13% +97.9% 1.538e+08 ± 20% perf-stat.ps.node-load-misses
85143339 ± 11% +58.1% 1.346e+08 ± 20% perf-stat.ps.node-loads
3.797e+08 -2.8% 3.69e+08 perf-stat.ps.node-stores
7.112e+12 -42.4% 4.096e+12 perf-stat.total.instructions
fio.read_bw_MBps
46000 +-------------------------------------------------------------------+
44000 |..+.+..+.+..+..+.+..+..+.+.. .+.. .+.+..+.+..+ |
| + +. |
42000 |-+ |
40000 |-+ |
38000 |-+ |
36000 |-+ |
| |
34000 |-+ |
32000 |-+ |
30000 |-+ |
28000 |-+ |
| |
26000 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
24000 +-------------------------------------------------------------------+
fio.read_iops
23000 +-------------------------------------------------------------------+
22000 |..+.+..+.+..+..+.+..+..+.+.. .+.. .+.+..+.+..+ |
| + +. |
21000 |-+ |
20000 |-+ |
19000 |-+ |
18000 |-+ |
| |
17000 |-+ |
16000 |-+ |
15000 |-+ |
14000 |-+ |
| |
13000 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
12000 +-------------------------------------------------------------------+
fio.read_clat_mean_us
1.2e+08 +-----------------------------------------------------------------+
| O O O O O O O O O O O O O O O O O O O O O O O O |
1.1e+08 |-+ |
| |
| |
1e+08 |-+ |
| |
9e+07 |-+ |
| |
8e+07 |-+ |
| |
| |
7e+07 |..+.+.. .+..+. .+..+.+..+..+.+..+.+.. |
| +.+..+ +..+ + |
6e+07 +-----------------------------------------------------------------+
fio.read_clat_90__us
1.3e+08 +-----------------------------------------------------------------+
| |
1.2e+08 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
| |
1.1e+08 |-+ |
| |
1e+08 |-+ |
| |
9e+07 |-+ |
| |
8e+07 |-+ |
| |
7e+07 |..+.+..+.+..+.+..+.+..+.+..+.+..+..+.+..+.+..+ |
| |
6e+07 +-----------------------------------------------------------------+
fio.read_clat_95__us
1.3e+08 +-----------------------------------------------------------------+
| O O O O O |
1.2e+08 |-+O O O O O O O O O O O O O O O O O O O O |
| |
1.1e+08 |-+ |
| |
1e+08 |-+ |
| |
9e+07 |-+ |
| |
8e+07 |-+ |
| .+. .+.. |
7e+07 |..+.+..+.+..+.+..+.+..+.+. +. +.+..+.+..+ |
| |
6e+07 +-----------------------------------------------------------------+
fio.read_slat_mean_us
4e+06 +-----------------------------------------------------------------+
3.8e+06 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
| |
3.6e+06 |-+ |
3.4e+06 |-+ |
| |
3.2e+06 |-+ |
3e+06 |-+ |
2.8e+06 |-+ |
| |
2.6e+06 |-+ |
2.4e+06 |-+ |
| |
2.2e+06 |..+.+..+.+..+.+..+.+..+.+..+.+..+..+.+..+.+..+ |
2e+06 +-----------------------------------------------------------------+
fio.latency_10ms_
0.15 +--------------------------------------------------------------------+
0.14 |..+.+..+..+.+..+..+.+..+..+.+..+..+.+..+.+..+..+ |
| |
0.13 |-+ |
0.12 |-+ |
0.11 |-+ |
0.1 |-+ |
| |
0.09 |-+ |
0.08 |-+ |
0.07 |-+ |
0.06 |-+ |
| |
0.05 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
0.04 +--------------------------------------------------------------------+
fio.latency_20ms_
0.55 +--------------------------------------------------------------------+
| + + + |
0.5 |-+ : + + :: |
0.45 |++ : + + + : : |
| + : + + : |
0.4 |-+ : : : : : |
0.35 |-+ : : : : : |
| : : : : : + |
0.3 |-+ : : : +. : + + |
0.25 |-+ : : .. +.. : + + |
| + +..+ +..+ +..+ |
0.2 |-+ |
0.15 |-+ |
| O O O O O O O O O O O O O O O O O O O O O O O O O |
0.1 +--------------------------------------------------------------------+
fio.latency_100ms_
100 +---------------------------------------------------------------------+
90 |-+ + +.+..+..+ +. |
| |
80 |-+ |
70 |-+ |
| |
60 |-+ |
50 |-+ |
40 |-+ |
| |
30 |-+ |
20 |-+ |
| |
10 |-+ |
0 +---------------------------------------------------------------------+
fio.latency_250ms_
100 +---------------------------------------------------------------------+
90 |-+ O |
| |
80 |-+ |
70 |-+ |
| |
60 |-+ |
50 |-+ |
40 |-+ |
| |
30 |-+ |
20 |-+ |
| |
10 |-+ |
0 +---------------------------------------------------------------------+
fio.workload
4.6e+06 +-----------------------------------------------------------------+
4.4e+06 |..+.+..+.+..+.+..+.+..+.+.. .+.. .+.+..+.+..+ |
| + +. |
4.2e+06 |-+ |
4e+06 |-+ |
3.8e+06 |-+ |
3.6e+06 |-+ |
| |
3.4e+06 |-+ |
3.2e+06 |-+ |
3e+06 |-+ |
2.8e+06 |-+ |
| |
2.6e+06 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
2.4e+06 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-csl-2sp6: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/mount_option/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
2M/gcc-9/performance/2pmem/xfs/sync/x86_64-rhel-8.3/dax/50%/debian-10.4-x86_64-20200603.cgz/200s/read/lkp-csl-2sp6/200G/fio-basic/tb/0x5002f01
commit:
7476b91d4d ("x86, powerpc: Rename memcpy_mcsafe() to copy_mc_to_{user, kernel}()")
a0ac629ebe ("x86/copy_mc: Introduce copy_mc_generic()")
7476b91d4db369d8 a0ac629ebe7b3d248cb93807782
---------------- ---------------------------
%stddev %change %stddev
\ | \
0.61 ± 15% -0.4 0.22 ± 94% fio.latency_1000us%
0.01 ± 11% +1.3 1.27 ± 25% fio.latency_10ms%
96.06 -95.5 0.60 ± 80% fio.latency_2ms%
1.27 ± 33% +96.2 97.48 fio.latency_4ms%
1.29 ± 55% -1.2 0.05 ± 54% fio.latency_500us%
75143 -55.6% 33381 fio.read_bw_MBps
1372160 +118.5% 2998272 fio.read_clat_90%_us
1409024 +116.9% 3055616 fio.read_clat_95%_us
2142208 ± 19% +120.3% 4718592 ± 17% fio.read_clat_99%_us
1272849 +125.4% 2869293 fio.read_clat_mean_us
228201 ± 15% +103.6% 464620 ± 14% fio.read_clat_stddev
37571 -55.6% 16690 fio.read_iops
69.28 ± 2% -40.3% 41.38 ± 3% fio.time.user_time
7514438 -55.6% 3338252 fio.workload
0.11 ± 3% +0.0 0.14 ± 5% mpstat.cpu.all.soft%
0.43 ± 3% -0.1 0.28 ± 2% mpstat.cpu.all.usr%
115069 -2.3% 112454 proc-vmstat.nr_shmem
20846 ± 6% -27.8% 15052 ± 3% proc-vmstat.pgactivate
967.50 ± 27% -50.0% 483.75 ± 78% slabinfo.xfs_buf_item.active_objs
967.50 ± 27% -50.0% 483.75 ± 78% slabinfo.xfs_buf_item.num_objs
100.00 -2.0% 98.00 vmstat.io.bo
1672 -3.3% 1616 vmstat.system.cs
9.059e+09 ± 6% -32.3% 6.131e+09 ± 54% cpuidle.C1E.time
19004364 ± 3% -22.4% 14741281 ± 34% cpuidle.C1E.usage
4.034e+08 ±133% +713.0% 3.28e+09 ±100% cpuidle.C6.time
570211 ±122% +571.6% 3829822 ± 86% cpuidle.C6.usage
61.80 ± 9% -17.6 44.19 perf-profile.calltrace.cycles-pp.copy_mc_fragile.copy_mc_to_user.copyout_mc._copy_mc_to_iter.dax_iomap_actor
0.00 +7.8 7.81 ± 6% perf-profile.calltrace.cycles-pp.copy_mc_generic.copy_mc_to_user.copyout_mc._copy_mc_to_iter.dax_iomap_actor
0.00 +29.2 29.21 ± 5% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.copy_mc_generic.copy_mc_to_user.copyout_mc._copy_mc_to_iter
61.92 ± 9% -17.7 44.25 perf-profile.children.cycles-pp.copy_mc_fragile
3.47 ±132% +11.7 15.21 ± 5% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.00 +22.3 22.32 perf-profile.children.cycles-pp.copy_mc_generic
61.16 ± 9% -17.4 43.78 perf-profile.self.cycles-pp.copy_mc_fragile
0.00 +22.1 22.09 perf-profile.self.cycles-pp.copy_mc_generic
212.00 ± 38% +288.6% 823.90 ± 67% sched_debug.cfs_rq:/.exec_clock.min
34013 ± 3% -17.1% 28181 ± 2% sched_debug.cfs_rq:/.exec_clock.stddev
36118 ± 5% -15.0% 30710 ± 2% sched_debug.cfs_rq:/.min_vruntime.stddev
36118 ± 5% -15.0% 30707 ± 2% sched_debug.cfs_rq:/.spread0.stddev
9.52 ± 11% +33.8% 12.73 ± 9% sched_debug.cpu.clock.stddev
17832 ± 13% -47.5% 9368 ± 17% sched_debug.cpu.sched_count.max
2475 ± 9% -34.4% 1624 ± 8% sched_debug.cpu.sched_count.stddev
8858 ± 13% -48.3% 4577 ± 18% sched_debug.cpu.sched_goidle.max
1260 ± 9% -33.2% 841.68 ± 8% sched_debug.cpu.sched_goidle.stddev
8285 ± 16% -32.1% 5622 ± 7% sched_debug.cpu.ttwu_count.max
1169 ± 9% -24.9% 878.40 ± 4% sched_debug.cpu.ttwu_count.stddev
26587 ± 8% -21.9% 20773 ± 22% softirqs.CPU1.SCHED
19906 ± 37% -55.7% 8824 ± 96% softirqs.CPU10.SCHED
21997 ± 34% -82.2% 3910 ± 55% softirqs.CPU20.SCHED
5126 ± 70% +166.6% 13666 ± 15% softirqs.CPU30.SCHED
5567 ± 56% +165.3% 14772 ± 29% softirqs.CPU31.SCHED
10027 ± 35% +101.3% 20182 ± 18% softirqs.CPU33.SCHED
4868 ± 50% +112.6% 10349 ± 14% softirqs.CPU44.SCHED
6304 ± 60% +154.5% 16043 ± 22% softirqs.CPU46.SCHED
4127 ± 76% +198.6% 12326 ± 32% softirqs.CPU49.SCHED
6313 ± 62% +98.5% 12530 ± 19% softirqs.CPU51.SCHED
8249 ± 58% +148.7% 20515 ± 31% softirqs.CPU57.SCHED
6971 ±109% +268.6% 25698 ± 8% softirqs.CPU68.SCHED
25116 ± 15% -32.4% 16974 ± 12% softirqs.CPU78.SCHED
24757 ± 12% -36.8% 15657 ± 27% softirqs.CPU79.SCHED
20231 ± 14% -45.5% 11024 ± 24% softirqs.CPU81.SCHED
21830 ± 23% -55.4% 9733 ± 67% softirqs.CPU9.SCHED
24043 ± 16% -39.9% 14449 ± 23% softirqs.CPU94.SCHED
42.31 +68.3% 71.22 perf-stat.i.MPKI
9.958e+09 -54.7% 4.511e+09 perf-stat.i.branch-instructions
0.05 ± 2% +0.0 0.08 ± 4% perf-stat.i.branch-miss-rate%
3682118 ± 2% -8.2% 3381534 perf-stat.i.branch-misses
67.34 +10.4 77.74 perf-stat.i.cache-miss-rate%
1.709e+09 -12.2% 1.501e+09 perf-stat.i.cache-misses
2.531e+09 -24.0% 1.923e+09 perf-stat.i.cache-references
1639 -4.1% 1571 perf-stat.i.context-switches
2.25 +121.4% 4.98 perf-stat.i.cpi
99.03 -1.8% 97.24 perf-stat.i.cpu-migrations
85.60 +14.2% 97.78 perf-stat.i.cycles-between-cache-misses
0.00 ± 18% +0.0 0.00 ± 44% perf-stat.i.dTLB-load-miss-rate%
9.996e+09 -54.5% 4.549e+09 perf-stat.i.dTLB-loads
0.00 ± 7% +0.0 0.00 ± 6% perf-stat.i.dTLB-store-miss-rate%
9.904e+09 -54.9% 4.466e+09 perf-stat.i.dTLB-stores
44.79 +4.2 48.99 perf-stat.i.iTLB-load-miss-rate%
2535885 -13.8% 2185118 perf-stat.i.iTLB-load-misses
3134177 -27.3% 2278467 perf-stat.i.iTLB-loads
5.952e+10 -54.9% 2.687e+10 perf-stat.i.instructions
23480 -47.6% 12304 perf-stat.i.instructions-per-iTLB-miss
0.45 -54.6% 0.20 perf-stat.i.ipc
342.39 -51.0% 167.90 perf-stat.i.metric.M/sec
1.165e+08 ± 30% +72.8% 2.013e+08 ± 9% perf-stat.i.node-load-misses
1.257e+08 ± 26% +41.1% 1.773e+08 ± 9% perf-stat.i.node-loads
2.42e+08 +19.6% 2.895e+08 perf-stat.i.node-stores
42.53 +68.3% 71.58 perf-stat.overall.MPKI
0.04 ± 2% +0.0 0.07 perf-stat.overall.branch-miss-rate%
67.52 +10.5 78.07 perf-stat.overall.cache-miss-rate%
2.24 +122.4% 4.99 perf-stat.overall.cpi
78.17 +14.3% 89.34 perf-stat.overall.cycles-between-cache-misses
0.00 ± 25% +0.0 0.00 ± 12% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 13% +0.0 0.00 ± 10% perf-stat.overall.dTLB-store-miss-rate%
44.72 +4.2 48.96 perf-stat.overall.iTLB-load-miss-rate%
23499 -47.6% 12306 perf-stat.overall.instructions-per-iTLB-miss
0.45 -55.0% 0.20 perf-stat.overall.ipc
1587395 +1.5% 1611895 perf-stat.overall.path-length
9.912e+09 -54.7% 4.489e+09 perf-stat.ps.branch-instructions
3650903 ± 2% -8.4% 3345674 perf-stat.ps.branch-misses
1.701e+09 -12.2% 1.494e+09 perf-stat.ps.cache-misses
2.52e+09 -24.1% 1.914e+09 perf-stat.ps.cache-references
1616 -3.7% 1556 perf-stat.ps.context-switches
9.95e+09 -54.5% 4.526e+09 perf-stat.ps.dTLB-loads
9.859e+09 -54.9% 4.445e+09 perf-stat.ps.dTLB-stores
2521574 -13.8% 2172342 perf-stat.ps.iTLB-load-misses
3116655 -27.3% 2264894 perf-stat.ps.iTLB-loads
5.925e+10 -54.9% 2.673e+10 perf-stat.ps.instructions
1.159e+08 ± 30% +72.8% 2.003e+08 ± 9% perf-stat.ps.node-load-misses
1.25e+08 ± 26% +41.1% 1.764e+08 ± 9% perf-stat.ps.node-loads
2.407e+08 +19.6% 2.878e+08 perf-stat.ps.node-stores
1.193e+13 -54.9% 5.381e+12 perf-stat.total.instructions
0.00 +2.7e+105% 2689 ±171% interrupts.115:PCI-MSI.31981648-edge.i40e-eth0-TxRx-79
62.75 ± 27% +51.8% 95.25 ± 21% interrupts.CPU1.RES:Rescheduling_interrupts
6530 ± 17% -44.1% 3647 ± 35% interrupts.CPU17.NMI:Non-maskable_interrupts
6530 ± 17% -44.1% 3647 ± 35% interrupts.CPU17.PMI:Performance_monitoring_interrupts
62.00 ± 74% +187.9% 178.50 ± 5% interrupts.CPU20.RES:Rescheduling_interrupts
365.00 ± 78% -76.0% 87.50 ± 53% interrupts.CPU25.TLB:TLB_shootdowns
170.50 ± 15% -26.8% 124.75 ± 10% interrupts.CPU30.RES:Rescheduling_interrupts
7605 -43.3% 4316 ± 32% interrupts.CPU31.NMI:Non-maskable_interrupts
7605 -43.3% 4316 ± 32% interrupts.CPU31.PMI:Performance_monitoring_interrupts
169.00 ± 12% -37.1% 106.25 ± 23% interrupts.CPU31.RES:Rescheduling_interrupts
7145 ± 11% -33.0% 4786 ± 18% interrupts.CPU36.NMI:Non-maskable_interrupts
7145 ± 11% -33.0% 4786 ± 18% interrupts.CPU36.PMI:Performance_monitoring_interrupts
136.50 ± 27% -44.7% 75.50 ± 60% interrupts.CPU39.TLB:TLB_shootdowns
149.25 ± 24% -24.6% 112.50 ± 30% interrupts.CPU4.RES:Rescheduling_interrupts
7599 -46.6% 4061 ± 35% interrupts.CPU41.NMI:Non-maskable_interrupts
7599 -46.6% 4061 ± 35% interrupts.CPU41.PMI:Performance_monitoring_interrupts
6661 ± 24% -52.1% 3191 ± 51% interrupts.CPU44.NMI:Non-maskable_interrupts
6661 ± 24% -52.1% 3191 ± 51% interrupts.CPU44.PMI:Performance_monitoring_interrupts
7622 -43.5% 4307 ± 33% interrupts.CPU46.NMI:Non-maskable_interrupts
7622 -43.5% 4307 ± 33% interrupts.CPU46.PMI:Performance_monitoring_interrupts
7613 -43.1% 4331 ± 31% interrupts.CPU47.NMI:Non-maskable_interrupts
7613 -43.1% 4331 ± 31% interrupts.CPU47.PMI:Performance_monitoring_interrupts
5823 ± 32% -36.4% 3703 ± 34% interrupts.CPU5.NMI:Non-maskable_interrupts
5823 ± 32% -36.4% 3703 ± 34% interrupts.CPU5.PMI:Performance_monitoring_interrupts
89.25 ± 48% -61.1% 34.75 ± 31% interrupts.CPU53.TLB:TLB_shootdowns
5698 ± 33% -42.5% 3277 ± 49% interrupts.CPU55.NMI:Non-maskable_interrupts
5698 ± 33% -42.5% 3277 ± 49% interrupts.CPU55.PMI:Performance_monitoring_interrupts
172.00 ± 14% -35.2% 111.50 ± 41% interrupts.CPU56.RES:Rescheduling_interrupts
64.00 ± 42% -39.5% 38.75 ± 29% interrupts.CPU56.TLB:TLB_shootdowns
156.00 ± 17% -36.2% 99.50 ± 21% interrupts.CPU57.RES:Rescheduling_interrupts
146.25 ± 28% -48.9% 74.75 ± 67% interrupts.CPU58.RES:Rescheduling_interrupts
7627 -47.0% 4043 ± 31% interrupts.CPU62.NMI:Non-maskable_interrupts
7627 -47.0% 4043 ± 31% interrupts.CPU62.PMI:Performance_monitoring_interrupts
174.75 ± 12% -29.9% 122.50 ± 30% interrupts.CPU62.RES:Rescheduling_interrupts
76.00 ± 29% -48.4% 39.25 ± 29% interrupts.CPU62.TLB:TLB_shootdowns
7159 ± 11% -50.2% 3564 ± 32% interrupts.CPU63.NMI:Non-maskable_interrupts
7159 ± 11% -50.2% 3564 ± 32% interrupts.CPU63.PMI:Performance_monitoring_interrupts
7628 -62.9% 2831 interrupts.CPU66.NMI:Non-maskable_interrupts
7628 -62.9% 2831 interrupts.CPU66.PMI:Performance_monitoring_interrupts
174.50 ± 10% -36.4% 111.00 ± 50% interrupts.CPU66.RES:Rescheduling_interrupts
4370 ± 18% -34.7% 2853 interrupts.CPU69.NMI:Non-maskable_interrupts
4370 ± 18% -34.7% 2853 interrupts.CPU69.PMI:Performance_monitoring_interrupts
6885 ± 18% -45.8% 3731 ± 28% interrupts.CPU74.NMI:Non-maskable_interrupts
6885 ± 18% -45.8% 3731 ± 28% interrupts.CPU74.PMI:Performance_monitoring_interrupts
5900 ± 18% -57.5% 2510 ± 24% interrupts.CPU77.NMI:Non-maskable_interrupts
5900 ± 18% -57.5% 2510 ± 24% interrupts.CPU77.PMI:Performance_monitoring_interrupts
62.00 ± 41% +58.9% 98.50 ± 14% interrupts.CPU78.RES:Rescheduling_interrupts
0.00 +2.7e+105% 2689 ±171% interrupts.CPU79.115:PCI-MSI.31981648-edge.i40e-eth0-TxRx-79
49.75 ± 47% +119.6% 109.25 ± 28% interrupts.CPU79.RES:Rescheduling_interrupts
61.50 ± 54% +115.0% 132.25 ± 31% interrupts.CPU8.RES:Rescheduling_interrupts
5871 ± 19% -38.8% 3594 ± 32% interrupts.CPU80.NMI:Non-maskable_interrupts
5871 ± 19% -38.8% 3594 ± 32% interrupts.CPU80.PMI:Performance_monitoring_interrupts
60.50 ± 19% +120.2% 133.25 ± 14% interrupts.CPU81.RES:Rescheduling_interrupts
36.00 ± 79% +179.2% 100.50 ± 35% interrupts.CPU86.RES:Rescheduling_interrupts
6322 ± 21% -60.6% 2490 ± 25% interrupts.CPU88.NMI:Non-maskable_interrupts
6322 ± 21% -60.6% 2490 ± 25% interrupts.CPU88.PMI:Performance_monitoring_interrupts
32.50 ± 40% +150.0% 81.25 ± 41% interrupts.CPU92.RES:Rescheduling_interrupts
124.00 ± 11% -22.8% 95.75 ± 4% interrupts.IWI:IRQ_work_interrupts
538989 ± 8% -28.0% 387910 ± 2% interrupts.NMI:Non-maskable_interrupts
538989 ± 8% -28.0% 387910 ± 2% interrupts.PMI:Performance_monitoring_interrupts
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
1 year, 10 months
[torture] 345fdd0aab: WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_fwd_prog_nr
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 345fdd0aabfa4aaa469e782ada7c26e29dd61c7f ("torture: Make torture_stutter() use hrtimer")
https://git.kernel.org/cgit/linux/kernel/git/paulmck/linux-rcu.git rcu/next
in testcase: trinity
version:
with following parameters:
runtime: 300s
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-------------------------------------------------------------+------------+------------+
| | a6afecad1b | 345fdd0aab |
+-------------------------------------------------------------+------------+------------+
| boot_successes | 4 | 0 |
| boot_failures | 0 | 3 |
| WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_fwd_prog_nr | 0 | 3 |
| EIP:rcu_torture_fwd_prog_nr | 0 | 3 |
+-------------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 77.789569] WARNING: CPU: 0 PID: 124 at kernel/rcu/rcutorture.c:1951 rcu_torture_fwd_prog_nr+0x52c/0x625
[ 77.790167] Modules linked in:
[ 77.790373] CPU: 0 PID: 124 Comm: rcu_torture_fwd Not tainted 5.9.0-rc1-00146-g345fdd0aabfa4 #2
[ 77.790932] EIP: rcu_torture_fwd_prog_nr+0x52c/0x625
[ 77.791253] Code: e8 42 a0 ff ff 83 05 20 eb d3 83 01 83 15 24 eb d3 83 00 85 db 75 23 83 f8 01 77 1e 83 05 28 eb d3 83 01 83 15 2c eb d3 83 00 <0f> 0b 83 05 30 eb d3 83 01 83 15 34 eb d3 83 00 50 53 ff 75 e0 68
[ 77.792425] EAX: 00000000 EBX: 00000000 ECX: 81123459 EDX: 00000000
[ 77.792824] ESI: e5955e00 EDI: ffffa8b1 EBP: efae1f74 ESP: efae1f44
[ 77.793222] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00010246
[ 77.793653] CR0: 80050033 CR2: 080a2190 CR3: 6f8d6aa0 CR4: 000406b0
[ 77.794054] DR0: 76a67000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 77.794453] DR6: fffe0ff0 DR7: 00000600
[ 77.794710] Call Trace:
[ 77.794880] ? rcu_torture_fwd_cb_cr+0x1d1/0x1d1
[ 77.795179] rcu_torture_fwd_prog+0xb2/0x19b
[ 77.795457] kthread+0x2ce/0x2e1
[ 77.795672] ? rcu_torture_fwd_prog_nr+0x625/0x625
[ 77.795982] ? kthread_create_worker+0x38/0x38
[ 77.796269] ret_from_fork+0x19/0x24
[ 77.796503] ---[ end trace b9e70890d3ca1c49 ]---
To reproduce:
# build kernel
cd linux
cp config-5.9.0-rc1-00146-g345fdd0aabfa4 .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
1 year, 10 months