[PATCH 0/2] device-dax: fix unaligned munmap handling
by Dan Williams
Hi Andrew,
Here is another device-dax fix that requires touching some mm code. When
device-dax is operating in huge-page mode we want it to behave like
hugetlbfs and fail attempts to split vmas into unaligned ranges. It
would be messy to teach the munmap path about device-dax alignment
constraints in the same (hstate) way that hugetlbfs communicates this
constraint. Instead, these patches introduce a new ->split() vm
operation.
---
Dan Williams (2):
mm, hugetlbfs: introduce ->split() to vm_operations_struct
device-dax: implement ->split() to catch invalid munmap attempts
drivers/dax/device.c | 12 ++++++++++++
include/linux/mm.h | 1 +
mm/hugetlb.c | 8 ++++++++
mm/mmap.c | 8 +++++---
4 files changed, 26 insertions(+), 3 deletions(-)
3 years, 1 month
[PATCH v3 0/5] fix device-dax pud crash and fixup {pte, pmd, pud}_write
by Dan Williams
Changes since v2 [1]:
* Switch from the "#define __HAVE_ARCH_PUD_WRITE" to "#define
pud_write". This incidentally fixes a powerpc compile error.
(Stephen)
* Add a cleanup patch to align pmd_write to the pud_write definition
scheme.
---
Andrew,
Here is another attempt at the pud_write() fix [2], and some follow-on
patches to use the '_access_permitted' helpers in fault and
get_user_pages() paths where we are checking if the thread has access to
write. I explicitly omit conversions for places where the kernel is
checking the _PAGE_RW flag for kernel purposes, not for userspace
access.
Beyond fixing the crash, this series also fixes get_user_pages() and
fault paths to honor protection keys in the same manner as
get_user_pages_fast(). Only the crash fix is tagged for -stable as the
protection key check is done just for consistency reasons since
userspace can change protection keys at will.
These have received a build success notification from the 0day robot.
[1]: https://lists.01.org/pipermail/linux-nvdimm/2017-November/013254.html
[2]: https://lists.01.org/pipermail/linux-nvdimm/2017-November/013237.html
---
Dan Williams (5):
mm: fix device-dax pud write-faults triggered by get_user_pages()
mm: switch to 'define pmd_write' instead of __HAVE_ARCH_PMD_WRITE
mm: replace pud_write with pud_access_permitted in fault + gup paths
mm: replace pmd_write with pmd_access_permitted in fault + gup paths
mm: replace pte_write with pte_access_permitted in fault + gup paths
arch/arm/include/asm/pgtable-3level.h | 1 -
arch/arm64/include/asm/pgtable.h | 1 -
arch/mips/include/asm/pgtable.h | 2 +-
arch/powerpc/include/asm/book3s/64/pgtable.h | 1 -
arch/s390/include/asm/pgtable.h | 8 +++++++-
arch/sparc/include/asm/pgtable_64.h | 2 +-
arch/sparc/mm/gup.c | 4 ++--
arch/tile/include/asm/pgtable.h | 1 -
arch/x86/include/asm/pgtable.h | 8 +++++++-
fs/dax.c | 3 ++-
include/asm-generic/pgtable.h | 12 ++++++++++--
include/linux/hugetlb.h | 8 --------
mm/gup.c | 2 +-
mm/hmm.c | 8 ++++----
mm/huge_memory.c | 6 +++---
mm/memory.c | 8 ++++----
16 files changed, 42 insertions(+), 33 deletions(-)
3 years, 1 month
Re: KVM "fake DAX" flushing interface - discussion
by Dan Williams
On Tue, Nov 21, 2017 at 10:19 AM, Rik van Riel <riel(a)redhat.com> wrote:
> On Fri, 2017-11-03 at 14:21 +0800, Xiao Guangrong wrote:
>> On 11/03/2017 12:30 AM, Dan Williams wrote:
>> >
>> > Good point, I was assuming that the mmio flush interface would be
>> > discovered separately from the NFIT-defined memory range. Perhaps
>> > via
>> > PCI in the guest? This piece of the proposal needs a bit more
>> > thought...
>> >
>>
>> Consider the case that the vNVDIMM device on normal storage and
>> vNVDIMM device on real nvdimm hardware can both exist in VM, the
>> flush interface should be able to associate with the SPA region
>> respectively. That's why I'd like to integrate the flush interface
>> into NFIT/ACPI by using a separate table. Is it possible to be a
>> part of ACPI specification? :)
>
> It would also be perfectly fine to have the
> virtio PCI device indicate which vNVDIMM
> range it flushes.
>
> Since the guest OS needs to support that kind
> of device anyway, does it really matter which
> direction the device association points?
>
> We can go with the "best" interface for what
> could be a relatively slow flush (fsync on a
> file on ssd/disk on the host), which requires
> that the flushing task wait on completion
> asynchronously.
>
> If that kind of interface cannot be advertised
> through NFIT/ACPI, wouldn't it be perfectly fine
> to have only the virtio PCI device indicate which
> vNVDIMM range it flushes?
>
Yes, we could do this with a custom PCI device, however the NFIT is
frustratingly close to being able to define something like this. At
the very least we can start with a "SPA Range GUID" that is Linux
specific to indicate "call this virtio flush interface on FUA / flush
cache requests" as a stop gap until a standardized flush interface can
be defined.
3 years, 1 month
转发： 技术人员如何适应管理
by 贾瀚漠
从技术走向管理——研发经理的领导力与执行力
培训对象 企业CEO/总经理、研发总经理/副总、公司总工/技术总监、研发项目经理/产品经理、中试部经理、研发质量部经理、PMO（项目管理办公室）主任、走上管理岗位的技术人员等课程介绍
11月27-28日北京 11月30-12月1日上海 11月23-24日深圳
【培训费用】4980元买一送一，单独一人参加收费3200元，不再折扣。
【培训方式】案例分享、实务分析、互动讨论、项目模拟、培训游戏
■课程背景
根据我们多年从事研发管理咨询的经验发现中国企业95%以上的研发中基层主管都是从技术能力比较强的工程师中提拔起来的:
很多刚刚走上管理岗位的研发人员角色不能转换，过度关注技术细节；
认真帮助下属可是他们并不买账；
凡事亲力亲为，忙得焦头烂额，可是上司却嫌效率太低；
希望下属多提意见，可是他们却什么都不说，不愿意承担责任；
上司让制定工作计划，可却无从下手；
不知道如何分派工作，如何领导团队，更不知道如何确保你的团队不出差错
……致使疲惫不堪却还不能有效达到整体目标。
从一名只对技术负责的技术人员转变为对全流程负责的项目经理和对某一专业领域负责的部门经理，在这个转变的过程中，技术人员要实现哪些蜕变、要掌握哪些管理技能、如何培养自己的领导力等是本课程重点探讨的内容。
■培训收益
1.分享讲师上百场研发管理培训的专业经验，通过现场的互动帮助学员理清走向管理的困惑
2.总结和分析技术人员从技术走向管理过程中常见的问题
3.掌握实现从技术走向管理的过程中要实现的几个转变
4.了解从技术走向管理的五个好习惯（结果导向、综观全局、聚焦重点、发挥优势、集思广益）
5.掌握与领导沟通的方法技巧
6.掌握走上管理工作岗位后需要掌握的四个核心管理技能（目标与计划、组织与分派工作、控制与纠偏、领导与激励）
7.了解成功实现从技术走向管理转变的几个关键要素
8.分享讲师数十个咨询项目的研发管理的案例资料（模板、表格、样例……），帮助学员制定 Action Plan，使得学员参训后回到自己的公司能够很好实践
■课程大纲
一、案例分析（0.5）
1)讨论：技术走向管理的烦恼
二、从技术走向管理的角色定位和角色转换（1.5）
1)为什么要从技术走向管理（背景、原因）
2)技术人员的角色定位和素质模型
3)管理人员的角色定位和素质模型
4)技术型管理者的角色与核心工作
（技术管理者的不是说不要技术，而是层次越高的技术管理者，越需要技术广度、技
术敏锐度与市场敏锐度，而且更需要沟通、管理与领导技能）
5)技术人员与管理人员的特质
6)研发人员的特点
7)角色转换过程中常见的问题分析
8）角色转换的成长之路（角色、态度、知识、技能）
9）演练与问题讨论
三、从技术走向管理必备的好习惯（3.5）
1)习惯的价值与培养
2)习惯与原则
3)习惯之一：成果导向
4)习惯之二：综观全局
5)习惯之三：聚焦重点
6)习惯之四：发挥优势
7)习惯之五：集思广益
四、研发管理者如何与领导沟通（1.5）
1)研发管理者自己沟通能力不强而领导又不懂技术怎么办？
2)为什么研发工作自己觉得开展的很好却得不到老板或领导的认可？
3)与领导沟通的重要性
4)无数“革命先烈”的教训分享
5)领导的沟通类型
6)领导的沟通类型对沟通的影响
7)与领导沟通的难题（尤其是没有技术背景的领导）
8)与领导沟通的要点
9)高层领导喜欢的沟通方式
10)与领导沟通的方式、方法与技巧
11)与领导沟通谨慎换位思考
12)向领导汇报方式和工具
13)汇报会上领导常问的问题分类
14)为什么领导在会上总是不断追着问？
15)高层管理者对研发的沟通信息需求（开发状况、资源状况、管理优化状况）
详细介绍和模板演示
16)分辨领导的真正需求
17)要想成功从技术走向管理首先做个成功的下属
18)如何做个成功的下属
19)研讨：学习本单元的体会列出以后改进的三个要点
五、从技术走向管理的四个核心管理技能之一：目标与计划（1.0）
1)目标对我们的影响
2)个人目标和团队目标的关系
3)如何根据公司的战略要求制定研发部门和研发项目的目标
4)研发部门和项目的目标如何分解到个人
5)如何帮助下属制定工作目标
6)目标的制定与下达（SMART化、愿景化、共享化、承诺化（PBC））
7)研发项目的目标为什么不容易SMART
8)为什么培训了很多次SMART研发项目目标还是做不到SMART
9)研发工作计划的PDCA循环
10)产品平台规划、产品路标规划和产品计划、版本计划、特性计划之间的关系
11)研发流程与计划的关系
12)研发项目计划制定的流程
13)PERT、关键路径和GANNT
14)为什么研发项目计划不用PERT图
15)产品开发计划如何分成四级（这四级计划的责任主体和制定时间点）
16)研发计划与资源计划的管理
17)资源计划指导书举例
18)资源实施对开发进度的影响
19)演练：每个小组制定一个半年计划，发表！
六、从技术走向管理的四个核心管理技能之二：组织与分派工作（2.0）
1)常见研发组织形式及优缺点
2)如何对研发工作进行分解
3)给研发人员分派工作的原则
4)给研发人员分派工作的步骤
5)给研发人员分派工作中容易出现的问题
6)研发沟通管理的内容
7)沟通的目的与功能
8)沟通的种类与方式
9)有效沟通的障碍/约哈里窗
10)面对面沟通避免的小动作
11)如何给其它部门分派研发工作
12)研发执行力缺失的原因分析
13)研发管理人员在分派工作中容易存在的问题、原因和克服
14)给研发技术人员创造愿景、描绘愿景，尤其是关于项目与团队前途
15)目标共享与任务分配（目标的共享化、目标的承诺化、目标的系统化、
任务分配的原则、分配时的抗拒处理、分配时的控制紧度、
分配后的跟踪与控制、结果反馈）
16)给研发技术人员空间（猴子管理、授权的五个原则、共享与参与管理）
17)活动演练 30 分钟：扑克游戏——上中下三层互动（体验：管理对人与对事
，三层角色定位，目标下达，控制与跟踪，愿景与目标共享，结果反馈等）
18)案例研讨：研发技术型团队的成员常被迫承担紧急的项目周期，该如何处理？
19)案例研讨：给予研发技术人员的空间到底多大，犯什么样的错误可以接受？
20)案例研讨：任务下达后完成得不好但因为是碰到困难又怎么处理？
21)案例研讨：一个人承担多个项目遇到资源冲突怎么办
22)案例研讨：两个领导意见不一致，怎么办？
七、从技术走向管理的四个核心管理技能之三：控制与纠偏（1.5）
1)研发工作为什么难以控制
2)研发工作的问题管理与风险管理
3)研发工作追踪的步骤
4)研发工作控制方法之一：会议（具体操作与模板）
5)研发工作控制方法之二：报告机制（具体操作与模板）
6)研发工作控制方法之三：审计（具体操作与模板）
7)研发工作控制方法之四：合同书与任务书（具体操作与模板）
8)研发工作控制方法之五：预警系统（具体操作与模板）
9)研发工作控制方法之六：经验教训总结（具体操作与模板）
10)研发工作控制方法之七：测评（具体操作与模板）
11)研发工作控制方法之八：非正规控制（具体操作与模板）
12)研发工作如何度量、量化管理（有哪些量化指标、PCB）
13)研发部门中的“因人而异”的管理方法
14)研发财经与成本管理的最佳实践
15)研发费用和成本的控制要点
16)贯穿开发全流程的研发费用跟踪和控制要点
17)研发费用管理的全流程
18)设计成本的管理方法——目标成本法及应用
19)产品生命周期成本的定义与沟通要素
20)如何控制产品生命周期成本
21)如何从财务和投资角度控制研发多项目资源
22)关于控制的误区（用人不疑、甩手掌柜、与创新的矛盾）
23)关于研发执行力
八、从技术走向管理的四个核心管理技能之四：领导与激励（2.0）
1)研发领导权威力的来源
2)研发领导如何发展个人魅力
3)如何针对不同环境和不同的研发人员进行情景领导
4)讨论：如何增进研发团队的凝聚力和士气
5)研发领导如何授权
6)研发领导如何辅导下属和培养接班人
7)研发人员的考核与激励（专题讲解）
8)建立功能型团队与项目型团队面向结果的绩效考核办法
9)研发技术型人才的培育与任职资格管理
10)研发技术型人才的非物质激励与物质激励方法
11)演练与讨论
九、成功实现从技术走向管理转变的关键（0.5）
1)成功的实现角色换位
2)管理技能的培养
3)个人修炼（习惯、领导力、沟通能力）
4)组织的融合和团队的打造
5)给刚走上管理岗位的技术人员推荐的书籍和电影 专家介绍 Giles 资深顾问
■专业背景
10多年高科技企业产品研发和研发管理工作经历，具备十多年的产品开发、项目管理、研发管理、高层管理、研发咨询等丰富的技术和管理经验。在国内某大型知名企业工作期间，负责多个大型项目开发、管理工作，主持开发了多款成功的产品；长期与国际顶尖咨询顾问一起工作，作为研发唯一的核心小组成员全程参与了IPD项目的母项目ITS&P项目，作为IPD的筹备小组副组长，筹备并全程参与该公司研发管理变革项目（IPD）的规划和实施，推动该公司研发体系的创新文化建设。
■咨询背景
作为项目总监、带领咨询团队，成功为某电气集团、国内某知名网络公司、某知名电子公司、交通运输业某集团、某电子（芯片开发商）、中国电子科技集团某研究所等数十个企业进行了研发流程、流程管理、项目管理、技术创新及研发人力资源管理体系等方面的管理咨询项目，帮助这些企业建立高效、完备的研发管理体系，对中国企业研发管理体系及创新体制有着深刻的研究和理解，受到客户的高度评价。
■培训背景
曾在各地多次举办产品开发流程设计、研发项目管理、从技术走向管理等公开课，为数千家企业提供了研发管理公开课的培训，为数百家企业进行了研发管理的内训；在培训咨询的10年间，作为咨询项目总监和项目经理成功完成了数十个研发管理咨询项目体系的建设（技术创新文化、研发流程体系、项目管理体系、研发人力资源体系），有着丰富的研发管理咨询经验，涉及的行业包括通信、软件、家电、电信运营商、芯片等，项目范围涉及需求工程、产品规划、产品开发流程、研发项目管理、研发绩效管理、技术创新等模块体系，帮助这些企业建立高效、完备的研发管理体系，对中国企业研发管理体系及创新体制有着深刻的研究和理解，受到客户的高度评价。
报名咨询热线
报名咨询电话：0755-61288035 010-51661863 021-31261580
在线咨询 QQ：6983436 报名信箱：6983436(a)qq.com
手机：18890700600 （微信同号） 赵先生
温馨提示：(报名请回复索取报名表）
3 years, 1 month
转 发： linux-nvdimm
by 车珂晨
附件详细大纲，请查阅 》》》从技术走向管理——研发经理的领导力与执行力
从一名只对技术负责的技术人员转变为对全流程负责的项目经理和对某一专业领域负责的部门经理，在这个转变的过程中，技术人员要实现哪些蜕变、要掌握哪些管理技能、如何培养自己的领导力等是本课程重点探讨的内容。
创业者要常保五气【创业五常】：义气――干云，正气――凛然，意气――风发，志气――恢宏，运气――相随。
3 years, 2 months
[xfsprogs PATCH 0/2] Add necessary items for MAP_SYNC testing
by Ross Zwisler
As suggested by Dave Chinner:
As I say to all these sorts of one-off test prgrams: please add the
new MAP_SYNC flag to xfs_io rather than writing a one-off
test program to set it and write some data.
And if we're going to be adding special custom tests just because
we need to insert dm-log marks, add that functionality to xfs_io,
too.
That way we can create complex custom dm logwrite tests without
needing one-off test programs for them all...
This series enhances xfs_io by adding support for the MAP_SYNC mmap() flag
and for dm-log-writes marks. This allows the resulting xfstest for
MAP_SYNC to be much simpler and have no custom C programs.
You can find an xfsprogs branch with this series here:
https://git.kernel.org/pub/scm/linux/kernel/git/zwisler/xfsprogs-dev.git/...
Both MAP_SYNC and the DAX enhancements for dm-log-writes will be found in
the upcoming v4.15-rc1. For ease of testing I've posted a kernel that is
v4.14 plus just those two patch series here:
https://git.kernel.org/pub/scm/linux/kernel/git/zwisler/linux.git/log/?h=...
Ross Zwisler (2):
xfs_io: add MAP_SYNC support to mmap()
xfs_io: add a new 'log_writes' command
configure.ac | 1 +
include/builddefs.in | 1 +
include/linux.h | 5 ++++
io/Makefile | 5 ++--
io/init.c | 1 +
io/io.h | 2 ++
io/log_writes.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++
io/mmap.c | 19 +++++++++----
m4/package_libcdev.m4 | 16 +++++++++++
man/man8/xfs_io.8 | 25 ++++++++++++++++-
10 files changed, 145 insertions(+), 8 deletions(-)
create mode 100644 io/log_writes.c
--
2.9.5
3 years, 2 months
[ndctl PATCH] ndctl, rpmspec: use %{version} for the git version override
by Dan Williams
When we generate the spec file we're already teaching it the version
number so we don't need to create custom echo line each time.
Reported-by: Jeff Moyer <jmoyer(a)redhat.com>
Signed-off-by: Dan Williams <dan.j.williams(a)intel.com>
---
ndctl.spec.in | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/ndctl.spec.in b/ndctl.spec.in
index 4aa133dd70aa..fa60518e43a9 100644
--- a/ndctl.spec.in
+++ b/ndctl.spec.in
@@ -91,7 +91,7 @@ control API for these devices.
%setup -q ndctl-%{version}
%build
-echo "VERSION" > version
+echo %{version} > version
./autogen.sh
%ifarch x86_64
%configure --disable-static --enable-local --disable-silent-rules --with-libpmem
3 years, 2 months
[PATCH 1/2] dm log writes: Add support for inline data buffers
by Ross Zwisler
Currently dm-log-writes supports writing filesystem data via BIOs, and
writing internal metadata from a flat buffer via write_metadata().
For DAX writes, though, we won't have a BIO, but will instead have an
iterator that we'll want to use to fill a flat data buffer.
So, create write_inline_data() which allows us to write filesystem data
using a flat buffer as a source, and wire it up in log_one_block().
Signed-off-by: Ross Zwisler <ross.zwisler(a)linux.intel.com>
---
drivers/md/dm-log-writes.c | 90 +++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 86 insertions(+), 4 deletions(-)
diff --git a/drivers/md/dm-log-writes.c b/drivers/md/dm-log-writes.c
index 8b80a9c..c65f9d1 100644
--- a/drivers/md/dm-log-writes.c
+++ b/drivers/md/dm-log-writes.c
@@ -246,27 +246,109 @@ static int write_metadata(struct log_writes_c *lc, void *entry,
return -1;
}
+static int write_inline_data(struct log_writes_c *lc, void *entry,
+ size_t entrylen, void *data, size_t datalen,
+ sector_t sector)
+{
+ int num_pages, bio_pages, pg_datalen, pg_sectorlen, i;
+ struct page *page;
+ struct bio *bio;
+ size_t ret;
+ void *ptr;
+
+ while (datalen) {
+ num_pages = ALIGN(datalen, PAGE_SIZE) >> PAGE_SHIFT;
+ bio_pages = min(num_pages, BIO_MAX_PAGES);
+
+ atomic_inc(&lc->io_blocks);
+
+ bio = bio_alloc(GFP_KERNEL, bio_pages);
+ if (!bio) {
+ DMERR("Couldn't alloc inline data bio");
+ goto error;
+ }
+
+ bio->bi_iter.bi_size = 0;
+ bio->bi_iter.bi_sector = sector;
+ bio_set_dev(bio, lc->logdev->bdev);
+ bio->bi_end_io = log_end_io;
+ bio->bi_private = lc;
+ bio_set_op_attrs(bio, REQ_OP_WRITE, 0);
+
+ for (i = 0; i < bio_pages; i++) {
+ pg_datalen = min(datalen, PAGE_SIZE);
+ pg_sectorlen = ALIGN(pg_datalen, lc->sectorsize);
+
+ page = alloc_page(GFP_KERNEL);
+ if (!page) {
+ DMERR("Couldn't alloc inline data page");
+ goto error_bio;
+ }
+
+ ptr = kmap_atomic(page);
+ memcpy(ptr, data, pg_datalen);
+ if (pg_sectorlen > pg_datalen)
+ memset(ptr + pg_datalen, 0,
+ pg_sectorlen - pg_datalen);
+ kunmap_atomic(ptr);
+
+ ret = bio_add_page(bio, page, pg_sectorlen, 0);
+ if (ret != pg_sectorlen) {
+ DMERR("Couldn't add page of inline data");
+ __free_page(page);
+ goto error_bio;
+ }
+
+ datalen -= pg_datalen;
+ data += pg_datalen;
+ }
+ submit_bio(bio);
+
+ sector += bio_pages * PAGE_SECTORS;
+ }
+ return 0;
+error_bio:
+ bio_free_pages(bio);
+ bio_put(bio);
+error:
+ put_io_block(lc);
+ return -1;
+}
+
static int log_one_block(struct log_writes_c *lc,
struct pending_block *block, sector_t sector)
{
struct bio *bio;
struct log_write_entry entry;
- size_t ret;
+ size_t metadlen, ret;
int i;
entry.sector = cpu_to_le64(block->sector);
entry.nr_sectors = cpu_to_le64(block->nr_sectors);
entry.flags = cpu_to_le64(block->flags);
entry.data_len = cpu_to_le64(block->datalen);
- if (write_metadata(lc, &entry, sizeof(entry), block->data,
- block->datalen, sector)) {
+
+ metadlen = (block->flags & LOG_MARK_FLAG) ? block->datalen : 0;
+ if (write_metadata(lc, &entry, sizeof(entry), block->data, metadlen,
+ sector)) {
free_pending_block(lc, block);
return -1;
}
+ sector += dev_to_bio_sectors(lc, 1);
+
+ if (block->datalen && metadlen == 0) {
+ if (write_inline_data(lc, &entry, sizeof(entry), block->data,
+ block->datalen, sector)) {
+ free_pending_block(lc, block);
+ return -1;
+ }
+ /* we don't support both inline data & bio data */
+ goto out;
+ }
+
if (!block->vec_cnt)
goto out;
- sector += dev_to_bio_sectors(lc, 1);
atomic_inc(&lc->io_blocks);
bio = bio_alloc(GFP_KERNEL, min(block->vec_cnt, BIO_MAX_PAGES));
--
2.9.5
3 years, 2 months
[GIT PULL] libnvdimm for 4.15
by Williams, Dan J
Hi Linus, please pull from:
git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm tags/libnvdimm-for-4.15
...to receive the libnvdimm and dax update for 4.15.
Save for a few late fixes, all of these commits have shipped in -next
releases since before the merge window opened, and 0day has given a
build success notification. The MAP_SYNC work included some re-
factoring of dax_insert_mapping(), to break out a common
dax_iomap_pfn() helper, that collided with other small changes in
fs/dax.c. A suggested merge resolution for that collision and a few
other minor collisions is included below after the diffstat.
The ext4 touches came from Jan, and the xfs touches have Darrick's
reviewed-by. An xfstest for the MAP_SYNC feature [1] has been through a
few round of reviews and is on track to be merged. The final policy of
how MAP_SHARED_VALIDATE and MAP_SYNC flags behave was discussed by you
and Jan here: [2].
[1]: https://lists.01.org/pipermail/linux-nvdimm/2017-October/012974.ht
ml
[2]: https://lists.01.org/pipermail/linux-nvdimm/2017-October/012894.ht
ml
---
The following changes since commit 8a5776a5f49812d29fe4b2d0a2d71675c3facf3f:
Linux 4.14-rc4 (2017-10-08 20:53:29 -0700)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm tags/libnvdimm-for-4.15
for you to fetch changes up to 4247f24c23589bcc3bc3490515ef8c9497e9ae55:
Merge branch 'for-4.15/dax' into libnvdimm-for-next (2017-11-15 16:56:11 -0800)
----------------------------------------------------------------
libnvdimm for 4.15
* Introduce MAP_SYNC and MAP_SHARED_VALIDATE, a mechanism to enable
'userspace flush' of persistent memory updates via filesystem-dax
mappings. It arranges for any filesystem metadata updates that may be
required to satisfy a write fault to also be flushed ("on disk") before
the kernel returns to userspace from the fault handler. Effectively
every write-fault that dirties metadata completes an fsync() before
returning from the fault handler. The new MAP_SHARED_VALIDATE mapping
type guarantees that the MAP_SYNC flag is validated as supported by the
filesystem's ->mmap() file operation.
* Add support for the standard ACPI 6.2 label access methods that
replace the NVDIMM_FAMILY_INTEL (vendor specific) label methods. This
enables interoperability with environments that only implement the
standardized methods.
* Add support for the ACPI 6.2 NVDIMM media error injection methods.
* Add support for the NVDIMM_FAMILY_INTEL v1.6 DIMM commands for latch
last shutdown status, firmware update, SMART error injection, and
SMART alarm threshold control.
* Cleanup physical address information disclosures to be root-only.
* Fix revalidation of the DIMM "locked label area" status to support
dynamic unlock of the label area.
* Expand unit test infrastructure to mock the ACPI 6.2 Translate SPA
(system-physical-address) command and error injection commands.
Acknowledgements that came after the commits were pushed to -next:
957ac8c421ad dax: fix PMD faults on zero-length files
Reviewed-by: Ross Zwisler <ross.zwisler(a)linux.intel.com>
a39e596baa07 xfs: support for synchronous DAX faults
Reviewed-by: Darrick J. Wong <darrick.wong(a)oracle.com>
7b565c9f965b xfs: Implement xfs_filemap_pfn_mkwrite() using __xfs_filemap_fault()
Reviewed-by: Darrick J. Wong <darrick.wong(a)oracle.com>
----------------------------------------------------------------
Arvind Yadav (1):
dax: pr_err() strings should end with newlines
Christoph Hellwig (1):
xfs: support for synchronous DAX faults
Colin Ian King (1):
libnvdimm, namespace: make a couple of functions static
Dan Williams (18):
libnvdimm, dimm: clear 'locked' status on successful DIMM enable
libnvdimm, region : make 'resource' attribute only readable by root
libnvdimm, namespace: make 'resource' attribute only readable by root
libnvdimm, pfn: make 'resource' attribute only readable by root
libnvdimm, namespace: fix label initialization to use valid seq numbers
acpi, nfit: add support for the _LSI, _LSR, and _LSW label methods
libnvdimm: introduce 'flags' attribute for DIMM 'lock' and 'alias' status
acpi, nfit: hide unknown commands from nmemX/commands
acpi, nfit: add support for NVDIMM_FAMILY_INTEL v1.6 DSMs
mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
acpi, nfit: validate commands against the device type
tools/testing/nvdimm: unit test clear-error commands
fs, dax: unify IOMAP_F_DIRTY read vs write handling policy in the dax core
dax: quiet bdev_dax_supported()
brd: remove dax support
dax: stop requiring a live device for dax_flush()
acpi, nfit: add 'Enable Latch System Shutdown Status' command support
Merge branch 'for-4.15/dax' into libnvdimm-for-next
Dave Jiang (2):
libnvdimm: move poison list functions to a new 'badrange' file
nfit_test: add error injection DSMs
Jan Kara (17):
mm: Handle 0 flags in _calc_vm_trans() macro
mm: Remove VM_FAULT_HWPOISON_LARGE_MASK
dax: Simplify arguments of dax_insert_mapping()
dax: Factor out getting of pfn out of iomap
dax: Create local variable for VMA in dax_iomap_pte_fault()
dax: Create local variable for vmf->flags & FAULT_FLAG_WRITE test
dax: Inline dax_insert_mapping() into the callsite
dax: Inline dax_pmd_insert_mapping() into the callsite
dax: Fix comment describing dax_iomap_fault()
dax: Allow dax_iomap_fault() to return pfn
dax: Allow tuning whether dax_insert_mapping_entry() dirties entry
mm: Define MAP_SYNC and VM_SYNC flags
dax, iomap: Add support for synchronous faults
dax: Implement dax_finish_sync_fault()
ext4: Simplify error handling in ext4_dax_huge_fault()
ext4: Support for synchronous DAX faults
xfs: Implement xfs_filemap_pfn_mkwrite() using __xfs_filemap_fault()
Jeff Moyer (1):
dax: fix PMD faults on zero-length files
Mikulas Patocka (1):
dax: fix general protection fault in dax_alloc_inode
Ross Zwisler (2):
MAINTAINERS: Add entry for device DAX
dev/dax: fix uninitialized variable build warning
Vishal Verma (3):
libnvdimm, badrange: remove a WARN for list_empty
nfit_test: when clearing poison, also remove badrange entries
tools/testing/nvdimm: stricter bounds checking for error injection commands
Yasunori Goto (3):
nfit_test Make private definitions to command emulation
acpi nfit: Enable to show what feature is supported via ND_CMD_CALL for nfit_test
acpi nfit: nfit_test supports translate SPA
MAINTAINERS | 8 +-
arch/alpha/include/uapi/asm/mman.h | 1 +
arch/mips/include/uapi/asm/mman.h | 1 +
arch/parisc/include/uapi/asm/mman.h | 1 +
arch/xtensa/include/uapi/asm/mman.h | 1 +
drivers/acpi/nfit/core.c | 274 ++++++++++++++++++++++-
drivers/acpi/nfit/mce.c | 2 +-
drivers/acpi/nfit/nfit.h | 37 +++-
drivers/block/Kconfig | 12 -
drivers/block/brd.c | 65 ------
drivers/dax/device.c | 3 +-
drivers/dax/super.c | 14 +-
drivers/nvdimm/Makefile | 1 +
drivers/nvdimm/badrange.c | 293 ++++++++++++++++++++++++
drivers/nvdimm/bus.c | 24 +-
drivers/nvdimm/core.c | 260 +---------------------
drivers/nvdimm/dimm.c | 3 +
drivers/nvdimm/dimm_devs.c | 19 ++
drivers/nvdimm/label.c | 2 +-
drivers/nvdimm/namespace_devs.c | 6 +-
drivers/nvdimm/nd-core.h | 3 +-
drivers/nvdimm/nd.h | 7 +-
drivers/nvdimm/pfn_devs.c | 8 +
drivers/nvdimm/region_devs.c | 8 +-
fs/dax.c | 319 ++++++++++++++++++---------
fs/ext2/file.c | 2 +-
fs/ext4/file.c | 26 ++-
fs/ext4/inode.c | 15 ++
fs/jbd2/journal.c | 17 ++
fs/proc/task_mmu.c | 1 +
fs/xfs/xfs_file.c | 44 ++--
fs/xfs/xfs_iomap.c | 5 +
fs/xfs/xfs_trace.h | 2 -
include/linux/dax.h | 4 +-
include/linux/fs.h | 1 +
include/linux/iomap.h | 5 +
include/linux/jbd2.h | 1 +
include/linux/libnvdimm.h | 21 +-
include/linux/mm.h | 9 +-
include/linux/mman.h | 48 +++-
include/trace/events/fs_dax.h | 3 +-
include/uapi/asm-generic/mman-common.h | 1 +
include/uapi/asm-generic/mman.h | 1 +
mm/mmap.c | 15 ++
tools/include/uapi/asm-generic/mman-common.h | 1 +
tools/testing/nvdimm/Kbuild | 1 +
tools/testing/nvdimm/test/nfit.c | 319 ++++++++++++++++++++++++---
tools/testing/nvdimm/test/nfit_test.h | 52 +++++
48 files changed, 1406 insertions(+), 560 deletions(-)
create mode 100644 drivers/nvdimm/badrange.c
---
commit 82f3359eb04e3a3b5d23655eee58d31a1b17c902
Merge: 18c83d2c0390 4247f24c2358
Author: Dan Williams <dan.j.williams(a)intel.com>
Date: Thu Nov 16 13:20:35 2017 -0800
Merge branch 'libnvdimm-for-next' into test
diff --cc drivers/block/brd.c
index 588360d79fca,b2391bbd7e5a..8028a3a7e7fd
--- a/drivers/block/brd.c
+++ b/drivers/block/brd.c
@@@ -20,12 -20,6 +20,7 @@@
#include <linux/radix-tree.h>
#include <linux/fs.h>
#include <linux/slab.h>
+#include <linux/backing-dev.h>
- #ifdef CONFIG_BLK_DEV_RAM_DAX
- #include <linux/pfn_t.h>
- #include <linux/dax.h>
- #include <linux/uio.h>
- #endif
#include <linux/uaccess.h>
@@@ -449,23 -401,9 +401,10 @@@ static struct brd_device *brd_alloc(in
disk->flags = GENHD_FL_EXT_DEVT;
sprintf(disk->disk_name, "ram%d", i);
set_capacity(disk, rd_size * 2);
+ disk->queue->backing_dev_info->capabilities |= BDI_CAP_SYNCHRONOUS_IO;
- #ifdef CONFIG_BLK_DEV_RAM_DAX
- queue_flag_set_unlocked(QUEUE_FLAG_DAX, brd->brd_queue);
- brd->dax_dev = alloc_dax(brd, disk->disk_name, &brd_dax_ops);
- if (!brd->dax_dev)
- goto out_free_inode;
- #endif
-
-
return brd;
- #ifdef CONFIG_BLK_DEV_RAM_DAX
- out_free_inode:
- kill_dax(brd->dax_dev);
- put_dax(brd->dax_dev);
- #endif
out_free_queue:
blk_cleanup_queue(brd->brd_queue);
out_free_dev:
diff --cc fs/dax.c
index 3652b26a0048,f757cd0e2d07..95981591977a
--- a/fs/dax.c
+++ b/fs/dax.c
@@@ -825,38 -820,42 +825,42 @@@ out
}
EXPORT_SYMBOL_GPL(dax_writeback_mapping_range);
- static int dax_insert_mapping(struct address_space *mapping,
- struct block_device *bdev, struct dax_device *dax_dev,
- sector_t sector, size_t size, void *entry,
- struct vm_area_struct *vma, struct vm_fault *vmf)
+ static sector_t dax_iomap_sector(struct iomap *iomap, loff_t pos)
{
- unsigned long vaddr = vmf->address;
- void *ret, *kaddr;
- return iomap->blkno + (((pos & PAGE_MASK) - iomap->offset) >> 9);
++ return (iomap->addr + (pos & PAGE_MASK) - iomap->offset) >> 9;
+ }
+
+ static int dax_iomap_pfn(struct iomap *iomap, loff_t pos, size_t size,
+ pfn_t *pfnp)
+ {
+ const sector_t sector = dax_iomap_sector(iomap, pos);
pgoff_t pgoff;
+ void *kaddr;
int id, rc;
- pfn_t pfn;
+ long length;
- rc = bdev_dax_pgoff(bdev, sector, size, &pgoff);
+ rc = bdev_dax_pgoff(iomap->bdev, sector, size, &pgoff);
if (rc)
return rc;
-
id = dax_read_lock();
- rc = dax_direct_access(dax_dev, pgoff, PHYS_PFN(size), &kaddr, &pfn);
- if (rc < 0) {
- dax_read_unlock(id);
- return rc;
+ length = dax_direct_access(iomap->dax_dev, pgoff, PHYS_PFN(size),
+ &kaddr, pfnp);
+ if (length < 0) {
+ rc = length;
+ goto out;
}
+ rc = -EINVAL;
+ if (PFN_PHYS(length) < size)
+ goto out;
+ if (pfn_t_to_pfn(*pfnp) & (PHYS_PFN(size)-1))
+ goto out;
+ /* For larger pages we need devmap */
+ if (length > 1 && !pfn_t_devmap(*pfnp))
+ goto out;
+ rc = 0;
+ out:
dax_read_unlock(id);
-
- ret = dax_insert_mapping_entry(mapping, vmf, entry, sector, 0);
- if (IS_ERR(ret))
- return PTR_ERR(ret);
-
- trace_dax_insert_mapping(mapping->host, vmf, ret);
- if (vmf->flags & FAULT_FLAG_WRITE)
- return vm_insert_mixed_mkwrite(vma, vaddr, pfn);
- else
- return vm_insert_mixed(vma, vaddr, pfn);
+ return rc;
}
/*
diff --cc fs/ext4/inode.c
index 8d2b582fb141,ee4d907a4251..0992d76f7ab1
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@@ -3384,6 -3393,20 +3384,19 @@@ static int ext4_releasepage(struct pag
return try_to_free_buffers(page);
}
-#ifdef CONFIG_FS_DAX
+ static bool ext4_inode_datasync_dirty(struct inode *inode)
+ {
+ journal_t *journal = EXT4_SB(inode->i_sb)->s_journal;
+
+ if (journal)
+ return !jbd2_transaction_committed(journal,
+ EXT4_I(inode)->i_datasync_tid);
+ /* Any metadata buffers to write? */
+ if (!list_empty(&inode->i_mapping->private_list))
+ return true;
+ return inode->i_state & I_DIRTY_DATASYNC;
+ }
+
static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
unsigned flags, struct iomap *iomap)
{
diff --cc include/linux/iomap.h
index ca10767ab73d,73e3b7085dbe..d187cf7c4757
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@@ -22,8 -21,12 +22,13 @@@ struct vm_fault
/*
* Flags for all iomap mappings:
*/
-#define IOMAP_F_NEW 0x01 /* blocks have been newly allocated */
+#define IOMAP_F_NEW 0x01 /* blocks have been newly allocated */
+#define IOMAP_F_BOUNDARY 0x02 /* mapping ends at metadata boundary */
+ /*
+ * IOMAP_F_DIRTY indicates the inode has uncommitted metadata needed to access
+ * written data and requires fdatasync to commit them to persistent storage.
+ */
-#define IOMAP_F_DIRTY 0x02
++#define IOMAP_F_DIRTY 0x04
/*
* Flags that only need to be reported for IOMAP_REPORT requests:
3 years, 2 months
[ndctl PATCH 1/4] ndctl, create-namespace: fix --map= reconfiguration for device-dax
by Dan Williams
Allow the memmap location to be changed when reconfiguring a device-dax
mode namespace.
Fixes: 3139dfda1158 ("ndctl: create namespace")
Signed-off-by: Dan Williams <dan.j.williams(a)intel.com>
---
ndctl/namespace.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/ndctl/namespace.c b/ndctl/namespace.c
index 58f23ad0877a..0020be354670 100644
--- a/ndctl/namespace.c
+++ b/ndctl/namespace.c
@@ -198,7 +198,8 @@ static int set_defaults(enum device_action mode)
}
if (!param.reconfig && param.mode
- && strcmp(param.mode, "memory") != 0) {
+ && strcmp(param.mode, "memory") != 0
+ && strcmp(param.mode, "dax") != 0) {
error("--map only valid for a memory mode pmem namespace\n");
rc = -EINVAL;
}
3 years, 2 months
