Linux 4.9 has been released
Summary: This release adds support for shared extents ({{{cp --reflink}}} support) and copy-on-write support on XFS; virtually mapped kernel stacks that make the kernel more reliable and secure; a more efficient BPF profiler that brings Linux on part with Dtrace; a new optional BBR TCP congestion control algorithm based on bandwidth measurements instead of packet loss; syscalls to use the protection keys hardware feature; support for the Greybus bus from Project Ara; a hardware latency tracer to detect firmware-induced latencies, and many other improvements and new drivers.
Shared data extents + copy-on-write support on XFS
This release adds several key features to the XFS file system, based on the reverse mapping work introduced in the past release
This release adds the ability to share data extents between different files. That is, support for {{{cp --reflink=always}}}, the ability to deduplicate data, and the ability to unshared data through the {{{FALLOC_FL_UNSHARE}}} fallocate(2) interface.
It also adds copy-on-write support for data: instead of overwriting data, it copies data to a new location.
All these features are a huge chunk of new experimental functionality with new on-disk format features and internal infrastructure.
Code: merge
Virtually mapped stacks for more secure and reliable stack handling
The Linux kernel has always mapped the memory used by kernel stacks directly in the kernel memory, an approach that makes harder to allocate stacks under memory load and proves no protection against stack overflows. This release allows to map the kernel stacks in virtual memory, which makes easier to allocate stacks under memory pressure, provides protection against stack overflows, and other benefits.
Recommended LWN articles: Virtually mapped kernel stacks
Code: commit
Efficient BPF-based profiler
This release includes the necessary infrastructure to allow BPF programs to be atached to hardware and software perf events, which in turn allows the kernel to profile via timed sampling and summarize stack traces. For more details, see Brendan Gregg's blog post
Code: commit
BBR TCP congestion control algorithm
This release adds another TCP congestion control algorithm: BBR (Bottleneck Bandwidth and RTT). The Internet has predominantly used loss-based congestion control, relying on packet loss as the signal to slow down. While this worked well for many years, loss-based congestion control is considered -by BBR authors- to be out-dated in today's networks. On today's Internet, loss-based congestion control causes the infamous bufferbloat problem, often causing seconds of needless queuing delay, since it fills the bloated buffers in many last-mile links. On today's high-speed long-haul links using commodity switches with shallow buffers, loss-based congestion control has abysmal throughput because it over-reacts to losses caused by transient traffic bursts.
BBR creates an explicit model of the network pipe by sequentially probing the bottleneck bandwidth and RTT. On the arrival of each ACK, BBR derives the current delivery rate of the last round trip, and feeds it through a windowed max-filter to estimate the bottleneck bandwidth. Conversely it uses a windowed min-filter to estimate the round trip propagation delay. The max-filtered bandwidth and min-filtered RTT estimates form BBR's model of the network pipe. BBR has significantly increased throughput and reduced latency for connections on Google's internal backbone networks and google.com and YouTube Web servers.
ACM Queue paper: BBR: Congestion-Based Congestion Control
Recommended LWN article: BBR congestion control
Code: commit
Protection keys syscall support
Protection keys is a memory protection hardware feature merged in Linux 4.6
This release adds new syscalls that offer a more complete API to use protection keys. For more details, see the recommended LWN article: System calls for memory protection keys
Code: commit
Greybus support
This release adds support for Greybus, a new subsystem that brings support for the Greybus bus: a bus designed for the modular, hot-pluggable hardware of the (now dead) Ara Project. Despite the dead of the main project, the code is still being used
Recommended LWN article: Greybus
Code: drivers/staging/greybus
Hardware latency tracer
Coming from the real-time patchset, the hardware latency tracer is a special purpose tracer that is used to detect large system latencies induced by the behavior of certain underlying hardware or firmware interruptions, like SMIs (System Management Interrupts) on x86 systems, that the kernel is unaware of. The kernel does not even know these interruptions are happening. SMIs are instead set up by BIOS code and are serviced by BIOS code, usually for "critical" events such as management of thermal sensors and fans.
The hardware latency detector works by simply creating a thread that spins on a single CPU polling the CPU Time Stamp Counter for a specified amount of time (width) within a periodic window (window), and trying to find gaps where the polling was interrupted. This is useful for testing if a system is reliable for Real Time tasks.
Documentation: Documentation/trace/hwlat_detector.txt
Code: commit