Two servers hung last night. Both events happened while a file was being copied into the btrfs file system. No references or vm's were involved. It was a simple file copy.
I had the opportunity to observe the system while it happened. It turns out that the CPU time goes to 50% or more I/O wait. The servers have 6 cores/12 threads running at 3.5GHz. It is like the operating system goes into an infinite loop, banging away at the disk. Letting it sit for hours does not resolve the problem. The only solution I'm aware of at this time is to reboot the server. Restarting the copy process after a reboot sometimes triggers the same bug, and sometimes it works fine.
--
Russell Mosemann