Hi, On Thu, 20 Apr 2023 at 20:02:27 +0200, Cyril Brulebois wrote: >> * Backport upstream MR !498, let it mature in sid for a few >> weeks then upload 2:2.6.1-4~deb12u1 via t-p-u. There are only 2 >> upstream commits to cherry-pick and neither is large nor intrusive; >> moreover like the commits previously cherry-picked they are no-op on >> “normal” systems (only systems without swap are affected). For >> convenience I attach a debdiff for 2:2.6.1-3~deb12u2 and you'll also >> find binary packages for amd64 at >> https://people.debian.org/~guilhem/tmp/cryptsetup_2.6.1-3~deb12u2/ >> Tested: autopkgtests (incl. full upstream test suite), d-i in both >> graphical and text install on VMs with 1024M RAM (now memory cost >> won't exceed ~250M resp. ~300M thus leaving plenty of headroom for >> the rest). > > Since you're happy with that approach, let's go for an upload to > unstable for the time being, I'll conduct some tests shortly, and once > it's indeed confirmed to work fine, go via t-p-u (because of the same > fun as before with some library) so that it can be used for rc3 (if it's > ready by then — we haven't really defined when it's going to happen > besides “somewhen before end of April”). Just uploaded 2:2.6.1-4 to sid, and locally prepared a rebuild for bookworm (2:2.6.1-4~deb12u1). Comparing PBKDF benchmark results obtained using default settings (guided “encrypted LVM” partitioning scheme) between the last 3 releases and 1, 2, or 4G RAM (the first luksDump is what I got out of d-i, the second shows benchmark results on the final system — with swap), I get the following parameters (summary at the bottom). Buster (debian-10.12.0-amd64-netinst.iso, text install), 1024M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 504962 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 8 Memory: 505350 Threads: 2 Buster (debian-10.12.0-amd64-netinst.iso, text install), 2048M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 538914 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 1021446 Threads: 2 Buster (debian-10.12.0-amd64-netinst.iso, text install), 4096M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 533886 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 1048576 Threads: 2 Bullseye (debian-11.6.0-amd64-netinst.iso, text install), 1024M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 499892 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 8 Memory: 499888 Threads: 2 Bullseye (debian-11.6.0-amd64-netinst.iso, text install), 2048M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 582804 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 1015216 Threads: 2 Bullseye (debian-11.6.0-amd64-netinst.iso, text install), 4096M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 518981 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2i Time cost: 4 Memory: 948373 Threads: 2 Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso, text install), 1024M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 5 Memory: 489820 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 8 Memory: 490598 Threads: 2 Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso, text install), 2048M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 4 Memory: 553835 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 4 Memory: 1005926 Threads: 2 Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso, text install), 4096M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 4 Memory: 546642 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 4 Memory: 1048576 Threads: 2 Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup 2:2.6.1-4~deb12u1, graphical install), 1024M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 10 Memory: 223780 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 8 Memory: 490598 Threads: 2 Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup 2:2.6.1-4~deb12u1, text install), 1024M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 8 Memory: 294302 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 8 Memory: 490598 Threads: 2 Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup 2:2.6.1-4~deb12u1, text install), 2048M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 4 Memory: 590553 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 4 Memory: 1005926 Threads: 2 Bookworm (debian-bookworm-DI-rc1-amd64-netinst.iso + cryptsetup 2:2.6.1-4~deb12u1, text install), 4096M RAM: root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 4 Memory: 613826 Threads: 2 root@debian:~# cryptsetup luksConvertKey /dev/vda5 <<<test root@debian:~# cryptsetup luksDump /dev/vda5 | grep -A3 PBKDF PBKDF: argon2id Time cost: 4 Memory: 1048576 Threads: 2 Bottom line: * The upstream patches in the patch-queue (the 2 backported earlier from upstream MR !490 plus the new other two from upstream MR !498) only affect systems with <2G RAM (i.e., those where half the amount of physical memory is lower than DEFAULT_LUKS2_MEMORY_KB). And only those without swap. On such systems the memory cost is set to a lower value at the expense of a higher time cost, which is the intended behavior; it appear to leave enough head-room for the graphical installer to succeed with 1G RAM, so I believe the errata can be removed if the changes makes it to bookworm. * I was surprised to see the memory cost settle at ~550-600M on systems with a decent amount of RAM in d-i. Would have expected to see 1G here just like after running `cryptsetup luksConvertKey` in the normal system. I get a similarily low memory cost after dropping to a rescue shell early in d-i and running `luksFormat` manually: ~ # grep -c ^processor /proc/cpuinfo 6 ~ # free total used free shared buff/cache available Mem: 6062584 107888 5647804 260000 306892 5543168 Swap: 0 0 0 ~ # echo test | cryptsetup luksFormat --debug --batch-mode /dev/sda […] # Running argon2id() benchmark. # PBKDF benchmark: memory cost = 65536, iterations = 4, threads = 4 (took 229 ms) # PBKDF benchmark: memory cost = 71545, iterations = 4, threads = 4 (took 242 ms) # PBKDF benchmark: memory cost = 73910, iterations = 4, threads = 4 (took 249 ms) # PBKDF benchmark: memory cost = 74206, iterations = 4, threads = 4 (took 246 ms) # PBKDF benchmark: memory cost = 75412, iterations = 4, threads = 4 (took 254 ms) # PBKDF benchmark: memory cost = 593795, iterations = 4, threads = 4 (took 3527 ms) # PBKDF benchmark: memory cost = 336713, iterations = 4, threads = 4 (took 1196 ms) # PBKDF benchmark: memory cost = 563065, iterations = 4, threads = 4 (took 2035 ms) # Benchmark returns argon2id() 4 iterations, 563065 memory, 4 threads (for 512-bits key). […] I think what happens here is that compared to the final system d-i is a bit crippled so the 2s threshold is reached earlier in the benchmark. For comparison, running the benchmark in the initramfs shell of the final system (after installation, but also without swap): (initramfs) free total used free shared buff/cache available Mem: 6064140 66752 5797144 56 200244 5675728 Swap: 0 0 0 (initramfs) echo test | cryptsetup luksConvertKey --debug --batch-mode /dev/sda5 […] # Running argon2id() benchmark. # PBKDF benchmark: memory cost = 65536, iterations = 4, threads = 4 (took 94 ms) # PBKDF benchmark: memory cost = 174297, iterations = 4, threads = 4 (took 239 ms) # PBKDF benchmark: memory cost = 182319, iterations = 4, threads = 4 (took 242 ms) # PBKDF benchmark: memory cost = 188346, iterations = 4, threads = 4 (took 243 ms) # PBKDF benchmark: memory cost = 193771, iterations = 4, threads = 4 (took 232 ms) # PBKDF benchmark: memory cost = 208804, iterations = 4, threads = 4 (took 274 ms) # PBKDF benchmark: memory cost = 1048576, iterations = 5, threads = 4 (took 1721 ms) # Benchmark returns argon2id() 5 iterations, 1048576 memory, 4 threads (for 512-bits key). […] And now in the final system fully booted (same result as in initramfs): root@debian:~# free -h total used free shared buff/cache available Mem: 5.8Gi 270Mi 5.6Gi 476Ki 78Mi 5.5Gi Swap: 975Mi 0B 975Mi root@debian:~# cryptsetup luksConvertKey --debug --batch-mode /dev/sda5 <<<test […] # Running argon2id() benchmark. # PBKDF benchmark: memory cost = 65536, iterations = 4, threads = 4 (took 93 ms) # PBKDF benchmark: memory cost = 176172, iterations = 4, threads = 4 (took 248 ms) # PBKDF benchmark: memory cost = 177592, iterations = 4, threads = 4 (took 242 ms) # PBKDF benchmark: memory cost = 183462, iterations = 4, threads = 4 (took 226 ms) # PBKDF benchmark: memory cost = 202944, iterations = 4, threads = 4 (took 274 ms) # PBKDF benchmark: memory cost = 1048576, iterations = 5, threads = 4 (took 1795 ms) # Benchmark returns argon2id() 5 iterations, 1048576 memory, 4 threads (for 512-bits key). […] Never noticed that before, but that's not a regression since buster and bullseye both have the same behavior. (At least in my test VMs; didn't compare on real hardware.) Cheers -- Guilhem.
Attachment:
signature.asc
Description: PGP signature