Re: HIP ISA Compatibility Unlocked

[Cordell Bloor <cgmb@slerp.xyz>]

Hi Gavin,

On 2024-02-21 07:35, Gavin Zhao wrote:

>       
>       Hi Cory,
>         
>         This is Gavin. I messaged you a few weeks ago regarding the
>         possibility of upstreaming patches that extends HIP ISA
>         compatibility.
>         
>         Recently, I was able to patch ROCm-CompilerSupport, rocBLAS,
>         MIOpen, and rocFFT so that combining with your patches to the
>         HIP runtime, I can run Stable Diffusion (PyTorch) and llama.cpp
>         on my RX6600M (gfx1032) without HSA_OVERRIDE_GFX_VERSION.
>         Hopefully these efforts would help Debian and other distros to
>         also get this compatibility support as well. My patches are uploaded to
>         my personal GitHub fork GZGavinZhao/<component-name> with
>         branch name solus-rocm-<version>. So, the patches for
>         rocBLAS 6.0 are in the solus-rocm-6.0.0 branch of
>         GZGavinZhao/rocBLAS. 
>         
>         Please note that these patches haven't been thoroughly tested
>         and the ROCm 6.0 packages are not in Solus's repo yet, so please
>         let me know if there are any issues. In addition, the patches
>         may need to be adapted to ROCm 5.7 since ROCm 6.0 doesn't have
>         the int gcnArch as a member of hipDeviceProp_t, instead it has
>         the string gcnArchName, which complicates the arch coercion
>         logic a bit. ROCm 6.0 also requires an additional patch to ROCm-CompilerSupport, so
>         you may want to double check if 5.7 needs that too.
>         
>         Lastly, I have a working PR
>         for sccache that can cache HIP
>         compilations. Combining with a patch
>         for HIPCC I was able to greatly speed up my workflow by
>         exporting the environment variable HIP_CLANG_LAUNCHER=sccache so
>         that I can rapidly edit, clean, and rebuild. I'm not too
>         familiar with how Debian packaging works, but if you've also
>         been bothered by the long compilation times hopefully this can
>         help.
>         
>         Have a great day :)
>         
>         Best,
>         Gavin
>         
>         P.S. I'm not sure if the rocFFT patch is even needed because
>         rocFFT has an "any" fallback arch. Perhaps the patch
>         would allow rocFFT to select the most optimal algorithm instead
>         of the generic fallback one but I haven't benchmarked it.
>

Thanks for the information. I'm sharing this message with the Debian AI mailing list as per our discussion, because I think your patches are highly relevant to the interests of the Debian ROCm Team.

I'm a little overwhelmed with work at the moment, but you're touching on a lot of areas that I've been studying over the past few years. I can provide a lot of feedback once I have a moment to take a breath and organize my thoughts.

Sincerely,
Cory Bloor