Dockerfile

[nsailor]

Initial attempt to write a Dockerfile for AMDGPU.jl, based on the latest Ubuntu ROCm image.

It seems to work (at least for the simple vector sum in the documentation), however, you have to ignore several error messages.

Unfortunately, the build fails with:

 Installing known registries into `~/.julia`
######################################################################## 100.0%
      Added registry `General` to `~/.julia/registries/General`
  Resolving package versions...
  Installed AbstractFFTs ─────── v0.5.0
  Installed TimerOutputs ─────── v0.5.6
  Installed ConstructionBase ─── v1.0.0
  Installed Compat ───────────── v3.15.0
  Installed BinaryProvider ───── v0.5.10
  Installed DataStructures ───── v0.18.2
  Installed LLVM ─────────────── v3.0.0
  Installed AMDGPU ───────────── v0.2.0
  Installed CEnum ────────────── v0.4.1
  Installed MacroTools ───────── v0.5.5
  Installed Adapt ────────────── v2.0.2
  Installed Requires ─────────── v1.0.1
  Installed GPUArrays ────────── v5.1.0
  Installed Setfield ─────────── v0.7.0
  Installed GPUCompiler ──────── v0.7.1
  Installed OrderedCollections ─ v1.3.0
Updating `~/.julia/environments/v1.5/Project.toml`
  [21141c5a] + AMDGPU v0.2.0
Updating `~/.julia/environments/v1.5/Manifest.toml`
  [21141c5a] + AMDGPU v0.2.0
  [621f4979] + AbstractFFTs v0.5.0
  [79e6a3ab] + Adapt v2.0.2
  [b99e7846] + BinaryProvider v0.5.10
  [fa961155] + CEnum v0.4.1
  [34da2185] + Compat v3.15.0
  [187b0558] + ConstructionBase v1.0.0
  [864edb3b] + DataStructures v0.18.2
  [0c68f7d7] + GPUArrays v5.1.0
  [61eb1bfa] + GPUCompiler v0.7.1
  [929cbde3] + LLVM v3.0.0
  [1914dd2f] + MacroTools v0.5.5
  [bac558e1] + OrderedCollections v1.3.0
  [ae029012] + Requires v1.0.1
  [efcf1570] + Setfield v0.7.0
  [a759f4b9] + TimerOutputs v0.5.6
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [9fa8497b] + Future
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
   Building AMDGPU → `~/.julia/packages/AMDGPU/nnddY/deps/build.log`
┌ Error: Error building `AMDGPU`:
│ paths = ["/opt/rocm/hsa/lib"]
│ Initializing HSA runtime failed with code 4104.
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Operations.jl:949
   Building AMDGPU → `~/.julia/packages/AMDGPU/nnddY/deps/build.log`
┌ Error: Error building `AMDGPU`:
│ paths = ["/opt/rocm/hsa/lib"]
│ Initializing HSA runtime failed with code 4104.
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Operations.jl:949

[jpsamaroo]

How did you run the container? You need to pass a number of flags to ensure HSA support in the container.

[jpsamaroo]

LGTM once it's working 😄

[jpsamaroo]

It would be better if we could run this as a non-privileged user, like gpuuser. You'd just have to make sure /dev/kfd and /dev/dri/* have the right perms so the user has RW on them.

[jpsamaroo]

Here's how I invoke my local development container (with a similar Dockerfile):

docker run --rm -it --device=/dev/kfd --device=/dev/dri \
           --group-add=video --security-opt seccomp=unconfined \
           --privileged=true --cap-add=SYS_PTRACE \
           "$@" rocm-jps

It may not actually require --privileged=true anymore, I haven't checked recently. Putting this in a script also lets me add volume mounts for faster development.