A static checker pointed out, that bo_va->base.bo was already derefenced
earlier in the same scope. Therefore this check is unnecessary here.
Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
Fixes: 50661eb1a2 ("drm/amdgpu: Auto-validate DMABuf imports in compute VMs")
Reviewed-by: Kent Russell <kent.russell@amd.com>
Signed-off-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
DMABuf imports in compute VMs are not wrapped in a kgd_mem object on the
process_info->kfd_bo_list. There is no explicit KFD API call to validate
them or add eviction fences to them.
This patch automatically validates and fences dymanic DMABuf imports when
they are added to a compute VM. Revalidation after evictions is handled
in the VM code.
v2:
* Renamed amdgpu_vm_validate_evicted_bos to amdgpu_vm_validate
* Eliminated evicted_user state, use evicted state for VM BOs and user BOs
* Fixed and simplified amdgpu_vm_fence_imports, depends on reserved BOs
* Moved dma_resv_reserve_fences for amdgpu_vm_fence_imports into
amdgpu_vm_validate, outside the vm->status_lock
* Added dummy version of amdgpu_amdkfd_bo_validate_and_fence for builds
without KFD
v4: Eliminate amdgpu_vm_fence_imports. It's not needed because the
reservation with its fences is shared with the export, as long as all
imports are from KFD, with the exports already reserved, validated and
fenced by the KFD restore worker.
v5: Reintroduced separate evicted_user state to simplify the state machine
and CS error handling when amdgpu_vm_validate is called without a ticket.
Signed-off-by: Felix Kuehling <felix.kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Idle bo's PTE needs to be re-created when resetting VM state machine.
Set idle bo's vm_bo as moved to mark it as invalid.
Fixes: 55bf196f60 ("drm/amdgpu: reset VM when an error is detected")
Signed-off-by: ZhenGuo Yin <zhenguo.yin@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Fixes the below:
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c:1373: warning: Function parameter or member 'xcc_mask' not described in 'amdgpu_vm_flush_compute_tlb'
Cc: Felix Kuehling <Felix.Kuehling@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Pan, Xinhui" <Xinhui.Pan@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This will make it possible for amdgpu GEM ioctls to flush TLBs on compute
VMs.
This removes VMID-based TLB flushing and always uses PASID-based
flushing. This still works because it scans the VMID-PASID mapping
registers to find the right VMID. It's only slightly less efficient. This
is not a production use case.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When clearing the root PD fails we need to properly release it again.
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
mem = bo->tbo.resource may be NULL in amdgpu_vm_bo_update.
Fixes: 1802537820 ("drm/ttm: stop allocating dummy resources during BO creation")
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
Let amdgpu_vm_handle_moved update all BO VA mappings of BOs reserved by
the caller. This will be useful for handling extra BO VA mappings in
KFD VMs that are managed through the render node API.
v2: rebase against drm_exec changes (Alex)
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On gfx943 APU, EXT_COHERENT should give MTYPE_CC for local and
MTYPE_UC for nonlocal memory.
On NUMA systems, local memory gets the local mtype, set by an
override callback. If EXT_COHERENT is set, memory will be set as
MTYPE_UC by default, with local memory MTYPE_CC.
Add an option in the override function for this case, and
add a check to ensure it is not used on UNCACHED memory.
V2: Combined APU and NUMA code into one patch
V3: Fixed a potential nullptr in amdgpu_vm_bo_update
Signed-off-by: David Francis <David.Francis@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
-----BEGIN PGP SIGNATURE-----
iQFSBAABCAA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAmU1ngkeHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGrsIH/0k/+gdBBYFFdEym
foRhKir9WV3ZX4oIozJjA1f7T+qVYclKs6kaYm3gNepRBb6AoG8pdgv4MMAqhYsf
QMe2XHi0MrO/qKBgfNfivxEa9jq+0QK5uvTbqCRqCAB8LfwVyDqapCmg3EuiZcPW
UbMITmnwLIfXgPxvp9rabmCsTqO6FLbf0GDOVIkNSAIDBXMpcO1iffjrWUbhRa7n
oIoiJmWJLcXLxPWDsRKbpJwzw2cIG08YhfQYAiQnC3YaeRm1FKLDIICRBsmfYzja
rWv9r4dn4TDfV4/AnjggQnsZvz2yPCxNaFSQIT88nIeiLvyuUTJ9j8aidsSfMZQf
xZAbzbA=
=NoQv
-----END PGP SIGNATURE-----
BackMerge tag 'v6.6-rc7' into drm-next
This is needed to add the msm pr which is based on a higher base.
Signed-off-by: Dave Airlie <airlied@redhat.com>
abo->tbo.resource may be NULL in amdgpu_vm_bo_update.
Fixes: 1802537820 ("drm/ttm: stop allocating dummy resources during BO creation")
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Don't update the fault cache if status is 0. In the multiple
fault case, subsequent faults will return a 0 status which is
useless for userspace and replaces the useful fault status, so
only update if status is non-0.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When we get a GPU page fault, cache the fault for later
analysis.
Cc: samuel.pitoiset@gmail.com
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Guchun Chen <guchun.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use an inline function for version check. Gives more flexibility to
handle any format changes.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Merge all developer debug options available as separated module
parameters in one, making it obvious that are for developers.
Drop the obsolete module options in favor of the new ones.
Signed-off-by: André Almeida <andrealmeid@igalia.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Hamza Mahfooz <hamza.mahfooz@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use min_t to replace min, min_t is a bit fast because min use
twice typeof.
And using min_t is cleaner here since the min/max macros
do a typecheck while min_t()/max_t() to an explicit type cast.
Fixes the below checkpatch warning:
WARNING: min() should probably be min_t()
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Pan, Xinhui" <Xinhui.Pan@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Recent code set xcp_id stored from file private data when opening
device to amdgpu bo for accounting memory usage etc, but not all
VMs are attached to this fpriv structure like the vm cases in
amdgpu_mes_self_test, otherwise, KASAN will complain below out
of bound access. And more importantly, VM code should not touch
fpriv structure, so drop fpriv code handling from amdgpu_vm_pt.
[ 77.292314] BUG: KASAN: slab-out-of-bounds in amdgpu_vm_pt_create+0x17e/0x4b0 [amdgpu]
[ 77.293845] Read of size 4 at addr ffff888102c48a48 by task modprobe/1069
[ 77.294146] Call Trace:
[ 77.294178] <TASK>
[ 77.294208] dump_stack_lvl+0x49/0x63
[ 77.294260] print_report+0x16f/0x4a6
[ 77.294307] ? amdgpu_vm_pt_create+0x17e/0x4b0 [amdgpu]
[ 77.295979] ? kasan_complete_mode_report_info+0x3c/0x200
[ 77.296057] ? amdgpu_vm_pt_create+0x17e/0x4b0 [amdgpu]
[ 77.297556] kasan_report+0xb4/0x130
[ 77.297609] ? amdgpu_vm_pt_create+0x17e/0x4b0 [amdgpu]
[ 77.299202] __asan_load4+0x6f/0x90
[ 77.299272] amdgpu_vm_pt_create+0x17e/0x4b0 [amdgpu]
[ 77.300796] ? amdgpu_init+0x6e/0x1000 [amdgpu]
[ 77.302222] ? amdgpu_vm_pt_clear+0x750/0x750 [amdgpu]
[ 77.303721] ? preempt_count_sub+0x18/0xc0
[ 77.303786] amdgpu_vm_init+0x39e/0x870 [amdgpu]
[ 77.305186] ? amdgpu_vm_wait_idle+0x90/0x90 [amdgpu]
[ 77.306683] ? kasan_set_track+0x25/0x30
[ 77.306737] ? kasan_save_alloc_info+0x1b/0x30
[ 77.306795] ? __kasan_kmalloc+0x87/0xa0
[ 77.306852] amdgpu_mes_self_test+0x169/0x620 [amdgpu]
v2: without specifying xcp partition for PD/PT bo, the xcp id is -1.
Link: https://gitlab.freedesktop.org/drm/amd/-/issues/2686
Fixes: 3ebfd221c1 ("drm/amdkfd: Store xcp partition id to amdgpu bo")
Signed-off-by: Guchun Chen <guchun.chen@amd.com>
Tested-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
UAPI Changes:
* fbdev:
* Make fbdev userspace interfaces optional; only leaves the
framebuffer console active
* prime:
* Support dma-buf self-import for all drivers automatically: improves
support for many userspace compositors
Cross-subsystem Changes:
* backlight:
* Fix interaction with fbdev in several drivers
* base: Convert struct platform.remove to return void; part of a larger,
tree-wide effort
* dma-buf: Acquire reservation lock for mmap() in exporters; part
of an on-going effort to simplify locking around dma-bufs
* fbdev:
* Use Linux device instead of fbdev device in many places
* Use deferred-I/O helper macros in various drivers
* i2c: Convert struct i2c from .probe_new to .probe; part of a larger,
tree-wide effort
* video:
* Avoid including <linux/screen_info.h>
Core Changes:
* atomic:
* Improve logging
* prime:
* Remove struct drm_driver.gem_prime_mmap plus driver updates: all
drivers now implement this callback with drm_gem_prime_mmap()
* gem:
* Support execution contexts: provides locking over multiple GEM
objects
* ttm:
* Support init_on_free
* Swapout fixes
Driver Changes:
* accel:
* ivpu: MMU updates; Support debugfs
* ast:
* Improve device-model detection
* Cleanups
* bridge:
* dw-hdmi: Improve support for YUV420 bus format
* dw-mipi-dsi: Fix enable/disable of DSI controller
* lt9611uxc: Use MODULE_FIRMWARE()
* ps8640: Remove broken EDID code
* samsung-dsim: Fix command transfer
* tc358764: Handle HS/VS polarity; Use BIT() macro; Various cleanups
* Cleanups
* ingenic:
* Kconfig REGMAP fixes
* loongson:
* Support display controller
* mgag200:
* Minor fixes
* mxsfb:
* Support disabling overlay planes
* nouveau:
* Improve VRAM detection
* Various fixes and cleanups
* panel:
* panel-edp: Support AUO B116XAB01.4
* Support Visionox R66451 plus DT bindings
* Cleanups
* ssd130x:
* Support per-controller default resolution plus DT bindings
* Reduce memory-allocation overhead
* Cleanups
* tidss:
* Support TI AM625 plus DT bindings
* Implement new connector model plus driver updates
* vkms
* Improve write-back support
* Documentation fixes
-----BEGIN PGP SIGNATURE-----
iQEzBAABCAAdFiEEchf7rIzpz2NEoWjlaA3BHVMLeiMFAmSvvRAACgkQaA3BHVML
eiNpGQgAs8jq1XjN9t8jZsdgXnoCbkZyVUI2NO0HwoVwpRCLgbXp5AX5qq2oRciE
TBhe4Fceh/ZsYqHTZQahnguxgRKM5JgXwbI4Z0iiOVcqasNbycaKAqipxJJ7kdo1
qPhGCbgQFVX7oIq2xjfXehh6O0SYX+R9r88X8dMJxMYv/pcLwOHG74kS040WOcQq
uATgcnobOf/D8ZmlqvfKGAeTUoFo/RSR2Uhlauka58qgeUbicrTELZT2barY9d+k
as6U5vv4wx2zMklTkjrlkMpAT1ZpbB9d3jGHwL27VEnjlfd3wV2bdH7Dzn9qZRf/
gn0ALg/b3u5yBWk/k7YBvijXyNcH6Q==
=bBuG
-----END PGP SIGNATURE-----
Merge tag 'drm-misc-next-2023-07-13' of git://anongit.freedesktop.org/drm/drm-misc into drm-next
drm-misc-next for v6.6:
UAPI Changes:
* fbdev:
* Make fbdev userspace interfaces optional; only leaves the
framebuffer console active
* prime:
* Support dma-buf self-import for all drivers automatically: improves
support for many userspace compositors
Cross-subsystem Changes:
* backlight:
* Fix interaction with fbdev in several drivers
* base: Convert struct platform.remove to return void; part of a larger,
tree-wide effort
* dma-buf: Acquire reservation lock for mmap() in exporters; part
of an on-going effort to simplify locking around dma-bufs
* fbdev:
* Use Linux device instead of fbdev device in many places
* Use deferred-I/O helper macros in various drivers
* i2c: Convert struct i2c from .probe_new to .probe; part of a larger,
tree-wide effort
* video:
* Avoid including <linux/screen_info.h>
Core Changes:
* atomic:
* Improve logging
* prime:
* Remove struct drm_driver.gem_prime_mmap plus driver updates: all
drivers now implement this callback with drm_gem_prime_mmap()
* gem:
* Support execution contexts: provides locking over multiple GEM
objects
* ttm:
* Support init_on_free
* Swapout fixes
Driver Changes:
* accel:
* ivpu: MMU updates; Support debugfs
* ast:
* Improve device-model detection
* Cleanups
* bridge:
* dw-hdmi: Improve support for YUV420 bus format
* dw-mipi-dsi: Fix enable/disable of DSI controller
* lt9611uxc: Use MODULE_FIRMWARE()
* ps8640: Remove broken EDID code
* samsung-dsim: Fix command transfer
* tc358764: Handle HS/VS polarity; Use BIT() macro; Various cleanups
* Cleanups
* ingenic:
* Kconfig REGMAP fixes
* loongson:
* Support display controller
* mgag200:
* Minor fixes
* mxsfb:
* Support disabling overlay planes
* nouveau:
* Improve VRAM detection
* Various fixes and cleanups
* panel:
* panel-edp: Support AUO B116XAB01.4
* Support Visionox R66451 plus DT bindings
* Cleanups
* ssd130x:
* Support per-controller default resolution plus DT bindings
* Reduce memory-allocation overhead
* Cleanups
* tidss:
* Support TI AM625 plus DT bindings
* Implement new connector model plus driver updates
* vkms
* Improve write-back support
* Documentation fixes
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
From: Thomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/msgid/20230713090830.GA23281@linux-uq9g
Rename struct amdgpu_vm_tlb_seq_cb {...} to struct amdgpu_vm_tlb_seq_struct
{...}, so as to not conflict with documentation processing tools. Of course, C
has no problem with this.
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Alex Deucher <Alexander.Deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Link: https://lore.kernel.org/r/b5ebc891-ee63-1638-0377-7b512d34b823@infradead.org
Signed-off-by: Luben Tuikov <luben.tuikov@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Avoids quite a bit of logic and kmalloc overhead.
v2: fix multiple problems pointed out by Felix
v3: two more nit picks from Felix fixed
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20230711133122.3710-4-christian.koenig@amd.com
When kfd uses cpu to update vm iterates all current PDs/PTs bos, adds
AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED flag and kmap them to kernel virtual
address space before kfd updates the vm that was created by gfx.
Signed-off-by: Xiaogang Chen <Xiaogang.Chen@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Update the invalid PTE flag setting with TF enabled.
This is to ensure, in addition to transitioning the
retry fault to a no-retry fault, it also causes the
wavefront to enter the trap handler. With the current
setting, the fault only transitions to a no-retry fault.
Additionally, have 2 sets of invalid PTE settings, one for
TF enabled, the other for TF disabled. The setting with
TF disabled, doesn't work with TF enabled.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Since we allow kfd and graphic operate on same GPU VM to have interoperation
between them GPU VM may have been used by graphic vm operations before kfd turns
a GPU VM into a compute VM. Remove vm clean checking at amdgpu_vm_make_compute.
Signed-off-by: Xiaogang Chen <xiaogang.chen@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Per VM BOs must be marked as moved or otherwise their ranges are not
updated on use which might be necessary when the replace operation
splits mappings.
This fixes random GPU hangs when replacing sparse mappings from the
userspace, while OP_MAP/OP_UNMAP works fine because always valid BOs
are correctly handled there.
Cc: stable@vger.kernel.org
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead of using the VRAM lost counter add a 64bit token which indicates
if a context or job is still valid to use.
Should the VRAM be lost or the page tables need re-creation the token will
change indicating that userspace needs to act and re-create the contexts
and re-submit the work.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Luben Tuikov <luben.tuikov@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When some problem with the updates of page tables is detected reset the
state machine of the VM and re-create all page tables from scratch.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Luben Tuikov <luben.tuikov@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
We need to grab the lock of the BO or otherwise can run into a crash
when we try to inspect the current location.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Guchun Chen <guchun.chen@amd.com>
Tested-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
enforce process isolation between graphics and compute via using the same reserved vmid.
v2: remove params "struct amdgpu_vm *vm" from
amdgpu_vmid_alloc_reserved and amdgpu_vmid_free_reserved.
Signed-off-by: Chong Li <chongli2@amd.com>
Reviewed-by: Christian Koenig <Christian.Koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This is motivated by OOB access in amdgpu_vm_update_range when
offset_in_bo+map_size overflows.
v2: keep the validations in amdgpu_vm_bo_map
v3: add the validations to amdgpu_vm_bo_map/amdgpu_vm_bo_replace_map
rather than to amdgpu_gem_va_ioctl
Fixes: 9f7eb5367d ("drm/amdgpu: actually use the VM map parameters")
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
None have been defined yet, so reject anybody setting any. Mesa sets
it to 0 anyway.
Signed-off-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
Update interrupt handling in CPX mode for GFX9.4.3 by using the
VMID space instead of SDMA client id to determine if an interrupt
should be processed by a KFD node. This is especially needed for
handling retry faults from MMHUB.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This work is required for GC 9.4.3, previous to support memory
partitions per node at SVM. When multiple partition is configured,
every BO should be allocated inside one specific partition which
corresponds to the current amdgpu_device and kfd_node.
v2: squash in compilation fix (Alex)
v3: squash in fix for pre-gfx 9.4.3 (Alex)
v4: squash in best_loc fix (Alex)
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
v1: Each partition has its own gfxhub or mmhub. adjust
the num of MAX_VMHUBS and the GFXHUB/MMHUB layout (Le)
v2: re-design the AMDGPU_GFXHUB/AMDGPU_MMHUB layout (Le)
v3: apply the gfxhub/mmhub layout to new IPs (Hawking)
v4: fix up gmc11 (Alex)
v5: rebase (Alex)
Signed-off-by: Le Ma <le.ma@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When using cpu to update page tables, vm update fences are unused.
Install stub fence into these fence pointers instead of NULL
to avoid NULL dereference when calling dma_fence_wait() on them.
Suggested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
It looks better to place this field in ring
structure. Also drop the repeated ring funcs definitions
if there's no difference except for vmhub field.
v2: rename the field to vm_hub like others (Le)
v3: apply the changes to new ip blocks (Hawking)
v4: fix vcn sw ring (Alex)
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This will be used for performance investigations.
Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Treat the last page in a non-contiguous range as part of the following
contiguous range.
Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-and-tested-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
UAPI Changes:
* connector: Support analog-TV mode property
* media: Add MEDIA_BUS_FMT_RGB565_1X24_CPADHI,
MEDIA_BUS_FMT_RGB666_1X18 and MEDIA_BUS_FMT_RGB666_1X24_CPADHI
Cross-subsystem Changes:
* dma-buf: Documentation fixes
* i2c: Introduce i2c_client_get_device_id() helper
Core Changes:
* Improve support for analog TV output
* bridge: Remove unused drm_bridge_chain functions
* debugfs: Add per-device helpers and convert various DRM drivers
* dp-mst: Various fixes
* fbdev emulation: Always pick 32 bpp as default
* KUnit: Add tests for managed helpers; Various cleanups
* panel-orientation: Add quirks for Lenovo Yoga Tab 3 X90F and DynaBook K50
* TTM: Open-code ttm_bo_wait() and remove the helper
Driver Changes:
* Fix preferred depth and bpp values throughout DRM drivers
* Remove #CONFIG_PM guards throughout DRM drivers
* ast: Various fixes
* bridge: Implement i2c's probe_new in various drivers; Fixes; ite-it6505:
Locking fixes, Cache EDID data; ite-it66121: Support IT6610 chip,
Cleanups; lontium-tl9611: Fix HDMI on DragonBoard 845c; parade-ps8640:
Use atomic bridge functions
* gud: Convert to DRM shadow-plane helpers; Perform flushing synchronously
during atomic update
* ili9486: Support 16-bit pixel data
* imx: Split off IPUv3 driver; Various fixes
* mipi-dbi: Convert to DRM shadow-plane helpers plus rsp driver changes;i
Support separate I/O-voltage supply
* mxsfb: Depend on ARCH_MXS or ARCH_MXC
* omapdrm: Various fixes
* panel: Use ktime_get_boottime() to measure power-down delay in various
drivers; Fix auto-suspend delay in various drivers; orisetech-ota5601a:
Add support
* sprd: Cleanups
* sun4i: Convert to new TV-mode property
* tidss: Various fixes
* v3d: Various fixes
* vc4: Convert to new TV-mode property; Support Kunit tests; Cleanups;
dpi: Support RGB565 and RGB666 formats; dsi: Convert DSI driver to
bridge
* virtio: Improve tracing
* vkms: Support small cursors in IGT tests; Various fixes
-----BEGIN PGP SIGNATURE-----
iQEzBAABCAAdFiEEchf7rIzpz2NEoWjlaA3BHVMLeiMFAmO0B0QACgkQaA3BHVML
eiMKtAf9EXt5yaEonR4gsGNVD70VkebW+jPEWbEMg5hMFDHE9sjBja8T7bOxeeL1
BVofJiAuEZW9176eqeWvShuwOuiE7lf4WQLMvXfFmtHF/Nac9HUtEvOcvc1vEDUB
y6VFlVHe8mSp+Iy0WPLyZCtT4d7v2eM+VYm0HgFa74dSTzQTLGPiUI/XVb/YDaA7
FHKB5NvsMX9S1XvjxCsq0jA5bo8SSzh5CVKerdAZkBJMCSmKiY09o2p5C7vw3EAz
WUXL5CXqd9bwgvZa/9ZClQtpJaikNGoOaMSEl67rbWdVTk0LIpqx3nTY15py5LXd
SG4Wtfor3Vf1REa9TrR2uCIOmh+1gQ==
=0/PC
-----END PGP SIGNATURE-----
Merge tag 'drm-misc-next-2023-01-03' of git://anongit.freedesktop.org/drm/drm-misc into drm-next
drm-misc-next for v6.3:
UAPI Changes:
* connector: Support analog-TV mode property
* media: Add MEDIA_BUS_FMT_RGB565_1X24_CPADHI,
MEDIA_BUS_FMT_RGB666_1X18 and MEDIA_BUS_FMT_RGB666_1X24_CPADHI
Cross-subsystem Changes:
* dma-buf: Documentation fixes
* i2c: Introduce i2c_client_get_device_id() helper
Core Changes:
* Improve support for analog TV output
* bridge: Remove unused drm_bridge_chain functions
* debugfs: Add per-device helpers and convert various DRM drivers
* dp-mst: Various fixes
* fbdev emulation: Always pick 32 bpp as default
* KUnit: Add tests for managed helpers; Various cleanups
* panel-orientation: Add quirks for Lenovo Yoga Tab 3 X90F and DynaBook K50
* TTM: Open-code ttm_bo_wait() and remove the helper
Driver Changes:
* Fix preferred depth and bpp values throughout DRM drivers
* Remove #CONFIG_PM guards throughout DRM drivers
* ast: Various fixes
* bridge: Implement i2c's probe_new in various drivers; Fixes; ite-it6505:
Locking fixes, Cache EDID data; ite-it66121: Support IT6610 chip,
Cleanups; lontium-tl9611: Fix HDMI on DragonBoard 845c; parade-ps8640:
Use atomic bridge functions
* gud: Convert to DRM shadow-plane helpers; Perform flushing synchronously
during atomic update
* ili9486: Support 16-bit pixel data
* imx: Split off IPUv3 driver; Various fixes
* mipi-dbi: Convert to DRM shadow-plane helpers plus rsp driver changes;i
Support separate I/O-voltage supply
* mxsfb: Depend on ARCH_MXS or ARCH_MXC
* omapdrm: Various fixes
* panel: Use ktime_get_boottime() to measure power-down delay in various
drivers; Fix auto-suspend delay in various drivers; orisetech-ota5601a:
Add support
* sprd: Cleanups
* sun4i: Convert to new TV-mode property
* tidss: Various fixes
* v3d: Various fixes
* vc4: Convert to new TV-mode property; Support Kunit tests; Cleanups;
dpi: Support RGB565 and RGB666 formats; dsi: Convert DSI driver to
bridge
* virtio: Improve tracing
* vkms: Support small cursors in IGT tests; Various fixes
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
From: Thomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/msgid/Y7QIwlfElAYWxRcR@linux-uq9g
This is completely pointless since the VMID always stays allocated until
the VM is idle.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This should probably not access job->vm and also emit the SPM switch
under the conditional execute.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Bas pointed out that this isn't working as expected and could cause
crashes. Fix the handling by storing the marker that a switch is needed
inside the job instead.
Reported-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Merge and cleanup the two headers into a single description of the
object API. Also move all the documentation to the implementation and
drop unnecessary includes from the header.
No functional change.
v2: minimal checkpatch.pl cleanup
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20221125102137.1801-4-christian.koenig@amd.com
Expand the GPUVM documentation to better describe the
hardware functionality and use cases it serves.
v2: Fixed a couple of spelling mistakes.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Link: https://lore.kernel.org/r/20221201214153.8453-2-alexander.deucher@amd.com
Signed-off-by: Luben Tuikov <luben.tuikov@amd.com>
Reviewed-by: Luben Tuikov <luben.tuikov@amd.com>
Trigger Mid-Command Buffer Preemption according to the priority of the software
rings and the hw fence signalling condition.
The muxer saves the locations of the indirect buffer frames from the software
ring together with the fence sequence number in its fifo queue, and pops out
those records when the fences are signalled. The locations are used to resubmit
packages in preemption scenarios by coping the chunks from the software ring.
v2: Update comment style.
v3: Fix conflict caused by previous modifications.
v4: Remove unnecessary prints.
v5: Fix corner cases for resubmission cases.
v6: Refactor functions for resubmission, calling fence_process in irq handler.
v7: Solve conflict for removing amdgpu_sw_ring.c.
v8: Add time threshold to judge if preemption request is needed.
v9: Correct comment spelling. Set fence emit timestamp before rsu assignment.
Cc: Christian Koenig <Christian.Koenig@amd.com>
Cc: Luben Tuikov <Luben.Tuikov@amd.com>
Cc: Andrey Grodzovsky <Andrey.Grodzovsky@amd.com>
Cc: Michel Dänzer <michel@daenzer.net>
Signed-off-by: Jiadong.Zhu <Jiadong.Zhu@amd.com>
Acked-by: Luben Tuikov <luben.tuikov@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Re-take the eviction lock immediately again after the allocation is
completed, to fix circular locking warning with drm_buddy allocator.
Move amdgpu_vm_eviction_lock/unlock/trylock to amdgpu_vm.h as they are
called from multiple files.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For asic with VF MMIO access protection avoid using CPU for VM table updates.
CPU pagetable updates have issues with HDP flush as VF MMIO access protection
blocks write to mmBIF_BX_DEV0_EPF0_VF0_HDP_MEM_COHERENCY_FLUSH_CNTL register
during sriov runtime.
v3: introduce virtualization capability flag AMDGPU_VF_MMIO_ACCESS_PROTECT
which indicates that VF MMIO write access is not allowed in sriov runtime
Signed-off-by: Danijel Slivka <danijel.slivka@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Free page table BO from vm resv unlocked context generate below
warnings.
Add a pt_free_work in vm to free page table BO from vm->pt_freed list.
pass vm resv unlock status from page table update caller, and add vm_bo
entry to vm->pt_freed list and schedule the pt_free_work if calling with
vm resv unlocked.
WARNING: CPU: 12 PID: 3238 at
drivers/gpu/drm/ttm/ttm_bo.c:106 ttm_bo_set_bulk_move+0xa1/0xc0
Call Trace:
amdgpu_vm_pt_free+0x42/0xd0 [amdgpu]
amdgpu_vm_pt_free_dfs+0xb3/0xf0 [amdgpu]
amdgpu_vm_ptes_update+0x52d/0x850 [amdgpu]
amdgpu_vm_update_range+0x2a6/0x640 [amdgpu]
svm_range_unmap_from_gpus+0x110/0x300 [amdgpu]
svm_range_cpu_invalidate_pagetables+0x535/0x600 [amdgpu]
__mmu_notifier_invalidate_range_start+0x1cd/0x230
unmap_vmas+0x9d/0x140
unmap_region+0xa8/0x110
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use vm_status_lock to protect all vm_status state transitions to allow
them to happen without a reservation lock in unlocked page table
updates.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use vm_status_lock to protect all vm_status state transitions to allow
them to happen without a reservation lock in unlocked page table
updates.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use vm_status_lock to protect all vm_status state transitions to allow
them to happen without a reservation lock in unlocked page table
updates.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use vm_status_lock to protect all vm_status state transitions to allow
them to happen without a reservation lock in unlocked page table
updates.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The vm status_lock will be used to protect all vm status lists.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This patch updates the PTE flags when translate further (TF) is
enabled:
- With translate_further enabled, invalid PTEs can be 0. Reading
consecutive invalid PTEs as 0 is considered a fault. To prevent
this, ensure invalid PTEs have at least 1 bit set.
- The current invalid PTE flags settings to translate a retry fault
into a no-retry fault, doesn't work with TF enabled. As a result,
update invalid PTE flags settings which works for both TF enabled
and disabled case.
Fixes: 352e683b72 ("drm/amdgpu: Enable translate_further to extend UTCL2 reach")
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
We don't need to validate and map root PD specially here,
it would be validated and mapped by amdgpu_vm_validate_pt_bos
if it is evicted.
The special case is when turning a GFX VM to a compute VM,
if vm_update_mode changed, we should make sure root PD gets
mapped. So just map root PD after updating vm->update_funcs
in amdgpu_vm_make_compute whether the vm_update_mode changed
or not.
v3:
- Add some comments suggested by Christian.
v2:
- Don't rename vm_validate_pt_pd_bos and make it public.
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
there is an unexpected word "the" in the comments that need to be dropped
file: drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
line: 57
* the kernel tells the the ring what VMID to use for that command
changed to
* the kernel tells the ring what VMID to use for that command
Signed-off-by: Jiang Jian <jiangjian@cdjrlc.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The TLB on GFX8 stores each block of 8 PTEs where any of the valid bits
are set.
Fixes: 5255e146c9 ("drm/amdgpu: rework TLB flushing")
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Tested-by: Michal Kubecek <mkubecek@suse.cz>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Flush TLBs when existing PDEs are updated because a PTB or PDB moved,
but avoids unnecessary TLB flushes when new PDBs or PTBs are added to
the page table, which commonly happens when memory is mapped for the
first time.
Suggested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
If page table PDEs is evicted and restored, after updating PDEs, need
increase vm->tlb_seq, then amdgpu_vm_flush will flush TLB before command
submission.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For mes manages gds allocation, skip gds switch.
Signed-off-by: Jack Xiao <Jack.Xiao@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
VM might already be freed when amdgpu_vm_tlb_seq_cb() is called.
We see the calltrace below.
Fix it by keeping the last flush fence around and wait for it to signal
BUG kmalloc-4k (Not tainted): Poison overwritten
0xffff9c88630414e8-0xffff9c88630414e8 @offset=5352. First byte 0x6c
instead of 0x6b Allocated in amdgpu_driver_open_kms+0x9d/0x360 [amdgpu]
age=44 cpu=0 pid=2343
__slab_alloc.isra.0+0x4f/0x90
kmem_cache_alloc_trace+0x6b8/0x7a0
amdgpu_driver_open_kms+0x9d/0x360 [amdgpu]
drm_file_alloc+0x222/0x3e0 [drm]
drm_open+0x11d/0x410 [drm]
Freed in amdgpu_driver_postclose_kms+0x3e9/0x550 [amdgpu] age=22 cpu=1
pid=2485
kfree+0x4a2/0x580
amdgpu_driver_postclose_kms+0x3e9/0x550 [amdgpu]
drm_file_free+0x24e/0x3c0 [drm]
drm_close_helper.isra.0+0x90/0xb0 [drm]
drm_release+0x97/0x1a0 [drm]
__fput+0xb6/0x280
____fput+0xe/0x10
task_work_run+0x64/0xb0
Suggested-by: Christian König <christian.koenig@amd.com>
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add an usage for submissions independent of implicit sync but still
interesting for memory management.
v2: cleanup the kerneldoc a bit
v3: separate amdgpu changes from this
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20220407085946.744568-10-christian.koenig@amd.com
This change adds the dma_resv_usage enum and allows us to specify why a
dma_resv object is queried for its containing fences.
Additional to that a dma_resv_usage_rw() helper function is added to aid
retrieving the fences for a read or write userspace submission.
This is then deployed to the different query functions of the dma_resv
object and all of their users. When the write paratermer was previously
true we now use DMA_RESV_USAGE_WRITE and DMA_RESV_USAGE_READ otherwise.
v2: add KERNEL/OTHER in separate patch
v3: some kerneldoc suggestions by Daniel
v4: some more kerneldoc suggestions by Daniel, fix missing cases lost in
the rebase pointed out by Bas.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20220407085946.744568-2-christian.koenig@amd.com
Audit all the users of dma_resv_add_excl_fence() and make sure they
reserve a shared slot also when only trying to add an exclusive fence.
This is the next step towards handling the exclusive fence like a
shared one.
v2: fix missed case in amdgpu
v3: and two more radeon, rename function
v4: add one more case to TTM, fix i915 after rebase
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20220406075132.3263-2-christian.koenig@amd.com
For VG20 + XGMI bridge, all mappings PTEs cache in TC, this may have
stall invalid PTEs in TC because one cache line has 8 pages. Need always
flush_tlb after updating mapping.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Testing the valid bit is not enough to figure out if we
need to invalidate the TLB or not.
During eviction it is quite likely that we move a BO from VRAM to GTT and
update the page tables immediately to the new GTT address.
Rework the whole function to get all the necessary parameters directly as
value.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Philip Yang <Philip.Yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Fix two incorrect kerneldocs for the recent VM code changes.
v2: fix one more typo
Signed-off-by: Christian König <christian.koenig@amd.com>
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Flush TLB needs wait for GPU update fence done. MMU notify callback to
unmap range from GPUs uses unlocked GPU page table update, so add tlb_cb
to unlocked update fence to increase vm->tlb_seq.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead of providing the bulk move structure for each LRU update set
this as property of the BO. This should avoid costly bulk move rebuilds
with some games under RADV.
v2: some name polishing, add a few more kerneldoc words.
v3: add some lockdep
v4: fix bugs, handle pin/unpin as well
v5: improve kerneldoc
Signed-off-by: Christian König <christian.koenig@amd.com>
Tested-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20220321132601.2161-5-christian.koenig@amd.com
This way we finally fix the problem that new resource are
not immediately evict-able after allocation.
That has caused numerous problems including OOM on GDS handling
and not being able to use TTM as general resource manager.
v2: stop assuming in ttm_resource_fini that res->bo is still valid.
v3: cleanup kerneldoc, add more lockdep annotation
v4: consistently use res->num_pages
Signed-off-by: Christian König <christian.koenig@amd.com>
Tested-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20220321132601.2161-1-christian.koenig@amd.com
Better to leave the decision when to flush the VM changes in the TLB to
the VM code.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Philip Yang<Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Instead of tracking the VM updates through the dependencies just use a
sequence counter for page table updates which indicates the need to
flush the TLB.
This reduces the need to flush the TLB drastically.
v2: squash in NULL check fix (Christian)
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Separate the VM page table backend operations from the state machine since
the amdgpu_vm.c file is becoming to complex.
The allocating, freeing and updating page tables and page directories can
easily be moved into a separate file.
While at it cleanup everything checkpatch.pl reported and rename the
functions a bit to make more clear that they belong together.
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Move the page tables to the idle list after updating the PDEs.
We have gone back and forth with that a couple of times because of problems
with the inter PD dependencies, but it should work now that we have the
state handling cleanly separated.
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Regression has been reported that suspend/resume may hang with
the previous vm ready check commit.
So bring back the evicted list check as a temp fix.
Bug: https://gitlab.freedesktop.org/drm/amd/-/issues/1922
Fixes: c1a66c3bc4 ("drm/amdgpu: check vm ready by amdgpu_vm->evicting flag")
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Qiang Yu <qiang.yu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
-----BEGIN PGP SIGNATURE-----
iQFSBAABCAA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAmIb/PEeHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGugMH/R8icwG5gOjAuxBr
fuz9032ECVFS36Cy3ps9Eqgf5EdS4G6yz3joM4aNtp4B7e5FzI9lzBaHS8OAguNL
y7puFtBr9CywsnniJumZzciB9pEHmF/yyEKfMlRZA3JsRfLDacFstETp+duJnXoA
+s49IWsy1ot5zoherhDXcFLqDoFhLVU4hYwE1xpLpW/cllqmSnW8SDZMtWW1Ui/B
Of7zqINR4zBMmRjP4ymGq/ZrPWlFyWLdtOo0xxVoAQkeMgm33kfaaeGzfyK25FqR
JDGUZ7lkKvwz3PYh2hqJ7dc5K+vhJ18I+F1UmOiL6QAAUF/k8jq7i3Qaf6MKqh2Z
2v+A5k0=
=Hiar
-----END PGP SIGNATURE-----
Backmerge tag 'v5.17-rc6' into drm-next
This backmerges v5.17-rc6 so I can merge some amdgpu and some tegra changes on top.
Signed-off-by: Dave Airlie <airlied@redhat.com>
Workstation application ANSA/META v21.1.4 get this error dmesg when
running CI test suite provided by ANSA/META:
[drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't update BO_VA (-16)
This is caused by:
1. create a 256MB buffer in invisible VRAM
2. CPU map the buffer and access it causes vm_fault and try to move
it to visible VRAM
3. force visible VRAM space and traverse all VRAM bos to check if
evicting this bo is valuable
4. when checking a VM bo (in invisible VRAM), amdgpu_vm_evictable()
will set amdgpu_vm->evicting, but latter due to not in visible
VRAM, won't really evict it so not add it to amdgpu_vm->evicted
5. before next CS to clear the amdgpu_vm->evicting, user VM ops
ioctl will pass amdgpu_vm_ready() (check amdgpu_vm->evicted)
but fail in amdgpu_vm_bo_update_mapping() (check
amdgpu_vm->evicting) and get this error log
This error won't affect functionality as next CS will finish the
waiting VM ops. But we'd better clear the error log by checking
the amdgpu_vm->evicting flag in amdgpu_vm_ready() to stop calling
amdgpu_vm_bo_update_mapping() later.
Another reason is amdgpu_vm->evicted list holds all BOs (both
user buffer and page table), but only page table BOs' eviction
prevent VM ops. amdgpu_vm->evicting flag is set only for page
table BOs, so we should use evicting flag instead of evicted list
in amdgpu_vm_ready().
The side effect of this change is: previously blocked VM op (user
buffer in "evicted" list but no page table in it) gets done
immediately.
v2: update commit comments.
Acked-by: Paul Menzel <pmenzel@molgen.mpg.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Qiang Yu <qiang.yu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
Since newly added BOs don't have any mappings it's ok to add them
without holding the VM lock. Only when we add per VM BOs the lock is
mandatory.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reported-by: Bhardwaj, Rajneesh <Rajneesh.Bhardwaj@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Some people complained about the name and this matches much
more Linux naming conventions for object functions.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Whenever a bo_va structure is added or removed the VM and eventually
added BO should be locked.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
trace_amdgpu_vm_update_ptes trace unable to log when nptes too large
Signed-off-by: Somalapuram Amaranath <Amaranath.Somalapuram@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
adev_to_drm is used everywhere, so improve recent changes
when accessing drm_device pointer from amdgpu_device.
Signed-off-by: Guchun Chen <guchun.chen@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
We hit soft hang while doing memory pressure test on one numa system.
After a qucik look, this is because kfd invalid/valid userptr memory
frequently with process_info lock hold.
Looks like update page table mapping use too much cpu time.
perf top says below,
75.81% [kernel] [k] __srcu_read_unlock
6.19% [amdgpu] [k] amdgpu_gmc_set_pte_pde
3.56% [kernel] [k] __srcu_read_lock
2.20% [amdgpu] [k] amdgpu_vm_cpu_update
2.20% [kernel] [k] __sg_page_iter_dma_next
2.15% [drm] [k] drm_dev_enter
1.70% [drm] [k] drm_prime_sg_to_dma_addr_array
1.18% [kernel] [k] __sg_alloc_table_from_pages
1.09% [drm] [k] drm_dev_exit
So move drm_dev_enter/exit outside gmc code, instead let caller do it.
They are gart_unbind, gart_map, vm_clear_bo, vm_update_pdes and
gmc_init_pdb0. vm_bo_update_mapping already calls it.
Signed-off-by: xinhui pan <xinhui.pan@amd.com>
Reviewed-and-tested-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_bo_get_preferred_pin_domain is used for page tables
creation, which is not involved with page pinning. And it is used in
more cases than display scanout, modify its documentation as well.
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Check range access permission to restore GPU retry fault, if GPU retry
fault on address which belongs to VMA, and VMA has no read or write
permission requested by GPU, failed to restore the address. The vm fault
event will pass back to user space.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Why: Previously hw fence is alloced separately with job.
It caused historical lifetime issues and corner cases.
The ideal situation is to take fence to manage both job
and fence's lifetime, and simplify the design of gpu-scheduler.
How:
We propose to embed hw_fence into amdgpu_job.
1. We cover the normal job submission by this method.
2. For ib_test, and submit without a parent job keep the
legacy way to create a hw fence separately.
v2:
use AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT to show that the fence is
embedded in a job.
v3:
remove redundant variable ring in amdgpu_job
v4:
add tdr sequence support for this feature. Add a job_run_counter to
indicate whether this job is a resubmit job.
v5
add missing handling in amdgpu_fence_enable_signaling
Signed-off-by: Jingwen Chen <Jingwen.Chen2@amd.com>
Signed-off-by: Jack Zhang <Jack.Zhang7@hotmail.com>
Reviewed-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This reverts commit 4e7b93ca52.
Revert reason: The issue has been resolved.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This reverts commit 024d8811c9.
Revert reason: The issue has been resolved.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This reverts commit 4e7b93ca52.
Revert reason: The issue has been resolved.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This reverts commit 024d8811c9.
Revert reason: The issue has been resolved.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Felix Kuehling
ZhenGuo Yin
Srinivasan Shanmugam
Christian König
David Francis
Dave Airlie
Alex Deucher
Lijo Lazar
André Almeida
Daniel Vetter
Guchun Chen
Luben Tuikov
Xiaogang Chen
Mukul Joshi
Samuel Pitoiset
Chong Li
Chia-I Wu
Bas Nieuwenhuizen
Alex Sierra
Hawking Zhang
Lang Yu
Le Ma
Marek Olšák
Jiadong.Zhu
Philip Yang
Danijel Slivka
Jiang Jian
Jack Xiao
xinhui pan
Maxime Ripard
Qiang Yu
Somalapuram Amaranath
Yann Dirson
Yifan Zhang
Jack Zhang
Eric Huang