linux/hdmi.h: Split out logic into .c file.

Shoulda done this ages ago.

only build linux_acpi.c if we have acpi(4) in the kernel

drm: Move acpi_check_dsm &c. from intel_acpi.c to new linux_acpi.c.

drm: Move io_mapping to very own .c file.

drm: linux kthread stubs

linux: Rename linux_sg.c -> linux_sgt.c.

Avoids collision with compat_linux.

linux: Kludgey radix tree shims

linux: Draft half-arsed xarray shims

linux: dma fence array stubs

linux: Draft irq_work shims, in terms of a softint.

Doesn't actually run in hardirq context, but who's gonna notice...

i915: more work

drm: Another pass over i915 and some supporting logic.

This makes a shim around sg_table, which essentially represents two
things:

1. an array of pages (roughly corresponding to bus_dma_segment_t[])
2. an array of DMA addresses stored in a bus_dmamap_t

Both parts are optional; different parts of i915 use sg_tables to
pass around one or both of the two parts. This helps to reduce the
ifdefs by quite a bit, although it's not always clear which part of
an sg_table any particular interface is actually using which is why I
was reluctant to do this before.

linux compat: Draft hrtimer shims.

(not actually high-resolution, just a wrapper around callout(9))

linux: Draft rwsem with bodgy semantics.

Narrow the CPPFLAGS for drm subsystems.

No need for the whole kernel to be built with -I flags for the drm
header files.

Move Linux firmware shims to their own .c file.

dma fence chain stubs

backlight stubs

Draft sync_file.

Rename reservation_object -> dma_resv.

Move Linux SRCU to common.

Draft stop_machine. Hope the callback runs quickly under splhigh...

Move Linux RCU to common.

linux/wait_bit.h

Draft SRCU implementation.

Rename fence -> dma_fence, step 2: files files.

Draft support for drm prime.

Hash-locked atomic64 to give ports like powerpc32 a chance at radeon.

Move the Linux PCI shim into its very own .c file.

The poor thing has deserved this for a very long time, even though it
is full of kludges. Also I'm tired of recompiling everything every
time I touch it.

Sort.

Draft implementation of the Linux reservation and fence APIs.

Implement some of the Linux RCU API.

Sort in the standard English alphabetical order, please.

Sort.

branches: 1.9.16; 1.9.18;
Move linux_work.c and workqueue.h from sys/external/bsd/drm2 to
sys/external/common so that they can be used by others.

LGTM from riastradh@

These should really live outside sys/external, but that can be decided
later

Move Linux ww_mutex code into a .c file where it belongs.

branches: 1.7.2; 1.7.4; 1.7.6;
Remove needless linux_gfp.c.

Trying to make do with Linux's page allocation interface is pretty
much always wrong: either we let uvm allocate pages in uvm objects
for us, or we use bus_dma.

Make it build and boot on my test machines.

Screen blanks on boot on the Ivy Bridge system with

DRM error in cpt_serr_int_handler: PCH transcoder A FIFO underrun

But after that everything is OK. Appears to be an upstream problem.
To investigate...

I think there's a cache flushing issue somewhere -- there are little
display artefacts on my T60.

branches: 1.5.2; 1.5.4;
Implement Linux dmi_check_system API.

Use it to get a Thinkpad quirk for i915 graphics.

Patch from nonoka@, PR 48708.

branches: 1.4.2;
Make drmkms find header files from sys/external/bsd/common.

Move the drmkms i2c dependencies to where they belong.

Merge riastradh-drm2 to HEAD.

branches: 1.1.2;
file files.drmkms_linux was initially added on branch riastradh-drm2.

drm: Fix ACPI crud in ALL kernel build.

drm: Move acpi_check_dsm &c. from intel_acpi.c to new linux_acpi.c.

branches: 1.3.2; 1.3.6;
Implement atomic64_add_return and atomic64_inc_return.

Fix argument types of atomic64_add/sub to be sensible.

Technically Linux does use long long rather than int64_t, but that's
silly.

Hash-locked atomic64 to give ports like powerpc32 a chance at radeon.

backlight stubs

drm: Fix missing bounds checks in dma buf mmap.

drm: Teach dmabuf to handle lseek.

Needed by libdrm_amdgpu.

Based on patch from Jeff Frasca -- thanks!

XXX pullup-10

branches: 1.15.4;
drm: Convert membar_enter/exit stragglers to membar_acquire/release.

drm: Fix membars around dma_buf_put reference count release.

amdgpu: amdgpu_dma_buf.c

drm: Omit half-baked dma_buf_file indirection.

drm: Note NetBSD->Linux errno conversion.

drm: Another pass over i915 and some supporting logic.

This makes a shim around sg_table, which essentially represents two
things:

1. an array of pages (roughly corresponding to bus_dma_segment_t[])
2. an array of DMA addresses stored in a bus_dmamap_t

Both parts are optional; different parts of i915 use sg_tables to
pass around one or both of the two parts. This helps to reduce the
ifdefs by quite a bit, although it's not always clear which part of
an sg_table any particular interface is actually using which is why I
was reluctant to do this before.

Rename reservation_object -> dma_resv.

Update dma-buf API enough to compile drm_prime.c.

No need for <linux/fence.h> in linux_dma_buf.c.

correct error handling.

from Ilja Van Sprundel, thanks.

branches: 1.5.4; 1.5.6;
dma_buf_fd(): fd_putfile() does not belong here

PR kern/53834 ok riastradh@

branches: 1.4.2;
Implement poll for reservations and dma bufs.

Simplify. Destroy the mutex too.

Return the fd we just allocated.

Draft support for drm prime.

drm: Fix dma fence stub fix so the lock is actually initialized.

Tested but forgot to amend change before exporting to CVS again.

drm: Fix dma fence stub so it works with locking operations.

drm: Convert membar_enter/exit stragglers to membar_acquire/release.

drm: Use atomic_load_consume/relaxed to simplify code.

drm: Support 64-bit fence context and sequence numbers.

drm: Make sure dma_fence_wait gets traced too. Deduplicate.

drm: Add dtrace probes to fences.

drm: Fix signalling return value if no enable_signaling op.

drm: Use atomic_load_relaxed for fence->flags when not locked.

Shouldn't make a semantic difference on any machines we run on, but
will make sanitizers happier and emphasize to readers that the result
may not be reliable at these points.

drm: Assert dma_fence_wait internal return value.

drm: Rework timeout return code logic.

drm: Rework dma_fence_wait_any_timeout.

- Yield the first index, not the most recently signalled one.
- Check again after last wakeup.
- Make sure to return positive for nontimeout.
- Handle ip=null.

drm: Set fence->ops for stub fences too.

drm: set f_magic in the stub fence.


Author: phone <mrg@NetBSD.org>
Committer: Taylor R Campbell <riastradh@NetBSD.org>

drm: In dma_fence_wait_timeout, allow MAX_SCHEDULE_TIMEOUT.

In the Linux API, this means wait forever.

drm: Print bad timeout in dma_fence_wait_timeout kasserts.

drm: dma_fence_get allows fence to be null; fix assertion.

While here, update comments about semantics for dma_fence_get and
dma_fence_get_rcu (which does not allow null fence).

drm: Set fence timestamps -- i915 uses them.

drm: Sprinkle assertions about fence lifetime.

drm: Add a kassert for fence destruction.

drm: Skip enabling signalling for fence if already signalled.

drm: Ensure the fence callback is initialized on failure.

drm: Rework enable-signal logic to match Linux.

amdgpu_fb.c
amdgpu_fence.c
amdgpu_gart.c
amdgpu_fixpt31_32.c
amdgpu_fiji_smumgr.c
amdgpu_gem.c
amdgpu_gfx_v10_0.c
amdgpu_gfx_v6_0.c through amdgpu_gfx_v9_0.c
amdgpu_gmc.c
amdgpu_gmc_v6_0.c through amdgpu_gmc_v10_0.c
amdgpu_gtt_mgr.c
some amdgpu_h* files
some amdgpu_i* files
some amdgpu_j* files
amdgpu_kms.c
some amdgpu_m* and amdgpu_n* files

i915: Defer destroying request lock and fence until page destruction.

Add missing unlock at exit


Author: Maya Rashish <maya@NetBSD.org>
Committer: Taylor R Campbell <riastradh@NetBSD.org>

drm/dma-fence: Fix zero timeout semantics in dma_fence_default_wait.

linux dma-fence: Fix up more callback uses.

linux dma-fence: Use dma_fence_default_wait if .wait is null.

Fix typo.

Move grody i915 dma-fence hacks into dma-fence proper.

Teach dma_fence_any_wait_timeout to return index of signalled fence.

dma_fence_get_status

First approximation of dma_fence_get_stub.

Issue __insn_barrier in dma_fence_get_rcu.

This way it is safe to use as dma_fence_get_rcu(*p) when concurrent
writes to *p might be happening, so that we load *p only once before
using the value.

Allow const volatile fence pointer pointers in dma_fence_get_rcu_safe.

We spell it ELAST.

dma_fence_set_error

Linux wants to violate abstractions, so fine, let them.

Add dma_fence_get_rcu_safe.

Rename fence -> dma_fence, step 3 of 3: code.

Rename fence -> dma_fence, step 1: files.

drm: Make dma fence array name strings match Linux.

drm: Another missing spin_lock_destroy.

drm: Draft dma-fence-array.

linux: dma fence array stubs

drm: Convert membar_enter/exit stragglers to membar_acquire/release.

drm: Implement dma fence chains.

drm: Release fence in dma_fence_chain_walk.

dma fence chain stubs

drm: Use KM_SLEEP to allocate reservation fence arrays.

Except as a fast path in an RCU reader.

The array sizes appear to be reasonably small and not trivially
controlled by userland, from what I can tell, so if my impression is
accurate, it is reasonable to sleep for allocation here.

drm: Fix comment about dma_resv_lock_* return values.

drm: Paranoia: handle fencep=null, fence!=null, and 0 shared.

Not sure this is possible but I don't know the API well enough to
prove it can't happen.

drm: Fix missing restart on race in dma_resv_get_fences_rcu.

drm: Support dma_resv_reserve_shared(..., num_fences>1).

drm: Implement drm_resv_get_fences_rcu with null fencep.

drm: Factor dma_fence_put out of multiple dma_resv_do_poll branches.

drm: Fix dma_resv_test_signaled_rcu and dma_resv_wait_timeout_rcu.

Make them implemented the semantics as documented like Linux does:
only test the exclusive fence if there's no shared fences.

drm: Make sure to check read ticket after reading shared fences.

Not entirely sure it was a bug not to do that, but let's make it easy
to prove that we got a consistent snapshot of the whole picture.

drm: Write comment for dma_resv_get_fences_rcu semantics.

drm: Factor out logic to read snapshot of fences in dma_resv.

Should make auditing a little easier.

amdgpu: Another pass through.

amdgpu_object.c
amdgpu_pci.c
amdgpu_pm.c
amdgpu_polaris10_smumgr.c
amdgpu_pp_psm.c
amdgpu_ppatomctrl.c
amdgpu_ppatomfwctrl.c
amdgpu_process_pptables_v1_0.o
amdgpu_psp.o
amdgpu_psp_v10_0.c
amdgpu_psp_v11_0.c
amdgpu_psp_v12_0.c
amdgpu_psp_v3_1.c
amdgpu_ras.c
amdgpu_ras_eeprom.c
amdgpu_rc_calc.c
amdgpu_rc_calc_dpi.c
amdgpu_ring.c
amdgpu_rlc.c
amdgpu_rn_clk_mgr.c
amdgpu_sdma_v3_0.c
amdgpu_sdma_v4_0.c
amdgpu_sdma_v5_0.c
amdgpu_si.c
amdgpu_si_dma.c
amdgpu_si_dpm.c
amdgpu_smu.c
amdgpu_smu10_hwmgr.c
amdgpu_smu7_hwmgr.c
amdgpu_smu8_hwmgr.c
amdgpu_smu8_smumgr.c
amdgpu_smu_v11_0.c (and a sprinkling of consts)
amdgpu_smu_v11_0_i2c.c
amdgpu_soc15.c
amdgpu_tonga_smumgr.c
amdgpu_ttm.c
amdgpu_ucode.c
amdgpu_uvd.c
amdgpu_uvd_v5_0.c
amdgpu_uvd_v6_0.c
amdgpu_uvd_v7_0.c
amdgpu_vce.c
amdgpu_vce_v2_0.c
amdgpu_vce_v4_0.c
amdgpu_vcn_v1_0.c
amdgpu_vcn_v2_0.c
amdgpu_vcn_v2_5.c
amdgpu_vega10_hwmgr.c
amdgpu_vega10_powertune.c
amdgpu_vega10_processpptables.c
amdgpu_vega10_reg_init.c
amdgpu_vega12_hwmgr.c
amdgpu_vega20_hwmgr.c
amdgpu_vega20_ppt.c
amdgpu_vega20_reg_init.c
amdgpu_vi.c
amdgpu_virt.c
amdgpu_vram_mgr.c
amdgpu_xgmi.c -- disable (for now?)
amdgpufb.c

drm: Fence leak audit. No functional change intended.

Sprinkle nulling out variables, add kasserts to reflect them, and
propagate their consequences to eliminate dead code. Should make it
easier to detect similar leak bugs.

drm: Fix sense of conditional to avoid null pointer dereference.

drm: Paranoia: null out fence if unreferenceable.

drm: Membar audit for dma_resv.

Try to pacify kcsan (untested) and make it clearer what ordering
matters.

drm: Use atomic_store_release/load_consume in linux_dma_resv.c.

At least, for robj->fence and robj->fence_excl. Need to review
list->shared_count and list->shared[i] too.

drm: Plug fence leak

Implement dma_resv_locking_ctx more carefully.

dma_resv_reserve_shared takes one more argument, num_fences

Fortunately it's ~always 1, and (XXX) I assume our implementation
assumes this, too. But assert it.


Author: Maya Rashish <maya@NetBSD.org>

Add slow path versions of to drm_resv_lock.

Rename reservation_object -> dma_resv.

Implement dmi_match.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.10; 1.1.34; 1.1.36;
Implement Linux dmi_check_system API.

Use it to get a Thinkpad quirk for i915 graphics.

Patch from nonoka@, PR 48708.

Rename fence -> dma_fence, step 1: files.

hardclock_ticks -> getticks()

branches: 1.15.4;
fix various problems i've seen where cv_*wait*() return ERESTART,
which is -3 in netbsd, which we have mapped linux ERESTARTSYS to.

this has a problem because linux code often returns errors and
pointers in the same value, and pointer values between -4095 and
-1 are considered as error returns, but -3 ends up as 3 and thus
is not considered an error, and mayhem ensues.

with this in place my kabylake system seems actually stable, i
have not triggered any of my prior issues in almost 4 weeks now.


Taylor asked me to write up a description and then wrote most of
the text below for me :-)


In Linux code, we always work with ERESTARTSYS so the code meaning
start over is a positive NetBSD errno safe for PTR_ERR/ERR_PTR.
To achieve this:
1. adapt all cv_waits that return to Linux so they map ERESTART to
ERESTARTSYS, and
2. adapt all returns to userland so they convert ERESTARTSYS to
ERESTART.
Leave EINTR and all other error codes alone.

fence_referenced_p(): mark as __diagused

branches: 1.13.2;
Allow fence_get(NULL).

This is kind of silly: if the caller has exclusive access to the
fence, they should be able to guarantee it's there, so this could
serve to suppress legitimate bugs. But better to preserve Linux
semantics here than to be picky about API design.

Bug by me, fix from Tobias Ulmer in PR kern/53565.

Issue __insn_barrier after loading hardclock_ticks.

For some reason this isn't volatile...

Assert EWOULDBLOCK happens only with timeout.

With no timeout, fence_default_wait must return 1 on success, never 0.

KASSERT -> KASSERTMSG, to help debug panic observed by mrg@.

Linux fence_put accepts NULL as noop. Match this.

From mrg@.

Make extremely noisy fence trace output conditional.

Lotsa kasserts about referencedness of frences.

Add fence_is_later and fence_wait_any_timeout.

Fix prototype of function in comment.

Fence release callback may be absent; default to fence_free.

Use atomics to manage fence->flags.

Outside users use this field too, and expect it to be managed with
atomics.

Draft implementation of the Linux reservation and fence APIs.

linux: request_firmware_direct

Move Linux firmware shims to their own .c file.

Remove needless linux_gfp.c.

Trying to make do with Linux's page allocation interface is pretty
much always wrong: either we let uvm allocate pages in uvm objects
for us, or we use bus_dma.

branches: 1.2.2; 1.2.4; 1.2.6;
Merge riastradh-drm2 to HEAD.

branches: 1.1.2;
file linux_gfp.c was initially added on branch riastradh-drm2.

linux/hdmi: Fix size of product id.

linux/hdmi: Fix return value in unpack.

linux/hdmi: Fix hdmi_infoframe_unpack to record the unpacked header.

linux/hdmi: Handle audio infoframes in hdmi_infoframe_pack.

Sort cases by type number.

linux/hdmi: Convert failure branch to kassert for internal invariant.

linux/hdmi: Use ssize_t for all hdmi_*_infoframe_pack functions.

linux/hdmi: Zero entire union hdmi_infoframe on unpack.

Don't just zero the part that we're about to initialize -- i915 does
memcmp on the whole structure.

linux/hdmi: Handle unpacking audio frames too. Sort by number.

linux/hdmi: Make some private functions static.

linux/hdmi.h: Split out logic into .c file.

Shoulda done this ages ago.

drm: Eliminate hrtimer_destroy and teardown_timer.

These are non-Linux shims added to the Linux API to pair with
hrtimer_setup and setup_timer, but they really only serve to call
callout_destroy, which is not strictly necessary in the NetBSD API.
So although we lose a little bit of diagnostic help this way, we also
reduce diffs a bit and save some trouble that bit us last week.

Add hrtimer_destroy


Author: Maya Rashish <maya@NetBSD.org>
Committer: Taylor R Campbell <riastradh@NetBSD.org>

linux compat: Draft hrtimer shims.

(not actually high-resolution, just a wrapper around callout(9))

linux: Repeat i2c transfer if driver fails with EAGAIN.

The Intel GMBUS (graphics management bus, i2c controller) relies on
this now to fall back from interrupt-driven xfers to bit-banging.

Make intel_psr.c build
netbsd-ify
Stub out intel_lpe_audio.c (for now?).
Hammer intel_opregion.c into shape.
Hack up intel_lrc.c
Adapt intel_reset.c
Stub out intel_region_lmem.c
Adapt intel_ring_submission.c
Adapt intel_rps.c
Adapt intel_sdvo.c
Adapt intel_runtime_pm.c
Adapt intel_sideband.c
Adapt intel_sprite.c
Adapt intel_tv.c
Adapt intel_tc.c
Adapt intel_sseu.c
Adapt intel_timeline.c
Adapt intel_uc_fw.c and fix typo
Fix up intel_uncore.c
intel_vga.c
intel_wakeref.c
intel_vdsc.c
intel_wopcm.c
intel_workarounds.c
i915_globals.c
Fix up intelfb.c
Stub out intel_guc_log.c
Stub out intel_dsi_dcs_backlight.c
i915_perf.c
intel_hdmi.c
intel_hdcp.c


Author: Maya Rashish <maya@NetBSD.org>
Committer: Taylor R Campbell <riastradh@NetBSD.org>

Provide a definition of I2C_M_STOP XXX


Author: Maya Rashish <maya@NetBSD.org>

Support i2c bus lock operations.

Reorganize Linux i2c header file and add a few more shims.

branches: 1.2.4; 1.2.6; 1.2.8; 1.2.10; 1.2.12;
Merge riastradh-drm2 to HEAD.

branches: 1.1.2;
file linux_i2c.c was initially added on branch riastradh-drm2.

linux/idr: Remove meaningless stray membar.

Teach idr_remove to accept failure.

But teach it to return the data if it succeeds too.

Define idr_init_base.

Nothing actually uses this at the moment, because callers already
specify the base. Not really sure what the point is!

sdt probe naming style.

Work around broken empty expansion of SDT_PROBE* if !KDTRACE_HOOKS.

sdt dtrace probes for linux idr.

Zero-initialize idr cache.

Draft rewrite of idr preload.

Previous idr code assumed every caller would definitely call
idr_preload, idr_alloc, idr_preload_end, but some callers skip
idr_alloc if an intermediate failure happens first, and would
therefore leak idr nodes.

Use a per-lwp single-node cache instead, and print warnings about
leakers.

Add idr_get_next and idr_for_each_entry.

branches: 1.6.2; 1.6.4;
Tweak slightly sketchy logic in linux_idr.

1. idr_preload can fail if you don't set __GFP_WAIT.
2. If idr_preload fails, it is wrong for idr_alloc to assert.
3. There is no way for idr_alloc to know what flags idr_preload got.

Probably won't *fix* any bugs, but if there is a bug with a missing
__GFP_WAIT, then we will learn about a trifle sooner.

branches: 1.5.10;
due to hangs seen by several folks, for now revert:
http://mail-index.netbsd.org/source-changes/2014/11/04/msg060120.html

Log Message:
This code should be MP-safe. Use IPL_SCHED in place of IPL_DRM/IPL_VM and set
D_MPSAFE flag in cdevsw.

branches: 1.4.2;
This code should be MP-safe. Use IPL_SCHED in place of IPL_DRM/IPL_VM and set
D_MPSAFE flag in cdevsw.

branches: 1.3.4;
Make it build and boot on my test machines.

Screen blanks on boot on the Ivy Bridge system with

DRM error in cpt_serr_int_handler: PCH transcoder A FIFO underrun

But after that everything is OK. Appears to be an upstream problem.
To investigate...

I think there's a cache flushing issue somewhere -- there are little
display artefacts on my T60.

branches: 1.2.2; 1.2.4; 1.2.6;
Merge riastradh-drm2 to HEAD.

branches: 1.1.2;
file linux_idr.c was initially added on branch riastradh-drm2.

drm: Move io_mapping to very own .c file.

linux: Use pointer indirection for irq_work_cpu.

Can't store locks in percpu since it moves around.

linux: Draft irq_work shims, in terms of a softint.

Doesn't actually run in hardirq context, but who's gonna notice...

Ifdef out label used only by !__HAVE_MM_MD_DIRECT_MAPPED_PHYS.

Work around `kmap' namespace clash another way.

Work around a namespacing clash between dtrace and linux symbol.

sdt dtrace probes for linux kmap.

branches: 1.12.16; 1.12.18;
due to hangs seen by several folks, for now revert:
http://mail-index.netbsd.org/source-changes/2014/11/04/msg060120.html

Log Message:
This code should be MP-safe. Use IPL_SCHED in place of IPL_DRM/IPL_VM and set
D_MPSAFE flag in cdevsw.

branches: 1.11.2;
This code should be MP-safe. Use IPL_SCHED in place of IPL_DRM/IPL_VM and set
D_MPSAFE flag in cdevsw.

Use direct map if available in linux_kmap.

Yields 20% increase in glxgears framerate.

Tweak style.

No functional change intended. Assembly differences appear to be
only related to kassert line numbers.

__diagused, not __unused.

Assert sleepable in Linux kmap/kunmap.

Simplify empty test for linux_kmap_entries.

Convert linux_kmap_lock to an adaptive lock.

kmap/kunmap can't be used in interrupt context anyway, so there is no
need for this to be a spin lock.

branches: 1.4.4; 1.4.6; 1.4.8; 1.4.10;
Don't leak KVA by forgetting to call uvm_km_free...oops!

From rmind@.

Remove needless splvm/splx calls.

From rmind@, who explained that the pmap(9) man page is wrong.

Merge riastradh-drm2 to HEAD.

branches: 1.1.2;
file linux_kmap.c was initially added on branch riastradh-drm2.

drm: Work around busted kthread_join.

drm: Fix error return for kthread_run: error pointer, not null.

drm: Take advantage of kthread interlock to reduce diff.

drm: Rework Linux `kthread' abstraction to avoid race to sleep.

Requires passing in the caller's lock and condvar to kthread_run, but
for the one user that appears not to be an onerous requirement.

drm: Trigger `spurious' wakeup for kthread_stop too.

linux: In kthread_park, wake thread to re-check kthread_shouldpark.

Otherwise, the thread to pakr may be sleeping on some condvar not
noticing it has to notify kthread_park --> deadlock.

linux: Make kthread_park/stop work at IPL_VM.

Needed in order to hold spin locks around it.

linux: Paranoia: Don't read kt_shouldstop unlocked.

drm: linux kthread stubs

branches: 1.2.4; 1.2.6; 1.2.10;
Merge riastradh-drm2 to HEAD.

branches: 1.1.2;
file linux_list_sort.c was initially added on branch riastradh-drm2.

drm: Fix dma fence stub so it works with locking operations.

drm: linux kthread stubs

linux: Draft irq_work shims, in terms of a softint.

Doesn't actually run in hardirq context, but who's gonna notice...

linux/wait_bit.h

Draft Linux tasklet implementation.

Hash-locked atomic64 to give ports like powerpc32 a chance at radeon.

Draft implementation of the Linux reservation and fence APIs.

Implement some of the Linux RCU API.

branches: 1.6.16; 1.6.18;
Update module dependencies:

drmkms does not depend on iic
drmkms_linux depends on i2cexec

branches: 1.5.2;
finalize the idr code.

branches: 1.4.2; 1.4.4;
First whack at radeon driver.

No hardware to test yet, but it builds.

Make it build and boot on my test machines.

Screen blanks on boot on the Ivy Bridge system with

DRM error in cpt_serr_int_handler: PCH transcoder A FIFO underrun

But after that everything is OK. Appears to be an upstream problem.
To investigate...

I think there's a cache flushing issue somewhere -- there are little
display artefacts on my T60.

branches: 1.2.2; 1.2.4; 1.2.6;
Merge riastradh-drm2 to HEAD.

branches: 1.1.2;
file linux_module.c was initially added on branch riastradh-drm2.

Match function name to code


Author: Maya Rashish <maya@NetBSD.org>
Committer: Taylor R Campbell <riastradh@NetBSD.org>

i915: more work

linux_pci: Fix shifto in pci_get_class.

It looks like Linux's pci_get_class also matches the interface part
of the PCI class register (but not the revision part), and I hadn't
noticed that in the previous shim structured differently.

linux_pci: Nix pci enumeration kludges.

Now that we can pass a cookie through, this stuff will be a little
less fragile.

linux: Add a few more cases to pci_get_class.

Should fix crash on boot with amdgpu now that the ACPI business is
enabled.

drm: enable almost all PCIe functionality

linux_pci.c revisions 1.24 and 1.25 implemented most of the remaining
missing PCIe backends, but only enabled them for some amdgpu portions.

this enables all code marked with "XXX amdgpu pcie", "XXX radeon pcie",
and "XXX pcie speed". for most of it, simply removing #ifndefs __NetBSD__
to enable compliation was required, once the new "bus->max_bus_speed"
member was added to struct pci_bus. add an "always fails" backend for
pci_enable_atomic_ops_to_root() which seems to only be necessary
for virtual GPU functionality (and could be implemented if needed.)

tested on radeon 5450, 7750, R7 240 [radeon], and RX 550 [amdgpu], and
nvidia 750 and 1030 [nouveau].


this still does not quite work on nvidia cards. there are two problems
that remain:

- the call to set the link speed is skipped because the speed is set
to the default value of "-1". nvkm_pcie_set_link() will actually
determine the right value for this and for some cards, calling this
function if the current speed is -1 helps set the link speed. it
may be that on linux other paths we don't have enabled properly
would set this (there's one via debugfs, and a jetson specific one,
though perhaps setting either AC or DC speed values as boot options
(after hooking up these for netbsd) would currently work.

- worse, cards newer than kepler - geforce 900, 1000, and newer, are
all lacking the backing support to set pcie link speed. the GT 1030
card i have been testing with remains at pcie 1.0.

convert a KASSERT() into an if () panic() sequence to appease GCC 12.

OK riastradh@.

branches: 1.25.2;
add pcie capability and read request size linux compat, some pci root support

implement support for:
- pcie_capability_read_dword()
- pcie_capability_read_word()
- pcie_capability_write_dword()
- pcie_capability_write_word()
- pcie_get_readrq()
- pcie_set_readrq()

implement the "struct pci_dev" bus->self member by creating a minimal fake
"struct pci_dev" for the pci bus itself. this is kind of gross. it checks
that the current device's parent is a netbsd "pci" device, and that it has
a (grand) parent "ppb" device, and then fills in the fake device based upon
the pci and ppb devices.

add some PCIE_LCSR2_TGT_LSPEED encodings, and map them to linux names.
map several other PCIE_LCSR and PCIE_LCAP names.

uncomment several pcie code segments in radeon and amdgpu. (not sure that
we can test the amdgpu_si.c change, as we use the radeon version and the
amdgpu version hangs on the one machine i have.)

tested on amdgpu (RX550) and radeon (7750 & 3650).


ok @riastradh

fill out more of the linux pci API compat

- implement pcie_get_speed_cap(), pcie_bandwidth_available(), and
pci_is_root_bus().
- expand "enum pci_bus_speed" to add PCIe 5.x and 6.x speeds.
- add "enum pcie_link_width".
- add defines for PCIE_LCSR_LINKSPEED (PCIe generation) and PCIE_LCSR_NLW
(negotiated lane width) to pcireg.h
- enable amdgpu_device_get_pcie_info() code now it works.

ok riastradh@

drm: Use real pci segment number, not pciN device unit.

drm: Nix drm_agp_borrow.

This horrible kludge dates from before I understood the relation of
genfb_pci and pci drm drivers in the old and new worlds of drm.

The only user of it, in i915, was changed to use agp_i810_borrow
directly in the last drm update, so this hack can die.

drm: Deconditionalize pci_get_segment.

linux: pci_name

i915: another whack at it

drm: Remove local diffs for drm_pci_set_busid.

Instead, move it to its own file that can be safely used without
bringing in a link/run-time dependency on pci.

Include acpica.h for NACPICA.

Move acpi includes to linux_pci.c to avoid pollution.

Stub dev_is_pci


Author: Maya Rashish <maya@NetBSD.org>

Define pci_is_thunderbolt_attached (answer: no).

Update change to Linux struct resource members: end, not size.

Define pci_get/set_drvdata properly.

pci_get_bus_and_slot -> pci_get_domain_bus_and_slot

re-enable MSI - this actually enables MSI only for intel/radeon/generic
which are reported to work fine, while nouveau (which breaks with
MSI) still uses INTx

see the thread
http://mail-index.netbsd.org/current-users/2020/02/06/msg037661.html

Turn MSI off again.

Re-enable PCI MSI for DRM devices

Use pci_get_segment if available

branches: 1.6.2; 1.6.6; 1.6.10;
Rework PCI and AGP conditionalization.

- Push drm_pci_set_unique into driver callback.
- Eliminate drm_pci_set_unique_hook.
- Gather all drm_agp_* functions into struct drm_agp_hooks.
- Replace the nonsensical old atomic garbage by serious locking.
- Make drm_agpsupport.c its own module.
- Eliminate NDRMKMS_PCI.
- Use NAGP from "agp.h" only in drm_module.c for horrible hack.
=> See comment in file for rationale.
- Always define CONFIG_PCI=1 and CONFIG_AGP=1.
- Always go through the drm_agp_* function hooks.
- Ifdef out nouveau agp stuff that doesn't go through drm_agp_*
for reasons that I'm too frustrated to figure out tonight.
- pci_iomap no longer automagically does agp_i810_borrow.
=> Use drm_agp_borrow instead.

Try agp_i810_borrow only if CONFIG_AGP is enabled.

Treat pre-enabled devices as 1 enablecnt.

Zero the whole thing for good measure.

Make sure the resources are marked unmapped initially.

Move the Linux PCI shim into its very own .c file.

The poor thing has deserved this for a very long time, even though it
is full of kludges. Also I'm tired of recompiling everything every
time I touch it.

linux: GFP_ATOMIC, not GFP_NOWAIT

linux: membar_exit not needed if we use locks here.

linux: Kludgily make radix tree safe for RCU

Using a lock is completely stupid (gotta be cpu_simple_lock because
there's no radix_tree_destroy or anything), and the nonsense to do
unvolatile/unconst is garbage, but it will serve to make this safe
for RCU until someone makes our radix tree code RCU-safe (which
Someone^TM should totally do).

linux: Kludgey radix tree shims

Move Linux RCU to common.

Implement kfree_rcu.

Make sure the RCU GC thread processes all callbacks before dying.

Implement rcu_barrier to wait for call_rcu callbacks.

gc.cv has multiple purposes now, so use cv_broadcast instead of
cv_signal.

need <sys/param.h> for COHERENCY_UNIT

branches: 1.4.2; 1.4.6; 1.4.20;
dtrace sdt probes for linux rcu.

Make gc static. No need to use this private state elsewhere.

Don't forget to call kthread_exit.

Implement some of the Linux RCU API.

Rename reservation_object -> dma_resv.

Draft implementation of reservation_object_copy_fences.

Judiciously insert __insn_barrier into reservation logic.

Nothing is volatile here, so we must handle the barriers ourselves.

Also note where membar_producer is implied.

const const const const const reservations

Add some more reservation API.

Define reservation_object_assert_held.

Just declare reservation_object_copy_fences for now, will implement
later.

Rename more fields that Linux treats as part of the API, not opaque.

Let's try spelling the member the same way Linux does.

Linux API wants seqcount exposed in reservation object, so do that.

reservation_object_lock/lock_interruptible/trylock/unlock

reservation_object_get_excl_rcu

fence -> dma_fence in linux_reservation.c.

drm: Plug another fence leak.

XXX pullup-9

drm: Release fence if we're about to return too.

Should fix another fence leak.

XXX pullup-9

drm: Release fence after use.

May plug the dreaded radeon kmem memory leak.

XXX pullup-9

branches: 1.11.2; 1.11.6; 1.11.8; 1.11.20;
Preserve signedness to return failure from fence_wait_timeout.

From mrg@.

Expand docstring for reservation_object_poll.

Let's try not to step on each others' toes, shall we?

Implement poll for reservations and dma bufs.

Clarify with comments and assertions.

There may not be an old list.

Nihil ex nihilo -- don't dereference null.

Acquire references in add_shared/excl.

Draft reservation_object_get_fences_rcu.

Apparently one can reserve_shared only once in a row. Simplify.

Wait for the read to be valid!

Draft implementation of the Linux reservation and fence APIs.

linux/rwsem: Work around broken lockdebug macros.

linux: Teach rw semaphores about lockdebug.

linux: Use IPL_VM for lock inside rw_semaphore.

down_read_trylock needs to work under a spin lock.

linux: Draft rwsem with bodgy semantics.

linux: Rename linux_sg.c -> linux_sgt.c.

Avoids collision with compat_linux.

linux: Accept multipage segments in sg_alloc_table_from_bus_dmamem.

i915: another whack at it

drm: Another pass over i915. Most of i915 gem builds now.

drm: Another pass over i915 and some supporting logic.

This makes a shim around sg_table, which essentially represents two
things:

1. an array of pages (roughly corresponding to bus_dma_segment_t[])
2. an array of DMA addresses stored in a bus_dmamap_t

Both parts are optional; different parts of i915 use sg_tables to
pass around one or both of the two parts. This helps to reduce the
ifdefs by quite a bit, although it's not always clear which part of
an sg_table any particular interface is actually using which is why I
was reluctant to do this before.

drm: Sprinkle some assertions into sg dma logic.

drm: Destroy dma map in sg_table_free.

Helps to reduce a little diff.

linux: Fix dma mapping to set READ|WRITE if bidi; clarify cases.

linux: Rename linux_sg.c -> linux_sgt.c.

Avoids collision with compat_linux.

Move Linux SRCU to common.

Draft SRCU implementation.

sys: Use membar_release/acquire around reference drop.

This just goes through my recent reference count membar audit and
changes membar_exit to membar_release and membar_enter to
membar_acquire -- this should make everything cheaper on most CPUs
without hurting correctness, because membar_acquire is generally
cheaper than membar_enter.

sys: Membar audit around reference count releases.

If two threads are using an object that is freed when the reference
count goes to zero, we need to ensure that all memory operations
related to the object happen before freeing the object.

Using an atomic_dec_uint_nv(&refcnt) == 0 ensures that only one
thread takes responsibility for freeing, but it's not enough to
ensure that the other thread's memory operations happen before the
freeing.

Consider:

Thread A Thread B
obj->foo = 42; obj->baz = 73;
mumble(&obj->bar); grumble(&obj->quux);
/* membar_exit(); */ /* membar_exit(); */
atomic_dec -- not last atomic_dec -- last
/* membar_enter(); */
KASSERT(invariant(obj->foo,
obj->bar));
free_stuff(obj);

The memory barriers ensure that

obj->foo = 42;
mumble(&obj->bar);

in thread A happens before

KASSERT(invariant(obj->foo, obj->bar));
free_stuff(obj);

in thread B. Without them, this ordering is not guaranteed.

So in general it is necessary to do

membar_exit();
if (atomic_dec_uint_nv(&obj->refcnt) != 0)
return;
membar_enter();

to release a reference, for the `last one out hit the lights' style
of reference counting. (This is in contrast to the style where one
thread blocks new references and then waits under a lock for existing
ones to drain with a condvar -- no membar needed thanks to mutex(9).)

I searched for atomic_dec to find all these. Obviously we ought to
have a better abstraction for this because there's so much copypasta.
This is a stop-gap measure to fix actual bugs until we have that. It
would be nice if an abstraction could gracefully handle the different
styles of reference counting in use -- some years ago I drafted an
API for this, but making it cover everything got a little out of hand
(particularly with struct vnode::v_usecount) and I ended up setting
it aside to work on psref/localcount instead for better scalability.

I got bored of adding #ifdef __HAVE_ATOMIC_AS_MEMBAR everywhere, so I
only put it on things that look performance-critical on 5sec review.
We should really adopt membar_enter_preatomic/membar_exit_postatomic
or something (except they are applicable only to atomic r/m/w, not to
atomic_load/store_*, making the naming annoying) and get rid of all
the ifdefs.

Provisionally work around cycle in x86 header files.

Draft stop_machine. Hope the callback runs quickly under splhigh...

linux_sync_file: Fix missing init/fini steps.

Noted by rjs@.

PR kern/58210

branches: 1.2.4;
Add inline functions to manipulate the klists that link up knotes
via kn_selnext:

- klist_init()
- klist_fini()
- klist_insert()
- klist_remove()

These provide some API insulation from the implementation details of these
lists (but not completely; see vn_knote_attach() and vn_knote_detach()).
Currently just a wrapper around SLIST(9).

This will make it significantly easier to switch kn_selnext linkage
to a different kind of list.

Draft sync_file.

linux: Fix wait_bit semantics.

- wait_on_bit is supposed to wait until the bit is cleared, not set.

- wait_on_bit_timeout is supposed to return 0 on success, -EAGAIN on
faiure.

Omit wake_up_bit; nothing uses it and clear_and_wake_up_bit is a more
semantically coherent operation.

linux: Add clear_and_wake_up_bit.

wait_on_bit

Fix thinko: ERESTART, not EWOULDBLOCK.

linux/wait_bit.h

Move linux_work.c and workqueue.h from sys/external/bsd/drm2 to
sys/external/common so that they can be used by others.

LGTM from riastradh@

These should really live outside sys/external, but that can be decided
later

Avoid NULL dereference in linux_worker_intr() in WORK_DELAYED_CANCELLED case.

Potentially fixes PR#49560.

include sys/cpu.h for CPU_INFO_ITERATOR

due to hangs seen by several folks, for now revert:
http://mail-index.netbsd.org/source-changes/2014/11/04/msg060120.html

Log Message:
This code should be MP-safe. Use IPL_SCHED in place of IPL_DRM/IPL_VM and set
D_MPSAFE flag in cdevsw.

branches: 1.9.2;
This code should be MP-safe. Use IPL_SCHED in place of IPL_DRM/IPL_VM and set
D_MPSAFE flag in cdevsw.

Don't forget to lock wq_lock around handling wq_delayed.

branches: 1.7.2; 1.7.4;
Linux wq locks are taken with spin locks held, so need IPL_VM.

Linux work is queued in intr context, so block intrs when locking.

(Yes, this getting out of hand.)

This case is not supposed to fall through!

First whack at radeon driver.

No hardware to test yet, but it builds.

Make it build and boot on my test machines.

Screen blanks on boot on the Ivy Bridge system with

DRM error in cpt_serr_int_handler: PCH transcoder A FIFO underrun

But after that everything is OK. Appears to be an upstream problem.
To investigate...

I think there's a cache flushing issue somewhere -- there are little
display artefacts on my T60.

branches: 1.2.2; 1.2.4; 1.2.6;
Merge riastradh-drm2 to HEAD.

branches: 1.1.2;
file linux_work.c was initially added on branch riastradh-drm2.

Misc stubs.

KASSERT -> KASSERTMSG

phys_wc_to_mtrr_index got renamed to arch_phys_wc_index upstream.

Initialize n before mtrr_set in arch_phys_wc_del.

From mrg@.

XXX pullup-7
XXX pullup-8

Exit, don't reenter, the mutex...!@#*@$!&.

From mrg@.

XXX pullup-7
XXX pullup-8

branches: 1.4.16; 1.4.18;
only use mtrr api if options MTRR is present and we are on x86

due to hangs seen by several folks, for now revert:
http://mail-index.netbsd.org/source-changes/2014/11/04/msg060120.html

Log Message:
This code should be MP-safe. Use IPL_SCHED in place of IPL_DRM/IPL_VM and set
D_MPSAFE flag in cdevsw.

branches: 1.2.2;
This code should be MP-safe. Use IPL_SCHED in place of IPL_DRM/IPL_VM and set
D_MPSAFE flag in cdevsw.

branches: 1.1.2; 1.1.6;
Make it build and boot on my test machines.

Screen blanks on boot on the Ivy Bridge system with

DRM error in cpt_serr_int_handler: PCH transcoder A FIFO underrun

But after that everything is OK. Appears to be an upstream problem.
To investigate...

I think there's a cache flushing issue somewhere -- there are little
display artefacts on my T60.

drm/linux_ww_mutex: Fix ww acquire context ordering.

XXX pullup-8
XXX pullup-9
XXX pullup-10

drm/linux_ww_mutex: Fix wait loops.

If cv_wait_sig returns because a signal is delivered, we may
nonetheless have been granted the lock. It is harmless for us to
ignore this fact in three of the four paths, but in
ww_mutex_state_wait_sig, we may now have ownership of the lock and
MUST NOT return failure because the caller MUST release the lock
before destroying the ww_acquire_ctx.

While here, restructure the other three loops for clarity, so they
match the structure of the fourth and so they have a little less
impenetrable negation.

PR kern/57537

XXX pullup-8
XXX pullup-9
XXX pullup-10

branches: 1.14.4;
drm: In ww_mutex_unlock, do lockdebug check first.

This way we get a full lockdebug dump when LOCKDEBUG is enabled,
instead of just the panic message (which includes the lock address
you could pass to `show lock' in ddb, but let's get the dump by
default even if you don't enter ddb).

Also in the KASSERT print the mutex.

drm: Allow ww_mutex_lock after ww_acquire_done if we already hold it.

In other words, if ww_mutex_lock would return -EALREADY, that's OK and
does not warrant an assertion.

PR kern/56557

Revert "drm: Fix missing newline in DRM_WARN."

Accidentally included more than I intended here.

drm: Fix missing newline in DRM_WARN.

drm: Spruce up ww_mutex comments. Audit return values.

linux/ww_mutex: Disable locking-against-self asserts in trylock.

This seems to be done intentionally in Linux.

Implement dma_resv_locking_ctx more carefully.

branches: 1.7.2;
Avoid prepending a timestamp to lock debug outputs on ddb

Lock printer functions (lockops_t#lo_dump) use printf_nolog to print, but
printf_nolog now prepends a timestamp which is unnecessary for ddb:

db{0}> show all locks/t
[Locks tracked through LWPs]
Locks held by an LWP (iperf):
Lock 0 (initialized at soinit)
lock address : 0xffffedeb84b06080 type : sleep/adaptive
initialized : 0xffffffff806d8c3f
shared holds : 0 exclusive: 1
shares wanted: 0 exclusive: 11
current cpu : 0 last held: 1
current lwp : 0xffffedeb849ff040 last held: 0xffffedeb7dfdb240
last locked* : 0xffffffff806d8335 unlocked : 0xffffffff806d8385
[ 79103.0868574] owner field : 0xffffedeb7dfdb240 wait/spin: 1/0

Fix it by passing a printer function to lo_dump functions, i.e., make the
functions use db_printf on ddb.

fix various problems i've seen where cv_*wait*() return ERESTART,
which is -3 in netbsd, which we have mapped linux ERESTARTSYS to.

this has a problem because linux code often returns errors and
pointers in the same value, and pointer values between -4095 and
-1 are considered as error returns, but -3 ends up as 3 and thus
is not considered an error, and mayhem ensues.

with this in place my kabylake system seems actually stable, i
have not triggered any of my prior issues in almost 4 weeks now.


Taylor asked me to write up a description and then wrote most of
the text below for me :-)


In Linux code, we always work with ERESTARTSYS so the code meaning
start over is a positive NetBSD errno safe for PTR_ERR/ERR_PTR.
To achieve this:
1. adapt all cv_waits that return to Linux so they map ERESTART to
ERESTARTSYS, and
2. adapt all returns to userland so they convert ERESTARTSYS to
ERESTART.
Leave EINTR and all other error codes alone.

Use Linux atomic64 for ww mutex class.

This way we can take advantage of the hash-locked atomic64 on
platforms that lack native atomic_inc_64_nv.

branches: 1.4.2; 1.4.4; 1.4.6;
more const

Fix lockdebug_locked annotations.

When thread A grants ownership to thread B waiting with a context,
thread B needs to assert lockdebug_locked; otherwise, when it
releases, lockdebug_unlocked thinks it's releasing an unlocked
ww_mutex.

Fixes LOCKDEBUG failure with radeon noticed by martin@.

branches: 1.2.10;
Adapt ww_mutex to use LOCKDEBUG.

Should help track down PR 49862.

branches: 1.1.2; 1.1.4;
Move Linux ww_mutex code into a .c file where it belongs.

linux_xa: Delete and replace collision in xa_store as intended.

Don't free the colliding node that's still in the tree.

Noted by rjs@.

branches: 1.3.4;
linux: Rework radix tree shims.

linux: Fix xa_find_after to actually find the entry _after_.

linux: Draft half-arsed xarray shims