<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> <meta name="Generator" content="Microsoft Word 14 (filtered medium)"> <style></style> </head> <body lang="ZH-CN" link="blue" vlink="purple" style="text-justify-trim:punctuation"> <div class="WordSection1"> This RFC is based on previous discussion to set up a generic communication channel between display and audio driver and<o:p></o:p> an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial draft and your advice would be appreciated<o:p></o:p> to improve the design.<o:p></o:p> <o:p> </o:p> The basic idea is to create a new avsink module and let both drm and alsa depend on it.<o:p></o:p> This new module provides a framework and APIs for synchronization between the display and audio driver. <o:p></o:p> <o:p> </o:p> 1. Display/Audio Client<o:p></o:p> <o:p> </o:p> The avsink core provides APIs to create, register and lookup a display/audio client.<o:p></o:p> A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver) can create a client, add some resources <o:p></o:p> objects (shared power wells, display outputs, and audio inputs, register ops) to the client, and then register this <o:p></o:p> client to avisink core. The peer driver can look up a registered client by a name or type, or both. If a client gives<o:p></o:p> a valid peer client name on registration, avsink core will bind the two clients as peer for each other. And we <o:p></o:p> expect a display client and an audio client to be peers for each other in a system.<o:p></o:p> <o:p> </o:p> int avsink_new_client ( const char *name, <o:p></o:p> int type, /* client type, display or audio */<o:p></o:p> struct module *module,<o:p></o:p> void *context,<o:p></o:p> const char *peer_name,<o:p></o:p> struct avsink_client **client_ret);<o:p></o:p> <o:p> </o:p> int avsink_free_client (struct avsink_client *client);<o:p></o:p> <o:p> </o:p> int avsink_register_client(struct avsink_client *client);<o:p></o:p> int avisink_unregister_client(int client_handle);<o:p></o:p> <o:p> </o:p> struct avsink_client *avsink_lookup_client(const char *name, int type);<o:p></o:p> <o:p> </o:p> struct avsink_client {<o:p></o:p> const char *name; /* client name */<o:p></o:p> int type; /* client type*/<o:p></o:p> void *context;<o:p></o:p> struct module *module; /* top-level module for locking */<o:p></o:p> <o:p></o:p> struct avsink_client *peer; /* peer client */<o:p></o:p> <o:p> </o:p> /* shared power wells */<o:p></o:p> struct avsink_power_well *power_well; <o:p></o:p> int num_power_wells;<o:p></o:p> <o:p></o:p> /* endpoints, display outputs or audio inputs */<o:p></o:p> struct avsink_endpoint * endpoint;<o:p></o:p> int num_endpints;<o:p></o:p> <o:p> </o:p> struct avsink_registers_ops *reg_ops; /* ops to access registers of a client */<o:p></o:p> void *private_data;<o:p></o:p> …<o:p></o:p> }; <o:p></o:p> <o:p> </o:p> On system boots, the avsink module is loaded before the display and audio driver module. And the display and audio <o:p></o:p> driver may be loaded on parallel. <o:p></o:p> * If a specific display driver (eg. i915) supports avsink, it can create a display client, add power wells and display<o:p></o:p> outputs to the client, and then register the display client to the avsink core. Then it may look up if there is any <o:p></o:p> audio client registered, by name or type, and may find an audio client registered by some audio driver.<o:p></o:p> <o:p> </o:p> * If an audio driver supports avsink, it usually should look up a registered display client by name or type at first, <o:p></o:p> because it may need the shared power well in GPU and check the display outputs’ name to bind the audio inputs. If <o:p></o:p> the display client is not registered yet, the audio driver can choose to wait (maybe in a work queue) or return <o:p></o:p> –EAGAIN for a deferred probe. After the display client is found, the audio driver can register an audio client with<o:p></o:p> the display client’s name as the peer name, the avsink core will bind the display and audio clients to each other.<o:p></o:p> <o:p> </o:p> Open question:<o:p></o:p> If the display or audio driver is disabled by the black list, shall we introduce a time out to avoid waiting for the <o:p></o:p> other client registered endlessly?<o:p></o:p> <o:p> </o:p> 2. Shared power wells (optional)<o:p></o:p> <o:p> </o:p> The audio and display devices, maybe only part of them, may share a common power well (e.g. for Intel Haswell and <o:p></o:p> Broadwell). If so, the driver that controls the power well should define a power well object, implement the get/put ops,<o:p></o:p> and add it to its avsink client before registering the client to avsink core. Then the peer client can look up this <o:p></o:p> power well by its name, and get/put this power well as a user. <o:p></o:p> <o:p> </o:p> A client can have multiple power well objects.<o:p></o:p> <o:p> </o:p> struct avsink_power_well {<o:p></o:p> const char *name; /* name of the power well */<o:p></o:p> void *context; /* parameter of get/put ops, maybe device pointer for this power well */<o:p></o:p> struct avsink_power_well_ops *ops<o:p></o:p> };<o:p></o:p> <o:p> </o:p> struct avsink_power_well_ops {<o:p></o:p> int (*get)(void *context);<o:p></o:p> int (*put)(void *context);<o:p></o:p> };<o:p></o:p> <o:p> </o:p> API:<o:p></o:p> int avsink_new_power(struct avsink_client *client, <o:p></o:p> const char *power_name,<o:p></o:p> void * power_context,<o:p></o:p> struct avsink_power_well_ops *ops,<o:p></o:p> struct avsink_power_well **power_ret);<o:p></o:p> <o:p> </o:p> struct avsink_power_well *avisnk_lookup_power(const char *name);<o:p></o:p> <o:p> </o:p> int avsink_get_power(struct avsink_power_well *power); /* Reqesut the power */<o:p></o:p> int avsink_put_power(struct avsink_power_well *power); /* Release the power */<o:p></o:p> <o:p> </o:p> For example, the i915 display driver can create a device for the shared power well in Haswell GPU, implement its PM <o:p></o:p> functions, and use the device pointer as the context when creating the power well object, like below <o:p></o:p> <o:p> </o:p> struct avsink_power_well_ops i915_power_well_ops = {<o:p></o:p> .get = pm_runtime_get_sync;<o:p></o:p> .put = pm_runtime_put_sync;<o:p></o:p> };<o:p></o:p> ...<o:p></o:p> avsink_new_power ( display_client,<o:p></o:p> "i915_display_power_well",<o:p></o:p> pdev, /* pointer of the power well device */<o:p></o:p> &i915_power_well_ops,<o:p></o:p> …)<o:p></o:p> <o:p> </o:p> Power domain is not used here since a single device seems enough to represent a power well. <o:p></o:p> <o:p> </o:p> 3. Display output and audio input endpoints<o:p></o:p> <o:p> </o:p> A display client should register the display output endpoints and its audio peer client should register the audio input<o:p></o:p> endpoints. A client can have multiple endpoints. The avsink core will bind an audio input and a display output as peer <o:p></o:p> to each other. This is to allow the audio and display driver to synchronize with each other for each display pipeline.<o:p></o:p> <o:p> </o:p> All endpoints should be added to a client before the client is registered to avsink core. Dynamic endpoints are not <o:p></o:p> supported now.<o:p></o:p> <o:p> </o:p> A display out here represents a physical HDMI/DP output port. And as long as it’s usable in the system (i.e. physically<o:p></o:p> connected to the HDMP/DP port on the machine board), the display output should be registered not matter the port is <o:p></o:p> connected to an external display device or not. And if HW and display driver can support DP1.2 daisy chain (multiple DP<o:p></o:p> display devices can be connected to a single port), multiple static displays outputs should be defined for the DP port <o:p></o:p> according to the HW capability. The port & display device number can be indicated by the name (e.g. "i915_DDI_B", <o:p></o:p> "i915_DDI_B_DEV0", "i915_DDI_B_DEV1", or "i915_DDI_B_DEV2"), defined by the display driver.<o:p></o:p> <o:p> </o:p> The audio driver can check the endpoints of its peer display client and use an display endpoint’s name, or a presumed <o:p></o:p> display endpoint name, as peer name when registering an audio endpoint, thus the avsink core will bind the two display<o:p></o:p> and audio endpoints as peers.<o:p></o:p> <o:p> </o:p> struct avsink_endpoint {<o:p></o:p> const char *name; /*name of the endpoint */<o:p></o:p> int type; /* DISPLAY_OUTPUT or AUDIO_INPUT */<o:p></o:p> void *context; /* private data, used as parameter of the ops */<o:p></o:p> struct avsink_endpoint_ops *ops;<o:p></o:p> <o:p> </o:p> struct avsink_endpoint *peer; /* peer endpoint */<o:p></o:p> };<o:p></o:p> <o:p> </o:p> struct avsink_endpoint_ops {<o:p></o:p> int (*get_caps) (enum had_caps_list query_element,<o:p></o:p> void *capabilities,<o:p></o:p> void *context);<o:p></o:p> int (*set_caps) (enum had_caps_list set_element,<o:p></o:p> void *capabilities,<o:p></o:p> void *context);<o:p></o:p> int (*event_handler) (enum avsink_event_type event_type, void *context);<o:p></o:p> };<o:p></o:p> <o:p> </o:p> API:<o:p></o:p> int avsink_new_endpoint (struct avsink_client *client,<o:p></o:p> const char *name,<o:p></o:p> int type, /* DISPLAY_OUTPUT or AUDIO_INPUT*/<o:p></o:p> void *context,<o:p></o:p> const char *peer_name, /* can be NULL if no clue */<o:p></o:p> avsink_endpoint_ops *ops,<o:p></o:p> struct avsink_endpoint **endpoint_ret);<o:p></o:p> <o:p> </o:p> int avsink_endpoint_get_caps(struct avsink_endpoint *endpoint, <o:p></o:p> enum avsink_caps_list t get_element, <o:p></o:p> void *capabilities);<o:p></o:p> int avsink_endpoint_set_caps(struct avsink_endpoint *endpoint, <o:p></o:p> enum had_caps_list set_element, <o:p></o:p> void *capabilities);<o:p></o:p> <o:p> </o:p> int avsink_endpoint_post_event(struct avsink_endpoint *endpoint, <o:p></o:p> enum avsink_event_type event_type);<o:p></o:p> <o:p> </o:p> 4. Get/Set caps on an endpoint<o:p></o:p> <o:p> </o:p> The display or audio driver can get or set capabilities on an endpoint. Depending on the capability ID, the avsink core<o:p></o:p> will call get_caps/set_caps ops of this endpoint, or call get_caps/set_caps ops of its peer endpoint and return the <o:p></o:p> result to the caller.<o:p></o:p> <o:p> </o:p> enum avsink_caps_list {<o:p></o:p> /* capabilities for display output endpoints */<o:p></o:p> AVSINK_GET_DISPLAY_ELD = 1,<o:p></o:p> AVSINK_GET_DISPLAY_TYPE, /* HDMI or DisplayPort */<o:p></o:p> AVSINK_GET_DISPLAY_NAME, /* Hope to use display device name under /sys/class/drm, like “card0-DP-1”, for user<o:p></o:p> * space to figure out which HDMI/DP output on the drm side corresponds to which audio <o:p></o:p> * stream device on the alsa side */<o:p></o:p> AVSINK_GET_DISPLAY_SAMPLING_FREQ, /* HDMI TMDS clock or DP link symbol clock, for audio driver to <o:p></o:p> * program N value<o:p></o:p> */<o:p></o:p> AVSINK_GET_DISPLAY_HDCP_STATUS,<o:p></o:p> AVSINK_GET_DISPLAY_AUDIO_STATUS, /* Whether audio is enabled */ <o:p></o:p> AVSINK_SET_DISPLAY_ENABLE_AUDIO, /* Enable audio */<o:p></o:p> AVSINK_SET_DISPLAY_DISABLE_AUDIO, /* Disable audio */<o:p></o:p> AVSINK_SET_DISPLAY_ENABLE_AUDIO_INT, /* Enable audio interrupt */<o:p></o:p> AVSINK_SET_DISPLAY_DISABLE_AUDIO_INT, /* Disable audio interrupt */<o:p></o:p> <o:p> </o:p> /* capabilities for audio input endpoints */<o:p></o:p> AVSINK_GET_AUDIO_IS_BUSY, /* Whether there is an active audio streaming */<o:p></o:p> OTHERS_TBD,<o:p></o:p> };<o:p></o:p> <o:p> </o:p> For example, the audio driver can query ELD info on an audio input endpoint by using caps AVSINK_GET_DISPLAY_ELD, and<o:p></o:p> avsink core will call get_caps() on the peer display output endpoint and return the ELD info to the audio driver.<o:p></o:p> <o:p> </o:p> Some audio driver may only use part of these caps. E.g. HD-Audio driver can use bus commands instead of the ops to <o:p></o:p> control the audio on gfx side, so it doesn’t use caps like ENABLE/DISABLE_AUDIO or ENABLE/DISABLE_AUDIO. <o:p></o:p> <o:p> </o:p> When the display driver want to disable a display pipeline for hot-plug, mode change or power saving, it can use caps <o:p></o:p> AVSINK_GET_AUDIO_IS_BUSY to check if the audio input is busy (active streaming) on this display pipeline. And if audio<o:p></o:p> is busy, the display driver can choose to wait or go ahead to disable display pipeline anyway. For the latter case, the<o:p></o:p> audio input endpoint will be notified by an event and should abort audio streaming.<o:p></o:p> <o:p> </o:p> 5. Event handling of endpoints<o:p></o:p> <o:p> </o:p> A driver can post events on an endpoint. Depending on the event type, the avsink core will call the endpoint’s event <o:p></o:p> handler or pass the event to its peer endpoint and trigger the peer’s event handler function if defined.<o:p></o:p> <o:p> </o:p> int avsink_endpoint_post_event(struct avsink_endpoint *endpoint, <o:p></o:p> enum avsink_event_type event_type);<o:p></o:p> <o:p> </o:p> Now we only defined event types which should be handled by the audio input endpoints. The event types can be extended<o:p></o:p> in the future.<o:p></o:p> <o:p> </o:p> enum avsink_event_type {<o:p></o:p> AVSINK_EVENT_DISPLAY_DISABLE = 1, /* The display pipeline is disabled for hot-plug, mode change or <o:p></o:p> * suspend. Audio driver should stop any active streaming.<o:p></o:p> */<o:p></o:p> AVSINK_EVENT_DISPLAY_ENABLE, /* The display pipeline is enabled after hot-plug, mode change or <o:p></o:p> * resume. Audio driver can restore previously interrupted streaming <o:p></o:p> */<o:p></o:p> AVSINK_EVENT_DISPLAY_MODE_CHANGE, /* Display mode change event. At this time, the new display mode is <o:p></o:p> * configured but the display pipeline is not enabled yet. Audio driver<o:p></o:p> * can do some configurations such as programing N value */<o:p></o:p> AVSINK_EVENT_DISPLAY_AUDIO_BUFFER_DONE, /* Audio Buffer done interrupts. Only for audio drivers if DMA and <o:p></o:p> * interrupt are handled by GPU<o:p></o:p> */<o:p></o:p> AVSINK_EVENT_DISPLAY_AUDIO_BUFFER_UNDERRUN, /* Audio Buffer under run interrupts. Only for audio drivers if <o:p></o:p> * DMA and interrupt are handled by GPU<o:p></o:p> */<o:p></o:p> };<o:p></o:p> <o:p> </o:p> So for a display driver, it can post an event on a display output endpoint and get processed by the peer audio input <o:p></o:p> endpoint. Or it can also directly post an event on a peer audio input endpoint, by using the 'peer' pointer on a <o:p></o:p> display output endpoint. <o:p></o:p> <o:p> </o:p> 6. Display register operation (optional)<o:p></o:p> <o:p> </o:p> Some audio driver needs to access GPU audio registers. The register ops are provided by the peer display client. <o:p></o:p> <o:p> </o:p> struct avsink_registers_ops {<o:p></o:p> int (*read_register) (uint32_t reg_addr, uint32_t *data, void *context);<o:p></o:p> int (*write_register) (uint32_t reg_addr, uint32_t data, void *context);<o:p></o:p> int (*read_modify_register) (uint32_t reg_addr, uint32_t data, uint32_t mask, void *context);<o:p></o:p> <o:p> </o:p> int avsink_define_reg_ops (struct avsink_client *client, struct avsink_registers_ops *ops);<o:p></o:p> <o:p> </o:p> And avsink core provides API for the audio driver to access the display registers:<o:p></o:p> <o:p> </o:p> int avsink_read_display_register(struct avsink_client *client , uint32_t offset, uint32_t *data);<o:p></o:p> int avsink_write_display_register(struct avsink_client *client , uint32_t offset, uint32_t data);<o:p></o:p> int avsink_read_modify_display_register(struct avsink_client *client, uint32_t offset, uint32_t data, uint32_t mask);<o:p></o:p> <o:p> </o:p> If the client is an audio client, the avsink core will find it peer display client and call its register ops; <o:p></o:p> and if the client is a display client, the avsink core will just call its own register ops.<o:p></o:p> <o:p> </o:p> Thanks<o:p></o:p> Mengdong<o:p></o:p> <o:p> </o:p> </div> </body> </html>