<html><body><p> <pre> On Thu, 2024-05-09 at 11:27 +0200, AngeloGioacchino Del Regno wrote: > Il 09/05/24 07:42, CK Hu (胡俊光) ha scritto: > > On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno > > wrote: > > > Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: > > > > On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del Regno > > > > wrote: > > > > > Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: > > > > > > On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del > > > > > > Regno > > > > > > wrote: > > > > > > > Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: > > > > > > > > Hi, Angelo: > > > > > > > > > > > > > > > > On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del > > > > > > > > Regno > > > > > > > > wrote: > > > > > > > > > Document OF graph on MMSYS/VDOSYS: this supports up > > > > > > > > > to > > > > > > > > > three > > > > > > > > > DDP > > > > > > > > > paths > > > > > > > > > per HW instance (so potentially up to six displays > > > > > > > > > for > > > > > > > > > multi- > > > > > > > > > vdo > > > > > > > > > SoCs). > > > > > > > > > > > > > > > > > > The MMSYS or VDOSYS is always the first component in > > > > > > > > > the > > > > > > > > > DDP > > > > > > > > > pipeline, > > > > > > > > > so it only supports an output port with multiple > > > > > > > > > endpoints - > > > > > > > > > where > > > > > > > > > each > > > > > > > > > endpoint defines the starting point for one of the > > > > > > > > > (currently > > > > > > > > > three) > > > > > > > > > possible hardware paths. > > > > > > > > > > > > > > > > > > Signed-off-by: AngeloGioacchino Del Regno < > > > > > > > > > angelogioacchino.delregno@collabora.com> > > > > > > > > > --- > > > > > > > > > .../bindings/arm/mediatek/mediatek,mmsys.yaml | > > > > > > > > > 23 > > > > > > > > > +++++++++++++++++++ > > > > > > > > > 1 file changed, 23 insertions(+) > > > > > > > > > > > > > > > > > > diff --git > > > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/medi > > > > > > > > > atek > > > > > > > > > ,mms > > > > > > > > > ys.y > > > > > > > > > aml > > > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/medi > > > > > > > > > atek > > > > > > > > > ,mms > > > > > > > > > ys.y > > > > > > > > > aml > > > > > > > > > index b3c6888c1457..4e9acd966aa5 100644 > > > > > > > > > --- > > > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/medi > > > > > > > > > atek > > > > > > > > > ,mms > > > > > > > > > ys.y > > > > > > > > > aml > > > > > > > > > +++ > > > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/medi > > > > > > > > > atek > > > > > > > > > ,mms > > > > > > > > > ys.y > > > > > > > > > aml > > > > > > > > > @@ -93,6 +93,29 @@ properties: > > > > > > > > > '#reset-cells': > > > > > > > > > const: 1 > > > > > > > > > > > > > > > > > > + port: > > > > > > > > > + $ref: /schemas/graph.yaml#/properties/port > > > > > > > > > + description: > > > > > > > > > + Output port node. This port connects the > > > > > > > > > MMSYS/VDOSYS > > > > > > > > > output > > > > > > > > > to > > > > > > > > > + the first component of one display pipeline, > > > > > > > > > for > > > > > > > > > example > > > > > > > > > one > > > > > > > > > of > > > > > > > > > + the available OVL or RDMA blocks. > > > > > > > > > + Some MediaTek SoCs support up to three display > > > > > > > > > outputs > > > > > > > > > per > > > > > > > > > MMSYS. > > > > > > > > > + properties: > > > > > > > > > + endpoint@0: > > > > > > > > > + $ref: > > > > > > > > > /schemas/graph.yaml#/properties/endpoint > > > > > > > > > + description: Output to the primary display > > > > > > > > > pipeline > > > > > > > > > + > > > > > > > > > + endpoint@1: > > > > > > > > > + $ref: > > > > > > > > > /schemas/graph.yaml#/properties/endpoint > > > > > > > > > + description: Output to the secondary display > > > > > > > > > pipeline > > > > > > > > > + > > > > > > > > > + endpoint@2: > > > > > > > > > + $ref: > > > > > > > > > /schemas/graph.yaml#/properties/endpoint > > > > > > > > > + description: Output to the tertiary display > > > > > > > > > pipeline > > > > > > > > > + > > > > > > > > > + required: > > > > > > > > > + - endpoint@0 > > > > > > > > > + > > > > > > > > > > > > > > > > mmsys/vdosys does not output data to the first > > > > > > > > component of > > > > > > > > display > > > > > > > > pipeline, so this connection looks 'virtual'. Shall we > > > > > > > > add > > > > > > > > something > > > > > > > > virtual in device tree? You add this in order to decide > > > > > > > > which > > > > > > > > pipeline > > > > > > > > is 1st, 2nd, 3rd, but for device it don't care which > > > > > > > > one is > > > > > > > > first. > > > > > > > > In > > > > > > > > computer, software could change which display is the > > > > > > > > primary > > > > > > > > display. > > > > > > > > I'm not sure it's good to decide display order in > > > > > > > > device > > > > > > > > tree? > > > > > > > > > > > > > > > > > > > > > > Devicetree describes hardware, so nothing virtual can be > > > > > > > present > > > > > > > - > > > > > > > and in any case, > > > > > > > the primary/secondary/tertiary pipeline is in relation to > > > > > > > MM/VDO > > > > > > > SYS, > > > > > > > not referred > > > > > > > to software. > > > > > > > > > > > > > > Better explaining, the primary pipeline is not > > > > > > > necessarily > > > > > > > the > > > > > > > primary display in > > > > > > > DRM terms: that's a concept that is completely detached > > > > > > > from > > > > > > > the > > > > > > > scope of this > > > > > > > series and this graph - and it's something that shall be > > > > > > > managed > > > > > > > solely by the > > > > > > > driver (mediatek-drm in this case). > > > > > > > > > > > > > > Coming back to the connection looking, but *not* being > > > > > > > virtual: > > > > > > > the > > > > > > > sense here is > > > > > > > that the MM/VDOSYS blocks are used in the display > > > > > > > pipeline to > > > > > > > "stitch" together > > > > > > > the various display pipeline hardware blocks, or, said > > > > > > > differently, > > > > > > > setting up the > > > > > > > routing between all of those (P.S.: > > > > > > > mmsys_mtxxxx_routing_table!) > > > > > > > through the VDO > > > > > > > Input Selection (VDOx_SEL_IN) or Output Selection > > > > > > > (VDOx_SEL_OUT) > > > > > > > and > > > > > > > with the > > > > > > > assistance of the VDO Multiple Output Mask (VDOx_MOUT) > > > > > > > for > > > > > > > the > > > > > > > multiple outputs > > > > > > > usecase, both of which, are described by this graph. > > > > > > > > > > > > I agree this part, but this is related to display device OF > > > > > > graph. > > > > > > These display device would output video data from one > > > > > > device > > > > > > and > > > > > > input > > > > > > to another video device. These video device would not input > > > > > > or > > > > > > output > > > > > > video data to mmsys/vdosys. > > > > > > > > > > > > > > > > > > > > This means that the VDOSYS is really the "master" of the > > > > > > > display > > > > > > > pipeline since > > > > > > > everything gets enabled, mixed and matched from there - > > > > > > > and > > > > > > > that's in > > > > > > > the sense > > > > > > > of hardware operation, so we are *really* (and not > > > > > > > virtually!) > > > > > > > flipping switches. > > > > > > > > > > > > I agree mmsys/vdosys is master of video pipeline, so let's > > > > > > define > > > > > > what > > > > > > the port in mmsys/vdosys is. If the port means the master > > > > > > relationship, > > > > > > mmsys/vdosys should output port to every display device. Or > > > > > > use > > > > > > a > > > > > > simply way to show the master relation ship > > > > > > > > > > > > mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, > > > > > > &rdma1, > > > > > > &color1, > > > > > > ...>; > > > > > > > > > > > > > > > > There's no need to list all of the VDO0/VDO1/mmsys devices in > > > > > one > > > > > big > > > > > array > > > > > property, because the actual possible devices can be defined: > > > > > 1. In the bindings; and > > > > > 2. In the actual OF graph that we write for each > > > > > SoC+board > > > > > combination. > > > > > > > > > > A graph cannot contain a connection to a device that cannot > > > > > be > > > > > connected to > > > > > the previous, so, your "mmsys-subdev" list can be retrieved > > > > > by > > > > > looking at the > > > > > graph: > > > > > - Start from VDO0/1 or MMSYS > > > > > - Walk through (visually, even) OUTPUT ports > > > > > - VDO0 (read output ep) -> ovl0 (read output ep) -> > > > > > rdma0 > > > > > (read > > > > > output ep) -> > > > > > color0 (...) -> etc > > > > > - Nothing more - it's all defined there. > > > > > > > > > > > > > > > > > Another problem is how to group display device? If two > > > > > > pipeline > > > > > > could > > > > > > be route to the same display interface, such as > > > > > > > > > > > > rdma0 -> dsi > > > > > > rdma1 -> dsi > > > > > > > > > > > > Would this be single group? > > > > > > > > > > There are multiple ways of doing this, but one that comes to > > > > > my > > > > > mind > > > > > right now and > > > > > that looks clean as well is the following: > > > > > > > > > > ovl0@ef01 { > > > > > ..... > > > > > ports { > > > > > port@0 { > > > > > reg = <0>; > > > > > ovl0_in: endpoint { > > > > > remote-endpoint = <&vdosys0_out>; > > > > > }; > > > > > }; > > > > > > > > I'm not sure how do you define this port from OVL to vdosys. If > > > > this > > > > port means 'master relationship', others could add port in > > > > COLOR to > > > > point to vdosys because COLOR and vdosys has the 'master > > > > relationship' > > > > and I could not reject this. So we need more specific > > > > definition of > > > > this port. > > > > > > > > > > Only the 'first' device in pipeline could have this port? > > > > > > Correct. Only the first device in a pipeline - and this is > > > actually a > > > restriction > > > that the generic binding definition of port already gives, in a > > > way. > > > > > > > > > > In mt8173, one pipeline is > > > > > > > > ovl -> color -> aal -> od -> rdma -> ufo -> dsi > > > > > > > > But rdma has an option to read data from od or directly from > > > > DRAM. > > > > If > > > > from DRAM, the pipeline would be changed to > > > > > > > > rdma -> ufo -> dsi > > > > > > > > > > > > So it's confused which one is 'first'. > > > > > > That's why the pipeline is *board-specific* and not soc-generic! > > > > > > And what you described is *exactly* the reason why I'm adding > > > support > > > for the > > > OF graphs in mediatek-drm: specifying the correct pipeline for > > > each > > > board as per > > > what each board wants to use (said differently: for each board's > > > *capabilities*). > > > > > > So, if on a certain board you want to skip OD, you can hook RDMA > > > up > > > directly to > > > MMSYS/VDOSYS. > > > > > > In MT8173, one pipeline for one board uses endpoints IN/OUT like > > > this: > > > > > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > > > > > and for another board, endpoints will be like > > > > > > MMSYS -> RDMA -> UFO -> DSI > > > > > > ...which is the exact same as you described, and I think that > > > your > > > confusion comes > > > from the fact that you didn't put MMSYS at the beginning of the > > > pipeline :-) > > > > In one board, both OVL and RDMA could switch dynamically. Because > > each > > one could be the first in one board, mmsys point to both ovl and > > rdma? > > > > No. > > MMSYS would still point ONLY to OVL, because OVL is the "earliest > point" > of the pipeline that this one board does support. > > In that case, RDMA being present at a later point in the pipeline > does not > matter and does not prevent us from *temporarily* skipping OVL-COLOR- > AAL-OD > and going MMSYS->RDMA *directly*. > > Switching dynamically is a driver duty and will be 100% possible (as > much > as it is right now) to dynamically switch OVL and RDMA as long as > both are > present in the pipeline. > > With this exact pipeline: > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > the driver _can switch dynamically_ between MMSYS->OVL->...->RDMA and > MMSYS->RDMA as the driver itself *is allowed to* temporarily ignore > part > of the pipeline. > > Please note that, in case it is needed (trust me on this: it's not > needed) > a custom property in the endpoint node can always be introduced > later, so > that you can declare a node like > > endpoint@0 { > remote-endpoint = <&ovl0_in>; > mediatek,short-path = <&rdma0_in>; > }; > > ...but again, that's never going to be needed, as the driver already > does > have knowledge of the fact that RDMA is in the pipeline, so it is > possible > to simply do a temporary override in the driver. > > What the OF Graph support does is to build the same arrays, that we > currently > have hardcoded in mediatek-drm, by reading from device tree. > > Nothing else and nothing more - for now. > > Having the OF Graph support makes us able to also add new dual-path > support > with more complicated connections than the current ones, without any > problem > and, in many cases, without even modifying the bindings from what I > provided > in this series. OK, please add the information we discussed into binding document to prevent anyone misusing this binding. Regards, CK > > Cheers, > Angelo > > > Regards, > > CK > > > > > > > > > > > > > > > > > In case you need any *temporary override* on any board that > > > defines a > > > pipeline like > > > > > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > > > > > so that the pipeline *temporarily* becomes (for power management, > > > or > > > for any other > > > reason) RDMA -> UFO -> DSI .... that's not a concern: the graph > > > is > > > present, and it > > > is used to tell to the driver what is the regular pipeline to > > > use. > > > Eventual temporary overrides can be managed transparently inside > > > of > > > the driver with > > > C code and no changes to the devicetree are required. > > > > > > > > > > I don't know how to decide which device could point to > > > > mmsys/vdosys. So > > > > please give a specific definition. > > > > > > > > > > Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing to a > > > device. > > > > > > So, mmsys/vdosys must point to the *first device in the > > > pipeline*. > > > > > > Any other doubt? > > > > > > Cheers, > > > Angelo > > > > > > > Regards, > > > > CK > > > > > > > > > > > > > > port@1 { > > > > > reg = <1>; > > > > > ovl0_out0: endpoint@0 { > > > > > remote-endpoint = <&rdma0_in>; > > > > > }; > > > > > ovl0_out1: endpoint@1 { > > > > > remote-endpoint = <&rdma1_in>; > > > > > }; > > > > > }; > > > > > }; > > > > > }; > > > > > > > > > > rdma0@1234 { > > > > > ..... > > > > > ports { > > > > > port@0 { > > > > > reg = <0>; > > > > > rdma0_in: endpoint { > > > > > remote-endpoint = <&ovl0_out0>; /* assuming ovl0 > > > > > outputs to > > > > > rdma0...*/ > > > > > }; > > > > > }; > > > > > port@1 { > > > > > reg = <1>; > > > > > rdma0_out: endpoint@1 { > > > > > remote-endpoint = <&dsi_dual_intf0_in>; > > > > > }; > > > > > }; > > > > > }; > > > > > }; > > > > > > > > > > > > > > > rdma1@5678 { > > > > > ..... > > > > > ports { > > > > > port@0 { > > > > > reg = <0>; > > > > > rdma1_in: endpoint { > > > > > /* assuming ovl0 outputs to rdma1 as well... can > > > > > be > > > > > something else. */ > > > > > remote-endpoint = <&ovl0_out1>; > > > > > }; > > > > > }; > > > > > port@1 { > > > > > reg = <1>; > > > > > rdma1_out: endpoint { > > > > > remote-endpoint = <&dsi_dual_intf1_in>; > > > > > }; > > > > > }; > > > > > }; > > > > > }; > > > > > > > > > > > > > > > dsi@9abcd { > > > > > ..... > > > > > ports { > > > > > port@0 { > > > > > reg = <0>; > > > > > /* Where endpoint@0 could be always DSI LEFT CTRL */ > > > > > dsi_dual_intf0_in: endpoint@0 { > > > > > remote-endpoint = <&rdma0_out>; > > > > > }; > > > > > /* ...and @1 could be always DSI RIGHT CTRL */ > > > > > dsi_dual_intf1_in: endpoint@1 { > > > > > remote-endpoint = <&rdma1_out>; > > > > > }; > > > > > }; > > > > > > > > > > port@1 { > > > > > reg = <1>; > > > > > dsi0_out: endpoint { > > > > > remote-endpoint = <&dsi_panel_in>; > > > > > }; > > > > > }; > > > > > }; > > > > > }; > > > > > > > > > > ...for a dual-dsi panel, it'd be a similar graph. > > > > > > > > > > Cheers, > > > > > Angelo > > > > > > > > > > > > > > > > > mmsys-subdev = <&rdma0, &rdma1, &dsi>; > > > > > > > > > > > > Or two group? > > > > > > > > > > > > mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; > > > > > > > > > > > > I think we should clearly define this. > > > > > > > > > > > > Regards, > > > > > > CK > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > Angelo > > > > > > > > > > > > > > > Regards, > > > > > > > > CK > > > > > > > > > > > > > > > > > > > > > > > > > required: > > > > > > > > > - compatible > > > > > > > > > - reg > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > </pre> </p></body></html><pre>************* MEDIATEK Confidentiality Notice ******************** The information contained in this e-mail message (including any attachments) may be confidential, proprietary, privileged, or otherwise exempt from disclosure under applicable laws. It is intended to be conveyed only to the designated recipient(s). Any use, dissemination, distribution, printing, retaining or copying of this e-mail (including its attachments) by unintended recipient(s) is strictly prohibited and may be unlawful. If you are not an intended recipient of this e-mail, or believe that you have received this e-mail in error, please notify the sender immediately (by replying to this e-mail), delete any and all copies of this e-mail (including any attachments) from your system, and do not disclose the content of this e-mail to any other person. Thank you! </pre>