<div dir="auto">I don't think this was preventable given how many things went wrong. Sometimes we have to accept that shit happens. We can fix that in the next release.<div dir="auto"> </div><div dir="auto">Marek</div></div> <div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu., Apr. 23, 2020, 07:01 Danylo Piliaiev, <<a href="mailto:danylo.piliaiev@gmail.com">danylo.piliaiev@gmail.com</a>> wrote: </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I want to sum up what happened from my perspective, I think it could be useful to improve the process: 1) There was a regression in 20.* (<a href="https://gitlab.freedesktop.org/mesa/mesa/-/issues/2758" rel="noreferrer noreferrer" target="_blank">https://gitlab.freedesktop.org/mesa/mesa/-/issues/2758</a>) 2) I "fixed" the regression and broke non-iris drivers (<a href="https://gitlab.freedesktop.org/mesa/mesa/-/commit/d684fb37bfbc47d098158cb03c0672119a4469fe" rel="noreferrer noreferrer" target="_blank">https://gitlab.freedesktop.org/mesa/mesa/-/commit/d684fb37bfbc47d098158cb03c0672119a4469fe</a>) 3) I "fixed" the new regression of fix 2) (<a href="https://gitlab.freedesktop.org/mesa/mesa/-/commit/829013d0cad0fa2513b32ae07cf8d745f6e5c62d" rel="noreferrer noreferrer" target="_blank">https://gitlab.freedesktop.org/mesa/mesa/-/commit/829013d0cad0fa2513b32ae07cf8d745f6e5c62d</a>) 4) After that, it appeared that due to a bug in piglit, Intel CI didn't run piglit tests which gave me a false sense of commit's correctness (<a href="https://gitlab.freedesktop.org/mesa/mesa/-/issues/2815" rel="noreferrer noreferrer" target="_blank">https://gitlab.freedesktop.org/mesa/mesa/-/issues/2815</a>) 5) I aimed to have a fix before the next minor release on 2020-04-29 by consulting the release calendar. 6) I accidentally saw 20.0.5 being released with one of the two of my commits. I see multiple failure points: 1) Me not carefully examining all code paths, because at least one that failed should have been obvious to test. 2) CI not communicating that piglit tests were not executed. Again, I could have seen this, examined CI results, but did not. 3) After restoration of CI capabilities test what added to "expected failure" and this may have contributed to regression being missed when testing the release. I'm not sure about this part so correct me if I'm wrong. 4) I didn't know about this release and that this release was help up for the fix of 2758. 5) There were now window between announcing the scope of the release and release itself. Since I knew about regression I could have notified about it. Also there is no milestone for minor releases so it's problematic to link issue and release. It's a second release in a row with clear regression crept in. I believe that we can use this to improve the process and safeguard against such regressions in the future. P.S. I'm preparing and will test a final fix which will be sent soon. Danylo On 23.04.20 07:40, Dylan Baker wrote: > Quoting Ilia Mirkin (2020-04-22 15:47:59) >> On Wed, Apr 22, 2020 at 6:39 PM Danylo Piliaiev >> <<a href="mailto:danylo.piliaiev@gmail.com" target="_blank" rel="noreferrer">danylo.piliaiev@gmail.com</a>> wrote: >>> I'm sorry for this trouble. However looking at it I think: maybe something could be >>> changed about applying patches to stable to safeguard against such issues. >> We used to get pre-announcements a few days prior to a release which >> would allow developers to look things over, which would allow us to >> notice things like that. Not sure when this got dropped. >> >> Cheers, >> >> -ilia > That was dropped in favor of a live staging branch that is updated daily (at > least on week days). > > Dylan _______________________________________________ mesa-dev mailing list <a href="mailto:mesa-dev@lists.freedesktop.org" target="_blank" rel="noreferrer">mesa-dev@lists.freedesktop.org</a> <a href="https://lists.freedesktop.org/mailman/listinfo/mesa-dev" rel="noreferrer noreferrer" target="_blank">https://lists.freedesktop.org/mailman/listinfo/mesa-dev</a> </blockquote></div>