Compatibility/Meetings/2022-07-05

• Web Compatibility Meeting Meeting - 2022-07-05
• Minutes: Previous 2022-06-28

Minutes

Scribed by Dennis

KB: Whiteboard flag for bugs that are in KB? (Honza)

• Honza: Did we agree on adding a flag? Did we agree on how to store that information?
• Dennis: What's your motivation for tracking it on Bugzilla as well?
• Tom: The motivation is for people looking at Bugzilla being able to discover the Knowledge Base. A whiteboard flag might not be the best since it should contain a link to an entry, but maybe a see-also.
• James: For people in the team, searching the KB is more important. If you want to link people to something, you can link to GitHub. If you want to have a broader audience, a more presented version would be better rather than pointing to a YAML file. There might be some value in the future for people outside the team.
• Tom: I agree. We have to make sure that all the Bugzilla bugs are tracked in the entries.
• James: Every KB entry should have a link to at least one bug. Maybe we should enforce this (see the open issues about requiring some fields).
• Dennis: We have a field in the schema for a list of bugs. We should make that required. Linking on bugzilla could be useful for people outside the team, but at the moment we are the main audience so we don't need a backlink in the short term; we have enough information to add it later.
• Tom: The list of bugs might not be fully accurate right away.
• Dennis: That seems OK, but we should make the field required.

KB: How to track duplicates on GH issues? (Honza)

• Honza: we were talking about that in the triage meeting. We currently don't have a way to track duplicatates.
• Dennis: It shows the issue as mentioned in GH, but there isn't the explicit concept of "duplicate" in GH. We should use the site_breakage entry in the kb to track all dupes. If we just close an entry we can miss adding the kb entry. Maybe we should have a label so that we can keep track of the ones that we need to add. Then we set the label synchronously when closing and link later.
• Honza: we also filed issues for creating new entries.
• Dennis: We could just have one label "needs-knowledge-base" and then rely on people being able to remember what the desired outcome is, or use a comment to specify what the action should be.
• James: I agree. We should have a list of all duplicates. Some end on bugzilla, some get tracked on Bugzilla, but it's really hard to keep track of duplicates.
• Honza: What do you think about labels vs. comments?
• James: If I understood Dennis correctly, the label is to indicate "some follow-up work should be happening here", and the comment should clarify what should be done.
• Dennis: James is correct.
• [TODO] Dennis to create a `action-needs-knowledgebase` label.

KB: Adding entries about things that are not supported? (Honza)

• Honza: Do we want to add KB entries for things Firefox does not support.
• James: For context, we came accros an issue based on Web Serial. Standards Position is "harmful", so we're unlikely to implement it. Some issues are caused by things we *probably* implement, but there are no specific plans yet.
• James: The question was if we should track those things, so that we're able to say "hey, there's this 1000 sites that break due to Web Serial"
• Tom: I'd err on tracking too much vs too little. At least there should be a metabug.
• Dennis: I think the overhead of creating a new entry is small enough to justify just doing it.
• Joe: Is it possible to go through all the "harmful" standards positions and create an entry for those?
• James: yes. But there's not too many of those, so manually going through the list might just be quicker.
• Tom: Should we create entries for all harmful positions, or just for the ones we're seeing breakage?
• Dennis: I think we're fine just creating entries when we see things breaking. But the number is small enough, it probably doesn't matter.
• James: Do we need a specific field to track the standard position, or does it fit into one of the existing fields?
• Tom: It seems reasonable to add an optional field for the standards position.
• Dennis: I think it's reasonable to create a new field.
• Tom: Do we want to keep track of discussions in standards bodies around known incompatibilities?
• Dennis: A field like `standards_discussions` might be good for both Mozilla Standards Positions, but also for links pointing towards WG discussions etc.
• James: I think that happens frequently enough. Those could be just be tracked as platform issues, or in a new section. I think there should be no more than one standards position about a spec, but there could be multiple spec discussions.
• Tom: There's also the standard positions of other vendors.
• Joe: I think it's relevant to be able to see that everyone else in favor of something if we prioritize something.
• Tom: That might also be useful in the opposite case: if everyone but one vendor is against a spec, then that's also a signal.
• James: I think the only case where the standards position can contribute weight is when we receive a signal that other vendors are supporting/shipping it. This field would one that is probably more useful to a wider audience if we ever do that. I think it's worth linking to it, just so that we are aware of the standards position for things.
• [TODO] Ksenia to create a two fields (`standard_position` and `standard_discussions`) in the schema.

KB: How to identify (and compare) trends? (Honza)

• Honza: This is a tricky question, but it's one of our OKRs. Is our database/schema ready to track importance? Do we have enough data to justify the importance of an issue?
• Tom: Do we keep track of the timestamps when we create/update an entry?
• James: Well, git is doing that. :)
• Dennis: What's the definition of "trend" here?
• Joe: We're good at spotting trends and patterns in a broad sense. Like the "do we see an uptick in Japan-based issues" question. That doesn't have to be a time-based trend, but just a pattern that we see. The knowledge-base is well-defined, but this kind of information doesn't really fit there. Another example: we had a whole bunch of Flexbox issues, but we didn't but them together, and didn't realize that a lot of issues just boiled down to a negative margin issue.
• Honza: We have to compare KB entries, but it's hard to base this on fact, and feels more like a judgement call.
• Joe: It is a judgement call. There's probably a lot of information that feels pretty obvious to us, but it's not obvious to others because they're not experts in it.
• Ksenia: A way to identify trends is how many comments or see-also's a Bugzilla issue has. If there's suddenly a large spikes of see-alsos or new comments, this might be a signal as well.
• James: There's two things: The Top 20 list, which will probably be based on some objective criteria, like the number of breakage, telemetry signals, etc. The trend part seems to be the subjective part, like groups of issues where people are worried about or that we notice frequently, and we should pay attention even though we have no objective signal (yet). The Japan case was an example where we took a "subjective" data (the origin country) and tried to link that to objective data like platform issues etc.
• Joe: To answer the question "how do we identify trends". I don't think we can document on how to do this. I encourage everyone to speek up if we feel like we see something.
• Honza: There might be signals in the KB that we use to see patterns, like the number at see-alsos etc.
• Tom: The problem is that we have no specific trends we're looking for, so it's hard to know what data we're looking for.
• James: Spending a lot of time now to talk about how to "identify groups of issues that we aren't grouping yet" feels like a big premature optimisation. Maybe at some point in the future there's a project.
• Ksenia: I wonder if there should be a field for that somehow.
• Dennis: In theory, tags could be useful for that. In practise, using tags is very free-form, and might not be useful.
• James: I wanted to add a feature to the tooling that shows the tags other people are using, so maybe you can use that as a tool to learn what tags to use.
• Dennis: I think we should have a tool that lists tags, and also counts issues tagged. That might help with discovering things.
• Tom: Maybe we should notify the team if new tags are added? Or a tool that shows you "similar" tags if you add a new one.
• James: That feels pretty heavy for tagging. If we'd have a frontend for that, we could list all the tags, and people can observe that. As a first step, just a tool that lists all the tags may be enough.

KB: How many entries are we aiming for? (Joe)

• Joe: Context - OKRs
• Joe: How many investigations do we need to be sure that we have the top 20?
• Dennis: I thinkit's hard to put a number on this. If we have a specific numeric goal it could be hard to achieve.
• Joe: We could know we need at least as many as there are open webcompat bugs
• Dennis: Maybe. There is a possibility that we have more open issues than there are actual platform issues. That doesn't seem very likely but not impossible.
• Tom: It might be too early to estimate how many entries will come out. For the first round, we should aim to add as much as possible, without putting a number on it.
• James: In the short term, we probably don't have too many entries, and we have to figure out a way of getting through the backlog and creating entries where needed. We're a long way from having everything with a webcompat flag on bugzilla corespond to a KB entry.
• Joe: What James is saying is the motivation of my question. Currently, we're roughly adding three entires a week, which might leave us with 60 entries at the end of the year. So the question is if we should ramp this up.
• Dennis: How do we actually arrange for the issues to be created? We currently doing it as part of the triage, but how can we address the backlog?
• Honza: Can we go through the backlog of webcompat-priority bugs?
• Ksenia: We already picked some of the P1s, but there are some left.
• Dennis: There isn't too many P1s, but we can look at the P2s as well if we need more data.
• James: Yeah, we should finish the P1s first, and then move on.
• Tom: My plan was that in the next interventions round, I want to check that all bugs where we shipped an intervention have a KB entry.