DXR Roadmap

From MozillaWiki
Jump to: navigation, search

Top of the Heap

Here's the next round of improvements we'll do, not necessarily in order. These either make DXR a lot better for some or a little better for all and are biased toward our goal of being able to retire MXR. These items turn into filed bugs (not necessarily with a 1-to-1 correspondence), and then we do the bugs. When this list gets short, we pull more up from below.

  • Indexing and search improvements
    • (1) Move to a line-based search, as proposed in https://github.com/mozilla/dxr/pull/161#issuecomment-25201532.
      • This should solve jruderman's need to sometimes load all the search results onto the page and then cmd-F through them.
    • (3) Stop differentiating between macros, functions, consts, etc.: just find me an *anything* called "fred". People coming up to the Innovation Fair booth were confused that they had to know the kind of target.
    • (3) Just typing a filename (or path segment?) without a path: specifier should find files by name or path. DXR Result Mixing should help with this.
      • [Should be solved by ^^] Trying to find files is hard. (Still haven't figured out how to get easily from the main page to Navigator.cpp on dom/base)
      • [Should be solved by ^^] A basic "file:" keyword hint with simple wildcard globbing could do most of it well enough, I think... "file:*.css", "file:nsILogin*", "file:/test*". (We already search for "path" matches as unanchored substrings. We just need to add the globbing. Why this isn't super slow I don't know.)
      • [Should be solved by ^^] As a jump point when I know a filename (eg, "nsILogin", click search, click the particular IDL I wanted).
      • Oddly enough, the direct result finder does search for paths ending in whatever you type, so that's an inconsistency (noticed by jmaher).
    • (4) Design for being able to search for multiple extensions at once. For one idea, multiple ext: filters should be ORed together.
  • (1) Index multiple trees (starting with comm-central and mozilla-aurora, the most commonly used ones on MXR.
  • MXR parity bugs are likely the main reasons people still use MXR.

Research Projects

These are going on concurrently with the above, in spare cycles and in volunteer contributors' time.

  • Rust analysis support, led by nrc
  • Incremental indexing, started by abbeyj and continued by seth
  • DXR_Parallel_Tree_Indexing, gestating in ErikRose's head

Rest of the Heap

The rest of the heap is a synthesis of feedback and ideas from (for instance) this dev-platform thread. Found a bug? File it! Have a broader idea or noticed a problem whose solution isn't as clear? Scribble it here!

We use the following numbers (and letter) to rank items with regard to MXR retireability, which is our overarching goal at the moment. These express nothing about difficulty. Order of attack will take this and other factors into account.

  1. Must have. Everybody will be mad otherwise. Not having would be silly. MXR retirement blocker.
  2. Must have for >= 10% of the audience. Likely MXR blocker.
  3. Can wait but should get to to feel proud of the project. Might be able to turn off MXR without it.
  4. A useful thing for later
  5. A rare edge case, out of scope, or there's probably a better solution

B: Behind the scenes. A non-user-visible change that will enable other changes.

Search

  • (3) Better ranking: if you type an exact identifier name, put the definition at the top. (If we implement "id:", this is easy.)
  • (3) Stop differentiating between macros, functions, consts, etc.: just find me an *anything* called "fred". People coming up to the booth were confused that they had to know the kind of target.
  • (4) Search just within strings, for error messages and such.
  • (4) Let us structurally query stuff that gets #ifdef'd out on x86, like ARM stuff.
  • Multi-tree search. "Having a unified mxr or dxr that includes all the m-c stuff _and_ gaia would help for some sorts of changes (like API removals). People are trying to make sure to search both repos, but they slip up sometimes."

Indexing

  • (B) Index trees independently. A build failure in one (with an unchanged format version) shouldn't totally scuttle an instance update.
  • (B) Parallel and/or incremental indexing. This will let us scale to more trees.
  • (B) (2?) Dynamically generate markup, rather than building zillions of static files. This means we won't have to increment the format version nearly as often, so deployments won't block on time-consuming whole-instance rebuilds as often, so markup changes get into production faster. This might reduce the need for the above. It will also enable us to think about serving up (unanalyzed but syntax-colored) older revisions of files, for permalinking.
  • (3) Support for generated files in indexing. (Is this about IDLs, for example? We should just make DXR understand IDLs. Here's the code that turns IDL attr names into C++ and JS ones: http://dxr.mozilla.org/mozilla-central/source/xpcom/idl-parser/header.py#l34, http://dxr.mozilla.org/mozilla-central/source/dom/bindings/Codegen.py (search for "binaryNames and makeNativeName").)
  • (3) Better support for JS, Python, etc.
    • Remove the C++ assumptions from the core so we can support structured queries on other languages. Python keeps coming up. Java and Scala did once as well. People mentioned JS a long time ago.
  • (4) Merge multiple build configuration databases somehow.
    • The code base is compiled for multiple platforms. Currently I cannot find the functions which are defined on ARM unless we use a search as we used to do on MXR.
  • (4) Include Doxygen/Javadocs-like documentation. For C, C++, IDL, Java, JS, etc.

Blame/VCS integration

  • (4) Diff between trees
  • (4 - Does MXR offer this now? yes -MattN I see it only in MXR's links through to hgweb. Does it offer it internally as well?) Be able to refer to specific revisions of code somehow, so links don't rot.
  • (4) Blame link should preserve the line-number fragment so it hops right to the highlighted line in hgweb or whatever.
  • (4) Be able to refer to certain ranges of code.
  • (5 - Shouldn't we delegate to hgweb or github for this?) Blame integrated into main file view
    • Be able to navigate blame history better, stepping back and back in time until we find the change we were looking for.

Other

  • (4) Support for image browsing would be super helpful for front-end stuff.
  • Compare http://dxr.mozilla.org/mozilla-central/source/toolkit/themes/windows/global/icons with http://mxr.mozilla.org/mozilla-central/source/toolkit/themes/windows/global/icons/
  • (5) DXR gives a nice contextual navigation, but the size of the code base is overwhelming to have a clear understanding of what is going on. One of the thing that I am looking at in general is to understand the conditionals which are giving a particular result, or the consequences of a statement. Such overview is hard to get when you have ~30 DXR tabs opened. I would love to have a graph overview of these relations, as well as seeing the conditionals/guards as part of the graph.
  • (huh?) "cycleCollection" on the right side may or may not do something useful. In most cases it just ignores all the stuff, so it might be better to not have it at all.

UI/UX

  • (2) Mook (and Dave Townsend) switches trees a lot while looking at a single file in MXR. He'd like to be able to do that without losing his scroll position, as it typically lands him at a similar-enough place in the code that he can reorient himself. We'll add a tree switcher to the navigation panel, which appears when a file is being viewed. The nav panel will be pinned.
  • (3) Enable integration with IDEs. I'm scoping to not include writing any plugins, but we should at least expose a well-documented public API. Then we can see what develops. We've had interest from a couple directions on this.
  • (4) Count of search results, so we can use DXR to gather continuous metrics
  • (4) Direct results: some love them, some hate them (because they just want to see the file pathname (don't we show that with the file? Is it bothersome because it's slow to load?))
  • (4) Show context around results (with a clickable control? by saying "context:3[lines]" in the query? show the entire statement?)
  • (4) Currently, you always get the context menu. But I (erikrose) suspect there is a by-far-most-common case: jumping to a symbol's definition. If we were to map that to a normal click and save the context menu for context-clicks (right-clicks, etc.), it would let people bounce around the codebase faster. It would be great to back up my supposition with some measurements. Risk: This makes the existence of the context menu (and thus a lot of DXR's capabilities) non-obvious.
  • (4) Find a place to put release notes, preferably accessible from in-product.

Just Bugs

Retrieved from "https://wiki.mozilla.org/index.php?title=DXR_Roadmap&oldid=1075363"