DXR Future Work Plan

From MozillaWiki
Jump to: navigation, search

This page is currently fairly outdated, some of these future ideas are certainly still applicable. Whilst many of the issue are no-longer applicable, in the current testing branch. At time of writing I strongly recommend reading "docs/" folder from current testing branch before continuing to read this page.

What is DXR?

DXR is a two things. First, it’s a method for collecting type, member, statement, macro, etc. information about C++, IDL, and soon JavaScript using Mozilla’s static analysis tools (e.g., Dehydra), a hacked build system, and a bunch of scripts. Second, it’s a web-based tool for mapping this information back onto source code, and allowing users to query and look-up this information.

You can read more about it here.

DXR was written and is being actively developed by David Humphrey.

DXR Bugs

The complete list of DXR bugs is available here (WebTools:DXR )

Current Task List

The current set of tasks can be summarized into the following high-level categories:

  • Improve the DXR web-app backend
  • Improve the indexing of source code and creation of the database and HTML
  • Improve user access to data stored in the database, making it easier to query, search, etc.
  • Improve the presentation and accuracy of data presented within the source code itself

Each of these three have sub-tasks, many of which interrelate.

Web App Back-end Work

  • Switch python web cgi to use django
  • Audit web app for security
  • Start using caching
  • Investigate switching from SQLite to a db server
  • Investigate adding some basic auditing and user feedback:
    • What are people searching for most often?
    • Allow users to star or otherwise mark things that are wrong in the HTML data
    • Perhaps allow users to leave notes, links, etc. in various parts of the code

Indexing, HTML Generation

  • Build test harness for indexing/html generation
  • Deal with edge cases for builds failing (e.g., don't delete existing an existing index/html set until it's clear that the new build/index is working, cf. builds failing back-to-back)
  • Figure out best method for doing continuous re-indexing of the tree
    • Answer: Build DXR from tinderbox!
  • Add dead-code info (talk to ehren for details)
  • Improve post-processing speed, investigate better parallelism
    • And memory usage
  • Add proper IDL data to source code such that clicking types, members, etc. shows a popup with links to implementations, callers, etc.
  • Fix C++ snippets in IDL to use cpp position indicators (# line file). Add any other missing IDL features.
    • Switch from using xpidl to xpidl.py and add __FILE__ and __LINE__ data to IDL C++ snippets
  • Fix use of IPIDL ${...} stuff for .cpp.html generation (e.g., Got invalid token in ? @ 353 token:$: 'TED =====\n${INCLUDES' from http://mxr.mozilla.org/mozilla-central/source/ipc/ipdl/test/cxx/IPDLUnitTests.template.cpp#19 -- ${INCLUDES}
  • Look for speed-ups in how analysis builds are done
  • Add links for #include header files
  • Add support for JS source. Consider using jshydra to build static HTML. See js token info in jsscan.h.
  • Add support for Obj-C/C++ source
  • Integrate various callgraphs (ehren's, sixgill): convert to using treehydra, investigate --with-static-checking in mozilla's build sys, include dwitte's callgraph.js and my dxr.js (include "callgraph.js").
  • Tokenize macros so that embedded macros are clickable (e.g., NS_ProcessNextEvent resolves to NS_ProcessNextEvent_P)
  • Make IDL pages, and other decl locations do a better job at providing links
  • Fix srcdir vs. objdir links (e.g., PRInt32)
  • Remove .html from source file names (mod_rewrite?)
  • Make dxr scripts path agnostic and more portable (python path stuff)
  • Fix ctor html linkifying code so Foo::Foo works properly
  • Figure out how to deal with code being ifdef'ed out (e.g., debug vs. non-debug build)
  • Audit dehydra script for improvements
  • Add a download-db link
  • Link to the hg revision of the indexed db (i.e., a timestamp)
  • Investigate minifying html - http://perfectionkills.com/experimenting-with-html-minifier/

Querying, Search

  • Make Users queries differentiate decls/defs
  • Add ability to do Users:nsIFrameFrame::GetDocShell
  • Add base:type search query
  • Show Classname:: as part of member sidebar on the right (for searches)
  • Make search results clickable anywhere vs. just on line number and make line number plain text vs. link.
  • Make search sidebars links so they can be turned into searches. For example, members sidebar could be a link that takes you to a member:foo query
  • Search UI should encourage using built-in queries, for example, suggesting other keywords (members:) that could be added.
  • When searching of nsFoo, display a little summary of the class bases,derived, members
  • For Type search results, give a link to do a type:: search to show all members (if there are any) -- nsFoo (class) [members]
  • In search results, favour c++ style name ordering (e.g., class nsFoo vs. nsFoo (class) ). Make the name more prominent.
  • In search results, make all types and member names clickable
  • Remove side bar from searches, and copy Google Code search results by showing other queries at top of results (Narrow your search: 53 Types, 74 Members, ...)
  • In decl/def search results (e.g., member) only show line number once.
  • Add counts to all search results (glimpse and otherwise)
  • Add pagination to search results
  • Limit search nav divs to some lower number (12?) and give a link to see the whole thing.
  • Remove JavaScript munging of search box input, so "Add a keyword for this search" works

Source Code View

  • Pin Popup when scrolling (Jeff and jdm want)
  • Make source code nav bar narrow if contents are narrower, and expand to a max size (current size).
  • When navigating to search result in code for a type or member, sync the source code navigation pane accordingly
  • Lazily keep source code nav and source view in sync. Clicking any stmt should highlight/scroll into view/etc the containing member/class
  • Make source code nav panel sortable (alpha, file order)
  • Make types clickable in source code nav bar, perhaps open the popup to show type info
  • Make types in source code nav bar always clickable, and add icon to take you to decl (should be able to jump to header file from cpp)
  • Add hrefs to links in source code dom nodes on hover
  • Standardize on look and feel of source code nav bar for members and popups, etc (gray, icons, font)
  • Tighten-up look of popup box
  • Provide mechanism to go from one type to another, showing intermediary steps (QI, method calls, etc. to get from nsIOne to nsITwo) (cjones wants)
  • Fix pop-up position so it isn't off the screen
  • Fix Users query so it includes uses of the type directly in addition to uses of derived types (e.g., nsIFoo might have direct users if not an idl type)
  • Add Callees and Callers searches to all popups keyed on loc
  • Add info to popups that indicates that empty popups are bugs in data/dxr or TODOs (maybe with link to this wiki page or bug filing info).
  • Collapse Type Info and Bases tabs in popups
  • Convert popup to use fewer/no tabs, and dynamic/tree regions to show data like members, bases, etc.
  • Make http://dxr.proximity.on.ca/dxr/mozilla-central/xpcom/ds/nsHashtable.h.html#144/Put work

Future Ideas (not currently in scope)

  • Integrate other tools:
    • Should we integrate Mossop's XPCOM docs project?
    • Connect with MDC (e.g., for XUL docs)
    • Integrate with bug/patch data in bugzilla
    • Dead code, should-be-static and other analysis data
  • Add Macro decl locations
  • Investigate strategy for increment re-indexing vs. full clobbers
  • Add support for html/xml, perhaps with links to MDC doc pages
  • Multi-platform indexing
  • Add build dependency map info - "what do I have to remake if I change something in content/html/document/src?"
  • Add control mechanism for running dxr continuously. Maybe web based? Maybe as part of buildbot with waterfall?
  • Patch/Review features: try-server type system that checks the patch for issues, but perhaps dxr should be used to visualize the results. Show things like, new warnings/issues/deadcode. Also, given a patch, show everything DXR knows about the patch and code affected by it.
  • derived: search should also provide an interactive inheritance diagram
Retrieved from "https://wiki.mozilla.org/index.php?title=DXR_Future_Work_Plan&oldid=466359"