User:JoeyArmstrong/makefiles/makemakefile

From MozillaWiki
Jump to: navigation, search

post #2

Build system tools: make-makefile, file generation

This post will be article #2 in a series about the build-config tool make-makefile.
The series is being written to formally document current tool functionality and
provide a base for future topics related to the container makefile project.
Container structure, functionality and example usage.

When makefile generation is needed the tool will perform a few steps.

   1) gmake -f client.mk is invoked.  During traversal if gmake detects one
      of two conditions the tool will be invoked to generate a Makefile:
        o $(obj)/Makefile does not exist.
        o $(src)/Makefile.in is newer than $(obj)/Makefile.
   2) Target rules used to invoke mm live in config/rules.mk near line 1342 and 1347
        o 1342 Makefile: Makefile.in
        o 1343 	@$(PERL) $(AUTOCONF_TOOLS)/make-makefile -t $(topsrcdir) -d $(DEPTH)
   3) make-makefile will be passed -d and -t command line arguments:
        o -t  path to root of a development sandbox ~$(TOPSRCDIR)
        o -d  relative path --depth from file to hierarchy root directory.
              Makefile: $(MOZ_OBJDIR), Makefile.in: $(TOPSRCDIR)
   4) When invoked make-makefile is aware of the directory hierarchy
      and will expect cwd for the shell to be $(MOZ_OBJDIR) or a subdir
      of it [1].  config/rules.mk logic will place make-makefile in the
      target directory for each makefile to be generated [2].
   5) Step 1 - obtain values for depth, topsrcdir and objdir.
      Values will be:
        o used to construct directory and file paths.
        o perform text substitution within Makefile.in templates.
      Values can be:
        o explicitly passed on the command line.
        o determined from the filesystem based on arguments and/or cwd.
        o extracted from file arguments.
      o $depth can be set by:
        o Explicitly using the -d command line argument.
        o Assignment of DEPTH=../.. specified within an individual Makefile.
        o Assignment of DEPTH=../.. within any other files passed on the command line (is this a bug or feature?).
        o Assignment of DEPTH=../.. contained within a parent directory Makefile [3], [4].
   6) Step 2 - Using cwd and arguments, derive paths to the source template
      Makefile.in and generated target file Makefile.
   7) Step 3 - Slurp the source template Makefile.in.  Perform value substitutions
      on tokens embedded within the template of the form @token@.

browser/branding/official/Makefile.in
=====================================
DEPTH		= ../../..
topsrcdir	= @topsrcdir@
srcdir		= @srcdir@
VPATH		= @srcdir@

      Some directory/file paths are derived internally by the tool for quick
      substitution.  For any @${unknown}@ tokens make-makefile will delegate
      expansion with a call to system("./config.status") [5] to obtain values,
      create directories and who knows what else.  Shell overhead will be
      imposed by this step so avoid unnecessary tokens when possible.  When only
      a handful of tokens are in play check if mm could derive these values and
      avoid calling the config.status script [6].
   8) Step 4 - update or preserve generated Makefile timestamps.  If
      obj/Makefile does not exist create it.  If the file exists compare
      generated content against the original and only update when modified.

[1] - The requirement of cwd==$(MOZ_OBJDIR)/ can be removed with use of the --enhanced flag.
      This option and other new flags will be covered in a future post.
[2] - Enhancement: a response file could be used to support bulk makefile processing and
      directory creation (by config.status) saving some shell overhead.  Bulk processing
      might also reduce the number of times config.status must be invoked independently.
[3] - Potential bug.  DEPTH= searches within multiple files could set $depth incorrectly
      when it is not explicitly set within a makefile.  One ex would be passing browser/.../Makefile
      and js/src/.../Makefile as command line arguments.  Though this error condition
      exists it is not likely to be triggered building in the current hierarchy.  Makefiles
      are processed/generated individually by directory so there should be no chance for
      file interaction.
[4] - Potential bug: checking ../Makefile for 'DEPTH=' will fail for makefiles
      invoked from a parent directory several layers above cwd.
[5] - Enhancement - modify make-makefile to parse and extract fully expanded
      values from config* to avoid invoking ./config.status for a subset of
      substitution tokens.
[6] - make-makefile will issue a warning when the ./config.status script will be launched
      WARNING: token SET_MAKE not defined
      line 2, src: [.....]/js/src/ctypes/libffi/Makefile.in

resources

make-makefile

Unit tests for program and module

makefiles/target logic

Pending tool enhancements

  • Expand test coverage for make-makefile
  • Add Response file support.
  • Rewrite unit tests in python
  • Porting make-makefile from perl to python.
  • Container makefile support: transition to make-makefile --enhanced as a default

Future Topics


post #1

Build system tools: make-makefile

The build process consists of several steps, one of which is the dynamic generation of makefiles - the creation of recipes that control how and when elements of firefox and friends are gnerated. The tool currently used for generation is make-makefile, a utility written way back in the days of Netscape (circa 1999). Intent for this post and a few more that will follow will be to document the tool and usage.

Scripts and Config

  • build/autoconf/make-makefile
  • build/autoconf/makemakefile.pm
  • build/autoconf/make-makefile.excl

make-makefile will be automatically invoked while building whenever a Makefile does not exist beneath the object directory or a template (Makefile.in) is newer than a previously generated Makefile.


Retrieved from "https://wiki.mozilla.org/index.php?title=User:JoeyArmstrong/makefiles/makemakefile&oldid=398856"