Collabora Notes

From MozillaWiki
Jump to: navigation, search

These are helpful tips, made by Collaborans, for those struggling to build Mozilla sources on different platforms.

Collaborans are interested in building code from the Electrolysis project (E10s) on a machine on which Scratchbox is also installed with the newest Maemo SDKs setup for ARM cross-compilation.

XEN IMAGES

One route to a working build environment is via a pre-configured virtual machine image. robin_bb has developed two such images. Each is Linux-based.

Mozilla Central on Debian 5.0 32-bit i386

The Xen image called "lenny-32-mc" (1.1GB) (removed 16 July 2011 - contact me for another) is a Debian 5.0 ("Lenny") installation on top of a 32-bit i386 Xen VM. It has a copy of the Mozilla Central code repository, and all of the Debian packages required to build that. The root password is "Collabora".

To use that image, you'll need the Xen virtual machine. While installing Xen is quite an ordeal, it is significantly easier than configuring the build environment described above!

Maemo + Scratchbox + E10s + on Debian 5.0 32-bit i386

Coming soon.

VIRTUALBOX IMAGE

Request from robin_bb.

MANUAL SETUP

For those who don't have proper setup or recommended OS to install the provided Xen images -- or for those who want to understand the odds of setting up a cross-platform development environment for the mozilla codebase -- the following are three step-by-step guides focussing on building the Electrolysis projects (e10s) under three different platforms, namely linux, maemo and win32.

Important considerations when using scratchbox

If you are interested in building electrolysis specifically for only one of the abovementionned platforms, then you can just go right to the proper section hereunder and follow the steps from there. However, if you intend to build for the maemo platform (using scratchbox) along with any (or both) of the other platforms, then, you might want to start setting up your maemo environment first and be sure to put the Mozilla sources somewhere under the /scratchbox directory. Preferably, something like :

/scratchbox/users/<your_username>/home/<your_username>/mozilla/e10s

In fact, because Scratchbox performs "chroot" on the /scratchbox directory at startup, all other system directories outside of /scratchbox, including home/, will become unreacheable for all scratchbox sessions. This chroot operation is essential for seamless cross-compilation and in fact, comes in very handy.

This way, you'll be able to use this one and only codebase for all of your maemo, linux and win32 platform builds. Of course, to make your cross-platform environment even more seamless, a very good idea is to create symbolic links that map to your /scratchbox/... working dirs, as follows:

cd /home/<your_username>
ln -s /scratchbox/users/<your_username>/home/<your_username>/mozilla mozilla 

Building e10s under linux

Supported Linux versions

The mozilla source supports many linux flavors, but the most common ones among Mozilla developers are Debian/Ubuntu. See this page for a list of the supported distros.

Installing build dependencies

Many third-party libraries are necessary to build the mozilla codebase. Depending on your linux distro, you can find all the commands to properly install the linux prerequisites in here.

Mozconfig

For general information on how to create mozconfig files, read this.

Alternatively, here are two examples of mozconfig that you can use directly :

#DEBUG build under linux<br> 

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../objdirs/DEBUG_LINUX_X86
mk_add_options MOZ_BUILD_PROJECTS="browser"
ac_add_app_options browser --enable-application=browser 

ac_add_options --disable-optimize
ac_add_options --enable-debug
ac_add_options --enable-debugger-info-modules 
ac_add_options --disable-tests
ac_add_options --disable-crashreporter
#RELEASE build under linux<br> 

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/../objdirs/RELEASE_LINUX_X86
mk_add_options MOZ_BUILD_PROJECTS="browser"
ac_add_app_options browser --enable-application=browser

ac_add_options --enable-optimize 
ac_add_options --disable-debug
ac_add_options --disable-tests

Pulling the electrolysis project

see the Pulling_the_electrolysis_project section below.

Building e10s

You can perform what we call an "srcdir" build. See the Building_the_electrolysis_project section below.

Note: Once you've built the code, you can rebuild "from top", that is, from the <objdir/browser> directory instead, using only the 'make' command. It's faster than building from the "scrdir".

HOWEVER, subsequent modifications to the code will rarely require a full build from top as you can rebuild only the required components. See E10S_Build_WikiPage for more information.

Building e10s under maemo

Installing Scratchbox Apophis r4

NOTE: There are actually two scratchbox versions for maemo developers, namely Scratchbox1 and Scratchbox2. Both allow electrolysis to build under maemo for ARM processors, but this document will only pertain to version 1.0.16 Apophis r4

Install from your regular repositories

If you're running DEBIAN or UBUNTU, get the most recent stable release packages from the scratchbox apophis repository :

Do the following :

sudo echo "deb http://scratchbox.org/debian/ apophis main" >> /etc/apt/sources.list 
apt-get update
yes | sudo apt-get install scratchbox-core scratchbox-libs scratchbox-toolchain-host-gcc scratchbox-devkit-apt-https

Install manually

In any other case, you may want to manually install Scratchbox apophis by downloading an installing the latest tarballs from here.

  • scratchbox-core-<latest version>.tar.gz
  • scratchbox-libs-<latest version>.tar.gz
  • scratchbox-toolchain-host-gcc-<latest version>.tar.gz
  • scratchbox-devkit-apt-https-<latest version>.tar.gz

Once downloaded, do the following for each tarball :

cd /
sudo tar -xvf <tarball_file>

Installing Maemo SDK

Before installing your SDK, you have to add your username to scratchbox by typing the following :

sudo /scratchbox/sbin/sbox_adduser $USER yes

Note : If you get the following error : "Host kernel has vdso support (which is uncompatible with SB)", do add/change the following lines to your /etc/sysctl.conf file :

vm.vdso_enabled = 0
vm.mmap_min_addr = 4096

then run :

sudo sysctl -p

... then retry to add your username and don't forget to restart your Xserver before going on with the following steps.


You might be targeting a specific Maemo platform version and target device. Below are different devices with their associated Maemo SDK.

Note: links to the most recent SDK version (when this document was written) are also provided.

For your chosen SDK, you can download and run the "Maemo SDK Installer" script provided on the SDK webpage. The script should automatically download and install all the needed devkits, toolchains and targets for you.

Note : the installer will show you the required devkits that you have to download and install for your chosen target SDK installer. Follow the instructions and the setup process should run smoothly towards completion.

installing without an installer script

For other not-yet-released maemo SDKs, you simply start by downloading and installing the appropriate set of devkits and toolchains .deb packages in your repo. Here's a list of what you'll normally use :

  • scratchbox-devkit-qemu
  • scratchbox-devkit-cputransp
  • scratchbox-devkit-debian
  • scratchbox-devkit-doctools
  • scratchbox-devkit-maemo3
  • scratchbox-devkit-perl
  • scratchbox-devkit-apt-https

... and then, you have to create compilation targets in scratchbox. For example, to create an ARM compilation target for the DIABLO SDK, do the following :

sudo /scratchbox/sbin/sbox_adduser $USER yes

##############################################
## restart your X server with CTRL-ALT-BACKSPC
##############################################

/scratchbox/login

sb-conf setup DIABLO_ARMEL -c cs2007q3-glibc2.5-arm6 \
  -d cputransp:debian-etch:doctools:maemo3-tools:perl \
  -t /scratchbox/devkits/cputransp/bin/qemu-arm-cvs-m
sb-conf rootstrap DIABLO_ARMEL /<path_to_downloaded_rootstrap>/maemo-sdk-rootstrap_4.0.1_armel.tgz
sb-conf install DIABLO_ARMEL -cedFGS
sb-conf select DIABLO_ARMEL

Installing missing dependencies

Now that you've got at least one ARM target created inside scratchbox, you have to install some libs required to build electrolysis. For example, if you've setup a DIABLO_ARM target, then do the following from your target scratchbox session :

fakeroot echo "deb http://repository.maemo.org/extras-devel diablo free non-free" >> /etc/apt/sources.list
fakeroot echo "deb file:/<path_to_nokia_binaries_if_any>/maemo-sdk-nokia-binaries_4.0.1 diablo explicit" >> /etc/apt/sources.list
yes | fakeroot apt-get update
yes | fakeroot apt-get install libgtk2.0-dev libxaw7-dev libfreetype6-dev libxi-dev libhildonmime-dev libosso-dev libidl-dev liblocation-dev libasound2-dev libhildon1-dev libiw-dev

Note: This set of missing libs was correct and complete for the DIABLO_ARM target when this document was written, but it is very likely that, for other targets, and over time, this list may have to be completed.

Pulling the electrolysis project

Pulling the e10s project, as well as updating your worktree with 'hg', should be done outside of scratchbox, See the Pulling_the_electrolysis_project section below.

Building e10s

See the Building the electrolysis project section below.

Note: The scratchbox '_X86' targets don't add extra value to the build and since your SB build environment might not be as complete as your native linux environment, it's really likely that'll you'll have problems building with those targets. So there's really little need for the _X86 targets in Scratchbox. When you want to build for X86 processors, do it natively from your linux console.

Building e10s under win32

Installing a proper development environment

Dependencies and tools required to build Mozilla under Windows are evolving and changing rapidly. Generally though, one must install the following three components :

  • Microsoft Visual C++ Tools
  • Microsoft Windows SDK(s)
  • MozillaBuild

Microsoft Visual C++ Tools

To build Electrolysis, you must at least use Visual C++ 8 (Visual Studio 2005). You must also make sure you've got the newest Service Packs. See this for more information.

Microsoft Windows SDK(s)

Read this to find which Windows SDK will make your Visual Studio happy.

MozillaBuild

Mozilla is not built natively under Windows with the usual development tools. It's usually built from MinGW (Minimalist GNU for Windows). All the required tools (including MinGW, MSys, HG and Perl) needed to build the code under Windows are provided by mozilla as an executable installer. Download and install the latest version..

Installing an ext2/ext3 fileSystem

This step is optional if you're only building e10s under windows, or if you're using a virtual machine, but comes in very handy if you're developing on both win32 and linux (or Maemo) platforms while having a dual-boot setup. "Ext2 IFS" lets you mount an ext2 or ext3 linux partition onto Windows so that you can build e10s from the same codebase that you're using from linux or scratchbox.

Get Ext2 IFS from here.

Setting MSVC up

There are as many ways of configuring MSVC projects as there are developers hacking on mozilla, but to get all the benefits from MSVC, you'll generally want to do the following steps :

  • Create one project per module and create convenient folders/filters for it. (For example, create one project called "widget", another "xpcom", "content", etc.... If you put all the mozilla files into one huge sigle MSVC folder, it'll be a mess!)
  • For each project, choose the "Makefile" configuration type from its "properties" menu.
  • For each projects, add appropriate include folders to make Intellisense kick in properly.
  • You might want to include the <objdir>/browser/dist/include/ folder in your include path. The build copies any header marked as EXPORT in the makefiles there. However, be careful NOT to edit these files. Intellisense will open them happily and they are writeable, but you'll lose any changes at the next build.


I've created some example MSVC project files. It's easier to start with them and, of course, rename/replace all the absolute and relative directories found in them prior to load them with MSVC.

Note : in this .zip, you'll find "/template/Template.vcproj". This is an empty project template file that you can use to add another module to your solution. It contains convenient folder filters.

Getting a command prompt for pulling and building

Before using 'hg' or 'make', you'll have to get a command prompt. Carefully follow these following steps. This .bat file will open a MinGW32 window.

Note: It's very likely that you'll have to tweak the start-msvcXX.bat batch file to add more paths to the "PATH" environment variable. Also, you'll probably need to add some "LIB" and "INCLUDE" paths, or preset the "MOZCONFIG" variable for convenience.

Pulling the electrolysis project

Once you've got that MinGW32 window with command prompt, just pull the code as you would normally do on other platforms by following these steps.

Note: If the 'hg' command is unrecognized, log out from your minGW32 console, update the "PATH" environment variable in "start-msvcXX.bat" and start the console again

Building e10s

Even under Windows, e10s is built manually from a console (though you can setup MSVC projects to drive builds for you, but it's not recommended, as build dependencies are very complicated, making incremental build from MSVC a *real* challenge!).

From the minGW32 window, build the code as you would normally do on other platforms, by following these steps.

Common steps to all three platforms

Pulling the electrolysis project for the first time

The electrolysis repository is a parallel branch to the "mozilla-central" branch, from which the main Mozilla apps are built (firefox, thunderbird, etc...).

You must have Mercurial installed on your system to clone the electrolysis repository.

Do the following :

hg clone http://hg.mozilla.org/projects/electrolysis e10s

This will create the "e10s" directory under the current directory.

Building the electrolysis project

Once you have pulled the code and created a mozconfig, you're ready to build the electrolysis project. You can perform what we call an "srcdir" build :

export MOZCONFIG=/scratchbox/<path_to_mozconfig>/<your_mozconfig_file>
cd /scratchbox/users/<your_username>/home/<your_username>/mozilla/<your_code_directory>
make -f client.mk build

Note #1 : The directories mentioned above are just examples, but are a reminder that, for cross-platform developers, mozilla code should be pulled somewhere under the "/scratchbox" directory. For more information, see this.

Note #2: You don't have to specify the chosen mozconfig file throught a MOZCONFIG export. You can just as well create your mozconfig as a .mozconfig file laid out in <your_code_directory>. This, though, doesn't work well under windows.

Note #3: Once you've built the code, you can rebuild "from top", that is, from the <objdir/browser> directory instead, using only the 'make' command. It's faster than building from the "scrdir". HOWEVER, subsequent modifications to the code will rarely require a full build from top as you can rebuild only the required components. See E10S_Build_WikiPage for more information.

Running electrolysis

Eee the "Running electrolysis" section from [1]

Notes on Building

IPDL

When adding the first IPDL files to a Mozilla component, one must ensure that the following lines are added to the (sub)component's Makefile.in:

include $(topsrcdir)/config/config.mk
include $(topsrcdir)/ipc/chromium/chromium-config.mk

Also, make sure prtypes.h is NEVER included before basictypes.h, as the definition of int32 is different (int/long). You'll get an unresolved external error while linking otherwise, complaining the IPC::Message constructor is not found.

XPIDL

When changing an .idl file, one must always rebuild "from top", because it's very likely that some other component (that you did not recompile) is making use of your modified .idl interface, thus leading to vtable problems and crashes.