BSDCan2010 - Final Release
BSDCan 2010
The Technical BSD Conference
| Speakers | |
|---|---|
|
Craig Rodrigues |
| Schedule | |
|---|---|
| Day | Talks - 2 - 2010-05-14 |
| Room | DMS 1160 |
| Start time | 13:30 |
| Duration | 01:00 |
| Info | |
| ID | 198 |
| Event type | Lecture |
| Track | Hacking |
| Language used for presentation | English |
jbuild
next generation build tool for FreeBSD
jbuild - next generation build tool for FreeBSD
This presentation will discuss a modified version of the FreeBSD make(1) utility named "jbuild". jbuild was originally developed by FreeBSD developer, John Birrell, to solve large scale software build issues by improving the dependency tracking of make(1). jbuild improves dependency tracking by tracing all file system reads and writes by programs invoked by fork(). This traced data is used to augment the dependency tracking in make(1).
This presentation will be an in-depth technical presentation of jbuild, and will discuss some possible ways in which it can be used to improve the software build of FreeBSD.
jbuild - next generation build tool for FreeBSD
The make(1) utility has been part of the FreeBSD base system since the beginning of the project. Before FreeBSD, the make utility was included with PWB/Unix 1.0 In FreeBSD, one of make(1)'s most important uses is used to build all the software that comprises the FreeBSD base system (userland and kernel).
Over time, the size of the FreeBSD codebase has grown, as has the number of architectures that it can build on. The scale of the software build requirements for FreeBSD has exposed some fundamental limitations in make(1) and the core sys.mk Makefile rules used to build FreeBSD. Some of the problems that end-users see due to these limitations include: -> Parallel builds with very high values of -j [n] often fail. -> "make universe" rebuilds a lot of the same software multiple times, and takes a very long time. Consequently most FreeBSD developers do not test their code against this target, so sometimes code changes are introduced which break the build on less frequently used machine architectures.
For the FreeBSD project, these problems are annoying, but may be somewhat tolerable. However, for commercial products that are derived from FreeBSD, with much larger codebases, these problems can result in huge productivity losses.
FreeBSD developer John Birrell saw many of these problems in the software build at his employer, Juniper Networks. John decided to address some of these fundamental software build problems by: (1) modifying FreeBSD's make(1) to trace all programs invoked from make(1) via fork(), in order to log all file system read and write calls (2) using this tracing information to generate dependency information which could be used by make(1) (3) writing core Makefile rules in sys.mk to make it easier to add new machine architectures to the build, and make it easy to test compilation of code across multiple architectures
John implemented his changes as a fork of the make(1) utility, named "build", and later "jbuild". The code for jbuild is available in a branch of the FreeBSD SVN repository.
This presentation will discuss some of technical details of jbuild, and also John's vision for how this utility could be applied to improving the FreeBSD build.